Data Engineer

4 weeks ago

Mangalore, Karnataka, India R Systems Full time

Job Title: Data Engineer

Contract Period: 12 Months

Location: Offshore candidates accepted (Singapore Based Company)

Work Timing : 6.30 AM to 3.30 PM or 7.00 AM to 4.00 PM (IST - India timing)

Experience

  • Minimum 4+ years as a Data Engineer or similar role.(Please don't apply if less than 4 years exp in Data Engineer)
  • Proven experience in Python, Spark, and PySpark (non-negotiable).(mandatory*)
  • Hands-on in building ETL pipelines, real-time streaming, and data transformations.
  • Worked with data warehouses, cloud platforms (AWS/Azure/GCP), and databases.

Technical Skills

  • Spark Core API: RDDs, transformations/actions, DAG execution.
  • Spark SQL: DataFrames, schema optimization, UDFs.
  • Streaming: Structured Streaming, Kafka integration.
  • Data Handling: S3, HDFS, JDBC, Parquet, Avro, ORC.
  • Orchestration: Airflow/Prefect.
  • Performance Tuning: Partitioning, caching, broadcast joins.
  • Cloud Deployment: Databricks, AWS EMR, Azure HDInsight, GCP Dataproc.
  • CI/CD: Pytest/unittest for Spark jobs, Jenkins, GitHub Actions.

Education & Soft Skills

  • Bachelor's/Master's in Computer Science, Computer Engineering, or equivalent.
  • Strong analytical, problem-solving, and communication skills.

About the Role

We are seeking an experienced Data Engineer to join our team and support data-driven initiatives. This role involves building scalable pipelines, working with streaming data, and collaborating with data scientists and business stakeholders to deliver high-quality solutions.

Key Responsibilities

  • Design, build, and optimize data pipelines and ETL workflows.
  • Manage and process large datasets using Spark, PySpark, and SQL.
  • Build and maintain real-time streaming applications with Spark Streaming/Kafka.
  • Collaborate with data scientists and product teams to integrate AI/ML models into production.
  • Ensure data quality, scalability, and performance in all pipelines.
  • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks).
  • Automate testing and deployment of Spark jobs via CI/CD pipelines.

Requirements

  • Bachelor's/Master's degree in Computer Science, Computer Engineering, or related field.
  • Minimum 4 years of professional experience as a Data Engineer.
  • Strong expertise in Python, Spark, PySpark.
  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution.
  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).

Budget

  • Onshore: As per market standards.

  • Data Engineer

    3 weeks ago

    Mangalore, Karnataka, India R Systems Full time

    Job Title: Data EngineerContract Period: 12 MonthsLocation: Offshore candidates accepted (Singapore Based Company)Work Timing : 6.30 AM to 3.30 PM or 7.00 AM to 4.00 PM (IST - India timing)ExperienceMinimum 4+ years as a Data Engineer or similar role.(Please don't apply if less than 4 years exp in Data Engineer)Proven experience in Python, Spark, and Py...

  • Data Engineer

    3 weeks ago

    Mangalore, Karnataka, India People Prime Worldwide Full time

    About Client:Our Client is a multinational IT services and consulting company headquartered in USA, With revenues 19.7 Billion USD, with Global work force of 3,50,000 and Listed in NASDAQ, It is one of the leading IT services firms globally, known for its work in digital transformation, technology consulting, and business process outsourcing, Business Focus...

  • Data Engineer

    3 weeks ago

    Mangalore, Karnataka, India Logic Pursuits Full time

    Job Title: Data Engineer (Snowflake + dbt)Location: Hyderabad, IndiaJob Type: Full-timeJob Description:We are looking for an experienced and results-driven Data Engineer to join our growing Data Engineering team. The ideal candidate will be proficient in building scalable, high-performance data transformation pipelines using Snowflake and dbt, and able to...

  • Senior Data Engineer

    1 day ago

    Mangalore, Karnataka, India UnifyCX Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Required Experience8+ Years for Senior Data EngineerPreferred SkillsetsVery Strong Python skills.Very efficient in SQL/Stored procedures.At least 8 years of Hands on Experience in AWS Step function, AWS Lambda, AWS S3, AWS ECS, AWS Cloud Watch, AWS Event Bridge, AWS Athena, AWS Glue.At least 8 years of Experience in TerraformAt least 8 years of Experience in...

  • Quality Assurance Lead

    3 weeks ago

    Mangalore, Karnataka, India DATA POEM Full time

    Company DescriptionData Poem is a core AI company focused on uncovering true causation in data, going beyond correlation. We have developed the world's first Large Causal Architecture, enabling organizations to derive deep, actionable insights. Our flagship product, POEM365, is an enterprise-scale AI model that leverages a vast data universe to deliver...

  • Data Scientist

    4 weeks ago

    Mangalore, Karnataka, India Three Across Full time

    Job Title: Data ScientistExperience: 5+ YearsLocation: Remote/ Indore/ Mumbai/ Chennai/ GurugramIndustry: Must be from BPO/KPO or Healthcare Org. Or Shared servicesKey Responsibilities:AI/ML Development & ResearchDesign, develop, and deploy advanced machine learning and deep learning models to solve complex business problems.Implement and optimize Large...

  • Analytics Engineer

    6 days ago

    Mangalore, Karnataka, India UnifyCX Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Who We AreunifyCX is an emerging Global Business Process Outsourcing company with a strong presence in the U.S., Colombia, Dominican Republic, India, Jamaica, Honduras, and the Philippines. We provide personalized contact centers, business processing, and technology outsourcing solutions to clients worldwide. In nearly two decades, unifyCX has grown from a...

  • Python AI Engineer

    2 weeks ago

    Mangalore, Karnataka, India beBeeDataScience Full time ₹ 50,00,000 - ₹ 84,00,000

    Job DescriptionWe are seeking a highly skilled Python AI Engineer to lead the design and implementation of intelligent frameworks for synthetic and automated test data generation in support of clinical trials, regulatory submissions, and GxP-compliant systems.This role will leverage AI/ML and Python automation to deliver high-quality, audit-ready datasets...

  • AI Engineer

    3 weeks ago

    Mangalore, Karnataka, India MightyBot Full time

    Join our team as an AI Engineer, where we're focused on graduating AI from interesting demos to indispensable products. You will build reliable, self-improving systems that empower subject matter experts to automate their most complex, high-value work. This is a role for engineers who want to solve real-world business challenges and create AI tools that are...

  • OpenAI Engineer

    4 weeks ago

    Mangalore, Karnataka, India Hyqoo Full time

    Title - Application Engineer AI SMEType - Contract Location - Remote Roles and Responsibilities:- Configure and manage data privacy controls to prevent the training of AI models on sensitive data and enforce zero-retention policies.- Deliver comprehensive audit logs, compliance reporting, and monitoring for OpenAI usage to ensure adherence to regulatory...