Data Engineer

2 days ago

India R Systems Full time

Job Title: Data Engineer

Contract Period: 12 Months

Location: Offshore candidates accepted (Singapore Based Company)

Work Timing : 6.30 AM to 3.30 PM or 7.00 AM to 4.00 PM (IST - India timing)

Experience

  • Minimum 4+ years as a Data Engineer or similar role.(Please don't apply if less than 4 years exp in Data Engineer)
  • Proven experience in Python, Spark, and PySpark (non-negotiable).(mandatory*)
  • Hands-on in building ETL pipelines, real-time streaming, and data transformations .
  • Worked with data warehouses, cloud platforms (AWS/Azure/GCP) , and databases .

Technical Skills

  • Spark Core API : RDDs, transformations/actions, DAG execution.
  • Spark SQL : DataFrames, schema optimization, UDFs.
  • Streaming : Structured Streaming, Kafka integration.
  • Data Handling : S3, HDFS, JDBC, Parquet, Avro, ORC.
  • Orchestration : Airflow/Prefect.
  • Performance Tuning : Partitioning, caching, broadcast joins.
  • Cloud Deployment : Databricks, AWS EMR, Azure HDInsight, GCP Dataproc.
  • CI/CD : Pytest/unittest for Spark jobs, Jenkins, GitHub Actions.

Education & Soft Skills

  • Bachelor's/Master's in Computer Science, Computer Engineering, or equivalent.
  • Strong analytical, problem-solving, and communication skills.

About the Role

We are seeking an experienced Data Engineer to join our team and support data-driven initiatives. This role involves building scalable pipelines, working with streaming data, and collaborating with data scientists and business stakeholders to deliver high-quality solutions.

Key Responsibilities

  • Design, build, and optimize data pipelines and ETL workflows .
  • Manage and process large datasets using Spark, PySpark, and SQL .
  • Build and maintain real-time streaming applications with Spark Streaming/Kafka.
  • Collaborate with data scientists and product teams to integrate AI/ML models into production.
  • Ensure data quality, scalability, and performance in all pipelines.
  • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks) .
  • Automate testing and deployment of Spark jobs via CI/CD pipelines.

Requirements

  • Bachelor's/Master's degree in Computer Science, Computer Engineering, or related field.
  • Minimum 4 years of professional experience as a Data Engineer.
  • Strong expertise in Python, Spark, PySpark .
  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution .
  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc .
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).

Budget

  • Onshore: As per market standards.

  • Senior Data Engineer

    3 weeks ago

    India Aptus Data Labs Full time

    Job Title: Senior Data EngineerLocation: RemoteExperience: 4+Employment Type: Full-TimeNotice- Looking for immediate joiners or who can join within 15-20 days.About the RoleAptus Data Labs is looking for a talented and proactive Senior Data Engineer to help build the backbone of our enterprise data and AI initiatives. You'll work on modern data lake...

  • Data Analyst

    2 days ago

    India Quant-data Full time

    We're Hiring: Data Analyst (Credit Analytics | Remote | Full-Time) Leverage financial and credit data to power AI-driven lending solutions We are seeking a skilled Data Analyst (Credit Analytics) with 4+ years of experience to join our AI-driven lending platform team . In this role, you will analyze financial, customer, and transaction datasets ,...

  • AI Security Engineer

    10 hours ago

    India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a AI Security Engineer - Shared Security Services Engineering to join our team in Banaglore, Karntaka (IN-KA), India...

  • Data Engineer

    4 weeks ago

    India NP Group Full time

    Data Engineer - Python, Pyspark & Typescript Fully Remote Long Term Contract Rate: $20 - $25 per hour ($3,000 - $4,000 per month)We have an immediate requirement for an experienced Data Engineer to join the global engineering team for a International Enterprise Organisation.You will bring Data Engineering expertise onto greenfield multi year project...

  • Data Engineer

    4 weeks ago

    India NP Group Full time

    Data Engineer - Python, Pyspark & Palantir FoundryFully Remote6 month full time contractRate: $13.00 - $15.00 per hourWe have an immediate requirement for an experienced Data Engineer to join the global engineering team for a International Enterprise Organisation.You will bring Data Engineering expertise onto greenfield multi year project initiative.You...

  • Data Engineer

    4 weeks ago

    India NP Group Full time

    Data Engineer - Python, Pyspark & TypescriptFully RemoteLong Term ContractRate: $14 - $17 per hourWe have an immediate requirement for an experienced Data Engineer to join the global engineering team for a International Enterprise Organisation.You will bring Data Engineering expertise onto greenfield multi year project initiative.You should offer at least...

  • Data Engineer

    12 hours ago

    India NP Group Full time

    Data Engineer - Palantir Foundry, Workshop, Pyspark & Typescript Fully Remote 6 month full time contract Rate: $15.00 per hour We have an immediate requirement for an experienced Data Engineer to join the global engineering team for a International Enterprise Organisation.You will bring Data Engineering expertise onto greenfield multi year project...

  • Data Engineer

    4 weeks ago

    India NP Group Full time

    Data Engineer - Python, Pyspark & Palantir FoundryFully Remote 6 month full time contractRate: $13.00 - $15.00 per hourWe have an immediate requirement for an experienced Data Engineer to join the global engineering team for a International Enterprise Organisation.You will bring Data Engineering expertise onto greenfield multi year project initiative.You...

  • Data Engineer

    4 weeks ago

    India NP Group Full time

    Data Engineer - Python, Pyspark & TypescriptFully Remote Long Term Contract Rate: $14 - $17 per hour We have an immediate requirement for an experienced Data Engineer to join the global engineering team for a International Enterprise Organisation.You will bring Data Engineering expertise onto greenfield multi year project initiative.You should offer at least...

  • Data Engineer

    4 days ago

    India Insight Global Full time

    100% Remote Data EngineerRequired Skills & Experience5+ years of experience as a Data EngineerExpertise in Python, Pyspark and SQLHands-on Python/PySpark coding experienceStrong Big Data experience, GCP preferredTelemetry experienceStreaming experience with analytics and data processingBusiness analytics experience and working directly with business partners...