Data Engineer

4 hours ago

bangalore, India R Systems Full time

Job Title: Data Engineer

Contract Period: 12 Months

Location: Offshore candidates accepted (Singapore Based Company)

Work Timing : 6.30 AM to 3.30 PM or 7.00 AM to 4.00 PM (IST - India timing)


Experience

  • Minimum 4+ years as a Data Engineer or similar role.(Please don't apply if less than 4 years exp in Data Engineer)
  • Proven experience in Python, Spark, and PySpark (non-negotiable).(mandatory*)
  • Hands-on in building ETL pipelines, real-time streaming, and data transformations.
  • Worked with data warehouses, cloud platforms (AWS/Azure/GCP), and databases.


✅ Technical Skills

  • Spark Core API: RDDs, transformations/actions, DAG execution.
  • Spark SQL: DataFrames, schema optimization, UDFs.
  • Streaming: Structured Streaming, Kafka integration.
  • Data Handling: S3, HDFS, JDBC, Parquet, Avro, ORC.
  • Orchestration: Airflow/Prefect.
  • Performance Tuning: Partitioning, caching, broadcast joins.
  • Cloud Deployment: Databricks, AWS EMR, Azure HDInsight, GCP Dataproc.
  • CI/CD: Pytest/unittest for Spark jobs, Jenkins, GitHub Actions.


✅ Education & Soft Skills

  • Bachelor’s/Master’s in Computer Science, Computer Engineering, or equivalent.
  • Strong analytical, problem-solving, and communication skills.


About the Role

We are seeking an experienced Data Engineer to join our team and support data-driven initiatives. This role involves building scalable pipelines, working with streaming data, and collaborating with data scientists and business stakeholders to deliver high-quality solutions.


Key Responsibilities

  • Design, build, and optimize data pipelines and ETL workflows.
  • Manage and process large datasets using Spark, PySpark, and SQL.
  • Build and maintain real-time streaming applications with Spark Streaming/Kafka.
  • Collaborate with data scientists and product teams to integrate AI/ML models into production.
  • Ensure data quality, scalability, and performance in all pipelines.
  • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks).
  • Automate testing and deployment of Spark jobs via CI/CD pipelines.


Requirements

  • Bachelor’s/Master’s degree in Computer Science, Computer Engineering, or related field.
  • Minimum 4 years of professional experience as a Data Engineer.
  • Strong expertise in Python, Spark, PySpark.
  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution.
  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).


Budget

  • Onshore: As per market standards.

  • Data Engineer

    5 hours ago

    bangalore, India Bahwan CyberTek Full time

    Job Title: Data Engineer – Google Cloud Platform (GCP) Job Summary We are seeking a skilled and motivated Data Engineer with hands-on experience in building scalable data pipelines and cloud-native data solutions on Google Cloud Platform. The ideal candidate will be proficient in GCP services like Pub/Sub, Dataflow, Cloud Storage, and BigQuery, with a...

  • Data Engineer

    24 hours ago

    Bangalore, India Synechron Full time

    Good-day, We have immediate opportunity for Data Engineer. Job Role: Data Engineer Job Location: Synechron ( Bengaluru) Experience- 7 to 12 years Notice : Only immediate joiner . About Company: At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity...

  • Data Engineer

    4 hours ago

    bangalore, India Synechron Full time

    Good-day,We have immediate opportunity for Data Engineer.Job Role: Data EngineerJob Location: Synechron ( Bengaluru)Experience- 7 to 12 yearsNotice : Only immediate joiner .About Company:At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver...

  • Data Engineer

    24 hours ago

    Bangalore, India Alp Consulting Ltd. Full time

    Role: Data Engineer Location : Bangalore Experience: 3 to 8Years SKILLS & CERTIFICATIONS Strong interest and acumen for solving large-scale data engineering problems. Hands-on technical and functional experience in building large scale data engineering capabilities in diverse big-data environments Strong hands-on expertise in data...

  • Data Engineer

    3 hours ago

    bangalore, India Alp Consulting Ltd. Full time

    Role: Data EngineerLocation : BangaloreExperience: 3 to 8YearsSKILLS & CERTIFICATIONSStrong interest and acumen for solving large-scale data engineering problems.Hands-on technical and functional experience in building large scale data engineering capabilities in diverse big-data environmentsStrong hands-on expertise in data engineering:Enterprise grade ETL...

  • Data Engineer

    6 hours ago

    bangalore, India Insight Global Full time

    100% Remote Data Engineer Required Skills & Experience5+ years of experience as a Data EngineerExpertise in Python, Pyspark and SQLHands-on Python/PySpark coding experienceStrong Big Data experience, GCP preferredTelemetry experienceStreaming experience with analytics and data processingBusiness analytics experience and working directly with business...

  • Data Engineer

    24 hours ago

    Bangalore, India Recro Full time

    Role - Data Engineer Location - Bangalore Experience - 2-4 Yrs Key Responsibilities Design, build, and optimize scalable data pipelines for batch and real-time data processing. Develop and maintain ETL (Extract, Transform, Load) processes to integrate data from multiple sources. Manage and optimize data storage solutions (data warehouses,...

  • Data Engineer

    4 hours ago

    bangalore, India Recro Full time

    Role - Data EngineerLocation - BangaloreExperience - 2-4 YrsKey ResponsibilitiesDesign, build, and optimize scalable data pipelines for batch and real-time data processing.Develop and maintain ETL (Extract, Transform, Load) processes to integrate data from multiple sources.Manage and optimize data storage solutions (data warehouses, data lakes, relational...

  • Data Engineer

    4 hours ago

    bangalore, India VAANTECH Full time

    We’re Hiring: Data EngineerPosition: Data EngineerExperience Required: 3+ YearsNotice Period: Immediate Joiner (or within 15 days)Location: Chennai (Remote)Shift: Includes Night Shifts (US Shift)Preferred Candidates: From Chennai, Andhra Pradesh, Kerala & BangaloreSkills We’re Looking For:SQL, Python, ETLApache AirflowData Pipeline Optimization &...

  • Data Engineer

    39 minutes ago

    Bangalore, India Us Technology International Private Limited Full time

    Role: Data Engineer Experience 3 -6 years Location: Bangalore 1.Bachelor’s or master’s degree in computer science, Data Science, Engineering, or a related field. 2.Proven experience in predictive modeling using Python. 3.Strong analytical and problem-solving skills. 4.Proficiency in MATLAB and Databricks. 5.Familiarity with Python libraries like...