Data Engineer

2 days ago

Mangalore, Karnataka, India R Systems Full time

Job Title: Data Engineer

Contract Period: 12 Months

Location: Offshore candidates accepted (Singapore Based Company)

Work Timing : 6.30 AM to 3.30 PM or 7.00 AM to 4.00 PM (IST - India timing)

Experience

  • Minimum 4+ years as a Data Engineer or similar role.(Please don't apply if less than 4 years exp in Data Engineer)
  • Proven experience in Python, Spark, and PySpark (non-negotiable).(mandatory*)
  • Hands-on in building ETL pipelines, real-time streaming, and data transformations.
  • Worked with data warehouses, cloud platforms (AWS/Azure/GCP), and databases.

Technical Skills

  • Spark Core API: RDDs, transformations/actions, DAG execution.
  • Spark SQL: DataFrames, schema optimization, UDFs.
  • Streaming: Structured Streaming, Kafka integration.
  • Data Handling: S3, HDFS, JDBC, Parquet, Avro, ORC.
  • Orchestration: Airflow/Prefect.
  • Performance Tuning: Partitioning, caching, broadcast joins.
  • Cloud Deployment: Databricks, AWS EMR, Azure HDInsight, GCP Dataproc.
  • CI/CD: Pytest/unittest for Spark jobs, Jenkins, GitHub Actions.

Education & Soft Skills

  • Bachelor's/Master's in Computer Science, Computer Engineering, or equivalent.
  • Strong analytical, problem-solving, and communication skills.

About the Role

We are seeking an experienced Data Engineer to join our team and support data-driven initiatives. This role involves building scalable pipelines, working with streaming data, and collaborating with data scientists and business stakeholders to deliver high-quality solutions.

Key Responsibilities

  • Design, build, and optimize data pipelines and ETL workflows.
  • Manage and process large datasets using Spark, PySpark, and SQL.
  • Build and maintain real-time streaming applications with Spark Streaming/Kafka.
  • Collaborate with data scientists and product teams to integrate AI/ML models into production.
  • Ensure data quality, scalability, and performance in all pipelines.
  • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks).
  • Automate testing and deployment of Spark jobs via CI/CD pipelines.

Requirements

  • Bachelor's/Master's degree in Computer Science, Computer Engineering, or related field.
  • Minimum 4 years of professional experience as a Data Engineer.
  • Strong expertise in Python, Spark, PySpark.
  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution.
  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).

Budget

  • Onshore: As per market standards.

  • Data Engineer Opportunity

    3 days ago

    Mangalore, Karnataka, India beBeeData Full time ₹ 1,25,00,000 - ₹ 2,25,00,000

    Job SummaryWe are seeking a highly skilled data engineer to join our team. As a key member of our engineering team, you will be responsible for designing, developing and implementing large-scale data solutions.This role involves working with various technologies such as distributed computing frameworks, data quality tools and scalable architectures to design...

  • Cloud Data Engineer

    3 days ago

    Mangalore, Karnataka, India beBeeData Full time US$ 70,000 - US$ 1,00,000

    Key Data Engineer RoleThe ideal candidate will have experience with cloud-based technologies, including GCP, Python, and ETL tools. They should be able to design, develop, and maintain large-scale data processing systems.Responsibilities include:Designing and developing data pipelines using GCP services such as BigQuery, Cloud Storage, and Cloud...

  • Senior Data Engineer

    5 days ago

    Mangalore, Karnataka, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineering LeaderJob DescriptionThe successful candidate will lead the design, development, and maintenance of scalable, secure, and high-performance data solutions on Databricks.He/She will collaborate with business stakeholders, data scientists, and analysts to gather requirements and deliver actionable data solutions.This role requires a...

  • Mastering Data Engineering

    1 day ago

    Mangalore, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Senior Data EngineerWe are seeking a skilled and experienced GCP data engineer to join our dynamic team. In this role, you will design, develop, and maintain data pipelines and data warehouse solutions on Google Cloud Platform (GCP). You will play a critical role in ensuring the seamless flow of data across systems, enabling data-driven...

  • Data Engineer

    6 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Seeking an accomplished Data Engineer to craft, build and oversee our data infrastructure and pipelines.This role is perfect for someone who has a proven track record of delivering multiple projects using GCP-Big Query, Data Proc technologies.The ideal candidate will be a collaborative team player with excellent problem-solving skills, strong communication...

  • Data Engineer Architect

    2 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    We're seeking a seasoned professional with expertise in Data Engineering and Generative AI.The ideal candidate will have 12–20 years of experience in data engineering, with at least 3 years (last 2 years acceptable) of hands-on work in Generative AI.This role will involve collaborating with Presales Managers to design technical solutions, leading client...

  • Leading Data Engineering Solutions

    3 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Job Title: Senior Data Engineer is a pivotal role in our data engineering team.As a seasoned and proficient Senior Python Data Engineer, you will play a crucial role in designing, implementing, and optimizing data pipelines ensuring seamless integration with cloud platforms. You will work closely with data scientists, analysts, and other teams to understand...

  • Senior Data Engineer

    20 hours ago

    Mangalore, Karnataka, India UnifyCX Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Required Experience8+ Years for Senior Data EngineerPreferred SkillsetsVery Strong Python skills.Very efficient in SQL/Stored procedures.At least 8 years of Hands on Experience in AWS Step function, AWS Lambda, AWS S3, AWS ECS, AWS Cloud Watch, AWS Event Bridge, AWS Athena, AWS Glue.At least 8 years of Experience in TerraformAt least 8 years of Experience in...

  • Master Data Engineer

    6 days ago

    Mangalore, Karnataka, India beBeeData Full time ₹ 2,50,00,000 - ₹ 3,50,00,000

    About the RoleAs a seasoned data expert, you'll craft datasets in Snowflake that fuel advanced analytics and machine learning across the business.The MissionExtracting value from powerful marketing platforms is your focus. You'll uncover hidden features, build intelligence layers, and enable others to scale smarter and faster.What You'll DoReverse-engineer...

  • Cloud Data Engineering Specialist

    3 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Overview:We are seeking a highly skilled Data Engineer to join our team.Key Responsibilities:Collaborate with data scientists to develop and implement complex algorithms for data insightsApply agile software development processes to improve back-end systems iteratively, ensuring efficient data processing and analysisIntegrate front-end and back-end data...