Data Engineer-Apache Spark

2 weeks ago

Bengaluru Chennai, India Wroots Global Full time ₹ 8,00,000 - ₹ 12,00,000 per year

Immediate Joiners-Preferred

Contact

Amreen

Rajeshwari

Tejaswini

Role Summary

We are seeking Senior Data Engineer resources to work on the migration of applications from

our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires

strong hands-on development skills in data engineering, with the ability to deliver high-quality

pipelines under guidance from internal leads.

Key Responsibilities

Develop and optimize data pipelines using Spark 3.5 and Python/Scala.

Migrate existing Hive, Spark, and Control-M jobs to Airflow and DBT-based workflows.

Integrate data pipelines with messaging systems (Kafka, Solace) and object stores (S3,

MinIO).

Troubleshoot and optimize distributed jobs running in Kubernetes environments.

Collaborate closely with internal leads and architects to implement best practices.

Design and implement migration/acceleration framework to automate end to end

migration.

Continuous enhancements to the frameworks to ensure the stability, scalability and

support for diverse use cases and scenarios.

Work with various data applications to enable and support the migration process.

Deliver assigned migration tasks within agreed timelines.

Required Skills

6 9 years of hands-on data engineering experience.

Strong expertise in Apache Spark (batch + streaming) and Hive.

Proficiency in Python, Scala, or Java.

Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation

frameworks (DBT preferred).

Experience working with Kafka, Solace, and object stores (S3, MinIO).

Exposure to Docker/Kubernetes for deployment.

Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).

INTERNAL

Engagement Expectations

Vendor resource will be expected to work independently on assigned modules and

deliver production-ready solutions.

Must participate in daily/weekly status calls, reviews, and issue resolution.

Should adapt to client processes, documentation standards, and compliance

requirements.

  • Staff Apache Spark Engineer

    2 weeks ago

    Bengaluru, India BairesDev Full time

    Job Description At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this...

  • Principal Data Engineer

    4 weeks ago

    Bengaluru, India Neemtree Full time

    Description :- Define and lead the data architecture vision and strategy, ensuring it supports analytics, ML, and business operations at scale.- Architect and manage cloud-native data platforms using Databricks and AWS, leveraging the lakehouse architecture to unify data engineering and ML workflows.- Build and optimize large-scale batch and streaming...

  • Apache scala spark

    2 weeks ago

    Bengaluru, Chennai, India Athena Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    email-client -MNCApache ,spark ,SCALA requirementlocation -Chennai and Bangalorenotice period - immediate joining - joining 1 decexperience -6-9 years and 9+ years- Grade B2 and B3Mandatory skill Apache Spark and Scala (Python not reqd)

  • Senior Data Architect

    2 weeks ago

    Bengaluru, India Data Engineer Academy Full time

    We're Hiring: Data Engineering Subject Matter Expert (SME) – Snowflake,Cloud & DatabricksRemote | Part Time | Flexible Timings | Pay: $600 – $1000/MonthlyAbout the OpportunityWe are seeking a highly experienced SME with extensive expertise in Snowflake, Databricks, and AWS cloud platforms.What We're Looking ForMinimum 9-15 years of experience in cloud...

  • Data Engineering Subject Matter Expert

    2 weeks ago

    Bengaluru, Karnataka, India Data Engineer Academy Full time US$ 4,80,000 - US$ 12,00,000 per year

    We're Hiring: Data Engineering Subject Matter Expert (SME) – Snowflake,Cloud & DatabricksRemote | Part Time | Flexible Timings | Pay: $600 – $1000/MonthlyAbout the OpportunityWe are seeking a highly experienced SME with extensive expertise in Snowflake, Databricks, and AWS cloud platforms.What We're Looking ForMinimum 9-15 years of experience in cloud...

  • Data Engineering Subject Matter Expert

    2 weeks ago

    Bengaluru, India Data Engineer Academy Full time

    We're Hiring: Data Engineering Subject Matter Expert (SME) – Snowflake,Cloud & Databricks Remote | Part Time | Flexible Timings | Pay: $600 – $1000/MonthlyAbout the OpportunityWe are seeking a highly experienced SME with extensive expertise in Snowflake, Databricks, and AWS cloud platforms.What We're Looking ForMinimum 9-15 years of experience in...

  • Data Engineering Subject Matter Expert

    2 weeks ago

    Bengaluru, India Data Engineer Academy Full time

    We're Hiring: Data Engineering Subject Matter Expert (SME) – Snowflake,Cloud & DatabricksRemote | Part Time | Flexible Timings | Pay: $600 – $1000/MonthlyAbout the OpportunityWe are seeking a highly experienced SME with extensive expertise in Snowflake, Databricks, and AWS cloud platforms.What We're Looking ForMinimum 9-15 years of experience in cloud...

  • Data Engineering Subject Matter Expert

    2 weeks ago

    Bengaluru, India Data Engineer Academy Full time

    We're Hiring: Data Engineering Subject Matter Expert (SME) – Snowflake,Cloud & DatabricksRemote | Part Time | Flexible Timings | Pay: $600 – $1000/MonthlyAbout the OpportunityWe are seeking a highly experienced SME with extensive expertise in Snowflake, Databricks, and AWS cloud platforms.What We're Looking ForMinimum 9-15 years of experience in cloud...

  • Data Engineering Subject Matter Expert

    2 weeks ago

    Bengaluru, India Data Engineer Academy Full time

    We're Hiring: Data Engineering Subject Matter Expert (SME) – Snowflake,Cloud & DatabricksRemote | Part Time | Flexible Timings | Pay: $600 – $1000/MonthlyAbout the OpportunityWe are seeking a highly experienced SME with extensive expertise in Snowflake, Databricks, and AWS cloud platforms.What We're Looking For- Minimum 9-15 years of experience in cloud...

  • Data Engineering Subject Matter Expert

    2 weeks ago

    Bengaluru, India Data Engineer Academy Full time

    We're Hiring: Data Engineering Subject Matter Expert (SME) – Snowflake,Cloud & DatabricksRemote | Part Time | Flexible Timings | Pay: $600 – $1000/MonthlyAbout the OpportunityWe are seeking a highly experienced SME with extensive expertise in Snowflake, Databricks, and AWS cloud platforms.What We're Looking ForMinimum 9-15 years of experience in cloud...