Lead Data Engineer

4 weeks ago

Pune, Maharashtra, India VY SYSTEMS PRIVATE LIMITED Full time

Immediate to 15 days and are highly encouraged to apply

Experiences : 9+ Years

Location : Pune

Relocation candidates are also welcome.

Job Summary :


We are looking for a highly experienced and technically proficient Lead Data Engineer with 9+ years of experience to design, build, and optimize data pipelines and platforms in a cloud-native environment. This role requires deep expertise in Python, SQL, Apache Airflow, Apache Spark, and AWS services such as S3, Lambda, and Glue. You will play a key role in leading data engineering initiatives, mentoring engineers, and driving data architecture strategy.

Key Responsibilities :


- Lead the design, development, and deployment of scalable data pipelines using Apache Airflow, Apache Spark, and Python.

- Build and maintain ETL/ELT workflows for structured and unstructured data across various data sources.

- Architect data solutions using AWS cloud-native services including S3, Lambda, Glue, and Athena.

- Optimize large-scale data processing and storage, ensuring performance, security, and cost-efficiency.

- Collaborate with cross-functional teams including data scientists, analysts, and DevOps to deliver high quality data products.

- Drive best practices for coding, testing, version control (Git), and CI/CD in data engineering.

- Mentor junior data engineers and promote a high-performance, collaborative team culture.

- Monitor data pipelines, troubleshoot production issues, and ensure data quality and reliability.

Required Skills and Experience :


- 9+ years of hands-on experience in data engineering with strong architectural understanding.

- Expert in Python and SQL for data transformation and pipeline development.

- Deep experience with Apache Airflow for orchestration and Apache Spark for distributed data processing.

- Proven expertise in AWS cloud services: S3, Lambda, Glue, Redshift, and related tools.

- Strong experience in designing and maintaining ETL/ELT pipelines and batch/stream processing frameworks.

- Solid understanding of data modeling, data warehousing, and data lake architecture.

- Familiarity with infrastructure-as-code (Terraform or CloudFormation) and CI/CD tools.

- Strong communication skills and experience leading technical teams or projects.

(ref:hirist.tech)

  • Data Engineering Lead

    9 hours ago

    Pune, Maharashtra, India Ascentt Full time US$ 1,25,000 - US$ 1,75,000 per year

    Job Title: Data Engineering LeadJob Summary:We are seeking an experienced Data Engineering Lead to oversee and guide our organization's data engineering function. The ideal candidate will combine deep technical expertise in big data and data engineering with strong leadership skills to drive innovation, best practices, and excellence across our data...

  • Lead Data Engineer

    3 days ago

    Pune, Maharashtra, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering LeaderA seasoned Data Engineer to spearhead our data strategy and drive business outcomes.Job SummaryWe are seeking an experienced Data Engineer to lead our team in driving the development, implementation, and maintenance of data engineering solutions. The successful candidate will have a strong background in data engineering principles,...

  • Lead Data Engineer

    2 days ago

    Pune, Maharashtra, India Mastercard Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Title:Lead Data Engineer Overview:OverviewWe are the global technology company behind the world's fastest payments processing network. We are a vehicle for commerce, a connection to financial systems for the previously excluded, a technology innovation lab, and the home of Priceless. We ensure every employee has the opportunity to be a part of something...

  • Lead Data Engineer

    2 days ago

    Pune, Maharashtra, India Mastercard Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...

  • Lead Data Engineer

    9 hours ago

    Pune, Maharashtra, India Mastercard Full time US$ 1,25,000 - US$ 1,75,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...

  • Lead Data Engineer

    22 hours ago

    Pune, Maharashtra, India MSD Full time US$ 1,50,000 - US$ 2,00,000 per year

    Job DescriptionLead Data EngineerAt our Company we are leveraging analytics and technology, as we invent for life on behalf of patients around the world. We are seeking those who have a passion for using data, analytics, and insights to drive decision making, that will allow us to tackle some of the world's greatest health threats.Within our commercial...

  • Lead Salesforce QA Engineer(AccelQ/Opp reg/ CPQ)

    2 weeks ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Salesforce QA Engineer(AccelQ/Opp reg/ CPQ) to join our team in Pune, Mahrshtra (IN-MH), India (IN).Role: Lead QA...

  • Lead Data Engineer

    1 week ago

    Pune, Maharashtra, India TerraGiG Full time

    Job Title : Data Engineer Experience : 8-12 Years Location : Noida, Chennai & Pune Skills : Python, Pyspark, Snowflake & Redshift, AWS, DBT, Data Modeling, Airflow, SQL Key Responsibilities • Migration & Modernization • Lead the migration of data pipelines, models, and workloads from Redshift to Snowflake /Yellowbrick. • Design and...

  • Lead Big Data Engineer

    4 weeks ago

    Pune, Maharashtra, India algoleap Full time

    Job DescriptionDescriptionWe are seeking a talented Lead Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks.Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable,...

  • 2723 - Lead Data Engineer

    3 weeks ago

    Pune, Maharashtra, India EXL Full time

    Role -Lead Data Engineer Years of experience: 10+Must Have SkillsProficient in snowflake and SQL4+ years of experience in snowflake and 8+ years of experience in SQLAtleast 10+ years of experience in data engineering development projectAtleast 6+ years of experience in Data Engineering in the cloud technologyStrong expertise with Snowflake data warehouse...