Lead Data Engineer

1 week ago

Pune, Maharashtra, India TerraGiG Full time

Job Title : Data Engineer

Experience : 8-12 Years

Location : Noida, Chennai & Pune

Skills : Python, Pyspark, Snowflake & Redshift, AWS, DBT, Data Modeling, Airflow, SQL

Key Responsibilities


• Migration & Modernization


• Lead the migration of data pipelines, models, and workloads from Redshift to Snowflake /Yellowbrick.


• Design and implement landing, staging, and curated data zones to support scalable ingestion and consumption patterns.


• Evaluate and recommend tools and frameworks for migration, including file formats, ingestion tools, and orchestration.


• Design and build robust ETL/ELT pipelines using Python, PySpark, SQL, and orchestration tools (e.g., Airflow, dbt).


• Support both batch and streaming pipelines, with real-time processing via Kafka, Snowpipe, or Spark Structured Streaming.


• Build modular, reusable, and testable pipeline components that handle high volume and ensure data integrity.


• Define and implement data modeling strategies (star, snowflake, normalization/denormalization) for analytics and BI layers.


• Implement strategies for data versioning, late-arriving data, and slowly changing dimensions.


• Implement automated data validation and anomaly detection (using tools like dbt tests, Great Expectations, or custom checks).


• Build logging and alerting into pipelines to monitor SLA adherence, data freshness, and pipeline health.


• Contribute to data governance initiatives including metadata tracking, data lineage, and access control.

Required Skills & Experience


• 10+ years in data engineering roles with increasing responsibility.


• Proven experience leading data migration or re-platforming projects.


• Strong command of Python, SQL, and PySpark for data pipeline development.


• Hands-on experience with modern data platforms like Snowflake, Redshift, Yellowbrick, or BigQuery.


• Proficient in building streaming pipelines with tools like Kafka, Flink, or Snowpipe.


• Deep understanding of data modeling, partitioning, indexing, and query optimization.


• Expertise with ETL orchestration tools (e.g., Apache Airflow, Prefect, Dagster, or dbt).


• Comfortable working with large datasets and solving performance bottlenecks.


• Experience in designing data validation frameworks and implementing DQ rules.

  • Data Engineering Lead

    2 days ago

    Pune, Maharashtra, India Ascentt Full time US$ 1,25,000 - US$ 1,75,000 per year

    Job Title: Data Engineering LeadJob Summary:We are seeking an experienced Data Engineering Lead to oversee and guide our organization's data engineering function. The ideal candidate will combine deep technical expertise in big data and data engineering with strong leadership skills to drive innovation, best practices, and excellence across our data...

  • Lead Data Engineer

    4 days ago

    Pune, Maharashtra, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering LeaderA seasoned Data Engineer to spearhead our data strategy and drive business outcomes.Job SummaryWe are seeking an experienced Data Engineer to lead our team in driving the development, implementation, and maintenance of data engineering solutions. The successful candidate will have a strong background in data engineering principles,...

  • Lead Data Engineer

    4 weeks ago

    Pune, Maharashtra, India VY SYSTEMS PRIVATE LIMITED Full time

    Immediate to 15 days and are highly encouraged to applyExperiences : 9+ YearsLocation : PuneRelocation candidates are also welcome.Job Summary :We are looking for a highly experienced and technically proficient Lead Data Engineer with 9+ years of experience to design, build, and optimize data pipelines and platforms in a cloud-native environment. This role...

  • Lead Data Engineer

    4 days ago

    Pune, Maharashtra, India Mastercard Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Title:Lead Data Engineer Overview:OverviewWe are the global technology company behind the world's fastest payments processing network. We are a vehicle for commerce, a connection to financial systems for the previously excluded, a technology innovation lab, and the home of Priceless. We ensure every employee has the opportunity to be a part of something...

  • Data Engineering Lead

    1 day ago

    Pune, Maharashtra, India beBeeDataEngineering Full time US$ 1,30,000 - US$ 1,70,000

    Job Title: Data Engineering LeadWe are seeking a highly skilled and experienced data engineering lead to design, implement, and manage large-scale data solutions using Azure technologies. The ideal candidate will have a proven track record of designing, implementing, and managing complex system integration processes focused primarily on the use of Azure SQL,...

  • Lead Data Engineer

    4 days ago

    Pune, Maharashtra, India Mastercard Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...

  • Lead Data Engineer

    2 days ago

    Pune, Maharashtra, India Mastercard Full time US$ 1,25,000 - US$ 1,75,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...

  • Lead Data Engineer

    2 days ago

    Pune, Maharashtra, India MSD Full time US$ 1,50,000 - US$ 2,00,000 per year

    Job DescriptionLead Data EngineerAt our Company we are leveraging analytics and technology, as we invent for life on behalf of patients around the world. We are seeking those who have a passion for using data, analytics, and insights to drive decision making, that will allow us to tackle some of the world's greatest health threats.Within our commercial...

  • Technical Lead – Data Engineering

    1 day ago

    Pune, Maharashtra, India Zywave Full time

    Job Description – Technical Lead (Data Engineering, TurboRater RQR: CPQ Rating)Location: Pune, IndiaWork Mode: 5 Days Work from OfficeAbout the RoleZywave is seeking a Technical Lead – Data Engineering (TurboRater RQR: CPQ Rating) with expertise in Snowflake, SQL Server, and modern data architecture principles including Medallion Architecture and Data...

  • Lead Salesforce QA Engineer(AccelQ/Opp reg/ CPQ)

    2 weeks ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Salesforce QA Engineer(AccelQ/Opp reg/ CPQ) to join our team in Pune, Mahrshtra (IN-MH), India (IN).Role: Lead QA...