Lead Data Engineer

6 days ago

Gurugram, India SUPERSOURCING TECHNOLOGIES PRIVATE LIMITED Full time


About the Role:


We are looking for an experienced Lead Data Engineer with deep expertise in Big Data technologies, particularly within the Google Cloud Platform (GCP) ecosystem. The ideal candidate should have a strong command of PySpark/Spark, SQL, and Python, and a proven track record in building, optimizing, and managing large- scale data pipelines and cloud- native data platforms.

Key Responsibilities:


- Lead the design and implementation of scalable ETL/ELT pipelines using Spark (batch and stream) and Python


- Architect and optimize BigQuery solutions using advanced SQL, partitioning, clustering, and materialized views


- Guide the team on GCP services: Dataproc, GCS, BigQuery, Cloud Composer, Cloud Functions, IAM, and Cloud Logging


- Conduct code reviews and mentor team members on Spark optimization (caching, memory management, broadcast joins, skew handling)


- Drive Airflow DAG development, configuration management, and orchestration workflows


- Solve complex data engineering problems and contribute to architectural decisions for performance, scalability, and cost- efficiency


- Ensure data quality, governance, and security best practices are enforced across all data platforms


- Support team readiness through technical ramp- up and ongoing skill enhancement

Must- Have Skills:


- Hands- on experience with PySpark/Spark core concepts, internal workings, transformations, and tuning


- Strong knowledge of SQL and BigQuery (including window functions, CTEs, performance tuning, joins)


- Proficiency in Python with strong problem- solving abilities


- Deep experience with GCP components: BigQuery, Dataproc, GCS, and Cloud Composer


- Understanding of Airflow, including XComs, variables, schema- based DAG creation, and branching


- Exposure to Hive, partitioning (static/dynamic), and bucketed tables


- Familiarity with data pipeline orchestration, monitoring, and failure handling


- Solid grasp of data security (column- level, row- level, IAM roles)

(ref:hirist.tech)

  • Lead Data Engineer

    4 days ago

    Gurugram, India Vipsa Talent Solutions Full time

    • Lead the data engineering team by driving the design, build, test, and launch of new data pipelines and models on the production data platform

  • Principal Software/Data Engineering

    4 days ago

    Gurugram, India CoPoint Data Full time

    About CoPoint Data CoPoint Data is a specialized consulting firm focused on transforming businesses through process improvement, data insights, and technology-driven innovation. We leverage AI technologies, Microsoft cloud platforms, and modern web development frameworks to deliver intelligent, scalable solutions that drive measurable impact for our clients....

  • Lead Data Engineer

    4 days ago

    Gurugram, India WOW Softech Full time

    Exp- 8-15 years 3+ years of hands on experience in Azure Data Engineering development project 2+ years of experience in Databricks development 2+ years of experience in PySpark. 2+ years of experience in SQL Proficient in SQL 2+ years of experience in leading at least a 5+ member team in data engineering delivery Must be fluent in communication Role &...

  • Lead Data Engineer

    4 days ago

    Gurugram, India CBRE Full time

    Gurgaon - Haryana - India, Hyderabad - Telangana - India, Noida - Uttar Pradesh - India About The RoleAs a Senior Lead Data Engineer, you will play a crucial role in designing, building, and optimizing our data pipeline and infrastructure. You will work closely with stakeholders, analysts, and other to ensure data integrity, availability, and security. This...

  • Senior Web Developer and Data Engineer

    4 days ago

    Gurugram, India CoPoint Data Full time

    About CoPoint AI CoPoint AI is a specialized consulting firm focused on transforming businesses through process improvement, data insights, and technology-driven innovation. We leverage AI technologies, Microsoft cloud platforms, and modern web development frameworks to deliver intelligent, scalable solutions that drive measurable impact for our clients. Our...

  • Senior/Lead Data Engineer

    4 days ago

    Gurugram, India Srijan Technologies Pvt Ltd Full time

    About US:- We turn customer challenges into growth opportunities. Material is a global strategy partner to the world's most recognizable brands and innovative companies. Our people around the globe thrive by helping organizations design and deliver rewarding customer experiences. We use deep human insights, design innovation and data to create experiences...

  • Data Engineer Lead

    4 days ago

    Gurugram, India Bounteous x Accolite Full time

    Key Responsibilities: Responsible for implementation of scalable, secure, and high-performance data pipelines. Design and develop ETL processes using AWS services (Lambda, S3, Glue, Step Functions, etc.). Own and enhance API design and integrations for internal and external data systems. Work closely with data scientists, analysts, and software engineers to...

  • Data Engineer

    4 days ago

    Gurugram, India NatWest Group Full time

    Join us as a Data Engineering Lead This is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiences You'll be simplifying the bank through developing innovative data driven solutions, inspiring to be commercially successful through insight, and keeping our customers' and...

  • Lead AI Data Engineer

    4 days ago

    Gurugram, India S&P Global Full time

    Lead AI Data Engineer Gurgaon, India; Noida, India Information Technology Job Description About The Role: Grade Level (for internal use): 11 Summary: As a Data Engineer & AI Developer, you will play a key role in transforming complex data into clear, actionable insights. You will be responsible for building robust data pipelines, developing interactive...

  • Data Engineering Lead

    2 weeks ago

    Delhi, Gurugram, NCR, Noida, India Bounteous x Accolite Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Role & responsibilitiesSkill - Data engineer python AWSExp - 10+Location - GurugramNP - only immediate joiners neededPreferred candidate profileWe are seeking an experienced Lead Data Engineer with strong expertise in Python, AWS cloud services, ETL pipelines, and system integrations. The ideal candidate will lead the design, development, and optimization of...