Lead Data Engineer

2 days ago

Hyderabad, Telangana, India Zorba Consulting Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Description :

Primary Job Title : Data Engineering Lead.

About The Opportunity :

We are seeking a highly skilled Lead Data Engineer with strong expertise in Python, Pandas, PySpark, AWS, and SQL to design, build, and manage scalable data solutions.

The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and business teams to ensure high data quality and performance across cloud-based environments.

Role & Responsibilities :

- Lead the design and development of data pipelines for ingestion, transformation, and integration from multiple sources into the enterprise data platform.

- Implement data quality frameworks, validation checks, and monitoring solutions using Python and SQL.

- Optimize PySpark jobs for performance and scalability in a distributed computing environment (AWS EMR/Glue).

- Develop reusable ETL frameworks using PySpark and Pandas for data transformation and analytics.

- Manage and maintain cloud-based infrastructure and data storage (AWS S3, Redshift, Lambda, Glue, Athena).

- Collaborate with data scientists, analysts, and stakeholders to provide clean, structured, and accessible datasets.

- Oversee code reviews, performance tuning, and mentoring junior data engineers.

- Establish best practices for data governance, version control, and CI/CD integration.

- Troubleshoot production data issues and ensure system reliability, availability, and efficiency.

Must-Have Skills :

- Programming : Python (advanced scripting, Pandas, PySpark).

- Cloud : AWS (S3, Glue, Lambda, Redshift, Athena, EMR).

- Database / Querying : Advanced SQL (Joins, Window Functions, Query Optimization).

- Big Data Tools : PySpark, Spark SQL.

- ETL Development : Data ingestion, transformation, and validation pipelines.

- Version Control : Git / GitHub / Bitbucket.

- Workflow Orchestration : Apache Airflow or equivalent (preferred).

Good-to-Have Skills :

- Experience with Docker or Kubernetes for containerization.

- Familiarity with CI/CD pipelines and DevOps practices.

- Exposure to data modeling, schema design, and partitioning strategies.

- Understanding of data lake and data warehouse architecture.

- Knowledge of monitoring tools (CloudWatch, Datadog, or Prometheus).

Qualifications :

- Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.

years of total experience with at least 3- 4 years leading data engineering projects.

- Proven track record in handling large-scale data pipelines and cloud migrations.

  • Azure Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Data Delivery Lead

    2 weeks ago

    Hyderabad, Telangana, India Data Unveil Full time

    Description : About us : Data Unveil is the next generation provider of best-in-class data aggregation services. Our solutions and methodologies have been honed over the past 10 years by experienced practitioners who have collectively supported 100 brands, 100 data partners and 25 new drug launches. Our integrated capabilities create a...

  • Azure Data Engineer

    5 days ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025)Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...

  • PeopleSoft To Workday Data Conversion Technical Lead

    2 days ago

    Hyderabad, Telangana, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job summaryThe PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools.The lead will manage the technical conversion team,...

  • Lead Data Engineer

    2 days ago

    Hyderabad, Telangana, India Proclink Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    DescriptionProclink is looking for Data Engineer Lead.Experience :9+ years.Location :Hyderabad.Position OverviewWe are seeking an experienced Data Engineer with over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions.The ideal candidate will lead our data engineering strategy, architect cloud-based data platforms,...

  • Lead Data Engineer

    2 days ago

    Hyderabad, Telangana, India Proclink Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description : Proclink is looking for Data Engineer Lead.Experience : 9 years.Location : Hyderabad.Position Overview : We are seeking an experienced Data Engineer with over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions. The ideal candidate will lead our data engineering strategy, architect...

  • Data Engineer Lead

    4 days ago

    Hyderabad, Telangana, India Proclink Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Proclinkis looking forData Engineer LeadExperience -9+ yearsLocation- HyderabadPosition OverviewWe are seeking an experiencedData Engineerwith over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions. The ideal candidate will lead our data engineering strategy, architect cloud-based data platforms, and mentor...

  • Lead Data Engineer

    7 days ago

    Hyderabad, Telangana, India EverestDX Full time ₹ 12,00,000 - ₹ 30,00,000 per year

    Description : We are seeking a highly skilled Lead Data Engineer to design, build, and scale robust data platforms and pipelines that empower business decisions and analytics. This role combines deep technical expertise with strategic leadership, guiding a team of data engineers to deliver scalable, reliable, and high-performing data solutions. The ideal...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025)Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...