Lead Data Engineer

4 days ago

Hyderabad, Telangana, India Zorba AI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Primary Job Title:
Data Engineering Lead

About The Opportunity
We are seeking a highly skilled
Lead Data Engineer
with strong expertise in
Python, Pandas, PySpark, AWS, and SQL
to design, build, and manage scalable data solutions. The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and business teams to ensure high data quality and performance across cloud-based environments.

Role & Responsibilities

  • Lead the design and development of data pipelines for ingestion, transformation, and integration from multiple sources into the enterprise data platform.
  • Implement data quality frameworks, validation checks, and monitoring solutions using Python and SQL.
  • Optimize PySpark jobs for performance and scalability in a distributed computing environment (AWS EMR/Glue).
  • Develop reusable ETL frameworks using PySpark and Pandas for data transformation and analytics.
  • Manage and maintain cloud-based infrastructure and data storage (AWS S3, Redshift, Lambda, Glue, Athena).
  • Collaborate with data scientists, analysts, and stakeholders to provide clean, structured, and accessible datasets.
  • Oversee code reviews, performance tuning, and mentoring junior data engineers.
  • Establish best practices for data governance, version control, and CI/CD integration.
  • Troubleshoot production data issues and ensure system reliability, availability, and efficiency.

Must-Have Skills

  • Programming: Python (advanced scripting, Pandas, PySpark)
  • Cloud: AWS (S3, Glue, Lambda, Redshift, Athena, EMR)
  • Database / Querying: Advanced SQL (Joins, Window Functions, Query Optimization)
  • Big Data Tools: PySpark, Spark SQL
  • ETL Development: Data ingestion, transformation, and validation pipelines
  • Version Control: Git / GitHub / Bitbucket
  • Workflow Orchestration: Apache Airflow or equivalent (preferred)

Good-to-Have Skills

  • Experience with Docker or Kubernetes for containerization.
  • Familiarity with CI/CD pipelines and DevOps practices.
  • Exposure to data modeling, schema design, and partitioning strategies.
  • Understanding of data lake and data warehouse architecture.
  • Knowledge of monitoring tools (CloudWatch, Datadog, or Prometheus).

Qualifications

  • Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
  • 11–16 years of total experience with at least 3–4 years leading data engineering projects.
  • Proven track record in handling large-scale data pipelines and cloud migrations.

Skills: python,aws,sql,pyspark

  • Azure Data Engineer

    8 hours ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • PeopleSoft to Workday Data Conversion Technical Lead

    3 days ago

    Hyderabad, Telangana, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job summary The PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools. The lead will manage the technical conversion team,...

  • Azure Data Engineer With Snowflake

    2 days ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Azure Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    16 hours ago

    Hyderabad, Telangana, India The Modern Data Company Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job InformationDate Opened11/07/2025IndustrySoftware ProductJob TypeFull timeCityHyderabadState/ProvinceTelanganaCountryIndiaZip/Postal Code500081About UsThe Modern Data Company is redefining data management for the AI era, transforming data from a technical challenge into a company's most powerful business asset. Modern's DataOS platform is the world's...

  • PeopleSoft To Workday Data Conversion Technical Lead

    1 week ago

    Hyderabad, Telangana, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job summaryThe PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools.The lead will manage the technical conversion team,...

  • Lead Data Engineer

    1 week ago

    Hyderabad, Telangana, India Proclink Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    DescriptionProclink is looking for Data Engineer Lead.Experience :9+ years.Location :Hyderabad.Position OverviewWe are seeking an experienced Data Engineer with over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions.The ideal candidate will lead our data engineering strategy, architect cloud-based data platforms,...

  • Lead Data Engineer

    1 week ago

    Hyderabad, Telangana, India Proclink Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description : Proclink is looking for Data Engineer Lead.Experience : 9 years.Location : Hyderabad.Position Overview : We are seeking an experienced Data Engineer with over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions. The ideal candidate will lead our data engineering strategy, architect...

  • Lead Data Engineer

    1 week ago

    Hyderabad, Telangana, India Zorba Consulting Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description : Primary Job Title : Data Engineering Lead.About The Opportunity : We are seeking a highly skilled Lead Data Engineer with strong expertise in Python, Pandas, PySpark, AWS, and SQL to design, build, and manage scalable data solutions. The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate...

  • Data Engineer Lead

    2 weeks ago

    Hyderabad, Telangana, India Proclink Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Proclinkis looking forData Engineer LeadExperience -9+ yearsLocation- HyderabadPosition OverviewWe are seeking an experiencedData Engineerwith over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions. The ideal candidate will lead our data engineering strategy, architect cloud-based data platforms, and mentor...