Lead Data Engineer

6 days ago

Hyderabad, Telangana, India Zorba AI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Primary Job Title:
Data Engineering Lead

About The Opportunity
We are seeking a highly skilled
Lead Data Engineer
with strong expertise in
Python, Pandas, PySpark, AWS, and SQL
to design, build, and manage scalable data solutions. The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and business teams to ensure high data quality and performance across cloud-based environments.

Role & Responsibilities

  • Lead the design and development of data pipelines for ingestion, transformation, and integration from multiple sources into the enterprise data platform.
  • Implement data quality frameworks, validation checks, and monitoring solutions using Python and SQL.
  • Optimize PySpark jobs for performance and scalability in a distributed computing environment (AWS EMR/Glue).
  • Develop reusable ETL frameworks using PySpark and Pandas for data transformation and analytics.
  • Manage and maintain cloud-based infrastructure and data storage (AWS S3, Redshift, Lambda, Glue, Athena).
  • Collaborate with data scientists, analysts, and stakeholders to provide clean, structured, and accessible datasets.
  • Oversee code reviews, performance tuning, and mentoring junior data engineers.
  • Establish best practices for data governance, version control, and CI/CD integration.
  • Troubleshoot production data issues and ensure system reliability, availability, and efficiency.

Must-Have Skills

  • Programming: Python (advanced scripting, Pandas, PySpark)
  • Cloud: AWS (S3, Glue, Lambda, Redshift, Athena, EMR)
  • Database / Querying: Advanced SQL (Joins, Window Functions, Query Optimization)
  • Big Data Tools: PySpark, Spark SQL
  • ETL Development: Data ingestion, transformation, and validation pipelines
  • Version Control: Git / GitHub / Bitbucket
  • Workflow Orchestration: Apache Airflow or equivalent (preferred)

Good-to-Have Skills

  • Experience with Docker or Kubernetes for containerization.
  • Familiarity with CI/CD pipelines and DevOps practices.
  • Exposure to data modeling, schema design, and partitioning strategies.
  • Understanding of data lake and data warehouse architecture.
  • Knowledge of monitoring tools (CloudWatch, Datadog, or Prometheus).

Qualifications

  • Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
  • 11–16 years of total experience with at least 3–4 years leading data engineering projects.
  • Proven track record in handling large-scale data pipelines and cloud migrations.

Skills: python,aws,sql,pyspark

  • Azure Data Engineer

    2 days ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • PeopleSoft to Workday Data Conversion Technical Lead

    4 days ago

    Hyderabad, Telangana, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job summary The PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools. The lead will manage the technical conversion team,...

  • Azure Data Engineer

    2 days ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Azure Data Engineer

    4 days ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • PeopleSoft To Workday Data Conversion Technical Lead

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job summaryThe PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools.The lead will manage the technical conversion team,...

  • Data Conversion Technical Lead

    2 days ago

    Hyderabad, Telangana, India NTT DATA Global Delivery Services Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    PeopleSoft to Workday Data Conversion Technical Lead Job summary The PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools....

  • Lead Data Engineer

    1 week ago

    Hyderabad, Telangana, India Applore Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Company DescriptionApplore Technologies is a premium consulting and product development company, specializing in AI-driven strategy, data-led decision-making, digital product innovation, and high-impact digital marketing. The company helps businesses unlock new opportunities through AI, data science, and full-stack engineering. With a global presence in...

  • Lead Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Lera Technologies Pvt. Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    What You Bring : years of experience in Data Engineering, including SQL/PLSQL/TSQL, ETL development, and data pipeline architecture - Strong command of ETL tools (SSIS or equivalent) and Data Warehousing concepts - Expertise in data modeling, architecture, and integration frameworks - Experience leading data teams and managing end-to-end data...

  • Lead Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Lera Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    What You Bring7+ years of experience in Data Engineering, including SQL/PLSQL/TSQL, ETL development, and data pipeline architectureStrong command of ETL tools (SSIS or equivalent) and Data Warehousing conceptsExpertise in data modeling, architecture, and integration frameworksExperience leading data teams and managing end-to-end data delivery across...

  • PeopleSoft to Workday Data Conversion Technical Lead

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA North America Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job summaryThe PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools. The lead will manage the technical conversion team,...