Lead Data Engineer

3 days ago

Hyderabad, Telangana, India Zorba Consulting Full time ₹ 12,00,000 - ₹ 24,00,000 per year

About The Opportunity :

We are seeking a highly skilled Lead Data Engineer with strong expertise in Python, Pandas, PySpark, AWS, and SQL to design, build, and manage scalable data solutions.

The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and business teams to ensure high data quality and performance across cloud-based environments.

Role & Responsibilities :

  • Lead the design and development of data pipelines for ingestion, transformation, and integration from multiple sources into the enterprise data platform.

  • Implement data quality frameworks, validation checks, and monitoring solutions using Python and SQL.

  • Optimize PySpark jobs for performance and scalability in a distributed computing environment (AWS EMR/Glue).

  • Develop reusable ETL frameworks using PySpark and Pandas for data transformation and analytics.

  • Manage and maintain cloud-based infrastructure and data storage (AWS S3, Redshift, Lambda, Glue, Athena).

  • Collaborate with data scientists, analysts, and stakeholders to provide clean, structured, and accessible datasets.

  • Oversee code reviews, performance tuning, and mentoring junior data engineers.

  • Establish best practices for data governance, version control, and CI/CD integration.

  • Troubleshoot production data issues and ensure system reliability, availability, and efficiency.

Must-Have Skills :

  • Programming : Python (advanced scripting, Pandas, PySpark).

  • Cloud : AWS (S3, Glue, Lambda, Redshift, Athena, EMR).

  • Database / Querying : Advanced SQL (Joins, Window Functions, Query Optimization).

  • Big Data Tools : PySpark, Spark SQL.

  • ETL Development : Data ingestion, transformation, and validation pipelines.

  • Version Control : Git / GitHub / Bitbucket.

  • Workflow Orchestration : Apache Airflow or equivalent (preferred).

Good-to-Have Skills :

  • Experience with Docker or Kubernetes for containerization.

  • Familiarity with CI/CD pipelines and DevOps practices.

  • Exposure to data modeling, schema design, and partitioning strategies.

  • Understanding of data lake and data warehouse architecture.

  • Knowledge of monitoring tools (CloudWatch, Datadog, or Prometheus).

Qualifications :

  • Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.

  • 11- 16 years of total experience with at least 3- 4 years leading data engineering projects.

  • Proven track record in handling large-scale data pipelines and cloud migrations.

  • Azure Data Engineer

    1 week ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • PeopleSoft to Workday Data Conversion Technical Lead

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job summary The PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools. The lead will manage the technical conversion team,...

  • Azure Data Engineer With Snowflake

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    1 week ago

    Hyderabad, Telangana, India The Modern Data Company Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job InformationDate Opened11/07/2025IndustrySoftware ProductJob TypeFull timeCityHyderabadState/ProvinceTelanganaCountryIndiaZip/Postal Code500081About UsThe Modern Data Company is redefining data management for the AI era, transforming data from a technical challenge into a company's most powerful business asset. Modern's DataOS platform is the world's...

  • Lead Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Zorba AI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Primary Job Title:Data Engineering LeadAbout The OpportunityWe are seeking a highly skilledLead Data Engineerwith strong expertise inPython, Pandas, PySpark, AWS, and SQLto design, build, and manage scalable data solutions. The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and...

  • Lead Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India SourcingXPress Full time ₹ 2,50,00,000 - ₹ 3,00,00,000 per year

    Company:KanerikaWebsite:Visit WebsiteBusiness Type:StartupCompany Type:Product & ServiceBusiness Model:B2BFunding Stage:Pre-seedIndustry:Data and AISalary Range:₹ 25-30 Lacs PAJob DescriptionLead Data Engineer (Architect):We're looking for an experiencedData Engineering Lead/Architectto design, build, and scale modern, cloud-based data ecosystems that...

  • Data Conversion Technical Lead

    1 week ago

    Hyderabad, Telangana, India NTT DATA Global Delivery Services Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    PeopleSoft to Workday Data Conversion Technical Lead Job summary The PeopleSoft to Workday Data Conversion Technical Lead will be responsible for the technical design, development, and execution of the data migration from PeopleSoft to Workday. This hands-on role requires deep technical expertise in both PeopleSoft and Workday data structures and tools....

  • Lead - Data Engineering

    3 days ago

    Hyderabad, Telangana, India Lera Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position:Lead - Data EngineeringLocations: HyderabadExperience Level:7-10 yearsAbout UsLera Technologies is a future-focused, AI-led digital transformation company that empowers businesses to innovate and grow in today's fast-paced technology landscape. Our core strength lies in our flagship products like the 9X Data Platform which is a state-of-the-art...

  • Lead Data Engineer

    3 days ago

    Hyderabad, Telangana, India Talent Worx Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    one of the leading healthcare technology Product organizations with $1.4 Billion revenue with year-on-year growth of 22% and client retention rate of 120%. we offer wide range of innovative solutions to the healthcare payers, providers, and consumers. Our services include network analytics, payment integrity and optimization, provider credentialing, and...

  • Lead Data Engineer

    5 days ago

    Hyderabad, Telangana, India Talent Worx Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    one of the leading healthcare technology Product organizations with $1.4 Billion revenue with year-on-year growth of 22% and client retention rate of 120%. we offer wide range of innovative solutions to the healthcare payers, providers, and consumers. Our services include network analytics, payment integrity and optimization, provider credentialing, and...