Lead Data Engineer

1 week ago

Hyderabad, Telangana, India Zorba Consulting Full time ₹ 12,00,000 - ₹ 24,00,000 per year

About The Opportunity :

We are seeking a highly skilled Lead Data Engineer with strong expertise in Python, Pandas, PySpark, AWS, and SQL to design, build, and manage scalable data solutions.

The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and business teams to ensure high data quality and performance across cloud-based environments.

Role & Responsibilities :

  • Lead the design and development of data pipelines for ingestion, transformation, and integration from multiple sources into the enterprise data platform.

  • Implement data quality frameworks, validation checks, and monitoring solutions using Python and SQL.

  • Optimize PySpark jobs for performance and scalability in a distributed computing environment (AWS EMR/Glue).

  • Develop reusable ETL frameworks using PySpark and Pandas for data transformation and analytics.

  • Manage and maintain cloud-based infrastructure and data storage (AWS S3, Redshift, Lambda, Glue, Athena).

  • Collaborate with data scientists, analysts, and stakeholders to provide clean, structured, and accessible datasets.

  • Oversee code reviews, performance tuning, and mentoring junior data engineers.

  • Establish best practices for data governance, version control, and CI/CD integration.

  • Troubleshoot production data issues and ensure system reliability, availability, and efficiency.

Must-Have Skills :

  • Programming : Python (advanced scripting, Pandas, PySpark).

  • Cloud : AWS (S3, Glue, Lambda, Redshift, Athena, EMR).

  • Database / Querying : Advanced SQL (Joins, Window Functions, Query Optimization).

  • Big Data Tools : PySpark, Spark SQL.

  • ETL Development : Data ingestion, transformation, and validation pipelines.

  • Version Control : Git / GitHub / Bitbucket.

  • Workflow Orchestration : Apache Airflow or equivalent (preferred).

Good-to-Have Skills :

  • Experience with Docker or Kubernetes for containerization.

  • Familiarity with CI/CD pipelines and DevOps practices.

  • Exposure to data modeling, schema design, and partitioning strategies.

  • Understanding of data lake and data warehouse architecture.

  • Knowledge of monitoring tools (CloudWatch, Datadog, or Prometheus).

Qualifications :

  • Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.

  • 11- 16 years of total experience with at least 3- 4 years leading data engineering projects.

  • Proven track record in handling large-scale data pipelines and cloud migrations.

  • Azure Data Engineer

    5 days ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Lead Data Engineer

    2 days ago

    Hyderabad, Telangana, India Proclink Full time

    DescriptionProclink is looking for Data Engineer Lead.Experience :9+ years.Location :Hyderabad.Position OverviewWe are seeking an experienced Data Engineer with over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions.The ideal candidate will lead our data engineering strategy, architect cloud-based data platforms,...

  • Data Engineer Lead

    5 days ago

    Hyderabad, Telangana, India Proclink Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Proclinkis looking forData Engineer LeadExperience -9+ yearsLocation- HyderabadPosition OverviewWe are seeking an experiencedData Engineerwith over 10 years of experience in designing, developing, and optimizing enterprise-scale data solutions. The ideal candidate will lead our data engineering strategy, architect cloud-based data platforms, and mentor...

  • Lead Data Engineer

    1 day ago

    Hyderabad, Telangana, India Argano Full time

    JOB DATA JOB TITLE: Lead - Data Engineer Argano is the world's largest global digital consultancy, exclusively connecting design and delivery for the transformation of high-performance business operations, extending our clients' commercial agility, profitability, customer experience, and growth. Our strategic consulting, bolstered by proprietary IP,...

  • Lead Data Engineer

    1 week ago

    Hyderabad, Telangana, India Talent Worx Full time ₹ 14,15,000 - ₹ 21,05,000 per year

    one of the leading healthcare technology Product organizations with $1.4 Billion revenue with year-on-year growth of 22% and client retention rate of 120%. we offer wide range of innovative solutions to the healthcare payers, providers, and consumers. Our services include network analytics, payment integrity and optimization, provider credentialing, and...

  • Lead Data Engineer

    3 days ago

    Hyderabad, Telangana, India Saras Analytics Full time

    About Saras Analytics:We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value. Our suite of work spans extraction, transformation, visualization & analysis of data delivered via industry leading products, solutions & services. Our flagship product is Daton, an ETL...

  • Snowflake Data Lead

    5 days ago

    Hyderabad, Telangana, India Boolean Data Systems Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Tittle: Snowflake Data LeadJob Id: JPC - 67Location: HyderabadAbout The Company :Boolean Data Systems is a system integration company focusing on SAP, Snowflake and cloud infrastructure products.We focus on helping companies implement innovative solutions combining People+ Process+ Technology to reach their maximum ROI.We are headquartered in Atlanta, GA...

  • Data Scientist

    7 days ago

    Hyderabad, Telangana, India Hudson Data Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Why Hudson DataWork on real data problems across fintech, lending, and AI-ops platforms.Exposure to full-stack analytics: modeling, automation, and product integration.Flat structure, high ownership, and visible impact on business outcomes.Continuous learning and mentorship from experienced data and product leaders.Role & responsibilitiesHudson Data is a...

  • Azure .NET Technical Lead

    5 days ago

    Hyderabad, Telangana, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Senior Team...

  • Lead Data Engineer

    1 week ago

    Hyderabad, Telangana, India Salesforce Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts.Job CategorySoftware EngineeringJob DetailsAbout SalesforceSalesforce is the #1 AI CRM, where humans with agents drive customer success together. Here, ambition meets action. Tech meets trust. And innovation isn't...