Lead Data Engineer

1 week ago

IND Bangalore MSR Vaishnavi No Union Street Off Cubbon Road India LabCorp Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Lead Data Engineer

Job Description:

We are looking for a Lead Data Engineer with strong Databricks expertise who can also design and build accelerators for automation. The role requires hands-on development in Databricks (PySpark, Spark SQL, Delta Lake,Autoloader, DLT etc) along with experience in creating frameworks for test data generation, report validation, and backend validation. You will be leading technical work, mentoring team members, and building reusable solutions that speed up project delivery.

Key Responsibilities:

·Design, develop, and optimize ETL/ELT pipelines on Databricks using PySpark, Spark SQL,Delta Lake, DLT live.·Build automation accelerators such as:Synthetic test data generators for large-scale data sets.Automated report validation tools to compare dashboards with backend data.Backend validation frameworks for reconciliation and data quality checks.·Implement data validation frameworks (Great Expectations, Deequ, or custom solutions).·Ensure data quality, lineage, and governance using Unity Catalog.·Collaborate with QA, BI, and business teams to integrate accelerators into workflows.·Drive best practices in coding, performance tuning, and cost optimization on Databricks.·Lead and mentor data engineers, review code, and set technical standards.

Must-Have Skills


•5-7 years of experience in data engineering, with at least 3-5 years hands-on experience in databricks

·Databricks: PySpark, Spark SQL, Delta Lake, Delta Live Tables (DLT).·Automation & Accelerators: Experience creating test data generators, validation frameworks, report reconciliation tools.·Programming language: Strong in Python,PySpark and SQL.·Validation Tools: open source tools like Great Expectations, Deequ, or equivalent custom frameworks.·Data Pipelines: Batch and streaming pipeline design.·Data Governance: Unity Catalog for access, lineage, and compliance.·DevOps/CI-CD: Git, Azure DevOps, Jenkins, or GitHub Actions for deployment automation.·Cloud & Storage: Azure (preferred) / AWS / GCP with hands-on in ADLS/S3/GCS.

Good-to-Have Skills

·Streaming: Kafka, EventHub, or Kinesis.·Infra-as-Code: Terraform for Databricks and cloud provisioning.·Reporting Tools: Power BI, Tableau, or Looker (for validation accelerators).·Testing: Exposure to pytest or unittest for automation.

Qualifications:


•Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

Labcorp is proud to be an Equal Opportunity Employer:

Labcorp strives for inclusion and belonging in the workforce and does not tolerate harassment or discrimination of any kind. We make employment decisions based on the needs of our business and the qualifications and merit of the individual. Qualified applicants will receive consideration for employment without regard to race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), family or parental status, marital, civil union or domestic partnership status, sexual orientation, gender identity, gender expression, personal appearance, age, veteran status, disability, genetic information, or any other legally protected characteristic. Additionally, all qualified applicants with arrest or conviction records will be considered for employment in accordance with applicable law.

We encourage all to apply

If you are an individual with a disability who needs assistance using our online tools to search and apply for jobs, or needs an accommodation, please visit our accessibility site or contact us at Labcorp Accessibility. For more information about how we collect and store your personal data, please see our Privacy Statement.

Experience LevelSenior Level

  • Data Associate Engineer

    1 week ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    DA Associate Engineer • Proficiency in programming languages such as Java, Python, C++, or JavaScript. • Understanding of software development methodologies (e.g., Agile, Scrum). • Knowledge of cloud computing platforms such as AWS, Azure. • Familiarity with data integration tools, ETL processes, and data pipeline architectures. •...

  • Lead Back-end Engineer

    1 week ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Lead Back-end Engineer - Java Corporate IT is actively searching for an experienced, expert-level Senior Java Engineer who will leverage their technology skills to develop business solutions within the healthcare industry. The engineer will contribute to creating and implementing enterprise standards for a web service platform (RESTful APIs). The applicant...

  • Assistant Data Programmer

    12 hours ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 2,50,000 - ₹ 7,50,000 per year

    Asst Data ProgrammerSupport data management during start up. Act as a data liaison between Clinical Data Management and all other LabCorp departments and business units. Manage moderate client queries. Provide general support to lead Data Managers Create standard data transfer format(s) in compliance with client requests while adhering to both...

  • Assistant Data Programmer

    4 days ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 2,00,000 - ₹ 4,00,000 per year

    Asst Data ProgrammerMust be able to communicate effectively in English with end users, technical teams, and stakeholders. This includes written and verbal communication, especially when explaining technical issues or gathering information. Skilled at tracing data entries and system actions to specific users or departments using available tools and logs....

  • Data Enablement Integration Analyst

    6 days ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Data Enablement Integration Analyst I Duties and Responsibilities: Participate in the administration, troubleshooting, and maintenance of Intelligent Cloud Services environments to ensure high availability and performance. Participate in the performance of regular upgrades, security patches, and system health checks. Participate in...

  • DevOps Engineer

    6 days ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Sr. DevOps Engineer Job Description: The Individual in this role is responsible for implementing and growing our Infrastructure as Code practices across the enterprise. They should be able to grow our IaC library to meet the needs of any teams that need to add new infrastructure or to add new functionality to existing infrastructure. They should...

  • Front-End Engineer

    22 hours ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Lead Front-End Engineer Corporate IT is actively searching for experienced Senior UI/Web Application Engineers who will leverage their technology skills to develop business solutions within the healthcare industry. You will be challenged with cutting-edge technologies to provide a modern framework for container, cloud, and serverless Angular-based single...

  • Azure DevOps Engineer

    6 days ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Sr. Azure DevOps Engineer Job Description: The Individual in this role is responsible for implementing and growing our Infrastructure as Code practices across the enterprise. They should be able to grow our IaC library to meet the needs of any teams that need to add new infrastructure or to add new functionality to existing infrastructure. They should...

  • Senior Software Engineer

    12 hours ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Sr. Software EngineerDesign, develop, and maintain high-quality Java-based applications. Contribute to the development of a LIMS systems. Ensure the scalability, security, and performance of applications deployed on AWS. Collaborate with product managers, domain experts, and team members to understand and refine requirements. Integrate healthcare...

  • Senior Marketing and Customer Analytics Director

    1 week ago

    IND - Bangalore - MSR Vaishnavi, No Union Street - Off Cubbon Road, India LabCorp Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Senior Director – Marketing and Customer Analytics Overview: Labcorp, a Forbes 2025 "Best Large Employer," is looking for a visionary data leader to join our Global Insights team in Bangalore. As Senior Director of Marketing and Customer Analytics, you'll implement the strategy for how Labcorp leverages data to drive marketing effectiveness, customer...