IT engineer data lakehouse

2 weeks ago

Bengaluru, Karnataka, India Continental AG Full time ₹ 9,00,000 - ₹ 12,00,000 per year

- Design, develop, and operate scalable and maintainable data pipelines in the Azure Databricks environment
- Develop all technical artefacts as code, implemented in professional IDEs, with full version control and CI/CD automation
- Enable data-driven decision-making in Human Resources (HR), Purchasing (PUR) and Finance (FIN) by ensuring high data availability, quality, and reliability
- Implement data products and analytical assets using software engineering principles in close alignment with business domains and functional IT
- Apply rigorous software engineering practices such as modular design, test-driven development, and artifact reuse in all implementations

- Global delivery footprint; cross-functional data engineering support across HR, PUR & FIN domains
- Collaboration with business stakeholders, functional IT partners, product owners, architects, ML/AI engineers, and Power BI developers
- Agile, product-team structure embedded in an enterprise-scale Azure environment

Main Tasks:


• Design scalable batch and streaming pipelines in Azure Databricks using PySpark and/or Scala

• Implement ingestion from structured and semi-structured sources (e.g., SAP, APIs, flat files)

• Build bronze/silver/gold data layers following the defined lakehouse layering architecture & governance


• Implement use-case driven dimensional models (star/snowflake schema) tailored to HR, PUR & FIN needs

• Ensure compatibility with reporting tools (e.g., Power BI) via curated data marts and semantic models


• Implement enterprise-level data warehouse models (domain-driven 3NF models) for HR, PUR & FIN data, closely aligned with data engineers for other business domains

• Develop and apply master data management strategies (e.g., Slowly Changing Dimensions)


• Develop automated data validation tests using frameworks

• Monitor pipeline health, identify anomalies, and implement quality thresholds

• Establish data quality transparency by defining and implementing meaningful data quality rules with source system and business stakeholders and implementing related reports


• Develop and structure pipelines using modular, reusable code in a professional IDE

• Apply test-driven development (TDD) principles with automated unit, integration, and validation tests

• Integrate tests into CI/CD pipelines to enable fail-fast deployment strategies

• Commit all artifacts to version control with peer review and CI/CD integration

• Work closely with Product Owners to refine user stories and define acceptance criteria

• Translate business requirements into data contracts and technical specifications

• Participate in agile events such as sprint planning, reviews, and retrospectives


• Document pipeline logic, data contracts, and technical decisions in markdown or auto-generated docs from code

• Align designs with governance and metadata standards (e.g., Unity Catalog)

• Track lineage and audit trails through integrated tooling


• Profile and tune data transformation performance

• Reduce job execution times and optimize cluster resource usage

• Refactor legacy pipelines or inefficient transformations to improve scalability

Read more Read less

  • Senior Data Solutions Architect-Data Lakehouse

    2 weeks ago

    Bengaluru, Karnataka, India Analyttica Datalab Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    About the RoleWe're seeking aSenior Data Solutions Architecttodesign and implemententerprise-gradedata lakehouse architecturesto power analytics, reporting, and AI use cases. You'll work closely with business and technical teams to translate needs into scalable, high-performance, and secure data solutions that deliver measurable value.Key...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India AKSHAYA BUSINESS IT SOLUTIONS PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position : Data Engineer - AzureExperience : 4 to 8 YearsLocation : BangaloreJob Type : Full-timeJob Summary : We are seeking a highly skilled Data Engineer with 4 to 8 years of experience, specializing in the Microsoft Azure ecosystem. The ideal candidate will have a minimum of 3 years of core experience in MS Azure and Databricks. This role...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India ALIQAN Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Greetings from ALIQAN TechnologiesHiring Data Engineer (Microsoft Fabric & Lakehouse) for one of our client MNCJob Title: Data Engineer (Microsoft Fabric & Lakehouse)Location: Hybrid Bangalore, IndiaExperience: 5 YearsJoining: ImmediateHiring Process: One interview + One case study roundAbout the Role-We are looking for a skilled Data Engineer with 25...

  • Senior Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Sigmoid Full time

    We are looking for a skilled Data Engineer with 8+ years of experience in big data technologies, particularly Python, PYSpark, SQL and data lakehouse architectures. The ideal candidate will have a strong background in building scalable data pipelines and experience with modern data storage formats, including Apache Iceberg. You will work closely with...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Ethiraj Associates Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Responsibilities:Design and implement scalable data pipelines using Microsoft Fabric, including Dataflows Gen2, Lakehouse, Notebooks and SQL endpoints.Develop ETL/ELT solutions using PySpark, T-SQL and Spark Notebooks within Fabric and Azure Synapse.Manage and optimize data storage and compute in OneLake supporting Lakehouse and Warehouse use cases.Implement...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Sprouts Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Company Description is an innovative martech platform designed to streamline the B2B buying and deploying process by integrating multiple GTM solutions into one. Our platform enhances demand generation, drives incremental sales, and significantly improves marketing efficiency. We guarantee impressive metrics such as a 200% increase in MQLs and SQLs, a 3X...

  • AWS Data engineer/Architect

    4 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Job Summary: In this key leadership role, you will lead the development of foundational components for a Lakehouse architecture on AWS and drive the migration of existing data processing workflows to the new Lakehouse solution. You will work across the Data Engineering organisation to design and implement scalable data infrastructure and processes using...

  • Lead Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Aptean Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    India5836Employee Regular Full TimeJob DescriptionOverviewJob Title - Lead Data Engineer(MS Fabrics)Location - BengaluruAptean is changing. Our bespoke ERP solutions are transforming a huge range of global businesses, from food producers to manufacturers. In a world of generic enterprise software, we provide targeted solutions that bring together the very...

  • Lead Data Engineer

    3 days ago

    Bengaluru, Karnataka, India Aptean Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    OverviewJob Title - Lead Data Engineer(MS Fabrics)Location - BengaluruAptean is changing. Our bespoke ERP solutions are transforming a huge range of global businesses, from food producers to manufacturers. In a world of generic enterprise software, we provide targeted solutions that bring together the very best technology and drive greater results. With over...

  • Senior Data Engineer

    1 week ago

    Bengaluru, Karnataka, India USEReady Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title: Senior Databricks EngineerExperience Level: 5-8 YearsJob SummaryAs a Senior Databricks Engineer, you will be responsible for designing, developing, and optimizing our data architecture and pipelines on the Databricks Lakehouse Platform. You will leverage your deep expertise in Spark, Delta Lake, and cloud technologies to build scalable and...