2666 - Lead Data Engineer (Azure)

3 weeks ago

Pune, Maharashtra, India EXL Full time

Exp - 8-11 Years

Job Summary:

We are looking for an experienced and visionary Lead Data Engineer to architect and drive the development of scalable, secure, and high-performance data solutions . This role requires deep technical expertise in Python, Apache Spark, Delta Lake, and orchestration tools like Databricks Workflows or Azure Data Factory . The ideal candidate will also bring a strong understanding of data governance, metadata management, and regulatory compliance in the insurance and financial services domains. Proficient in developing Python Applications, Spark-based workflows, leveraging Delta Lake, and orchestrating jobs using Databricks Workflows or Azure Data Factory. Good to understand retention metadata, business rules, and data governance policies into reusable pipelines. Strong understanding of data privacy, security, and regulatory needs in insurance and financial domains.

Key Responsibilities:

  • Lead the design and architecture of end-to-end data engineering solutions across cloud platforms.
  • Develop and oversee robust data pipelines and ETL workflows using Python and Apache Spark.
  • Architect and implement scalable Delta Lake solutions for structured and semi-structured data.
  • Orchestrate complex workflows using Databricks Workflows or Azure Data Factory.
  • Translate business rules, retention metadata, and data governance policies into reusable, modular, and scalable pipeline components.
  • Ensure adherence to data privacy, security, and compliance standards (e.g., GDPR, HIPAA, etc.).
  • Mentor and guide junior data engineers, fostering best practices in coding, testing, and deployment.
  • Collaborate with cross-functional teams including data architects, analysts, and business stakeholders to align data solutions with business goals.
  • Drive performance optimization, cost-efficiency, and innovation in data engineering practices.

Required Skills & Qualifications:

  • 8+ years of experience in data engineering, with at least 2 years in a lead or architect role.
  • Expert-level proficiency in Python, Apache Spark, and Delta Lake.
  • Strong experience with Databricks Workflows and/or Azure Data Factory.
  • Deep understanding of data governance, metadata management, and business rule integration.
  • Proven track record in implementing data privacy, security, and regulatory compliance in insurance or financial domains.
  • Strong leadership, communication, and stakeholder management skills.
  • Experience with cloud platforms such as Azure, AWS, or GCP.

Preferred Qualifications:

  • Experience with CI/CD pipelines and DevOps practices in data engineering.
  • Familiarity with data cataloging and data quality tools.
  • Certifications in Azure Data Engineering or related technologies.
  • Exposure to enterprise data architecture and modern data stack tools.

  • Senior Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Manuh Technologies Full time

    Job Code: MGT-EXL-2666 Job Title: Lead Data Engineer Job Type: Permanent Technology: Data Engineering Experience: 8-12 Years Work Location: Pune/Bangalore/Noida/Gurugram/Hyderabad Requirements: 8+ years of experience in data engineering, with at least 2 years in a lead or architect role. Expert-level proficiency in Python, Apache Spark, and Delta...

  • Senior Data Engineer

    1 week ago

    Pune, Maharashtra, India Manuh Technologies Full time

    Job Code: MGT-EXL-2666 Job Title: Lead Data Engineer Job Type: Permanent Technology: Data Engineering Experience: 8-12 Years Work Location: Pune/Bangalore/Noida/Gurugram/Hyderabad Requirements: 8+ years of experience in data engineering, with at least 2 years in a lead or architect role. Expert-level proficiency in Python, Apache Spark, and Delta...

  • Lead Azure Data Engineer

    4 weeks ago

    Pune, Maharashtra, India ACL Digital Full time

    Lead Data Engineer7+ yrs expLocation: Balewadi, PuneImmediate joiners only.ResponsibilitiesMulti-year work experience as a Platform Lead, Delivery Leader or similar role.Good working knowledge of Microsoft Azure Cloud, Azure Data Engineering Services – ADF, Azure Synapse (Spark), SQL Database.Knowledge of columnar and NoSQL databases, predictive analytics,...

  • Azure Lead Data Specialist

    19 hours ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Lead Azure Data EngineerThis position requires a highly skilled data engineer to lead and manage our team's efforts in designing, developing, and optimizing data pipelines using Azure and Databricks.The ideal candidate will have hands-on experience with Databricks Asset Bundles (DABs) for CI/CD and environment management, as well as a strong understanding of...

  • Azure Data Engineer

    23 hours ago

    Pune, Maharashtra, India Coforge Full time

    We are Hiring For Azure Data Engineer at Coforge. Note:- We are looking for an immediate joiner. This is an urgent project, So we are looking for someone who can join immediately. We are seeking an experienced Data Engineer with expertise in Azure technologies to join our team. The ideal candidate will have a strong background in designing, developing, and...

  • Azure Data Engineer

    1 day ago

    Pune, Maharashtra, India Coforge Full time

    We are Hiring For Azure Data Engineer at Coforge.Note:- We are looking for an immediate joiner. This is an urgent project, So we are looking for someone who can join immediately.We are seeking an experienced Data Engineer with expertise in Azure technologies to join our team. The ideal candidate will have a strong background in designing, developing, and...

  • Azure Data Engineer

    3 days ago

    Pune, Maharashtra, India Lumiq Full time

    Location Pune 5 Days WFO Who we are LUMIQ is the leading Data and Analytics company in the Financial Services and Insurance FSI industry We are trusted by the world s largest FSIs including insurers banks AMCs and NBFCs to address their data challenges Our clients include 40 enterprises with over 10B in deposits AUM collectively representing...

  • Azure Data Engineer

    1 day ago

    Pune, Maharashtra, India Saxon AI Full time

    Role – Azure Data EngineerRelevant Experience – 6 to 8 YearsJob Description – We are seeking a highly skilled and experienced Azure Data Engineer with expertise in Azure Data Platforms to join our dynamic and innovative technology team. As a Data Engineer, you will play a pivotal role in designing, developing, and implementing robust and scalable...

  • Senior Azure Data Engineer

    1 day ago

    Pune, Maharashtra, India Fusemachines Full time

    About FusemachinesFusemachines is a leading AI strategy talent and education services provider Founded by Sameer Maskey Ph D Adjunct Associate Professor at Columbia University Fusemachines has a core mission of democratizing AI With a presence in 4 countries Nepal United States Canada and Dominican Republic and more than 400 full-time employees ...

  • Azure Data Engineer

    2 days ago

    Pune, Maharashtra, India Future Focus Infotech Full time

    Job DescriptionOverview:We are seeking a highly motivated and detail-oriented individual to join our team as a Azure Data Engineer. This role requires a dynamic professional who can adapt to evolving business needs and drive value through their expertise.Key Responsibilities:- Provide support and expertise in the domain of Azure Data Engineer.- Collaborate...