Data Engineer

4 weeks ago

Pune, Maharashtra, India Optimum Solutions Pte Ltd Full time

Job Summary:

We are seeking a skilled and experienced Azure Databricks Engineer to join our growing data engineering team. The ideal candidate will have deep hands-on expertise in building scalable data pipelines and streaming architectures using Azure-native technologies. Prior experience in the banking or financial services domain is highly desirable, as you will be working with critical data assets and supporting regulatory, risk, and operational reporting use cases.

Key Responsibilities:

  • Design, develop, and optimize data pipelines using Databricks (PySpark) for batch and real-time data processing.
  • Implement CDC (Change Data Capture) and Delta Live Tables/Autoloader to support near-real-time ingestion.
  • Integrate various structured and semi-structured data sources using ADF, ADLS, and Kafka (Confluent).
  • Develop CI/CD pipelines for data engineering workflows using GitHub Actions or Azure DevOps.
  • Write efficient and reusable SQL and Python code for data transformations and validations.
  • Ensure data quality, lineage, governance, and security across all ingestion and transformation layers.
  • Collaborate closely with business analysts, data scientists, and data stewards to support use cases in risk, finance, compliance, and operations.
  • Participate in code reviews, architectural discussions, and documentation efforts.

Required Skills & Qualifications:

  • Strong proficiency in SQL, Python, and PySpark.
  • Proven experience with Azure Databricks, including notebooks, jobs, clusters, and Delta Lake.
  • Experience with Azure Data Lake Storage (ADLS Gen2) and Azure Data Factory (ADF).
  • Hands-on with Confluent Kafka for streaming data integration.
  • Strong understanding of Autoloader, CDC mechanisms, and Delta Lake-based architecture.
  • Experience implementing CI/CD pipelines using GitHub and/or Azure DevOps.
  • Knowledge of data modeling, data warehousing, and data security best practices.
  • Exposure to regulatory and risk data use cases in the banking/financial sector is a strong plus.

Preferred Qualifications:

  • Azure certifications (e.g., Azure Data Engineer Associate).
  • Experience with tools such as Delta Live Tables, Unity Catalog, and Lakehouse architecture.
  • Familiarity with business glossaries, data lineage tools, and data governance frameworks.
  • Understanding of financial data including GL, loan, customer, transaction, or market risk domains.

  • Senior QE Engineer

    3 weeks ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:330977NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior QE Engineer to join our team in Pune, Mahrshtra (IN-MH), India (IN).Senior Quality Engineer I...

  • Senior QE Engineer

    2 days ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:330974NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior QE Engineer to join our team in Pune, Mahrshtra (IN-MH), India (IN).Senior Quality Engineer I...

  • Senior Software Engineer

    3 weeks ago

    Pune, Maharashtra, India Data Dynamics Full time

    About the Role : We are seeking a Senior Software Engineer to join our team in Pune. As a key member of our engineering group, you will play a crucial role in the entire software development lifecycle, from initial design to hands-on development. You will work with cross-functional teams to create innovative solutions that are scalable and reliable. This...

  • Data Engineer

    3 weeks ago

    Pune, Maharashtra, India Whitehorse Manpower Full time

    Responsibilities: Design, develop, and deploy data solutions on Azure, leveraging SQL Azure, Azure Data Factory, and Databricks. Build and maintain scalable data pipelines to ingest, transform, and load data from various sources into Azure data repositories. Implement data security and compliance measures to safeguard sensitive information....

  • Mid .Net Engineer

    2 days ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:331146NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Mid .Net Engineer to join our team in Pune, Mahrshtra (IN-MH), India (IN).Who we are:NTT DATA America...

  • Data Engineer

    13 hours ago

    Pune, Maharashtra, India aQb Solutions Pvt Ltd Full time

    We are looking for a highly skilled Data Engineer with strong expertise in PostgreSQL and hands-on experience in building data warehouses and data lakes. In this role, you will design and implement a scalable PostgreSQL-based data warehouse and manage a data lake on AWS infrastructure, leveraging primarily open-source technologies.The ideal candidate has a...

  • Data Engineer

    2 weeks ago

    Pune, Maharashtra, India People Prime Worldwide Full time

    About Client: Our Client is a global IT services company headquartered in Southborough, Massachusetts, USA. Founded in 1996, with a revenue of $1.8B, with 35,000+ associates worldwide, specializes in digital engineering, and IT services company helping clients modernize their technology infrastructure, adopt cloud and AI solutions, and accelerate...

  • Data Engineer

    2 weeks ago

    Pune, Maharashtra, India People Prime Worldwide Full time

    About Client: Our Client is a global IT services company headquartered in Southborough, Massachusetts, USA. Founded in 1996, with a revenue of $1.8B, with 35,000+ associates worldwide, specializes in digital engineering, and IT services company helping clients modernize their technology infrastructure, adopt cloud and AI solutions, and accelerate...

  • Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Mount Talent Consulting Pvt Ltd. Full time

    We are hiring a Data Engineer for Pune/Hyderabad/Bangalore. Experience: 6+ Years Designation: Senior Software Engineer/Lead Software Engineer –Data Engineer Skill Tech stack: AWS Data Engineer, Python, PySpark, SQL, Data Pipeline, AWS, AWS Glue, Lambda JD: 6+ years of experience in data engineering, specifically in cloud environments like AWS....

  • Data Engineer

    3 weeks ago

    Pune, Maharashtra, India Mount Talent Consulting Pvt Ltd. Full time

    We are hiring a Data Engineer for Pune/Hyderabad/Bangalore.Experience: 6+ YearsDesignation: Senior Software Engineer/Lead Software Engineer –Data EngineerSkill Tech stack: AWS Data Engineer, Python, PySpark, SQL, Data Pipeline, AWS, AWS Glue, LambdaJD:6+ years of experience in data engineering, specifically in cloud environments like AWS.Proficiency in...