Senior Data Engineer

7 days ago

Pune, India Xlr8 Talent Search Full time

Job Description :


We are looking for a Senior Data Engineer with strong expertise in Azure Databricks, PySpark, and distributed computing to develop and optimize scalable ETL pipelines for manufacturing analytics. The role involves working with high-frequency industrial data to enable real-time and batch data processing.

KRA :

Build scalable real-time and batch processing workflows using Azure Databricks, PySpark, and Apache Spark.

- Perform data pre-processing, including cleaning, transformation, deduplication, normalization, encoding, and scaling to ensure high-quality input for downstream analytics.

- Design and maintain cloud-based data architectures, including data lakes, lakehouses, and warehouses, following Medallion Architecture.

- Deploy and optimize data solutions on Azure (preferred), AWS, or GCP with a focus on performance, security, and scalability.

- Develop and optimize ETL/ELT pipelines for structured and unstructured data from IoT, MES, SCADA, LIMS, and ERP systems.

- Automate data workflows using CI/CD and DevOps best practices, ensuring security and compliance with industry standards

- Monitor, troubleshoot, and enhance data pipelines for high availability and reliability.

- Utilize Docker and Kubernetes for scalable data processing.

- Collaborate with automation team, data scientists and engineers to provide clean, structured data for AI/ML models.

(ref:hirist.tech)

  • Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Data Axle Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    About Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging proprietary...

  • Data Engineer

    5 days ago

    Pune, India Data Axle Full time

    About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging...

  • Data Dynamics

    7 days ago

    Pune, India Data Dynamics Full time

    About the Role : We are seeking a Senior Software Engineer to join our team in Pune. As a key member of our engineering group, you will play a crucial role in the entire software development lifecycle, from initial design to hands-on development. You will work with cross-functional teams to create innovative solutions that are scalable and reliable. This...

  • Data Engineer

    5 days ago

    Pune, India Jash Data Sciences Full time

    Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team Then come join our high-energy team of passionate data people. Jash Data Sciences is the right place for you. We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India. We believe in continuous learning and...

  • Senior QE Engineer

    3 weeks ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionReq ID:330974NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior QE Engineer to join our team in Pune, Mahrshtra (IN-MH), India (IN).Senior Quality Engineer I...

  • Senior GenAI Engineers

    7 days ago

    Pune, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior GenAI Engineers to join our team in Pune, Mahrshtra (IN-MH), India (IN). Location: | Experience: 8+...

  • Data Engineer

    5 days ago

    Pune, Maharashtra, India NTT DATA Full time

    Req ID 324612NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Data Engineer to join our team in Pune Mahxc4x81rxc4x81shtra IN-MH India IN Key Responsibilities Design and...

  • Senior Data Engineer

    2 weeks ago

    Pune, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 20,10,000

    Job Title: Senior Data Engineer">Job Description:As a senior data engineer, you will be responsible for designing and building robust data pipelines that process and transform data from diverse sources.Key Responsibilities:Design and build efficient data pipelines that ensure data quality, reliability, and availability across the pipeline...

  • Senior Data Engineer

    2 weeks ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionAbout the Role:We are seeking an experienced Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, implementing and maintaining large-scale data warehouses that support our analytics and intelligence platform.Design and develop advanced SQL queries, stored...

  • Data Scientist

    1 week ago

    Pune, Maharashtra, India Data Dynamics Full time

    **Objectives of this role**: - Collaborate with product design and engineering teams to develop an understanding of needs. - Research and devise innovative statistical models for data analysis. - Build innovative models and integrate them in our product. - Communicate findings to all stakeholders. - Keep current with technical and industry...