Senior Data Engineer

1 week ago

Pune, Maharashtra, India USIL Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Job Title: Lead/Senior Data Engineer

Location: Pune

Job Type: Full-Time

Job Description:

We are looking for a Senior Data Engineer with strong expertise in Azure Databricks, PySpark, and distributed computing to develop and optimize scalable ETL pipelines for manufacturing analytics. The role involves working with high-frequency industrial data to enable real-time and batch data processing.

Key Responsibilities

· Build scalable real-time and batch processing workflows using Azure Databricks, PySpark, and Apache Spark.

· Perform data pre-processing, including cleaning, transformation, deduplication, normalization, encoding, and scaling to ensure high-quality input for downstream analytics.

· Design and maintain cloud-based data architectures, including data lakes, lakehouses, and warehouses, following Medallion Architecture.

· Deploy and optimize data solutions on Azure (preferred), AWS, or GCP with a focus on performance, security, and scalability.

· Develop and optimize ETL/ELT pipelines for structured and unstructured data from IoT, MES, SCADA, LIMS, and ERP systems. · Automate data workflows using CI/CD and DevOps best practices, ensuring security and compliance with industry standards

· Monitor, troubleshoot, and enhance data pipelines for high availability and reliability.

· Utilize Docker and Kubernetes for scalable data processing.

· Collaborate with automation team, data scientists and engineers to provide clean, structured data for AI/ML models.

Desired Skills and Qualifications ·

Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

· 7+ years of experience in core data engineering, with a strong focus on cloud platforms such as Azure (preferred), AWS, or GCP · Proficiency in PySpark, Azure Databricks, Python and Apache Spark, etc.

. 2 years of team handling experience.

· Expertise in relational databases (e.g., SQL Server, PostgreSQL), time series databases (e.g. Influx DB), and NoSQL databases (e.g., MongoDB, Cassandra) · Experience in containerization (Docker, Kubernetes).

· Strong analytical and problem-solving skills with attention to detail.

· Good to have MLOps, DevOps including model lifecycle management

· Excellent communication and collaboration skills, with a proven ability to work effectively as a team player.

· Comfortable working in a dynamic, fast-paced startup environment, adapting quickly to changing priorities and responsibilities.

Interested candidate can send resume

  • Data Engineer

    4 days ago

    Pune, Maharashtra, India Data Axle Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    About Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging proprietary...

  • Senior QE Engineer

    1 week ago

    Pune, Maharashtra, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Req ID: 330974NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior QE Engineer to join our team in Pune, Mahārāshtra (IN-MH), India (IN). Senior Quality Engineer I (JavaScript)...

  • Data Engineer

    7 days ago

    Pune, Maharashtra, India Jash Data Sciences Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team Then come join our high-energy team of passionate data people. Jash Data Sciences is the right place for you.We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India.We believe in continuous learning and...

  • Senior Software Engineer

    1 week ago

    Pune, Maharashtra, India Data Dynamics Full time US$ 1,20,000 - US$ 2,00,000 per year

    About The RoleWe are seeking a Senior Software Engineer to join our team in Pune. As a key member of our engineering group, you will play a crucial role in the entire software development lifecycle, from initial design to hands-on development. You will work with cross-functional teams to create innovative solutions that are scalable and reliable. This...

  • Senior Data Engineer

    7 days ago

    Pune, Maharashtra, India NCSI Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: Senior Data Engineer Company Name: NCSi Job Description: NCSi is seeking a highly skilled Senior Data Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and optimizing data pipelines and architectures to support our data-driven initiatives. You will collaborate with cross-functional teams to...

  • Senior Data Engineer

    2 weeks ago

    Pune, Maharashtra, India BlueCloud Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    ResponsibilitiesWe are looking for a Senior Data Engineer to join their team and contribute to the design and implementation of data engineering solutions. This role requires strong hands-on experience withDBT and Snowflake. The Senior Data Engineer will be responsible for building, maintaining, and optimizing data pipelines, as well as implementing...

  • Senior Data Engineer

    7 days ago

    Pune, Maharashtra, India Vanderlande Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job TitleSenior Data EngineerJob DescriptionIf you are passionate about unlocking the power to gain next level insights into a great variety of data, powering data driven decision making through advanced analytics and data science practises, you might have just found your perfect next career step.In this job you will have versatile responsibilities,...

  • Senior Data Engineer

    7 days ago

    Pune, Maharashtra, India True Talents Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Title :Senior Data EngineerLocation :Pune / NoidaExperience :4 - 8 YearsEmployment Type :Full-timeNotice Period :Up to 30 DaysAbout The RoleWe are looking for an experienced Senior Data Engineer to design, build, and optimize scalable data pipelines and data platforms. The ideal candidate should have strong expertise in modern data engineering tools,...

  • Senior Data Engineer

    7 days ago

    Pune, Maharashtra, India NCS Group Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: Senior Data EngineerCompany Name: NCSiJob Description:NCSi is seeking a highly skilled Senior Data Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and optimizing data pipelines and architectures to support our data-driven initiatives. You will collaborate with cross-functional teams to understand...

  • Senior Data Engineer

    4 weeks ago

    Pune, Maharashtra, India SpeedMart Full time

    Company Profile :Our client is a global IT services company that helps businesses with digital transformation with offices in India and the United States.It helps businesses with digital transformation, provide IT collaborations and uses technology, innovation, and enterprise to have a positive impact on the world of business.With expertise is in the fields...