Technical Lead: Open Source Data Engineering Platform

6 days ago

Pune, Maharashtra, India BAJAJ FINSERVE Full time ₹ 5,00,000 - ₹ 8,00,000 per year

Role & responsibilities

We are looking for an accomplished Technical Lead - Data Engineering to design and develop of a cloud-agnostic, open-source data ingestion and processing solution that will be embedded within a larger Low-Code platform to strengthen the overall low code platform capabilities.

The ideal candidate will bring deep expertise in large-scale Data Engineering platforms design, development and execution experience, with hands-on experience in ETL, Real Time Streaming, Distributed Data Processing, Python, Spark, and will have the ability to guide a product development team through the entire lifecycle - from concept to deployment.

This role involves architecting and delivering a robust, scalable, and future-ready data ingestion and processing framework, leveraging strong technical proficiency in Open Source Softwares along with modern data engineering best practices.

Preferred candidate profile

Experience Required:

  • Overall 8+ Years of experience in Data Engineering domain
  • Ability to lead the team to develop a cloud-agnostic, open-source data engineering platform
  • Hands on Architecture, Design and Programming skills to develop an Open Source based Data Engineering Application and Framework within a larger Enterprise Application product suite.
  • Strong Hands On experience in Python, Spark, API and preferably with other open source product like Apache Nifi etc.
  • Hands on experience in Java is preferrable
  • Strong experience in ETL/ELT pipelines, data streaming, CDC etc.
  • Design and implement real-time data streaming solutions as a part of product feature
  • Design and develop robust data quality assurance, failure logging features
  • Design and develop Monitoring, diagnostic and logging features to maintain production and non-production systems proactively
  • Develop automation scripts/tools (Python, Perl, Shell) to eliminate manual tasks and accelerate issue resolution
  • Engage with stakeholders to continuously enhance the product
  • Capture user feedback and translate it into actionable architecture or support initiatives.
  • Maintain structured documentation, including performance metrics and enhancement logs.

  • Data Platform Engineer

    1 week ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 5,00,000 - ₹ 8,00,000

    About the RoleThis is an opportunity to work with a leading company in the field of data engineering. We are seeking a talented professional to support, develop and maintain our data and analytics platform.Key ResponsibilitiesThe successful candidate will be responsible for implementing and automating the deployment of our distributed system for ingesting...

  • Open Source Advocate

    7 days ago

    Pune, Maharashtra, India beBeeDeveloper Full time US$ 90,000 - US$ 1,20,000

    As a pioneering figure in the realm of Developer Relations, you will play a pivotal role in bridging the gap between our organization and the open-source community.We are seeking an accomplished Developer Relations Engineer to spearhead our efforts in engaging directly with open-source communities and developer-centric organizations. This individual will be...

  • Data Platform Lead

    4 weeks ago

    Pune, Maharashtra, India Kalyani Technologies Full time

    About Us We are building a state-of-the-art, on-premise data platform from the ground up to revolutionize how our organization leverages data. Inspired by a unified and modern approach, our mission is to create a single, cohesive ecosystem that eliminates silos between data engineering, business analytics, and machine learning. We are looking for a key...

  • Data Platform Lead

    2 weeks ago

    Pune, Maharashtra, India Kalyani Technologies Full time

    About UsWe are building a state-of-the-art, on-premise data platform from the ground up to revolutionize how our organization leverages data. Inspired by a unified and modern approach, our mission is to create a single, cohesive ecosystem that eliminates silos between data engineering, business analytics, and machine learning. We are looking for a key...

  • Data Platform Engineer

    2 weeks ago

    Pune, Maharashtra, India Deutsche Bank Full time US$ 1,50,000 - US$ 2,00,000 per year

    :  Job Title - Data Platform Engineer - Tech Lead Location - Pune, India Role Description DB Technology is a global team of tech specialists, spread across multiple trading hubs and tech centers. We have a strong focus on promoting technical excellence our engineers work at the forefront of financial services innovation using cutting-edge...

  • Technical Lead

    2 weeks ago

    Pune, Maharashtra, India NCS Group Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    1 Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks2 Ensure data pipelines run smoothly and efficiently3 Developing scalable and re-usable frameworks for ingesting of data sets4 Integrating the end-to-end data pipeline - to take data from source systems to target data repositories ensuring the quality and...

  • Data Engineer

    2 weeks ago

    Pune, Maharashtra, India aQb Solutions Pvt Ltd Full time

    We are looking for a highly skilled Data Engineer with strong expertise in PostgreSQL and hands-on experience in building data warehouses and data lakes. In this role, you will design and implement a scalable PostgreSQL-based data warehouse and manage a data lake on AWS infrastructure, leveraging primarily open-source technologies.The ideal candidate has a...

  • Senior Open Source Cloud Migration Specialist

    2 weeks ago

    Pune, Maharashtra, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    As a senior cloud migration expert, you will lead the technical onboarding process for customers using managed open source and managed kubernetes platforms.About the RoleYou will work closely with clients to understand their business needs, architect tailored solutions, and ensure seamless integration and deployment of cloud-native technologies.Key...

  • Technical Lead – Data Engineering

    3 weeks ago

    Pune, Maharashtra, India Zywave Full time

    Job Description – Technical Lead (Data Engineering, TurboRater RQR: CPQ Rating)Location: Pune, IndiaWork Mode: 5 Days Work from OfficeAbout the RoleZywave is seeking a Technical Lead – Data Engineering (TurboRater RQR: CPQ Rating) with expertise in Snowflake, SQL Server, and modern data architecture principles including Medallion Architecture and Data...

  • Technical Lead – Data Engineering

    2 weeks ago

    Pune, Maharashtra, India Zywave Full time

    Job Description – Technical Lead (Data Engineering, TurboRater RQR: CPQ Rating) Location: Pune, India Work Mode: 5 Days Work from Office About the Role Zywave is seeking a Technical Lead – Data Engineering (TurboRater RQR: CPQ Rating) with expertise in Snowflake, SQL Server, and modern data architecture principles including Medallion Architecture and...