Data Platform Engineer

5 days ago

Mumbai, Maharashtra, India TIH | IIT Bombay Full time
Job Summary:
BharatGen is on a mission to create AI that truly represents the diversity, culture, and unique context of India. At the heart of this mission lies the need for robust, scalable infrastructure to build multilingual and multimodal datasets that power foundational AI models. We're seeking a skilled Data Platform Engineer to build scalable tools, platforms, and pipelines tailored for processing large-scale, multilingual, multimodal datasets critical for foundational AI models.
In this role, you will build scalable data pipelines to ingest, transform, and prepare data from diverse sources—text, speech, images, and video—making it ready for Generative AI model training. Your work will involve developing and managing the underlying platform while addressing challenges like governance, security, observability, lineage, and scalability. The outcomes of your work will include efficient tools for data processing, a reliable data platform, and high-quality datasets tailored to the evolving needs of large-scale AI and LLM training.
Collaborating closely with researchers and ML engineers, you will play a pivotal role in enabling BharatGen to deliver state-of-the-art AI models, contributing to the advancement of India's AI ecosystem through innovative data engineering solutions.

Key Responsibilities:
Design and Build Scalable Platforms: Develop distributed infrastructure for ingesting, processing, and transforming diverse datasets (text, speech, images, video) at terabyte to petabyte scale.
Develop Robust Data Pipelines: Create reliable, scalable pipelines to prepare datasets for Generative AI and LLM training.
Implement Governance and Observability: Build frameworks for data lineage, monitoring, and access control to ensure data quality and operational reliability.
Optimize Performance and Cost: Enhance platform performance and resource utilization using cost-effective strategies, including GPU-accelerated preprocessing.
Collaborate and Innovate: Work closely with researchers and ML engineers to adapt platforms and data pipelines to evolving LLM requirements, addressing various data challenges.
Drive Innovation: Stay updated on emerging tools, frameworks, and best practices to implement cutting-edge solutions for large-scale dataset creation.

Minimum Qualifications and Experience:
Education:
Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
[Preferred] Advanced degrees or certifications in Distributed Systems, Data Engineering, or Big Data technologies

Experience and Expertise:
3+ years of overall industry experience in engineering roles, demonstrating strong foundations in software development, systems engineering, or related disciplines.
1+ years of specific hands-on experience in developing large-scale, distributed data pipelines and platforms, preferably in high-performance AI or ML environments.
Expertise in managing unstructured data (text, speech, or multimodal datasets) for high-performance use cases, ideally in the context of LLM/AI datasets.
Understanding of challenges in scalable data engineering, including ingestion, transformation, and storage optimization for large-scale accelerated workflows.

Skills:
1.Technical
Proficiency in distributed systems and frameworks (e.g., Kafka, Ray, PySpark) for scalable data workflows.
Exposure to end-to-end data lifecycle management, including DataOps.
Strong programming skills in Python, Scala, or Go, with a focus on high-performance pipeline development.
Experience with building and optimizing data pipelines, including ETL processes, data modeling, and integration into scalable workflows.
Expertise in data scraping, crawling frameworks, and modern dataset development techniques such as synthetic data generation techniques.
Experience with cloud platforms (AWS, GCP, Azure) and container orchestration (Docker, Kubernetes).
Deep understanding of data platform design, including data architecture, metadata tracking, data lineage, observability, monitoring, and scalability best practices.
Familiarity with Infrastructure-as-Code tools (e.g., Terraform, CloudFormation), CI/CD pipelines, relational/NoSQL databases, and GPU-accelerated workflows.
Familiarity with visualization and monitoring tools for lifecycle management and pipeline performance tracking.

2.Soft Skills
Adaptability and innovation in fast-paced, dynamic environments.
Strong collaboration skills for interdisciplinary teamwork.
Proactive problem-solving and a growth mindset to thrive in a mission-driven organization.

  • Platform Support Engineer

    3 weeks ago

    Mumbai, Maharashtra, India NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT...

  • Data Platform Engineer

    2 days ago

    Mumbai, Maharashtra, India Apollo Global Management, Inc. Full time

    Job Overview:Apollo Global Management, Inc. is seeking a highly skilled Enterprise Data Platform Engineer to design and build foundational data products for enterprise consumption. As a key member of our team, you will work closely with stakeholders to define product roadmaps and bring them to fruition.Key Responsibilities:- Build, own, and scale data...

  • Data Platform Architect

    4 days ago

    Mumbai, Maharashtra, India TIH | IIT Bombay Full time

    Job SummaryBharatGen is on a mission to create AI that truly represents the diversity, culture, and unique context of India. At the heart of this mission lies the need for robust, scalable infrastructure to build multilingual and multimodal datasets that power foundational AI models. We're seeking a skilled Data Platform Engineer to build scalable tools,...

  • 15h Left Cloud Data Platform Engineer

    3 weeks ago

    Mumbai, Maharashtra, India Capgemini Full time

    Job DescriptionChoosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of...

  • L1 Data Platform Engineer

    3 weeks ago

    Mumbai, Maharashtra, India HARP Technologies and Services Full time

    Job Title : Data Platform Engineer L1Exp : 2 to 5 yearsLocation : Mumbai/ChennaiKey Skills : GCP, SQL, JIRA, Python, SupportJob Description :Responsibilities :1. Production Monitoring :- Monitor and ensure the smooth execution of production data pipelines and workflows.- Identify and promptly address anomalies or failures in the production environment.-...

  • L1 Data Platform Engineer

    2 weeks ago

    Mumbai, Maharashtra, India HARP Technologies and Services Full time

    Job Title : Data Platform Engineer L1Exp : 2 to 5 yearsLocation : Mumbai/ChennaiKey Skills : GCP, SQL, JIRA, Python, SupportJob Description :Responsibilities :1. Production Monitoring :- Monitor and ensure the smooth execution of production data pipelines and workflows.- Identify and promptly address anomalies or failures in the production environment.-...

  • Platform Engineer

    3 weeks ago

    Mumbai, Maharashtra, India Nielsen Full time

    At Nielsen, we are passionate about our work to power a better media future for all people by providing powerful insights that drive client decisions and deliver extraordinary results. Our talented, global workforce is dedicated to capturing audience engagement with content - wherever and whenever it's consumed. Together, we are proudly rooted in our deep...

  • Data Engineer Lead for Global Data Platforms

    6 days ago

    Mumbai, Maharashtra, India CPP Investments Full time

    Job DescriptionThe Role: We are seeking an experienced Data Engineer Lead to oversee the design, development, and implementation of our global data platforms. As a key member of our team, you will be responsible for building high-performing teams with diverse skillsets to deliver services that meet and exceed business expectations.Key...

  • Director, Data Science Platform Engineer

    3 weeks ago

    Mumbai, Maharashtra, India Pfizer Full time

    ROLE SUMMARY Pfizer's Director – Team Lead, Data Science Platform Engineer leads the transformation of Pfizer into a digital powerhouse that will generate patient superior experiences which results in better health outcomes. The Analytics Experience team, which is part of the Artificial Intelligence, Data and Advanced Analytics (AIDA) organization...

  • Manager - Data Platform

    3 weeks ago

    Mumbai, Maharashtra, India Godrej Industries Group Full time

    JOB DESCRIPTION—————————————————————————————————————Data Engineer (Manager)Godrej Consumer Products Limited (GCPL)Mumbai, Maharashtra, India—————————————————————————————————————Job Title: Data EngineerJob Type: ...

  • Manager - Data Platform

    4 weeks ago

    Mumbai, Maharashtra, India Godrej Industries Group Full time

    JOB DESCRIPTION ————————————————————————————————————— Data Engineer (Manager) Godrej Consumer Products Limited (GCPL) Mumbai, Maharashtra, India ————————————————————————————————————— Job Title: Data Engineer...

  • Manager, Platform Engineer

    1 week ago

    Mumbai, Maharashtra, India Pfizer Full time

    Job Description Pfizer's Chief Digital Office (CDO) leads the transformation of Pfizer into a digital powerhouse that will generate patient superior experiences which results in better health outcomes. The Analytics Experience team, which is part of the Artificial Intelligence, Data and Advanced Analytics (AIDA) organization, is responsible for the...

  • Platform Engineer

    2 weeks ago

    Mumbai, Maharashtra, India 5100 Kyndryl Solutions Private Limited Full time

    Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward – always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The...

  • Sr. ML Platform Engineer

    3 weeks ago

    Mumbai, Maharashtra, India 5 Star Recruitment Full time

    We are seeking a seasoned ML Platform Engineer. This role involves designing and implementing scalable MLOps platforms for Fortune 500 clients, enabling seamless development, deployment, and maintenance of machine learning models. The successful candidate will leverage deep expertise in cloud platforms and DevOps practices to bridge the gap between data...

  • Data Engineer

    4 days ago

    Mumbai, Maharashtra, India XHire Full time

    Responsibilities : - As a Data Engineer you will be a key operational and hands-on engineer delivering business tenancies as part of the data platform strategy execution across all areas of the business. - You will work closely with other teams, especially the London core team in a multi-functional set-up. - The Data Engineer will be an experienced data...

  • Cloud Data Platform Specialist

    4 days ago

    Mumbai, Maharashtra, India XHire Full time

    X Hire is seeking a highly skilled Cloud Data Platform Specialist to join our team. As a Cloud Data Platform Specialist, you will be responsible for designing and implementing data pipelines and data structures that meet the needs of our business-critical applications.Key ResponsibilitiesDevelop and maintain data pipelines and data structures using...

  • Customer Data Platform

    5 days ago

    Mumbai, Maharashtra, India i4 Consulting : Reimagining HR Blueprints Full time

    Position: Customer Data Platform CDP - ConsultantYears of Exp: 5+ yearsAreas of Work: Customer Data Platform, Adobe Experience Platform, Digital Audience ManagementLevel: ConsultantLocation: Mumbai, Bangalore, Gurgaon, Hyderabad, PuneQualifications: Master of Business Administration / Post Graduate Diploma in ManagementMarketing platform Certifications: such...

  • Data engineer

    7 days ago

    Mumbai, Maharashtra, India Zemoso Technologies Full time

    Location: Pune / Mumbai Chennai / Hyderabad / (Hybrid)Notice Period : upto 60daysKey Responsibilities:Python Proficiency:● Demonstrate a strong command of Python programming language, actively contributing to thedevelopment and maintenance of data engineering solutions.Data Engineering Expertise:● Set up and maintain efficient data pipelines, ensuring...

  • Platform Engineer, Enterprise Platforms

    3 weeks ago

    Mumbai, Maharashtra, India Astellas Pharma Inc. Full time

    Job Description Platform Engineer, Enterprise Platforms (SAP/OTC Configuration)Do you want to be part of an inclusive team that works to develop innovative therapies for patients? Every day, we are driven to develop and deliver innovative and effective new medicines to patients and physicians. If you want to be part of this exciting work, you belong at...

  • Platform Engineer, Enterprise Platforms

    3 weeks ago

    Mumbai, Maharashtra, India Astellas Pharma Inc. Full time

    Responsibilities :Platform Development and Configuration: Design, develop, and configure business platforms to meet the specific needs of our organization. This could involve programming, configuring settings, and integrating various software solutions.System Integration: Ensure seamless integration between different business platforms and systems (e.g.,...