Large Scale Data Processing Expert

1 day ago

Salem, Tamil Nadu, India beBeeDataEngineering Full time ₹ 12,00,000 - ₹ 24,00,000
Data Engineering Role

At Incept Labs, we are dedicated to building domain-specific AI solutions to enhance education and research. We believe in the power of human-AI collaboration, where each partner strengthens the other.

  • We are seeking a skilled Software Engineer with expertise in large-scale data processing to join our high-impact team responsible for both architecture and scaling of our data and model development infrastructure.

This is a remote position based in India. As a member of our compact team, you will work directly with researchers to accelerate experiments, develop new datasets, improve infrastructure efficiency, and enable key insights across our data assets.

About the Role

We are looking for an individual who can design, build, and operate scalable, fault-tolerant data infrastructure to support distributed computing and data orchestration for LLM Research.

  • Key Responsibilities:
  • Create high-throughput systems for data ingestion, processing, and transformation to support LLM model development
  • Develop synthetic datasets using state-of-the-art solutions
  • Collaborate with research teams to deliver critical data assets for model development and evaluation
  • Implement and maintain monitoring and alerting to support platform reliability and performance
  • Build systems for traceability, reproducibility, and robust quality control to ensure adherence to industry compliance standards
Requirements
  • 5+ years of experience in data infrastructure, ideally supporting high-scale applications or research platforms.
  • Fluent in distributed computing frameworks.
  • Deeply familiar with cloud infrastructure, data storage architectures, and batch + streaming pipelines.
  • Experience with specialized hardware (GPUs, TPUs) computing and GPU cluster.
  • Strong knowledge of databases, storage systems, and how architecture choices impact performance at scale.
  • Familiar with microservices architectures, containerization and orchestration, and both synchronous and asynchronous processing.
  • Extensive experience with performance optimization and memory management in high-volume data systems.
  • Proactive about documentation, automation, testing, and empowering your teammates with good tooling.
Benefits

This role offers a unique opportunity to work on cutting-edge AI projects, collaborate with top researchers, and contribute to the development of innovative solutions. If you are passionate about data engineering and want to make a meaningful impact, apply now.

Tech Stack
  • Distributed computing frameworks
  • Cloud infrastructure
  • Data storage architectures
  • Batch + streaming pipelines
  • Specialized hardware (GPUs, TPUs)

  • Solution Architect for Large Scale Data Pipelines

    5 days ago

    Salem, Tamil Nadu, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Solution Architect for Large Scale Data Pipelines

  • HR Manager

    5 days ago

    Salem, Tamil Nadu, India beBeeOperations Full time ₹ 1,20,00,000 - ₹ 1,40,00,000

    Job Opportunity:We are seeking a highly skilled professional to oversee and streamline HR functions for a large-scale workforce.Key ResponsibilitiesManage HR operations processes across the employee life cycle, ensuring seamless integration of new hires.Oversee onboarding, induction, and employee ID creation, guaranteeing efficient workflows.Drive compliance...

  • Large Scale Business Intelligence Solutions Specialist

    1 day ago

    Salem, Tamil Nadu, India beBeeBusiness Full time ₹ 10,00,000 - ₹ 20,00,000

    Job Title: Business Intelligence DeveloperKey Responsibilities:Develop and maintain large-scale business intelligence applications.Experience with data visualization, ETL processes, and database administration.About the Role:This role requires a skilled professional to design and implement business intelligence solutions. The successful candidate will have...

  • Transforming Data Engineering Expert

    6 days ago

    Salem, Tamil Nadu, India beBeeDataEngineer Full time US$ 2,00,000 - US$ 2,20,000

    Data Engineer Job DescriptionOur team is looking for a skilled Data Engineer to join our organization. As a Data Engineer, you will be responsible for designing and implementing large-scale data processing systems that are fast, reliable, and scalable.The ideal candidate will have a strong background in computer science, experience with modern data...

  • Expertise in Large Scale Data Engineering

    6 days ago

    Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 20,00,000

    Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team. The ideal candidate will have 4–5 years of experience in designing and implementing high-quality, secure, and scalable data pipelines and systems that power reporting, analytics, and operational use cases.This is a remote-first opportunity with a strong preference...

  • Chief Data Solutions Expert

    1 day ago

    Salem, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Role OverviewWe are seeking a highly skilled and experienced Chief Data Solutions Expert to join our team. This individual will be responsible for leading the design and implementation of enterprise-wide data solutions, working closely with cross-functional teams to deliver business value through data-driven insights.The ideal candidate will have a strong...

  • Senior Data Architecture Specialist

    7 days ago

    Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Are you a highly skilled data professional looking for a challenging role?We have an exciting opportunity for a Senior Data Engineer to join our team of experts.As a Senior Data Engineer, you will be responsible for designing, constructing, and optimizing large-scale data processing systems that enable advanced analytics and business intelligence.Key...

  • Big Data Pipeline Specialist

    2 days ago

    Salem, Tamil Nadu, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    GCP Data Engineering ExpertThe ideal candidate will possess in-depth knowledge of data engineering principles and practices, specifically within the Google Cloud Platform (GCP) ecosystem. They will be responsible for designing, building, and maintaining large-scale data pipelines that cater to various business requirements.The GCP Data Engineer will work...

  • Expert Chemist for Large Language Model Development

    7 days ago

    Salem, Tamil Nadu, India beBeeChemistry Full time ₹ 15,00,000 - ₹ 20,00,000

    Advanced Chemistry ExpertTuring seeks a highly skilled and analytical chemistry professional to work on cutting-edge projects.About the RoleThis is an opportunity for experts in chemistry to apply their knowledge and skills to help fine-tune large language models. The ideal candidate will have a strong foundation in chemistry, comparable to the level...

  • Senior Data Pipeline Specialist

    3 days ago

    Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 90,00,000 - ₹ 1,50,00,000

    Data Engineering is a vital role that involves designing, building and maintaining production-level data pipelines to handle structured and unstructured data. We are seeking a skilled Data Engineer to join our team.The ideal candidate has 1–2 years of experience in data engineering and possesses strong knowledge of AWS cloud services and big data best...