Senior Large-Scale Data Processing Specialist

1 day ago

Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

Job Overview

We are seeking a highly skilled Data Engineer to join our compact team. The ideal candidate will have expertise in large-scale data processing and experience working with distributed computing frameworks.

The role requires the design, build, and operation of scalable, fault-tolerant data infrastructure to support LLM Research.

  • Design and Build Scalable Data Infrastructure: Create high-throughput systems for data ingestion, processing, and transformation to support LLM model development.
  • Develop Synthetic Datasets: Utilize state-of-the-art solutions to create synthetic datasets.
  • Collaborate with Researchers: Work closely with research teams to deliver critical data assets for model development and evaluation.
  • Implement Monitoring and Alerting: Develop monitoring and alerting systems to ensure platform reliability and performance.
  • Build Systems for Traceability: Establish systems for traceability, reproducibility, and robust quality control to ensure adherence to industry compliance standards.

Required Skills and Qualifications

  • 5+ Years of Experience: In data infrastructure, preferably supporting high-scale applications or research platforms.
  • Distributed Computing Frameworks: Proficient in fluent distributed computing frameworks.
  • Cloud Infrastructure: Experienced in cloud infrastructure, data storage architectures, and batch + streaming pipelines.
  • Specialized Hardware: Familiar with specialized hardware (GPUs, TPUs) computing and GPU cluster.
  • Database Knowledge: Strong knowledge of databases, storage systems, and how architecture choices impact performance at scale.
  • Microservices Architecture: Experienced in microservices architectures, containerization and orchestration, and both synchronous and asynchronous processing.

Benefits

This role is fully remote, India-based, offering competitive compensation and benefits based on background, skills, and experience levels.

Other Opportunities

Join a dynamic team passionate about building next-generation AI solutions to enhance education and research.

  • Mastering Large-Scale Data Engineering

    1 week ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer RoleWe are seeking a seasoned data engineer to collaborate on designing and building large-scale data platforms using Google Cloud.Key Responsibilities:Design, build, and maintain robust data pipelines for ingestion, transformation, and analytics.Develop scalable workflows utilizing tools like Airflow and Tivoli.Implement data...

  • Lead Data Processing Specialist

    5 days ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time US$ 10,59,000 - US$ 14,10,000

    Data Engineer PositionWe are seeking a skilled Data Engineer to join our team.Design, develop, and maintain large-scale data processing systems on Google Cloud Platform.Collaborate with cross-functional teams to implement data engineering solutions.Develop and implement ETL (Extract, Transform, Load) processes using Python and other tools.Requirements:3-7...

  • Senior Big Data Specialist

    4 days ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 23,40,000 - ₹ 30,14,000

    Job Title: Senior Data EngineerJob Summary:Highly skilled data engineer with 5+ years of experience in Python, PySpark, and SQL.Key Responsibilities:Expertise in big data processing, telemetry, and streaming analytics.Strong background in backend engineering and system architecture.Familiarity with GCP and experience working with IoT systems.Ability to...

  • Senior Data Engineering Specialist

    6 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe are seeking a highly skilled Senior Data Engineer with expertise in ETL pipeline design, data modeling, and data infrastructure.We are looking for a detail-oriented professional with strong experience in designing and implementing efficient ETL/ELT pipelines and data flows.This role is ideal for someone who thrives in a fast-paced engineering...

  • Data Migration Specialist

    1 day ago

    Thoothukudi, Tamil Nadu, India beBeeMigration Full time US$ 1,50,000 - US$ 2,50,000

    Job Title:Data Engineer for Large-Scale SQL Migration\DescriptionWe are seeking a skilled professional to lead the migration of large-scale SQL workloads from on-premise environments to Azure. The ideal candidate will have hands-on experience in modern data engineering tools and platforms within the Azure ecosystem, with a strong foundation in data...

  • Design and Execute Large-Scale Distributed Systems

    1 day ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Data Engineer SpecialistWe are seeking an experienced Data Engineer to join our organization. The ideal candidate will have a strong background in big data technologies and relational databases, with expertise in designing, executing, and managing complex distributed data systems.

  • Senior Data Engineer

    5 days ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,10,00,000

    Job Title: Senior Data EngineerJob Description:This is an exciting opportunity to work on large-scale, high-availability systems that handle complex supply chain operations and personalized customer experiences.As a senior data engineer, you will design, develop, and automate scalable ETL workflows using Python and Pandas. You will implement and manage data...

  • Senior HCM Implementation Specialist

    1 day ago

    Thoothukudi, Tamil Nadu, India beBeeImplementation Full time ₹ 25,00,000 - ₹ 40,00,000

    Job Title: Senior HCM Implementation SpecialistRole Overview:Lead large-scale HCM implementation projects with a proven track record of 8-10 years in the field.Key Responsibilities: Spearhead project management, overseeing all aspects from planning to delivery. Expertly navigate OTL, US Payroll, Talent Management, Benefits, Security, and Absence Management...

  • Cloud Data Specialist

    5 days ago

    Thoothukudi, Tamil Nadu, India beBeeData Remote Work Freelance Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Cloud Data SpecialistWe are seeking a skilled Cloud Data Specialist with expertise in designing and implementing cloud-based solutions using Azure and Google Cloud platforms.The ideal candidate will have a good understanding of data integration, data cleanup, and related products. They should be able to design and implement efficient data pipelines and...

  • Data Integration Specialist

    7 days ago

    Thoothukudi, Tamil Nadu, India beBeeInformatica Full time ₹ 9,00,000 - ₹ 14,00,000

    We are seeking a Data Integration Specialist to join our team. This role involves developing, enhancing and maintaining ETL processes using Informatica PowerCenter / IICS.The ideal candidate will have 3-7 years of experience in data integration and should be proficient in designing, developing and tuning ETL processes. Experience with SAP, Salesforce and...