Developer of Large-Scale Data Solutions

2 days ago

Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000
Software Engineer (Data)

This is a Software Engineer position focused on data infrastructure and large-scale data processing.

You will be working directly with researchers to accelerate experiments, develop new datasets, improve infrastructure efficiency, and enable key insights across our data assets.

Responsibilities
  • Design and Operate Scalable Data Infrastructure: Develop scalable, fault-tolerant data infrastructure to support distributed computing and data orchestration for LLM Research.
  • High-Throughput Systems: Develop and maintain high-throughput systems for data ingestion, processing, and transformation to support LLM model development.
  • Synthetic Datasets: Develop synthetic datasets using state-of-the-art solutions.
  • Collaboration: Collaborate with research teams to deliver critical data assets for model development and evaluation.
  • Monitoring and Alerting: Implement and maintain monitoring and alerting to support platform reliability and performance.
  • Systems for Traceability: Build systems for traceability, reproducibility, and robust quality control to ensure adherence to industry compliance standards.
Requirements

  • Data Infrastructure Experience: 5+ years of experience in data infrastructure, ideally supporting high-scale applications or research platforms.
  • Distributed Computing Frameworks: Fluent in distributed computing frameworks.
  • Cloud Infrastructure: Deeply familiar with cloud infrastructure, data storage architectures, and batch + streaming pipelines.
  • Specialized Hardware: Experience with specialized hardware (GPUs, TPUs) computing and GPU cluster.
  • Database Knowledge: Strong knowledge of databases, storage systems, and how architecture choices impact performance at scale.
  • Microservices Architecture: Familiar with microservices architectures, containerization and orchestration, and both synchronous and asynchronous processing.
  • Performance Optimization: Extensive experience with performance optimization and memory management in high-volume data systems.
  • Proactive Documentation: Proactive about documentation, automation, testing, and empowering your teammates with good tooling.
Working Conditions

This role is fully remote, India-based. Compensation and benefits vary based on background, skills, and experience levels.

  • Large Scale Data Architect

    20 hours ago

    Bellary, Karnataka, India beBeeData Full time ₹ 40,00,000 - ₹ 50,00,000

    Architect Large-Scale Data SolutionsThe successful candidate will be responsible for designing and implementing data solutions on Snowflake, including data ingestion, transformation, and optimization.They will have hands-on experience with Snowflake utilities, SnowSQL, SnowPipe, ETL data pipelines, Big Data model techniques using Python/Java.Proficiency in...

  • Large Scale Data Processing Expert

    7 days ago

    Bellary, Karnataka, India beBeeDataEngineering Full time ₹ 6,00,000 - ₹ 12,00,000

    Data Engineering Specialist As a seasoned data expert, you will play a pivotal role in designing, building, and maintaining large-scale data processing systems. With a strong background in computer science and experience in data engineering, you will be responsible for developing and implementing efficient data pipelines, integrating with various tools and...

  • Expert Large-Scale Data Pipeline Specialist

    6 days ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 25,00,000

    Data Engineering OpportunityWe are seeking a seasoned Freelance Data Engineer with expertise in designing, building, and optimizing large-scale data pipelines.This role involves developing and maintaining ETL/ELT pipelines using PySpark, AWS, and Databricks. Additionally, the successful candidate will perform data profiling, validation, and transformation...

  • Senior Data Solutions Developer

    1 week ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data EngineerWe are seeking an experienced Data Engineer to join our team. The ideal candidate will have a strong background in designing, developing, and maintaining large-scale data engineering solutions using Azure technologies.Create, maintain, and support data pipelines using Azure Data Factory, Databricks, and Synapse.Design and implement...

  • Large Scale Data Engineer

    2 weeks ago

    Bellary, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Description:We are looking for a skilled data engineer to join our team. The ideal candidate will have experience with designing and building robust data pipelines, as well as strong knowledge of the Google Cloud ecosystem.The successful candidate will be responsible for working with big data tools such as Hadoop, Spark, and Kafka, as well as hands-on...

  • Data Solutions Architect

    2 weeks ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 12,62,926 - ₹ 25,14,151

    We are seeking a highly skilled Data Solutions Architect with 4+ years of experience in designing, building, and optimizing large-scale data processing workflows. The ideal candidate will have hands-on experience with ETL pipeline design, development, and maintenance using Azure Data Factory, Databricks, and Spark.Key ResponsibilitiesDesign and develop...

  • Senior Digital Media Specialist

    7 days ago

    Bellary, Karnataka, India beBeePerformance Full time US$ 4,80,000 - US$ 6,00,000

    Seeking an experienced paid media leader to drive high-impact campaigns for a leading beauty brand.Key Responsibilities:Develop, execute, and scale targeted Facebook ad initiatives to maximize ROI.Manage and optimize large-scale digital budgets with precision and accountability.Conduct A/B testing, audience targeting, and performance analysis to inform...

  • Top Software Solutions Developer

    20 hours ago

    Bellary, Karnataka, India beBeeEngineer Full time US$ 1,50,000 - US$ 1,75,000

    Software Engineer OpportunityAN exceptional opportunity exists for a skilled engineer to drive technical excellence and contribute to the development of cutting-edge software solutions.Design, develop, and implement large-scale software solutions in production environments.Write, test, and document technical work products according to organizational...

  • Senior Data Developer

    1 week ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 20,00,000

    We are seeking a skilled Data Architect with hands-on experience in designing, developing and maintaining large-scale data systems.Required Skills & Experience:5+ years of experience as a Data EngineerExpertise in Python, Pyspark and SQLHands-on Python/PySpark coding experienceStrong Big Data experience, GCP preferredTelemetry experienceStreaming experience...

  • Large Data Migration Specialist

    6 days ago

    Bellary, Karnataka, India beBeeDataIntegration Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Migration Specialist: Large Dataset IntegrationAbout the Role:We are seeking a skilled Migration Specialist to join our team. As a key member of our data integration project, you will be responsible for migrating large datasets from various sources to target platforms using advanced tools and techniques.Key Responsibilities:Execute migrations using bulk...