Data Pipeline Specialist

2 days ago

Ajmer, Rajasthan, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

Job Summary

We are seeking a highly skilled Data Engineer to design, build and maintain large-scale data pipelines. This individual will work with streaming data, collaborate with data scientists and stakeholders to deliver high-quality solutions.

Key Responsibilities:

  • Design and optimize complex data pipelines using Spark and ETL workflows.

  • Manage and process massive datasets using Spark, PySpark and SQL.

  • Build and maintain real-time streaming applications with Spark Streaming/Kafka.

  • Collaborate with data scientists to integrate AI/ML models into production environments.

  • Ensure data quality, scalability and performance in all pipelines.

  • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks).

  • Automate testing and deployment of Spark jobs via CI/CD pipelines.

Requirements:

  • Bachelor's/Master's degree in Computer Science, Computer Engineering or related field.

  • Minimum 4 years of professional experience as a Data Engineer.

  • Strong expertise in Python, Spark, PySpark.

  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution.

  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).

  • Proficiency in Airflow/Prefect for scheduling and orchestration.

  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).

  • Experience deploying on Databricks, AWS EMR, Azure HDInsight or GCP Dataproc.

  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).

Why You'll Love This Role

This position offers the opportunity to work on challenging projects, develop skills in data engineering and contribute to business growth.

  • Data Pipeline Specialist

    4 days ago

    Ajmer, Rajasthan, India beBeeEtl Full time ₹ 15,00,000 - ₹ 20,00,000

    ETL DeveloperWe are seeking a skilled ETL professional to join our data engineering team. The ideal candidate will have expertise in designing, developing and maintaining scalable data pipelines using IBM DataStage and AWS Glue.The successful applicant will collaborate with architects, business analysts and data modelers to ensure timely delivery of critical...

  • Data Pipeline Specialist

    12 hours ago

    Ajmer, Rajasthan, India beBeeData Full time ₹ 18,95,000 - ₹ 25,10,000

    Our organization seeks a Senior Data Engineer to contribute to the design and development of scalable, high-performance data pipelines using Databricks and PySpark.This role is part of a dynamic team that collaborates with data scientists, analysts, and other engineering teams to support end-to-end data solutions. As a Senior Data Engineer, you will be...

  • Data Engineering Specialist

    2 days ago

    Ajmer, Rajasthan, India beBeeDataEngineering Full time ₹ 1,68,00,000 - ₹ 2,28,00,000

    Job Title: Data Engineering SpecialistRole Overview: We are seeking an experienced Data Engineering Specialist to join our team. The ideal candidate will have a strong background in designing and developing scalable data pipelines using cloud-based technologies such as Azure Databricks and Pyspark.Key Responsibilities:Develop and implement efficient data...

  • Senior Data Integration Specialist

    1 day ago

    Ajmer, Rajasthan, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Job OverviewAs an experienced data integration specialist, you will work on developing, configuring, and troubleshooting large-scale data models, reference datasets, and integration workflows.Design and implement data pipelines using file-based, FTP/SFTP, API, real-time, and batch integrations.Develop and maintain reference datasets and support their...

  • Data Scientist

    2 days ago

    Ajmer, Rajasthan, India beBeeDatabricks Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Engineer with Databricks ExperienceWe are seeking an Ingestion Specialist to join our team.The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient ingestion workflows.Data Engineering Responsibilities:Design, develop, and optimize data ingestion...

  • Data Infrastructure Specialist

    12 hours ago

    Ajmer, Rajasthan, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer OpportunityWe are seeking a highly skilled Senior Data Engineer to design, build, and maintain our modern data platform.This role is perfect for someone with deep technical expertise in ETL pipeline design, data modeling, and data infrastructure who thrives in a fast-paced, engineering-driven environment.You'll work across the full...

  • Data Insights Specialist

    11 hours ago

    Ajmer, Rajasthan, India beBeeData Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Data Insights SpecialistJob DescriptionWe are seeking a skilled Data Insights Specialist to collect, process, and analyze large datasets for strategic decision-making. This role involves gathering, cleaning, and interpreting complex data from various sources to inform business decisions.The ideal candidate has strong skills in data querying,...

  • Data Transformation Specialist

    10 hours ago

    Ajmer, Rajasthan, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OverviewWe are seeking an experienced professional to fill a critical role in our organization.Key ResponsibilitiesDevelop and maintain Extract, Transform, Load (ETL) pipelines using FiveTran, DBT Cloud, and custom frameworks.Implement client-specific data transformations for HighTouch integration.Support data ingestion and quality checks during client...

  • Advanced Cloud Data Specialist

    14 hours ago

    Ajmer, Rajasthan, India beBeeDataEngineering Full time ₹ 25,00,000 - ₹ 40,00,000

    Job SummaryWe are seeking an experienced Senior Data Engineer to design and maintain cloud-based data architectures, build scalable real-time and batch processing workflows, and develop ETL/ELT pipelines.Key ResponsibilitiesDesign and maintain cloud-based data architectures, including data lakes, lakehouses, and warehousesBuild scalable real-time and batch...

  • Senior Data Scientist

    2 days ago

    Ajmer, Rajasthan, India beBeeMachineLearning Full time ₹ 15,00,000 - ₹ 25,00,000

    Key Skills for Machine Learning EngineerThis role requires a highly skilled Machine Learning Engineer to lead end-to-end machine learning pipelines on AWS SageMaker.Design, develop and deploy complex machine learning models and algorithms into production-ready code.Collaborate with data scientists to optimize model performance and reliability.Manage the...