Scalable Data Pipeline Specialist

19 hours ago

Udaipur, Rajasthan, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000
Azure Data Engineer Position

Our company is seeking a skilled data engineer to join our team. The ideal candidate will have hands-on experience in designing, developing, and managing scalable data pipelines using modern Azure tools and best practices.

Job Description

The successful candidate will work closely with data architects, platform engineers, and DevOps teams to build efficient data engineering solutions that enable analytics, reporting, and business intelligence capabilities across the enterprise.

  • Key Responsibilities:
  • Design and build scalable ETL/ELT pipelines using Azure Data Factory (ADF), Azure Databricks (Spark), and Azure Synapse Analytics.
  • Develop and optimize PySpark/Spark SQL jobs for large-scale batch and streaming data transformations.
  • Ingest data from various sources including Apache Kafka, REST APIs, and RDBMS, ensuring real-time or near-real-time availability.
  • Implement data modeling strategies (star schema, snowflake schema) for analytics consumption layers in Synapse or ADLS.
  • Collaborate with DevOps teams to automate deployment using CI/CD pipelines (Azure DevOps, GitHub Actions, etc.).
  • Monitor, troubleshoot, and optimize data workflows for performance, cost-efficiency, and reliability.
  • Follow coding standards, participate in peer reviews, and maintain version-controlled code in Git repositories.
  • Support data quality checks, logging, alerting, and observability mechanisms for production workloads.
  • Participate in sprint ceremonies and contribute to task estimation and delivery planning.
Requirements and Qualifications

To be considered for this position, applicants should possess the following skills and qualifications:

  • 5+ years of experience in data engineering roles.
  • Hands-on experience with:
    • Azure Data Factory (ADF) – building pipelines, triggers, linked services.
    • Azure Databricks – building and managing Spark jobs in PySpark.
    • Azure Synapse Analytics – data warehousing, SQL queries, workspace orchestration.
    • Apache Kafka – consuming and processing real-time data streams.
  • Strong in SQL, Python, and Spark for data manipulation and transformation.
  • Exposure to CI/CD practices (Azure DevOps, Git workflows, build/release pipelines).
  • Understanding of data lake architecture and modern data warehousing principles.
Preferred Certifications

Applicants with the following certifications are preferred:

  • Microsoft Certified: Azure Data Engineer Associate.
  • Databricks Certified Associate Developer for Apache Spark.
  • Azure Fundamentals (AZ-900) – optional but helpful.

Work Environment

The selected candidate will work in a dynamic and collaborative environment with a talented team of professionals.

Benefits

The company offers a competitive compensation package and opportunities for professional growth and development.

  • Building Scalable Data Pipelines

    7 days ago

    Udaipur, Rajasthan, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a seasoned Cloud Data Engineer to join our team and take charge of designing and building scalable data pipelines.About the RoleThe ideal candidate will have a strong background in Azure Data Factory, with expertise in ETL/ELT processes and pipeline design.Design and build data pipelines using Azure Data Factory that ensure smooth, secure, and...

  • Data Pipeline Specialist

    7 days ago

    Udaipur, Rajasthan, India beBeeFinancial Full time ₹ 20,00,000 - ₹ 25,00,000

    We are seeking a highly skilled Big Data Engineer to join our growing team. In this role, you will be responsible for designing, building, and maintaining robust data pipelines that handle high-volume financial data, including stocks, cryptocurrencies, and third-party data sources. You will play a critical role in ensuring data integrity, scalability, and...

  • Data Pipeline Specialist

    1 week ago

    Udaipur, Rajasthan, India beBeeData Full time ₹ 15,00,000 - ₹ 22,50,000

    Unlock Your Potential as a Data EngineerWe are seeking an experienced Data Engineer to drive innovation in our data pipeline team.Your expertise in designing, developing, and running complete end-to-end data pipelines on the Azure platform will be invaluable.As a key member of our team, you will work closely with Agile teams to handle data validation,...

  • Cloud Data Pipeline Specialist

    1 week ago

    Udaipur, Rajasthan, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data EngineerWe are seeking an experienced data engineer with expertise in designing, building, and optimizing scalable data pipelines and cloud solutions. The ideal candidate will have strong experience in AWS cloud services, data engineering, and knowledge of upstream, midstream or downstream processes to support advanced analytics, reporting,...

  • Senior Data Pipeline Specialist

    1 week ago

    Udaipur, Rajasthan, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking an experienced Senior Data Engineer to design and develop scalable data pipelines using Databricks and PySpark.Our ideal candidate will have a strong understanding of distributed data processing, CI/CD pipelines, and infrastructure as code using Terraform.The role involves implementing and managing data solutions on AWS, including...

  • Data Pipeline Architect

    2 days ago

    Udaipur, Rajasthan, India beBeeDataPlatformEngineer Full time US$ 1,02,600 - US$ 1,58,400

    Job DescriptionWe are seeking a skilled Data Platform Engineer to design, develop and maintain robust data pipelines.Develop and manage complex data pipelines using ETL processes to extract, transform and load (ETL) data from various sources into our data warehouse or lake environment.Automate, manage and scale the underlying infrastructure for our data...

  • Lead Data Pipeline Architect

    1 week ago

    Udaipur, Rajasthan, India beBeeDataEngineer Full time US$ 1,50,000 - US$ 1,75,000

    Job Description:The Senior Data Engineer plays a pivotal role in constructing and optimizing the data pipeline architecture. This individual will lead the optimization and scalability of data delivery pipelines, ensuring performance and reliability for cross-functional stakeholders. This key position is responsible for architecting, building, and maintaining...

  • Data Architect for Scalable Solutions

    1 week ago

    Udaipur, Rajasthan, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Key Responsibilities:Develop and maintain robust ETL pipelines for scalable data processing using Snowflake, Matillion, and Python.Implement data models to support analytics and reporting across the organization.Maintain medallion architecture (Bronze, Silver, Gold layers) for scalable data processing.Write complex SQL queries for data manipulation and...

  • Cloud Data Pipeline Architect

    1 week ago

    Udaipur, Rajasthan, India beBeeDataEngineer Full time ₹ 14,40,000 - ₹ 25,12,000

    Job Title:Data EngineerAbout Us:We're leading the transformation of the insurance industry by developing innovative benefits management solutions.We focus on modernizing complex back-office systems to create scalable, secure, and high-performing platforms that streamline operations for our clients.Position Overview:As a self-driven Data Engineer, you'll...

  • Scalable Data Solutions Professional

    6 days ago

    Udaipur, Rajasthan, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Scalable Data Solutions EngineerWe are seeking a highly skilled and experienced Data Solutions Engineer to design, build, and maintain scalable data pipelines and workflows.Proficiency in Python (Pandas, SQLAlchemy) and hands-on experience with ETL/ELT processes is required.Intermediate to advanced proficiency in SQL and knowledge of data quality and...