Data Pipeline Developer

7 days ago

Hyderabad, Telangana, India beBeeData Full time ₹ 80,00,000 - ₹ 1,20,00,000

Data Pipeline Developer

Job Summary:

We are seeking a skilled data pipeline developer to design, build, and maintain scalable and reliable data pipelines across our Data & AI ecosystem.

This role is crucial in enabling downstream analytics, machine learning, and business intelligence solutions by ensuring robust and automated data acquisition from various internal and external sources.

Key Responsibilities:

  • Design, develop, and maintain scalable and reusable data ingestion pipelines to onboard structured and semi-structured data from APIs, flat files, databases, and external systems.
  • Work with Azure-native services and tools like Databricks or Apache Spark for data ingestion and transformation.
  • Develop and manage metadata-driven ingestion frameworks to support dynamic and automated onboarding of new sources.
  • Collaborate closely with source system owners, analysts, and data stewards to define data ingestion specifications and implement monitoring/alerting on ingestion jobs.
  • Ensure data quality, lineage, and governance principles are embedded into ingestion processes.
  • Optimize ingestion processes for performance, reliability, and cloud cost efficiency.
  • Support batch and real-time ingestion needs, including streaming data pipelines where applicable.

Required Skills and Qualifications:

  • 3+ years of hands-on experience in data engineering (bonus: with a specific focus on data ingestion or integration).
  • Hands-on experience with Azure Data Services or equivalent cloud-native tools.
  • Experience in Python (PySpark) for data processing tasks (bonus: SQL knowledge).
  • Experience with ETL frameworks, orchestration tools, and working with API-based data ingestion.
  • Familiarity with data quality and validation strategies, including schema enforcement and error handling.
  • Good understanding of CI/CD practices, version control, and infrastructure-as-code (e.g., Terraform, Git).
  • Bonus: Experience with streaming ingestion (e.g., Kafka, Event Hubs, Spark Structured Streaming).

Benefits:

This role offers the opportunity to work on cutting-edge data engineering projects and collaborate with experienced professionals in the field.

Others:

Please note that this job requires strong problem-solving skills, attention to detail, and excellent communication skills.

],

  • Senior Data Pipeline Developer

    4 days ago

    Hyderabad, Telangana, India beBeeEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Data Engineering LeaderWe are seeking a highly skilled data engineer to join our organization. The successful candidate will be responsible for designing, implementing and optimizing data pipelines using Azure Data Factory (ADF) and Power BI.Key Responsibilities:Develop and maintain data pipelines using ADF for data integration and ETL processes.Design,...

  • Data Pipelines Developer

    6 days ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 1,20,00,000

    Job Title: Junior Data Engineer">Design and implement data pipelines using Python, PySpark, and AWS.Technical Skills Required:Languages: Proficient in Python, with working knowledge of PySparkCloud: Practical exposure to AWS (S3, Lambda, CloudWatch)Palantir Foundry: Foundational understanding or hands-on experienceVersion Control & CI/CD: Git, Jenkins or AWS...

  • Senior Data Pipeline Developer

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

    Immerse yourself in a role that requires creativity, analytical skills, and technical expertise. As a Data Engineer, you will be responsible for crafting data pipelines that drive business decisions.Key aspects of this position include designing scalable ETL/ELT pipelines using Python, collaborating with cross-functional teams to understand data...

  • Data Pipeline Specialist

    1 week ago

    Hyderabad, Telangana, India beBeeDataPipelineManager Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Job Title: Data Team LeadData Pipeline ManagerWe are seeking a seasoned data pipeline manager with experience in designing and developing robust data pipelines using Azure Databricks, Data Factory, and Datalake.The role involves overseeing the development of complex ETL solutions, managing large volumes of data, and working closely with business teams to...

  • Data Pipeline Specialist

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataPipeline Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking an accomplished Data Pipeline Specialist to join our team. The successful candidate will be responsible for creating and maintaining data pipelines that drive business growth.Main ResponsibilitiesCreate and implement new data pipelines to meet business needsEssential Skills and Qualifications4-6 years of experience in designing and...

  • Data Pipeline Specialist

    3 days ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 8,00,000 - ₹ 12,00,000

    Data Engineer PositionWe are seeking a skilled data engineer to join our team. The ideal candidate will be responsible for designing, implementing, and optimizing data pipelines on our platform.Data Pipeline Development: Design, develop, and maintain scalable data pipelines using Azure Synapse, Databricks & Apache Spark (PySpark).Data Integration: Integrate...

  • Lead Data Pipeline Developer

    7 days ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878

    Data Engineer RoleWe are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing and building scalable data pipelines using PySpark and Python.

  • Data Pipeline Architect

    1 week ago

    Hyderabad, Telangana, India beBeeBusiness Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Role Summary:Lead Business Intelligence DeveloperWe are seeking an experienced and skilled Lead Business Intelligence Developer to design, develop, and maintain robust data pipelines using Azure Data Factory.This is a key position that requires strong technical expertise and leadership skills. You will work closely with cross-functional teams to understand...

  • Data Pipeline Architect

    1 week ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Data EngineerWe are seeking a seasoned professional to fill the role of Senior Data Engineer. The successful candidate will design and develop end-to-end data integration solutions, optimize workflows, and collaborate with stakeholders to deliver scalable and high-performing data pipelines.Develop ETL/ELT workflows using Informatica...

  • Mastering Data Pipelines

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data EngineerWe are seeking a highly skilled data professional to design, build, and optimize data pipelines.Key Responsibilities:Develop scalable, high-performance data architectures using Spark and SQL.Process large datasets using PySpark and drive business insights.Create real-time streaming applications with Spark Streaming/Kafka.Collaborate...