Senior Data Pipeline Developer

6 days ago


Ghaziabad, Uttar Pradesh, India beBeeTechnical Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

Job Description

  • Design and build a shared component library for data pipelines, including ingestion, parsing/normalization, extraction, validation, enrichment, and publishing.
  • Define patterns/templates for Apache Beam pipelines and Databricks jobs, standardizing configuration, packaging, versioning, CI/CD, and documentation.
  • Create interfaces so multiple teams can swap extractors, OCR providers, and EMR publishers without code rewrites.

Requirements

  • 7+ years of Python experience with strong depth in performance and concurrency, profiling, and memory tuning.
  • Observability expertise: Elastic APM instrumentation and dashboarding; Splunk for logs and correlation.
  • Hands-on experience with large-scale data pipelines, including Apache Beam and/or Spark, and Databricks.
  • Document processing: OCR, PDF parsing, text normalization.
  • LLM/SLM integration experience, prompt/eval frameworks, PII redaction/guardrails.

Skills We Value

  • Concurrent programming skills, including asyncio, concurrent.futures, multiprocessing.
  • Proficiency in Docker and Terraform for infrastructure management.
  • Strong architecture/design skills: clean interfaces, packaging shared libs, versioning, CI/CD.
  • API engineering skills for high-throughput integrations, resilience patterns, and secure handling of sensitive data.


  • Ghaziabad, Uttar Pradesh, India beBeeDeveloper Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Senior ETL Developer – Data Pipelines EngineerLocation: RemoteJob Type: Full-timeJob DescriptionWe are seeking an experienced ETL Developer to join our data engineering team. The successful candidate will work on designing and developing efficient data pipelines using IBM DataStage, AWS Glue, and Snowflake.The ideal candidate will collaborate...


  • Ghaziabad, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 25,00,000 - ₹ 35,00,000

    Freelance Data Engineer PositionWe are seeking an experienced Freelance Data Engineer to join our team and play a key role in designing, building, and optimizing large-scale data pipelines.The successful candidate will collaborate with internal teams and clients to ensure robust, scalable, and high-performance data solutions.Designing, developing, and...


  • Ghaziabad, Uttar Pradesh, India beBeeData Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Job Title: Data Pipeline ArchitectWe are seeking an experienced professional to design, develop, and optimize data ingestion pipelines.About the Role:Design and implement data ingestion pipelines for integrating multiple sources into Databricks.Implement and maintain CI/CD pipelines for data workflows.Deploy and manage containerized applications using...


  • Ghaziabad, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking a highly skilled and experienced Data Engineer to drive our data strategy.This role is responsible for designing, developing, and maintaining scalable data pipelines using PySpark.Design and develop complex data pipelines utilizing PySpark and Spark SQL.Collaborate with cross-functional teams to ensure accurate data processing,...


  • Ghaziabad, Uttar Pradesh, India beBeeTalend Full time ₹ 20,00,000 - ₹ 25,00,000

    Talend ETL Developer Job DescriptionWe are seeking a skilled professional to take on the role of Talend ETL Developer with expertise in Talend Open Studio for an immediate contractual position.**Job Overview:**The successful candidate will be responsible for designing and optimizing complex ETL pipelines, debugging, and deployment using TMC.**Key...


  • Ghaziabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 22,00,000

    Azure Data Engineer RoleSeeking a skilled data engineer to design, build and maintain automated data pipelines supporting business intelligence and analytics initiatives.We're looking for an expert who can leverage their knowledge of Azure Databricks, PySpark, SQL and ETL/ELT pipeline design to develop robust data pipelines that ingest, transform and curate...

  • Data Engineer

    2 weeks ago


    Ghaziabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    ETL Developer RoleWe are seeking an experienced ETL Developer to join our data engineering team.This role involves designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.About the RoleThe ideal candidate will have 4+ years of experience in ETL development with at least...


  • Ghaziabad, Uttar Pradesh, India beBeeDatabricks Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Databricks Engineer Job DescriptionThe primary objective of this role is to design, develop, and maintain scalable, secure, and high-performance data pipelines and ETL processes on Databricks.Key Responsibilities:Develop and implement data workflows that ensure performance, cost-efficiency, and reliability.Collaborate with cross-functional teams to gather...


  • Ghaziabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 20,00,000

    We are looking for a skilled Data Engineer to join our team.About the RoleThis is a challenging opportunity for a seasoned Data Engineer to design, implement and maintain robust data pipelines using CI/CD practices, Databricks (Spark), Python and SQL.The ideal candidate will have hands-on experience with modern DevOps/configuration management tools and...

  • Data Engineer

    6 days ago


    Ghaziabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Engineer - Designing Scalable Data PipelinesAbout the RoleWe're seeking a talented data engineer to join our team. As a key member of our data engineering team, you will play a crucial role in designing and implementing scalable data pipelines to support our growing business needs.Key ResponsibilitiesData Pipeline Development: Develops and...