Senior Data Pipeline Engineer

19 hours ago

Salem, Tamil Nadu, India beBeeIngestion Full time US$ 1,00,000 - US$ 1,50,000
Job Description

We are seeking a Data Ingestion Specialist with expertise in continuous integration and deployment (CI/CD) and container orchestration to design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks.

The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient ingestion workflows.

  • Design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks.
  • Implement and maintain CI/CD pipelines for data workflows.
  • Deploy and manage containerized applications using Kubernetes.
  • Ensure data integrity, security, and compliance throughout the ingestion process.
  • Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies.
  • Troubleshoot and optimize ingestion pipelines for performance and scalability.
Required Skills & Qualifications
  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools such as GitHub Actions, Jenkins, or Azure DevOps.
  • Strong knowledge of Kubernetes and container orchestration.
  • Experience with Databricks, Spark, and data lake architectures.
  • Proficiency in Python, Scala, or SQL for data processing.
  • Familiarity with cloud platforms such as AWS, Azure, or GCP.
Preferred Qualifications
  • Experience with Infrastructure as Code (IaC) using Terraform or Helm.
  • Background in streaming data ingestion using Kafka or Kinesis.
  • Knowledge of data governance and security best practices.

  • Data Pipeline Test Engineer

    16 hours ago

    Salem, Tamil Nadu, India beBeeTest Full time ₹ 15,00,000 - ₹ 21,00,000

    Job TitleA senior test engineer is needed to lead the testing strategy and execution for a project using Databricks, Data Factory pipelines, and Power BI.5+ years of experience in data pipeline/ETL testing with at least 2+ years as a Test Lead.Key responsibilities include:Driving the testing strategy and execution for the project.Leading a team of test...

  • Senior Data Pipelines Architect

    16 hours ago

    Salem, Tamil Nadu, India beBeeData Full time ₹ 1,51,00,000 - ₹ 2,01,00,000

    Job OverviewWe are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines on Databricks.The ideal candidate will have a strong background in data engineering, experience with cloud platforms, and expertise in Databricks (including Databricks SQL, Delta Lake, and MLflow).ResponsibilitiesDesign and implement data...

  • Senior Big Data Engineer

    16 hours ago

    Salem, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    About the RoleThis exciting opportunity is for a seasoned Big Data Engineer to join our data-driven organization. As a key member of our Professional Services team, you will play a crucial role in designing, building, and maintaining high-performance data pipelines that process massive amounts of data.

  • Senior Data Engineer

    6 hours ago

    Salem, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking a seasoned expert to lead full lifecycle implementations — from secure platform setup and cloud integration to high-performance data pipeline development.This is an opportunity for someone who thrives at the intersection of data engineering, infrastructure, and DevOps, and wants to be part of enterprise-grade analytics and AI...

  • Senior Data Architecture Specialist

    16 hours ago

    Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Are you a highly skilled data professional looking for a challenging role?We have an exciting opportunity for a Senior Data Engineer to join our team of experts.As a Senior Data Engineer, you will be responsible for designing, constructing, and optimizing large-scale data processing systems that enable advanced analytics and business intelligence.Key...

  • Senior Technical Lead for Azure Data Solutions

    16 hours ago

    Salem, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Lead Azure Data Engineer Opportunity\We are seeking a highly skilled and experienced Lead Data Engineer to join our team in a senior technical role. This position requires hands-on expertise in Azure and Databricks, as well as strong problem-solving skills and ability to mentor team members.\The ideal candidate will have 8+ years of experience in Data...

  • Senior Microservices Engineer

    4 weeks ago

    Salem, Tamil Nadu, India BuildingWorld Full time

    Company: BuildingWorld (Stealth Startup by Mukesh Buildtech)Location: Remote / India preferredJob Type: Full-timeJob Function: Engineering / BackendIndustry: SaaS / ConstructionTech / MarketplacesSeniority Level: Senior-LevelJob DescriptionWe're building the backend infrastructure for an India-first marketplace — and we need an experienced...

  • Senior Microservices Engineer

    4 weeks ago

    Salem, Tamil Nadu, India BuildingWorld Full time

    Company: BuildingWorld (Stealth Startup by Mukesh Buildtech)Location: Remote / India preferredJob Type: Full-timeJob Function: Engineering / BackendIndustry: SaaS / ConstructionTech / MarketplacesSeniority Level: Senior-LevelJob DescriptionWe're building the backend infrastructure for an India-first marketplace — and we need an experienced Microservices...

  • Senior Microservices Engineer

    4 weeks ago

    Salem, Tamil Nadu, India BuildingWorld Full time

    Company: BuildingWorld (Stealth Startup by Mukesh Buildtech) Location: Remote / India preferred Job Type: Full-time Job Function: Engineering / Backend Industry: SaaS / ConstructionTech / Marketplaces Seniority Level: Senior-Level Job Description We're building the backend infrastructure for an India-first marketplace — and we need an experienced...

  • Senior Microservices Engineer

    4 weeks ago

    Salem, Tamil Nadu, India BuildingWorld Full time

    Company: BuildingWorld (Stealth Startup by Mukesh Buildtech)Location: Remote / India preferredJob Type: Full-timeJob Function: Engineering / BackendIndustry: SaaS / ConstructionTech / MarketplacesSeniority Level: Senior-LevelJob DescriptionWe're building the backend infrastructure for an India-first marketplace — and we need an experienced Microservices...