Senior Data Pipeline Specialist

3 days ago

Bengaluru, Karnataka, India beBeeDataPipelineSpecialist Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

We are seeking an experienced Senior Data Pipeline Specialist to lead the development and maintenance of complex data pipelines and enterprise-grade data systems.

Job Description

The ideal candidate will have 6-15 years of experience in ETL testing, software development, and data engineering. They must be proficient in Python, SQL, DWH, CI/CD, AWS, Automation, Jenkins, and GitHub.

Key Responsibilities:
  • Design, develop, and execute advanced data pipeline architectures for large-scale data operations.
  • Develop and execute sophisticated SQL queries for data validation, reconciliation, and transformation checks.
  • Build and maintain robust data validation automation frameworks using Python, with seamless integration into CI/CD pipelines (e.g., GitHub Actions, Jenkins).
  • Collaborate with DevOps teams for automation workflows and contribute to version-controlled repositories on GitHub.
  • Participate in performance Testing/Monitoring and data integrity assurance across structured and semi-structured data sets.
Required Skills:
  • Extensive hands-on experience in ETL testing, software development, and data engineering mandatory.
  • Expertise in advanced SQL for large-scale data operations mandatory.
  • Proven ability to automate data validation using Python, with exposure to CI/CD tools and GitHub mandatory.
Desirable Skills:
  • Working knowledge of AWS cloud services, especially S3 and Airflow.
  • Experience with PostgreSQL, Oracle, or SQL Server databases.

  • Senior Data Pipeline Specialist

    7 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a highly skilled Senior Data Pipeline Specialist to join our team.As a Senior Data Pipeline Specialist, you will be responsible for designing and developing real-time and batch data pipelines using tools like Apache NiFi, Apache Kafka, and Apache Airflow.Design and develop scalable data pipelines that meet the needs of various business...

  • Data Pipeline Engineer

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    The Role of Senior Data Integration SpecialistJob Overview:We are seeking an experienced Senior Data Integration Specialist to join our team. This is a key position that will play a vital role in the smooth operation of our enterprise data integration platform.As a Senior Data Integration Specialist, you will be responsible for designing, implementing, and...

  • Data Pipeline Specialist

    5 days ago

    Bengaluru, Karnataka, India beBeeDataPipeline Full time ₹ 12,00,000 - ₹ 25,00,000

    Job Opportunity: Data Pipeline SpecialistWe are seeking a skilled data pipeline expert to validate and optimize our data transformations.ETL testing methodologies and best practicesDebugging data issues and resolving discrepanciesWorking with large datasets in cloud environmentsThe ideal candidate should have hands-on experience in Databricks, Python, SQL,...

  • Data Pipeline Specialist

    6 days ago

    Bengaluru, Karnataka, India beBeeDataPipeline Full time

    Job Title: Data Pipeline SpecialistA leading IT consulting and software solutions provider is seeking a highly skilled professional to design, build, and maintain scalable and reliable data pipelines.This role involves developing ETL processes to ensure data quality, integrity, and consistency. The ideal candidate will collaborate with analysts, scientists,...

  • Data Pipeline Specialist

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataPipeline Full time US$ 1,00,000 - US$ 1,50,000

    Job Overview- We are seeking a highly skilled Data Pipeline Specialist to join our team. The ideal candidate will have experience building and productionizing data pipelines using industry-leading technologies.- A strong background in working with complex SQL datasets is required, as well as expertise in building efficient SQL scripts using BigQuery Syntax-...

  • Data Pipeline Specialist

    2 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 28,00,000

    Job Title : Data Pipeline SpecialistKey Responsibilities :Develop and manage scalable data pipelines using PySpark and SQL to drive business growth.Collaborate with cross-functional teams to implement data-driven solutions leveraging AWS services.Utilize Databricks for advanced data processing and analytics, working closely with data scientists and analysts...

  • Senior Data Solutions Specialist

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,20,00,000 - ₹ 1,80,00,000

    Senior Data Solutions SpecialistWe are seeking a highly skilled data solutions specialist to design and implement cutting-edge data pipelines and workflows.

  • Data Pipeline Specialist

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

    Job TitleSenior Data EngineerJob DescriptionAs a Senior Data Engineer, you will design, implement, and optimize data pipelines for large-scale data processing. You will work with data scientists, analysts, and engineers to ensure data availability and quality.Key responsibilities include developing and maintaining ETL/ELT workflows using Spark, Hadoop, Hive,...

  • Principal Data Pipeline Architect

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,25,000 - US$ 1,75,000

    About the RoleWe are seeking a Senior Data Engineer with expertise in building scalable data pipelines and integrating healthcare data. Reporting to the Head of AI and Engineering, you will design, build, and operate the backbone of our AI-powered applications.You will work closely with software engineers, ML specialists, and clinical partners to bring our...

  • Data Pipeline Architect

    1 week ago

    Bengaluru, Karnataka, India beBeeAzure Full time ₹ 15,00,000 - ₹ 25,00,000

    Freelance Azure Data Factory SpecialistWe are seeking an experienced Freelance Azure Data Factory (ADF) Specialist to support our data integration and ETL needs.The ideal consultant will have strong hands-on expertise in Azure ADF pipelines, data orchestration, and transformations to ensure smooth, scalable, and secure data flows across multiple sources and...