Senior Data Pipeline Developer

4 days ago

Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

Job Overview:


Our team is seeking an experienced Data Engineer to join our ranks. As a key member of our data infrastructure, you will be responsible for developing and executing test strategies focused on data pipelines, including ingestion, transformation, storage, and retrieval.


  • Develop and execute comprehensive test plans to validate data pipeline performance, scalability, and reliability.
  • Perform in-depth data integrity validation by verifying schema consistency, data completeness, and correctness across distributed systems.
  • Conduct Change Data Capture (CDC) testing to ensure seamless data updates and synchronization, minimizing data inconsistencies and errors.
  • Optimize and validate queries for performance and correctness using SQL, Trino, and Amazon Redshift, ensuring efficient data processing and reduced latency.
  • Implement automated data validation and regression tests to monitor for anomalies, data drift, and pipeline failures, providing proactive insights for improvement.
  • Perform load testing and stress testing for large-scale data processing workflows, identifying potential bottlenecks and areas for optimization.

Key Responsibilities:


  1. Design and implement robust data pipelines to support business growth and innovation.
  2. Collaborate with cross-functional teams to define and implement data governance standards, ensuring security, retention, and access controls.
  3. Develop and maintain technical documentation to ensure knowledge sharing and continuous improvement.
  4. Identify opportunities for process improvements and optimize data workflows to reduce costs and increase efficiency.

Requirements:


  • 5 to 7 years of experience in data engineering, with a strong focus on data pipeline development and testing.
  • Primary skills: Python, Pyspark, Postgres, Data Pipeline QA.

What We Offer:


An immediate opportunity to join our team and contribute to the success of our organization.

  • Senior Data Pipeline Specialist

    1 week ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a highly skilled Senior Data Pipeline Specialist to join our team.As a Senior Data Pipeline Specialist, you will be responsible for designing and developing real-time and batch data pipelines using tools like Apache NiFi, Apache Kafka, and Apache Airflow.Design and develop scalable data pipelines that meet the needs of various business...

  • Senior Pipeline and GTM Data Expert

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,04,000 - ₹ 1,30,878

    Job OpportunityPipeline and GTM Senior Data AnalystWe are seeking a skilled Senior Data Analyst to join our team. This role will involve contributing to pipeline and GTM analytics, dashboard creation and maintenance, and providing operational support for the Sales Development Representative (SDR) team.Key Responsibilities:

  • Senior Data Pipeline Specialist

    5 days ago

    Bengaluru, Karnataka, India beBeeDataPipelineSpecialist Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking an experienced Senior Data Pipeline Specialist to lead the development and maintenance of complex data pipelines and enterprise-grade data systems.Job DescriptionThe ideal candidate will have 6-15 years of experience in ETL testing, software development, and data engineering. They must be proficient in Python, SQL, DWH, CI/CD, AWS, Automation,...

  • Senior Data Pipeline Developer

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 30,00,000

    Data Engineer - Design, Build, and Deliver Robust Data PipelinesWe are seeking a skilled data engineer to design, develop, and maintain high-performance data pipelines. The ideal candidate will have hands-on expertise with ETL tools and data platforms, a strong understanding of database systems, and proficiency in developing complex SQL queries.Key...

  • Senior Data Pipeline Developer

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering RoleWe are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines that meet business needs. The ideal candidate will have a strong understanding of data engineering principles, databases, and data platforms.Main Responsibilities:Develop and optimize Extract, Transform, Load (ETL)/Extract, Load,...

  • Data Pipeline Developer

    3 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job TitleWe are seeking a skilled Data Pipeline Developer to join our team. As a key member of our engineering group, you will be responsible for designing, building, and maintaining data pipelines that drive business success.About the RoleThis is an exciting opportunity to work with cutting-edge technology and collaborate with cross-functional teams. Your...

  • Senior Data Pipeline Architect

    7 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OverviewAs a senior data pipeline architect, you will be responsible for designing, developing, and deploying scalable data architectures using big data technologies.You will collaborate with stakeholders to understand data requirements and translate them into technical solutions. Your expertise in writing complex SQL queries and knowledge of advanced...

  • Senior Data Pipeline Developer

    1 week ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Big Data Lead Engineer Job SummaryThe role of Big Data Lead Engineer is crucial in developing and optimizing data pipelines and ETL workflows.

  • Senior Data Pipeline Developer

    2 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,000 - ₹ 4,00,000

    As a Senior Python Data Engineer, you will be responsible for designing and building high-performance data pipelines using Python. Your goal will be to develop scalable and efficient solutions that meet the needs of our organization.You will work closely with cross-functional teams to understand business requirements and design data architectures that...

  • Senior Data Pipeline Developer

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 30,00,000

    Data Engineering SpecialistWe are seeking a highly skilled Data Engineer to join our organization. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining scalable data pipelines to support our growing business needs.You will collaborate with analytics and business teams to improve data models that...