Senior Data Pipeline Specialist

19 hours ago

Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

We are seeking a highly skilled Senior Data Pipeline Specialist to join our team.

As a Senior Data Pipeline Specialist, you will be responsible for designing and developing real-time and batch data pipelines using tools like Apache NiFi, Apache Kafka, and Apache Airflow.

  • Design and develop scalable data pipelines that meet the needs of various business stakeholders.
  • Implement and maintain ETL/ELT processes, ensuring data quality and integrity across various data sources.
  • Collaborate with cross-functional teams to integrate and manage data flows across diverse systems using NiFi and Kafka topics.
  • Monitor and optimize the performance of data pipelines and troubleshoot issues proactively.
  • Work with the ELK stack (Elasticsearch, Logstash, Kibana) for logging, monitoring, and real-time analytics.
  • Write efficient and optimized SQL queries for data extraction, transformation, and reporting.
  • Develop meaningful Power BI dashboards and reports in collaboration with data analysts and business stakeholders.
Required Skills:
  • Strong experience with Apache NiFi for data flow management.
  • Expertise in Kafka (streaming data, Kafka Connect, schema management).
  • Proficiency in Apache Airflow for orchestrating workflows.
  • Solid understanding of SQL (writing complex queries, tuning, indexing).
  • Experience working with the ELK stack Elasticsearch for search and analytics, Logstash for ingestion, Kibana for visualization.
  • Experience with Power BI data modeling, DAX, dashboard development.
  • Knowledge of data warehousing concepts, data modeling, and data lake architecture.

  • Data Pipeline Engineer

    4 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    The Role of Senior Data Integration SpecialistJob Overview:We are seeking an experienced Senior Data Integration Specialist to join our team. This is a key position that will play a vital role in the smooth operation of our enterprise data integration platform.As a Senior Data Integration Specialist, you will be responsible for designing, implementing, and...

  • Senior Data Pipeline Specialist

    1 week ago

    Bengaluru, Karnataka, India beBeeAutomatedTesting Full time

    Job Opportunity:The role of Senior Consultant involves designing, developing and implementing automated test cases for complex data pipelines.Ability to design, develop, and implement automated test cases for complex data pipelines is a mustAWS experience is essentialPharma expertise is also requiredProficiency in programming languages such as Python, R, or...

  • Data Pipeline Specialist

    4 days ago

    Bengaluru, Karnataka, India beBeeDataPipeline Full time US$ 1,00,000 - US$ 1,50,000

    Job Overview- We are seeking a highly skilled Data Pipeline Specialist to join our team. The ideal candidate will have experience building and productionizing data pipelines using industry-leading technologies.- A strong background in working with complex SQL datasets is required, as well as expertise in building efficient SQL scripts using BigQuery Syntax-...

  • Senior Data Solutions Specialist

    4 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,20,00,000 - ₹ 1,80,00,000

    Senior Data Solutions SpecialistWe are seeking a highly skilled data solutions specialist to design and implement cutting-edge data pipelines and workflows.

  • Data Pipeline Specialist

    1 week ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

    Job TitleSenior Data EngineerJob DescriptionAs a Senior Data Engineer, you will design, implement, and optimize data pipelines for large-scale data processing. You will work with data scientists, analysts, and engineers to ensure data availability and quality.Key responsibilities include developing and maintaining ETL/ELT workflows using Spark, Hadoop, Hive,...

  • Principal Data Pipeline Architect

    5 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,25,000 - US$ 1,75,000

    About the RoleWe are seeking a Senior Data Engineer with expertise in building scalable data pipelines and integrating healthcare data. Reporting to the Head of AI and Engineering, you will design, build, and operate the backbone of our AI-powered applications.You will work closely with software engineers, ML specialists, and clinical partners to bring our...

  • Data Pipeline Architect

    3 days ago

    Bengaluru, Karnataka, India beBeeAzure Full time ₹ 15,00,000 - ₹ 25,00,000

    Freelance Azure Data Factory SpecialistWe are seeking an experienced Freelance Azure Data Factory (ADF) Specialist to support our data integration and ETL needs.The ideal consultant will have strong hands-on expertise in Azure ADF pipelines, data orchestration, and transformations to ensure smooth, scalable, and secure data flows across multiple sources and...

  • Senior Pipeline and GTM Data Expert

    2 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,04,000 - ₹ 1,30,878

    Job OpportunityPipeline and GTM Senior Data AnalystWe are seeking a skilled Senior Data Analyst to join our team. This role will involve contributing to pipeline and GTM analytics, dashboard creation and maintenance, and providing operational support for the Sales Development Representative (SDR) team.Key Responsibilities:

  • Lead Data Pipeline Specialist

    5 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,04,000 - US$ 1,30,878

    Job Title: Senior Data EngineerAbout the Role:This is an exceptional opportunity to join our organization as a Senior Data Engineer. As a key member of our data team, you will be responsible for designing, building, and optimizing scalable data pipelines using cloud-native technologies.Key Responsibilities:Design and architect high-throughput ETL pipelines...

  • Senior Data Scientist

    2 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Scientist - Scalable Data PipelinesJob Title: Senior Data ScientistRole Overview:We are seeking a highly skilled and experienced Senior Data Scientist to lead the design, development, and maintenance of scalable data pipelines that process and transform large volumes of structured and unstructured data. The ideal candidate will have a strong...