Senior Data Pipeline Specialist

1 week ago

Palakkad, Kerala, India beBeeBigDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

Job Overview

We are seeking a highly skilled Data Engineer to join our team. The ideal candidate will have strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka.

Key Responsibilities:

  • Design, develop, and optimize large-scale data pipelines and distributed data processing systems.
  • Work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.
  • Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.
  • Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.
  • Collaborate with cross-functional teams to design scalable cloud-based data architectures.
  • Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.
  • Build monitoring and alerting systems leveraging Splunk or equivalent tools.
  • Participate in CI/CD workflows using Git, Jenkins, and other DevOps tools.
  • Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills:

  • Scala – Minimum 3+ years of hands-on experience.
  • Strong expertise in Spark (PySpark) and Python.
  • Hands-on experience with Apache Kafka.
  • Knowledge of NiFi / Airflow for orchestration.
  • Strong experience in Distributed Data Systems (5+ years).
  • Proficiency in SQL and query optimization.
  • Good understanding of Cloud Architecture.

Preferred Skills:

  • Exposure to messaging technologies like Apache Kafka or equivalent.
  • Experience in designing intuitive, responsive UIs for data analytics visualization.
  • Familiarity with Splunk or other monitoring/alerting solutions.
  • Hands-on experience with CI/CD tools (Git, Jenkins).
  • Strong grasp of software engineering concepts, data modeling, and optimization techniques.

  • Senior Big Data Specialist

    2 weeks ago

    Palakkad, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Big Data SpecialistDescriptionWe are seeking a highly skilled Senior Big Data Specialist to join our team. The ideal candidate will have a minimum of 3-5 years of experience in designing and implementing data pipelines using various technologies.The successful candidate will be responsible for developing and maintaining large-scale data...

  • Senior Data Pipeline Specialist

    2 weeks ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Cloud Data Engineer PositionLemongrass is seeking a skilled Cloud Data Engineer to design, develop, and maintain scalable data pipelines using cloud-native tools. The ideal candidate will have extensive experience with AWS, Azure, and GCP, as well as knowledge of data governance, security, and compliance best practices.">Key Responsibilities:Design, develop,...

  • Data Pipeline Specialist

    2 weeks ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Engineering ProfessionalWe are seeking a skilled data engineer to join our analytics division.You will be part of our data analytics and business intelligence vertical, responsible for conceptualizing and owning the build out of problem-solving data marts for consumption by data science and business intelligence teams. You will evaluate design and...

  • Senior Data Engineering Specialist

    1 week ago

    Palakkad, Kerala, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data Engineering SpecialistDescription:Our organization is seeking a highly skilled data engineer to lead our data engineering practice and drive initiatives to develop and continuously improve the Data/AI Practice.Key Responsibilities:Develop, implement, and optimize data pipelines for large-scale structured and unstructured datasets using...

  • Chief Data Pipeline Architect

    2 weeks ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineer Role OverviewSeeking a skilled data engineer to design, develop, and maintain robust data pipelines. Responsibilities include extracting data from Oracle Symphony via APIs, processing and storing it in the Databricks Lakehouse platform, and integrating it into Oracle EPM.Key Responsibilities:Design and implement end-to-end data pipelines from...

  • Senior Data Pipeline Developer

    1 week ago

    Palakkad, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data Engineer PositionWe are seeking a skilled Data Engineer to join our team. As a key member of the data team, you will be responsible for designing and developing scalable data pipelines using Google Cloud Platform (GCP) tools.This is a remote opportunity that requires 10+ years of experience in data engineering with a focus on real-time and big query...

  • Senior Data Protection Specialist

    1 week ago

    Palakkad, Kerala, India beBeeDataProtection Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data Protection SpecialistAbout the Role:We are seeking an experienced data protection professional to join our team as a senior data protection specialist. The role involves leading and executing data protection engagements, ensuring compliance with global data protection regulations, and embedding data protection by design into client...

  • Expert Data Pipeline Architect

    2 weeks ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Job Title:A Data Engineer is needed to design, build, and maintain large-scale data pipelines using Google Cloud Platform.Responsibilities:Develop, construct, test, and maintain data acquisition pipelines for structured and unstructured data in GCP.Create complex datasets based on business requirements.Construct 'big data' pipeline architecture.Identify...

  • Data Engineer for Scalable Data Pipelines

    2 weeks ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 19,00,000

    We are seeking a highly skilled Data Engineer to lead the design and development of scalable data pipelines. The ideal candidate will have expertise in data engineering principles, with a strong understanding of cloud-based data platforms.Key Responsibilities:Design and develop efficient data pipelines using Airflow and SnowflakeCollaborate with...

  • Data Engineer Leader

    2 weeks ago

    Palakkad, Kerala, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineering Specialist Job Title: Senior Data Engineering SpecialistExperience Level: 5+ YearsKey Responsibilities:Develop and optimize ETL/ELT pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics.Design and implement scalable data architectures following Medallion Architecture and best practices.Perform data ingestion,...