Data Pipeline Developer

1 week ago


Palakkad, Kerala, India beBeeIngestion Full time ₹ 15,00,000 - ₹ 25,00,000

Job Opportunity

We are seeking an Ingestion Specialist with expertise in data pipeline development and Kubernetes to fill a key role in our organization.

This position requires a strong understanding of data ingestion processes, focusing on integrating various data sources into Databricks.

The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient ingestion workflows.

Key Responsibilities:

  • Design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks.
  • Implement and maintain CI/CD pipelines for data workflows.
  • Deploy and manage containerized applications using Kubernetes.
  • Ensure data integrity, security, and compliance throughout the ingestion process.
  • Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies.
  • Troubleshoot and optimize ingestion pipelines for performance and scalability.

Required Skills & Qualifications:

  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps, etc.).
  • Strong knowledge of Kubernetes and container orchestration.
  • Experience with Databricks, Spark, and data lake architectures.
  • Proficiency in Python, Scala, or SQL for data processing.
  • Familiarity with cloud platforms such as AWS, Azure, or GCP.
  • Strong problem-solving and analytical skills.


  • Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking a skilled Data Engineer to join our team in Mumbai. The ideal candidate will have strong experience in PySpark and hands-on expertise in building scalable data pipelines.The successful candidate will work on designing, developing, and maintaining batch and streaming data pipelines using Spark SQL, DataFrame, and RDD APIs. They...


  • Palakkad, Kerala, India beBeeETLDeveloper Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    We are seeking a skilled professional to fill the role of an ETL Specialist with expertise in designing and optimizing complex data pipelines using Talend Open Studio.Key qualifications for this position include 8 to 10 years of experience working with large datasets, proficiency in SQL and Python programming languages, as well as hands-on experience with...


  • Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Job Title:A Data Engineer is needed to design, build, and maintain large-scale data pipelines using Google Cloud Platform.Responsibilities:Develop, construct, test, and maintain data acquisition pipelines for structured and unstructured data in GCP.Create complex datasets based on business requirements.Construct 'big data' pipeline architecture.Identify...


  • Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineer Role OverviewSeeking a skilled data engineer to design, develop, and maintain robust data pipelines. Responsibilities include extracting data from Oracle Symphony via APIs, processing and storing it in the Databricks Lakehouse platform, and integrating it into Oracle EPM.Key Responsibilities:Design and implement end-to-end data pipelines from...


  • Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Engineering ProfessionalWe are seeking a skilled data engineer to join our analytics division.You will be part of our data analytics and business intelligence vertical, responsible for conceptualizing and owning the build out of problem-solving data marts for consumption by data science and business intelligence teams. You will evaluate design and...


  • Palakkad, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking a highly skilled professional to join our team as a Big Data Engineer. In this role, you will be responsible for designing, building, and maintaining robust data pipelines that handle high-volume financial data.The ideal candidate will have experience with data pipeline tools, cloud platforms, and data warehousing. They will...


  • Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 19,00,000

    We are seeking a highly skilled Data Engineer to lead the design and development of scalable data pipelines. The ideal candidate will have expertise in data engineering principles, with a strong understanding of cloud-based data platforms.Key Responsibilities:Design and develop efficient data pipelines using Airflow and SnowflakeCollaborate with...


  • Palakkad, Kerala, India beBeeETL Full time ₹ 18,00,000 - ₹ 25,40,000

    Job OpportunitySeeking a highly skilled ETL Developer to design, develop, and maintain data pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources.As an experienced ETL professional, you will be responsible for building and optimizing data flows for loading curated datasets into Snowflake, leveraging best practices for...


  • Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Cloud Data Engineer PositionLemongrass is seeking a skilled Cloud Data Engineer to design, develop, and maintain scalable data pipelines using cloud-native tools. The ideal candidate will have extensive experience with AWS, Azure, and GCP, as well as knowledge of data governance, security, and compliance best practices.">Key Responsibilities:Design, develop,...


  • Palakkad, Kerala, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Key Responsibilities:We are seeking a skilled Data Architect to develop and maintain data pipelines using Python and AWS (S3, Redshift, Lambda, Glue).Develop scalable data architectures using AWS services.Implement and optimize ETL processes for large datasets using Python and AWS Glue.Collaborate with cross-functional teams to design and implement data...