Data Pipeline Specialist

4 days ago

Kollam, Kerala, India beBeeCloud Remote Work Freelance Full time ₹ 1,50,00,000 - ₹ 2,50,00,000
Cloud Computing Engineer

We are seeking a skilled Cloud Computing Engineer to design and implement cloud-based solutions using Azure and Google Cloud platforms.

The ideal candidate will have expertise in automating data pipelines to facilitate efficient data flow and processing. They will also develop and manage data pipelines using technologies like PySpark and Databricks, ensuring the scalability and reliability of data processing workflows.

In addition, the candidate will work with Large Language Models (LLMs) such as GPT and OpenAI models to enhance applications with natural language processing (NLP) capabilities. They will design and implement prompt engineering strategies to optimize model performance in business applications.

The ideal candidate will have a Bachelor or Master degree and +3 years of hands-on experience in Python, Pyspark, Databricks, and cloud platforms like Azure and Google Cloud.

Key responsibilities include:

  • Designing and implementing cloud-based solutions using Azure and Google Cloud platforms;
  • Automating data pipelines to facilitate efficient data flow and processing;
  • Developing and managing data pipelines using technologies like PySpark and Databricks;
  • Ensuring the scalability and reliability of data processing workflows;
  • Working with Large Language Models (LLMs) such as GPT and OpenAI models to enhance applications with natural language processing (NLP) capabilities;

Requirements:

  • Bachelor or Master degree;
  • +3 years of hands-on experience in Python, Pyspark, Databricks, and cloud platforms like Azure and Google Cloud;
  • Experience in designing and implementing cloud-based solutions and automating data pipelines;
  • Experience in working with Large Language Models (LLMs) such as GPT, OpenAI models, or enhancing solutions with natural language processing (NLP) capabilities;
  • Proven ability in metadata management and schema mapping;
  • Cloud certifications (Azure, Google Cloud, or similar) are a plus;

We offer a competitive salary and benefits package to the successful candidate.

  • Senior Data Pipeline Specialist

    3 days ago

    Kollam, Kerala, India beBeeDataSpecialist Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Pipeline SpecialistWe are seeking an experienced Senior Data Pipeline Specialist to build and maintain production-grade data pipelines at scale using AWS services.The ideal candidate has 3–5 years of professional experience in designing and implementing scalable data architectures.About the Role:We need someone with hands-on expertise in...

  • Senior Data Pipeline Architect

    5 days ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will design, develop, and maintain robust data pipelines, data warehouses, and analytics platforms that serve as the backbone of our digital marketing product development.You will collaborate with cross-functional teams including Data Scientists, Product Managers,...

  • Data Pipeline Developer

    16 hours ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 19

    Senior data engineers design and develop complex data pipelines using Airflow and Python. The role involves creating efficient workflows that meet business needs.Key Responsibilities:Develop, maintain, and optimize data workflowsCollaborate with cross-functional teams to integrate data pipelinesWrite high-quality code in PythonThe ideal candidate has...

  • High-Performance Data Pipeline Specialist

    4 days ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    We are seeking an experienced Data Engineer to fill a critical role in our organization. This position will be responsible for designing and implementing efficient data pipelines using leading tools such as FiveTran, DBT Cloud, and custom frameworks.Key Responsibilities:Develop and maintain scalable data pipelines to meet business needsImplement data...

  • Head of Data Pipeline Optimization

    5 days ago

    Kollam, Kerala, India beBeeSeniorDataEngineer Full time ₹ 30,00,000 - ₹ 40,00,000

    Transform Data Infrastructure Our company is seeking a skilled Senior Data Engineer to lead the optimization and scalability of data delivery pipelines. This critical role will design, build, and maintain highly scalable data pipelines, optimize data delivery, and automate data processes. 

  • Data Pipeline Architect

    16 hours ago

    Kollam, Kerala, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 35,00,000

    Senior Data Engineer RoleThis senior data engineer position involves working with cross-functional teams to design, develop, and maintain large-scale data pipelines. The ideal candidate will have a strong background in data engineering, excellent communication skills, and the ability to collaborate effectively.Key ResponsibilitiesDevelop and implement ETL...

  • Data Architect Specialist

    7 days ago

    Kollam, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data Architect SpecialistWe are seeking an experienced Data Architect Specialist to join our team.The ideal candidate will have a strong background in data engineering and architecture, with expertise in designing and developing scalable data pipelines, architectures, and models.Responsibilities:Design and Develop Data Solutions: Design, develop, and...

  • Data Scientist/ML Engineer

    1 week ago

    Kollam, Kerala, India Quant-data Full time

    We're Hiring: Machine Learning Engineer / Data Engineer (Remote | Full-Time) Build AI-powered credit decisioning systems on Microsoft AzureWe're looking for a Machine Learning Engineer / Data Engineer with 5+ years of experience to join our AI-driven credit lending platform team. In this role, you'll design and deploy scalable ML solutions that power loan...

  • Expert Data Integration Specialist

    4 days ago

    Kollam, Kerala, India beBeeAzure Full time ₹ 9,00,000 - ₹ 12,00,000

    Job DescriptionThreatXIntel is a startup cyber security company dedicated to protecting businesses and organizations from cyber threats.Role DescriptionFreelance Azure Data Factory (ADF) Specialists are required to support data integration and ETL needs. The ideal consultant will have strong hands-on expertise in Azure ADF pipelines, data orchestration, and...

  • Senior Data Integration Specialist

    1 week ago

    Kollam, Kerala, India beBeeDataSpecialist Full time ₹ 30,00,000 - ₹ 50,00,000

    Job DescriptionWe are seeking an experienced Data Integration Specialist to develop and maintain scalable data pipelines using IBM DataStage, AWS Glue, and Snowflake. This role will involve designing, building, and optimizing ETL processes for loading curated datasets into a cloud-based data warehouse.The ideal candidate will have hands-on experience with...