Databricks Pipeline Developer

4 days ago

Kannur, Kerala, India beBeePipeline Full time ₹ 15,00,000 - ₹ 25,00,000

Job Title: Databricks Pipeline Developer

 

We seek an experienced Databricks pipeline developer to design and implement robust data ingestion pipelines for integrating multiple sources into Databricks.

Key Responsibilities:

  • Develop high-performance data ingestion pipelines for integrating multiple sources into Databricks.
  • Implement and maintain continuous integration and continuous deployment (CI/CD) pipelines for data workflows.
  • Deploy and manage containerized applications using Kubernetes orchestration.
  • Ensure data integrity, security, and compliance throughout the ingestion process.
  • Collaborate with cross-functional teams to optimize data ingestion strategies.
  • Troubleshoot and optimize ingestion pipelines for performance and scalability.

Required Skills & Qualifications:

  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps).
  • Strong knowledge of Kubernetes and container orchestration.
  • Experience with Databricks, Spark, and data lake architectures.
  • Proficiency in Python, Scala, or SQL for data processing.
  • Familiarity with cloud platforms such as AWS, Azure, or GCP.

Preferred Qualifications:

  • Experience with infrastructure as code (Terraform, Helm).
  • Background in streaming data ingestion (Kafka, Kinesis).
  • Knowledge of data governance and security best practices.

 

Why Choose This Role:

  • Opportunity to work on complex data engineering projects.
  • Collaborative team environment.
  • Ongoing training and professional development opportunities.

 

How to Apply:

Send your resume and a brief cover letter explaining why you are a fit for this role.

  • Data Ingestion Developer

    7 days ago

    Kannur, Kerala, India beBeeIngestion Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Description:We are seeking a skilled Data Ingestion Specialist to oversee the design, development and optimization of data ingestion pipelines.Key Responsibilities:Analyzing and refining data ingestion processes for integrating multiple sources into Databricks.Implementing and maintaining CI/CD pipelines for data workflows using tools like GitHub Actions...

  • Data Architect

    2 days ago

    Kannur, Kerala, India beBeeDatabricks Full time ₹ 20,00,000 - ₹ 25,00,000

    Job OverviewWe are seeking a seasoned Databricks PySpark Developer to join our data platform implementation team.This role is responsible for designing, developing, and maintaining ETL processes to ensure efficient extraction, transformation, and loading of data from various sources into our data lake and data warehouse.Key ResponsibilitiesETL...

  • Freelance Data Expert – Big Data Analytics Specialist

    23 hours ago

    Kannur, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    ThreatXIntel is a startup cyber security company that offers customized solutions to protect businesses from cyber threats.We are seeking an experienced Freelance Data Engineer with expertise in data pipelines design and development.Design and develop ETL/ELT pipelines using PySpark, AWS, and Databricks for large-scale data integration.Perform data...

  • Senior ETL Developer for Data Pipeline Development

    2 days ago

    Kannur, Kerala, India beBeeDataPipeline Full time ₹ 18,00,000 - ₹ 21,00,000

    Job Description:">Seeking a skilled ETL Developer to design, develop and maintain scalable data pipelines using IBM DataStage and AWS Glue. The ideal candidate will have experience working with Snowflake and proficient in SQL, Unix scripting and Python.Requirements:">4+ years of experience in ETL development with at least 1–2 years in IBM...

  • Highly Skilled Cloud Data Engineer Wanted

    3 days ago

    Kannur, Kerala, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Cloud Data Engineer PositionWe are seeking a highly skilled Cloud Data Engineer with strong hands-on experience in Databricks and AWS to join our dynamic data engineering team. The ideal candidate will be well-versed in PySpark, CI/CD pipelines, and Terraform for infrastructure as code.Key Responsibilities:Design, develop, and maintain scalable and...

  • Data Architect

    4 days ago

    Kannur, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Solutions EngineerWe are seeking a highly skilled Data Solutions Engineer to design and implement scalable, secure, and efficient data platforms using Databricks.About the Role:This is an excellent opportunity for a talented data professional to join our team and work on cutting-edge data projects. As a Data Solutions Engineer, you will be...

  • Cloud-Based AI Engineer

    3 days ago

    Kannur, Kerala, India beBeeAI Full time ₹ 25,00,000 - ₹ 35,00,000

    Job Description:We are seeking a highly skilled Azure AI_ML SRE to join our team. The ideal candidate will have strong knowledge of SRE principles, Databricks, and Terraform.As an experienced Azure AI_ML SRE, you will work closely with cross-functional teams to ensure the reliability, scalability, and performance of our cloud-based AI/ML applications on...

  • Senior Data Solutions Architect

    2 days ago

    Kannur, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job TitleWe are seeking an experienced Senior Software Engineer/Technical Specialist to join our organization. As a key member of our technology department, you will be responsible for designing and implementing scalable data architectures, developing and optimizing ETL/ELT pipelines, and ensuring data governance, security, and compliance in Azure-based data...

  • Cloud Scalable Data Solutions Specialist

    2 days ago

    Kannur, Kerala, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,90,000

    Job Overview:We are seeking a self-driven Data Engineer with 4–7 years of experience to build and optimize scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake.Work collaboratively with cross-functional teams to develop high-quality database programming requirements.Possess extensive knowledge in Azure cloud platforms like Advanced...

  • Ingestion Engineer

    6 days ago

    Kannur, Kerala, India Saxon Global Full time

    Hi Hope you are doing good todayKindly go through the JD and let me know your interest you can reach me at SAI.K@SAXONGLOBAL.COMTitle: Ingestion Engineer Opportunity Available for Immediate Joining Location - Anywhere from India TIMINGS - 10.00 AM IST to 7.00 PM ISTNeed good comm skills – they will be coordinating with Americans on daily basesWe are...