Senior Data Ingestion Specialist

5 days ago

Malappuram, Kerala, India beBeeDataIngestion Full time ₹ 10,00,000 - ₹ 12,00,000
Job Title:

Azure Databricks Engineer with Injection Experience

Description:

This position requires a deep understanding of data ingestion processes, focusing on integrating various data sources into Azure Databricks. The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient ingestion workflows.

The successful candidate will design, develop, and optimize data ingestion pipelines for integrating multiple sources into Azure Databricks. They will implement and maintain Continuous Integration/Continuous Deployment (CI/CD) pipelines for data workflows and deploy and manage containerized applications using Kubernetes.

Key Responsibilities:

  • Design, develop, and optimize data ingestion pipelines for integrating multiple sources into Azure Databricks.
  • Implement and maintain CI/CD pipelines for data workflows.
  • Deploy and manage containerized applications using Kubernetes.
  • Ensure data integrity, security, and compliance throughout the ingestion process.
  • Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies.
  • Troubleshoot and optimize ingestion pipelines for performance and scalability.

Required Skills & Qualifications:

  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps, etc.).
  • Strong knowledge of Kubernetes and container orchestration.
  • Experience with Azure Databricks, Spark, and data lake architectures.
  • Proficiency in Python, Scala, or SQL for data processing.
  • Familiarity with cloud platforms such as AWS, Azure, or GCP.
  • Strong problem-solving and analytical skills.

Preferred Qualifications:

  • Experience with Infrastructure as Code (Terraform, Helm, etc.).
  • Background in streaming data ingestion (Kafka, Kinesis, etc.).
  • Knowledge of data governance and security best practices.

  • Data Quality Specialist

    3 days ago

    Malappuram, Kerala, India beBeeDataQuality Full time ₹ 18,00,000 - ₹ 25,00,000

    As a skilled data quality specialist, you will be responsible for ensuring the accuracy and integrity of our data assets.Key responsibilities include developing and executing tests for data pipelines, ETL processes, and ingestion testing. Additionally, you will work closely with cross-functional teams to identify and resolve data quality...

  • Senior Cloud Data Engineer

    3 days ago

    Malappuram, Kerala, India beBeeGlueDeveloper Full time ₹ 10,00,000 - ₹ 15,00,000

    Job Title - Senior AWS Glue DeveloperRole Overview:We are looking for a skilled AWS Glue developer to join our team.The ideal candidate will have experience in serverless architecture and the AWS ecosystem of services, specifically in data ingestion and transformation.Key Responsibilities:Design and implement data pipelines using AWS Glue, ensuring seamless...

  • Senior Data Visualization Specialist

    5 days ago

    Malappuram, Kerala, India beBeeSplunk Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Title:Location: Remote (US Timings)Contract Duration: 1 YearExperience Required: 6–7 YearsJob Summary:Expert Splunk developers with strong expertise in developing, configuring, and maintaining environments to support monitoring, alerting, and advanced analytics are sought for long-term contracts. The ideal candidate will have the ability to design,...

  • Senior Data Infrastructure Professional

    4 days ago

    Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    About the Role:","We are seeking an experienced Data Pipeline Specialist with expertise in building and integrating healthcare data.","This critical role will create scalable pipelines that process clinical encounter data, retrieve associated medical documents, and deliver validated information to our machine learning systems for inference.","","Design and...

  • Senior Data Systems Specialist

    3 days ago

    Malappuram, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Lead Data ArchitectOverview:We seek a highly skilled and motivated Lead Data Architect to design, develop, and maintain large-scale data systems. This role requires strong expertise in data architecture, cloud technologies, and software engineering principles.Key Responsibilities:Design and implement robust data architectures for processing structured and...

  • Senior Data Pipeline Specialist

    3 days ago

    Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 35,00,000

    Are you a data expert looking to leverage your skills in designing and building robust data pipelines?We are seeking an experienced Azure Data Engineer to join our team.The ideal candidate will have strong SQL skills, experience with Fabric, and proficiency in Python (PySpark).Additionally, they will have hands-on experience with Azure Data Services, CI/CD...

  • Senior Data Architecture Specialist

    2 days ago

    Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    About this PositionWe are seeking an experienced professional to design, build, and optimize data pipelines and architectures that empower analytics and data science initiatives.Key Responsibilities:Develop scalable data pipelines using Azure Data Factory and Azure Synapse Analytics.Create lakehouse architectures on Azure using Delta Lake.Collaborate with...

  • Data Governance Professional

    3 days ago

    Malappuram, Kerala, India beBeeDataGovernance Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title - Data Catalogue SpecialistWe are seeking a skilled professional to join our team as a Data Catalogue Specialist.The ideal candidate will have hands-on experience with Collibra Data Intelligence Platform, including metadata ingestion, data lineage stitching and workflow configuration and customization.Mandatory Skills:Hands-on experience with...

  • Senior Data Engineering Specialist

    3 days ago

    Malappuram, Kerala, India beBeedataengineering Full time ₹ 2,08,20,000 - ₹ 2,59,50,000

    **Data Engineering Leadership Opportunity**Our organization is seeking a seasoned Data Engineer to lead our data engineering initiatives. The ideal candidate will have a strong background in enterprise-level data engineering, with expertise in OLAP design and deep understanding of data warehousing concepts.We are looking for someone with 8+ years of...

  • Highly Skilled Data Architect

    22 hours ago

    Malappuram, Kerala, India beBeeDataEngineering Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Senior Data Engineer RoleWe are seeking a Senior Data Engineer to play a pivotal part in designing, developing, and optimizing data ingestion, transformation, and governance frameworks, enabling real-time and batch data analytics across our data platform.Design and implement scalable, robust, and secure data pipelines using Azure Data Factory, Databricks,...