Data Ingestion Specialist

3 days ago

Anand, Gujarat, India beBeeDataIngestion Full time US$ 1,00,000 - US$ 1,25,000

Job Title: Data Ingestion Specialist

Description: We are seeking an experienced professional to design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks. The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient ingestion workflows.

Responsibilities:

  • Design and Develop Data Pipelines: Create efficient and scalable data pipelines to integrate various data sources into Databricks.
  • Implement CI/CD Pipelines: Implement and maintain continuous integration and continuous deployment (CI/CD) pipelines for data workflows.
  • Deploy Containerized Applications: Deploy and manage containerized applications using Kubernetes.
  • Data Integrity and Security: Ensure data integrity, security, and compliance throughout the ingestion process.
  • Collaboration: Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies.
  • Troubleshooting: Troubleshoot and optimize ingestion pipelines for performance and scalability.

Required Skills & Qualifications:

  • Data Ingestion Experience: Proven experience in data ingestion and pipeline development.
  • CID Tools: Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps, etc.).
  • Kubernetes Expertise: Strong knowledge of Kubernetes and container orchestration.
  • Databricks Experience: Experience with Databricks, Spark, and data lake architectures.
  • Programming Languages: Proficiency in Python, Scala, or SQL for data processing.
  • Cloud Platforms: Familiarity with cloud platforms such as AWS, Azure, or GCP.
  • Problem-Solving: Strong problem-solving and analytical skills.

Preferred Qualifications:

  • Infrastructure as Code: Experience with Infrastructure as Code (Terraform, Helm, etc.).

  • Azure Databricks Developer for Data Ingestion

    7 days ago

    Anand, Gujarat, India beBeeExpertise Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job TitleAzure Databricks Developer Needed for Data Ingestion ExpertiseAbout the Role:We are seeking a skilled Azure Databricks engineer to lead data ingestion efforts, design, develop, and optimize pipelines for integrating multiple sources into Databricks. The ideal candidate will have expertise in CI/CD and Kubernetes.Key Responsibilities:Data Ingestion...

  • Data Engineering Specialist

    5 days ago

    Anand, Gujarat, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Title: Data Engineering SpecialistAbout the Role:This position involves designing data pipelines using AWS Glue, PySpark, and Step Functions. The ideal candidate should have a solid understanding of data engineering principles and be able to implement ETL/ELT processes for ingesting, transforming, and delivering data from multiple sources.Key...

  • Data Engineering Specialist

    1 week ago

    Anand, Gujarat, India beBeeEngineering Full time ₹ 18,00,000 - ₹ 24,00,000

    The primary function of a Data Engineering Specialist involves designing and implementing efficient data pipelines using industry-standard tools.Key Responsibilities include:Designing, developing, and maintaining ETL processes for ingestion from various sourcesCreating optimized data flows for loading curated datasets into cloud-based data...

  • Data Governance Specialist

    5 days ago

    Anand, Gujarat, India beBeeCollaborator Full time ₹ 18,00,000 - ₹ 25,00,000

    We are seeking a highly skilled Data Governance Specialist to join our team.Hands-on experience with Collibra Data Intelligence Platform is required. This includes:Metadata ingestionData lineage stitchingWorkflow configuration and customizationStrong understanding of metadata management and data governance principlesExperience working with Teradata, Tableau,...

  • Data Architect Specialist

    1 day ago

    Anand, Gujarat, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job OverviewA data architect with experience in cloud technologies and digital channel activities is required to work on projects involving the integration and processing of channel activity data from multiple sources.Develop robust data pipelines for processing channel activity data, focusing on digital channels and integrating multiple sources of data such...

  • Chief Data Infrastructure Specialist

    6 days ago

    Anand, Gujarat, India beBeeDataEngineer Full time US$ 1,50,000 - US$ 2,00,000

    Job TitleWe are seeking an experienced Data Engineer to join our team. The ideal candidate will have a strong background in building and maintaining production-grade data pipelines.Key Responsibilities:Design, build, and operate scalable, event-driven data pipelines that support real-time clinical workflows.Integrate with EHR/EMR systems (e.g., Epic) using...

  • Data Pipeline Specialist

    5 days ago

    Anand, Gujarat, India beBeeDataQuality Full time ₹ 18,00,000 - ₹ 23,00,000

    About Data Quality EngineersWe are looking for skilled professionals to join our team of Data Quality Engineers.Data Quality Engineers play a vital role in ensuring the accuracy and reliability of our clients' data pipelines.Key Responsibilities:Develop and execute automated tests to identify and resolve data quality issues.Requirements:Hands-on experience...

  • Chief Data Pipeline Architect

    7 days ago

    Anand, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,10,000

    Data Engineer Role Summary"> \We are seeking a Data Engineer to design, build and maintain scalable data pipelines for data ingestion, processing and storage.\

  • Data Pipeline Specialist

    6 days ago

    Anand, Gujarat, India beBeedataengineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Software EngineerJob Description:Develop and optimize data pipelines using Azure Data Factory, Databricks, and Synapse Analytics.Design scalable data architectures following best practices.Perform data ingestion, transformation, and orchestration across structured and unstructured datasets.Key Responsibilities:Develop and maintain data lake solutions...

  • Senior Data Development Specialist

    1 day ago

    Anand, Gujarat, India beBeeDataDevelopment Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Job Overview:Data Architects design and develop scalable data systems to meet business needs. In this role, you will be responsible for creating ETL/ELT pipelines using task automation, streams, and stored procedures.Key Responsibilities:Design and implement optimized data models with clustering keysImplement real-time ingestion and external table...