Data Ingestion Specialist

2 days ago

Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 10,00,000 - ₹ 25,00,000
Job Description

We are seeking an experienced Data Integration Specialist to join our team. This role requires a strong understanding of data ingestion processes, focusing on integrating various data sources into Databricks.

The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient ingestion workflows.

This includes designing, developing, and optimizing data ingestion pipelines for integrating multiple sources into Databricks, implementing and maintaining CI/CD pipelines for data workflows, deploying and managing containerized applications using Kubernetes, ensuring data integrity, security, and compliance throughout the ingestion process, and collaborating with Data Engineers and other stakeholders to streamline data ingestion strategies.

Key Responsibilities:
  • Design and develop data ingestion pipelines for integrating multiple sources into Databricks.
  • Implement and maintain continuous integration and continuous delivery pipelines for data workflows.
  • Deploy and manage containerized applications using Kubernetes.
  • Evaluate and ensure data integrity, security, and compliance throughout the ingestion process.
Required Skills & Qualifications:
  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with continuous integration and continuous delivery tools.
  • Strong knowledge of container orchestration using Kubernetes.
  • Experience with Databricks, Spark, and data lake architectures.
  • Proficiency in programming languages such as Python, Scala, or SQL.
  • Familiarity with cloud platforms such as AWS, Azure, or GCP.
Benefits

This position offers a competitive compensation package including a comprehensive benefits program, opportunities for professional growth and development, and a collaborative work environment.

  • Data Ingestion Specialist

    1 week ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Description">As an Ingestion Engineer, you will play a vital role in crafting data ingestion pipelines to integrate multiple sources into Databricks. This involves designing, developing, and optimizing these pipelines for seamless data integration.Key aspects of this position include collaborating with Data Engineers and other stakeholders to streamline...

  • Data Ingestion Specialist

    6 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineering Full time US$ 70,000 - US$ 1,23,456

    Job Title: Azure Databricks Engineer with Injection ExperienceThis role involves the integration of diverse data sources into a scalable and efficient ingestion workflow. It requires expertise in CI/CD and Kubernetes to design, implement, and manage data pipelines that automate and support infrastructure management.The ideal candidate should possess hands-on...

  • Data Quality Specialist

    3 hours ago

    Vijayawada, Andhra Pradesh, India beBeeDataQuality Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Quality SpecialistOverview:The Data Quality Specialist is responsible for ensuring the accuracy and reliability of our data pipelines. This role requires a deep understanding of data quality principles and experience with testing tools.Key Responsibilities:Develop and execute tests using SQL, Tricentis, Python (PySpark), and data quality...

  • Data Validation Specialist

    4 days ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 12,00,000 - ₹ 20,00,000

    Job Title:Data Validation SpecialistAbout the RoleWe are looking for an experienced data validation specialist to join our team.Key ResponsibilitiesDesign and execute test cases to validate data transformations, ingestion, and processing logic.Proficiency in validating data across various layers using Databricks.Proficiency in writing complex SQL...

  • Data Solutions Specialist

    4 days ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 25,00,000 - ₹ 35,00,000

    Job TitleWe are seeking a highly skilled Data Solutions Specialist to drive the design and development of scalable big data architectures.Design and develop distributed data systems using Spark and Hive technologies.Collaborate with cross-functional teams to integrate data solutions into existing systems.Develop and maintain data pipelines for ingestion,...

  • Data Science Specialist

    6 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataScience Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data Science SpecialistWe are seeking a skilled Data Science Specialist to join our team and contribute to the development of innovative AI solutions.This is an exceptional opportunity to work on international projects, collaborate with cross-functional teams, and leverage cutting-edge AI research and tools.The ideal candidate will have expertise...

  • Data Engineer

    5 days ago

    Vijayawada, Andhra Pradesh, India beBeeIngestion Full time ₹ 15,00,000 - ₹ 20,00,000

    Ingestion engineers are crucial for our organization's data strategy, responsible for crafting efficient data ingestion pipelines that integrate multiple sources into Databricks.Key Responsibilities:Data Ingestion Pipelines: Develop and optimize data ingestion pipelines for integrating multiple sources into Databricks.CI/CD Pipeline Implementation: Implement...

  • Senior Data Engineer

    4 days ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    About the RoleAt our organization, we are seeking a skilled Senior Software Engineer/Technical Specialist to join our team. This role is ideal for an individual with expertise in data engineering, including experience with Azure Data Factory (ADF), Databricks, and Synapse Analytics.As a Senior Software Engineer/Technical Specialist, you will be responsible...

  • Senior Data Engineer Specialist

    5 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe're seeking a skilled professional to design, develop, and maintain large-scale data pipelines. Main Responsibilities:Data Pipeline Development: Design and develop ETL/ELT data pipelines to ingest data from various sources into our data warehouse.Cloud Services: Utilize AWS services for data ingestion and transformation processes.Database...

  • Cloud Data Architect

    9 hours ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 1,50,00,000

    GCP Data Engineer RoleWe are seeking a skilled GCP Data Engineer to join our team.The ideal candidate will have expertise in GCP and BigQuery for real-time analytics, as well as experience with data storage, ingestion, and processing.Key Responsibilities:Design, develop, and maintain large-scale data pipelines using GCP services.Analyze and process complex...