Azure Databricks Data Ingestion Expert

1 day ago

Pune, Maharashtra, India beBeeDataIngestion Full time ₹ 15,00,000 - ₹ 25,00,000

Job Overview:

Azure Databricks Data Ingestion Specialist

We are seeking a highly skilled Data Ingestion Specialist with expertise in CI/CD and Kubernetes to join our team. This role requires a strong understanding of data ingestion processes, focusing on integrating various data sources into Databricks.

  • Design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks.
  • Implement and maintain CI/CD pipelines for data workflows.
  • Deploy and manage containerized applications using Kubernetes.
  • Ensure data integrity, security, and compliance throughout the ingestion process.
  • Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies.
  • Troubleshoot and optimize ingestion pipelines for performance and scalability.

Required Skills & Qualifications:

  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps, etc.).
  • Strong knowledge of Kubernetes and container orchestration.
  • Experience with Databricks, Spark, and data lake architectures.
  • Proficiency in Python, Scala, or SQL for data processing.
  • Familiarity with cloud platforms such as AWS, Azure, or GCP.
  • Strong problem-solving and analytical skills.

Preferred Qualifications:

  • Experience with Infrastructure as Code (Terraform, Helm, etc.).
  • Background in streaming data ingestion (Kafka, Kinesis, etc.).
  • Knowledge of data governance and security best practices.

What We Offer:

Competitive compensation and benefits package.

Opportunities for career growth and professional development.

A dynamic and collaborative work environment.

  • Azure Databricks Developer

    3 weeks ago

    Pune, Maharashtra, India People Prime World Wide Full time

    About Client :Our Client is a multinational IT services and consulting company headquartered in USA, With revenues 19.7 Billion USD, with Global work force of 3,50,000 and Listed in NASDAQ, It is one of the leading IT services firms globally, known for its work in digital transformation, technology consulting, and business process outsourcing, Business Focus...

  • Azure Databricks

    3 weeks ago

    Pune, Maharashtra, India Phygital Insights Private Limited Full time

    Job DescriptionDescriptionWe are seeking an experienced Azure Databricks professional to join our team in India. The ideal candidate will have a strong background in data engineering and be proficient in leveraging Azure Databricks to build robust data solutions.Responsibilities- Design and implement data pipelines using Azure Databricks to process large...

  • Azure Databricks

    4 weeks ago

    Pune, Maharashtra, India People Prime Worldwide Full time

    Our client is a global technology company headquartered in Santa Clara, California. it focuses on helping organisations harness the power of data to drive digital transformation, enhance operational efficiency, and achieve sustainability. over 100 years of experience in operational technology (OT) and more than 60 years in IT to unlock the power of data from...

  • ai - Big Data Architect - Azure Databricks

    3 weeks ago

    Pune, Maharashtra, India Codvo Full time

    Job Title : Big Data Architect / Databricks Architect.Company Overview :At Codvo, software and people transformations go hand-in-hand.We are a global empathy-led technology services company.Product innovation and mature software engineering are part of our core DNA.Respect, Fairness, Growth, Agility, and Inclusiveness are the core values that we aspire to...

  • Pyspark Azure Databricks

    4 weeks ago

    Pune, Maharashtra, India Tata Consultancy Services Full time

    Greetings from Tata Consulting ServicesTCS is Hiring for Pyspark DatabricksExperience : 6- 10 Years Location: Pune onlyPlease find the JD belowRequired Technical Skill - Pyspark DatabricksData Engineer with professional experience on Databricks, managing data pipelines and infrastructure within Databricks Lake house Platform.Proven expertise in Databricks,...

  • Data Architectuur specialist Databricks/Azure

    1 day ago

    Pune, Maharashtra, India beBeeDatabriksspecialist Full time ₹ 20,00,000 - ₹ 25,00,000

    Wij zoeken een expert in data engineering die ervaring heeft met Databricks en Azure. Deze professional zal onze organisatie helpen bij het ontwerpen van gegevensarchitecturen die de behoeften van klanten voldoen.Verantwoordelijkheden:Ontwerp en implementeer gegevensarchitecturen voor grote datasetsImplementeer data pipelines voor automatische...

  • Big Data Engineer

    5 hours ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,04,000 - ₹ 1,30,878

    Data Engineer PositionWe are seeking a highly skilled data engineer to spearhead the development of robust data solutions using Azure Databricks. The ideal candidate will possess a strong background in data engineering and be proficient in leveraging Apache Spark to build efficient data pipelines.Responsibilities include:Data Pipeline Development: Design,...

  • Data Engineer – Azure

    2 days ago

    Pune, Maharashtra, India vueverse. Full time

    Hybrid Role: Bengaluru/Hyderabad/Pune/ChennaiYears of experience: 8-10 yearsJob DescriptionWe are looking for a skilled Data Engineer with strong expertise in the Azure Data Platform to design, develop, and optimize large-scale data pipelines and analytics solutions. The ideal candidate should have deep hands-on experience with Azure Databricks, PySpark,...

  • Azure Databricks Architect

    4 weeks ago

    Pune, Maharashtra, India Persistent Systems Full time

    About Position:We are hiring for skilled Azure Databrick Architect with 12 to 17 Years of experience in python, sql.- Role: Azure Databricks Architect- Location: All Persistent Locations- Experience: 12 to 17 years- Job Type: Full Time EmploymentWhat You'll Do:- 12+ years of experience in data architecture, data engineering, or analytics.- 5+ years of...

  • GenAI/ML Engineer(Azure, Databricks)

    5 days ago

    Pune, Maharashtra, India Enterprise Minds, Inc Full time

    We're Hiring | AI/ML & Generative AI Developer Location: (Specify Location or Remote/Hybrid if applicable) Join our AI Innovation Team and help shape the future of AI-powered solutions We're looking for an experienced AI/ML & Generative AI Developer who's passionate about building cutting-edge Generative AI, LLM-powered applications, and Agentic AI...