Azure Databricks Data Ingestion Specialist

5 days ago

Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 2,00,00,000
Job Opportunity: Azure Databricks Engineer

We are seeking a highly skilled Azure Databricks Engineer with expertise in data ingestion and pipeline development. As an Ingestion Engineer, you will play a crucial role in designing, developing, and optimizing data ingestion pipelines for integrating multiple sources into Databricks.

Key Responsibilities:

  • Design and develop high-performing data ingestion pipelines that integrate seamlessly with Databricks.
  • Implement and maintain Continuous Integration/Continuous Deployment (CI/CD) pipelines for data workflows using GitHub Actions, Jenkins, or Azure DevOps.
  • Deploy and manage containerized applications using Kubernetes to ensure scalability and reliability.
  • Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies and optimize pipeline performance.

Requirements:

  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools and cloud platforms such as AWS, Azure, or GCP.
  • Strong knowledge of Kubernetes, container orchestration, and Databricks architecture.
  • Proficiency in Python, Scala, or SQL for data processing.
  • Familiarity with data governance and security best practices.

Benefits:

  • Opportunity to work on cutting-edge technology and contribute to the growth of our organization.
  • Collaborative and dynamic work environment.
  • Competitive salary and benefits package.

Additional Information:

  • Experience with Infrastructure as Code (Terraform, Helm, etc.) is a plus.
  • Background in streaming data ingestion (Kafka, Kinesis, etc.) is desirable.

  • Azure Data Engineering Specialist

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking a highly skilled professional with 4+ years of experience in Azure Data Engineering, who has strong expertise in Python, Scala, Spark (PySpark & Scala Spark), and Azure Cloud technologies.Key ResponsibilitiesData Ingestion and ETL PipelinesData Transformation WorkflowsAzure Cloud ServicesTroubleshooting and DebuggingPerformance...

  • Azure Data Engineer Leader

    5 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 22,00,000

    Unlock your expertise as a Lead Azure Data Engineer and propel our organization forward.We seek a seasoned professional with hands-on experience in Azure and Databricks to spearhead our data engineering efforts.This is a senior technical role that demands exceptional skills, strategic thinking, and the ability to deliver high-quality solutions...

  • Azure Data Engineering Specialist

    5 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job DescriptionWe are seeking a highly skilled Data Engineer to join our team. As a key member of our data engineering group, you will design, develop, and maintain large-scale data processing systems.In this role, you will work closely with cross-functional teams to identify business requirements and develop scalable solutions using Azure Databricks,...

  • Chief Data Pipeline Architect

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title:Azure Data EngineerJob Description:We are seeking a skilled data engineer to develop, optimize and maintain automated data pipelines supporting governance and analytics initiatives.This role involves designing, building and deploying ETL/ELT pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform and curate metadata from multiple...

  • AI Workflow Automation Specialist

    15 hours ago

    Tirupati, Andhra Pradesh, India beBeeAutomation Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: AI Workflow Automation SpecialistLead AI Engineer & Intermediate AI Engineer (Regulatory Workflow Automation)Openings: 1 Lead + 1 IntermediateLocation: Bangalore (Hybrid — 2–3 days/week onsite)Category: AI Engineering – Regulatory & Compliance AutomationAbout the teamWe're building AI-powered, automated workflows for compliance-heavy...

  • Engineering Manager – Big Data, Azure

    3 days ago

    Tirupati, Andhra Pradesh, India DataE2E Technologies Full time

    Technical Manager – AzureLocation: 100% RemoteEmployment Type: Full-timeShift Timing : 3 pm – 11 pm ISTJob Summary:About the Role:We are seeking a highly experienced and strategic Engineering Manager with deep expertise in Big Data platforms, Microsoft Azure, and Databricks.The ideal candidate will lead a team of talented data engineers to design and...

  • Data engineer

    1 week ago

    Tirupati, Andhra Pradesh, India Firstsource Full time

    Key Skills and Responsibilities:We are seeking a Senior Data Engineer with a strong background in cloud-native data engineering, primarily on Microsoft Azure, and familiarity with AWS and GCP. The ideal candidate will have deep expertise in building scalable data pipelines, implementing enterprise-grade data governance, security, and AI-powered engineering...

  • Senior Data Architecture Specialist

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job SummaryWe are seeking a highly skilled Data Architect to design and implement scalable data pipelines and lakehouse architectures on Azure.This is an exciting opportunity for a talented professional to work with our team in building cutting-edge solutions that drive business growth and innovation.Design, develop, and maintain scalable data pipelines...

  • Senior Cloud Data Engineering Professional

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Role OverviewWe are seeking a highly skilled professional to fill the position of Senior Data Engineer.About the JobThis role will play a pivotal part in designing, developing, and optimizing data ingestion, transformation, and governance frameworks, enabling real-time and batch data analytics across our data platform.The Ideal CandidateWill have deep...

  • Chief Data Solutions Architect

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 17,01,120 - ₹ 30,24,000

    Job OverviewThis role is responsible for designing and overseeing the architecture of Microsoft Fabric solutions, encompassing integration with Power BI, Synapse, Data Factory, and other Azure services.Main ResponsibilitiesCreate and direct the design of Microsoft Fabric solutions.Lead a team comprising data engineers, BI developers, and analysts in...