Databricks Engineer for Large-Scale Data Integration

6 days ago

Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

Job Title: Data Engineer with Databricks Experience

We are seeking an Ingestion Expert to drive the development and optimization of data ingestion pipelines for integrating multiple sources into Databricks.

The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient ingestion workflows.

Key Responsibilities:
  • Design, develop, and optimize data ingestion pipelines for integrating various data sources into Databricks.
  • Implement and maintain Continuous Integration/Continuous Deployment (CI/CD) pipelines for data workflows.
  • Deploy and manage containerized applications using Kubernetes.
  • Ensure data integrity, security, and compliance throughout the ingestion process.
  • Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies.
  • Troubleshoot and optimize ingestion pipelines for performance and scalability.

Required Skills & Qualifications:

  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps, etc.).
  • Strong knowledge of Kubernetes and container orchestration.
  • Experience with Databricks, Spark, and data lake architectures.
  • Proficiency in Python, Scala, or SQL for data processing.
  • Familiarity with cloud platforms such as AWS, Azure, or GCP.
  • Strong problem-solving and analytical skills.

Preferred Qualifications:

  • Experience with Infrastructure as Code (Terraform, Helm, etc.).
  • Background in streaming data ingestion (Kafka, Kinesis, etc.).
  • Knowledge of data governance and security best practices.

  • Databricks Data Engineer

    5 days ago

    Gulbarga, Karnataka, India beBeeDataEngineering Full time ₹ 18,14,400 - ₹ 25,12,000

    Databricks Data EngineerKey Responsibilities:Design, develop, and maintain scalable data pipelines on Databricks.Lead the architecture and implementation of high-performance data solutions using Databricks.Collaborate with business stakeholders and data scientists to deliver actionable data insights.Optimize data workflows for performance, cost-efficiency,...

  • Cloud Data Engineer

    4 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Cloud Data Engineering SpecialistWe are seeking an experienced Cloud Data Engineering Specialist to design, implement and maintain large-scale data processing pipelines.Main Responsibilities:Designing, implementing and maintaining data processing pipelines using Azure Data Factory (ADF), Azure Databricks (ADB), PySpark and SQL.Collaborating with...

  • Key Data Engineer Role

    5 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Description:A Data Engineer with expertise in Informatica DEI will be responsible for designing, developing, and maintaining large-scale data engineering solutions.Required Skills and Qualifications:We are looking for individuals with the following skills and qualifications:Strong hands-on experience in Informatica DEI (Data Engineering Integration)Basic...

  • Chief Data Engineering Specialist

    3 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 22,00,000

    Senior Data Engineer RoleJob Summary:We are seeking a skilled and experienced data engineer to lead the design and implementation of scalable, high-performance data pipelines using AWS cloud services and Databricks.Key Responsibilities:Design, develop, and maintain complex data pipelines and workflows utilizing AWS (S3, Redshift, Glue, EMR, Lambda, Kinesis,...

  • Freelance Data Engineering Specialist

    3 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking an experienced Freelance Data Engineer to design, build, and optimize large-scale data pipelines using PySpark, AWS, and Databricks.The ideal candidate will have expertise in Python, SQL, Linux/Bash, and a strong background in data engineering.Design, develop, and maintain ETL/ELT pipelines using PySpark, AWS, and...

  • Big Data Specialist

    3 days ago

    Gulbarga, Karnataka, India beBeeDataEngineering Full time ₹ 12,00,000 - ₹ 25,00,000

    Job Title: Data EngineerWe are seeking an experienced Data Engineer to join our team.A minimum of 3-6 years of experience in data engineering, preferably with a focus on large-scale data pipelines.Strong analytical and problem-solving skills, with the ability to analyze complex data sets and identify trends and insights.About the Role:Develop and maintain...

  • High-Performing Data Engineer

    4 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 12,00,000 - ₹ 18,00,000

    Data Pipeline ArchitectAs a Data Pipeline Architect, you will play a critical role in designing and implementing scalable data architectures using modern DevOps/configuration management tools.Key Responsibilities:Design, implement, and manage CI/CD pipelines using GitHub Actions or equivalent tools to ensure seamless code integration and deployment.Configure...

  • High-Performance Data Engineer

    6 days ago

    Gulbarga, Karnataka, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data Engineering LeadAbout the Position:We are seeking a seasoned data engineering expert to lead our team of talented professionals in designing and delivering high-performance, scalable data pipelines and analytics platforms on Microsoft Azure.Key Responsibilities:Mentor and grow a team of data engineers working on large-scale distributed data...

  • Large Scale Data Project Manager

    3 days ago

    Gulbarga, Karnataka, India beBeeProject Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Enterprise Project StrategistRole Overview:We are seeking an experienced Enterprise Project Strategist with strong expertise in managing large-scale projects within complex data environments.The role involves delivering complex technical projects on time, ensuring client satisfaction and seamless execution. As an Enterprise Project Strategist, you...

  • Data Engineer

    4 weeks ago

    Gulbarga, Karnataka, India BL Consultants Full time

    We're Hiring | Data EngineersLooking for an exciting role where you can build, scale, and innovate?We're hiring for one of our clients and are on the lookout for Data Engineers (1–3 yrs) and Sr. / Lead Data Engineers (3–9 yrs) with strong hands-on experience in: Databricks | SQL | PySpark | Python Strong knowledge of Spark, Delta Lake, and Data Lakehouse...