Data Engineer with Databricks Expertise

2 days ago

Pune, Maharashtra, India Mphasis Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Senior Software Engineer

Job Description: Data Engineer with Databricks Expertise

Job Summary

We are seeking a highly skilled Data Engineer with Databricks certifications and extensive practical experience working with the Databricks Lakehouse platform and its supporting capabilities. The ideal candidate will possess a robust understanding of AWS services relevant to data engineering and demonstrate proficiency in data pipeline development, programming, data modeling, and governance.

Key Responsibilities

  • Design and implement data pipelines using AWS and Databricks.
  • Manage and orchestrate data workflows, ensuring efficient data ingestion processes.
  • Utilize Delta Lake for data storage and management, ensuring secure data sharing across platforms with Delta Sharing.
  • Implement data governance and access control using Unity Catalog.
  • Apply the Medallion architecture (Bronze, Silver, Gold layers) to projects.
  • Develop and manage ETL pipelines using AWS Glue.
  • Leverage Amazon S3 for data storage and management.
  • Utilize AWS Lambda and Step Functions for serverless computing in data processing.
  • Apply data warehousing solutions and best practices in data transformation.
  • Monitor and implement data quality checks and governance practices.
  • Collaborate with data scientists, analysts, and other stakeholders in an Agile environment.
  • Deliver large-scale data processing and analytics projects within cloud environments.

Required Skills and Qualifications

  • Databricks certifications with practical experience in the Databricks Lakehouse platform.
  • Proficient in AWS services: Glue, S3, Lambda, Step Functions, and data warehousing solutions.
  • Strong programming skills in Python, Scala, or Java.
  • Proficient in SQL for querying and manipulating data.
  • Experience in data modeling for Lakehouse architecture.
  • Knowledge of ETL best practices and data transformation techniques.
  • Understanding of data governance frameworks and practices.
  • Proven track record of delivering data engineering projects, particularly in cloud environments.
  • Experience with large-scale data processing and analytics projects.
  • Ability to work in Agile teams and collaborate effectively with various stakeholders.

Preferred Qualifications

  • Advanced AWS certifications.
  • Experience with additional cloud platforms (e.g., Azure).
  • Familiarity with other data engineering tools and frameworks.

  • Azure Databricks Data Engineer

    6 days ago

    Pune, Maharashtra, India Reveille Technologies,Inc Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Hiring: Azure Databricks Data Engineer (FTE | Face-to-Face Interview) Location: Pune Experience: 9+ Years Notice Period: 0–30 Days (Serving candidates preferred)We're looking for an experienced Data Engineer with strong hands-on expertise in Azure Databricks and a solid data background. Key Skills:Primary: Azure, Databricks, ADF, PySpark/PythonSecondary:...

  • Data Engineer- Databricks

    2 days ago

    Pune, Maharashtra, India InfoCepts Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Position: Data Engineer- DatabricksPurpose of the Position:Develop, support and steer end-to-end business intelligence using Databricks.Location: Nagpur/ Pune/ Chennai/ BangaloreKey Responsibilities:Work with business analysts and data architects to translate requirements into technical implementations.Design, develop, implement, and maintain pyspark code...

  • Databricks Engineer

    2 days ago

    Pune, Maharashtra, India Peoplefy Infosolutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Must have skills:Hands-on experience with Databricks platform, including Delta Lake/ Lakehouse architecturePerson should be familiar with Python / Pyspark code (not needed a developer)Knowledge of data structures, data modeling, ETL/ELT, and data warehousing.SQL basic in data querying and troubleshooting / manipulation.Key asks for the role:We need a...

  • Azure Databricks Engineer

    2 weeks ago

    Pune, Maharashtra, India Savi Technologies Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Opening: Azure Databricks EngineerLocation: Pune, Maharashtra (Onsite)Experience Range: 5-6 YearsApply at: -Required Skills:Azure Data Factory (ADF)PySparkAzure Databricks (ADB)SQLJob Description:We're looking for an experienced Azure Databricks Engineer with a background in financial crime analytics and hands-on expertise in:Databricks engineering &...

  • Databricks Engineer

    2 days ago

    Pune, Maharashtra, India Persistent Systems Full time ₹ 12,00,000 - ₹ 20,00,000 per year

    About Position:We are seeking data bricks engineer with hands on experience in Power BI, python, spark and data bricks.Role: Databricks EngineerLocation: Pune & MumbaiExperience: 8 to 12 YearsJob Type: Full Time EmploymentWhat You'll Do:3+ years of experience in data engineering or analytics roles.Strong hands-on experience with Databricks and Apache...

  • Databricks Engineer

    2 days ago

    Pune, Maharashtra, India Virtusa Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Key Responsibilities:Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • Senior Databricks Engineer

    2 weeks ago

    Pune, Maharashtra, India Newscape Consulting Full time US$ 80,000 - US$ 1,20,000 per year

    About the RoleWe are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant...

  • Databricks Engineer

    2 days ago

    Pune, Maharashtra, India InfoCepts Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Position: Data Engineer- DatabricksPurpose of the Position:Develop, support and steer end-to-end business intelligence using Databricks.Location: Nagpur/ Pune/ Chennai/ BangaloreKey Responsibilities:Work with business analysts and data architects to translate requirements into technical implementations.Design, develop, implement, and maintain pyspark code...

  • Senior AWS Databricks Data Engineer

    2 days ago

    Pune, Maharashtra, India Sapphire Manpower Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Candidate should have experience in Data Engineering with AWS Databricks Please don't require Azure Databricks

  • Databricks

    4 days ago

    Pune, Maharashtra, India Talent Worx Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Every day, your work will make an impact that matters, while you thrive in a dynamic culture of inclusion, collaboration, and high performance. As the undisputed leader in professional services, Our Client is where you will find unrivaled opportunities to succeed and realize your full potential.We are seeking highly skilled Databricks Data Engineers to join...