Senior Databricks Developer

5 days ago

Vadodara, Gujarat, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000
Key Databricks Developer Role

As a leading organization in the field of data engineering and analytics, we are seeking a highly skilled and experienced Databricks developer to join our team.

We are looking for an individual who can work closely with data engineers, data scientists, and business intelligence teams to design, develop, and maintain ETL processes that efficiently extract, transform, and load data from various sources into data lake and data warehouse. The ideal candidate will have extensive experience in developing and debugging Databricks PySpark applications, with a strong focus on performance and scalability.

Responsibilities:

  • Design and develop ETL processes using Databricks PySpark to extract, transform, and load data from heterogeneous sources into our data lake and data warehouse.
  • Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
  • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
  • Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
  • Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.

Requirements:

  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Minimum of 3 years of experience as an ETL developer, with a strong focus on Databricks PySpark development.
  • Proficiency in Python programming, with extensive experience in developing and debugging Databricks PySpark applications.
  • In-depth understanding of Spark architecture and internals, with hands-on experience in Spark RDDs, DataFrames, and Spark SQL.
  • Expertise in writing and optimizing complex SQL queries for data manipulation, aggregation, and analysis.
  • Proven experience in working with large-scale data warehousing and ETL frameworks.
  • Strong problem-solving skills and the ability to troubleshoot and resolve ETL process issues.
  • Excellent communication and collaboration skills, with the ability to work effectively in a team environment.

Preferred Qualifications:

  • Experience with cloud platforms with a preference for AWS.
  • Experience with data platform tools such as DataBricks, Snowflake, and Tableau.
  • Demonstrated ability to implement best practices for ETL processes and data management.
  • Strong understanding of data governance and data quality principles.
  • Relevant certifications in Databricks PySpark, Spark SQL, or related technologies.

  • Databricks Specialist

    7 days ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Key Responsibilities:">Design, develop and maintain scalable data pipelines and ETL processes on Databricks.Lead the architecture and implementation of secure high-performance data solutions.Collaborate with business stakeholders and data experts to deliver actionable insights.Optimize data workflows for performance cost-efficiency and reliability.Integrate...

  • Remote Azure Databricks Data Ingestion Specialist

    7 days ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Opportunity:We are seeking an experienced Azure Databricks Engineer to join our team in a 6-month project, extendable to 1+ year, working remotely in India. The ideal candidate will have expertise in CI/CD and Kubernetes, strong understanding of data ingestion processes, and hands-on experience with data pipelines, automation, and infrastructure...

  • High-Performance Software Developer

    5 days ago

    Vadodara, Gujarat, India beBeeSoftware Full time ₹ 20,00,000 - ₹ 27,00,000

    Senior Software Engineer - Azure Databricks, SQL, Python, Py-Spark, AutomationWe are seeking a seasoned Senior Software Engineer with 4-8 years of experience to join our high-performance engineering team. In this role, you will collaborate with senior engineers, product managers, and other cross-functional teams to design, develop, and enhance cutting-edge...

  • Senior Data Engineer

    2 days ago

    Vadodara, Gujarat, India Reflik Full time US$ 1,25,000 - US$ 1,75,000 per year

    Job Title: Senior Data EngineerBasic FunctionThe Senior Data Engineer should be an expert familiar with all areas of data warehousing technical components (e.g., ETL, Reporting, Data Model), connected infrastructure, and their integrations. The ideal candidate will be responsible for developing the overall architecture and high-level design of the data...

  • Senior Cloud Data Architect

    7 days ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    We are seeking an experienced Lead Data Engineer to join our team.This is a senior technical role with high expectations. The ideal candidate must be extremely hands-on, technically strong in Azure and Databricks, and capable of delivering high-quality solutions with speed and precision.Key ResponsibilitiesDesign, develop, and optimize data pipelines using...

  • Senior Data Engineer

    4 days ago

    Vadodara, Gujarat, India beBeeDataEngineering Full time ₹ 18,35,000 - ₹ 23,40,000

    Job Title: Senior Data Engineer - Cloud Services Specialist We are seeking a highly skilled Senior Data Engineer to design and develop high-performance data pipelines using AWS cloud services and Databricks. This role involves building scalable ETL/ELT workflows, managing big data environments, and ensuring seamless data availability for analytics, AI/ML,...

  • #Cloud-Based Development Expert

    4 days ago

    Vadodara, Gujarat, India beBeeSoftware Remote Work Freelance Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Cloud-Based Software Developer PositionWe are seeking an experienced and skilled Senior Cloud-Based Software Developer to design, implement and maintain scalable cloud-based solutions using Azure and Google Cloud platforms.The ideal candidate will have hands-on experience in designing and implementing cloud-based solutions, automating data pipelines...

  • Azure Data Engineering Leadership Role

    5 days ago

    Vadodara, Gujarat, India beBeeData Full time US$ 1,20,000 - US$ 2,00,000

    Job OverviewAs a Lead Azure Data Engineer, you will be responsible for designing, developing, and optimizing data pipelines using Azure and Databricks. This is a senior technical role with high expectations, requiring extreme hands-on expertise in Azure and Databricks.The ideal candidate is a hands-on problem solver who ensures efficiency in their own work...

  • Senior Data Architect

    22 hours ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,98,72,000 - ₹ 2,43,84,000

    Job SummaryWe are seeking a skilled Senior Data Engineer to develop and optimize ETL/ELT pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics. The successful candidate will design and implement scalable data architectures following Medallion Architecture best practices.Main Responsibilities:Data Engineering:Develop and maintain data...

  • Chief Data Architect

    5 days ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Key Role: Senior Data EngineerAbout the JobWe are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will play a crucial role in designing and implementing cloud-based data architectures, ensuring high-quality input for downstream analytics.Responsibilities:Design and implement scalable real-time and batch...