Databricks Developer

3 days ago

Vijayawada, Andhra Pradesh, India beBeeDataInsights Full time ₹ 15,00,000 - ₹ 25,00,000

Job Title: Databricks Developer - Data Insights Specialist

Job Description:

We are seeking a highly skilled Databricks developer to join our team as a Data Insights Specialist. In this role, you will be responsible for designing, developing, and maintaining ETL processes to extract, transform, and load data from various sources into our data lake and data warehouse.

You will work closely with data engineers, data scientists, and business intelligence teams to build and optimize data workflows that support the project's analytics and reporting needs.

Responsibilities:
  • ETL Development:
    • Design and develop ETL processes using Databricks PySpark to extract, transform, and load data from heterogeneous sources into our data lake and data warehouse.
    • Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
    • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
    • Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
    • Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.
  • Data Pipeline Management:
    • Hands-on experience in creating advanced data pipelines using Databricks workflows.
    • Develop and maintain data pipelines using Databricks PySpark, ensuring data quality, integrity, and reliability throughout the ETL lifecycle.
    • Collaborate with data engineering, data science, and business intelligence teams to translate data requirements into efficient ETL workflows and pipelines.
  • Data Analysis and Query Optimization:
    • Write and optimize complex SQL queries for data manipulation, aggregation, and analysis within Databricks PySpark applications.
  • Project Coordination and Continuous Improvement:
    • Participate in project planning and coordination activities to ensure timely delivery of ETL solutions.
    • Stay updated on the latest developments in Databricks PySpark, Spark SQL, and related technologies, recommending and implementing best practices and optimizations.
    • Document ETL processes, data lineage, and metadata to facilitate knowledge sharing and ensure compliance with data governance standards.

  • Databricks Systems Architect

    1 week ago

    Vijayawada, Andhra Pradesh, India beBeeCloud Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title: Databricks Systems ArchitectAbout the RoleWe are seeking an experienced Databricks professional to join our team as a Databricks Systems Architect. As a key member of our technical team, you will be responsible for designing and implementing scalable Databricks solutions to support our business growth.Key Responsibilities:Design and deploy...

  • Expert Large Scale Data Pipeline Developer

    2 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking an experienced Data Engineer to design, build, and optimize large-scale data pipelines. The ideal candidate will have expertise in Python, SQL, Linux/Bash, PySpark, AWS, and Databricks.The primary focus of this role is to develop, maintain, and optimize ETL/ELT pipelines using PySpark, AWS, and Databricks. Additionally, the...

  • Data Engineer

    3 days ago

    Vijayawada, Andhra Pradesh, India VSquare Systems Pvt. Ltd. Full time

    Job Description:We are looking for a highly skilled Databricks PySpark Developer to join our data platform implementation team. In this role, you will be instrumental in designing, developing, and maintaining ETL processes to ensure efficient extraction, transformation, and loading of data from various sources into data lake and data warehouse. You will work...

  • Lead Data Engineering Position

    4 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    We are seeking a Lead Data Engineer to drive our data strategy.This is a senior technical role with high expectations. The ideal candidate must be extremely hands-on, technically strong in Azure and Databricks, and capable of delivering high-quality solutions with speed and precision.The key responsibilities include designing, developing, and optimizing data...

  • Senior Data Science Developer Position

    5 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataScience Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Science DeveloperWe are seeking a highly skilled Data Science Developer to drive data-driven initiatives forward.Develop scalable data pipelines using Big Data technologies.Process large datasets using Spark, PySpark and SQL for meaningful insights.Build real-time streaming applications with Spark Streaming/Kafka.Collaborate with data scientists and...

  • Data Architecture Specialist

    2 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 14,00,000 - ₹ 20,10,000

    About the JobWe are seeking a skilled professional to fill the role of Data Engineer in our organization.Job Summary:The ideal candidate will have hands-on experience in building and automating data pipelines, managing multi-environment deployments, and working with modern DevOps/configuration management tools.Responsibilities:Design and Implement Continuous...

  • Transformative Data Solutions Specialist

    3 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data EngineerWe are seeking a talented Data Engineer with 4–7 years of experience to build and optimize scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake.The role involves working across scrum teams to develop data solutions, ensure data governance with Unity Catalog, and support real-time and batch processing.Develop...

  • Data Ingestion Specialist

    4 hours ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 10,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking an experienced Data Integration Specialist to join our team. This role requires a strong understanding of data ingestion processes, focusing on integrating various data sources into Databricks.The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and...

  • Chief Data Architecture Specialist

    2 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job Opportunity: Senior Data EngineerAs a seasoned Senior Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines and workflows using AWS cloud services and Databricks.Key Responsibilities:Develop scalable, high-performance data pipelines that meet business needs.Build and optimize ETL/ELT processes for large-scale...

  • Artificial Intelligence Workflow Automation Specialist

    9 hours ago

    Vijayawada, Andhra Pradesh, India beBeeAutomation Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    AI Engineering Specialist Role AI Engineering and AutomationWe are seeking an experienced AI Engineer to join our team in building automated workflows for compliance-heavy business processes. This is an exciting opportunity to work with cutting-edge technologies, including Databricks, Azure, and LangChain.You will collaborate closely with domain SMEs to...