Databricks Developer

2 days ago

Jaipur, Rajasthan, India beBeeEngineer Full time ₹ 1,20,00,000 - ₹ 2,00,00,000
Job Title: Data Engineer

We are seeking a seasoned Databricks PySpark Developer to join our team. In this role, you will be responsible for designing, developing, and maintaining ETL processes to ensure efficient extraction, transformation, and loading of data from various sources into data lake and data warehouse.

Key Responsibilities:

  1. ETL Development:
  • Design and develop ETL processes using Databricks PySpark to extract, transform, and load data from heterogeneous sources into our data lake and data warehouse.
  • Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
  • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
  • Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
  • Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.
Data Pipeline Management:
  • Develop and maintain data pipelines using Databricks PySpark, ensuring data quality, integrity, and reliability throughout the ETL lifecycle.
  • Collaborate with data engineering, data science, and business intelligence teams to translate data requirements into efficient ETL workflows and pipelines.
Data Analysis and Query Optimization:
  • Write and optimize complex SQL queries for data manipulation, aggregation, and analysis within Databricks PySpark applications.
Project Coordination and Continuous Improvement:
  • Participate in project planning and coordination activities to ensure timely delivery of ETL solutions.
  • Stay updated on the latest developments in Databricks PySpark, Spark SQL, and related technologies, recommending and implementing best practices and optimizations.
  • Document ETL processes, data lineage, and metadata to facilitate knowledge sharing and ensure compliance with data governance standards.

  • Databricks Solutions Architect

    6 days ago

    Jaipur, Rajasthan, India beBeeDatabricks Full time US$ 1,20,000 - US$ 2,00,000

    Job SummaryWe are seeking a highly skilled Databricks Engineer to design, develop and maintain data pipelines and ETL processes on Databricks.About the Role:The ideal candidate will have 7+ years of experience in Data Engineering and proven subject matter expertise in Databricks (including Databricks SQL, Delta Lake and MLflow).They will be responsible for...

  • Leading Databricks Data Engineering Expert

    3 days ago

    Jaipur, Rajasthan, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Databricks Data EngineerWe are seeking a highly skilled and experienced Senior Databricks Data Engineer with strong hands-on expertise in Databricks and AWS to join our dynamic data engineering team.The ideal candidate will be well-versed in PySpark, CI/CD pipelines, and Terraform for infrastructure as code. Experience with Data Access Broker (DAB) is...

  • Freelance Senior Databricks Data Engineer

    2 weeks ago

    Jaipur, Rajasthan, India ThreatXIntel Full time

    Company Description ThreatXIntel is a startup cyber security company that specializes in protecting businesses and organizations from cyber threats. Our team offers services in cloud security, web and mobile security testing, cloud security assessment, and DevSecOps. We provide customized and affordable solutions to meet the specific needs of our clients,...

  • Chief Data Architect

    4 days ago

    Jaipur, Rajasthan, India beBeeLead Full time ₹ 30,00,000 - ₹ 40,00,000

    Job Title: Lead Azure Data EngineerImmediate hiring for a highly skilled and experienced lead data engineer to join our team.We're looking for a senior technical expert with deep hands-on expertise in Azure and Databricks. This is a critical role with high expectations. The consultant must be extremely hands-on, technically strong in Azure and Databricks,...

  • Chief Data Strategist

    4 days ago

    Jaipur, Rajasthan, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Unlock Business PotentialOur team is seeking an experienced Data Architect to join our ranks. This role involves designing and implementing scalable, secure, and efficient data solutions using Databricks.Key Responsibilities Include:Developing data pipelines, architecting data lakes, and implementing data warehousing solutions using Databricks.Collaborating...

  • Software Developer Specialist

    1 day ago

    Jaipur, Rajasthan, India beBeeSoftwareDeveloper Remote Work Freelance Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Software Developer SpecialistDescriptionWe are seeking a skilled software developer to fill a key role in our organization. As a software developer specialist, you will design and implement cloud-based solutions using Azure and Google Cloud platforms.You will automate data pipelines to facilitate efficient data flow and processing, develop and...

  • Azure Data Solutions Developer

    4 days ago

    Jaipur, Rajasthan, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Job Opportunity">Data Engineer Job Opening:">We are seeking a highly skilled and experienced Data Engineer to join our team.The successful candidate will be responsible for designing, developing, and maintaining large-scale data systems using Azure Databricks.The ideal candidate will have a strong background in data engineering, experience with Pyspark, and...

  • Senior Data Architect

    3 days ago

    Jaipur, Rajasthan, India beBeeDataEngineer Full time US$ 90,000 - US$ 1,20,000

    Job Opportunity: Data EngineerJob SummaryWe are seeking an experienced data engineer to join our team and contribute to the design, implementation, and management of large-scale data pipelines using Spark, Python, and SQL.Key ResponsibilitiesDesign, implement, and manage continuous integration/continuous deployment (CI/CD) pipelines using GitHub Actions or...

  • Data Transformation Specialist

    1 day ago

    Jaipur, Rajasthan, India beBeeEtlTesting Full time ₹ 15,00,000 - ₹ 18,00,000

    ETL Tester Job DescriptionWe are seeking a skilled professional to fill the ETL Tester role.Design and execute comprehensive test plans to validate Databricks tables across multiple layers.Develop proficiency in validating data transformations, ingestion, and processing logic.Write complex SQL queries to ensure accurate results.The ideal candidate will...

  • Expert Data Engineer Wanted

    2 days ago

    Jaipur, Rajasthan, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking a highly skilled data engineer to build and optimize scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. The role involves collaborating with cross-functional teams to develop data solutions, ensure data governance with Unity Catalog, and support real-time and batch processing.The ideal candidate will...