Lead Azure Databricks Engineer

4 weeks ago

Bangalore, India ResourceTree Global Services Full time

Resource Tree Global Services Pvt Ltd is hiring Senior Databricks Data Engineer for one of its leading client in Content & Data Analytics. Below are the details .

Job Title : Senior Databricks Data Engineer

Skills Required : Azure, Databricks, Datalake, Python/Scala/Java, Data Migration Strategies, Data Warehiusing, Data Modelling, Strong knowledge on Data Engneering, ETL Concepts.

Location : Bangalore

Notice Period : Immediate

Job Summary :

We are seeking an experienced Senior Databricks Data Engineer with a very good understanding of Databricks, Lakehouse concepts, with architecture experience, and a sound knowledge of Azure and its services. The Senior Databricks Data Engineer will play a pivotal role in designing, developing, and maintaining data pipelines and data solutions on the Databricks platform, with a focus on building a Lakehouse architecture. This role requires a deep knowledge of data engineering fundamentals, a strong grasp of Databricks and Lakehouse principles, the ability to architect data solutions for scalability and efficiency while leveraging Azure services, and expertise in data migration strategies. Understanding Hadoop eco-system will add brownie points.

Key Responsibilities :

  • Architect Lakehouse Solutions: Design, implement, and architect Lakehouse solutions on Databricks, leveraging Delta Lake, and utilize Azure services to enhance data storage and processing.
  • In-depth Databricks Expertise: Demonstrate a deep understanding of Databricks platform features, including Spark SQL, Delta Lake, and Databricks notebooks, to optimize data engineering processes.
  • Data Transformation: Implement advanced data transformations and quality checks within the Lakehouse architecture to ensure data accuracy, completeness, and consistency.
  • Performance Optimization: Architect data pipelines for performance and scalability, making efficient use of Databricks clusters and Azure resources.
  • Data Modelling: Create and implement advanced data models and schemas on Databricks, aligning with Lakehouse principles for analytical and reporting needs.
  • Data Integration: Seamlessly integrate data from diverse sources, aligning with Lakehouse principles for data ingestion and storage, leveraging Azure Data Lake Storage and Azure SQL Data Warehouse.
  • Lakehouse Architecture Experience: Apply architecture principles to build and maintain a scalable, efficient, and resilient Lakehouse architecture on Databricks while utilizing Azure for enhanced data processing and storage.
  • Data Security: Implement and maintain comprehensive data security and access controls within the Lakehouse architecture, utilizing Azure security features to safeguard sensitive data.
  • Data Migration Expertise: Possess a deep understanding of data migration strategies and practices, particularly in Azure environments, for seamless and secure data transfers. Experience on Hadoop ecosystem will be given advantage.
  • Collaboration: Collaborate closely with data scientists, data analysts, and cross-functional teams to understand data requirements and deliver Lakehouse-based solutions that meet their needs, incorporating Azure-based solutions and data migration strategies as needed.
  • Documentation: Create detailed documentation for data pipelines, data models, Lakehouse architecture configurations, Azure integrations, and data migration plans.
  • Troubleshooting: Proactively identify and resolve complex data pipeline and architecture issues, ensuring data integrity and availability, with a focus on Azure monitoring and diagnostics.
  • Performance Monitoring: Employ advanced monitoring techniques and Azure-based tools to maintain the performance and health of the Lakehouse architecture, taking proactive measures to address potential bottlenecks or issues. Should also have understanding of cluster monitoring and sizing.
  • Data Governance: Ensure strict adherence to data governance and data management best practices within the Lakehouse architecture, utilizing Azure-based data governance solutions.
  • Continuous Learning: Stay updated with the latest trends and developments in Databricks, Lakehouse concepts, Azure, data migration strategies, and data engineering best practices, and incorporate them into projects.

Qualifications and Skills :

  • Bachelor's or higher degree in computer science, data engineering, or a related field.
  • Proven experience as a data engineer with a strong focus on Databricks, Lakehouse architecture, advanced data engineering concepts, Azure services, and data migration strategies.
  • Exceptional understanding of data engineering fundamentals, including data modeling, ETL, and data warehousing.
  • Proficiency in programming languages such as Python, Scala, or Java.
  • Extensive experience with Databricks platform features, advanced Delta Lake concepts, and a strong working knowledge of Azure services.
  • Strong problem-solving skills and the ability to work in a fast-paced, collaborative environment.
  • Excellent communication and teamwork skills.
  • Relevant certifications in Databricks, Lakehouse, Azure services, and data migration are highly desirable.
(ref:hirist.tech)

  • Azure Databricks Engineer

    2 weeks ago

    bangalore, India Tredence Inc. Full time

    Role: Senior Databricks Engineer / Databricks Technical Lead/ Data ArchitectExperience: 3-12 yearsLocation: Bangalore, Chennai, Delhi, Pune, KolkataAbout Tredence:Tredence focuses on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science and software engineering. The largest companies...

  • Technical Lead

    2 weeks ago

    bangalore, India Marlabs LLC Full time

    Marlabs is a digital innovation company helping enterprises innovate & capture digital opportunities to enable swifter solutions. Our focus is to help each of our clients find and capture their unique opportunities and empower them with digital technologies so that they can run a purposeful business. Visit and know more about us – .Work Location -...

  • Lead Azure Databricks Engineer

    2 weeks ago

    Bangalore, India VIDPRO CONSULTANCY SERVICES Full time

    Primary Roles and Responsibilities :- Developing Modern Data Warehouse solutions using Databricks and AWS/ AzureStack.- Ability to provide solutions that are forward-thinking in data engineering and analytics space.- Collaborate with DW/BI leads to understand new ETL pipeline development requirements.- Triage issues to find gaps in existing pipelines and fix...

  • Azure Databricks Engineer

    2 weeks ago

    Bangalore, India ResourceTree Global Services Full time

    Job Title : Senior Databricks Data EngineerSkillset : Databricks, Datalake Architecture, AzureExperience : 6 - 8 yearsLocation : ChennaiJob Summary : We are seeking an experienced Senior Databricks Data Engineer with a very good understanding of Databricks, Lakehouse concepts, with architecture experience, and a sound knowledge of Azure and its services....

  • Technical Lead

    2 weeks ago

    bangalore, India Marlabs LLC Full time

    Marlabs is a digital innovation company helping enterprises innovate & capture digital opportunities to enable swifter solutions. Our focus is to help each of our clients find and capture their unique opportunities and empower them with digital technologies so that they can run a purposeful business. Visit and know more about us – Work Location -...

  • Lead Databricks Engineer

    3 weeks ago

    bangalore, India Tredence Inc. Full time

    About Tredence: Tredence is a global analytics services and solutions company. We are one of the fastest growing private companies in the country for three straight years according to the Inc. 5000 and we continue to set ourselves apart from our competitors by attracting the greatest talent in the data analytics and data science space. Our capabilities range...

  • Data Engineer

    2 weeks ago

    Bangalore/Hyderabad/Chennai, India Mouri Tech (P) Ltd Full time

    Job Title : Azure Data EngineerWork Location : Hyderabad , Bangalore, ChennaiExperience : 5+ yearsRequired Skills : Azure, Python, Azure DatabricksResponsibilities : . 5+ years as a data engineer.- 2- 3 years' experience in python- 2 years of experience in Azure SQL- Experience in DataBricks- Familiarity with Azure DevOps & database CI/CD.- Knowledge of...

  • Lead - Azure Databricks, Pyspark, Sql

    6 days ago

    Bangalore City, India Infogain Full time

    Position: Lead Azure DatabricksOverall IT experience: 9 to 13 yearsPrimary Skills: Azure Databricks, Hadoop, Hive, Pig, Impala, ETL, Data warehouse Secondary Skills: U-SQL, Spark, Scala, PythonWork Location: Bengaluru, Pune, Mumbai, Noida, GurugramMode of work: Hybrid at Infogain office locations mentioned aboveNotice Period: 0 to 30 DaysEssential...

  • Lead Databricks Engineer

    4 weeks ago

    Bangalore Metropolitan Area, India Tredence Inc. Full time

    About Tredence:Tredence is a global analytics services and solutions company. We are one of the fastest growing private companies in the country for three straight years according to the Inc. 5000 and we continue to set ourselves apart from our competitors by attracting the greatest talent in the data analytics and data science space. Our capabilities range...

  • Azure Databricks Engineer

    3 weeks ago

    Bangalore, Karnataka, India ARR Recruitment Solutions Full time

    Azure Databricks Engineer About the Role :- We are seeking a highly skilled Azure Databricks Engineer to join our team and play a vital role in developing and maintaining our big data processing pipelines.- This role is ideal for a candidate with extensive experience in Azure data services and a strong understanding of Apache Spark for large-scale data...

  • Azure Databricks Engineer

    3 weeks ago

    Bangalore, India ARR Recruitment Solutions Full time

    Azure Databricks Engineer About the Role :- We are seeking a highly skilled Azure Databricks Engineer to join our team and play a vital role in developing and maintaining our big data processing pipelines.- This role is ideal for a candidate with extensive experience in Azure data services and a strong understanding of Apache Spark for large-scale data...

  • Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Career Soft Solutions Full time

    Job Description :- 3+ years of relevant experience in Pyspark and Azure Databricks.- Proficiency in integrating, transforming and consolidating data from various structured and -unstructured data sources.- Good experience in SQL or native SQL query languages.- Strong experience in implementing Databricks notebooks using Python.- Good experience in Azure Data...

  • Data Engineer

    4 weeks ago

    Bangalore, India Career Soft Solutions Full time

    Job Description : - 3+ years of relevant experience in Pyspark and Azure Databricks.- Proficiency in integrating, transforming and consolidating data from various structured and -unstructured data sources.- Good experience in SQL or native SQL query languages.- Strong experience in implementing Databricks notebooks using Python.- Good experience in Azure...

  • Architect - Azure Databricks

    2 weeks ago

    bangalore, India Marlabs LLC Full time

    Marlabs is a digital innovation company helping enterprises innovate & capture digital opportunities to enable swifter solutions. Our focus is to help each of our clients find and capture their unique opportunities and empower them with digital technologies so that they can run a purposeful business. Visit and know more about us – Work Location -...

  • ADC Data Engineering Lead

    2 weeks ago

    Bangalore, Karnataka, India Growel Softech Pvt. Ltd. Full time

    Job Description:We are seeking an experienced ADC (Advanced Data Computing) Data Engineering Lead with 10-13 years of experience to join our team. The ideal candidate will have a strong background in Big Data engineering, with expertise in tools such as Databricks, Azure Data Factory, and SQL. As an ADC Data Engineering Lead, you will be responsible for...

  • ADC Data Engineering Lead

    2 weeks ago

    Bangalore, India Growel Softech Pvt. Ltd. Full time

    Job Description:We are seeking an experienced ADC (Advanced Data Computing) Data Engineering Lead with 10-13 years of experience to join our team. The ideal candidate will have a strong background in Big Data engineering, with expertise in tools such as Databricks, Azure Data Factory, and SQL. As an ADC Data Engineering Lead, you will be responsible for...

  • ADC Data Engineering Lead

    1 week ago

    bangalore, India Growel Softech Pvt. Ltd. Full time

    Job Description:We are seeking an experienced ADC (Advanced Data Computing) Data Engineering Lead with 10-13 years of experience to join our team. The ideal candidate will have a strong background in Big Data engineering, with expertise in tools such as Databricks, Azure Data Factory, and SQL. As an ADC Data Engineering Lead, you will be responsible for...

  • Data Scientist

    4 weeks ago

    bangalore, India VIDPRO CONSULTANCY SERVICES Full time

    About the job:Job Description:We are seeking an experienced data scientist who apart from the required mathematical and statistical expertise also possesses the natural curiosity and creative mind to ask questions, connect the dots, and uncover opportunities that lie hidden with the ultimate goal of realizing the data's full potential.Primary Roles And...

  • Azure Data engineer

    2 weeks ago

    bangalore, India Birlasoft Full time

    Job Title: Azure Databricks LeadLocation: Bangalore/Mumbai/Pune/NoidaCompany Description:Birlasoft, a powerhouse where domain expertise, enterprise solutions, and digital technologies converge to redefine business processes. We take pride in our consultative and design thinking approach, driving societal progress by enabling our customers to run businesses...

  • Data Scientist

    3 weeks ago

    Bangalore, Karnataka, India VIDPRO CONSULTANCY SERVICES Full time

    About the job:Job Description:We are seeking an experienced data scientist who apart from the required mathematical and statistical expertise also possesses the natural curiosity and creative mind to ask questions, connect the dots, and uncover opportunities that lie hidden with the ultimate goal of realizing the data's full potential.Primary Roles And...