Azure Databricks Engineer

2 weeks ago

Bangalore, India ResourceTree Global Services Full time

Job Title : Senior Databricks Data Engineer

Skillset : Databricks, Datalake Architecture, Azure

Experience : 6 - 8 years

Location : Chennai

Job Summary :

We are seeking an experienced Senior Databricks Data Engineer with a very good understanding of Databricks, Lakehouse concepts, with architecture experience, and a sound knowledge of Azure and its services.

The Senior Databricks Data Engineer will play a pivotal role in designing, developing, and maintaining data pipelines and data solutions on the Databricks platform, with a focus on building a Lakehouse architecture.

This role requires a deep knowledge of data engineering fundamentals, a strong grasp of Databricks and Lakehouse principles, the ability to architect data solutions for scalability and efficiency while leveraging Azure services, and expertise in data migration strategies.

Understanding Hadoop eco-system will add brownie points.

Key Responsibilities :

- Architect Lakehouse Solutions : Design, implement, and architect Lakehouse solutions on Databricks, leveraging Delta Lake, and utilize Azure services to enhance data storage and processing.

- In-depth Databricks Expertise : Demonstrate a deep understanding of Databricks platform features, including Spark SQL, Delta Lake, and Databricks notebooks, to optimize data engineering processes.

- Data Transformation : Implement advanced data transformations and quality checks within the Lakehouse architecture to ensure data accuracy, completeness, and consistency.

- Performance Optimization : Architect data pipelines for performance and scalability, making efficient use of Databricks clusters and Azure resources.

- Data Modelling : Create and implement advanced data models and schemas on Databricks, aligning with Lakehouse principles for analytical and reporting needs.

- Data Integration : Seamlessly integrate data from diverse sources, aligning with Lakehouse principles for data ingestion and storage, leveraging Azure Data Lake Storage and Azure SQL Data Warehouse.

- Lakehouse Architecture Experience : Apply architecture principles to build and maintain a scalable, efficient, and resilient Lakehouse architecture on Databricks while utilizing Azure for enhanced data processing and storage.

- Data Security : Implement and maintain comprehensive data security and access controls within the Lakehouse architecture, utilizing Azure security features to safeguard sensitive data.

- Data Migration Expertise : Possess a deep understanding of data migration strategies and practices, particularly in Azure environments, for seamless and secure data transfers. Experience on Hadoop ecosystem will be given advantage.

- Collaboration : Collaborate closely with data scientists, data analysts, and cross-functional teams to understand data requirements and deliver Lakehouse-based solutions that meet their needs, incorporating Azure-based solutions and data migration strategies as needed.

- Documentation : Create detailed documentation for data pipelines, data models, Lakehouse architecture configurations, Azure integrations, and data migration plans.

- Troubleshooting : Proactively identify and resolve complex data pipeline and architecture issues, ensuring data integrity and availability, with a focus on Azure monitoring and diagnostics.

- Performance Monitoring : Employ advanced monitoring techniques and Azure-based tools to maintain the performance and health of the Lakehouse architecture, taking proactive measures to address potential bottlenecks or issues. Should also have understanding of cluster monitoring and sizing.

- Data Governance : Ensure strict adherence to data governance and data management best practices within the Lakehouse architecture, utilizing Azure-based data governance solutions.

- Continuous Learning : Stay updated with the latest trends and developments in Databricks, Lakehouse concepts, Azure, data migration strategies, and data engineering best practices, and incorporate them into projects.

Qualifications and Skills :

- Bachelor's or higher degree in computer science, data engineering, or a related field.

- Proven experience as a data engineer with a strong focus on Databricks, Lakehouse architecture, advanced data engineering concepts, Azure services, and data migration strategies.

- Exceptional understanding of data engineering fundamentals, including data modeling, ETL, and data warehousing.

- Proficiency in programming languages such as Python, Scala, or Java.

- Extensive experience with Databricks platform features, advanced Delta Lake concepts, and a strong working knowledge of Azure services.

- Strong problem-solving skills and the ability to work in a fast-paced, collaborative environment.

- Excellent communication and teamwork skills.

- Relevant certifications in Databricks, Lakehouse, Azure services, and data migration are highly desirable.

(ref:hirist.tech)

  • Azure Databricks Engineer

    2 weeks ago

    bangalore, India Tredence Inc. Full time

    Role: Senior Databricks Engineer / Databricks Technical Lead/ Data ArchitectExperience: 3-12 yearsLocation: Bangalore, Chennai, Delhi, Pune, KolkataAbout Tredence:Tredence focuses on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science and software engineering. The largest companies...

  • Lead Azure Databricks Engineer

    2 weeks ago

    Bangalore, India ResourceTree Global Services Full time

    Resource Tree Global Services Pvt Ltd is hiring Senior Databricks Data Engineer for one of its leading client in Content & Data Analytics. Below are the details .Job Title : Senior Databricks Data EngineerSkills Required : Azure, Databricks, Datalake, Python/Scala/Java, Data Migration Strategies, Data Warehiusing, Data Modelling, Strong knowledge on Data...

  • Data Engineer

    2 weeks ago

    Bangalore/Hyderabad/Chennai, India Mouri Tech (P) Ltd Full time

    Job Title : Azure Data EngineerWork Location : Hyderabad , Bangalore, ChennaiExperience : 5+ yearsRequired Skills : Azure, Python, Azure DatabricksResponsibilities : . 5+ years as a data engineer.- 2- 3 years' experience in python- 2 years of experience in Azure SQL- Experience in DataBricks- Familiarity with Azure DevOps & database CI/CD.- Knowledge of...

  • Azure Databricks Engineer

    3 weeks ago

    Bangalore, Karnataka, India ARR Recruitment Solutions Full time

    Azure Databricks Engineer About the Role :- We are seeking a highly skilled Azure Databricks Engineer to join our team and play a vital role in developing and maintaining our big data processing pipelines.- This role is ideal for a candidate with extensive experience in Azure data services and a strong understanding of Apache Spark for large-scale data...

  • Azure Databricks Engineer

    3 weeks ago

    Bangalore, India ARR Recruitment Solutions Full time

    Azure Databricks Engineer About the Role :- We are seeking a highly skilled Azure Databricks Engineer to join our team and play a vital role in developing and maintaining our big data processing pipelines.- This role is ideal for a candidate with extensive experience in Azure data services and a strong understanding of Apache Spark for large-scale data...

  • Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Career Soft Solutions Full time

    Job Description :- 3+ years of relevant experience in Pyspark and Azure Databricks.- Proficiency in integrating, transforming and consolidating data from various structured and -unstructured data sources.- Good experience in SQL or native SQL query languages.- Strong experience in implementing Databricks notebooks using Python.- Good experience in Azure Data...

  • Data Engineer

    4 weeks ago

    Bangalore, India Career Soft Solutions Full time

    Job Description : - 3+ years of relevant experience in Pyspark and Azure Databricks.- Proficiency in integrating, transforming and consolidating data from various structured and -unstructured data sources.- Good experience in SQL or native SQL query languages.- Strong experience in implementing Databricks notebooks using Python.- Good experience in Azure...

  • Lead Azure Databricks Engineer

    2 weeks ago

    Bangalore, India VIDPRO CONSULTANCY SERVICES Full time

    Primary Roles and Responsibilities :- Developing Modern Data Warehouse solutions using Databricks and AWS/ AzureStack.- Ability to provide solutions that are forward-thinking in data engineering and analytics space.- Collaborate with DW/BI leads to understand new ETL pipeline development requirements.- Triage issues to find gaps in existing pipelines and fix...

  • Senior Azure Data Engineer

    3 weeks ago

    Mumbai/Bangalore, Maharashtra, India Techno Wise Full time

    Responsibilities :1. Design, develop, and maintain scalable data pipelines and ETL processes using Databricks and PySpark.2. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and implement data solutions that align with business needs.3. Optimize and tune existing data pipelines for performance and...

  • Senior Azure Data Engineer

    2 weeks ago

    Mumbai/Bangalore, India Techno Wise Full time

    Responsibilities :1. Design, develop, and maintain scalable data pipelines and ETL processes using Databricks and PySpark.2. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and implement data solutions that align with business needs.3. Optimize and tune existing data pipelines for performance and...

  • Data Scientist

    4 weeks ago

    bangalore, India VIDPRO CONSULTANCY SERVICES Full time

    About the job:Job Description:We are seeking an experienced data scientist who apart from the required mathematical and statistical expertise also possesses the natural curiosity and creative mind to ask questions, connect the dots, and uncover opportunities that lie hidden with the ultimate goal of realizing the data's full potential.Primary Roles And...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Growel Softech Pvt. Ltd. Full time

    Job Title: Azure Data Engineer. Location: Bangalore, India. Experience Level: 4-8 years. Job Description:. We are looking for an experienced Azure Data Engineer to join our team in Bangalore. As an Azure Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and solutions using Azure Data Factory,...

  • Senior Data Engineer

    4 weeks ago

    bangalore, India Growel Softech Pvt. Ltd. Full time

    Job Title: Azure Data Engineer. Location: Bangalore, India. Experience Level: 4-8 years. Job Description:. We are looking for an experienced Azure Data Engineer to join our team in Bangalore. As an Azure Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and solutions using Azure Data Factory,...

  • Senior Data Engineer

    2 weeks ago

    Bangalore, India Growel Softech Pvt. Ltd. Full time

    Job Title: Azure Data Engineer. Location: Bangalore, India. Experience Level: 4-8 years. Job Description:. We are looking for an experienced Azure Data Engineer to join our team in Bangalore. As an Azure Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and solutions using Azure Data Factory,...

  • Architect - Azure Databricks

    2 weeks ago

    bangalore, India Marlabs LLC Full time

    Marlabs is a digital innovation company helping enterprises innovate & capture digital opportunities to enable swifter solutions. Our focus is to help each of our clients find and capture their unique opportunities and empower them with digital technologies so that they can run a purposeful business. Visit and know more about us – Work Location -...

  • Data Scientist

    3 weeks ago

    Bangalore, Karnataka, India VIDPRO CONSULTANCY SERVICES Full time

    About the job:Job Description:We are seeking an experienced data scientist who apart from the required mathematical and statistical expertise also possesses the natural curiosity and creative mind to ask questions, connect the dots, and uncover opportunities that lie hidden with the ultimate goal of realizing the data's full potential.Primary Roles And...

  • Data Scientist

    2 weeks ago

    Bangalore, India VIDPRO CONSULTANCY SERVICES Full time

    About the job:Job Description:We are seeking an experienced data scientist who apart from the required mathematical and statistical expertise also possesses the natural curiosity and creative mind to ask questions, connect the dots, and uncover opportunities that lie hidden with the ultimate goal of realizing the data's full potential.Primary Roles And...

  • Azure Databricks Support Engineer

    3 weeks ago

    Any Location/Bangalore, IN INSIGHT GLOBAL SOLUTIONS Full time

    Required Skills and Experience - - 6+ years of hands-on experience in managing and administrating Azure PaaS environments. - 3+ years of hands-on experience setting up, solutioning, configuring, provisioning, and administering Azure Databricks platform -must be a SME. - Any experience in providing best practices around Azure Databricks for enterprise...

  • Azure Databricks Support Engineer

    4 weeks ago

    Any Location,Bangalore, India INSIGHT GLOBAL SOLUTIONS Full time

    Required Skills and Experience - - 6+ years of hands-on experience in managing and administrating Azure PaaS environments. - 3+ years of hands-on experience setting up, solutioning, configuring, provisioning, and administering Azure Databricks platform -must be a SME. - Any experience in providing best practices around Azure Databricks for enterprise...

  • Vsquare Systems

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Vsquare Systems Pvt. Ltd. Full time

    Data Engineer (Azure Databricks)Job Title : Data Engineer (Azure Databricks). Location : Bangalore / Hybrid. Job Type : Full-time. Job Description : .- Experienced Data engineer & Solutions designer with a deep understanding of data solution design and implementation, seeking a challenging role to leverage my expertise in data architecture, cloud migrations,...