Natobotics - Senior Data Engineer - Azure Databricks

3 weeks ago

Mumbai, India Natobotics Technologies Pvt Limited Full time

Description : Senior Data Engineer - Azure ETL/ELT & : Mumbai, MH / Hyderabad, TG / Chennai, TN / Bhubaneswar, OR / Bangalore, : 6-8 YearsAbout the Role :We are seeking an experienced Senior Data Engineer with 6-8 years in data engineering to join our team across multiple locations. This role is focused on designing, developing, and maintaining scalable and fault-tolerant data pipelines using the Microsoft Azure ecosystem. The ideal candidate will have deep expertise in Azure Data Factory, Databricks, PySpark, and advanced SQL to drive complex data transformations and implement robust CI/CD :- ETL/ELT Pipeline Development : Design, develop, and maintain complex ETL/ELT pipelines for large-scale data ingestion and transformation using Azure Data Factory (ADF) and Azure Databricks.- Data Ingestion Flows : Implement robust data ingestion flows to integrate data from diverse sources, including Azure Blob Storage, Azure Data Lake (ADLS), On-Prem SQL, and SFTP, ensuring reliability and security.- Data Transformation & Optimization : Design and optimize dimensional and relational data models, and perform complex data transformations utilizing Spark SQL, PySpark, Oracle, SQL Server, and Progress DB SQL.- Orchestration Workflow : Build sophisticated orchestration workflows in ADF using a range of activities such as Lookup, ForEach, Execute Pipeline, and Set Variable, to manage complex, multi-stage data jobs.- Troubleshooting and Maintenance : Perform root cause analysis and quickly resolve critical production issues affecting data pipelines, Databricks notebooks, and data flow stability.- CICD and DevOps : Collaborate with DevOps teams on the creation and enhancement of CI/CD pipelines using tools like Azure DevOps and Jenkins to enable automated testing and deployment of data assets.- Performance Tuning : Apply specialized performance tuning techniques to optimize query execution and resource utilization within Azure Synapse Analytics (SQL DW) environments.- Documentation and Quality : Maintain comprehensive technical documentation, including detailed runbooks, technical design specifications, and QA test cases, ensuring knowledge transfer and operational efficiency.Required Skill Set (Mandatory Skills) :- Cloud Platforms : 6-8 years of hands-on experience utilizing key Microsoft Azure services for data engineering, including Azure Data Factory (ADF), Azure Databricks (ADB), Azure Data Lake Storage (ADLS), Azure SQL, and Azure Synapse Analytics.- Programming & Big Data : Expert proficiency in Python for Data Science, and deep experience with PySpark and Spark SQL for large-scale distributed data processing.- ETL Tools : Mandatory expertise in building and maintaining enterprise-level data pipelines using Azure Data Factory.- Database & SQL : Expert-level proficiency in SQL for complex querying, performance tuning, and data model design.Preferred Skills :- Practical experience applying performance tuning specifically to Azure Synapse Analytics (SQL DW).- Experience with Azure DevOps, Git, and CICD pipeline creation using either Azure DevOps or Jenkins.- Familiarity with non-relational databases such as Cosmos DB or other NoSQL platforms.- Experience ingesting data from older enterprise systems like DB2 or diverse formats like NoSQL sources. (ref:hirist.tech)

  • Microsoft Azure, Databricks

    1 week ago

    Mumbai, Maharashtra, India Natobotics Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Hi ConnectionsWe are Hiringhashtag#MicrosoftAzure, Digital: Python for Data Science, Digital: Databricks, Digital: PySpark, Azure Data Factory # Immediate Joiners Job Title: Developer Work Location: Mumbai, MH / Hyderabad, TG / Chennai, TN / Bhubaneswar, OR / Bangalore, KA Skill Required: Digital: Microsoft Azure, Digital: Python for Data Science, Digital:...

  • Senior azure databricks engineer

    4 weeks ago

    Mumbai, India MLAI Digital Private Ltd. Full time

    Company DescriptionMLAI_Digital is a Gen AI_Agentic Specialised Organisation(8 Years old) providing Various Cognitive Products well suited for major BFSI Institutions.Offices in US(ohio) , Singapore (PTE), Malaysia , India and Australia(MEL)Domain expertise:- FINANCIAL INSTITUTIONS(Banks & Insurance) and Fraud Analytics for Financial TransactionsRole...

  • Azure Databricks

    2 weeks ago

    Mumbai, India Teamware Solutions Full time

    Job Description Key Responsibilities: - Data Pipeline Development: Design, build, and maintain scalable ETL (Extract, Transform, Load) data pipelines using Azure Databricks, Apache Spark, and Python. - Spark Optimization: Develop and optimize Spark jobs for large-scale data processing on Databricks. Ensure that the jobs run efficiently, leveraging the...

  • Senior Azure Databricks Engineer

    1 week ago

    Mumbai Metropolitan Region, India SWITS DIGITAL Private Limited Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title:Azure Databricks EngineerLocation:PAN IndiaExperience:8–10 years total, with 4+ years relevant experienceJob SummaryWe are looking for a highly skilled Azure Databricks Engineer to join our data engineering team. The ideal candidate will have hands-on expertise in building scalable data pipelines, optimizing big data workloads, and leveraging...

  • Quantiphi - Senior Data Engineer - ETL/Azure Databricks

    2 weeks ago

    Mumbai, India Quantiphi Analytics Full time

    While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life,...

  • Azure Data Engineers with Synapse, Databricks

    2 days ago

    Kolkata, Mumbai, Pune, India Zensar Technologies Limited Full time

    Job Description Description We are seeking an experienced Azure Data Engineer with expertise in Azure Synapse and Databricks to join our dynamic team. The ideal candidate will have 3-6 years of experience in designing and implementing data solutions in the Azure ecosystem. You will play a crucial role in building scalable data pipelines, collaborating with...

  • Solutions Data Architect

    1 week ago

    Mumbai, India Databricks Full time

    As a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...

  • Senior azure engineer

    4 weeks ago

    Mumbai, India MLAI Digital Private Ltd. Full time

    MLAI_Digital is a Gen AI_Agentic Specialised Organisation(8 Years old) providing Various Cognitive Products well suited for major BFSI Institutions. FINANCIAL INSTITUTIONS(Banks & Insurance) and Fraud Analytics for Financial Transactions This is a full-time on-site role for a Senior Databricks Engineer located in Mumbai. As a Data Engineer, you will work...

  • Azure Data Engineer

    7 days ago

    Ahmedabad, Mumbai, Pune, India Technomech Placements Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Job Title:Senior Data Engineer Azure Synapse + Databricks (Immediate Joiners Preferred)Company:A Leading German MNC (via Agile Tech Solutions)Location:Pune (Hybrid)Experience Required:7 to 10 yearsNotice Period:Immediate to 30 Days PreferredJob Description:We are hiring a Senior Data Engineer for a prestigious German multinational client in the automotive...

  • Senior Data Engineer

    2 weeks ago

    Mumbai, Maharashtra, India Oracle Financial Services Software Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Senior Consultant - Databricks Senior Data Engineer – Databricks (Azure/AWS) Role Overview: We are looking for a hands-on Senior Data Engineer experienced in migrating and building large-scale data pipelines on Databricks using Azure or AWS platforms. The role will focus on implementing batch and streaming pipelines, applying the bronze-silver-gold...