Current jobs related to Sr. Data Engineer Azure Databricks - Pune - Fusemachines
-
Azure Data Engineer
4 weeks ago
Pune, Maharashtra, India SR analytics Full time**Job Title: Azure Data Engineer** Location: Pune, India Company: SR Analytics, a Data Analytics and Business Intelligence Services firm We are looking to add a Senior Data Engineer to our team. Responsibilities: Design, construct, and maintain scalable data management systems on Azure Cloud. Develop and optimize data pipelines and architectures using Azure...
-
Sr. Data Engineer Azure Databricks
2 weeks ago
Pune, Maharashtra, India Fusemachines Full timeAbout FusemachinesFusemachines is a leading AI strategy, talent, and education services and products provider. Founded by Sameer Maskey Ph.D., Adjunct Associate Professor at Columbia University, Fusemachines has a core mission of democratizing AI. With a presence in 4 countries (Nepal, United States, Canada, and Dominican Republic and more than 400 full-time...
-
Data Engineer
3 days ago
Pune, Maharashtra, India Wipro Full timeJob DescriptionWelcome to the role of a Data Engineer at Wipro, where you will be working on designing and developing large-scale data processing systems using Azure Databricks.You will be responsible for architecting, implementing, and maintaining these systems to ensure efficient data processing, storage, and analytics.The ideal candidate should have...
-
Azure Databricks
4 weeks ago
Pune, Maharashtra, India Exusia Full timeSr Data Engineers & Tech Leads – Python/Pyspark/Databricks Department: Sales and Delivery Team - EmpowerIndustry: Information Technology & Services, Computer Software, Management...
-
Azure Databricks Engineer
3 weeks ago
Bengaluru, Pune, India People Prime Worldwide Private Limited Full timeJob DescriptionAbout Company :Our client is a trusted global innovator of IT and business services. They help clients transform through consulting, industry solutions, business process services, digital &; IT modernization and managed services. Our client enables them, as well as society, to move confidently into the digital future. We are committed to our...
-
Azure Databricks
2 weeks ago
Pune, India Nityo Infotech Full time17 hours ago **Job Code**: JD-19482 **JOB DESCRIPTION**: Data Engineer(Databricks,NiFi,Microsoft Azure,Python,SQL,Hive,Linux **Experience Required**: 5 - 9 Years **Industry Type**: IT **Employment Type**: Permanent **Location**: India
-
Noida, Mumbai, Pune, India Birlasoft Limited Full timeJob DescriptionYears of experience 8 to 10 Years- Experience in Perform Design, Development & Deployment using Azure Services (Databricks, PySpark, SQL, Data Factory,)- Develop and maintain scalable data pipelines and build new Data Source integrations to support increasing data volume and complexity.- Experience in creating Technical Specification Design,...
-
Azure Data Engineer
5 days ago
Pune, Maharashtra, India VidPro Consultancy Services Full timeJob DescriptionLocation: Hyderabad, Bangalore and PuneExperience: 3-12 YearsWork Mode: HybridMandatory Skills: Python, Pyspark, Airflow, Databricks, ETL, Data Pipelines, and Azure Databricks, Synapse, Data Factory and Data Lake,SQL,ArchitectOverviewWe are seeking a skilled and motivated Data Engineer with experience in Python, SQL, Azure, and cloud-based...
-
SwiftIn-Azure Databricks Lead
4 weeks ago
Pune, Maharashtra, India Nexthire Full timeRole: Azure Databrick Lead Exp-10+ years Location- Pune/Bangalore We are seeking an experienced Azure Databricks Lead to drive the design, development, and implementation of data solutions using Azure Databricks. The ideal candidate will have a deep understanding of cloud-based data platforms, strong leadership skills, and a...
-
Azure Databricks Engineer
4 weeks ago
Pune, Maharashtra, India STEFANINI INDIA PRIVATE LIMITED Full timeJob Description :Responsibilities :- Design, develop, and optimize data pipelines and ETL workflows using Databricks.- Implement scalable data integration solutions for large datasets across diverse data sources.- Build and maintain data architectures for real-time and batch processing.- Collaborate with data scientists, analysts, and stakeholders to ensure...
-
Senior Azure Databricks Engineer
1 day ago
Pune, Chennai, India Citiustech Healthcare Technology Private Limited Full timeAbout the RoleWe are looking for a highly skilled Senior Azure Databricks Engineer to join our team at Citiustech Healthcare Technology Private Limited. As a key member of our engineering team, you will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure.Key Responsibilities:- Design and develop data integration...
-
Azure Data engineer
6 days ago
Pune, Maharashtra, India SMARTWORK IT SERVICES Full timeJob Role : Azure Data Engineer. Job Locations : Pune , Bangalore or Hyderabad. Required Experience : 5 7 Years. Skills : Azure, Azure ADF , Databricks, Azure Synapse, Python , Pyspark. Roles And Responsibilities :- You are detailed reviewing and analyzing structured, semi-structured and unstructured data sources for quality, completeness, and business...
-
Kolkata, Mumbai, Pune, India Zensar Technologies Limited Full timeJob DescriptionDescriptionWe are seeking an experienced Azure Data Engineer with expertise in Azure Synapse and Databricks to join our dynamic team. The ideal candidate will have 3-6 years of experience in designing and implementing data solutions in the Azure ecosystem. You will play a crucial role in building scalable data pipelines, collaborating with...
-
Senior Data Engineer Databricks
3 days ago
Pune, Maharashtra, India Tredence Inc. Full timeAbout the Job:We are seeking a highly skilled Databricks architect to join our team at Tredence Inc. The successful candidate will have 6+ years of IT experience, with 3+ years in data warehousing and ETL projects, and a strong understanding of Databricks Data & AI platform and Databricks Delta Lake Architecture.Responsibilities:Developing Modern Data...
-
Azure Data Engineering Lead
4 weeks ago
Pune, India NewVision Software Full timePosition Summary: We are seeking a talented Data Engineer with a strong background in data engineering to join our team. You will play a key role in designing, building, and maintaining data pipelines using a variety of technologies, with a focus on the Microsoft Azure cloud platform. Responsibilities: - Design, develop, and implement data pipelines using...
-
Azure(Data Engineering
4 weeks ago
Pune, Maharashtra, India OptimHire Full timeAzure Solution Engineer with professional experience in in both application development and data engineering, with extensive experience in architecting, designing, and implementing solutions on the Microsoft Azure platform.Core Responsibilities:Solution Architecture: Design and architect end-to-end solutions on the Microsoft Azure platform, considering...
-
Azure Data Engineering Lead
4 weeks ago
Pune, India NewVision Software Full timePosition Summary:We are seeking a talented Data Engineer with a strong background in data engineering to join our team. You will play a key role in designing, building, and maintaining data pipelines using a variety of technologies, with a focus on the Microsoft Azure cloud platform.Responsibilities:- Design, develop, and implement data pipelines using Azure...
-
Azure Data Engineering Lead
4 weeks ago
Pune, India NewVision Software Full timePosition Summary:We are seeking a talented Data Engineer with a strong background in data engineering to join our team. You will play a key role in designing, building, and maintaining data pipelines using a variety of technologies, with a focus on the Microsoft Azure cloud platform.Responsibilities:- Design, develop, and implement data pipelines using Azure...
-
Azure Data Engineer
4 weeks ago
Hyderabad, Bengaluru, Pune, India Coforge Solutions Private Limited Full timeJob DescriptionWe are Hiring For Azure Data Engineer at Coforge. We are seeking an experienced Data Engineer with expertise in Azure technologies to join our team.Note:- We are looking for an immediate joiner. This is an urgent project, So we are looking for someone who can join immediately.Share Your CV - [Confidential Information]WhatsApp Me For Any...
-
Azure Data Engineering Specialist
2 days ago
Pune, Maharashtra, India IT Full timeAbout the RoleThe Senior Azure Data Engineer will be responsible for building robust and scalable data pipelines, data lakes, and data warehouses to support critical business decisions within the Financial Services domain.Key ResponsibilitiesDesign, develop, and implement complex data pipelines and ETL/ELT processes using Azure Data Factory (ADF) and Azure...
Sr. Data Engineer Azure Databricks
2 weeks ago
About Fusemachines Fusemachines is a leading AI strategy, talent, and education services and products provider.
Founded by Sameer Maskey Ph.D., Adjunct Associate Professor at Columbia University, Fusemachines has a core mission of democratizing AI.
With a presence in 4 countries (Nepal, United States, Canada, and Dominican Republic and more than 400 full-time employees).
Fusemachines seeks to bring its global expertise in AI to transform companies around the world.
About the role This is a remote, contract position responsible for designing, building, and maintaining the infrastructure required for data integration, storage, processing, and analytics (BI, visualization and Advanced Analytics).
We are looking for a skilled Senior Data Engineer with a strong background in Python, SQL, PySpark, Azure, Databricks, Synapse, Azure Data Lake, DevOps and cloud-based large scale data applications with a passion for data quality, performance and cost optimization.
The ideal candidate will develop in an Agile environment, contributing to the architecture, design, and implementation of Data products in the Aviation Industry , including migration from Synapse to Azure Data Lake.
This role involves hands-on coding, mentoring junior staff and collaboration with multi-disciplined teams to achieve project objectives.
Qualification & Experience Must have a full-time Bachelor's degree in Computer Science or similar At least 5 years of experience as a data engineer with strong expertise in Databricks, Azure, DevOps, or other hyperscalers.
5+ years of experience with Azure DevOps, GitHub.Proven experience delivering large scale projects and products for Data and Analytics, as a data engineer, including migrations.
Following certifications: Databricks Certified Associate Developer for Apache Spark Databricks Certified Data Engineer Associate Microsoft Certified: Azure Fundamentals Microsoft Certified: Azure Data Engineer Associate Microsoft Exam: Designing and Implementing Microsoft DevOps Solutions (nice to have) Required skills/Competencies Strong programming Skills in one or more languages such as Python (must have), Scala, and proficiency in writing efficient and optimized code for data integration, migration, storage, processing and manipulation.
Strong understanding and experience with SQL and writing advanced SQL queries.
Thorough understanding of big data principles, techniques, and best practices.
Strong experience with scalable and distributed Data Processing Technologies such as Spark/ PySpark (must have: experience with Azure Databricks ), DBT and Kafka, to be able to handle large volumes of data.
Solid Databricks development experience with significant Python, PySpark, Spark SQL, Pandas, NumPy in Azure environment.
Strong experience in designing and implementing efficient ELT/ETL processes in Azure and Databricks and using open source solutions being able to develop custom integration solutions as needed.
Skilled in Data Integration from different sources such as APIs, databases, flat files, event streaming.
Expertise in data cleansing, transformation, and validation.
Proficiency with Relational Databases (Oracle, SQL Server, MySQL, Postgres, or similar) and NonSQL Databases (MongoDB or Table).
Good understanding of Data Modeling and Database Design Principles.
Being able to design and implement efficient database schemas that meet the requirements of the data architecture to support data solutions.
Strong experience in designing and implementing Data Warehousing, data lake and data lake house, solutions in Azure and Databricks.
Good experience with Delta Lake, Unity Catalog, Delta Sharing, Delta Live Tables (DLT).
Strong understanding of the software development lifecycle (SDLC), especially Agile methodologies.
Strong knowledge of SDLC tools and technologies Azure DevOps and GitHub, including project management software (Jira, Azure Boards or similar), source code management (GitHub, Azure Repos or similar), CI/CD system (GitHub actions, Azure Pipelines, Jenkins or similar) and binary repository manager (Azure Artifacts or similar).
Strong understanding of DevOps principles, including continuous integration, continuous delivery (CI/CD), infrastructure as code (IaC – Terraform, ARM including hands-on experience), configuration management, automated testing, performance tuning and cost management and optimization. Strong knowledge in cloud computing specifically in Microsoft Azure services related to data and analytics, such as Azure Data Factory, Azure Databricks , Azure Synapse Analytics , Azure Data Lake , Azure Stream Analytics, SQL Server, Azure Blob Storage, Azure Data Lake Storage, Azure SQL Database, etc.
Experience in Orchestration using technologies like Databricks workflows and Apache Airflow.
Strong knowledge of data structures and algorithms and good software engineering practices.
Proven experience migrating from Azure Synapse to Azure Data Lake, or other technologies.
Strong analytical skills to identify and address technical issues, performance bottlenecks, and system failures.
Proficiency in debugging and troubleshooting issues in complex data and analytics environments and pipelines.
Good understanding of Data Quality and Governance, including implementation of data quality checks and monitoring processes to ensure that data is accurate, complete, and consistent. Experience with BI solutions including PowerBI is a plus.
Strong written and verbal communication skills to collaborate and articulate complex situations concisely with cross-functional teams, including business users, data architects, DevOps engineers, data analysts, data scientists, developers, and operations teams.
Ability to document processes, procedures, and deployment configurations.
Understanding of security practices, including network security groups, Azure Active Directory, encryption, and compliance standards.
Ability to implement security controls and best practices within data and analytics solutions, including proficient knowledge and working experience on various cloud security vulnerabilities and ways to mitigate them. Self-motivated with the ability to work well in a team, and experienced in mentoring and coaching different members of the team.
A willingness to stay updated with the latest services, Data Engineering trends, and best practices in the field.
Comfortable with picking up new technologies independently and working in a rapidly changing environment with ambiguous requirements.
Care about architecture, observability, testing, and building reliable infrastructure and data pipelines.
Responsibilities Architect, design, develop, test and maintain high-performance, large-scale, complex data architectures, which support data integration (batch and real-time, ETL and ELT patterns from heterogeneous data systems: APIs and platforms), storage (data lakes, warehouses, data lake houses, etc), processing, orchestration and infrastructure.
Ensuring the scalability, reliability, and performance of data systems, focusing on Databricks and Azure.
Contribute to detailed design, architectural discussions, and customer requirements sessions.
Actively participate in the design, development, and testing of big data products..
Construct and fine-tune Apache Spark jobs and clusters within the Databricks platform.
Migrate out of Azure Synapse to Azure Data Lake or other technologies.
Assess best practices and design schemas that match business needs for delivering a modern analytics solution (descriptive, diagnostic, predictive, prescriptive).
Design and implement data models and schemas that support efficient data processing and analytics.
Design and develop clear, maintainable code with automated testing using Pytest, unittest, integration tests, performance tests, regression tests, etc.
Collaborating with cross-functional teams and Product, Engineering, Data Scientists and Analysts to understand data requirements and develop data solutions, including reusable components meeting product deliverables. Evaluating and implementing new technologies and tools to improve data integration, data processing, storage and analysis.
Evaluate, design, implement and maintain data governance solutions: cataloging, lineage, data quality and data governance frameworks that are suitable for a modern analytics solution, considering industry-standard best practices and patterns.
Continuously monitor and fine-tune workloads and clusters to achieve optimal performance.
Provide guidance and mentorship to junior team members, sharing knowledge and best practices.
Maintain clear and comprehensive documentation of the solutions, configurations, and best practices implemented.
Promote and enforce best practices in data engineering, data governance, and data quality.
Ensure data quality and accuracy.
Design, Implement and maintain data security and privacy measures.
Be an active member of an Agile team, participating in all ceremonies and continuous improvement activities, being able to work independently as well as collaboratively.
Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.
Powered by JazzHR
