Senior Data Engineer
2 days ago
Job Title: Data Engineer (Azure Databricks)
Location: Pune
Job Type: Full-time
Job Description:
Experienced Data engineer & Solutions designer with a deep understanding of data solution design and implementation, seeking a challenging role to leverage my expertise in data architecture, cloud migrations, and solution design, aiming to drive enterprise-wide transformations and deliver efficient, scalable, and cost-effective solutions.
- Highly analytical team player, with the aptitude for prioritization of needs/risks.
- Has 6+ years of overall experience and 5+ years in taming Big Data using Azure Databricks.
- A rich understanding of exploiting Azure resources such as Azure Data factory, Synapse, Databricks, Logic Apps, Key Vaults, Data lakes, and SQL DBs.
- Fluent in performing Business Intelligence operations: Data analytics and visual analytics using MS Power BI & Tableau.
- Encouraging and engaging with Team & Non-Team members in the exploration of new concepts, creating POCs, studying new technologies & certification prep.
- Excellent communication & interpersonal skills with strong analytical, team building, problem-solving, and organizational capabilities.
Software Skills:
Cloud: Azure Stack - Azure Data factory, Datalake, keyvaults , Logic Apps
Data warehousing: Databricks, Synapse
Programming Language: Pyspark, Spark SQL
CI/CD: Azure DevOps, GitHub
Frontend: Power BI, Tableau
PROJECTS
· Telemetry Data Engineering for UX & Product Enhancement: Engineered a telemetry data aggregation solution using Azure Event Hubs, Databricks, and Logic Apps, enabling the collection and processing of device usage metrics (battery, thermal, application performance). Implemented a robust data pipeline with ADLS Gen2 storage and Azure Data Factory to facilitate Machine Learning and Product Design teams in enhancing product quality and user experience. Contributed to customer service efficiency by integrating device history analytics, streamlining parts availability, and service readiness at HP service centers.
· Financial Data Engineering for Warranty Analysis: Developed and managed the data processing pipeline for warranty information of HP products, creating actionable consumption tables for business analytics. Enhanced data fidelity and accessibility for warranty-related decision-making by leveraging Azure services and Databricks for real-time Streaming Delta tables.
· Cloud Infrastructure Cost Optimization: Led a technical team to reduce cloud infrastructure expenses by 67%, optimizing performance, consolidating pipelines, and parameterizing runtime inputs. Spearheaded the strategic decommissioning of underutilized Databricks, Snowflake, and Synapse workspaces, migrating simple task workflows to Azure Data Factory (ADF) and eliminating the need for costly data warehousing solutions.
· GSK Attendance Analysis - Collaborated with Cisco to design an API, fetching real-time data about employees connected to the office network. Leveraged Azure Data Factory and Databricks to process data, yielding insights into global attendance patterns. Assisted GSK in real estate optimization.
· Capital Asset Management - Orchestrated the end-to-end development of CAM, a cloud-based system, leading data transformation processes and ETL flows with Azure tools.
· Finance Operations at Anheuser-Busch InBev - Managed sensitive financial data operations. Transformed SAP data using Aecorsoft CDC and processed it through Databricks.
· Solution Development at DecisionAttic for TVS Motors - Offered comprehensive technical solutions from data extraction to dashboard creation, addressing varied business domains.
Certifications:
- Databricks Certified Data Engineer Associate
- Databricks Certified Data Analyst Associate
-
india Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic Apps,...
-
india Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...
-
Senior Data Integration Engineer
7 hours ago
India Senior Data Integration Engineer Full timeMust Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic...
-
Senior Data Engineer
1 month ago
India iO Associates - UKEU Full timeJob Title: Senior Data Engineer Location: Mohali Salary: Negotiable on the current package iO Associates is looking for a Senior Data Engineer in Databricks, Python, Scala, Azure Synapse, and Azure Data Factory to join our dynamic team. Responsibilities include developing and troubleshooting complex data pipelines using Databricks, PySpark, Synapse,...
-
Lead data engineer
6 days ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Senior Data Engineer
3 weeks ago
India iO Associates - UKEU Full timeJob Title: Senior Data Engineer Location: Mohali Salary: up to 30 LPA iO Associates is looking for a Senior Data Engineer in Databricks, Python, Scala, Azure Synapse, and Azure Data Factory to join our dynamic team. Responsibilities include developing and troubleshooting complex data pipelines using Databricks, PySpark, Synapse, ADF, ADLS...
-
Lead Data Engineer
2 weeks ago
India Wavicle Data Solutions Full timeJob Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...
-
Lead Data Engineer
2 weeks ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Senior data engineer
3 weeks ago
India Synechron Full timeGood-day, We have an immediate opportunity for the Senior Data Engineer Job Role: Senior Data Engineer Job Location: Bangalore BCIT Experience- 7 to 10 years Notice : upto 15 Days. About Company: At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and...
-
Senior data engineer
1 month ago
India Recmatrix Consulting Full timeJob Title: Senior Data Engineer Location: Remote within India Employment Type: Contract Experience Level: 6+ years Position Overview: We are seeking a Senior Data Warehouse Engineer to join our Digital Business team. This role involves building and managing data warehouses to support various business functions such as Sales, Marketing, and...
-
Aws data engineer
3 days ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on Py Spark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
Senior data engineer
3 weeks ago
India Linuxpect Private Limited Full timeWe are seeking an experienced and skilled Senior Data Engineer with 5 to 10 years of hands-on experience in building and managing data pipelines and big data infrastructure. The ideal candidate will have a strong technical background in data engineering tools, big data ecosystems, and programming languages. This role will focus on designing, implementing,...
-
Senior Data Engineer
2 months ago
india ValueLabs Full timeDear Aspirants, We at ValueLabs have an Opening for Senior Azure Data Engineer role. Below is the JD for the same. Role: Sr Data Engineer Experience: 5+ Years Preferable Immediate Joiners The Azure Data Engineer is responsible for the design, development, and maintenance of data pipelines, data warehouses, and data lakes for the organization. This role...
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 weeks ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 weeks ago
india ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...
-
Senior Data Engineer
3 weeks ago
india Varcus infinity Full timeJob Title: Senior Data Engineer Location: India (Permanent Work From Home) Salary: ₹30 - ₹40 LPA (flexible for exceptional candidates based on experience) Experience Level: Minimum 5 years, with preference for candidates with e-commerce experience (Note: Candidate must be available to work in PST time zone and able to join within a...
-
Senior Data Engineer
3 weeks ago
India Varcus infinity Full time € 30,00,000 - € 40,00,000Job Title: Senior Data Engineer Location: India (Permanent Work From Home) Salary: ₹30 - ₹40 LPA (flexible for exceptional candidates based on experience) Experience Level: Minimum 5 years, with preference for candidates with e-commerce experience (Note: Candidate must be available to work in PST time zone and able to join within a month.) ...
