Data Engineer
3 weeks ago
Job Title: Data Engineer (Azure Databricks)
Location: Bangalore / Hybrid
Job Type: Full-time
Job Description:
Experienced Data engineer & Solutions designer with a deep understanding of data solution design and implementation, seeking a challenging role to leverage my expertise in data architecture, cloud migrations, and solution design, aiming to drive enterprise-wide transformations and deliver efficient, scalable, and cost-effective solutions.
- Highly analytical team player, with the aptitude for prioritization of needs/risks.
- Has 6+ years of overall experience and 5+ years in taming Big Data using Azure Databricks.
- A rich understanding of exploiting Azure resources such as Azure Data factory, Synapse, Databricks, Logic Apps, Key Vaults, Data lakes, and SQL DBs.
- Fluent in performing Business Intelligence operations: Data analytics and visual analytics using MS Power BI & Tableau.
- Encouraging and engaging with Team & Non-Team members in the exploration of new concepts, creating POCs, studying new technologies & certification prep.
- Excellent communication & interpersonal skills with strong analytical, team building, problem-solving, and organizational capabilities.
Software Skills:
Cloud: Azure Stack - Azure Data factory, Datalake, keyvaults , Logic Apps
Data warehousing: Databricks, Synapse
Programming Language: Pyspark, Spark SQL
CI/CD: Azure DevOps, GitHub
Frontend: Power BI, Tableau
PROJECTS
· Telemetry Data Engineering for UX & Product Enhancement: Engineered a telemetry data aggregation solution using Azure Event Hubs, Databricks, and Logic Apps, enabling the collection and processing of device usage metrics (battery, thermal, application performance). Implemented a robust data pipeline with ADLS Gen2 storage and Azure Data Factory to facilitate Machine Learning and Product Design teams in enhancing product quality and user experience. Contributed to customer service efficiency by integrating device history analytics, streamlining parts availability, and service readiness at HP service centers.
· Financial Data Engineering for Warranty Analysis: Developed and managed the data processing pipeline for warranty information of HP products, creating actionable consumption tables for business analytics. Enhanced data fidelity and accessibility for warranty-related decision-making by leveraging Azure services and Databricks for real-time Streaming Delta tables.
· Cloud Infrastructure Cost Optimization: Led a technical team to reduce cloud infrastructure expenses by 67%, optimizing performance, consolidating pipelines, and parameterizing runtime inputs. Spearheaded the strategic decommissioning of underutilized Databricks, Snowflake, and Synapse workspaces, migrating simple task workflows to Azure Data Factory (ADF) and eliminating the need for costly data warehousing solutions.
· GSK Attendance Analysis - Collaborated with Cisco to design an API, fetching real-time data about employees connected to the office network. Leveraged Azure Data Factory and Databricks to process data, yielding insights into global attendance patterns. Assisted GSK in real estate optimization.
· Capital Asset Management - Orchestrated the end-to-end development of CAM, a cloud-based system, leading data transformation processes and ETL flows with Azure tools.
· Finance Operations at Anheuser-Busch InBev - Managed sensitive financial data operations. Transformed SAP data using Aecorsoft CDC and processed it through Databricks.
· Solution Development at DecisionAttic for TVS Motors - Offered comprehensive technical solutions from data extraction to dashboard creation, addressing varied business domains.
Certifications:
-
Lead data engineer
4 days ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Lead Data Engineer
2 weeks ago
India Wavicle Data Solutions Full timeJob Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...
-
Lead Data Engineer
1 week ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Aws data engineer
18 hours ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on Py Spark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 weeks ago
india ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
2 weeks ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...
-
Lead Data Engineer: Strategic Data Architect
2 weeks ago
India Wavicle Data Solutions Full timeWavicle Data Solutions is seeking an experienced Lead Data Engineer to play a pivotal role in architecting and implementing data solutions.About the RoleThis position offers a competitive salary of $180,000 - $220,000 per annum, commensurate with experience. Key ResponsibilitiesDesign and develop scalable and efficient data pipelines using Python and...
-
india Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...
-
india Wavicle Data Solutions Full timeJob Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...
-
india Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...
-
Technical Lead
2 months ago
india Data Axle India Full timeData Axle Inc. has been an industry leader in data, marketing solutions, sales, and research for over 50 years in the USA. Data Axle now has an established strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and by leveraging...
-
Lead Data Scientist
2 months ago
india Data Axle India Full timeAbout Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales, and research for over 50 years in the USA. Data Axle now has an established strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and...
-
Lead Data Scientist
2 months ago
india Data Axle India Full timeAbout Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales, and research for over 50 years in the USA. Data Axle now has an established strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and by...
-
Data engineer
1 month ago
India Ryden Global Full timeWe're #hiring #Data Engineers for our USA based client at offshore in India (must support afternoon shift in IST). Role Description This is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics...
-
Data Engineer
1 month ago
india Ryden Global Full timeWe're #hiring #DataEngineers for our USA based client at offshore in India (must support afternoon shift in IST). Role DescriptionThis is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics on a...
-
Data Engineer
1 month ago
India Ryden Global Full timeWe're #hiring #DataEngineers for our USA based client at offshore in India (must support afternoon shift in IST). Role Description This is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics on a...
-
Data Engineer
1 month ago
india Ryden Global Full timeWe're #hiring #DataEngineers for our USA based client at offshore in India (must support afternoon shift in IST). Role Description This is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics on a...
