Data Engineer Lead
3 days ago
Job Title: Data Engineer Lead
Company Overview:
We are Everstream Analytics, a dynamic company seeking a skilled Data Engineer Lead to join our team. Our company specializes in providing cutting-edge data engineering solutions for the logistics and supply chain industry.
About the Role:
As a Data Engineer Lead, you will be responsible for designing and building efficient data systems, ensuring data integrity, and supporting business insights by wrangling, analyzing, and documenting complex data sets.
Key Responsibilities:
- Data Pipeline Development: Build and maintain scalable and efficient data pipelines using Python and PySpark. Write SQL queries to extract, transform, and load data from Amazon Athena. Work on ingestion of large-scale data from various sources using AWS tools (e.g., S3, Redshift, Glue, Lambda, Athena, etc.) with an emphasis on ensuring data quality and integrity.
- Data Exploration and Wrangling: Utilize NLP techniques and regular expressions to explore, clean, and process unstructured datasets. Perform data wrangling to structure raw data for further analysis using Pandas or PySpark as appropriate. Able to apply intermediate SQL techniques for data manipulation and aggregation, optimizing queries for large datasets. Handle geospatial data and processes related to geolocation, geofencing, and entity deduplication (e.g., address normalization and deduplication of entities such as companies).
- Exploratory Data Analysis (EDA): Conduct exploratory data analysis using Python, PySpark, and SQL to discover patterns, trends, and anomalies within large data sets, particularly in the logistics and supply chain management domain. Leverage knowledge of data from logistics, geolocation, and supply chain to identify key insights and make data-driven recommendations.
- Data Evaluation & Quality Assurance: Identify and troubleshoot data quality issues related to geolocation, company addresses, logistics, and supply chain data, ensuring data consistency, accuracy, and reliability.
Required Qualifications:
- Proficiency in Python: Must be highly skilled in Python for data manipulation, ETL (Extract, Transform, Load) processes, and data pipeline development. Ability to write and optimize SQL queries for data extraction, transformation, and analysis from large relational databases.
- Experience with PySpark: Hands-on experience with PySpark for big data processing is essential. Solid experience with AWS tools (e.g., S3, Glue, Lambda, Redshift, etc.) for data ingestion, processing, and storage. Understanding of basic Natural Language Processing (NLP) techniques and hands-on experience with regular expressions for data parsing and analysis.
- Domain Expertise: Knowledge or experience in logistics and supply chain management will be a significant advantage.
Estimated Salary Range:
Our estimated salary range for this position is between $120,000 - $160,000 per year, depending on experience and qualifications.
-
Data Analytics Engineer
4 weeks ago
Pune, Maharashtra, India Data Dynamics Full timeAbout Data Dynamics">Data Dynamics is a leading data-driven company that leverages cutting-edge technologies to drive business growth. Our team of experts collaborates with clients across various industries to deliver innovative solutions.Job Summary">We are seeking an experienced Senior Machine Learning Specialist to join our team. As a key member, you will...
-
Senior Software Engineer Lead
3 weeks ago
Pune, Maharashtra, India Data Labs Full timeJob Title: Senior Software Engineer LeadWe are seeking an experienced Senior Software Engineer Lead to join our team at Data Labs. This is a full-time position based in Pune.About the Role:As a Senior Software Engineer Lead, you will be responsible for leading the frontend development team and collaborating with the backend team to deliver high-quality...
-
Data Engineering Lead
3 weeks ago
Pune, Maharashtra, India Digital Group Infotech Pvt Ltd Full timeJob Description: We are seeking a Data Engineering Lead to join our team at Digital Group Infotech Pvt Ltd. The ideal candidate will have 5-10 years of experience in leading data engineering teams, designing and developing data models, ETL pipelines, and infrastructure for reporting, analytics, and data science.The successful candidate will have extensive...
-
Data Engineering Lead
3 weeks ago
Pune, Maharashtra, India M&G Global Services Full timeJob Title: Data Engineering LeadWe are seeking a highly skilled software engineer with extensive experience in developing and implementing data engineering solutions using Azure technologies.Key Responsibilities:Develop and implement software engineering skills with experience in SQL, C# and the .Net Framework.Work with data on Azure including SQL and no-sql...
-
Data Engineer Lead
3 weeks ago
Pune, Maharashtra, India Mindpool Technologies Full timeAbout the RoleWe are seeking a highly skilled Data Engineer Lead to join our EDWBI platform team at Mindpool Technologies in Pune. As a key member of our team, you will be responsible for building and maintaining a robust data validation framework, collaborating with cross-functional teams, and ensuring high-quality data delivery.
-
Senior Data Engineering Lead
3 weeks ago
Pune, Maharashtra, India Sampoorna Consultants Pvt. Ltd Full timeJob Title: Senior Data Engineering LeadSampoorna Consultants Pvt. Ltd is looking for a highly skilled Senior Data Engineering Lead to lead our data engineering team. If you have a strong background in data engineering, data quality, and data validation, and experience with data processing concepts like mapping documents and complex data relationships, we...
-
Data Engineering Team Lead
3 weeks ago
Pune, Maharashtra, India DFCS Technologies Full time**Job Title:** Data Engineering Team Lead**Location:** [Location: Assuming based on the job title and company]**Salary:** $180,000 - $220,000 per yearDFCS Technologies is seeking an experienced Data Engineering Team Lead to lead and mentor a team of data engineers, providing technical leadership and guidance to design, develop, and manage scalable, reliable,...
-
Cloud Data Engineering Lead
3 weeks ago
Pune, Maharashtra, India Tata Consultancy Services Full timeWe are seeking an experienced Cloud Data Engineering Lead to join our team at Tata Consultancy Services. This is a challenging and rewarding role that requires strong technical expertise, excellent communication skills, and the ability to lead cross-functional teams.About the RoleThis is a senior leadership position responsible for overseeing the development...
-
Senior Data Engineering Lead
4 weeks ago
Pune, Maharashtra, India Synechron Full timeWe are seeking a seasoned Senior Data Engineering Lead to spearhead the design and implementation of complex data processing applications within our organization. With 7 to 10 years of hands-on experience in Ab Initio and Apache Spark, you will be an expert in leveraging these technologies to deliver scalable and performance-driven data solutions.This senior...
-
Senior Data Engineering Lead
2 days ago
Pune, Maharashtra, India NewVision Software Full timeAbout the RoleWe are seeking an experienced Senior Data Engineering Lead to join our team at NewVision Software. In this role, you will be responsible for designing, developing, and managing data engineering solutions on the Microsoft Azure platform.Key ResponsibilitiesData Pipeline Development:Design and implement robust and scalable data pipelines using...
-
Data Engineering Lead
3 weeks ago
Pune, Maharashtra, India Infinity career edge Full timeAbout Infinity career edge">We are a cutting-edge technology firm dedicated to delivering innovative solutions. Our team is passionate about harnessing the power of big data and cloud platforms to drive business growth.Job OverviewThis role offers an exceptional opportunity for a highly skilled Data Engineer to lead our ETL pipeline development, ensuring...
-
Network Engineer Lead
4 weeks ago
Pune, Maharashtra, India Yotta Data Services Private Limited Full timeAbout the Role:We are seeking a highly skilled Network Engineer to lead our team at Yotta Data Services Private Limited.As a Network Engineer Lead, you will be responsible for designing, implementing, and managing our network infrastructure.The ideal candidate will have experience in daily network operational tasks, including configurations, communication...
-
Data Engineering Practice Lead
5 days ago
Pune, Maharashtra, India ACL Digital Full timeAbout ACL DigitalAt ACL Digital, we are seeking an experienced Data Engineering Practice Lead to join our team. The successful candidate will be responsible for defining and executing the strategic direction of our data engineering practice.The key role responsibilities include:Leading a team of data engineers to develop and maintain cutting-edge data...
-
Data Engineering Lead
4 weeks ago
Pune, Maharashtra, India vConstruct Private Limited Full timeVConstruct Private Limited is a dynamic Construction Technology company based in Pune, seeking an experienced Data Engineering Lead to join its Data Science and Analytics team.The ideal candidate will be responsible for designing, developing, and maintaining the company's data infrastructure, ensuring data integrity, and supporting various data-driven...
-
Azure Data Engineering Lead
3 weeks ago
Pune, Maharashtra, India Amk technologies Full timeAbout Amk Technologies: As a leading technology company, we provide innovative solutions to our clients.Job Title: Azure Data Engineering LeadThe successful candidate will lead the migration of traditional ETL processes from on-premise systems to the Azure cloud landscape. The ideal candidate will have strong experience with ETL processes, including...
-
Senior Data Engineering Lead
3 weeks ago
Pune, Maharashtra, India Synechron Full timeJob OverviewWe are seeking a highly skilled Senior Data Engineering Lead to lead the design and development of complex data processing applications within our organization.The ideal candidate will have 7 to 10 years of experience in leveraging the strengths of both Ab Initio and Apache Spark to deliver scalable and performance-driven data solutions.
-
Big Data Engineer Lead
2 weeks ago
Pune, Maharashtra, India LTIMindtree Full timeLTI Mindtree is looking for an experienced Big Data Engineer Lead to lead our team in Chennai, Pune, or Noida.Salary: ₹1,200,000 - ₹2,500,000 per annumThe successful candidate will have 5 to 12 years of experience in Big Data, with expertise in Spark, Pyspark, and resource utilization of instances.Responsibilities will include optimizing configurations,...
-
Pune, Maharashtra, India eClerx Full timeJob OvervieweClerx is seeking a highly skilled and experienced Data Engineering Lead to join our team. The ideal candidate will have a strong background in designing and implementing data infrastructure, data pipelines, and data lakes.About the RoleWe are looking for a talented individual to lead our data engineering efforts. As a Data Engineering Lead, you...
-
Chief Data Engineering Lead
3 weeks ago
Pune, Maharashtra, India Innovalus Technologies Pvt Ltd Full timeJob OverviewInnovalus Technologies Pvt Ltd is seeking an experienced Chief Data Engineering Lead to drive the design, development, and implementation of scalable and robust systems. This role requires a visionary leader who can effectively collaborate with international teams and stakeholders.Salary Estimate: ₹ 2000000 - ₹ 4000000 per annum (dependent on...
-
Data Engineering Lead
4 weeks ago
Pune, Maharashtra, India Wipro Full timeJob Title: Data Engineering Lead - Big Data Platform ArchitectAre you an experienced data engineering professional looking to take your career to the next level? We are seeking a skilled Data Engineering Lead - Big Data Platform Architect to join our team at Wipro.Estimated Salary: ₹20,00,000 - ₹30,00,000 per annumWe offer a highly competitive salary...
