Manager-Data Engineering-Big Data Engineering
4 days ago
About the Role
We are seeking a highly skilled Lead Data Engineer to drive end-to-end data engineering initiatives, lead cross-functional teams, and deliver scalable, cloud-based data solutions. The ideal candidate will bring deep technical expertise, strong leadership, and the ability to collaborate effectively with global stakeholders in a fast-paced environment.
Key Responsibilities
Leadership & Team Management
- Lead and mentor a team of onshore and offshore data engineers to ensure high-quality deliverables.
- Provide technical direction, coaching, and knowledge sharing to foster team growth and capability building.
- Establish and enforce engineering best practices, coding standards, and reusable frameworks.
- Champion innovation, continuous learning, and thought leadership within the data engineering function.
Project Delivery & Execution
- Oversee end-to-end project delivery, ensuring timely, high-quality execution aligned with project objectives.
- Define high-level solution designs, data architectures, and ETL/ELT frameworks for cloud-based data platforms.
- Drive development, code reviews, unit testing, and deployment to production environments.
- Ensure optimal performance, scalability, and reliability across data pipelines and systems.
Stakeholder Communication & Collaboration
- Collaborate with clients, product owners, business leaders, and offshore teams to gather requirements and define technical solutions.
- Communicate project updates, timelines, risks, and technical concepts effectively to both technical and non-technical stakeholders.
- Act as the primary point of contact for client technical discussions, solution design workshops, and progress reviews.
Risk & Issue Management
- Proactively identify project risks, dependencies, and issues; develop and execute mitigation plans.
- Ensure governance, compliance, and alignment with organizational standards and methodologies.
Must-Have Skills
- 12+ years of experience in Data Engineering with proven experience in delivering enterprise-scale projects.
- Strong expertise in Big Data concepts, distributed systems, and cloud-native architectures.
- Proficiency in Snowflake, SQL, and a wide range of AWS services (Glue, EMR, S3, Aurora, RDS, Lambda, Step Functions).
- Hands-on experience with Python, PySpark, and building cloud-based microservices.
- Strong problem-solving, analytical skills, and end-to-end ownership mindset.
- Proven ability to work in Agile/Scrum environments with iterative delivery cycles.
- Exceptional communication, leadership, and stakeholder management skills.
- Demonstrated capability in leading onshore–offshore teams and coordinating multi-region delivery efforts.
Good-to-Have Skills
- Experience with DevOps tools (Jenkins, Git, GitHub/GitLab) and CI/CD pipeline implementation.
- Experience with cloud migration and large-scale modernization projects.
- Familiarity with the US insurance/reinsurance domain + P&C Insurance knowledge
- Knowledge of Data Vault 2.0 and modern data modeling techniques.
About the Role
We are seeking a highly skilled Lead Data Engineer to drive end-to-end data engineering initiatives, lead cross-functional teams, and deliver scalable, cloud-based data solutions. The ideal candidate will bring deep technical expertise, strong leadership, and the ability to collaborate effectively with global stakeholders in a fast-paced environment.
Key Responsibilities
Leadership & Team Management
- Lead and mentor a team of onshore and offshore data engineers to ensure high-quality deliverables.
- Provide technical direction, coaching, and knowledge sharing to foster team growth and capability building.
- Establish and enforce engineering best practices, coding standards, and reusable frameworks.
- Champion innovation, continuous learning, and thought leadership within the data engineering function.
Project Delivery & Execution
- Oversee end-to-end project delivery, ensuring timely, high-quality execution aligned with project objectives.
- Define high-level solution designs, data architectures, and ETL/ELT frameworks for cloud-based data platforms.
- Drive development, code reviews, unit testing, and deployment to production environments.
- Ensure optimal performance, scalability, and reliability across data pipelines and systems.
Stakeholder Communication & Collaboration
- Collaborate with clients, product owners, business leaders, and offshore teams to gather requirements and define technical solutions.
- Communicate project updates, timelines, risks, and technical concepts effectively to both technical and non-technical stakeholders.
- Act as the primary point of contact for client technical discussions, solution design workshops, and progress reviews.
Risk & Issue Management
- Proactively identify project risks, dependencies, and issues; develop and execute mitigation plans.
- Ensure governance, compliance, and alignment with organizational standards and methodologies.
Must-Have Skills
- 12+ years of experience in Data Engineering with proven experience in delivering enterprise-scale projects.
- Strong expertise in Big Data concepts, distributed systems, and cloud-native architectures.
- Proficiency in Snowflake, SQL, and a wide range of AWS services (Glue, EMR, S3, Aurora, RDS, Lambda, Step Functions).
- Hands-on experience with Python, PySpark, and building cloud-based microservices.
- Strong problem-solving, analytical skills, and end-to-end ownership mindset.
- Proven ability to work in Agile/Scrum environments with iterative delivery cycles.
- Exceptional communication, leadership, and stakeholder management skills.
- Demonstrated capability in leading onshore–offshore teams and coordinating multi-region delivery efforts.
Good-to-Have Skills
- Experience with DevOps tools (Jenkins, Git, GitHub/GitLab) and CI/CD pipeline implementation.
- Experience with cloud migration and large-scale modernization projects.
- Familiarity with the US insurance/reinsurance domain + P&C Insurance knowledge
- Knowledge of Data Vault 2.0 and modern data modeling techniques.
Qualifications
Educational Qualifications
Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.
-
Data Engineer
1 week ago
Pune, Maharashtra, India Jash Data Sciences Full timeDo you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team Then come join our high-energy team of passionate data people. Jash Data Sciences is the right place for you.We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India.We believe in continuous learning and...
-
Big Data Engineer
1 week ago
Pune, Maharashtra, India Vodafone Full timeDesign, develop, and maintain scalable data pipelines for processing structured and unstructured data. Build and optimise data storage solutions including distributed file systems, NoSQL databases, and data warehouses. Utilise big data frameworks such as Apache Hadoop, Spark, Flink, and Kafka to develop data processing workflows. Ensure high performance and...
-
Data Engineer
1 week ago
Pune, Maharashtra, India Capgemini Engineering Full timeJob DescriptionKey Responsibilities:Design and develop data ingestion pipelines.Work on ETL/ELT processes for large-scale data systems.Good experince in Machine LearningCollaborate with cross-functional teams to ensure data quality and performance.Required Skills:PythonETLDatabricksPySparkSQLMachine LearningWorks in the area of Software Engineering, which...
-
Pune, Maharashtra, India EXL Full timeAbout The RoleWe are seeking a highly skilled Lead Data Engineer to drive end-to-end data engineering initiatives, lead cross-functional teams, and deliver scalable, cloud-based data solutions. The ideal candidate will bring deep technical expertise, strong leadership, and the ability to collaborate effectively with global stakeholders in a fast-paced...
-
Data Science Engineer
2 days ago
Pune, Maharashtra, India Capgemini Engineering Full timeChoosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and...
-
Big Data Developer
1 week ago
Pune, Maharashtra, India airtel Full timeJob Title:Big Data Developer (Manager)Location: PuneExperience: 3-5 yearsJob Description:Looking for an experienced Big Data Developer to develop, maintain, and optimize our big data solutions. The ideal candidate will have strong expertise in Java, Spark, API development, and a deep understanding of data engineering and big data tools.Key...
-
Data Engineer
2 days ago
Pune, Maharashtra, India Tlc Engineering Solutions Ltd Full timeRole DescriptionThis is a full-time, on-site Data Engineer role at Toyota Material Handling India Pvt. Ltd. The Data Engineer will be responsible for designing, building, and maintaining data pipelines and ETL processes to support business operations. The role includes developing and implementing efficient data models, enabling data warehousing solutions,...
-
Data Modeler
2 weeks ago
Pune, Maharashtra, India Capgemini Engineering Full time ₹ 5,00,000 - ₹ 25,00,000 per yearChoosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and...
-
Lead Big Data Engineer Manager
4 days ago
Pune, Maharashtra, India CIRCANA Full timeWhat will you be doing?As a part of this team, youll work alongside the brightest minds. As a Big Data Lead evaluate domain, financial and technical feasibility of solution ideas with help of all key stakeholders.Design, develop, and maintain highly scalable data processing applications.Job Responsibilities:Write efficient, reusable and well documented...
-
Big Data Specialist
2 days ago
Pune, Maharashtra, India Emergys Full timeWork Location:Pune(Work-from-Office)Notice Period:Immediate to 30 DaysExperience:8+ YearsMust Have Skills:Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.Data Modeling & Integration – Strong ability to build efficient data models and integrate data from multiple sources.Big...