Lead Data Engineer

4 weeks ago

Mumbai, India Yourhrpower Full time
Location: Mumbai Experience: 4yrs to 7yrs Advanced SQL, Python and associated libraries like Pandas, Numpy etc., Pyspark , Shell scripting, Data- Modelling, Big data, Hadoop , Hive, ETL pipelines and IaC tools like Terraform etc. Role: Lead Data Engineer Technologies / Skills: Responsibilities:

Proven success in communicating with users, other technical teams, and senior management to collect requirements, describe data modeling decisions and develop data engineering strategy.

Ability to work with business owners to define key business requirements and convert to user stories with required technical specifications.

Communicate results and business impacts of insight initiatives to key stakeholders to collaboratively solve business problems.

Working closely with the overall Enterprise Data & Analytics Architect and Engineering practice leads to ensure adherence with the best practices and design principles.

Assures quality, security and compliance requirements are met for supported area.

Design and create fault-tolerance data pipelines running on cluster

Excellent communication skills with the ability to influence client business and IT teams

Should have design data engineering solutions end to end. Ability to come up with scalable and modular solutions Required Qualification:

7 years of relevant IT experience

4+ years of hands-on experience Designing and developing Data Pipelines for Data Ingestion or Transformation using Python (PySpark)/Spark SQL in AWS cloud

Experience in design and development of data pipelines and processing of data at scale using technologies like EMR, Kinesis, Kafka, Lambda, Glue, Athena, Redshift, Step Functions.

Advanced experience in writing and optimizing efficient SQL queries with Python and Hive handling Large Data Sets in Big-Data Environments

Experience in debugging, tunning and optimizing PySpark data pipelines

Should have implemented concepts and have good knowledge of Pyspark dataframes, joins, caching, memory management, partitioning, parallelism etc.

Understanding of Spark UI, Event Timelines, DAG, Spark config parameters, in order to tune the long running data pipelines.

Experience working in Agile implementations

Experience with building data pipelines in streaming and batch mode.

Experience with Git and CI/CD pipelines to deploy cloud applications

Good knowledge of designing Hive tables with partitioning for performance Desired Qualification:

Experience in data modelling

Hands on creating workflows on any Scheduling Tool like Autosys, CA Workload Automation

Proficiency in using SDKs for interacting with native AWS services

Strong understanding of concepts of ETL, ELT and data modeling.

  • Lead, Data Engineer

    6 days ago

    Mumbai, India CPP Investments Full time

    Job DescriptionThe Data Engineering team is looking for people who are passionate about data and are looking to bring their expertise in big data and cloud by working in agile delivery environments to resolve engineering challenges of building robust and scalable data systems aligned to our enterprise data strategy.As a Lead, Data Engineer - you will be...

  • Lead, Data Engineer

    1 week ago

    mumbai, India CPP Investments Full time

    Job DescriptionThe Data Engineering team is looking for people who are passionate about data and are looking to bring their expertise in big data and cloud by working in agile delivery environments to resolve engineering challenges of building robust and scalable data systems aligned to our enterprise data strategy.   As a Lead, Data Engineer - you will be...

  • Team Lead

    1 month ago

    mumbai, India WIZSTAFFING PRIVATE LIMITED Full time

    Job Description : - We are seeking a highly skilled and motivated individual to join our team as a team lead - azure data engineer.- The ideal candidate will have a strong background in azure data technologies, a proven track record of leading data engineering teams, and the ability to design and implement scalable and efficient data solutions.- The team...

  • Lead Data Engineer

    1 month ago

    Mumbai, India JPMorgan Chase & Co. Full time

    Embrace this pivotal role as an essential member of a high performing team dedicated to reaching new heights in data engineering. Your contributions will be instrumental in shaping the future of one of the world’s largest and most influential companies. As a Senior Lead Data Engineer at JPMorgan Chase within the CCB Connected Commerce Technology Team,...

  • Lead Data Engineer

    1 month ago

    mumbai, India JPMorgan Chase & Co. Full time

    Embrace this pivotal role as an essential member of a high performing team dedicated to reaching new heights in data engineering. Your contributions will be instrumental in shaping the future of one of the world’s largest and most influential companies. As a Senior Lead Data Engineer at JPMorgan Chase within the CCB Connected Commerce Technology Team,...

  • Team Lead

    2 weeks ago

    Mumbai, India WIZSTAFFING PRIVATE LIMITED Full time

    Job Description : - We are seeking a highly skilled and motivated individual to join our team as a team lead - azure data engineer.- The ideal candidate will have a strong background in azure data technologies, a proven track record of leading data engineering teams, and the ability to design and implement scalable and efficient data solutions.- The team...

  • Team Lead

    2 weeks ago

    Mumbai, India WIZSTAFFING PRIVATE LIMITED Full time

    Job Description :- We are seeking a highly skilled and motivated individual to join our team as a team lead - azure data engineer.- The ideal candidate will have a strong background in azure data technologies, a proven track record of leading data engineering teams, and the ability to design and implement scalable and efficient data solutions.- The team lead...

  • Team Lead

    4 weeks ago

    Mumbai, Maharashtra, India WIZSTAFFING PRIVATE LIMITED Full time

    Job Description :- We are seeking a highly skilled and motivated individual to join our team as a team lead - azure data engineer.- The ideal candidate will have a strong background in azure data technologies, a proven track record of leading data engineering teams, and the ability to design and implement scalable and efficient data solutions.- The team lead...

  • Lead, Data Engineer

    1 week ago

    Mumbai, India CPP Investments Full time

    Job DescriptionThe Data Engineering team is looking for people who are passionate about data and are looking to bring their expertise in big data and cloud by working in agile delivery environments to resolve engineering challenges of building robust and scalable data systems aligned to our enterprise data strategy. As a Lead, Data Engineer - you will be...

  • Ventura - Lead Data Engineer

    1 month ago

    mumbai, India Ventura Securities Ltd. Full time

    Job description :About us :Ventura Securities is establishing its fintech team - a hybrid work environment with highly talented individuals who come from diverse backgrounds and looking to solve real client problems at scale. Ventura has been in the stock broking business for 25+ years and it is a robust and profitable franchise with 1000+ employees...

  • Ventura - Lead Data Engineer

    2 weeks ago

    Mumbai, India Ventura Securities Ltd. Full time

    Job description :About us :Ventura Securities is establishing its fintech team - a hybrid work environment with highly talented individuals who come from diverse backgrounds and looking to solve real client problems at scale. Ventura has been in the stock broking business for 25+ years and it is a robust and profitable franchise with 1000+ employees...

  • Lead Data Analyst

    2 days ago

    Mumbai, India Saaki Argus & Averil Consulting Full time

    Job Role – Lead Data AnalystJob Location- Kolkata/Mumbai (Hybrid mode with 3 days in office)Experience- 9-12 yearsJob Details:Technical Lead will design future-state, modern data architecture for transformation at the enterprise level using advanced cloud architectural principles. Using tools such as, they will also work with cutting edge DevOps...

  • Team Lead

    2 weeks ago

    mumbai, India NTT DATA, Inc. Full time

    Experience :7-12 yrsTeam Leadership:Lead, mentor, and supervise the service support desk team, ensuring a high level of motivation and performance.Foster a collaborative and customer-focused environment within the team.Conduct regular team meetings and training sessions to enhance technical skills and customer service capabilities.Service Desk...

  • Team Lead

    1 week ago

    Mumbai, India NTT DATA, Inc. Full time

    Experience :7-12 yrsTeam Leadership:Lead, mentor, and supervise the service support desk team, ensuring a high level of motivation and performance.Foster a collaborative and customer-focused environment within the team.Conduct regular team meetings and training sessions to enhance technical skills and customer service capabilities.Service Desk...

  • Team Lead

    2 weeks ago

    mumbai, India NTT DATA, Inc. Full time

    Experience :7-12 yrs Team Leadership: Lead, mentor, and supervise the service support desk team, ensuring a high level of motivation and performance. Foster a collaborative and customer-focused environment within the team. Conduct regular team meetings and training sessions to enhance technical skills and customer service capabilities. Service Desk...

  • Team Lead

    2 weeks ago

    Mumbai, India NTT DATA, Inc. Full time

    Experience :7-12 yrsTeam Leadership:Lead, mentor, and supervise the service support desk team, ensuring a high level of motivation and performance.Foster a collaborative and customer-focused environment within the team.Conduct regular team meetings and training sessions to enhance technical skills and customer service capabilities.Service Desk...

  • Ventura - Lead Data Engineer

    2 weeks ago

    Thane/Mumbai/Navi Mumbai, India Ventura Securities Ltd. Full time

    Job description :About us :Ventura Securities is establishing its fintech team - a hybrid work environment with highly talented individuals who come from diverse backgrounds and looking to solve real client problems at scale. Ventura has been in the stock broking business for 25+ years and it is a robust and profitable franchise with 1000+ employees...

  • Ventura - Lead Data Engineer

    4 weeks ago

    Thane/Mumbai/Navi Mumbai, Maharashtra, India Ventura Securities Ltd. Full time

    Job description :About us :Ventura Securities is establishing its fintech team - a hybrid work environment with highly talented individuals who come from diverse backgrounds and looking to solve real client problems at scale. Ventura has been in the stock broking business for 25+ years and it is a robust and profitable franchise with 1000+ employees...

  • Data Engineer

    4 weeks ago

    Mumbai, India Searce Technologies Inc Full time

    about the roleData Engineer identifies the business problem and translates these to data services and engineering outcomes. You will deliver data solutions that empower better decision making and flexibility of your solution that scales to respond to broader business questions.key responsibilitiesAs a Data Engineer, you are a full-stack data engineer that...

  • Lead Data Engineer

    2 weeks ago

    Mumbai, India XL Advisors Full time

    About us:JSW One Platforms is a venture founded by the world-renowned JSW group. JSW One is our integrated technology platform that seeks to transform India through increased transparency, trust, and ease-of-business. The JSW One Platforms are home to JSW One MSME and JSW One Homes. JSW One MSME, is a one-stop, multi-product digital marketplace, for MSMEs...