Lead Data Engineer

3 weeks ago

Gurgaon, Haryana, India Antal International Full time
Job Description

Lead Data engineer helps to architect, design, implement, and optimise our data ingestion, transformation pipelines and processes. He or she develops data models, processing pipelines, and back-end services supporting the data science teams, automates processes, builds integrations, and builds analytics.

Desired skills:

· A university degree from a reputed University with a record of academic excellence in Mathematics, Computer Science, Engineering, Physics, or similar.

· 8+ years of relevant experience in Data Engineering, warehousing, ETL, automation, big data and cloud technologies, etc.

· Ability to write clean, scalable, maintainable code in Python. Has a good understanding of software engineering concepts and patterns. Proficiency in other languages like Scala, Java, C#, C++ are an advantage.

· Proven record of building and maintaining data pipelines deployed in at least one of the big 3 cloud ML stacks (AWS, Azure, GCP).

· Hands-on experience with

o Open-source ETL, and data pipeline and orchestration of such tools as Apache Airflow and Nifi. o Large Scale/ Big Data technology, such as Hadoop, Spark, Hive, Impala, PrestoDb, Kafka.

o Fast and distributed data processing in Ray, Dask, DuckDb, Polars, cuDF

o Building performant code in C, C++ and interfaces with python in PyO3.rs, Cython

o Workflow orchestration tools like Apache Airflow. o Containerisation with Docker, deployment on Kubernetes.

o NoSql and graph databases.

o Unix server administration and shell scripting.

· Experience in building scalable data pipelines for highly unstructured data.

· Experience in building DWH and data lakes architectures.

· Experience in working in cross-functional team with software engineers, data scientists and machine learning engineers.

· Experience in working with or leading an off-shore team.

· Proven record of building data science environments deploying ML solutions in at least one of the big 3 cloud ML stacks (Azure/Aws/GCP) and on Kubernetes clusters.

· Excellent written and verbal command of English.

· Strong problem-solving, analytical, and quantitative skills.

· A professional attitude and service orientation with the ability to work with our international teams.

Check Your Resume for Match

Upload your resume and our tool will compare it to the requirements for this job like recruiters do.

  • Data Engineering Lead

    1 day ago

    Gurgaon, Haryana, India Eptura Full time

    About the Role:We are seeking a highly skilled Data Team Lead to join our team at Eptura. As a key member of our data engineering group, you will be responsible for designing and developing complex ETL processes that meet the needs of our stakeholders.You will work closely with cross-functional teams to analyze requirements, create data models, and implement...

  • Data Engineering Lead

    2 weeks ago

    Gurgaon, Haryana, India Metyis Full time

    What we offer   Interact with C-level at our clients on regular basis to drive their business towards impactful change Lead your team in creating new business solutions Seize opportunities at the client and at Metyis in our entrepreneurial environment Become part of a fast-growing international and diverse team What you will do Independently...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Falcon Full time

    Key Responsibilities : Data Engineering & Architecture :- Design and implement scalable, high-performance data pipelines on AWS.- Optimize ETL/ELT processes using AWS Glue, Apache Spark, EMR, and Lambda.- Build and maintain data lakes and warehouses (Redshift, Snowflake).- Ensure data governance, security, and compliance best practices. Technical Leadership...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Falcon Full time

    Key Responsibilities : Data Engineering & Architecture :- Design and implement scalable, high-performance data pipelines on AWS.- Optimize ETL/ELT processes using AWS Glue, Apache Spark, EMR, and Lambda.- Build and maintain data lakes and warehouses (Redshift, Snowflake).- Ensure data governance, security, and compliance best practices. Technical Leadership...

  • Data Engineering Lead

    1 day ago

    Gurgaon, Haryana, India Metyis Full time

    Job DescriptionWho We AreThe next step of your career starts here, where you can bring your own unique mix of skills and perspectives to a fast-growing team.Metyis is a global and forward-thinking firm operating across a wide range of industries, developing and delivering AI & Data, Digital Commerce, Marketing & Design solutions and Advisory services. At...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India IGT Solutions Full time

    Role: Lead Data Engineer Primary Skills: Python, Pyspark, AWS Glue, EMR, Lambda, Snowflake, ETL, PLSQL Experience: - 7+ Years Location: - Remote Notice period: - Immediate Joiner Job Overview : We are seeking a skilled and experienced Lead Data Engineer to join our data engineering team. The ideal candidate will have over 8 years of hands-on experience...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India IGT Solutions Full time

    Role: Lead Data Engineer Primary Skills: Python, Pyspark, AWS Glue, EMR, Lambda, Snowflake, ETL, PLSQL Experience: - 7+ Years Location: - Remote Notice period: - Immediate Joiner Job Overview : We are seeking a skilled and experienced Lead Data Engineer to join our data engineering team. The ideal candidate will have over 8 years of hands-on experience...

  • Lead Data Engineer

    3 weeks ago

    Gurgaon, Haryana, India Fidelity International Full time

    Job DescriptionAbout the OpportunityJob Type: PermanentApplication Deadline: 01 March 2025Job DescriptionTitle: Lead Data EngineerDepartment: ISSLocation: Gurgaon, India or Dalian, ChinaReports To: Head of Data Platform - ISSGrade : 7We're proud to have been helping our clients build better financial futures for over 50 years. How have we achieved this By...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India TALENTOLA SOLUTIONS PRIVATE LIMITED Full time

    Job Description :Key Responsibilities :- Team Leadership : Lead, mentor, and grow a team of data engineers, fostering a culture of excellence, collaboration, and continuous improvement.- Data Pipeline Development : Oversee the design, development, and maintenance of robust ETL and ELT pipelines to ingest, transform, and process data efficiently.- Cloud Data...

  • Lead Data Engineer

    4 weeks ago

    Gurgaon, Haryana, India TALENTOLA SOLUTIONS PRIVATE LIMITED Full time

    Job Description :Key Responsibilities :- Team Leadership : Lead, mentor, and grow a team of data engineers, fostering a culture of excellence, collaboration, and continuous improvement.- Data Pipeline Development : Oversee the design, development, and maintenance of robust ETL and ELT pipelines to ingest, transform, and process data efficiently.- Cloud Data...

  • Data Engineering Team Lead

    24 hours ago

    Gurgaon, Haryana, India Metyis Full time

    Required Skills and Qualifications7-10 years of professional hands-on experience in software engineering.A broad practice in multiple software engineering fields.Experience in managing and leading data engineering/warehousing projects.The ideal candidate will have a strong background in software engineering and experience in managing and leading data...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India IGT Solutions Full time

    Role: Lead Data EngineerPrimary Skills: Python, Pyspark, AWS Glue, EMR, Lambda, Snowflake, ETL, PLSQL Experience: - 7+ YearsLocation: - RemoteNotice period: - Immediate JoinerJob Overview: We are seeking a skilled and experienced Lead Data Engineer to join our data engineering team. The ideal candidate will have over 8 years of hands-on experience designing,...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India IGT Solutions Full time

    Role: Lead Data EngineerPrimary Skills: Python, Pyspark, AWS Glue, EMR, Lambda, Snowflake, ETL, PLSQL Experience: - 7+ YearsLocation: - RemoteNotice period: - Immediate JoinerJob Overview: We are seeking a skilled and experienced Lead Data Engineer to join our data engineering team. The ideal candidate will have over 8 years of hands-on experience designing,...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Antal International Full time

    Job Title - Lead Data Engineer Key Skills - Azure Data Factory Data Warehousing SQL SSIS Experience - 8 to 12 Years Location - Gurugram Gurgaon Industry - Insurance NP - Immediate to 30 Days Education B Tech or equivalent qualificationTechnical Expertise Should have experience of working on Microsoft Azure tools like Spark Databrick Synapse ...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Sound of the Sun Full time

    Job Title : Data Engineer. Contract Duration : 12 months. Total Experience Required : 5-9 years. Relevant Experience Required : 5+ years. Job Description :We are seeking experienced Data Engineers to join our team for a 12-month contract position. As a Data Engineer, you will play a key role in the development, deployment, and management of data engineering...

  • Lead Data Engineer

    1 week ago

    Gurgaon, Haryana, India Sound of the Sun Full time

    Job Title : Data Engineer. Contract Duration : 12 months. Total Experience Required : 5-9 years. Relevant Experience Required : 5+ years. Job Description :We are seeking experienced Data Engineers to join our team for a 12-month contract position. As a Data Engineer, you will play a key role in the development, deployment, and management of data engineering...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India MakeMyTrip Full time

    Position: Lead Data Engineer Experience: Must have 6+ years of experience About Role: We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products designed to enhance customer experiences. About us: Nurtured from the seed of a single great idea - to empower the traveler - MakeMyTrip...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India TransOrg Analytics Full time

    Job Description - Lead - Data Engineer Why would you like to join us ? TransOrg Analytics specializes in Data Science, Data Engineering and Generative AI, providing advanced analytics solutions to industry leaders and Fortune 500 companies across India, US, APAC and the Middle East. We leverage data science to streamline, optimize, and accelerate our...

  • Lead Data Engineer and Architect

    1 day ago

    Gurgaon, Haryana, India Metyis Full time

    Job ResponsibilitiesIndependently lead and manage the execution of data engineering projects.Engineer complete technical solutions to solve concrete business challenges in areas like digital marketing, eCommerce, Business Intelligence, and self-service analytics.Collect functional and non-functional requirements, consider technical environments, business...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India MakeMyTrip Full time

    Position: Lead Data EngineerExperience: Must have 6+ years of experienceAbout Role:We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products designed to enhance customer experiences.About us:Nurtured from the seed of a single great idea - to empower the traveler - MakeMyTrip went on...