Lead ETL Engineer

1 month ago


Bengaluru, India YITRO BUSINESS CONSULTANTS (INDIA) PRIVATE LIMITED Full time

Role : Lead ETL Feature Engineer

Years of Experience : 5 to 9 years

Job Location : Bangalore

Mode of Work : Work from office

Shift : General

Job Type : Permanent

Responsibilities :

1. Develop and maintain ETL pipelines to ingest and process large-scale datasets

2. Develop a Python connector designed for ETL applications, enabling the collection of historical samples and the generation of intelligent samples for AI/ML tasks

3. Demonstrate hands-on experience on Apache Kafka, Amazon Kinesis, and AWS Glue

4. ETL pipeline for AI/ML workload integrations and executions

5. Implement and maintain ETL pipeline and deep understanding of orchestration, scaling, and resource management.

6. Develop ETL samples to manage unstructured data tasks, covering data preprocessing for various types, including emails, Office 365 documents, pictures, voice recordings, videos, and PDFs.

7. Establish a solid understanding and practical knowledge of SQL databases, with a focus on query performance optimization and effective index management.

8. Implement an ETL pipeline with multiple databases to extract samples from both NoSQL and SQL

9. Work seamlessly in a multi-cloud environment, specifically AWS and Azure.

10. Execute hands-on development and deployment of microservices and containerized applications.

Requirements :

1. Minimum 4 years hands-on experience in setting up ETL & feature engineering part of the data pipelines on cloud or big data ecosystems.

2. Strong hands-on experience in Apache Kafka, Amazon Kinesis, and AWS Glue

3. Holds expert-level knowledge in at least one ETL tool (ETL/ Workflow - SeaTunnel/ Dolphin Scheduler, SSIS, Informatica, Talend, etc.)

4. Experience with big data technologies such as Hadoop, Spark, and Hive

5. Experience in NoSQL and SQL databases (e.g., MongoDB, SnowFlake, PostgreSQL, DeltaLake, Parquet).

6. Strong programming skills in Python, and experience with data manipulation libraries such as pandas, numpy, etc.

7. Handling unstructured data tasks, including preprocessing for various data types such as Mail, Office 365, Pictures, Voice, Video, and PDF

8. Possesses SQL knowledge including query performance tuning, index maintenance and an understanding of database structure

9. Good to have knowledge in Apache Spark, Apache NiFi, Apache Kafka, Apache Airflow, Apache Beam, KubeFlow, Apache Beam and TensorFlow Transform

10. Expertise in networking, security, and cloud platforms (AWS, Azure, GCP, etc.)

Interested candidate apply here or share your updated resume to deepika.subramanian@yitrobc.net

(ref:hirist.tech)

  • Bengaluru, India Huquo Full time

    Role Overview : As a Senior ETL QA Lead, you will be responsible for leading and executing the quality assurance strategy for ETL processes leveraging SnapLogic, DataStage, Informatica, Talend, and other relevant tools. You will collaborate closely with cross-functional teams to ensure the reliability, accuracy, and efficiency of our data integration...

  • ETL Pentaho Lead

    5 days ago


    Bengaluru, India Virtusa Full time

    ETL Pentaho Lead - CREQ187183 Description An ETL developer with Strong background on Pentaho ETL developments with large volumes of data set.Utilize agile methodologies and adhere to coding standards, procedures and techniques while contributing to the technical code documentationParticipate in design reviews and provide input to the design...

  • ETL Pentaho Lead

    1 week ago


    Bengaluru, India Virtusa Full time

    ETL Pentaho Lead - CREQ187183 Description An ETL developer with Strong background on Pentaho ETL developments with large volumes of data set. Utilize agile methodologies and adhere to coding standards, procedures and techniques while contributing to the technical code documentation Participate in design reviews and provide input to the design recommendations...

  • ETL Test Engineer

    3 weeks ago


    Bengaluru, India Corporate Resources Full time

    Job DescriptionDeveloping ETL test cases and test plans that ensure data quality, accuracy, and completeness.Conducting functional and non-functional testing of ETL processes to validate the integrity of the data being transferred.Identifying and documenting defects, issues, and potential improvements in the ETL process and sharing them with the development...

  • ETL Developer

    7 days ago


    Bengaluru, India Niotact Digital Solutions Full time

    Lead the design, development, and optimization of ETL processes using Ab Initio's Graphical Development Environment (GDE), ensuring data accuracy, consistency, and availability. Data Integration: Build, maintain, and optimize data integration workflows, enabling seamless data flow across various systems and platforms. Experience in Data Transformation,...

  • ETL Developer

    1 week ago


    Bengaluru, India Niotact Digital Solutions Full time

    Lead the design, development, and optimization of ETL processes using Ab Initio's Graphical Development Environment (GDE), ensuring data accuracy, consistency, and availability. Data Integration: Build, maintain, and optimize data integration workflows, enabling seamless data flow across various systems and platforms.  Experience in Data Transformation,...

  • QA Engineer

    3 days ago


    Bengaluru, India Syniverse Full time

    Syniverse is the world’s most connected company. Whether we’re developing the technology that enables intelligent cars to safely react to traffic changes or freeing travelers to explore by keeping their devices online wherever they go, we believe in leading the world forward.Which is why we work with some of the world’s most recognized brands. Eight of...

  • Azure Data Engineer

    2 weeks ago


    Bengaluru, India Connectio IT Pvt Ltd Full time

    Role Description :This is a full-time Azure Data Engineer role at Connect io IT Pvt Ltd located in Bengaluru. As an Azure Data Engineer, you will be responsible for tasks such as data engineering, data modeling, Extract Transform Load (ETL), data warehousing, and data analytics. You will work on-site in Bengaluru.Qualifications :- Data Engineering, Data...

  • Data Engineer

    1 month ago


    Bengaluru, India Mobile Programming LLC Full time

    Job Description :Please note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will play a crucial role in developing and maintaining our data infrastructure and ensuring the smooth operation of our data platforms. The ideal candidate should have a strong...


  • Bengaluru, India Quess Full time

    About Us “Quess IT Staffing is India’s largest IT staffing company with over 20 years of experience in staffing IT professionals in 300+ companies across levels and skillsets. Our 10,000+ associates deployed in 80+ cities and towns are proficient in over 500 technological skills. Our associates help enable cutting edge solutions some of the biggest names...

  • ETL Developer

    3 weeks ago


    Bengaluru, India State Street Corporation Full time

    ETL Developer - Team LeadWho we are looking for:Looking for a strong database engineer worked on experience in Snowflake, Oracle. cloud engineer worked on various cloud migration and implementation at large scale. Have experienced on Azure cloud primarily on the architecture, design and implementation.Highly-motivated and thrive in an environment that...


  • Bengaluru, India JLL Full time

    About the roleWe are looking for a Senior Resource to join our JLL Technologies - CoE organization in Bangalore, India. This role requires partnering with the broader JLL Technology team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experienceThe role will add immense...


  • Bengaluru, India JLL Full time

    About the role We are looking for a Senior Resource to join our JLL Technologies - CoE organization in Bangalore, India. This role requires partnering with the broader JLL Technology team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience The role will add...

  • Data Engineer

    1 week ago


    Bengaluru, India Sterlite Technologies Limited Full time

    Role as a Data engineer :- Develop and build AI/ML model ready data ponds using Azure Databricks Delta lake- Improve upon the data ingestion models, ETL jobs, and alarm to maintain data integrity and data availability.- Partner with AI/ML product teams proactively identify the data needs for rapid experimentation, stable production deployment and ensure the...


  • Bengaluru, India RS Consultants Full time

    ETL (Databricks)Job description : Responsibilities :- Build, document, test and launch new data extract, transform and load (ETL) processes for one or more core data sets.- Collaborate with Business Intelligence team and other stakeholders for data requirements- Collaborate with the Data Management team to provide quality data to stakeholders.- Build data...

  • Senior Manager

    3 weeks ago


    Bengaluru, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...


  • Bengaluru, India State Street Corporation Full time

    Automation - ETL - test engineer - Tech LeadRole Summary & Role DescriptionAs Software Quality Assurance Engineer you willCreate test scenarios and implement functional test cases (automated and otherwise) that will serve as acceptance test criteria for enhancements and new applicationsWork closely with developers and quality assurance members to run test...


  • Bengaluru, India State Street Full time

    Role Summary & Role DescriptionGoal of the role is to work on ETL automation. Will be reporting to me.5+ years of experience.As the ETL Automation Engineer in the Corporate Finance Techlology team you willDeveloping and maintaining robust ETL pipelines using PySpark to process large volumes of data efficiently.Writing and executing automated tests for ETL...

  • Senior Manager

    1 month ago


    Bengaluru, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people –...

  • Data Engineer

    1 month ago


    Bengaluru, India SKEP IT SYSTEMS Full time

    Role : Data EngineerMobilution IT Systems is a pioneering organization at the forefront of technological innovation, specializing in cutting-edge Tech solutions. With a strong commitment to leveraging advanced technologies for scientific advancements, Mobilution IT Systems is dedicated to revolutionizing the way data is analysed and utilized across various...