Lead ETL Engineer

1 month ago


Bangalore, India YITRO BUSINESS CONSULTANTS (INDIA) PRIVATE LIMITED Full time

Role : Lead ETL Feature Engineer

Years of Experience : 5 to 9 years

Job Location : Bangalore

Mode of Work : Work from office

Shift : General

Job Type : Permanent

Responsibilities :

1. Develop and maintain ETL pipelines to ingest and process large-scale datasets

2. Develop a Python connector designed for ETL applications, enabling the collection of historical samples and the generation of intelligent samples for AI/ML tasks

3. Demonstrate hands-on experience on Apache Kafka, Amazon Kinesis, and AWS Glue

4. ETL pipeline for AI/ML workload integrations and executions

5. Implement and maintain ETL pipeline and deep understanding of orchestration, scaling, and resource management.

6. Develop ETL samples to manage unstructured data tasks, covering data preprocessing for various types, including emails, Office 365 documents, pictures, voice recordings, videos, and PDFs.

7. Establish a solid understanding and practical knowledge of SQL databases, with a focus on query performance optimization and effective index management.

8. Implement an ETL pipeline with multiple databases to extract samples from both NoSQL and SQL

9. Work seamlessly in a multi-cloud environment, specifically AWS and Azure.

10. Execute hands-on development and deployment of microservices and containerized applications.

Requirements :

1. Minimum 4 years hands-on experience in setting up ETL & feature engineering part of the data pipelines on cloud or big data ecosystems.

2. Strong hands-on experience in Apache Kafka, Amazon Kinesis, and AWS Glue

3. Holds expert-level knowledge in at least one ETL tool (ETL/ Workflow - SeaTunnel/ Dolphin Scheduler, SSIS, Informatica, Talend, etc.)

4. Experience with big data technologies such as Hadoop, Spark, and Hive

5. Experience in NoSQL and SQL databases (e.g., MongoDB, SnowFlake, PostgreSQL, DeltaLake, Parquet).

6. Strong programming skills in Python, and experience with data manipulation libraries such as pandas, numpy, etc.

7. Handling unstructured data tasks, including preprocessing for various data types such as Mail, Office 365, Pictures, Voice, Video, and PDF

8. Possesses SQL knowledge including query performance tuning, index maintenance and an understanding of database structure

9. Good to have knowledge in Apache Spark, Apache NiFi, Apache Kafka, Apache Airflow, Apache Beam, KubeFlow, Apache Beam and TensorFlow Transform

10. Expertise in networking, security, and cloud platforms (AWS, Azure, GCP, etc.)

Interested candidate apply here or share your updated resume to

(ref:hirist.tech)

  • bangalore, India JLL Full time

    About the role We are looking for a Senior Resource to join our JLL Technologies - CoE organization in Bangalore, India. This role requires partnering with the broader JLL Technology team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience The role will add...


  • bangalore, India Huquo Full time

    Role Overview : As a Senior ETL QA Lead, you will be responsible for leading and executing the quality assurance strategy for ETL processes leveraging SnapLogic, DataStage, Informatica, Talend, and other relevant tools. You will collaborate closely with cross-functional teams to ensure the reliability, accuracy, and efficiency of our data integration...

  • ETL Pentaho Lead

    3 weeks ago


    bangalore, India Virtusa Full time

    ETL Pentaho Lead - CREQ187183 Description An ETL developer with Strong background on Pentaho ETL developments with large volumes of data set.Utilize agile methodologies and adhere to coding standards, procedures and techniques while contributing to the technical code documentationParticipate in design reviews and provide input to the design...

  • Lead ETL developer

    4 days ago


    Bangalore Urban, India CGI Full time

    Job Opportunity with CGI for Lead ETL Developer!!Experience: 8 to 10 yearsRelevant ExperienceSQL Server - 8 to 10 yearsSSIS- 8 to 10 yearsSSAS Server - 4 to 5 yearsAnd Strong analytic and trouble shooting skillsJob Location: Bangalore/Chennai/Hyderabad/Pune/MumbaiNotice: 30 daysJob Description:We need a Lead ETL Developer with Microsoft SQL Server, SSIS and...

  • ETL Developer

    3 weeks ago


    bangalore, India Niotact Digital Solutions Full time

    Lead the design, development, and optimization of ETL processes using Ab Initio's Graphical Development Environment (GDE), ensuring data accuracy, consistency, and availability. Data Integration: Build, maintain, and optimize data integration workflows, enabling seamless data flow across various systems and platforms.Experience in Data Transformation,...

  • ETL Test Engineer

    1 month ago


    bangalore, India Corporate Resources Full time

    Job Description Developing ETL test cases and test plans that ensure data quality, accuracy, and completeness. Conducting functional and non-functional testing of ETL processes to validate the integrity of the data being transferred. Identifying and documenting defects, issues, and potential...


  • Gurgaon/Gurugram/Bangalore, India Huquo Full time

    Role Overview : As a Senior ETL QA Lead, you will be responsible for leading and executing the quality assurance strategy for ETL processes leveraging SnapLogic, DataStage, Informatica, Talend, and other relevant tools. You will collaborate closely with cross-functional teams to ensure the reliability, accuracy, and efficiency of our data integration...

  • Data Engineer

    2 days ago


    bangalore, India Axtria - Ingenious Insights Full time

    Job Title: Data Engineer / ETL DeveloperJob Description:We are seeking an experienced ETL Developer / Data Engineer to join our team. The ideal candidate will have a strong background in Azure Data Factory (ADF), SQL, Pyspark/DatabricksResponsibilities:Design, develop, and implement ETL processes using Azure Data Factory (ADF), Matillion, Snowflake, Or...

  • Data Engineer

    2 days ago


    bangalore, India Axtria - Ingenious Insights Full time

    Job Title: Data Engineer / ETL Developer Job Description: We are seeking an experienced ETL Developer / Data Engineer to join our team. The ideal candidate will have a strong background in Azure Data Factory (ADF), SQL, Pyspark/Databricks Responsibilities: Design, develop, and implement ETL processes using Azure Data Factory (ADF), Matillion, Snowflake,...

  • QA Engineer

    3 weeks ago


    bangalore, India Syniverse Full time

    Syniverse is the world’s most connected company. Whether we’re developing the technology that enables intelligent cars to safely react to traffic changes or freeing travelers to explore by keeping their devices online wherever they go, we believe in leading the world forward.Which is why we work with some of the world’s most recognized brands. Eight...

  • Data Engineer

    1 week ago


    bangalore, India Careerzgraph Full time

    Total experience required : 15-20 years.Notice Period - Not More then 15 Days / Only Immediate Joiners For Location - Bengaluru (KA)/Electronic City Phase 1. - Work from Office Job Description :- Lead and mentor a team of data engineers in designing, developing, and maintaining efficient and reliable ETL pipelines.- Develop advanced scripts using Python...


  • bangalore, India ContinuServe Full time

    Continu Serve Softech  is a leading professional services, consulting and outsourcing firm that helps mid-market and Fortune 2000 organizations improve service levels and reduce costs within their back-office operations through the implementation and delivery of shared services solutions. We also provide Carve-out outsourcing services for financial and IT...

  • Data Engineer

    1 week ago


    Bangalore, India Careerzgraph Full time

    Total experience required : 15-20 years.Notice Period - Not More then 15 Days / Only Immediate Joiners For Location - Bengaluru (KA)/Electronic City Phase 1. - Work from Office Job Description :- Lead and mentor a team of data engineers in designing, developing, and maintaining efficient and reliable ETL pipelines.- Develop advanced scripts using Python...

  • Azure Data Engineer

    4 weeks ago


    Bangalore, India Connectio IT Pvt Ltd Full time

    Role Description :This is a full-time Azure Data Engineer role at Connect io IT Pvt Ltd located in Bengaluru. As an Azure Data Engineer, you will be responsible for tasks such as data engineering, data modeling, Extract Transform Load (ETL), data warehousing, and data analytics. You will work on-site in Bengaluru.Qualifications :- Data Engineering, Data...

  • Azure Data Engineer

    1 month ago


    bangalore, India Connectio IT Pvt Ltd Full time

    Role Description :This is a full-time Azure Data Engineer role at Connect io IT Pvt Ltd located in Bengaluru. As an Azure Data Engineer, you will be responsible for tasks such as data engineering, data modeling, Extract Transform Load (ETL), data warehousing, and data analytics. You will work on-site in Bengaluru.Qualifications :- Data Engineering, Data...

  • Data Engineer

    1 month ago


    Bangalore, India Mobile Programming LLC Full time

    Job Description : Please note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will play a crucial role in developing and maintaining our data infrastructure and ensuring the smooth operation of our data platforms. The ideal candidate should have a strong...


  • bangalore, India JLL Full time

    About the role We are looking for a Senior Resource to join our JLL Technologies - CoE organization in Bangalore, India. This role requires partnering with the broader JLL Technology team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience The role will add...


  • bangalore, India RS Consultants Full time

    ETL (Databricks)Job description : Responsibilities :- Build, document, test and launch new data extract, transform and load (ETL) processes for one or more core data sets.- Collaborate with Business Intelligence team and other stakeholders for data requirements- Collaborate with the Data Management team to provide quality data to stakeholders.- Build data...


  • Bangalore, India RS Consultants Full time

    ETL (Databricks)Job description : Responsibilities :- Build, document, test and launch new data extract, transform and load (ETL) processes for one or more core data sets.- Collaborate with Business Intelligence team and other stakeholders for data requirements- Collaborate with the Data Management team to provide quality data to stakeholders.- Build data...

  • Senior Data Engineer

    1 month ago


    Bangalore, India HuQuo Consulting Pvt. Ltd. Full time

    Job Description : - Drive best possible solutions for clients and work closely with a highly skilled Data Science team. - Lead on projects from a data engineering perspective, working with our clients to model their data landscape, obtain data extracts and define secure data exchange approaches while handling vast amounts of data - working on database...