Lead ETL Engineer
1 month ago
Role : Lead ETL Feature Engineer
Years of Experience : 5 to 9 years
Job Location : Bangalore
Mode of Work : Work from office
Shift : General
Job Type : Permanent
Responsibilities :
1. Develop and maintain ETL pipelines to ingest and process large-scale datasets
2. Develop a Python connector designed for ETL applications, enabling the collection of historical samples and the generation of intelligent samples for AI/ML tasks
3. Demonstrate hands-on experience on Apache Kafka, Amazon Kinesis, and AWS Glue
4. ETL pipeline for AI/ML workload integrations and executions
5. Implement and maintain ETL pipeline and deep understanding of orchestration, scaling, and resource management.
6. Develop ETL samples to manage unstructured data tasks, covering data preprocessing for various types, including emails, Office 365 documents, pictures, voice recordings, videos, and PDFs.
7. Establish a solid understanding and practical knowledge of SQL databases, with a focus on query performance optimization and effective index management.
8. Implement an ETL pipeline with multiple databases to extract samples from both NoSQL and SQL
9. Work seamlessly in a multi-cloud environment, specifically AWS and Azure.
10. Execute hands-on development and deployment of microservices and containerized applications.
Requirements :
1. Minimum 4 years hands-on experience in setting up ETL & feature engineering part of the data pipelines on cloud or big data ecosystems.
2. Strong hands-on experience in Apache Kafka, Amazon Kinesis, and AWS Glue
3. Holds expert-level knowledge in at least one ETL tool (ETL/ Workflow - SeaTunnel/ Dolphin Scheduler, SSIS, Informatica, Talend, etc.)
4. Experience with big data technologies such as Hadoop, Spark, and Hive
5. Experience in NoSQL and SQL databases (e.g., MongoDB, SnowFlake, PostgreSQL, DeltaLake, Parquet).
6. Strong programming skills in Python, and experience with data manipulation libraries such as pandas, numpy, etc.
7. Handling unstructured data tasks, including preprocessing for various data types such as Mail, Office 365, Pictures, Voice, Video, and PDF
8. Possesses SQL knowledge including query performance tuning, index maintenance and an understanding of database structure
9. Good to have knowledge in Apache Spark, Apache NiFi, Apache Kafka, Apache Airflow, Apache Beam, KubeFlow, Apache Beam and TensorFlow Transform
10. Expertise in networking, security, and cloud platforms (AWS, Azure, GCP, etc.)
Interested candidate apply here or share your updated resume to deepika.subramanian@yitrobc.net
-
Quality Assurance Lead
2 weeks ago
Bengaluru, India Huquo Full timeRole Overview : As a Senior ETL QA Lead, you will be responsible for leading and executing the quality assurance strategy for ETL processes leveraging SnapLogic, DataStage, Informatica, Talend, and other relevant tools. You will collaborate closely with cross-functional teams to ensure the reliability, accuracy, and efficiency of our data integration...
-
ETL Pentaho Lead
5 days ago
Bengaluru, India Virtusa Full timeETL Pentaho Lead - CREQ187183 Description An ETL developer with Strong background on Pentaho ETL developments with large volumes of data set.Utilize agile methodologies and adhere to coding standards, procedures and techniques while contributing to the technical code documentationParticipate in design reviews and provide input to the design...
-
ETL Pentaho Lead
1 week ago
Bengaluru, India Virtusa Full timeETL Pentaho Lead - CREQ187183 Description An ETL developer with Strong background on Pentaho ETL developments with large volumes of data set. Utilize agile methodologies and adhere to coding standards, procedures and techniques while contributing to the technical code documentation Participate in design reviews and provide input to the design recommendations...
-
ETL Test Engineer
3 weeks ago
Bengaluru, India Corporate Resources Full timeJob DescriptionDeveloping ETL test cases and test plans that ensure data quality, accuracy, and completeness.Conducting functional and non-functional testing of ETL processes to validate the integrity of the data being transferred.Identifying and documenting defects, issues, and potential improvements in the ETL process and sharing them with the development...
-
ETL Developer
7 days ago
Bengaluru, India Niotact Digital Solutions Full timeLead the design, development, and optimization of ETL processes using Ab Initio's Graphical Development Environment (GDE), ensuring data accuracy, consistency, and availability. Data Integration: Build, maintain, and optimize data integration workflows, enabling seamless data flow across various systems and platforms. Experience in Data Transformation,...
-
ETL Developer
1 week ago
Bengaluru, India Niotact Digital Solutions Full timeLead the design, development, and optimization of ETL processes using Ab Initio's Graphical Development Environment (GDE), ensuring data accuracy, consistency, and availability. Data Integration: Build, maintain, and optimize data integration workflows, enabling seamless data flow across various systems and platforms. Experience in Data Transformation,...
-
QA Engineer
3 days ago
Bengaluru, India Syniverse Full timeSyniverse is the world’s most connected company. Whether we’re developing the technology that enables intelligent cars to safely react to traffic changes or freeing travelers to explore by keeping their devices online wherever they go, we believe in leading the world forward.Which is why we work with some of the world’s most recognized brands. Eight of...
-
Azure Data Engineer
2 weeks ago
Bengaluru, India Connectio IT Pvt Ltd Full timeRole Description :This is a full-time Azure Data Engineer role at Connect io IT Pvt Ltd located in Bengaluru. As an Azure Data Engineer, you will be responsible for tasks such as data engineering, data modeling, Extract Transform Load (ETL), data warehousing, and data analytics. You will work on-site in Bengaluru.Qualifications :- Data Engineering, Data...
-
Data Engineer
1 month ago
Bengaluru, India Mobile Programming LLC Full timeJob Description :Please note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will play a crucial role in developing and maintaining our data infrastructure and ensuring the smooth operation of our data platforms. The ideal candidate should have a strong...
-
ETL Datastage Developer
3 days ago
Bengaluru, India Quess Full timeAbout Us “Quess IT Staffing is India’s largest IT staffing company with over 20 years of experience in staffing IT professionals in 300+ companies across levels and skillsets. Our 10,000+ associates deployed in 80+ cities and towns are proficient in over 500 technological skills. Our associates help enable cutting edge solutions some of the biggest names...
-
ETL Developer
3 weeks ago
Bengaluru, India State Street Corporation Full timeETL Developer - Team LeadWho we are looking for:Looking for a strong database engineer worked on experience in Snowflake, Oracle. cloud engineer worked on various cloud migration and implementation at large scale. Have experienced on Azure cloud primarily on the architecture, design and implementation.Highly-motivated and thrive in an environment that...
-
Senior Software Engineer
3 weeks ago
Bengaluru, India JLL Full timeAbout the roleWe are looking for a Senior Resource to join our JLL Technologies - CoE organization in Bangalore, India. This role requires partnering with the broader JLL Technology team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experienceThe role will add immense...
-
Senior Software Engineer
1 month ago
Bengaluru, India JLL Full timeAbout the role We are looking for a Senior Resource to join our JLL Technologies - CoE organization in Bangalore, India. This role requires partnering with the broader JLL Technology team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience The role will add...
-
Data Engineer
1 week ago
Bengaluru, India Sterlite Technologies Limited Full timeRole as a Data engineer :- Develop and build AI/ML model ready data ponds using Azure Databricks Delta lake- Improve upon the data ingestion models, ETL jobs, and alarm to maintain data integrity and data availability.- Partner with AI/ML product teams proactively identify the data needs for rapid experimentation, stable production deployment and ensure the...
-
ETL/Databricks Engineer
2 weeks ago
Bengaluru, India RS Consultants Full timeETL (Databricks)Job description : Responsibilities :- Build, document, test and launch new data extract, transform and load (ETL) processes for one or more core data sets.- Collaborate with Business Intelligence team and other stakeholders for data requirements- Collaborate with the Data Management team to provide quality data to stakeholders.- Build data...
-
Senior Manager
3 weeks ago
Bengaluru, India Genpact Full timeGenpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...
-
Automation - ETL - test engineer - Tech Lead
1 week ago
Bengaluru, India State Street Corporation Full timeAutomation - ETL - test engineer - Tech LeadRole Summary & Role DescriptionAs Software Quality Assurance Engineer you willCreate test scenarios and implement functional test cases (automated and otherwise) that will serve as acceptance test criteria for enhancements and new applicationsWork closely with developers and quality assurance members to run test...
-
ETL Automation engineer
3 weeks ago
Bengaluru, India State Street Full timeRole Summary & Role DescriptionGoal of the role is to work on ETL automation. Will be reporting to me.5+ years of experience.As the ETL Automation Engineer in the Corporate Finance Techlology team you willDeveloping and maintaining robust ETL pipelines using PySpark to process large volumes of data efficiently.Writing and executing automated tests for ETL...
-
Senior Manager
1 month ago
Bengaluru, India Genpact Full timeGenpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people –...
-
Data Engineer
1 month ago
Bengaluru, India SKEP IT SYSTEMS Full timeRole : Data EngineerMobilution IT Systems is a pioneering organization at the forefront of technological innovation, specializing in cutting-edge Tech solutions. With a strong commitment to leveraging advanced technologies for scientific advancements, Mobilution IT Systems is dedicated to revolutionizing the way data is analysed and utilized across various...