ETL Automation Engineer
1 week ago
Role Summary & Role Description
As the ETL Automation Engineer in the Corporate Finance Techlology team you will
Developing and maintaining robust ETL pipelines using PySpark to process large volumes of data efficiently.
Writing and executing automated tests for ETL processes, ensuring data quality and integrity.
Utilizing PySpark's DataFrame API for data manipulation and transformation during ETL processes.
Implementing best practices for PySpark code to optimize performance and scalability.
Collaborating with data engineers and analysts to understand data requirements and ensure ETL processes align with business needs.
Troubleshooting and debugging ETL jobs in PySpark, identifying bottlenecks and performance issues.
Documenting ETL processes, test cases, and results for future reference andpliance with dataernance standards.
These skills will help you succeed in this role
8 to 9 years of experience in Software testing minimum 5 years experience in ETL Automation using PySpark.
Proficiency in Python and experience with the PySpark library.
Strong understanding of ETL concepts and experience in automating ETL testing.
Familiarity with Spark architecture and its optimization techniques.
Experience with version control systems, such as Git, for collaborative development.
Knowledge of SQL and experience working with various data storage systems.
Familiarity with the Hadoop ecosystem
Experience with platforms such as Hive,impala,AWS, Yarn and Quantexa.
Excellent analytical and problem-solving skills
Excellentmunication and interpersonal skills
Excellent knowledge in SDLC process - include creating, maintaining and executing automation/functional test plans, writing test scripts against software platforms,municating test results, defect tracking and participating in day to day QA activities.
Should be able to work independently with minimum inputs and directions. Will be expected to mentor and guide the team on ETL automation.Core/Must have skills
Strong in PySpark, Python, BigData
Acquainted of SQL queries, RDBMS and ETL concepts
Exposure to JIRA and IBM RTC.
Good to have skills
DataLake, DataBricks, Tableau, UNIX, Finance background a plus
ESP is a plus
Proficiency in Agile Methodology is plus.
Good to have UI automation knowledge
Work Schedule
On-premise
Why this role is important to us
Our technology function, Global Technology Services (GTS), is vital to State Street and is the key enabler for our business to deliver data and insights to our clients.
We're driving thepany's digital transformation and expanding business capabilities using industry best practices and advanced technologies such as cloud, artificial intelligence and robotics process automation.
We offer a collaborative environment where technology skills and innovation are valued in a global organization.We're looking for top technical talent to join our team and deliver creative technology solutions that help us be an end-to-end, next-generation financial servicespany.
Join us if you want to grow your technical skills, solve real problems and make your mark on our industry.
About State StreetWhat we do.
State Street is one of the largest custodian banks, asset managers and asset intelligencepanies in the world. From technology to product innovation, we're making our mark on the financial services industry. For more than two centuries, we've been helping our clients safeguard and steward the investments of millions of people. We provide investment servicing, data & analytics, investment research & trading and investment management to institutional clients.
Work, Live and Grow.
We make all efforts to create a great work environment. Our benefits packages arepetitive andprehensive. Details vary by location, but you may expect generous medical care, insurance and savings plans, among other perks. You'll have access to flexible Work Programs to help you match your needs. And our wealth of development programs and educational support will help you reach your full potential.
Inclusion, Diversity and Social Responsibility.
We truly believe our employees' diverse backgrounds, experiences and perspectives are a powerful contributor to creating an inclusive environment where everyone can thrive and reach their maximum potential while adding value to both our organization and our clients.
Another fundamental value at State Street is active engagement with ourmunities around the world, both as a partner and a leader.
You will have tools to help balance your professional and personal life, paid volunteer days, matching gift programs and access to employee networks that help you stay connected to what matters to you.
State Street is an equal opportunity and affirmative action employer.Discover more at StateStreet/careers
Job ID R-751533
-
ETL Automation engineer
1 week ago
Bengaluru, Karnataka, India State Street Corporation Full timeETL Automation engineer - Assistant Manager Role Summary & Role Description As the ETL Automation Engineer in the Corporate Finance Technology team you will Developing and maintaining robust ETL pipelines using PySpark to process large volumes of data efficiently. Writing and executing automated tests for ETL processes, ensuring data quality and...
-
ETL Automation Engineer
1 week ago
Bengaluru, Karnataka, India State Street Full timeRole Summary & Role DescriptionAs the ETL Automation Engineer in the Corporate Finance Techlology team you willDeveloping and maintaining robust ETL pipelines using PySpark to process large volumes of data efficiently.Writing and executing automated tests for ETL processes, ensuring data quality and integrity.Utilizing PySpark's DataFrame API for data...
-
ETL Automation Engineer
1 week ago
Bengaluru, Karnataka, India State Street Full timeRole Summary & Role Description As the ETL Automation Engineer in the Corporate Finance Techlology team you will Developing and maintaining robust ETL pipelines using PySpark to process large volumes of data efficiently. Writing and executing automated tests for ETL processes, ensuring data quality and integrity. Utilizing PySpark's DataFrame API for...
-
Bengaluru, Karnataka, India Tech Mahindra Full timeHi All,We Have Urgent opening for ETL Automation Engineer @Bangalore Location.Skill- ETL Automation EngineerLocation: Bangalore 3 8 years of QA experience Strong experience in functional testing and test automation Strong DB testing experience and strong experience in SQL Ability to write python scripts for automation Able to test the data ingestion/curation...
-
ETL Test Engineer
1 week ago
Bengaluru, Karnataka, India Corporate Resources Full timeJob DescriptionDeveloping ETL test cases and test plans that ensure data quality, accuracy, and completeness.Conducting functional and non-functional testing of ETL processes to validate the integrity of the data being transferred.Identifying and documenting defects, issues, and potential improvements in the ETL process and sharing them with the development...
-
ETL Test Engineer
1 week ago
Bengaluru, Karnataka, India Corporate Resources Full timeJob Description Developing ETL test cases and test plans that ensure data quality, accuracy, and completeness. Conducting functional and non-functional testing of ETL processes to validate the integrity of the data being transferred. Identifying and documenting defects, issues, and potential improvements in the ETL process and sharing them with the...
-
Bengaluru, Karnataka, India Tech Mahindra Full timeHi All,We Have Urgent opening for ETL Automation Engineer @Bangalore Location.Skill- ETL Automation EngineerLocation: Bangalore3 -8 years of QA experienceStrong experience in functional testing and test automationStrong DB testing experience and strong experience in SQLAbility to writepython scripts for automationAble to test the data ingestion/curation...
-
Bengaluru, Karnataka, India Tech Mahindra Full timeJOB OPENING: ETL Automation Engineer in Bangalore!3-8 years of QA experienceStrong functional testing and test automation skillsExpertise in DB testing and SQLProficiency in writing Python scripts for automationAbility to test data pipelines and reconcile data across platformsExperience with Teradata, Hadoop, and Snowflake ecosystemKnowledge of test...
-
ETL Tester
1 week ago
Bengaluru, Karnataka, India F335 Deutsche India Private Limited, Bangalore Branch Full timeTesting ETL processes to ensure data accuracy, completeness and integrity throughout the extraction, transformation and loading processes.Scripting knowledge to automate the processes.Expert in tools like SQL, Python/Java, ETL testing frameworks, Tableau or Google looker.Develop and automate test scripts for ETL workflows, data pipelines to ensure database...
-
Automation - ETL - test engineer - Tech Lead
1 week ago
Bengaluru, Karnataka, India State Street Corporation Full timeAutomation - ETL - test engineer - Tech LeadRole Summary & Role DescriptionAs Software Quality Assurance Engineer you willCreate test scenarios and implement functional test cases (automated and otherwise) that will serve as acceptance test criteria for enhancements and new applicationsWork closely with developers and quality assurance members to run test...
-
Automation - ETL - test engineer - Tech Lead
1 week ago
Bengaluru, Karnataka, India State Street Corporation Full timeAutomation - ETL - test engineer - Tech Lead Role Summary & Role Description As Software Quality Assurance Engineer you will Create test scenarios and implement functional test cases (automated and otherwise) that will serve as acceptance test criteria for enhancements and new applications Work closely with developers and quality assurance members to run...
-
ETL Tester
1 week ago
Bengaluru, Karnataka, India F335 Deutsche India Private Limited, Bangalore Branch Full timeDescription Testing ETL processes to ensure data accuracy, completeness and integrity throughout the extraction, transformation and loading processes. Scripting knowledge to automate the processes. Expert in tools like SQL, Python/Java, ETL testing frameworks, Tableau or Google looker. Develop and automate test scripts for ETL workflows, data...
-
ETL Tester
1 week ago
Bengaluru, Karnataka, India idigilogic Full timeCompany OverviewiDigilogic is a dynamic and innovative Information Technology Company with a global presence providing state of the art solutions in the Insurance & Healthcare technology and Banking industry. We champion the value of inventiveness coupled with a determination and grit to serve the rapidly changing global market.iDigilogic is a dynamic IT...
-
ETL Testing
1 week ago
Bengaluru, Karnataka, India Virtusa Full timeETL Testing - CREQ190685 Description Job Description Skill: ETL Developer Role: T2, T1 Key responsibility: For Quality engineer ETL Testing & AWS cloud platform (services like S3, EC2, MWAA, DAG, Airflow, Glue, Glue Data Catalog, Athena, QuickSight, Redshift) BDD, Automation using Python Good understanding of Testing phases in Agile Data storage...
-
ETL/Data Engineering Manager
1 week ago
Bengaluru, Karnataka, India SemperFi Solutions Full timeShift timings : 12pm to 9 PMTotal experience required : 15-20 years.Job Description : Lead and mentor a team of data engineers in designing, developing, and maintaining efficient and reliable ETL pipelines. Develop advanced scripts using Python for data extraction, transformation, and loading (ETL) processes. Architect robust ETL solutions to support diverse...
-
170726 - Automation Testing with Etl
1 week ago
Bengaluru, Karnataka, India Black & White Business Solutions Full timeJob Information:IndustryIT Services***ProvinceKarnataka***CityBangalore***Postal Code560001***CountryIndiaRoles and Responsibilities:Automation Tester required with below skill set:1) Python ,Azure Cloud (Azure databricks,Azure data lakes)2) Experience in ETL Testing, Dataware house concepts3) SQL (joins,subqueries,CTE)4) Power BI basic knowledgeWork closely...
-
ETL Testing
1 week ago
Bengaluru, Karnataka, India Quess Full timeAbout Us "Quess IT Staffing is India's largest IT staffing company with over 20 years of experience in staffing IT professionals in 300+ companies across levels and skillsets. Our 10,000+ associates deployed in 80+ cities and towns are proficient in over 500 technological skills. Our associates help enable cutting edge solutions some of the biggest names...
-
ETL Developer
1 week ago
Bengaluru, Karnataka, India Domniclewis Full timePlease refer to the following job description: Job Responsibilities : Prepare designs for all metadata in accordance with various Data Integration processes. Develop and test data integration (ETL) processes, including design, development, and testing. Analyze source systems to transition and acquire data into the enterprise data warehouse. Define and...
-
ETL Developer
1 week ago
Bengaluru, Karnataka, India Beroe Inc Full timeJob description:Proficient on SQL, PL/SQL scripts: stored procedures, functions, views etc to implement the business logic.Well versed in Enterprise Data Modelling (Dimension & Relational) with a strong understanding of the Data modelling concept like Star-Schema Modelling, Snowflake's modelling, and fact and dimension tables modelling of data.Hands-on...
-
Automation - ETL - test engineer - Tech Lead
3 weeks ago
Bengaluru, Karnataka, India State Street Full timeRole Summary & Role DescriptionAs Software Quality Assurance Engineer you willCreate test scenarios and implement functional test cases (automated and otherwise) that will serve as acceptance test criteria for enhancements and new applicationsWork closely with developers and quality assurance members to run test suites and research errors.Find ways to...
