Etl Pyspark

1 day ago

Chennai Tamil Nadu, India Virtusa Full time

The ETL Tester is responsible for validating and verifying data extraction transformation and loading processes to ensure data accuracy and integrity This role involves designing and executing test cases identifying data quality issues and collaborating with data engineers to resolve discrepanciesThe ETL Tester will also develop automated testing frameworks to enhance testing efficiency and coverage A strong understanding of ETL processes and data warehousing concepts is essential for success in this role
Minimum Qualification
At least 8 years of IT experience with minimum 5 years of ETL QA experience with below skills
Work experience in ETL testing
Proficiency in SQL with ability to write complex queries in big data ecosystem
Should possess PySpark Work experience
Testing experience in cloud environment AWS
S3 SSS Glue EMR Data Catalog Crawler SQS ETL Jobs Event Bridge
Athena Redshift Spectrum Databases
AWS Commands for running the jobs file process and movements
Experience with Test Automation and DevOps practices in ETL Cloud Systems
Extensive data analysis skills and knowledge in scripting to optimize data testing
Good to have knowledge in Healthcare Insurance
Experience working in Agile team
Should have excellent business communication to interact with Business teams and crossfunctional teams

**Responsibilities**:
Review requirements specifications and technical design documents to understand and derive test scenarios
Experience in developing and implementing test strategies for complex initiatives
Design develop and execute automation scripts
Identify record document thoroughly and track bugs
Strong problem solving skills troubleshooting and root cause analysis skills
Perform thorough regression testing when bugs are resolved
Liaise with cross functional teams to understand system requirements

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 30,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • Etl Testing with Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    The ETL Tester is responsible for validating and verifying data extraction transformation and loading processes to ensure data accuracy and integrity This role involves designing and executing test cases identifying data quality issues and collaborating with data engineers to resolve discrepanciesThe ETL Tester will also develop automated testing frameworks...

  • Etl Databricks with Aws

    5 days ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Develop and maintain a metadata driven generic ETL framework for automating ETL code Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS. Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat files, streaming). Develop and maintain robust data pipelines for batch and streaming data using Delta...

  • Aws + Pyspark

    1 day ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Skills - AWS +Pyspark** **Experience: 4 to 9 years** **Location: AIA-Pune** **As an AWS Data Engineer, you will design, build, and manage robust data pipelines on the AWS cloud platform. You will leverage your expertise in AWS data services and programming languages to ingest, process, transform, and store data efficiently, enabling data-driven insights...

  • Pyspark / Reltio

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Develop and maintain data pipelines using PySpark in distributed computing environments (e.g., AWS EMR, Databricks). Integrate and synchronize data between enterprise systems and the Reltio MDM platform. Design and implement data transformation, cleansing, and enrichment processes. Collaborate with data architects, business analysts,...

  • Data Engineer

    1 week ago

    Chennai, India Ixceed Solutions Full time

    Role : Data Engineer Location : Chennai (Hybrid).Job Description : Key Responsibilities : - Lead the design, development, and deployment of scalable data pipelines and ETL processes using Scala and Apache Spark (Databricks). - Develop and maintain data processing workflows using PySpark and Python. - Collaborate with data scientists, analysts, and business...

  • ETL Tester

    3 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Role ; ETL TesterRequired Technical Skill Set : (ETL)SQL and Python programming languagesLocation: Pan IndiaMust-Have ETL Knowledge, SQL Programming, Azure, Databricks, Python Good-to-Have Python Automation ,Databricks,Pyspark,QuerySurge Automation Responsibility of / Expectations from the Role 1 Should have experience in database/ETL concepts & testing 2...

  • Etl Developer

    1 week ago

    Chennai, Tamil Nadu, India Decision Point Full time ₹ 2,50,000 - ₹ 7,50,000 per year

    Key Responsibilities:Design, develop, and optimize scalable and reliable ETL pipelines using Python and PySpark.Extract data from diverse data sources and transform it to meet analytical and business needs.Implement robust data validation, error handling, and quality checks within ETL pipelines.Work with large-scale datasets and ensure efficient performance...

  • ETL Support Engineer

    1 week ago

    Chennai, Tamil Nadu, India Star Health Insurance Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    ETL Support Engineer:Maintain ETL pipelines using Databricks and PySpark to extract, transform, and load data from various sources.Troubleshoot and resolve issues related to data transformations or loading processes in Databricks.Develop and maintain documentation for ETL processes, configurations, and data mappings.Strong proficiency in Databricks with...

  • ETL Developer

    2 weeks ago

    Chennai, Tamil Nadu, India Zion Hr Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    a) Pysparkb) PythonAWS experience 8+ Years of experience in ETL development

  • Pyspark Architect

    4 weeks ago

    Chennai, India Saaki Argus & Averil Consulting Full time

    About Company: A global IT services and consulting company specializing in digital transformation, cloud services, and automation solutions. Role : Pyspark Architect Experience : 15 -22 Years Location : Mumbai, Pune, Chennai, Bangalore & Noida. Job Description: Proven experience as a Technical Architect or similar role with a...