Etl Pyspark

6 days ago

Chennai Tamil Nadu, India Virtusa Full time

The ETL Tester is responsible for validating and verifying data extraction transformation and loading processes to ensure data accuracy and integrity This role involves designing and executing test cases identifying data quality issues and collaborating with data engineers to resolve discrepanciesThe ETL Tester will also develop automated testing frameworks to enhance testing efficiency and coverage A strong understanding of ETL processes and data warehousing concepts is essential for success in this role
Minimum Qualification
At least 8 years of IT experience with minimum 5 years of ETL QA experience with below skills
Work experience in ETL testing
Proficiency in SQL with ability to write complex queries in big data ecosystem
Should possess PySpark Work experience
Testing experience in cloud environment AWS
S3 SSS Glue EMR Data Catalog Crawler SQS ETL Jobs Event Bridge
Athena Redshift Spectrum Databases
AWS Commands for running the jobs file process and movements
Experience with Test Automation and DevOps practices in ETL Cloud Systems
Extensive data analysis skills and knowledge in scripting to optimize data testing
Good to have knowledge in Healthcare Insurance
Experience working in Agile team
Should have excellent business communication to interact with Business teams and crossfunctional teams

**Responsibilities**:
Review requirements specifications and technical design documents to understand and derive test scenarios
Experience in developing and implementing test strategies for complex initiatives
Design develop and execute automation scripts
Identify record document thoroughly and track bugs
Strong problem solving skills troubleshooting and root cause analysis skills
Perform thorough regression testing when bugs are resolved
Liaise with cross functional teams to understand system requirements

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 30,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • Pyspark - Hyderabad

    7 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **RR: - 60137021** **Job Title: - Data Engineer with Pyspark** **Location: -**Kolkata** **Job Type 3 days from office (Hybrid)** **Responsibilities**: - 2+ years of Experience with AWS Cloud on data integration with Spark & AWS Glue/EMR - In-depth understanding of Spark architecture & distributed systems - Good exposure to Spark job optimizations -...

  • Gcp + Big Query + Python

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Design and develop robust ETL pipelines using Python, PySpark, and GCP services. Build and optimize data models and queries in BigQuery for analytics and reporting. Ingest, transform, and load structured and semi-structured data from various sources. Collaborate with data analysts, scientists, and business teams to understand data...

  • Pyspark Dev

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Pyspark Dev & QA Mandatory Skills BIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PySpark Build CI/CD pipelines is required Outstanding coding, debugging and analytical skills Spark APIs to cleanse, explore, aggregate, transform, store & analyse available data Knowledge of installing, configuring, debugging...

  • Etl Tester

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Role: ETL Tester 1) Minimum 5+ years of work experience in ETL testing with SQL hands-on 2) Preferred technical knowledge on PySpark and Cloud experience 3) Extensive data analysis skills and Good communication skills 4) Good to have knowledge in Healthcare Insurance. **About Virtusa** Teamwork, quality of life, professional and personal development:...

  • Pyspark

    6 days ago

    Chennai, India SRM Technologies Full time

    SRM Technologies is a global IT services provider offering Cloud and Infrastructure, Digital Transformation, Data and AI, Application Lifecycle, Technology Consulting, Quality Assurance, eCommerce and Product Engineering services. The key industries that benefit from our offerings include Automotive, Manufacturing and Consumer Goods. We offer our customers...

  • Etl Developer

    2 days ago

    Chennai, Tamil Nadu, India Decision Point Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Key Responsibilities:Design, develop, and optimize scalable and reliable ETL pipelines using Python and PySpark.Extract data from diverse data sources and transform it to meet analytical and business needs.Implement robust data validation, error handling, and quality checks within ETL pipelines.Work with large-scale datasets and ensure efficient performance...

  • ETL Support Engineer

    2 days ago

    Chennai, Tamil Nadu, India Star Health Insurance Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    ETL Support Engineer:Maintain ETL pipelines using Databricks and PySpark to extract, transform, and load data from various sources.Troubleshoot and resolve issues related to data transformations or loading processes in Databricks.Develop and maintain documentation for ETL processes, configurations, and data mappings.Strong proficiency in Databricks with...

  • Etl Testing

    20 hours ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Excellent in writing SQL Scripts to validate data in ODS and Mart. Should be able to develop and execute test plans, test cases, and test scripts. Strong in validation of Visual Analytics reports using tools like Power BI, MicroStrategy, etc., and running different types of tests within the same. Identifies, manages, and resolves defects during testing...

  • ETL Developer

    1 day ago

    Chennai, Tamil Nadu, India Decision Point Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Roles and Responsibilities:Design, develop, and maintain scalable ETL pipelines using PySpark and Python for large-scale data processing.Work closely with data architects and analysts to understand business requirements and translate them into technical solutions.Develop and optimize SQL queries for data extraction, transformation, and loading from various...

  • PySpark Developer

    5 days ago

    Bengaluru / Bangalore, Chennai, Pune, India beBeeData Full time US$ 1,04,000 - US$ 1,30,878

    Job SummaryWe are seeking a highly skilled PySpark expert with hands-on experience in Databricks to join our data engineering team. This role focuses on designing, building, and optimizing large-scale data pipelines and processing solutions on the Databricks Unified Analytics Platform.Key Responsibilities:Develop and manage data pipelines using PySpark on...