PySpark QA

1 week ago


Hyderabad, India Virtusa Full time
PySpark QA - CREQ Description

JOB DESCRIPTION

Skill: PySpark QA

Role / Tier: Lead Software Engineer/Tier 2

Experience: 6 – 9 years

Job Description:

Primary Skills

BIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PYSpark.
build CI/CD pipelines
Spark APIs to cleanse, explore, aggregate, transform, store & analyse
installing, configuring, debugging and troubleshooting Hadoop clusters

Secondary Skills

Cloud-based infrastructure , AWS services EC2, EMR, S3, Lambda, EBS, IAM, Redshift, RDS
deploying and managing ETL pipelines, RDBMS technologies (PostgreSQL, MySQL, Oracle, etc.).
knowledge of data frames, Pandas , data visualization tools & data mining .
knowledge of JIRA, Bitbucket GitHub

JD
Seeking a developer to develop, enhance, in a Pyspark -QA role.
Responsibility
Experience with BIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PYSpark.
Hands-on experience in build CI/CD pipelines is required Outstanding coding, debugging and analytical skills, Core problem solving skills
eliminate possible solutions and select an optimal solution.
Experience in installing, configuring, debugging and troubleshooting Hadoop clusters is desired skill.

Primary Location Hyderabad, Andhra Pradesh, India Job Type Experienced Primary Skills Data Modelling Years of Experience 5 Travel No
  • PySpark QA

    1 week ago


    Hyderabad, India Virtusa Full time

    PySpark QA - CREQ Description JOB DESCRIPTION Skill: PySpark QA Role / Tier: Lead Software Engineer/Tier 2 Experience: 6 – 9 years Job Description: Primary Skills BIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PYSpark. build CI/CD pipelines Spark APIs to cleanse,...

  • Pyspark Developer

    3 weeks ago


    Hyderabad, Chennai, Pune, India Innova solutinos Full time

    Job Description - Develop and optimize scalable data pipelines using Databricks and Apache Spark. - Design and implement solutions for data integration, data processing, and data storage on Databricks. - Collaborate with data scientists and analysts to build end-to-end analytics solutions. - Build and maintain automated workflows, ensuring reliability and...


  • Hyderabad, Telangana, India Cognizant Full time

    **Exp: 9.5 to 13 years** **Skill: ETL QA with Azure data bricks and Pyspark** **Location : Bangalore/Hyderabad/Kolkata/Pune/Chennai** **Technical Skills**: Tableau ,Azure SQL Domain Skills **Roles & Responsibilities : -** Develop and maintain Tableau dashboards to provide actionable insights for various business units - Utilize Azure SQL to manage and...

  • Python Developer

    2 weeks ago


    Hyderabad, India PamTen Software Solutions Private Limited Full time

    Role : Python DeveloperLocation : HyderabadExperience : 5+ years- Develop clean, efficient, and scalable backend solutions using Python.- Design and build RESTful APIs and integrate with third-party services.- Work with frameworks like Pyspark, Pandas, SQLAlchemy, Django, Flask, or FastAPI to deliver robust applications.- Implement data storage solutions...

  • Sr. QA ETL Engineer

    2 weeks ago


    Gurugram, Hyderabad, India Spectramedix Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Designation: Sr. QA Engineer (ETL Testing)Experience: 6 - 8 YearsFull time | SpectraMedix | Hyderabad/GurgaonRoles & Responsibilities• Experience in understanding ETL Source to Target Mapping Document and create QA documents and artifacts accordingly.• Create test data according to new features/data platform upgrades and Business Rules.• Mentor the QA...


  • Hyderabad, India Insight Global, LLC Full time

    Job Description :The Data & Insights organization at Insight Global delivers Trusted Data & Verified Insights to drive strategic decision-making across the company. As the first Data Engineering Manager based in Hyderabad, India, you will play a foundational role in establishing and growing a high-performing offshore team. You will lead efforts to build and...


  • Hyderabad, Telangana, India Caspex Corp Full time ₹ 2,50,000 - ₹ 7,50,000 per year

    Title: QA Engineer with Python, Batch TestLocation: Hyderabad, India (Need to work from Experian- Hyderabad office at least 3 days in a week from day one)Experience: 6+Job Description:Job Responsibilities: - Design, develop, and execute automated and manual batch test cases for data processing pipelines - Validate data integrity and transformation logic...

  • Bigdata Validation

    2 weeks ago


    Hyderabad, Telangana, India Cognizant Full time

    **Job Summary** 4-8 Years of QA experience in Bigdata Ecosystem -Hortonworks (HiveQL HDFS) Sqoop Spark Strong in SQL and work experience in Azure data bricks PySpark.. Having knowledge in Scripting Python and Control-M is an added advantage.. **Responsibilities** 4-8 Years of QA experience in Bigdata Ecosystem -Hortonworks (HiveQL HDFS) Sqoop Spark...

  • Big Data Developer

    3 weeks ago


    Hyderabad, Bengaluru, Chennai, India Innova solutinos Full time

    Job Description Role & responsibilities Must have :Python(4), PySpark(4), Hive(4) Preferred candidate profile - 6 to 8 years of experience Exclusively in the below areas - People with Strong DE experience along with Python , Pyspark, Hive as primary skills. Must to Have: - Hands-on with Pyspark & python - Hive, Hadoop, worked on-prem cluster - Handled...

  • Big Data Engineer

    3 weeks ago


    Hyderabad, Bengaluru, Chennai, India Innova solutinos Full time

    Job Description Role & responsibilities Must have :Python(4), PySpark(4), Hive(4) 6 to 8 years of experience Exclusively in the below areas People with Strong DE experience along with Python , Pyspark, Hive as primary skills. Must to Have: - Hands-on with Pyspark & python - Hive, Hadoop, worked on-prem cluster - Handled Large volume Preferred...