Pyspark QA

4 days ago

Bengaluru, India Virtusa Full time

**Skill: Pyspark QA**

**Role / Tier: T2/Senior Consultant***

**Required Skills**:
Experience with BIG Data technology mentioned below Hadoop Big Data HDFS,PYTHON,SPARK SQL MapReduce with PYSpark.

Hands on experience in build CI CD pipelines is required Outstanding coding, debugging and analytical skills,Core problem solving skills and data migration from different databases.

Good knowledge at using Spark APIs to cleanse,explore,aggregate,transform, store analyse available data and potential solutions,

eliminate possible solutions and select an optimal solution.

Experience in distributed processing, storage frameworks,RDD,Dataframe with operation like diff Action Transformation

Experience in UDF,Lambda,pandas,numpy.

Experience in installing,configuring, debugging and troubleshooting Hadoop clusters is desired skill.

Preferred skills

knowledge of deploying and managing ETL pipelines,RDBMS technologies PostgreSQL,MySQL,Oracle,etc.

knowledge with modern DevOps toolchains.

knowledge of dataframes,Pandas,data visualization tools,data mining.

knowledge of statistical ,machine learning models and business intelligence.

knowledge of JIRA,Bitbucket GitHub and scheduling tools like Automic Airflow.

knowledge of cloud-based infrastructure, AWS services EC2,EMR,S3,Lambda,EBS, IAM,Redshift,RDS etc.***

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 36,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • QA

    2 days ago

    Bengaluru, India Near Full time

    **Description**: You will join Near, one of the fastest-growing Enterprise SaaS companies (NASDAQ: NIR) andl experience a truly agile culture with the freedom to experiment and innovate.. At Near, we believe that great culture is not just about work; it’s work + life. We not only encourage our employees to dream big but also give them the freedom and the...

  • Data Quality Assurance Specialist

    12 hours ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Data Quality Assurance SpecialistWe are seeking an experienced Data Quality Assurance Specialist to develop and execute test strategies focused on data pipelines, including ingestion, transformation, storage, and retrieval.About the Role:Perform data integrity validation by verifying schema consistency, data completeness, and correctness across...

  • Data Pipeline Specialist

    3 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About the JobAre you ready for a new career challenge?We are currently seeking an ETL Developer to join our team.The successful candidate will have at least 5 years of experience in designing and implementing data pipelines using Databricks (PySpark/Scala/SQL).The role involves assessing current-state Informatica ETL jobs, mappings, workflows, and...

  • Senior Data Engineer

    1 day ago

    Bengaluru, Karnataka, India beBeeDataEngineeringExpert Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job DescriptionOur organization seeks an accomplished Data Engineering Expert to spearhead data engineering initiatives, requiring hands-on expertise in developing and maintaining scalable data pipelines. The ideal candidate will be proficient in Python, SQL, Pyspark, Airflow, AWS, Databricks, and data warehousing with a strong understanding of Agile...

  • Senior ETL Developer

    4 days ago

    Bengaluru, Karnataka, India Teamware Solutions Full time

    About the JobGreetings from Teamware Solutions a division of Quantum Leap Consulting Pvt Ltd Job Description:Role: ETL DeveloperExperience: 5-8 YrsLocation : BangaloreWork Mode: Hybrid (2days of office work)Open Positions - 5Interview Mode: Virtual InterviewNotice Period: Immediate to 20 Days preferredConnect for Faster Comm LinkedInJob...

  • Data Pipeline Architect

    1 day ago

    Bengaluru, Karnataka, India beBeeETLDeveloper Full time ₹ 1,20,00,000 - ₹ 1,50,00,000

    About the RoleWe are seeking a skilled Senior ETL Developer to join our team.The successful candidate will be responsible for evaluating and understanding current-state Informatica ETL jobs, mappings, workflows, and dependencies.Key responsibilities include:Evaluating and redesigning data pipelines using Databricks (PySpark/Scala/SQL) to optimize performance...

  • Senior ETL Developer

    4 days ago

    Bengaluru, Karnataka, India Teamware Solutions Full time

    About the JobGreetings from Teamware Solutions a division of Quantum Leap Consulting Pvt Ltd Job Description:Role: ETL DeveloperExperience: 5-8 YrsLocation : BangaloreWork Mode: Hybrid (2days of office work)Open Positions - 5Interview Mode: Virtual InterviewNotice Period: Immediate to 20 Days preferredConnect for Faster Comm Job Responsibilities:• Assess...

  • Senior ETL Developer

    2 days ago

    Bengaluru, Karnataka, India Teamware Solutions Full time

    About the Job Greetings from Teamware Solutions a division of Quantum Leap Consulting Pvt Ltd Job Description: Role: ETL Developer Experience: 5-8 Yrs Location : Bangalore Work Mode: Hybrid (2days of office work) Open Positions - 5 Interview Mode: Virtual Interview Notice Period: Immediate to 20 Days preferred Connect for Faster Comm LinkedIn ...

  • 15h Left: Data Engineer

    1 day ago

    Bengaluru, Karnataka, India HCLTech Full time

    Job DescriptionHiring for Data QA Engineer in HCLTechPrimary skill- Python, Pyspark, Postgres, Data Pipeline QALocation- Bangalore, Noida, Chennai, Hyderabad, PuneNotice Period- Only Immediate joinersExperience- 5 to 7 YearsInterested candidates share your resume to [Confidential Information]Develop and execute test strategies focused on data pipelines,...

  • Data Engineer

    17 hours ago

    Bengaluru, Karnataka, India HCLTech Full time

    Hiring for Data QA Engineer in HCLTechPrimary skill- Python, Pyspark, Postgres, Data Pipeline QALocation- Bangalore, Noida, Chennai, Hyderabad, PuneNotice Period- Only Immediate joinersExperience- 5 to 7 YearsInterested candidates share your resume to selvakumari.b@hcltech.comDevelop and execute test strategies focused on data pipelines, including ingestion,...