QA Engineer III

7 days ago

Hyderabad, Telangana, India RealPage Inc Full time ₹ 6,00,000 - ₹ 12,00,000 per year

Overview:

As a member of our Agentic AI Engineering team, we seek individuals passionate about advancing the quality and reliability of AI systems through rigorous evaluation, experimentation, and tooling. You will play a key role in assessing the performance, safety, and alignment of AI models, leveraging platforms like OpenAI and LangSmith. Our team thrives in a culture of curiosity, continuous learning, and collaborative innovation, where your contributions directly impact the trustworthiness of AI solutions used globally.

Responsibilities:

  • Design and execute evaluation strategies for LLM-based systems using LangSmith, OpenAI tools, and custom frameworks.
  • Collaborate with product managers, researchers, and engineers to define evaluation metrics aligned with business and ethical goals.
  • Develop automated pipelines for regression testing, prompt evaluation, and model comparison.
  • Analyze model outputs for correctness, bias, safety, and hallucination using both qualitative and quantitative methods.
  • Build reusable test harnesses and datasets for benchmarking AI capabilities across tasks and domains.
  • Contribute to the development of internal tools for prompt engineering, evaluation visualization, and feedback loops.
  • Act as a quality advocate for AI-driven features across the product lifecycle.
  • Mentor junior team members in AI evaluation best practices and tooling.
  • Stay current with advancements in LLM evaluation, interpretability, and responsible AI.

Qualifications:

  • Bachelor's or master's degree in computer science or a related technical field.
  • Minimum 6-8 years of experience into QA and Automation
  • Basic level understanding of software development lifecycle and quality best practices and tools
  • Good knowledge on Automation Framework Development
  • Proficient in the use of various automation frameworks like Playwright (preferred), Protractor, Rest Assured, karate etc.
  • Hands-on experience with OpenAI APIs (e.g., GPT-4, function calling, Assistants API).
  • Familiarity with OpenAI Evals and LangSmith for prompt orchestration and evaluation.
  • Strong Python skills, especially in building test automation and data analysis scripts.
  • Experience with prompt engineering, LLM fine-tuning, or RAG pipelines is a plus.
  • Proficiency in using tools like Jupyter, Pandas, and/or SQL for evaluation analytics.
  • Understanding of ethical AI principles, model alignment, and safety concerns.
  • Experience with version control (GitHub), CI/CD pipelines, and test management tools (e.g., JIRA, Azure Test Plans).
  • Excellent communication and collaboration skills, with the ability to explain complex AI behaviors to diverse stakeholders.

  • Senior QA Engineer

    5 days ago

    Hyderabad, Telangana, India Globe Centrix Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    About UsGlobe Centrix is a rapidly growing organization committed to delivering excellence in technology and business solutions. We partner with global clients across industries, ensuring the highest standards of quality and efficiency. To strengthen our software delivery capabilities, we are seeking an experiencedSenior QA Engineerwith expertise in...

  • Software Engineer III

    7 days ago

    Hyderabad, Telangana, India JPMorganChase Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job DescriptionWe have an exciting and rewarding opportunity for you to take your software engineering career to the next level.As a Software Engineer III at JPMorganChase within the Consumer and community banking- CBC technology, you serve as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure,...

  • QA Engineer

    2 weeks ago

    Hyderabad, Telangana, India Acesoft Labs Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Hi all ,Job location : HyderabadEmployment type : C2HNP : Immediate - 15 daysWere Hiring QA EngineerWe are seeking an experienced Quality Assurance (QA) Engineer with 57 years of experience to join our team. This role offers the opportunity to work on exciting projects, ensuring high-quality product delivery through both manual and automated testing.Role: QA...

  • QA Engineer

    3 days ago

    Hyderabad, Telangana, India RAYVEN IT SOLUTIONS Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: QA EngineerLocation: HyderabadDuration: Long-termRequired SkillsAt least 10+ years of experience as a QA EngineerAt least 5+ years' Azure DevOps experience including developing Test Plans and experience in Automation testing using SeleniumShould have strong work experience on QA function within project TeamsShould Develop and maintain regression...

  • QA Engineer

    7 days ago

    Hyderabad, Telangana, India HIGH FLOWS IT SOLUTIONS PVT LTD Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Role DescriptionThis is a full-time on-site role for a QA Engineer (Automation + Manual) located in Hyderabad. The QA Engineer will be responsible for executing test cases, performing manual and automated testing, and ensuring the overall quality of software. Day-to-day tasks include developing test plans, writing and executing test cases, identifying...

  • QA Engineer

    7 days ago

    Hyderabad, Telangana, India Aayuv Technologies Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    We are looking for a QA Engineer with a strong background in software testing to join our Technology team at ekincare. The role involves designing test cases, executing functional and regression testing, identifying bugs, and ensuring high-quality product releases that impact millions of users.Job Title: QA EngineerLocation: Hyderabad (Work from...

  • Lead QA Engineer/ QA Engineer

    5 days ago

    Hyderabad, Telangana, India Master-Works Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Master-Works is searching for a dedicated and detail-oriented QA Engineer / Lead QA Engineer to join our team. In this pivotal role, you will be responsible for developing and leading the quality assurance strategy for our software products. Your leadership will guide a team of QA professionals to ensure the delivery of high-quality applications that meet...

  • sdet iii

    2 weeks ago

    Hyderabad, Telangana, India S&P Global Market Intelligence Full time ₹ 1,20,000 - ₹ 3,00,000 per year

    The Impact: As an SDET III, you will make a significant contribution in testing/building Salesforce solutions to test applications across Web/Windows/Mobile/API/Services platforms and framing efficient data comparison methods. Your challenge will be reducing the time to market for products without compromising quality, by leveraging automation and...

  • QA Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enterprise Minds, Inc Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title:QA Engineering Lead (Manual + Automation + API)Experience:8–12 yearsJob Description:We are seeking a highly experienced QA professional with expertise inManual Testing, Test Automation, and API Testing. The ideal candidate should have strong hands-on skills inSelenium OR Katalon, OR Playwright, with a proven track record of designing and...

  • Software Engineer III

    2 weeks ago

    Hyderabad, Telangana, India JPMorganChase Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    JOB DESCRIPTIONWe have an exciting and rewarding opportunity for you to take your software engineering career to the next level.As a Software Engineer III at JPMorgan Chase within the Consumer and community banking technology team , you serve as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure,...