QA Engineer III

6 days ago

Hyderabad, Telangana, India RealPage Inc Full time ₹ 6,00,000 - ₹ 12,00,000 per year

Overview:

As a member of our Agentic AI Engineering team, we seek individuals passionate about advancing the quality and reliability of AI systems through rigorous evaluation, experimentation, and tooling. You will play a key role in assessing the performance, safety, and alignment of AI models, leveraging platforms like OpenAI and LangSmith. Our team thrives in a culture of curiosity, continuous learning, and collaborative innovation, where your contributions directly impact the trustworthiness of AI solutions used globally.

Responsibilities:

  • Design and execute evaluation strategies for LLM-based systems using LangSmith, OpenAI tools, and custom frameworks.
  • Collaborate with product managers, researchers, and engineers to define evaluation metrics aligned with business and ethical goals.
  • Develop automated pipelines for regression testing, prompt evaluation, and model comparison.
  • Analyze model outputs for correctness, bias, safety, and hallucination using both qualitative and quantitative methods.
  • Build reusable test harnesses and datasets for benchmarking AI capabilities across tasks and domains.
  • Contribute to the development of internal tools for prompt engineering, evaluation visualization, and feedback loops.
  • Act as a quality advocate for AI-driven features across the product lifecycle.
  • Mentor junior team members in AI evaluation best practices and tooling.
  • Stay current with advancements in LLM evaluation, interpretability, and responsible AI.

Qualifications:

  • Bachelor's or master's degree in computer science or a related technical field.
  • Minimum 6-8 years of experience into QA and Automation
  • Basic level understanding of software development lifecycle and quality best practices and tools
  • Good knowledge on Automation Framework Development
  • Proficient in the use of various automation frameworks like Playwright (preferred), Protractor, Rest Assured, karate etc.
  • Hands-on experience with OpenAI APIs (e.g., GPT-4, function calling, Assistants API).
  • Familiarity with OpenAI Evals and LangSmith for prompt orchestration and evaluation.
  • Strong Python skills, especially in building test automation and data analysis scripts.
  • Experience with prompt engineering, LLM fine-tuning, or RAG pipelines is a plus.
  • Proficiency in using tools like Jupyter, Pandas, and/or SQL for evaluation analytics.
  • Understanding of ethical AI principles, model alignment, and safety concerns.
  • Experience with version control (GitHub), CI/CD pipelines, and test management tools (e.g., JIRA, Azure Test Plans).
  • Excellent communication and collaboration skills, with the ability to explain complex AI behaviors to diverse stakeholders.

  • Quality Assurance Engineer III

    1 day ago

    Hyderabad, Telangana, India GHX Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    QA Automation Engineer III (India) — Quality Through EmpowermentLocation: Remote / Hybrid — Hyderabad, India Department: Product Engineering Reports to: Software Engineering Manager / Director of Engineering---About the RoleWe're looking for a QA Automation Engineer III who believes that quality isn't a final step — it's built in from day one. You'll...

  • Qa Engineer

    2 weeks ago

    Hyderabad, Telangana, India 3cc6483f-15b0-482e-a309-b0731c5b44ae Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Area Key Roles and Specific Responsibilities Accountabilities I. Test Strategy & Planning Test Ownership Review product specifications and user stories to identify test requirements and coverage gaps. Design and write comprehensive test plans, test cases, and test scenarios. Participate actively in Agile ceremonies (Sprint Planning, Grooming) and...

  • QA Engineer III

    1 week ago

    Hyderabad, Telangana, India RealPage Inc Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Overview:As a member of our Software Engineering Group, we look first and foremost for people who are passionate around solving business problems through innovation and engineering practices. You'll be required to apply your depth of knowledge and expertise to all aspects of the software development lifecycle, as well as partner continuously with your many...

  • QA Engineer III

    1 week ago

    Hyderabad, Telangana, India RealPage, Inc. Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    OverviewAs a member of our Software Engineering Group, we look first and foremost for people who are passionate around solving business problems through innovation and engineering practices. You'll be required to apply your depth of knowledge and expertise to all aspects of the software development lifecycle, as well as partner continuously with your many...

  • Consumer QA Analyst III

    2 weeks ago

    Hyderabad, Telangana, India Coinbase Full time ₹ 14,88,400 per year

    Ready to be pushed beyond what you think you're capable of?At Coinbase, our mission is to increase economic freedom in the world. It's a massive, ambitious opportunity that demands the best of us, every day, as we build the emerging onchain platform — and with it, the future global financial system.To achieve our mission, we're seeking a very specific...

  • Software Engineer III

    1 week ago

    Hyderabad, Telangana, India GHX Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Software Engineer III — Product-Driven Builder About the Role We're looking for a Software Engineer III who thrives in an environment where engineers own problems end-to-end — from understanding user needs to shipping scalable, maintainable solutions. You'll work within a small, empowered product team, building modern web applications that...

  • QA Engineer

    5 days ago

    Hyderabad, Telangana, India RAYVEN IT SOLUTIONS Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: QA EngineerLocation: HyderabadDuration: Long-termRequired SkillsAt least 10+ years of experience as a QA EngineerAt least 5+ years' Azure DevOps experience including developing Test Plans and experience in Automation testing using SeleniumShould have strong work experience on QA function within project TeamsShould Develop and maintain regression...

  • Quality Assurance Engineer III

    1 week ago

    Hyderabad, Telangana, India GHX Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    ROLE - QUALITY ASSURANCE ENGINEER IIIOverview:We are seeking a skilled Quality Assurance Engineer III to join our dynamic team. As a Quality Assurance Engineer, you will be responsible for designing, developing, and executing automated test scripts to ensure the quality and reliability of our software products. You will work closely with developers, QA...

  • Software Engineer III

    22 hours ago

    Hyderabad, Telangana, India F5 Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    At F5, we strive to bring a better digital world to life. Our teams empower organizations across the globe to create, secure, and run applications that enhance how we experience our evolving digital world. We are passionate about cybersecurity, from protecting consumers from fraud to enabling companies to focus on innovation.  Everything we do centers...

  • Lead QA Engineer/ QA Engineer

    4 days ago

    Hyderabad, Telangana, India Master-Works Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Master-Works is searching for a dedicated and detail-oriented QA Engineer / Lead QA Engineer to join our team. In this pivotal role, you will be responsible for developing and leading the quality assurance strategy for our software products. Your leadership will guide a team of QA professionals to ensure the delivery of high-quality applications that meet...