The role involves ensuring the quality, safety, and reliability of AI-driven capabilities within digital products. The engineer validates applications such as chatbots, copilots, RAG-based search, and agentic workflows. Responsibilities include designing and executing automated testing strategies across UI, APIs, and model-level interactions. The position requires integrating quality checks into CI/CD pipelines and evaluating AI outputs for accuracy, grounding, hallucinations, bias, and safety. Additionally, the role monitors production systems for performance issues and supports root-cause analysis and defect resolution.

Candidates must possess at least six years of experience in software quality assurance or test automation, specifically including work with AI/ML-based products. Proficiency in Python or JavaScript is required, along with experience testing RESTful APIs and utilizing automation frameworks. The role necessitates a solid foundation in QA best practices and experience testing RAG-based systems or agentic AI workflows. Additionally, familiarity with LLM evaluation frameworks and vector database APIs is essential.