AI Quality Engineer (Contract)
- Level
- Mid
Not stated in the posting: salary, work type, visa.
Open. First seen 3 October 2026.
About the role
AI Quality Engineer
Location:- Hyderabad
Work Mode:- WFO (5 days a week)
Role Type:- Contractual (3 months and extension would depend on project requirement and performance)
Key Responsibilities
- Test and validate AI-generated insights, recommendations, and decision-making workflows.
- Evaluate LLM and RAG systems for accuracy, relevance, consistency, factuality, and hallucinations.
- Validate retrieval quality, context relevance, grounding, and response quality in RAG systems.
- Test AI agents and autonomous workflows across functional, negative, and edge-case scenarios.
- Define AI evaluation criteria, test datasets, quality metrics, and validation processes.
- Perform regression testing for models, prompts, RAG configurations, and AI workflows.
- Collaborate with AI/ML engineers to identify issues and improve AI system quality.
Requirements
Required Skills
- Strong understanding of AI/ML and Generative AI testing.
- Hands-on experience testing LLM and RAG-based applications.
- Knowledge of LLM evaluation, hallucination detection, relevance, and response quality.
- Understanding of AI agents and recommendation systems.
- Strong analytical and problem-solving skills.
Good to Have
- Experience with PyTest and automated testing frameworks.
- Experience building automated AI evaluation and regression frameworks.
- Familiarity with tools such as RAGAS, DeepEval, LangSmith, or equivalent.
- Experience with CI/CD-based test automation, performance testing, or AI guardrails.
- Familiarity with cloud platforms (AWS/Azure/GCP) and observability tools.
Success Metrics
- High accuracy, relevance, and reliability of AI outputs.
- Strong evaluation coverage across critical AI workflows.
- Early detection and reduction of hallucinations and AI regressions.
- Reduced production AI quality issues.
- Increased confidence and trust in AI-generated insights and recommendations.
What this role asks for
Standard for QA / testing roles
Test Automation
Common in QA / testing postings at any company.
Compared with Side's other open jobs.
The team: Quality Assurance HIRES STEADILY
This role is one of 6 open in Quality Assurance. 2 were posted in the last 30 days and the oldest in October: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
Android (4 of 6), CI/CD (4 of 6), Agile (3 of 6), iOS (3 of 6), Test Automation (3 of 6), AWS (2 of 6), C# (2 of 6)
OTHER ROLES ON THIS TEAM
- Test Engineer (HYD/Client) posted Nov 17
- Unity Software Development Engineer in Test (SDET) posted Nov 11
- Quality Engineering (Contract - 1 month) posted Oct 15
- Trainee Test Engineer (HYD) posted Aug 28
- Senior QA Tester (talent pool) posted Oct 22
From the team and dates on Side's own job board.
Similar jobs
- Senior AI Evals Engineer at Rubrik Job Board
Shares 10 skills: Pytest, RAG, Test Automation… · same country - Senior QA Engineer at Apna
Shares 10 skills: Pytest, RAG, Test Automation… · same country - QA Engineer at CUBE
Shares 9 skills: Pytest, Test Automation, Generative AI… · same seniority · same country - QA Engineer at JazzX AI
Shares 9 skills: Pytest, RAG, Test Automation… · same seniority · same country - Senior QA Engineer - AI Products (6 Months Contract) at Arabic.AI
Shares 9 skills: Pytest, RAG, Test Automation… - Systemic Mission Designer at Side
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.