Side

AI Quality Engineer (Contract)

Hyderabad
Level
Mid

Not stated in the posting: salary, work type, visa.

Open. First seen 3 October 2026.

About the role

AI Quality Engineer

Location:- Hyderabad
Work Mode:- WFO (5 days a week)
Role Type:- Contractual (3 months and extension would depend on project requirement and performance)

Key Responsibilities

  • Test and validate AI-generated insights, recommendations, and decision-making workflows.
  • Evaluate LLM and RAG systems for accuracy, relevance, consistency, factuality, and hallucinations.
  • Validate retrieval quality, context relevance, grounding, and response quality in RAG systems.
  • Test AI agents and autonomous workflows across functional, negative, and edge-case scenarios.
  • Define AI evaluation criteria, test datasets, quality metrics, and validation processes.
  • Perform regression testing for models, prompts, RAG configurations, and AI workflows.
  • Collaborate with AI/ML engineers to identify issues and improve AI system quality.

Requirements

Required Skills

  • Strong understanding of AI/ML and Generative AI testing.
  • Hands-on experience testing LLM and RAG-based applications.
  • Knowledge of LLM evaluation, hallucination detection, relevance, and response quality.
  • Understanding of AI agents and recommendation systems.
  • Strong analytical and problem-solving skills.

Good to Have

  • Experience with PyTest and automated testing frameworks.
  • Experience building automated AI evaluation and regression frameworks.
  • Familiarity with tools such as RAGAS, DeepEval, LangSmith, or equivalent.
  • Experience with CI/CD-based test automation, performance testing, or AI guardrails.
  • Familiarity with cloud platforms (AWS/Azure/GCP) and observability tools.

Success Metrics

  • High accuracy, relevance, and reliability of AI outputs.
  • Strong evaluation coverage across critical AI workflows.
  • Early detection and reduction of hallucinations and AI regressions.
  • Reduced production AI quality issues.
  • Increased confidence and trust in AI-generated insights and recommendations.

What this role asks for

Standard for QA / testing roles

Test Automation

Common in QA / testing postings at any company.

Compared with Side's other open jobs.

The team: Quality Assurance HIRES STEADILY

This role is one of 6 open in Quality Assurance. 2 were posted in the last 30 days and the oldest in October: a team that hires all year rather than one being built.

WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS

Android (4 of 6), CI/CD (4 of 6), Agile (3 of 6), iOS (3 of 6), Test Automation (3 of 6), AWS (2 of 6), C# (2 of 6)

OTHER ROLES ON THIS TEAM

From the team and dates on Side's own job board.

Similar jobs

More jobs like this

Is this your posting? Email [email protected] to have it removed.