True Talent Solutions Partners

Full-Stack Engineer, Agent Evaluation & Reliability

San Francisco
Salary
$120K - $180K
Level
Mid
Work type
On-site · San Francisco
Visa
Sponsors visas

This posting has closed on 13 November 2026.

About the role

About the Role

Join an AI team building backend infrastructure for agent evaluation and continuous improvement. This backend-leaning full-stack role offers ownership of architecture and product decisions while turning complex systems into useful customer-facing features.

What You'll Do

  • Build and maintain backend systems that power agent evaluation and continuous improvement.
  • Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings, and customer-defined quality criteria.
  • Develop and support enterprise integrations, including authentication, OAuth, webhooks, incremental syncs, backfills, rate limits, retries, and monitoring.
  • Own features end to end, from technical design and implementation through deployment, observability, customer feedback, and iteration.
  • Work directly with customers to diagnose system failures and turn individual needs into reusable product capabilities.
  • Debug distributed workflows and improve system reliability.

What We're Looking For

  • Two to four years of experience in a full-stack or backend engineering role building and shipping production systems.
  • Production experience with TypeScript and Node.js or Bun, PostgreSQL schema design, and queues or worker systems.
  • Hands-on machine learning engineering experience, including model development, and familiarity with agent infrastructure, LLM evaluation, tracing, or agent memory systems.
  • Experience building backend systems and third-party integrations, and owning projects from design through deployment and iteration.
  • Understanding of how evaluating nondeterministic agent systems differs from conventional unit testing.
  • Experience in a fast-paced startup environment and a track record of building, such as through shipped AI projects or open-source contributions.

Compensation & Benefits

Base salary is $120,000 to $180,000 USD annually. Visa sponsorship is available, including H-1B transfers, OPT, and O-1.

Location

On-site in San Francisco, California, United States.

The team: Client Roles

This role is one of 128 open in Client Roles. 128 were posted in the last 30 days, including 7 lead roles: the team is hiring the people who run it too.

WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS

Python (69 of 128), TypeScript (44 of 128), React (39 of 128), LLM (38 of 128), Machine Learning (34 of 128), JavaScript (28 of 128), AWS (26 of 128)

OTHER ROLES ON THIS TEAM

From the team and dates on True Talent Solutions Partners's own job board.

Posting history

Reposted 4 times since 25 October 2026: a posting for this role in San Francisco was taken down and the role was posted again.

Similar jobs

More jobs like this

Is this your posting? Email [email protected] to have it removed.