True Talent Solutions Partners

Full-Stack Engineer, Agent Evaluation & Reliability

San Francisco
Salary
$120,000 - $180,000 annually
Level
Mid
Work type
On-site · San Francisco
Visa
Sponsors visas

Open. First seen 3 November 2026.

Apply on the company's site →

About the role

About the Role

Join an AI/ML engineering team building infrastructure for agent evaluation and continuous improvement. This backend-leaning full-stack role combines foundational systems work with end-to-end product delivery, helping make agent workflows more reliable for enterprise customers.

What You'll Do

  • Build backend systems that power agent evaluation and continuous-improvement workflows.
  • Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings, and customer-defined quality criteria.
  • Build and maintain enterprise integrations, including authentication, OAuth, webhooks, incremental syncs, backfills, rate-limit handling, retries, and monitoring.
  • Own features from technical design and implementation through deployment, monitoring, customer feedback, and iteration.
  • Work with customers to understand their systems, diagnose failures, and turn specific needs into reusable product capabilities.
  • Take features through the full stack while contributing to architecture and product decisions.

What We're Looking For

  • Two to four years of experience in a full-stack or backend engineering role, building and shipping production systems.
  • Production experience with TypeScript and Node.js or Bun.
  • Hands-on machine learning model development experience, in addition to software engineering experience.
  • Experience at a fast-paced startup, from early stage through Series D.
  • Production experience designing PostgreSQL schemas and working with queues or worker systems.
  • A track record of owning projects end to end and building backend systems involving data modeling, integrations, observability, or distributed workflow debugging.
  • Understanding of how evaluating nondeterministic agent systems differs from conventional unit testing.
  • Experience with agent infrastructure, LLM evaluation, tracing, or agent memory systems in production or through substantial side projects is useful.

Compensation & Benefits

Base salary is $120,000 to $180,000 annually. Visa sponsorship is available, including H-1B transfers, OPT, and O-1.

Location

On-site in San Francisco, California, United States.

What this role asks for

Particular to this role

OAuth

Rare in True Talent Solutions Partners's other 123 open postings.

Standard for full stack roles

JavaScript · Node.js · PostgreSQL · TypeScript

Common in full stack postings at any company.

Compared with True Talent Solutions Partners's other open jobs.

The team: Client Roles

This role is one of 127 open in Client Roles. 127 were posted in the last 30 days, including 7 lead roles: the team is hiring the people who run it too.

WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS

Python (66 of 127), LLM (48 of 127), TypeScript (46 of 127), React (44 of 127), JavaScript (27 of 127), Machine Learning (27 of 127), AWS (25 of 127)

OTHER ROLES ON THIS TEAM

From the team and dates on True Talent Solutions Partners's own job board.

Posting history

Reposted 2 times since 25 October 2026: a posting for this role in San Francisco was taken down and the role was posted again.

Similar jobs

More jobs like this

SKILLSMachine Learning 19,843LLM 18,642JavaScript 15,430TypeScript 13,188
PLACESSan Francisco 7,728United States 73,429
COMPANYJobs at True Talent Solutions Partners 127

Is this your posting? Email info@careerholo.com to have it removed.