True Talent Solutions Partners

Software Engineer, RL Environments

San Francisco
Salary
$260,000 - $290,000 annually
Level
Mid
Work type
On-site · San Francisco
Visa
Sponsors visas

Open. First seen 28 October 2026.

Apply on the company's site →

About the role

About the Role

Design datasets, evaluation rubrics, and reward signals that help improve frontier AI models. You will work with research teams to test data-collection strategies, investigate model failure modes, and deliver evaluations and training data that inform model development.

What You'll Do

  • Design data slices that expose meaningful model failure modes across areas such as finance, software, and enterprise workflows.
  • Build and refine evaluation rubrics and reward signals for RLHF and RLVR training pipelines.
  • Model annotator behavior and run experiments to improve model capabilities.
  • Develop quantitative methods for assessing dataset quality, diversity, and downstream impact.
  • Create and manage real-world and synthetic data pipelines.
  • Translate research objectives into concrete data and evaluation specifications in collaboration with research teams.

What We're Looking For

  • One to four years of hands-on experience designing machine learning data-collection strategies and evaluation rubrics.
  • Strong familiarity with LLM training and evaluation methods, including experience creating data slices to reveal model failure modes.
  • Ability to design lightweight experiments, move quickly, and extract useful insights from messy results.
  • Comfort working across domains such as finance, software engineering, and policy.
  • A practical, builder-oriented approach and genuine interest in how data structure and quality shape model behavior.

Compensation & Benefits

Base salary: $260,000 to $290,000 annually. Visa sponsorship is available.

Location

On-site in San Francisco, California.

The team: Client Roles

This role is one of 105 open in Client Roles. 105 were posted in the last 30 days, including 5 lead roles: the team is hiring the people who run it too.

WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS

Python (56 of 105), TypeScript (43 of 105), LLM (37 of 105), React (37 of 105), Machine Learning (24 of 105), JavaScript (23 of 105), AWS (22 of 105)

OTHER ROLES ON THIS TEAM

From the team and dates on True Talent Solutions Partners's own job board.

Posting history

Reposted once since 25 October 2026: a posting for this role in San Francisco was taken down and the role was posted again.

  • 28 October 2026 – open (this posting)
  • 25 October 2026 – closed 26 October 2026

See Software Engineer salaries in San Francisco →

Similar jobs

More jobs like this

SKILLSMachine Learning 19,787LLM 18,620
PLACESSan Francisco 7,847United States 72,312
COMPANYJobs at True Talent Solutions Partners 105

Is this your posting? Email info@careerholo.com to have it removed.