AI Infrastructure Systems Engineer
- Level
- Mid
Not stated in the posting: salary, work type, visa.
Open. First seen 2 October 2026.
About the role
About the Role
At Together AI, you’ll build and operate one of the world’s largest GPU fleets used for frontier model training and inference. This isn’t a traditional infrastructure role—we’re looking for engineers who love building systems, automating everything, and solving problems at massive scale.
If you enjoy writing software more than clicking dashboards, obsess over eliminating manual work, and want to build infrastructure that manages tens of thousands of GPUs autonomously, we’d love to talk.
Hybrid at our office in Amsterdam or Remote in the UK.
Responsibilites
- Design and build fleet automation systems that provision, validate, deploy, upgrade, repair, and retire GPU clusters with minimal human intervention.
- Build AI Infrastructure Agents that automate deployment, root-cause failures, incident triage, and autonomous remediation.
- Develop Fleet Intelligence platforms that continuously monitor hardware health, firmware, networking, storage, thermals, and workload performance to predict failures before they impact customers.
- Build software that maximizes GPU availability, utilization, performance, and reliability across thousands of accelerators.
- Create automated validation systems for GPUs, InfiniBand/RoCE fabrics, NVLink/NVSwitch, storage, and distributed AI workloads.
- Build internal platforms and developer tools that allow infrastructure to be managed through software—not manual operations.
- Continuously improve deployment velocity, reliability, and operational efficiency through automation.
- Partner closely with hardware, networking, platform, and AI teams to push the limits of AI infrastructure.
Requirements
- 3+ years building distributed systems, infrastructure platforms, or large-scale backend software.
- Strong software engineering skills in Python, Go, or Rust.
- Experience building platforms, automation systems, or developer infrastructure.
- Experience with Linux, Kubernetes, Terraform, Ansible, or similar infrastructure technologies.
- Strong systems thinking with the ability to understand problems across hardware and software.
- A passion for solving complex infrastructure challenges through software.
- An automation-first mindset—if a task is repeated, your instinct is to build a system to eliminate it.
Bonus Experience
- GPU infrastructure, CUDA, NCCL, NVLink/NVSwitch
- InfiniBand or RoCE networking
- Bare-metal provisioning and lifecycle management
- Large-scale AI training or inference clusters
- Hardware health monitoring and predictive failure detection
- Distributed storage systems
- AI agents and autonomous infrastructure operations
Together AI's standard text
About Together AI
Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.
Application Policy
Please read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.
Equal Opportunity
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
Together AI's standard text
Please see our privacy policy at https://www.together.ai/privacy.
What this role asks for
Particular to this role
Embedded Systems
Rare in Together AI's other 51 open postings.
Standard for software engineering roles
Python
Common in software engineering postings at any company.
In most of Together AI's ads
Salesforce
Named in most of its open postings, whatever the role.
75% of this description is written for the role. The rest is Together AI's standard text, folded away below.
Compared with Together AI's other open jobs.
The team: Engineering HIRES STEADILY
This role is one of 28 open in Engineering. 3 were posted in the last 30 days and the oldest in February: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
Kubernetes (25 of 28), Salesforce (25 of 28), Golang (21 of 28), Ansible (17 of 28), Python (17 of 28), Terraform (15 of 28), Grafana (13 of 28)
OTHER ROLES ON THIS TEAM
- Engineering Manager, Kubernetes Bare Metal Infrastructure posted Nov 10
- (Senior/Junior AI Inference Reliability Engineer, Compute Infra posted Oct 27
- (Staff / Senior Staff) AI Inference Reliability Engineer, Compute Infra posted Oct 20
- Staff Software Engineer - AI Compute, Together Cloud posted Oct 16
- Staff Software Engineer - AI Compute, Together Cloud posted Sep 23
- Staff Engineer, Distributed Storage and HPC & AI Infrastructure posted Sep 11
- Senior Software Engineer — Infra Agent Systems posted Sep 10
- Senior Software Engineer — Infra Agent Systems posted Sep 6
- Senior Software Engineer — Infra Agent Systems UK posted Sep 1
- Senior Software Engineer — Infra Agent Systems Remote India posted Aug 25
- Senior Product Engineer, Fullstack posted Jul 27
- [Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering posted Jul 16
- AI Infrastructure System Engineer posted Jul 13
- Staff Platform Infrastructure Engineer posted Jul 10
- Senior Network Engineer posted Jul 7
- Data Center Operations Coordinator posted Jun 11
- Staff Engineer, Distributed Storage & AI Infrastructure posted Jun 4
- AI Infrastructure Systems Engineer posted May 15
- Lead/Manager AI Infra Systems Engineering Team posted Apr 1
- Senior Network Engineer (NL, India Remote or Singapore Remote) posted Jan 20
- Senior Software Engineer - AI Compute, Together Cloud posted Jan 20
- Senior Software Engineer, Observability posted Nov 8
- Senior Backend Engineer, Inference Platform posted Aug 22
- Senior Developer Productivity Engineer posted Jun 26
- Senior Software Engineer - Together Cloud Platform posted Jun 4
- Senior Software Engineer - AI Compute, Together Cloud posted Jun 3
- Infrastructure Security Lead posted Feb 12
From the team and dates on Together AI's own job board.
Similar jobs
- AI Developer at Salvo Software
Shares 7 skills: CUDA, Rust, Terraform… · same seniority - Senior Linux Systems Administrator at RapidFort, Inc.
Shares 7 skills: CUDA, Ansible, Embedded Systems… - Software Engineer - AI Compute Infrastructure at ByteDance
Shares 5 skills: CUDA, Rust, Golang… · same seniority - AI Infrastructure Engineer at Mercor
Shares 5 skills: CUDA, Ansible, Golang… · same seniority - HPC Systems Administrator at Helsing
Shares 6 skills: CUDA, Ansible, Embedded Systems… · same seniority - Lead Product Designer at Together AI
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.