Engineering Manager, Kubernetes Bare Metal Infrastructure
- Salary
- $300,000 - $350,000
- Level
- Lead
Not stated in the posting: work type, visa.
Open. First seen 10 November 2026.
About the role
About The Role
We are hiring an Engineering Manager/Tech Lead Manager to lead the team that runs Kubernetes on bare-metal GPU infrastructure at Together AI. You will own how thousands of GPUs across our data centers are provisioned, orchestrated and kept healthy for the customers training, fine-tuning and serving models on our platform.
Together AI is the AI Native Cloud: a full-stack platform for running open and custom generative AI models, from GPU clusters to serverless inference. Our research and engineering teams contribute widely used open-source work such as FlashAttention, and our infrastructure runs at the frontier of GPU hardware.
This team sits at the base of that stack. Every training run, inference endpoint and customer cluster depends on the reliability, utilization and speed of the bare-metal Kubernetes layer you will lead.
Responsibilities
Own the bare-metal Kubernetes platform
- Set the technical direction for how we run Kubernetes on bare metal across multiple data centers and regions.
- Drive automated, zero-touch provisioning of GPU nodes, from firmware and OS imaging through cluster join and validation.
- Own cluster lifecycle at scale: upgrades, capacity expansion, node health checks, automated remediation and decommissioning.
- Make GPU scheduling, topology-aware placement and high-performance networking (InfiniBand, RoCE, RDMA) first-class in the platform.
Drive reliability and efficiency
- Define and meet SLOs for cluster availability, node bring-up time and GPU utilization.
- Invest in observability across hardware, networking, Kubernetes and workloads so failures are found before customers see them.
- Cut the time from racked hardware to customer-ready capacity.
Lead and grow the team
- Hire, manage and develop a team of infrastructure and platform engineers, setting a high bar for technical depth and operational ownership.
- Run the team's planning, prioritization and delivery, balancing roadmap work against fleet growth and production support.
- Build a healthy on-call culture: sustainable rotations, blameless postmortems and follow-through on remediation.
- Work with data center operations, networking, hardware vendors, inference, training and product teams to plan capacity and launch new hardware generations.
- Represent the platform's needs and trade-offs to leadership, and turn customer requirements into a clear roadmap.
Requirements
- 8+ years in infrastructure, SRE or platform engineering, including 2+ years managing engineers.
- Deep hands-on experience running Kubernetes in production at scale, ideally on bare metal rather than only managed services.
- Solid grounding in Linux systems, networking and hardware: kernels, drivers, BIOS/BMC, PXE boot and out-of-band management.
- Experience building automation for fleet provisioning and lifecycle with tools such as Cluster API, Metal3, Ironic, MAAS, Ansible or Terraform.
- A track record of owning production systems with on-call, SLOs and incident response.
- Strong technical judgment: you can review designs, debug alongside the team and make clear trade-offs.
- Clear communication with engineers, leadership and external partners.
Nice to have
- Experience operating large NVIDIA GPU clusters (H100, H200, B200, GB200) and the NVIDIA GPU and Network Operators, DCGM and NCCL.
- Familiarity with InfiniBand, RoCE, SR-IOV and high-performance storage for AI workloads.
- Experience with Slurm, Kueue, Volcano or other batch schedulers for training workloads.
- Contributions to Kubernetes or related CNCF projects.
- Experience at an AI lab, GPU cloud or hyperscaler.
Together AI's standard text
About Together AI
Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.
Compensation
We offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $300,000 -350,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Together AI's standard text
Application Policy
Please read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.
Equal Opportunity
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
What this role asks for
Particular to this role
Embedded Systems · Serverless
Rare in Together AI's other 51 open postings.
In most of Together AI's ads
Salesforce
Named in most of its open postings, whatever the role.
83% of this description is written for the role. The rest is Together AI's standard text, folded away below.
Compared with Together AI's other open jobs.
The team: Engineering HIRES STEADILY
This role is one of 28 open in Engineering. 3 were posted in the last 30 days and the oldest in February: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
Kubernetes (25 of 28), Salesforce (25 of 28), Golang (21 of 28), Ansible (17 of 28), Python (17 of 28), Terraform (15 of 28), Grafana (13 of 28)
OTHER ROLES ON THIS TEAM
- (Senior/Junior AI Inference Reliability Engineer, Compute Infra posted Oct 27
- (Staff / Senior Staff) AI Inference Reliability Engineer, Compute Infra posted Oct 20
- Staff Software Engineer - AI Compute, Together Cloud posted Oct 16
- Staff Software Engineer - AI Compute, Together Cloud posted Sep 23
- Staff Engineer, Distributed Storage and HPC & AI Infrastructure posted Sep 11
- Senior Software Engineer — Infra Agent Systems posted Sep 10
- Senior Software Engineer — Infra Agent Systems posted Sep 6
- Senior Software Engineer — Infra Agent Systems UK posted Sep 1
- Senior Software Engineer — Infra Agent Systems Remote India posted Aug 25
- Senior Product Engineer, Fullstack posted Jul 27
- [Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering posted Jul 16
- AI Infrastructure System Engineer posted Jul 13
- Staff Platform Infrastructure Engineer posted Jul 10
- Senior Network Engineer posted Jul 7
- Data Center Operations Coordinator posted Jun 11
- Staff Engineer, Distributed Storage & AI Infrastructure posted Jun 4
- AI Infrastructure Systems Engineer posted May 15
- Lead/Manager AI Infra Systems Engineering Team posted Apr 1
- Senior Network Engineer (NL, India Remote or Singapore Remote) posted Jan 20
- Senior Software Engineer - AI Compute, Together Cloud posted Jan 20
- Senior Software Engineer, Observability posted Nov 8
- Senior Backend Engineer, Inference Platform posted Aug 22
- Senior Developer Productivity Engineer posted Jun 26
- Senior Software Engineer - Together Cloud Platform posted Jun 4
- Senior Software Engineer - AI Compute, Together Cloud posted Jun 3
- AI Infrastructure Systems Engineer posted Apr 28
- Infrastructure Security Lead posted Feb 12
From the team and dates on Together AI's own job board.
See Engineering Manager salaries in San Francisco →
Similar jobs
- Senior Platform Engineer, Embedded Linux & Security at swan
Shares 5 skills: Serverless, Ansible, Embedded Systems… · same city - Staff Field Engineer | US | Remote at Grafana Labs
Shares 5 skills: Serverless, Salesforce, Terraform… · same country - Senior Software Engineer, AI Infrastructure-Compute Efficiency at ByteDance
Shares 3 skills: Serverless, Linux, Kubernetes · same country - Staff Software Engineer - Cloud Infrastructure at Arena Intelligence
Shares 3 skills: Serverless, Terraform, Kubernetes · same country - Senior Cloud & Backend Software Engineer at Orbit Neuro Co.
Shares 4 skills: Serverless, Embedded Systems, Terraform… · same city - Lead Product Designer at Together AI
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.