Runtime Engineer
- Salary
- $160,000 - $250,000
- Level
- Mid
- Work type
- Hybrid · Mountain View
Not stated in the posting: visa.
Open. First seen 2 October 2026.
About the role
What MatX is Building
MatX is building custom silicon for large-language-model inference and training, with HW/SW co-design across ISA, RTL, simulator, compiler, and kernels so each layer benefits from the others. The runtime owns the host-side stack and the contracts that bind those teams together.
What You'll Do Here
- Build the host-side interface library — device memory management, DMA, streams and events, sync primitives — that every compiler-emitted program runs on top of
- Own and extend the executable format: the compiler→runtime contract, its versioning, the weight and quantization layouts that let compiler and runtime evolve independently
- Design the custom-kernel ABI — calling convention, sync semantics, lifecycle — and the host-side marshaling layer (DLPack, the buffer protocol, numpy) that gets Python tensors to the device
- Build Python bindings via PyO3, with a C-ABI shim as the alternative integration path for downstream consumers
- Build the LLM inference serving stack — paged KV cache, continuous batching, request scheduling, token streaming — and the cluster orchestration primitives underneath it
- Bring up interconnect topology from the host and own the failure-detection and clean-teardown path for stop-restructure-resume recovery across racks
- Design what the chip exposes to host-side profilers and debuggers — perf counters, traces, and the Python surfaces ML engineers actually use — and hit measurable performance targets on runtime overhead and serving throughput
Who You Are
- Strong experience in a systems programming language — Rust, C, C++, or Go — including memory management, allocator design, and FFI/ABI work
- Have built Python interop layers in production (PyO3, ctypes, pybind11, or equivalent C-ABI bridging)
- Have designed and maintained API or ABI contracts between teams — versioning, evolution, breaking-change discipline — not just consumed someone else's
- Hands-on with at least one accelerator programming model (CUDA, ROCm, oneAPI Level Zero, TPU, or comparable) — enough to reason about device memory, async execution, and kernel launch
- ML-systems literate — comfortable with the training and inference loop, what collectives do, what a tensor layout is. Research depth not required.
Bonus Points If You Have
- LLM inference internals — vLLM, TensorRT-LLM, or SGLang (paged attention, scheduler design)
- Rust at depth, including proc macros, unsafe with soundness reasoning, and complex lifetime/trait work
- Custom allocator design (slab, paged, arena) or other low-level memory work
- ML framework integration experience (PyTorch custom backends, JAX/XLA, ONNX runtime)
- Profiler or tracing infrastructure work (perfetto, Nsight, or a custom stack)
- Driver-adjacent or kernel-bypass work, or prior new-silicon bring-up
Compensation
The US base salary for this full-time position is determined based on a variety of factors including role, experience, location, job-related skills, and relevant education and training. Career length is only a guideline for compensation.
- Early Career - $160,000 - $250,000 + equity
- Mid Career - $175,000 - $362,500 + equity
- Senior Career - $250,000 - $475,000 + equity
What We Offer
- Time off: 4 weeks PTO (accrued) + 12 company Holidays + 3 religious observance days
- Health:
- Medical. Company-subsidized Medical (Kaiser or Anthem) for employees & dependents
- Dental + Vision. Guardian Dental and Vision insurances for employee & dependents
- HSA and FSA offerings via Lively (HSA includes MatX lump sum contribution)
- Concierge Care Advocates via Mishe. A white-glove service to help you navigate the maze of insurance, find providers, file (and appeal) claims
- Travel medical insurance. Covers both business and personal trips
- MatX Family Fund. $6k fund via Benepass to fill the gaps in traditional insurance. Covers family-building, maternity, menopause, mental health and bodywork.
- Financial Wellbeing:
- Retirement. Choose from Roth/ 401K (or both) retirement plans with up to 5% company contribution to 401K (even if you don't contribute).
- Wealthfront Workplace Accounts. Non-401K investment, high-yield savings & brokerage. MatX pays your fees for the 1st $100K
- Life, AD&D and Disability. Company-paid life insurance (up to $300K) and long-term disability insurances.
- Daily Meals: MatX provides $80/day budget for onsite and remote meals, via a plethora of options including Webox, Locale, plus Ramp virtual cards.
- Commute on Us: Commute on our company Uber account, or reimburse your train rides. Either way, we pay 100% for your daily commute.
- WFH Support:
- Home Setup. $1500 to make your home office comfortable & productive for you
- Cell & Internet Reimbursement. $60/mo for cellular and $60/mo for wifi
- Professional Development: $1500 Professional Development Budget (per year)
- MatX E[x]tras: $50/mo to use on whatever perk you value most
- Mental Wellbeing: 100% paid mental health benefit via SpringHealth and Guardian EAP
- Support to Parents: Up to 20 weeks paid parental leave regardless of path to parenthood, up to 18 weeks pregnancy disability leave, and flexible return-to-work hours
MatX's standard text
As part of our dedication to the diversity of our team and our focus on creating an inviting and inclusive work experience, MatX is committed to a policy of Equal Employment Opportunity and will not discriminate against an applicant or employee on the basis of race, color, religion, creed, national origin or ancestry, sex, gender, gender identity, gender expression, sexual orientation, age, physical or mental disability, medical condition, marital/domestic partner status, military and veteran status, genetic information or any other legally recognized protected basis under federal, state or local laws, regulations or ordinances.
---------
This position requires access to information that is subject to U.S. export controls. This offer of employment is contingent upon the applicant's capacity to perform job functions in compliance with U.S. export control laws without obtaining a license from U.S. export control authorities.
--------
MatX does not accept unsolicited resumes from individual recruiters or third-party recruiting agencies in response to job postings. No fee will be paid to third parties who submit unsolicited candidates directly to our hiring managers or People team and any resumes submitted are deemed to be the property of MatX.
What this role asks for
Particular to this role
CUDA · Golang · Machine Learning · NumPy · PyTorch
Rare in MatX's other 35 open postings.
Standard for software engineering roles
Python
Common in software engineering postings at any company.
80% of this description is written for the role. The rest is MatX's standard text, folded away below.
Compared with MatX's other open jobs.
The team: Software HIRES STEADILY
This role is one of 4 open in Software. 2 were posted in the last 30 days and the oldest in May: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
C (3 of 4), C++ (3 of 4), CUDA (3 of 4), Rust (3 of 4), Machine Learning (2 of 4)
OTHER ROLES ON THIS TEAM
- Technical Editor posted Nov 17
- Software Engineer - Kernels posted Oct 28
- Lead Software Engineer, Kernels posted Sep 22
From the team and dates on MatX's own job board.
Similar jobs
- Software Engineer - AI Compute Infrastructure at ByteDance
Shares 8 skills: CUDA, PyTorch, Rust… · same seniority · same country - Computer Vision Engineer at Awetomaton
Shares 7 skills: NumPy, CUDA, PyTorch… · same seniority · same country - Software Engineer, Full Stack (Data & Video Analysis) at SpaceX
Shares 7 skills: NumPy, PyTorch, Rust… · same seniority · same country - Machine Learning Engineer at Mach Industries
Shares 6 skills: CUDA, PyTorch, Rust… · same seniority · same country - ML Research Engineer at Microsoft
Shares 6 skills: CUDA, PyTorch, C… · same seniority · same country - Technical Editor at MatX
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.