Senior Backend Engineer, Infrastructure
- Salary
- $160K - $250K
- Level
- Senior
- Work type
- Hybrid · San Francisco
Not stated in the posting: visa.
Open. First seen 2 October 2026.
About the role
Ellipsis Health is creating cutting-edge AI/ML products that solve healthcare staffing challenges and administrative burdens using conversational AI and our patented vocal biomarker technology, helping deliver better healthcare experiences for clients and patients. We are headquartered in Silicon Valley and are funded and supported by preeminent venture capital groups and healthcare institutions.
In this role, you will be a core contributor on our AI Research & Engineering team, building the real-time voice infrastructure behind our conversational AI products. You will design and run the Python micro-services that power live voice conversations, from audio capture and streaming transcription to LLM logic, synthesis, and playback. You will work with ML, product, data engineering, and client-SDK teams to deliver secure, low-latency, HIPAA-compliant clinical applications.
Responsibilities:
Real-Time Voice Pipeline: Expand and maintain our real-time voice pipeline by designing, implementing, and maintaining Python micro-services for conversational AI orchestration, including audio capture, streaming transcription, prompt/LLM logic, synthesis, and playback.
Providers, Transports & APIs: Integrate new providers and transports by adding plug-ins for emerging ASR, TTS, LLM, and memory services; wiring up WebRTC, SIP, or phone endpoints; and building adapters that allow hot-swapping components without downtime. Build API endpoints.
Ultra-Low Latency: Deliver sub-500 ms round-trip latency by profiling async pipelines (asyncio, FastAPI, gRPC) and optimizing buffering, concurrency, and back-pressure handling.
Observability: Instrument and observe every hop by emitting structured traces (OpenTelemetry), metrics, and logs for each pipeline stage, and define SLOs for first-token latency, end-to-end latency, and streaming reliability.
Production Hardening: Implement graceful retries, idempotent message passing, circuit breakers, and HIPAA-compliant security (encryption in transit, per-tenant isolation, secrets rotation).
Cross-Functional Collaboration: Work with ML, product, data engineering, and client-SDK teams to deliver features such as voice cloning, multimodal hand-offs, and domain-specific memory retrieval.
Minimum Qualifications:
- 4+ years building production back-ends in modern Python.
- Proven experience with real-time streaming systems (WebRTC, WebSockets, or gRPC streaming) and proficiency with asyncio, FastAPI, or similar async frameworks.
- Deep understanding of concurrency, buffering, audio codecs (Opus, PCM), and distributed tracing.
- Solid understanding of AWS/GCP/Azure, including container orchestration (Kubernetes/EKS/GKE), message queues (Kafka/SQS/Pub/Sub), and IaC (Terraform).
- Solid grasp of relational (PostgreSQL) and in-memory (Redis) data stores, with the ability to model and persist conversational state.
- Excellent communication skills and a bias for measured, observable, and continuously deployable software.
Preferred Qualifications:
- B.S./M.S. in Computer Science, Electrical Engineering, or a related field.
- Familiarity with voice-agent frameworks.
- Hands-on experience with telephony (Twilio, Telnyx), SIP, or PSTN integrations.
- Experience integrating multimodal inputs (vision, text chat) into voice agents.
- Familiarity with GPU inference and streaming pipelines.
- Prior work in regulated industries (healthcare, finance) and comfort preparing for SOC 2 / HIPAA audits.
Salary and Benefits:
We offer competitive salary and benefits, including 401(k) matching, health, vision, and dental insurance, and very flexible paid time off.
The typical salary range for this role is $160,000 to $250,000 USD, depending on skills, qualifications, and relevant experience.
Ellipsis Health's standard text
Background Checks:
As a health technology company, we reserve the right to run background checks on candidates to whom we extend offers, in compliance with applicable laws. We evaluate candidates holistically and comply with all “ban the box” regulations.
Assistance:
If you have a disability or require accommodations during the application or recruitment process, please contact [email protected].
What this role asks for
Particular to this role
FastAPI · Kafka · OpenTelemetry · Redis · Terraform · WebRTC · WebSockets · gRPC
Rare in Ellipsis Health's other 12 open postings.
Standard for backend roles
AWS · Kubernetes · PostgreSQL · Python
Common in backend postings at any company.
In most of Ellipsis Health's ads
LLM
Named in most of its open postings, whatever the role.
89% of this description is written for the role. The rest is Ellipsis Health's standard text, folded away below.
Compared with Ellipsis Health's other open jobs.
The team: Engineering HIRES STEADILY
This role is one of 8 open in Engineering. 3 were posted in the last 30 days and the oldest in May: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
AWS (5 of 8), CI/CD (5 of 8), GCP (5 of 8), Kubernetes (5 of 8), LLM (5 of 8), Machine Learning (5 of 8), Python (5 of 8)
OTHER ROLES ON THIS TEAM
- Senior Data Platform Engineer posted Nov 17
- Senior AI Engineer posted Nov 9
- Senior ML Engineer, Agentic AI posted Nov 5
- Senior Backend Engineer, Core Services posted Sep 15
- Senior Backend Engineer, AI Evaluations posted May 13
- Senior Backend Engineer, Forward Deployed posted May 1
- Senior Full Stack Engineer, Core Services posted May 1
From the team and dates on Ellipsis Health's own job board.
See Backend Engineer salaries in San Francisco →
Similar jobs
- Senior AI Infrastructure Engineer at Hellyeah AI
Shares 11 skills: WebSockets, OpenTelemetry, Redis… · Senior level · same city - Full Stack Developer (Backend & DevOps) at 2070Health
Shares 13 skills: WebRTC, WebSockets, OpenTelemetry… - Backend AI Engineer at Nexxa.AI
Shares 10 skills: WebSockets, gRPC, Kafka… · same country - Software Engineer, Agentic Runtime at Glean
Shares 11 skills: WebSockets, OpenTelemetry, gRPC… · same country - Platform Engineer at Vectara
Shares 11 skills: OpenTelemetry, gRPC, Kafka… · same country - Senior AI Engineer at Ellipsis Health
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.