Member of Technical Staff (Index Engineer, Search Crawler)
- Level
- Staff
Not stated in the posting: salary, work type, visa.
Open. First seen 17 November 2026.
About the role
AI answers are only as trustworthy as the information behind them. Perplexity grounds its answers in fresh, real-world sources, and the index is what makes that possible. We're looking for an engineer to work on the indexing and data processing layer of Perplexity's Search Crawler.
You'll build systems that ingest, process, and organize large volumes of crawled content into efficient, queryable structures, working primarily on offline and nearline pipelines that keep data fresh, complete, and accessible to downstream retrieval. Throughput, scalability, and correctness are critical across datasets that change continuously.
Responsibilities
- Design, implement, and operate large-scale indexing systems and data pipelines at the core of Perplexity's search infrastructure.
- Develop and optimize indexing strategies that balance performance, freshness, and resource efficiency.
- Work on storage formats, compaction, and update mechanisms that keep data accessible and current.
- Keep pipelines reliable and predictable under high-throughput conditions.
- Build well-tested components with clear responsibilities and interface contracts that stay flexible as the system evolves.
- Define and implement observability across offline and near line pipelines, including structured logs, metrics, and data quality signals.
- Monitor throughput, resource usage, and cost, and drive optimisations as business needs change.
- Partner with retrieval, ranking, and ML teams to make sure indexing outputs meet their requirements.
- Enable safe experimentation on indexing strategies and processing logic through controlled rollouts and clearly defined quality signals.
Qualifications
- 5+ years of experience building production backend or data infrastructure systems.
- Strong programming experience in Go, C++, or Rust.
- Experience with high-throughput data processing systems and very large datasets.
- Experience building or operating databases, storage systems, data planes, or indexing pipelines.
- Strong understanding of distributed systems, fault tolerance, consistency, and scalability.
- Experience running production systems and handling operational incidents.
- A systems-thinking mindset and the ability to reason about end-to-end data flows.
What this role asks for
Particular to this role
C++
Rare in Perplexity's other 95 open postings.
Compared with Perplexity's other open jobs.
The team: Search HIRES STEADILY
This role is one of 17 open in Search. 4 were posted in the last 30 days and the oldest in March: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
Rust (12 of 17), C++ (9 of 17), Python (9 of 17), Golang (8 of 17), Kubernetes (7 of 17), LLM (7 of 17), AWS (6 of 17)
OTHER ROLES ON THIS TEAM
- Member of Technical Staff (Site Reliability Engineer, Crawler) posted Nov 13
- Member of Technical Staff (Web Access Engineer, Crawler) posted Nov 13
- Member of Technical Staff (Software Engineer, Fraud Prevention) posted Oct 23
- Member of Technical Staff (Search Crawler Analyst) posted Oct 8
- Member of Technical Staff (Machine Learning Engineer, Search Agent RL) posted Oct 6
- Member of Technical Staff (Search Core DevOps Engineer) posted Sep 25
- Member of Technical Staff (Search Core, Indexing & Data Pipeline Stream) posted Sep 18
- Member of Technical Staff (Search Quality Analyst) posted Sep 2
- Member of Technical Staff (Machine Learning Engineer, Ranking Quality - Search) posted Aug 12
- Internship - Search Backend Infra Engineer posted Jul 1
- Internship - Search Machine Learning Engineer posted Apr 8
- Internship - Search Machine Learning Engineer posted Jan 13
- Member of Technical Staff (Backend Software Engineer, Search Crawler) posted Jul 28
- Member of Technical Staff (Search Engine Developer, Search Core) posted Mar 31
- Member of Technical Staff (Machine Learning Engineer, Search) posted Mar 25
- Member of Technical Staff (Backend/Infrastructure Engineer, Search) posted Mar 7
From the team and dates on Perplexity's own job board.
Similar jobs
- Staff+ Software Engineer, Deployment at Anthropic
Shares 3 skills: Rust, Golang, C++ · Staff level · same city · ✓ Visa sponsorship - Software Engineer - EMS (London based) at Elwood Technologies
Shares 3 skills: Rust, Golang, C++ · same city - Senior/Staff Software Engineer, Search & Retrieval Infrastructure at Pinecone
Shares 3 skills: Rust, Golang, C++ · Staff level - Staff Forward Deployed Engineer at Swarm Aero
Shares 3 skills: Rust, Golang, C++ · Staff level - Technical Lead, Data Engineering at Distributed Spectrum
Shares 3 skills: Rust, Golang, C++ - Member of Technical Staff (Cybersecurity Solutions Architect) at Perplexity
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.