Respan's Rust LLM Gateway Claims 0.07 ms Overhead, but the Launch Post Calls It Pre-Alpha

Respan, a Y Combinator W24 company, released an open-source LLM gateway written in Rust that its README says adds 0.07 ms of median latency, against 4.34 ms for LiteLLM running one worker. The same company's Launch HN post calls the project "pre-alpha."

The repository, respanai/respan-gateway, was created on September 25, 2026. When we read it on October 23 it had 141 stars, 16 forks, 13 open issues and 5 open pull requests on GitHub, with its last push on October 22. GitHub reports the language as Rust.

What the gateway does

It is a single binary that accepts OpenAI Chat Completions, OpenAI Responses, Anthropic Messages and Gemini API clients and routes them to 16 providers plus any OpenAI-compatible endpoint, per the README. It translates between formats, so an app written for OpenAI Realtime can reach Gemini Live, though the Launch HN post warns that not every feature maps across.

The README says no config file is needed: "every vendor whose key is set becomes a provider." Install is pip install respan-gateway, cargo install respan-gateway or a release binary, and a web UI runs at localhost:8080/ui/.

The launch post was written by a user named Andy (Andydy25), who did not name other founders. It says the team built on LiteLLM from 2024, that its wrapper grew past 100,000 lines, and that a January LiteLLM upgrade caused connection errors and a rollback. That history is why the gateway is built around a small core that dispatches to plugins.

The benchmark table is the project's own

The README's comparison against Bifrost, Portkey and LiteLLM comes with no methodology on the page. The Launch HN post gives no performance numbers at all. Treat every figure below as vendor-supplied; no independent party is named as having run them.

Gateway	Added latency (p50)	Throughput	Memory
Respan	0.07 ms	56,734/s	31 MB
Bifrost	0.16 ms	18,182/s	488 MB
Portkey	0.17 ms	4,187/s	350 MB
LiteLLM (1 worker)	4.34 ms	268/s	418 MB

1. Respan57K req/s
2. Bifrost18K req/s
3. Portkey4187 req/s
4. LiteLLM (1 worker)268 req/s

Source: respanai/respan-gateway README, accessed 2026-10-23

The README also reports a p99 of 105 ms with 5% slow upstream calls, against roughly 2,000 ms for the other three. A gateway adding 0.07 ms matters little when a model call takes seconds, so the tail-latency row is the more interesting claim, and also the one with the least detail behind it.

Where the README and the launch post disagree

Three details do not line up. The README says the project is dual-licensed MIT or Apache-2.0, at the user's option, while GitHub's metadata lists Apache-2.0 alone. The launch post says MIT or Apache-2.0.

Second, the README lists a Docker image at ghcr.io/respanai/respan-gateway. Hacker News commenters wrote that there was no prebuilt Docker image and that users had to build from source or take release binaries. Either the image arrived after the post or the commenters missed it; we did not pull it to check.

Third, the maturity label. The README presents a finished product with a SECURITY.md file. The launch post says pre-alpha.

The objection that matters

Plugins are compiled into the binary as Rust crates, not loaded at runtime or run as WebAssembly, the post says. Commenters objected that third-party crates compiled into a process that holds every provider key are a supply-chain exposure few users will audit. One flagged the custom MCP OAuth code as a concern.

The README offers one mitigation: client keys are optional, and without them the gateway answers only on loopback and refuses to start on a public address.

The post also says the team used Claude to generate code and reviewed it. Its hosted platform, with observability, evals and unified billing, is how the company makes money; the gateway is the free part. Other open-source agent tooling with weak defaults is covered in our Omnara report, and a Go-versus-docs mismatch in the OpenAI CLI. The next thing to watch is whether a tagged release appears: the README's release section was empty when we read it.