NVIDIA Releases Switchyard, a Rust Proxy That Translates Between OpenAI and Anthropic APIs
NVIDIA published Switchyard under Apache 2.0: a Rust proxy and library that routes LLM traffic across providers and converts between OpenAI and Anthropic wire formats.
NVIDIA has released Switchyard, a Rust proxy and library for LLM traffic, under the Apache 2.0 license. It routes requests across providers, translates between OpenAI and Anthropic formats, records operational metrics and exposes typed, composable routing algorithms. Documentation lives at docs.nvidia.com/nemo/switchyard.
The problem it addresses is familiar to teams running coding agents: Claude Code speaks the Anthropic Messages API, Codex CLI speaks OpenAI, and the model a team actually wants to serve sits behind vLLM, NVIDIA NIM or Ollama. Rewriting the agent is not an option, so the translation layer has to live elsewhere.
How it works
Clients keep their native API. Switchyard decodes the inbound request into provider-neutral Rust types, runs a routing algorithm to pick a backend, re-encodes the request in that backend's own wire format, calls it, and translates the response, streaming events included, back into the shape the client expects.
The server accepts three inbound formats: OpenAI Chat Completions, OpenAI Responses and Anthropic Messages. Any of the three can address any route. A route is one client-visible model ID plus the algorithm behind it. Strong, weak, capable and efficient are roles inside a route rather than fixed properties of a model, so the same upstream model can serve different roles in different routes.
There are three installation paths: a launcher via uv tool install that runs switchyard launch claude, codex or openclaw; the standalone proxy via cargo install --locked switchyard-server, with config validation through --dry-run; and the switchyard-libsy library, which embeds the routing algorithms without owning an HTTP stack and hands every model call back to the caller.
Metrics and configuration
GET /metrics returns Prometheus text from the process-wide OpenTelemetry provider, covering requests, errors, latency and token families. switchyard_routing_overhead_ms reports the algorithm's run time minus the call that served the request, with buckets starting at 0.1 ms. A TOML deployment has three layers: llm_clients, targets and routes. Secrets stay out of the file because api_key_env only names an environment variable, and max_retries defaults to 2 for transport failures, timeouts, HTTP 408/429 and 5xx responses.
NVIDIA labels Switchyard pre-alpha and experimental, warns against production use and expects the API and algorithms to change significantly before v1.0. More detail in the MarkTechPost report.
