cerber · Trust-First AI Gateway — Architecture Dossier

The core idea

A team or a household accumulates many AI accounts — Claude Code subscriptions, a SuperGrok login, raw API keys, a local Ollama. cerber turns that pile into one endpoint with one key, routed and load-balanced — without ever handing those credentials to anyone else's server.

cerber is a from-scratch reimplementation that merges three upstream projects into a single binary: a multi-provider proxy (translation + OAuth logins + multi-account rotation), a per-account usage/quota tracker, and a web management UI. The upstreams are read as spec, not source — the credential store is the most security-critical package and is wholly ours, reviewed line by line.

CONCERN 01 — routing

One key in, the right provider out

A request names a model; cerber routes by name (prefix, alias, or live discovery) to the provider that serves it, translating dialects on the way.

CONCERN 02 — resilience

Many accounts, one healthy pool

Credentials rotate round-robin or fill-first; a failing account is sidelined with exponential backoff; OAuth tokens refresh themselves. A request fails over to the next account — or the next model.

CONCERN 03 — governance

Per-key budgets & limits

Each managed key can carry a rolling cost budget and request/token rate limits. Over budget → 402; over rate → 429.

CONCERN 04 — the line

Trust, not telemetry

The only outbound calls cerber makes are to the provider a request is routed to. No phone-home, no analytics, no remote code or asset fetch — ever.

Why this exists

The whole point is that we do not trust upstream gateways with our OAuth tokens and API keys. So cerber is written from scratch, vendors no upstream packages, and treats the credential store as the trust boundary. If you can read the code, you can verify your keys never go anywhere they shouldn't.

System map

Built in vertical slices — one provider end-to-end before the next. Every request descends the same stack; the credential pool and the upstream providers are the only places secrets and network egress live.

Fig.1 — layers & the single egress seam

Provider HTTP clients sit behind an interface, so unit tests never touch the network and the whole stack is exercised against generated mocks. The module holds a >85% coverage gate; new packages ship with their tests in the same change.

One request, end to end

Every model call walks the same path. Nothing is written to the client until a provider answers — which is what makes transparent fallback possible.

Fig.2 — the request pipeline

Authorizeaccess
Constant-time match of the presented key. A managed key then passes its governance gate — if its cost window is spent → 402, if its rate window is full → 429. The caller identity, IP and User-Agent are stashed on the context.
Resolve the modelcatalog
An alias (e.g. opus) is rewritten to its canonical name; the body is only touched when an alias actually applies.
Routerouter
The model picks a provider — configured prefix, then live-discovered names, then built-in prefixes (gpt*→openai, claude*→anthropic, …). An unknown model is rejected, never silently sent somewhere.
Translatetranslator
If dialects differ (OpenAI client → Claude), the request is converted; native /v1/messages is byte-faithful passthrough.
Dispatchcredential pool
Pick a credential (rotate or fill-first); refresh an OAuth token if it's near expiry; send. On a credential failure (401/403/429) sideline it and try the next; on a 401 OAuth, force-refresh and retry once.
Relayserver
Stream the SSE through (translating chunks where needed) or relay the JSON, faithfully forwarding the provider's rate-limit headers.
Recordusage
One point logs the outcome: tokens (parsed from the stream too), computed cost, and a recent-request entry — who, from where, on which model.

The credential pool

The most security-critical package, and the reason cerber exists. A thread-safe store hands out credentials and quietly keeps a broken account from poisoning the rotation.

rotation: round-robin (default) cycles every credential; fill-first drains one before moving on.
cooldown: a 401/403/429 or transport error sidelines the credential with exponential backoff — 60s → 120s → … capped at 30m — so a persistently broken (e.g. unpaid → 403) account is parked, not retried every minute.
recovery: a successful use clears the streak and cooldown immediately; ErrNoneAvailable only when every credential is cooling.
OAuth refresh: proactive near expiry and reactive on a 401, behind a singleflight so concurrent requests spend the rotating refresh token exactly once (Anthropic rotates it every refresh — clobbering it kills the account).
pinning: header X-Cerber-Cred: oauth | key | <name> forces which credential serves the request; omit to rotate.

Secrets never stringify

Credentials are never present in String(), logs, or any API response — they're readable only through explicit accessors, and the redacted views (key last4, account name) are all the dashboard ever sees.

Routing, aliases & fallback

Set model and cerber finds the provider. Three independent mechanisms, in order:

Config prefixes — providers.routing maps a model-name prefix to a provider.
Live discovery — providers are probed; whoever advertises the exact model (e.g. an Ollama tag) wins.
Built-in prefixes — gpt*/o1*/o3*/o4*/chatgpt*→openai, gemini*→gemini, grok*→xAI, claude*→anthropic; anything else → 400.

Aliases

A stable client-facing name maps to whatever dated model is current — opus → claude-opus-4-… — resolved before routing and before the upstream call (exact match first, else the longest prefix). Stable names don't break when a provider rolls a new snapshot.

Cross-provider fallback

On /v1/chat/completions, a model that fails with a retryable error — provider 5xx, or every account rate-limited/unavailable — is transparently retried against the next model in a chain, before any bytes are sent. A 4xx client error is returned as-is; a started stream is never re-attempted.

Fig.3 — the fallback decision (per target, pre-commit)

Declared two ways

Server-side chains in config (providers.fallbacks, matched by model prefix), or per-request via header X-Cerber-Fallback: gpt-4o,gemini-2.0-flash. Each target is a model routed like any other, so a fallback can cross providers. When every target fails, the last error is surfaced.

Two dialects, one gateway

Point an OpenAI SDK or an Anthropic SDK at cerber and reach every model. Translation happens in a pure, network-free layer.

You speak	Model lives on	What cerber does
OpenAI `/v1/chat/completions`	OpenAI / Grok / Ollama	passthrough with a rotated key
OpenAI `/v1/chat/completions`	Claude	translate request → Anthropic, translate (streamed) response back to OpenAI
OpenAI `/v1/chat/completions`	Gemini	translate OpenAI ⇄ Gemini both ways
Anthropic `/v1/messages`	Claude	byte-faithful passthrough (streaming, headers, count_tokens)

Streamed token usage is parsed out of the SSE on both the native and the OpenAI-translated Claude paths, so cost and per-key budgets count streaming traffic, not just buffered replies.

The surface

Endpoint	What it does
`POST /v1/chat/completions`	OpenAI-compatible chat for every provider incl. Claude — streaming + fallback
`POST /v1/messages`	native Anthropic Messages (transparent passthrough)
`POST /v1/messages/count_tokens`	Anthropic token counting via the pool
`POST /v1/embeddings · /v1/completions · /v1/responses`	OpenAI passthrough to the routed provider
`POST /v1/images/generations`	image generation (grok-imagine-* / gpt-image-*)
`GET /v1/models`	discovered model ids per provider
`GET /llm.md · /docs`	live, self-describing usage guide (agents) & HTML reference
`GET /admin/*`	stats · recent requests · accounts · client-key & limit management
`GET /metrics`	Prometheus: requests, errors, tokens, per-model cost

Every running instance serves its own live reference at /docs and an agent-oriented guide at /llm.md — reflecting that instance's providers, models, aliases and fallback chains.

Governance

Managed (dashboard-minted) keys can carry a rolling cost budget and rate limits. Static config keys and loopback callers are the operator's own and bypass governance.

budget

Cost cap

max_cost_usd over a period
minute · hour · day · week · month
over → 402

rate

Throughput cap

max_requests / max_tokens per window
reserved on admit
over → 429

lifecycle

Self-resetting

rolling windows, persisted with the key
survives restart
default_key_limits seeds new keys

Cost is computed from the configured per-model pricing; counters are flushed lazily by the periodic saver, not on every request. Limits are edited at runtime via POST /admin/keys/{name}/limits.

Observability & cost

Everything is measured in-process — no external collector, no data leaves the box.

aggregates · persisted

Usage & cost

Per-credential and per-model requests / errors / tokens, hourly time-series (~30-day retention), and a cumulative total cost from configured pricing — survives restarts.

ring · in-memory

Recent-request log

Who called what: time, client, IP (honouring X-Forwarded-For), User-Agent, provider, model, account, tokens, cost. Filter by model/provider/account, paginate. Holds IPs → kept in memory only, never on disk.

passive

Per-account quota

5h / 7d utilization read straight from Anthropic's rate-limit headers, per pooled account.

scrape · embed

Prometheus & dashboard

/metrics exposes counts + cerber_cost_usd_total; the binary-embedded dashboard shows cards, an hourly chart, per-model cost, accounts, and the recent-request table.

Trust principles

These are non-negotiable. They are the reason to run cerber instead of someone else's gateway.

01 · From scratch

No upstream code is copied or vendored. The upstreams are read to understand protocol and behaviour, then reimplemented. The credential store is reviewed line by line.

02 · No phone-home

cerber makes outbound calls to exactly one class of host: the legitimate provider APIs a request is being routed to. No telemetry, no analytics, no version pings, no auto-download of code or assets. A new outbound host is a red flag in review.

03 · Credentials never leave the box

…except as the documented auth header to the provider that owns them. Nowhere else, in no other form.

04 · Pinned, opt-in, never silent

No feature fetches and executes or persists remote content automatically. The upstream auto-updater and update-checks were deliberately dropped.

Deployment

Single static binary — CGO_ENABLED=0, shipped in a distroless image. Config is YAML + .env + ${ENV}; config is the single source of truth for which hosts cerber may talk to.
Zero-downtime blue-green — two identical containers behind Caddy with active health checks; deploy starts the idle colour and stops the active one, Caddy follows the /healthz probe — no reload.
Edge allow-list — Caddy fronts the public vhost: only the CDN edge + LAN/WG reach the origin; LAN gets a key injected for keyless local use, the public side presents its own.
TLS impersonation (opt-in) — a generated CA lets a first-party Claude Code's console/bootstrap calls run on cerber's pooled credential, keeping cerber the sole token owner.

Trade-offs of the design

What it buys

Verifiable trust — one small, from-scratch codebase; your keys provably go nowhere but the provider.
Account pooling — many subscriptions/keys behave as one resilient endpoint with automatic failover.
Dialect freedom — any OpenAI or Anthropic SDK reaches any model.
Attribution & cost — who-called-what and the running $ are visible without a third party.

What it costs

You run it — no managed SaaS; you own the box, the deploy, and the keys.
Narrower surface — focused on pooling personal/subscription accounts, not an enterprise feature catalogue.
Pricing upkeep — cost depends on a per-model pricing table you maintain.
Translation gaps — OpenAI→Anthropic tool-calling translation is still partial.

Glossary

credential: one upstream account: an API key or an OAuth/subscription token, pooled per provider.
client key: a key a caller presents to cerber. Static (config) or managed (dashboard-minted, cer_…, can carry limits).
rotation: handing successive requests to different credentials (round-robin / fill-first).
cooldown: the backoff window a failing credential is sidelined for.
alias: a stable model name mapped to a provider's canonical (often dated) model id.
fallback chain: ordered models tried when the primary fails retryably (OpenAI endpoint).
governance: per-key budgets + rate limits enforced before a request runs.
recent-request log: in-memory ring of who/where/which-model/tokens/cost for the last N calls.