Architecture
XSI is a single-process HTTP service written in Go. One port serves both model calls (the data plane) and account, key, and configuration management (the control plane). State lives in PostgreSQL and Redis, and two static consoles talk to the service through its HTTP API.
Components
ββββββββββββββββββββββββββββββββ
Apps/SDKs βββΆβ ββββΆ OpenAI / Azure
Codex βββΆβ XSI API (Go, :3443) ββββΆ Anthropic
Claude Code ββΆβ ββ Data plane: /v1/... ββββΆ Gemini / Vertex
Consoles βββΆβ ββ Control: /x-*, /dashboardββββΆ DeepSeek / xAI / ...
βββββββββ¬ββββββββββββββββ¬βββββββ
β β
PostgreSQL Redis βββΆ Mail service (SMTP)
Accounts, keys, ledger Config, cache, usage,
counters, messages, mail queue| Component | Image | Responsibility |
|---|---|---|
| API | proxyxai/xsi | Authentication, admission, routing, forwarding, metering, billing, and all management endpoints |
| Consoles | proxyxai/xsi-pages | Two static sites, configuration management (Admin) and account management (Manage), each in Chinese and English |
| Mail service | proxyxai/mail | Consumes the Redis mail queue and delivers over SMTP; the API never connects to SMTP directly |
| PostgreSQL | postgres:18 | Accounts, upstream keys, top-up ledger, notification outbox |
| Redis | redis:alpine | Tenant configuration, account cache, usage statistics, rate limit counters, sticky bindings, cross-instance messaging; DB 1 holds the mail queue |
Request lifecycle
Request ββΆ Size check ββΆ Auth ββΆ Account admission ββΆ Model resolution ββΆ Model admission ββΆ Level ββΆ Key selection ββΆ Forward
β
Settlement ββ Usage extraction (side-channel parse of response / SSE) ββ Retry (next key in group, may cross groups) ββ- Size check: if the declared
Content-Lengthexceeds the limit, the request is rejected with413before authentication. - Authentication: the key is read from
Authorization,x-api-key, orx-goog-api-key. An in-memory hit passes immediately; otherwise Redis and then PostgreSQL are queried. Failed authentication attempts are counted per IP and trigger a temporary ban. - Account admission: IP allowlist, account status and freeze, daily/monthly quota, balance, account-level rate limits, and resource allowlist.
- Model resolution: a single scan of the top-level JSON fields (
model,stream,prompt_cache_key,speed,service_tier) applies the mapping rules to produce the upstream model name, then rewrites the request body or URL in place. - Model admission: model allowlist and per-model limits.
- Level and key selection:
LEVEL_MAPPERdetermines the group, and a key is chosen from that group's key pool by sticky session or round robin. - Forwarding: caller credentials and forwarding headers are stripped, and the path and auth headers are rewritten for the upstream host.
- Retry: when the upstream returns a retryable error, the request is retried with another key before any byte is written to the client.
- Metering and settlement: non-streaming responses are parsed for usage after being read in full; SSE streams are parsed chunk by chunk while being forwarded, without buffering the whole stream. Settlement runs once at the end of the request, and a client that disconnects mid-stream is still billed for actual usage.
State and caching
| Layer | Location | Notes |
|---|---|---|
| Hot account state | Process memory β Redis β PostgreSQL | Reads fall back in this order; balance and usage accumulate in memory and are flushed to Redis and the database in batches |
| Tenant configuration | In-process snapshot | Loaded from Redis, copy-on-write; synced across instances via pub/sub plus a version check every 3 minutes |
| Key pool | In process, one round-robin ring per tenant per Level | Lazily loaded and rebuilt when keys change |
| Runtime mappings | Exact-match cache per account and tenant | Wildcard resolutions are stored as exact entries to avoid repeated matching |
| Prices | In process, one effective price table and model match index per tenant | pricing.json is loaded at startup; tenant overrides are merged on demand |
| Usage statistics | In process + Redis | One record per account per day and per month, broken down by model |
| Sticky bindings | Memory for a single instance, Redis for multiple instances | Retained for 24 hours by default |
PostgreSQL holds only authoritative data such as accounts, keys, and the ledger; fast-changing usage statistics are kept in Redis.
Encryption model
- Each tenant has a randomly generated tenant key. Upstream provider keys and Vertex private keys are encrypted with the tenant key before being stored, using XChaCha20-Poly1305.
- The tenant key itself is encrypted with the account's own API key and stored in the account record. The server keeps no plaintext material that can directly unlock a tenant key.
- As a result, a tenant's key pool can only be decrypted and loaded when an account in that tenant makes a request; if the database alone is exfiltrated, upstream keys cannot be decrypted directly.
- When an account key is rotated, the tenant key is re-encrypted with the new key and the old key is invalidated immediately.
- Core dumps are disabled at process startup.
Single instance and multiple instances
SINGLE_MODE defaults to true. In single-instance mode, rate limit counters, sticky bindings, and change notifications all stay in process for the lowest latency. Set it to false to scale horizontally:
| Aspect | Single instance (default) | Multiple instances (SINGLE_MODE=false) |
|---|---|---|
| Rate limit counters | In process | Redis |
| Key and account change notifications | In-process channel | Redis pub/sub |
| Sticky bindings | Memory | Redis |
| Usage writes to Redis | On maintenance and flush | On every update |
| Key auto-disable/delete broadcast | β | Redis channel |
In multi-instance deployments, group switchover (SWITCH_OVER), model failover, and failure counts are still evaluated independently per instance, and auto-sleep applies only to the instance that triggered it.
Performance design
- Connection reuse: all HTTP upstream requests share one
http.Transport(WebSocket uses a separate dialer; up to 256 idle and 2048 concurrent connections per host, with HTTP/2 and TLS session caching enabled), with 64 KiB read and write buffers reused throughsync.Pool. - Zero-copy streaming: SSE responses are copied straight from the upstream to the client while the usage tracker parses chunks on the side, without buffering the whole response (XSI requests uncompressed responses from upstreams; if an upstream still returns a compressed stream, metering buffers that response).
- Minimal parsing: the request body is scanned once for top-level fields, and model rewrites use in-place JSON edits rather than full deserialization.
- Asynchronous settlement: usage calculation, charging, and state persistence run in background workers sharded by account, off the request path.
- Hot-path caching: authentication, configuration, mappings, key pools, and prices are all served from process memory; configuration and key pool loads use singleflight to coalesce concurrent fetches.
- No timeout truncation: the HTTP server sets only a read-header timeout (15 seconds) and an idle timeout (2 minutes), so long streaming responses are never cut off by the server; the wait for upstream response headers defaults to a maximum of 1800 seconds.
Persistence and shutdown
SIGHUP: runs a checkpoint (flushes the usage queue, persists account state, runs RedisSAVE) without interrupting service.SIGTERM/SIGINT: stops accepting new requests and waits for in-flight requests (up to 20 seconds), closes WebSocket sessions and waits for their settlement, then runs a final checkpoint (up to 30 seconds). The deployment template'sstop_grace_period: 60scovers both phases.- If the process is killed forcibly (
SIGKILL, OOM), in-flight requests are not settled.