ai

One OpenAI / Anthropic / Gemini compatible endpoint for every tool and person on the instance, in front of subscription accounts (through CLIProxyAPI) and official API keys, with teams, keys, quotas, routing and usage analytics. Host: ai.<d> (API at ai.<d>/v1, /v1beta, Anthropic /v1/messages). Binary: aid. Stage 8. Built on CLIProxyAPI (ADR 0012).

Concepts

Provider account: a subscription account signed in through its CLI's OAuth (Claude Code, Codex, Gemini CLI, Antigravity, Grok, …, whatever CLIProxyAPI supports) or an API key (Anthropic, OpenAI, Google, OpenRouter, any OpenAI-compatible URL). Owned by the instance or a team; can be granted to other teams with a quota (ADR 0005). State: healthy, rate-limited until, expired, failing.
Pool: accounts of one upstream; requests rotate across them by remaining window (5-hour and weekly limits where the provider has them, tracked like cca does), with cool-down on rate limits.
Model: the public name clients use (claude-sonnet, gpt, gemini-pro, or exact upstream ids), mapped to one or more upstreams with priorities and fallbacks.
Key: sk-xvii-… per user, team or project (and per app in apps, delivered through env), with allowed models, rate limits and a budget.
Quota: per team, key or user: requests, input / output / cached tokens and cost per window (hour, day, month), hard or soft (soft notifies through notify). Cost from a price table per model kept in the repo and editable.
Usage: every request logged with key, team, project, model, upstream account, tokens, cost, latency, status; bodies not stored unless a team enables it for debugging (with retention).

Endpoints

Compatible APIs: OpenAI (/v1/chat/completions, /v1/responses, /v1/embeddings, /v1/models), Anthropic (/v1/messages, count tokens), Gemini (/v1beta/models/*). Streaming everywhere.
Quota endpoints: GET /v1/quota (remaining for the calling key: requests, tokens, budget, reset times) and per upstream account for admins, in a shape tools like opencode-quota can read.
Admin API for accounts, pools, models, keys, quotas, usage.

Panel

Overview (requests, tokens, cost by model, team, key), accounts and pools with live limits, models and routes, keys, quotas, usage explorer, a playground.

Updates

A scheduled CI job follows CLIProxyAPI releases, bumps the module, runs the compatibility tests (recorded upstream responses), and opens a merge request; home offers the new ai release like any other.

CLI and MCP

xvii ai keys create, xvii ai usage, xvii ai accounts add claude (runs the provider's OAuth through the browser), xvii ai quota. MCP tools for usage and quota questions.

Risk

Proxying consumer subscriptions can break the providers' terms; the panel says so when such an account is added (ADR 0012).

As built (stage 8)

aid embeds CLIProxyAPI v8 through its SDK (ADR 0012), on loopback (PROXY_PORT, 8813) with its own config, credentials and management key in DATA_DIR. Outside, ai.<d>/v1 and /v1beta go through aid's gate; /api/v1 and the panel are aid's.
Keys sk-xvii-… (stored as sha256, shown once): a team's, or a person's own in a team; allowed models (ids or * patterns); requests a minute and a day, tokens and dollars a month. The gate reads the key where the SDK puts it (Bearer, x-api-key, x-goog-api-key, ?key=), refuses in the caller's API's error shape, and passes only the key's id (X-Xvii-Key) to CLIProxyAPI, whose "xvii" access provider makes it the request's principal. GET /v1/quota answers what the calling key has left.
Usage: CLIProxyAPI's usage plugin reports every upstream call (key, model, provider, account, tokens from its accounting breakdown, latency, stream, failure); aid prices it from a built-in table of list prices (internal/meter/prices.go, estimates for subscriptions too) with the instance's overrides, and writes it in batches. The team limit ai.tokens (a month) and ai.keys apply.
Accounts: superusers sign in subscription accounts (claude, codex, antigravity, kimi, xai, devin, meta) through CLIProxyAPI's OAuth — the panel or xvii ai accounts add opens the provider's page and takes the address the browser ends on — import credential files (POST /api/v1/accounts/import, CLIProxyAPI's auths/*.json) and export one (GET /api/v1/accounts/export/{name}, journaled as provider.export), and keep a note and excluded models on each. The panel never shows keys, only their last four characters, and warns about subscription terms.
Sources (/api/v1/upstreams, superusers): a source's kind decides its CLIProxyAPI v8 groups (api-keys.<group>): anthropic → claude (with a base URL for Anthropic-compatible APIs), openai → openai-compatibility, codex, gemini, vertex, and cliproxy → claude for claude-*, gemini for gemini-*, openai-compatibility for the rest, all under the source's name. The sources table keeps what the config doesn't: kind, model sync (sync, sync_hours), a remote's management key (sealed with kit/seal, DATA_DIR/seal.key), excluded models (openai-compatibility has no excluded-models), the last test and sync. PATCH edits in place; a write that CLIProxyAPI refuses restores the groups already written. POST /upstreams/probe lists a source's models before saving, POST /upstreams/{name}/test sends one tiny request straight to it, POST /upstreams/{name}/sync reads its models now; every 10 minutes aid syncs the sources that follow their list and emits ai.upstream.models (and ai.upstream.sync on a failure). Disabling a group without a disabled field excludes *. With a management key, a remote CLIProxyAPI's accounts (source:account) and quotas are read through its /v0/management and shown in the pool; they are never parked by aid. A compact, centered provider column uses locally bundled brand icons with accessible names instead of a wide text heading. Model lists and the playground picker show the model family's icon, falling back to its known owner or a neutral symbol. The family takes precedence over the routing provider; OpenAI is black in light themes and white in dark themes. Icons make no external requests and do not affect routing.
Provider quotas: the Accounts table shows each subscription's remaining windows and reset times, separately from aid's usage metering. GET /api/v1/accounts/quota/{name} (superusers) uses CLIProxyAPI's quota provider when available; otherwise fixed read-only probes for Codex, Claude and Antigravity run through its management API with the account's token/proxy. Neither tokens nor raw provider responses leave aid. Checks are cached five minutes; ?refresh=1 allows a check at most once a minute. Missing data is unknown, never 100% free; a failed refresh keeps the last snapshot with its timestamp and an error. Providers without a quota API are explicitly unsupported. The panel groups quota windows vertically with separate reset details; provider failures use bounded EN/RU messages, never raw JSON. Exhaustion forecasts reject duration overflow and stale dates. When a window or the provider reports an exhausted quota, the quota cell keeps balances and reset details but omits exhaustion notices and forecasts; the account's Status column carries the explanation. Empty quota cells show a dash without a standalone refresh button; refresh remains available alongside quota details or a non-exhaustion check error.
Pool (v1.1): aid reads every enabled account's quota every five minutes into quota_snapshots (30 days): history (GET /api/v1/accounts/history/{name}), and GET /api/v1/accounts/pool with each window's pace and when it runs out if before it resets, the last day's calls per account, and what the last 30 days cost at list API prices. An account whose account-wide window (Claude 5h/7d, Codex primary/ secondary; not Antigravity's per-model ones) has under 2% left is disabled until that window resets (parked_accounts), unless no other account of its provider is in rotation; a person's toggle unparks it. Superusers hear through notify (ai.account.parked, .resumed, .exhausted, ai.quota.low under 10%, ai.account.failing when half of 10+ calls in 15 minutes failed).
Aliases (v1.2): the instance's own model names (aliases, superusers) for an ordered list of models; the gate tries them in order and moves on while one answers 429, 404 or 5xx (the last answer passes through, X-Xvii-Model names the model). /v1/models lists them. A key may call an alias if it may call its name or any of its models, and then only those models.
Analytics (v1.3): GET /api/v1/usage/stats — latency and TTFT p50/p95, failures by status, cache hits, output speed; by hour or day, model, provider, key and (superusers) account. A key's team hears when its month passes 80% or all of its budget or tokens (ai.key.budget, ai.key.tokens).
Playground (v1.4): /api/v1/playground/{key}/v1/chat/completions|messages|models calls the gate with a key the caller may change (a developer's team key or their own), so it counts and is limited on that key; the panel never holds a key.
Chat (v1.6): the playground is a chat built from @xvii items (conversation-list, chat-message, prompt-input, markdown). Conversations are a person's own (conversations: title, model, pinned, data opaque to the server, up to 32 MB with pictures), readable by a team they're shared with (team_id), never writable by others. Answers keep variants (regenerate, answer with another model); image models go to /v1/images/generations, or /v1/images/edits with the last picture or an attached one (the gate also reads the model of multipart forms). After the first answer a cheap model (haiku, mini, flash…) names the conversation. prompts are a person's system prompts and /command templates.
Service model (v1.7): settings keeps the instance's service model (names chats; "" picks a cheap one by name) and whether chats are named. A model's kind (chat or image, /v1/images) comes from its name as CLIProxyAPI routes them (gpt-image-, dall-e, imagen, grok-imagine-image), or by hand (model_kinds); /api/v1/models and aliases carry it.
Compression (v1.8): Headroom (chopratejas/headroom, Apache-2.0) runs in one Docker container aid manages through the Engine API (xvii-ai-headroom, pinned image, host network on loopback HEADROOM_PORT 8814, a volume for its state, 1 GB / 2 GB with ML; the manifest adds docker to ai's groups). Trust note (audit 2026-10): that socket is root-equivalent on the host — aid is only as isolated as Docker is; a break-out is aid's, not Headroom's (it has no socket). Where that's not acceptable, run a socket proxy (tecnativa/docker-socket-proxy) allowing only HEAD/GET/POST on containers and images, and point DOCKER_HOST at it. Its beacon, update checks and (without ML) downloads are off. The gate sends the messages of /v1/messages and /v1/chat/completions to /v1/compress — a session per key, model and first message keeps earlier turns byte-identical for the prompt cache — and forwards the result; slow (timeout_ms) or failing, the call goes as it came. Instance: off (container stopped), audit (count only, no session) or on; then a model rule (first match), the key, the team (team_settings, its admins), and the call's X-Xvii-Compress header, which can't turn on what the instance has off. compressions keeps each look (tokens before and after, added latency, transforms); /api/v1/compression/stats prices savings at the model's input price.
Logs (v1.5): a key's log_days (0, or up to 30) keeps its calls' bodies (request_logs, 256 KB each, streams as sent) for whoever sees the key (GET /api/v1/keys/{id}/logs[/{log}]); turning logs off or deleting the key deletes them; expired ones go every five minutes.
Updates: a weekly CI schedule runs scripts/upstream.sh, which opens a merge request when CLIProxyAPI releases (a project token in LAB_TOKEN).
Not yet: team-owned accounts and grants, per-team routing to accounts, MCP tools.

Observability (October 2026)

The gate keeps every model call's trace (kit/telemetry Keep) with child spans gate: key and limits, compress, upstream, and notes (key, model, trace) for the meter: CLIProxyAPI's usage record is matched to the nearest note of its key within 3 s and stored as usage.trace_id. The principal stays the key id (CLIProxyAPI scopes sessions by it). usage.error keeps the upstream's error.message (200 characters), never the body.
GET /api/v1/usage/live (server-sent events) streams calls as they are recorded, scoped like the list; GET /api/v1/usage/errors?days= groups failed calls by provider, account, status and message.
Circuit breaker (meter.Called → api.Called): three 401/403 in a row turn an OAuth account off (ai.account.auth); 80% of at least 10 calls failing (5xx, no answer) in 2 minutes park it for 10 minutes (ai.account.tripped, unparked by the pool). Never the provider's last working account.
Metrics: ai_upstream_calls_total, ai_tokens_total, ai_cost_dollars_total, ai_upstream_seconds, ai_ttft_seconds, ai_quota_remaining_ratio.

Routing (October 2026)

CLIProxyAPI serves a model from the highest priority tier with an available credential, by weight inside it (weighted-round-robin); session affinity binds a conversation to a credential. aid's policy (settings.routing): subs_first (accounts 10+, remote CLIProxyAPI sources 5, API-key sources 0), affinity + TTL, a strategy per provider — spread (weight = what the account-wide window has left), drain (the soonest reset with over 5% left gets tier 20), even, manual (a tier per account in the order given). Applied after every quota reading and on PUT /api/v1/routing (accounts' priority and weight credential fields, only when they change; config/routing; sources' group priority).
Aliases gain retry_on (429, 404, 5xx, none) and first_byte (seconds): the gate moves on when headers don't come in time (never on the last model).
Panel: Routing — pool lanes (request → tiers → "when all are busy" → remote → keys) with drag-to-order in manual, and routes as sortable cards with conditions on the arrows. Models shows routes read-only.

Results and the arena (October 2026)

GET /api/v1/stats/report?month=&scope=me|team|instance — a month's totals, models, providers, days, hours (UTC), the biggest call, keys used, the streak of active days, the previous month's totals; for the instance each account's value at API prices against its monthly_price (a credential field set through PATCH /api/v1/accounts/{name}). A person's calls are their own keys and the team keys they made.
GET /api/v1/stats/achievements — 14 milestones computed from a person's calls and arena votes; PUT /api/v1/stats/me {leaderboard} opts in (people); GET /api/v1/stats/leaderboard?month= lists only those who opted in, by tokens, with names from id's directory.
Arena: POST /api/v1/arena/votes (a, b, tie, bad), GET /api/v1/arena — Elo replayed over arena_votes (K 24, from 1000; "both bad" counts for neither). The panel runs both answers through the playground with the person's key.

Chat (v2.1, October 2026)

The playground became the chat at /chat (2.1.2; /playground is gone); its answers run on the server (internal/chat), so they go on when the tab closes, resume after a reload and show on another device.

Conversations are trees: messages (parent, parts, status streaming|done|error|stopped, the provider's output items in raw, body with a simple tsvector for search); conversations.head is the leaf shown. An edited message or another answer is a sibling. Pre-2.1 conversations move from data into messages on start (variants become sibling answers); data stays read-only for a release.
POST /api/v1/chats/{id}/runs {parent, parts, models, key} starts answers (1, or 2–4 side by side) and returns at once; GET /api/v1/runs/{run}/events streams them (SSE: a snapshot, then part, delta, status, ask, title, done; Last-Event-ID resumes); POST /runs/{run}/stop. Answers are written every 500 ms; on start, unfinished ones become stopped.
The loop calls the gate in-process on /v1/responses with the person's key (counted and limited like any call); CLIProxyAPI translates it for every provider. Claude, Codex and Gemini search natively ({type: web_search}); other models get web_search over the instance's SearXNG (search_url in settings). Tools: fetch_url (public addresses only, checked at connect time and on every redirect; 2 MB in, 100 KB of text out), run_python (asked of the person's open tab: Pyodide in a worker; 60 s, then "unavailable"), create_artifact (html, react, svg, markdown, mermaid, code; versions in artifacts), remember (a proposal the person confirms into memories). 8 tool rounds and 10 minutes an answer. Output items replay only to the same provider family; a 4xx retries once without native search, then without replayed items.
Instructions: the instance's (date, citing), the assistant's and the project's with their files (200k characters), "about me" and memories, then the conversation's own.
Files: POST /api/v1/files (20 MB each, 200 MB a person); text from PDF, Word, Excel, PowerPoint and text files is extracted on upload; pictures go to the model as images. Projects and assistants carry files and can be shared with a team to use.
GET /api/v1/chats/search?q= (prefix words, ⟦highlighted⟧), POST/DELETE /api/v1/chats/{id}/public and GET /api/v1/public/{token} (no sign-in: the shown branch's words, steps, sources and artifacts; never instructions, memory, files or keys), POST /api/v1/chats/import (ChatGPT or claude.ai export, 50 MB).
Artifacts render in iframe sandbox="allow-scripts" (no same-origin) from srcdoc with a content policy; React artifacts load pinned React, Babel and Tailwind from esm.sh and jsDelivr.
Metrics: ai_chat_runs_total{status}, ai_tool_calls_total{tool,result}, ai_tool_seconds{tool}; each answer is a trace with a span per tool.

Arena 2 (v2.2, October 2026)

Modes: text battles (two models drawn at random, unnamed until the vote), side (2–4 chosen models, named, never rated), webdev (both build one artifact; previews side by side) and image; each rated mode has its own Elo. Battles are conversations of kind arena (out of the chat list and search) answered by the chat engine on separate lines (StartInput.Lines), so they are multi-turn and survive a closed tab; an unjudged battle's messages and events carry no model names.
arena_battles (mode, category, search, models, roots, revealed, winner, public token); arena_votes gained battle, mode, category and search. The service model labels the first prompt (code, math, writing, creative, analysis, advice, other).
Pools: GET/PUT /api/v1/arena/settings (superusers; per mode, empty for every model of the kind); less voted models are drawn more (weight 1/√(1+votes)).
POST /api/v1/arena/battles, POST …/{id}/turns, POST …/{id}/vote (reveals, returns Elo before and after), GET /api/v1/arena/battles[/{id}], POST/DELETE …/{id}/public (after the vote) and GET /api/v1/public/arena/{token}, GET /api/v1/arena/leaderboard?mode=&category=&search= (Elo, 7-day change, win rate, W–T–L; the caller's taste: own ratings and families picked).