Model Pricing

Auto AI Router supports per-model cost calculation for spend logging. Prices are loaded from a JSON file or remote URL at startup, refreshed periodically in the background, and merged with any prices stored in the LiteLLM database.

Configuration

server:
  model_prices_link: "file://price.json"
  model_prices_sync_interval: 5m  # optional, default 5m

Both settings support os.environ/VAR_NAME substitution.

Accepted values for model_prices_link:

The file must be valid JSON and must not exceed 100 MB.

Organization Price Profiles

organization_policies can bind a verified LiteLLM organization to an immutable USD tariff profile. These profiles use the same source schemes, size limit, and timeout behavior as server.model_prices_link, but they are loaded synchronously during startup and are not refreshed until process restart.

organization_policies:
  - organization_id: os.environ/CLOUD_RU_ORGANIZATION_ID
    price_profile_id: cloud-ru-2026-09
    model_prices_link: /app/organization-prices/cloud-ru-2026-09.json
    model_allowlist:
      - openai/gpt-5.5-pro
    model_mappings:
      openai/gpt-5.5-pro: gpt-5.5-pro
    credential_denylist:
      - untrusted-provider-credential

The section is disabled when omitted. When present, it requires litellm_db.enabled: true, litellm_db.is_required: true, and litellm_db.disable_spend_logs_write: false.

Organization profile lookup is exact and case-sensitive. AIR prices mapped requests by the raw public model ID from the client request. It does not fall back to the default registry, database prices, canonical IDs, routed IDs, real provider IDs, or normalized names.

The organization tariff JSON is strict. Duplicate exact keys, unknown row fields, null rows, and empty price objects fail startup. An explicit free model must still include at least one recognized price field with a zero value.

When model_allowlist is omitted, the organization sees the global callable surface plus organization mapping keys, subject to exact profile prices in /v1/models. A callable request without an exact profile row returns 503 before provider selection. When model_allowlist is present and empty, the organization surface is empty. When present and non-empty, every listed ID must be routable and have an exact profile row at startup.

Provider credential exclusion

credential_denylist contains exact case-sensitive provider credential names. Matching provider credentials are excluded from initial selection, session affinity, retries, and fallback. Router credentials remain eligible. The restriction follows a request through chained AIR routers. Unknown names are ignored locally and remain available to downstream routers.

The list accepts up to 1024 names and 65536 encoded bytes. Each name may contain up to 256 bytes. Empty names, duplicate names, and control characters fail configuration loading. An omitted or empty list preserves standard routing.

Refresh interval

Prices are read once at startup and then re-read in the background every model_prices_sync_interval. Set it to any Go duration string:

server:
  model_prices_link: "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
  model_prices_sync_interval: 1h  # 30s, 15m, 1h, 24h ...

Behaviour:

• The interval applies only when model_prices_link is set. With an empty link no sync loop is started.
• The startup load happens immediately and does not wait for the first tick.
• A failed refresh (unreachable URL, unreadable file, invalid JSON) is logged as a warning and the previously loaded prices stay in the registry. The next tick retries.
• A successful refresh replaces the whole registry atomically; in-flight requests keep using the prices they already resolved.
• A missing or non-positive value falls back to the 5m default.

Choosing a value:

The effective interval is printed at startup:

26.08.26 10:15:03 [INFO] » Using model prices from link=file://price.json sync_interval=5m0s

Each successful refresh is logged at debug level (Model prices updated), so raise server.logging_level to debug when verifying that a new price file is actually picked up.

Price File Format

The file is a JSON object where each key is a model name and each value is a price descriptor:

{
  "gpt-4o-mini": {
    "input_cost_per_token": 1.5e-07,
    "output_cost_per_token": 6e-07
  },
  "gemini-2.5-flash": {
    "input_cost_per_token": 3e-07,
    "output_cost_per_token": 2.5e-06,
    "input_cost_per_audio_token": 1e-06,
    "output_cost_per_reasoning_token": 2.5e-06
  },
  "claude-opus-4-1": {
    "input_cost_per_token": 1.5e-05,
    "output_cost_per_token": 7.5e-05,
    "cache_read_input_token_cost": 1.5e-06,
    "cache_creation_input_token_cost": 1.875e-05,
    "cache_creation_input_token_cost_above_1hr": 3e-05,
    "cache_read_input_token_cost_above_200k_tokens": 3e-06,
    "cache_creation_input_token_cost_above_200k_tokens": 3.75e-05,
    "cache_creation_input_token_cost_above_1hr_above_200k_tokens": 6e-05
  },
  "imagen-4.0-fast-generate-001": {
    "output_cost_per_image": 0.02
  },
  "gpt-4o-search-preview": {
    "input_cost_per_token": 2.5e-06,
    "output_cost_per_token": 1e-05,
    "search_context_cost_per_query": {
      "search_context_size_low": 0.025,
      "search_context_size_medium": 0.0275,
      "search_context_size_high": 0.03
    }
  }
}

Why prices are per 1 token

All per-token prices are expressed as cost per one token (not per 1 000 or per 1 million). This matches the format used by LiteLLM's model_prices_and_context_window.json, making it straightforward to use the upstream file directly or maintain a custom override file in the same format.

For reference:

• $1.50 / 1M tokens → 1.5e-06 (0.0000015)
• $0.15 / 1M tokens → 1.5e-07 (0.00000015)

Available fields

Cost Calculation

All providers return specialised token counts as subsets of the totals:

• prompt_tokens (Vertex AI, OpenAI) already includes audio_input_tokens, cached_input_tokens
• completion_tokens (all providers) already includes reasoning_tokens, audio_output_tokens, prediction tokens
• Anthropic reports cache tokens separately; OpenAI-compatible APIs report them in prompt/input token details

To avoid billing the same tokens at two different rates, the calculator first computes regular (base-rate) token counts by subtracting all specialised sub-types, then adds each sub-type back at its own rate:

regular_input  = prompt_tokens - audio_input_tokens - cached_input_tokens - cache_creation_tokens
regular_output = completion_tokens - audio_output_tokens - reasoning_tokens
                                   - accepted_prediction_tokens - rejected_prediction_tokens

total = regular_input  × input_cost_per_token
      + regular_output × output_cost_per_token
      + audio_input_tokens  × input_cost_per_audio_token
      + audio_output_tokens × output_cost_per_audio_token
      + cached_text_tokens  × cache_read_input_token_cost
      + cached_audio_tokens × cache_read_input_audio_token_cost
      + cache_creation_5m_tokens × cache_creation_input_token_cost
      + cache_creation_1h_tokens × cache_creation_input_token_cost_above_1hr
      + cached_output_tokens   × output_cost_per_cached_token
      + reasoning_tokens            × output_cost_per_reasoning_token
      + accepted_prediction_tokens  × output_cost_per_prediction_token
      + rejected_prediction_tokens  × output_cost_per_token
      + image_count × output_cost_per_image
      + web_search_requests × search_context_cost_per_query[search_context_size]

This means every token is billed exactly once regardless of how the provider reported it.

Web Search billing

Web Search is billed as a separate tool cost, not as tokens. The calculator reads LiteLLM-compatible search_context_cost_per_query prices and selects one of:

• search_context_size_low
• search_context_size_medium
• search_context_size_high

AIR gets the request size from web_search_options.search_context_size or from a web_search / web_search_preview tool definition. If the request does not specify a size, medium is used.

AIR charges only confirmed response usage:

• usage.server_tool_use.web_search_requests
• usage.web_search_requests
• response.output[] or output[] items with type: "web_search_call"
• Chat Completions url_citation annotations when that API contract confirms a search
• Vertex/Gemini groundingMetadata.webSearchQueries

Merely enabling a tool does not count as execution. A successful response with no confirmed usage is billed for zero searches. Streaming requests use the final provider usage or completed response output. An incomplete tool event is not billed.

per_query multiplies the configured price by the confirmed query count. per_prompt clamps any positive count to one charge. LiteLLM Gemini 2.x entries without an explicit unit use per_prompt, while Gemini 3.x entries explicitly use per_query.

The count and selected context size are written to spend metadata under usage_object.server_tool_use and additional_usage_values.server_tool_use; the tool cost is written to cost_breakdown.tool_usage_cost and cost_breakdown.web_search_cost.

Regular input tokens

Vertex AI and OpenAI include audio and cached tokens inside prompt_tokens. Anthropic reports cache reads and writes separately on the wire, so AIR first normalises Anthropic usage to an inclusive prompt total. The formula then uses the same semantics for every provider:

• Vertex/OpenAI: 100 prompt − 5 audio − 20 cached = 75 regular, then +5 audio +20 cached at their rates
• Anthropic wire usage: 100 input + 20 cache read = 120 normalised prompt; billing uses 120 − 20 cached = 100 regular, then +20 cached at its rate

Regular output tokens

All providers include reasoning inside completion_tokens:

• OpenAI o-series: completion_tokens_details.reasoning_tokens is a subset of completion_tokens
• Vertex Gemini 2.5+: thinking tokens are included in candidatesTokenCount
• Anthropic with extended thinking: thinking tokens are included in output_tokens

The subtraction ensures reasoning is billed at output_cost_per_reasoning_token (not double-charged at the base output rate as well).

Tiered pricing (200k threshold)

Some models charge a higher rate once the context exceeds 200 000 tokens. When input_cost_per_token_above_200k_tokens is set:

below = min(prompt_tokens, 200_000)
above = prompt_tokens - 200_000          # only when prompt_tokens > 200_000

# regular tokens are split proportionally between below/above
regular_above = regular_input × above / prompt_tokens
regular_below = regular_input - regular_above

input_cost = regular_below × input_cost_per_token
           + regular_above × input_cost_per_token_above_200k_tokens

The same logic applies to output tokens using output_cost_per_token_above_200k_tokens.

Cache prices follow LiteLLM's full-session semantics: when prompt_tokens > 200_000, all cache read/write tokens use the matching *_above_200k_tokens rate. The 32k/128k/256k/272k full-session cache fields take precedence over the 200k tier whenever configured, with the highest exceeded threshold winning (see "Long-context pricing" below).

Long-context pricing (32k / 128k / 256k / 272k / 512k full-session tiers)

When the prompt exceeds one of these thresholds, the matching *_above_<N>k_tokens rate applies to the full session rather than only the tokens beyond the threshold — the prompt size selects the tier for regular input, output, cache reads, and cache writes. At exactly the threshold, base rates still apply (the check is strictly "greater than").

272k was added first for models such as GPT-5.6; 32k/128k/256k were added for Alibaba Cloud models (Qwen 3.x, GLM) whose published pricing is a flat rate per input-length bracket (e.g. 0–32k / 32k–128k / 128k–256k / >256k) rather than an incremental rate for the overflow. 512k was added for MiniMax-M3, whose pricing depends on total input token count with a 512k threshold.

A price entry may configure any subset of these five thresholds (e.g. only 32k and 256k, skipping 128k). For each cost component (input, output, cache read, cache write) independently, AIR picks the highest configured threshold that the prompt exceeds, in descending order 512k → 272k → 256k → 128k → 32k, and skips any threshold whose rate field isn't set. If none of the five apply, cost calculation falls back to the 200k proportional tier described above, then to the base rate.

Example: a model with only input_cost_per_token_above_128k_tokens and input_cost_per_token_above_256k_tokens set bills a 150k-token prompt at the 128k rate and a 300k-token prompt at the 256k rate; a 100k-token prompt still uses the base rate.

Specialised token types

How Prices Are Loaded

Loading is handled by internal/models/price_loader.go:

1. The value of model_prices_link is inspected to determine the source:
2. Paths starting with file:// or containing no :// are read from disk.
3. Paths starting with http:// or https:// are fetched via HTTP with a 100 MB limit.
4. The JSON is parsed into a map[string]*ModelPrice.
5. Every key is normalised: the provider prefix is stripped and the name is lowercased.
6. "openai/gpt-4-turbo" → "gpt-4-turbo"
7. "vertex_ai/gemini-2.5-pro" → "gemini-2.5-pro"
8. If two keys normalise to the same string, the last one wins and a warning is logged.
9. The resulting map is stored in a ModelPriceRegistry (thread-safe, sync.RWMutex).
10. A background goroutine repeats steps 1-4 every server.model_prices_sync_interval until shutdown — see Refresh interval.

DB price merging

When the LiteLLM database is enabled, prices defined in LiteLLM_ModelTable are merged on top of the file-based registry via MergeDB. Database prices take precedence for any model that appears in both sources. The file-based prices remain intact for all other models.

Two independent loops write to the registry: the price-file refresh (server.model_prices_sync_interval, default 5m) replaces the whole map, while the DB model-table sync (litellm_db.db_model_sync_interval, default 1m) merges DB prices back on top. A model that exists only in the database is therefore absent from the registry between a file refresh and the next DB sync. With spend logging enabled such a request is rejected with 503 Model pricing unavailable, so keep model_prices_sync_interval at or above db_model_sync_interval when DB-only models are in use.

Cache writes are read from cache_creation_tokens or the OpenAI-compatible cache_write_tokens alias in both Chat Completions and Responses API usage objects. Anthropic's cache_creation_token_details (ephemeral_5m_input_tokens and ephemeral_1h_input_tokens) is preserved in spend-log metadata while the existing aggregate cache-creation token columns remain backward-compatible. Gemini cached-audio counts are taken from cacheTokensDetails when the provider supplies a modality breakdown.

Spend storage contract

AIR keeps LiteLLM's upstream PostgreSQL schema unchanged. LiteLLM_SpendLogs.spend and the daily user, team, organization, and end-user tables contain the total cost. Cache and Web Search breakdowns are stored in LiteLLM_SpendLogs.metadata.

Kafka and ClickHouse expose the same breakdown as typed fields, including web_search_requests and web_search_cost. Use that analytics path when structured reconciliation by usage type is required.

Lookup

When a request completes, the router calls GetPrice(modelName) which normalises the name and returns the *ModelPrice. If no entry is found, cost calculation is skipped, spend is stored as 0, and the metadata cost breakdown is omitted.