Prompt caching

abliteration.ai automatically caches reusable prompt prefixes. You do not need to enable caching. When a request reads cached input tokens, those tokens are reported in the response and billed at 10% of the standard input rate.

Cache hits are best-effort. Repeating an unchanged prompt does not guarantee a hit. Cached-token counts are measured per request, not accumulated across a conversation, so they can rise, stay flat, fall, or return to zero.

Why an unchanged prefix can miss

Cache entries are not guaranteed to remain available for every request. Scaling, capacity changes, failover, deployment updates, or eviction can cause an unchanged prefix to be processed again. Routing hints such as prompt_cache_key and x-abliteration-session-id improve the chance that related requests reuse cached work. They do not reserve cache capacity or guarantee a cache hit.

What must match

Caching operates on the rendered token prefix, not only the visible user message. A reusable prefix generally requires:

the same project and model,
identical system and developer instructions,
identical earlier messages in the same order,
identical tool definitions and tool ordering,
compatible reasoning, output-format, and multimodal settings, and
enough unchanged tokens to form reusable cache blocks.

You can change content after the reusable prefix. For example, appending a new user message to an otherwise unchanged conversation can reuse the earlier conversation prefix.

Change	Expected effect
Append a new message	The unchanged earlier prefix can still hit
Edit or reorder an earlier message	Reuse stops at the first changed portion
Change system instructions or tools	The affected prefix usually misses
Compact, summarize, or truncate conversation history	Reuse stops at the first changed portion
Change the model or project	The request uses a separate cache namespace
The matching entry is no longer available	The prefix is processed again

Understand cache usage and billing

Cache usage is measured independently for each API request. A growing conversation does not mean the cached-token count must increase on every request.

Usage category	What it means	Price
Cache read	The request reused cached tokens	10% of the standard input rate
Cache creation or write	Tokens were processed and made eligible for possible later reuse	Standard input rate
Uncached input	Tokens were processed without cache reuse	Standard input rate

One request can include both cache reads and cache writes. A cache write does not promise that a later request will read those tokens.

Improve cache reuse

Place stable content first and variable content last. Keep system instructions, tool definitions, and their ordering consistent between requests. Use one stable, opaque affinity value for each logical conversation or reusable workload. Do not use one global value for unrelated traffic, and do not put prompts, personal data, or secrets in the value. For Chat Completions and Responses, set prompt_cache_key. This Chat Completions example uses one key for a conversation:

{
  "model": "abliterated-model-large-v2",
  "prompt_cache_key": "conv_7d2a9f",
  "messages": [
    { "role": "system", "content": "You are a concise coding assistant." },
    { "role": "user", "content": "Review this function." }
  ]
}

For any generation endpoint, you can send the same value in the Abliteration session header: x-abliteration-session-id: conv_7d2a9f. If both values are present, x-abliteration-session-id is used as the routing hint. For parallel workloads, let the first request begin returning output before sending the remaining requests so a reusable entry has time to be created.

Agent clients

Agent clients such as Claude Code can send several API requests for one visible turn, including tool-use continuations, retries, and conversation compaction. Usage and request counts are recorded per API request, not per visible turn. Compaction changes conversation history and can reduce cache reuse. For Claude Code, CLAUDE_CODE_ATTRIBUTION_HEADER=0 is not required for prompt caching on abliteration.ai.

Cache controls and retention

abliteration.ai does not promise a fixed cache-retention window. Entries can be evicted or become unavailable before a later request arrives.

OpenAI-compatible prompt_cache_retention is not supported.
Claude Code’s ENABLE_PROMPT_CACHING_1H flag does not extend or guarantee cache retention on abliteration.ai.
On /v1/messages, supported Anthropic-style cache_control objects are accepted for SDK compatibility, but caching remains automatic. A ttl value does not reserve an entry or guarantee that it remains available for that duration on abliteration.ai.

Do not apply OpenAI or Anthropic’s cache-retention guarantees to requests sent to the abliteration.ai base URL.

Check cache usage

Treat the response usage as the source of truth. A positive cached-token count confirms the tokens actually read from cache.

Endpoint	Cache-read usage field
`/v1/chat/completions`	`usage.prompt_tokens_details.cached_tokens`
`/v1/responses`	`usage.input_tokens_details.cached_tokens`
`/v1/messages`	`usage.cache_read_input_tokens`

For a streaming request, inspect the final usage event or chunk. On Chat Completions, set stream_options: { "include_usage": true } to receive the usage chunk. A reported cache-read count of zero means no tokens received the cache-read discount for that request. If usage is absent, inspect the final usage-bearing streaming event or the dashboard instead of inferring a miss. A cache miss is normal; the affected input tokens are billed at the standard input rate and processed again rather than reused.

Get started

Capabilities

Integrations

Policy Gateway

Reference

Why an unchanged prefix can miss

What must match

Understand cache usage and billing

Improve cache reuse

Agent clients

Cache controls and retention

Check cache usage

​Why an unchanged prefix can miss

​What must match

​Understand cache usage and billing

​Improve cache reuse

​Agent clients

​Cache controls and retention

​Check cache usage

Why an unchanged prefix can miss

What must match

Understand cache usage and billing

Improve cache reuse

Agent clients

Cache controls and retention

Check cache usage