# Response caching

Response caching stores the whole answer at the exchange. An identical request later gets that answer back, and no provider runs. A hit costs nothing.

It is off by default. Send `x-liquid-cache: true` to use it for one request. It works on Chat Completions, Messages and Responses, with and without streaming.

```bash
curl https://router.inference.ai.exchange/v1/chat/completions \
  -H "Authorization: Bearer sk-liquid-..." \
  -H "x-liquid-cache: true" \
  -H "x-liquid-cache-ttl: 3600" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-oss-120b", "messages": [{"role": "user", "content": "Name three rivers."}]}'
```

The first request returns `x-liquid-cache-status: MISS` and runs a job that you pay for. The same request within the hour returns `HIT` and costs nothing. In Python, read the status from the raw response:

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://router.inference.ai.exchange/v1",
    api_key=os.environ["LIQUID_API_KEY"],
)

raw = client.chat.completions.with_raw_response.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "Name three rivers."}],
    extra_headers={"x-liquid-cache": "true", "x-liquid-cache-ttl": "3600"},
)
print(raw.headers["x-liquid-cache-status"])  # MISS, then HIT
completion = raw.parse()
```

## Before you turn it on

- **A shared key shares the cache.** An entry belongs to one API key. If your application uses one key for many end users, an end user can get the stored answer of another end user who sent the same request. Do not turn the cache on for a request that holds private data.
- **A hit repeats the answer.** The content is the same each time, so a change of `temperature` does not produce a new answer. To get a new one, send `x-liquid-cache-clear: true`.
- **A chat rarely hits.** The whole message history must be the same. A second "Hi" in the same chat has a longer history, so it misses. The same "Hi" in a new chat hits. For a long chat, use [prompt caching](https://inference.ai.exchange/docs/caching.md) instead.

|                            | Prompt caching                              | Response caching                  |
| -------------------------- | ------------------------------------------- | --------------------------------- |
| What it stores             | The start of a long prompt, at the provider | The whole answer, at the exchange |
| What a repeat request does | Runs on a provider, with cheaper input      | Runs on no provider               |
| Charge                     | Cached input at a lower rate                | Nothing                           |
| Default                    | On, with `exchange.cache.mode: "auto"`      | Off                               |

## What a hit returns

A hit returns the stored content again, with a new `id` and a new `created` time. Every `usage` counter is `0`, and the charge is `0`. The `exchange` block holds a `cache` object in place of the receipt of a job, because no job was bought. A hit has no `x-liquid-job-id` header, and it moves no money.

To delete the entry and store a new answer, send `x-liquid-cache-clear`:

```python
client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "Name three rivers."}],
    extra_headers={"x-liquid-cache": "true", "x-liquid-cache-clear": "true"},
)
```

The request then runs, and you pay for it.

## What decides a hit

A request hits an entry when all of these are the same:

- The API key. Two keys never share an entry, also inside one workspace.
- The endpoint.
- The `stream` setting. A streamed answer and a buffered answer have separate entries.
- The request body. The fields can be in any order, and white space does not count. The whole message history counts.

The top-level fields `prompt_cache_key` and `session_id` do not count. They only name a conversation for the prompt cache of the provider. Headers do not decide a hit. A purchasing rule that you send as a header, such as `x-liquid-cap-usd`, does not change which entry the request finds.

## Limits

- Only a finished answer is stored. A refusal, an error, a failed job and a stream that did not end are never stored.
- The exchange can drop an entry before its TTL ends. A request can then miss.
- Two identical requests that arrive together both miss, and both are bought. The exchange does not join them.
- An answer larger than 4 MiB is returned but not stored.
- Images, speech, video and embeddings do not use the cache.

## Reference

### Request headers

| Request header         | Behaviour                                                                                |
| ---------------------- | ---------------------------------------------------------------------------------------- |
| `x-liquid-cache`       | `true` turns the cache on for this request. `false` or no header leaves it off.          |
| `x-liquid-cache-ttl`   | Seconds to keep a new entry, from 1 to 86400. The default is 300.                        |
| `x-liquid-cache-clear` | `true` deletes the entry for this request. The request then runs and stores a new entry. |

A header that cannot be read returns `400 invalid_header`. This includes a TTL outside 1 to 86400.

### Rules

To use the cache for every request of a preset, set `exchange.cache.response: true` in the rules of the preset. The rules `exchange.cache.response` and `exchange.cache.response_ttl_seconds` can also appear in the `exchange` block of a request. They win over the headers. Thus `response_cache: false` in the request turns the cache off, also if you send `x-liquid-cache: true`. A `x-liquid-cache-ttl` header wins over `exchange.cache.response_ttl_seconds`. A rule TTL outside 1 to 86400 is moved into that range.

### Response headers

| Response header                | Value                                                           |
| ------------------------------ | --------------------------------------------------------------- |
| `x-liquid-cache-status`        | `HIT` or `MISS`. Absent when the request did not use the cache. |
| `x-liquid-cache-ttl`           | The full TTL on a miss. The remaining TTL on a hit.             |
| `x-liquid-cache-age`           | On a hit, the seconds since the entry was stored.               |
| `x-liquid-cache-source-job-id` | On a hit, the job that produced the stored answer.              |

## Next steps

- [Prompt caching](https://inference.ai.exchange/docs/caching.md): cache the start of a long prompt.
- [Purchasing rules](https://inference.ai.exchange/docs/rules.md): `exchange.cache.response` and `exchange.cache.response_ttl_seconds` in a preset.