Response caching

Response caching stores the whole answer at the exchange. An identical request later gets that answer back, and no provider runs. A hit costs nothing.

It is off by default. Send x-liquid-cache: true to use it for one request. It works on Chat Completions, Messages and Responses, with and without streaming.

curl https://router.inference.ai.exchange/v1/chat/completions \
  -H "Authorization: Bearer sk-liquid-..." \
  -H "x-liquid-cache: true" \
  -H "x-liquid-cache-ttl: 3600" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-oss-120b", "messages": [{"role": "user", "content": "Name three rivers."}]}'

The first request returns x-liquid-cache-status: MISS and runs a job that you pay for. The same request within the hour returns HIT and costs nothing. In Python, read the status from the raw response:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://router.inference.ai.exchange/v1",
    api_key=os.environ["LIQUID_API_KEY"],
)

raw = client.chat.completions.with_raw_response.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "Name three rivers."}],
    extra_headers={"x-liquid-cache": "true", "x-liquid-cache-ttl": "3600"},
)
print(raw.headers["x-liquid-cache-status"])  # MISS, then HIT
completion = raw.parse()

Before you turn it on

• A shared key shares the cache. An entry belongs to one API key. If your application uses one key for many end users, an end user can get the stored answer of another end user who sent the same request. Do not turn the cache on for a request that holds private data.
• A hit repeats the answer. The content is the same each time, so a change of temperature does not produce a new answer. To get a new one, send x-liquid-cache-clear: true.
• A chat rarely hits. The whole message history must be the same. A second "Hi" in the same chat has a longer history, so it misses. The same "Hi" in a new chat hits. For a long chat, use prompt caching instead.

What a hit returns

A hit returns the stored content again, with a new id and a new created time. Every usage counter is 0, and the charge is 0. The exchange block holds a cache object in place of the receipt of a job, because no job was bought. A hit has no x-liquid-job-id header, and it moves no money.

To delete the entry and store a new answer, send x-liquid-cache-clear:

client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "Name three rivers."}],
    extra_headers={"x-liquid-cache": "true", "x-liquid-cache-clear": "true"},
)

The request then runs, and you pay for it.

What decides a hit

A request hits an entry when all of these are the same:

• The API key. Two keys never share an entry, also inside one workspace.
• The endpoint.
• The stream setting. A streamed answer and a buffered answer have separate entries.
• The request body. The fields can be in any order, and white space does not count. The whole message history counts.

The top-level fields prompt_cache_key and session_id do not count. They only name a conversation for the prompt cache of the provider. Headers do not decide a hit. A purchasing rule that you send as a header, such as x-liquid-cap-usd, does not change which entry the request finds.

Limits

• Only a finished answer is stored. A refusal, an error, a failed job and a stream that did not end are never stored.
• The exchange can drop an entry before its TTL ends. A request can then miss.
• Two identical requests that arrive together both miss, and both are bought. The exchange does not join them.
• An answer larger than 4 MiB is returned but not stored.
• Images, speech, video and embeddings do not use the cache.

Reference

Request headers

A header that cannot be read returns 400 invalid_header. This includes a TTL outside 1 to 86400.

Rules

To use the cache for every request of a preset, set exchange.cache.response: true in the rules of the preset. The rules exchange.cache.response and exchange.cache.response_ttl_seconds can also appear in the exchange block of a request. They win over the headers. Thus response_cache: false in the request turns the cache off, also if you send x-liquid-cache: true. A x-liquid-cache-ttl header wins over exchange.cache.response_ttl_seconds. A rule TTL outside 1 to 86400 is moved into that range.

Response headers

Next steps

• Prompt caching: cache the start of a long prompt.
• Purchasing rules: exchange.cache.response and exchange.cache.response_ttl_seconds in a preset.