Move from LiteLLM

LiteLLM can send a request to any OpenAI Chat Completions endpoint. The exchange needs a new model, api_base and api_key. Settings that LiteLLM runs by itself, such as fallbacks and num_retries, need more work. For setup, see LiteLLM.

Change these three things

1. Issue a spend key on the API keys page, and export it as LIQUID_API_KEY. See API keys.
2. Set api_base to https://router.inference.ai.exchange/v1 (keep /v1, add no path) and api_key to the key, not a LiteLLM key. In the proxy, set both in litellm_params of each model_list entry.
3. Set model to openai/ and a name that GET /v1/models lists. In the proxy, keep each model_name, so your clients do not change.

Then check these:

• Move the routing settings to the exchange object. See Settings that change and LiteLLM features.
• Run a dry run. See Check a request without a purchase.
• Read What is different about money. The exchange never lowers an output limit that you set. See Output limits.

What differs

• Retries and fallbacks. LiteLLM runs them before the request leaves it, and the exchange does not read them. Each retry is a new request with its own cap. To compete several models, send architect.basket with exchange.candidates.models.allow.
• Budgets and keys. A LiteLLM budget does not reach the exchange. Set a spending limit on the exchange key.
• Traffic weights. None. The exchange ranks the eligible offers with exchange.rank.strategy.

Example

A LiteLLM call with a retry, a fallback and a label, then the same call for the exchange:

import litellm

response = litellm.completion(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Say hello."}],
    max_tokens=500,
    num_retries=2,
    fallbacks=["gpt-4o-mini"],
    metadata={"team": "support"},
)

import os
import litellm

response = litellm.completion(
    model="openai/architect.basket",
    api_base="https://router.inference.ai.exchange/v1",
    api_key=os.environ["LIQUID_API_KEY"],
    messages=[{"role": "user", "content": "Say hello."}],
    max_tokens=500,
    extra_headers={"X-Title": "LiteLLM", "x-liquid-tag": "support"},
    extra_body={
        "exchange": {
            "candidates": {"models": {"allow": ["gpt-oss-120b", "gpt-oss-20b"]}},
            "cap_usd": 0.05,
        }
    },
)

The basket replaces fallbacks, and the cheapest eligible offer wins. x-liquid-tag replaces the metadata label. One failure returns one error. If you need retries, add them in your code with an idempotency-key header. The model names are illustrations.

Reference

Settings that change

LiteLLM features

Troubleshooting

For 402 insufficient_balance and 400 no_eligible_offer, see What is different about money.