LiteLLM

LiteLLM is a Python library that gives one interface to many model APIs. Use its OpenAI-compatible route, with the exchange as api_base, to send requests to the exchange.

Before you start

A workspace with a balance and a spend key. See API keys.

1. Export the key

export LIQUID_API_KEY=sk-liquid-...

2. Install LiteLLM

# Python 3.10 or later
pip install litellm

3. Send a request

Put openai/ before the model name and set api_base to the exchange. The prefix selects the API format. The exchange auctions the model that follows it.

The X-Title header names the client on your receipts.

import os
import litellm

response = litellm.completion(
    model="openai/liquid.auto",
    api_base="https://router.inference.ai.exchange/v1",
    api_key=os.environ["LIQUID_API_KEY"],
    extra_headers={"X-Title": "LiteLLM"},
    messages=[{"role": "user", "content": "hello"}],
)

To get the current model names, use GET /v1/models. See Market data.

Notes

The exchange sets a cap on each request. It is in the x-liquid-cap-usd response header. To limit total spend, set a spending limit on the key. See API keys.
To send purchasing rules, pass an exchange block in extra_body. See Purchasing rules.

Troubleshooting

Symptom	Cause and fix
`401`	The key is wrong, revoked or expired. Export it again.
`404`	Keep `/v1` at the end of `api_base`.
`model_not_found`	The model is not on the book. Use an exact name from `GET /v1/models`.
A request is refused for lack of funds.	The workspace balance cannot cover the cap. Add funds on the Credits page.

See Errors for the error codes.

Next steps

LangChain: another Python framework with an OpenAI chat model.
OpenAI SDK: streams and purchasing rules in code.
Purchasing rules: models, strategy and limits, and presets.