# Move from LiteLLM LiteLLM can send a request to any OpenAI Chat Completions endpoint. The exchange needs a new `model`, `api_base` and `api_key`. Settings that LiteLLM runs by itself, such as `fallbacks` and `num_retries`, need more work. For setup, see [LiteLLM](https://inference.ai.exchange/docs/connect-litellm.md). ## Change these three things 1. Issue a `spend` key on the **API keys** page, and export it as `LIQUID_API_KEY`. See [API keys](https://inference.ai.exchange/docs/keys.md). 2. Set `api_base` to `https://router.inference.ai.exchange/v1` (keep `/v1`, add no path) and `api_key` to the key, not a LiteLLM key. In the proxy, set both in `litellm_params` of each `model_list` entry. 3. Set `model` to `openai/` and a name that `GET /v1/models` lists. In the proxy, keep each `model_name`, so your clients do not change. Then check these: - Move the routing settings to the `exchange` object. See [Settings that change](#settings-that-change) and [LiteLLM features](#litellm-features). - Run a dry run. See [Check a request without a purchase](https://inference.ai.exchange/docs/move-from-openrouter.md#check-a-request-without-a-purchase). - Read [What is different about money](https://inference.ai.exchange/docs/move-from-openrouter.md#what-is-different-about-money). The exchange never lowers an output limit that you set. See [Output limits](https://inference.ai.exchange/docs/errors.md#output-limits). ## What differs - **Retries and fallbacks.** LiteLLM runs them before the request leaves it, and the exchange does not read them. Each retry is a new request with its own cap. To compete several models, send `architect.basket` with `exchange.candidates.models.allow`. - **Budgets and keys.** A LiteLLM budget does not reach the exchange. Set a spending limit on the exchange key. - **Traffic weights.** None. The exchange ranks the eligible offers with `exchange.rank.strategy`. ## Example A LiteLLM call with a retry, a fallback and a label, then the same call for the exchange: ```python import litellm response = litellm.completion( model="gpt-4o", messages=[{"role": "user", "content": "Say hello."}], max_tokens=500, num_retries=2, fallbacks=["gpt-4o-mini"], metadata={"team": "support"}, ) ``` ```python import os import litellm response = litellm.completion( model="openai/architect.basket", api_base="https://router.inference.ai.exchange/v1", api_key=os.environ["LIQUID_API_KEY"], messages=[{"role": "user", "content": "Say hello."}], max_tokens=500, extra_headers={"X-Title": "LiteLLM", "x-liquid-tag": "support"}, extra_body={ "exchange": { "candidates": {"models": {"allow": ["gpt-oss-120b", "gpt-oss-20b"]}}, "cap_usd": 0.05, } }, ) ``` The basket replaces `fallbacks`, and the cheapest eligible offer wins. `x-liquid-tag` replaces the `metadata` label. One failure returns one error. If you need retries, add them in your code with an `idempotency-key` header. The model names are illustrations. ## Reference ### Settings that change | LiteLLM setting | Exchange setting | | ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | `model="openai/"` | `openai/` selects the format. The exchange auctions the name after it: a model, router model, `architect.basket` or preset. | | `extra_body` | The `exchange` object. See [Purchasing rules](https://inference.ai.exchange/docs/rules.md). | | `extra_headers` | `X-Title` names the client on your receipts. `x-liquid-*` headers send rules. See [Purchasing rules](https://inference.ai.exchange/docs/rules.md#headers). | | `max_budget` and `budget_duration` | A spending limit on the exchange key, per day, week or month. See [API keys](https://inference.ai.exchange/docs/keys.md#spending-limits). | | `tags`, `metadata.tags`, `x-litellm-tags` | `x-liquid-tag` with one label. See [Cost tags](https://inference.ai.exchange/docs/rules.md#cost-tags). | ### LiteLLM features | LiteLLM feature | What the exchange does | | ---------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `num_retries`, `retry_after` | Not read. To get the first response again, send the same `idempotency-key` on each try. See [Purchasing rules](https://inference.ai.exchange/docs/rules.md#headers). | | `fallbacks` | Not read. Send `architect.basket` with `exchange.candidates.models.allow`. See [Purchasing rules](https://inference.ai.exchange/docs/rules.md#the-basket). | | `routing_strategy`, `weight`, `order`, `rpm`, `tpm`, many entries with one `model_name` | Not read. Rank with `exchange.rank.strategy`. See [Purchasing rules](https://inference.ai.exchange/docs/rules.md#strategy). | | Virtual keys and their `models` list | Not read. To limit the models of an exchange key, link a preset to it. See [Purchasing rules](https://inference.ai.exchange/docs/rules.md#a-preset-linked-to-a-key). | | `timeout` | Not read. `exchange.limits.ttft_ms` and `exchange.limits.completion_ms` limit which offers compete. They are not a client timeout. See [Purchasing rules](https://inference.ai.exchange/docs/rules.md#service). | | `user`, `metadata` | `user` and `metadata.user_id` name the end user for [guardrail limits](https://inference.ai.exchange/docs/guardrails.md#end-users). The rest of `metadata` is ignored. | | `tools`, `tool_choice`, `response_format`, `stream`, `stream_options`, `max_tokens`, `max_completion_tokens`, `temperature` | Applied. Only an offer that declares the feature can win. See [Other OpenRouter fields](https://inference.ai.exchange/docs/move-from-openrouter.md#other-openrouter-fields). | | `top_p`, `top_k`, `min_p`, `repetition_penalty`, `stop`, `seed`, `logprobs`, `logit_bias`, `frequency_penalty`, `presence_penalty` | Passed to a provider that declares support. If no offer supports one, `unsupported_parameter`. `n` greater than 1 is refused. | ### Troubleshooting For `402 insufficient_balance` and `400 no_eligible_offer`, see [What is different about money](https://inference.ai.exchange/docs/move-from-openrouter.md#what-is-different-about-money). | Error | Cause | Fix | | ----------------------------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | | `401 invalid_api_key` | `api_key` holds a LiteLLM key, or the exchange key is revoked or expired. | Set `api_key` to a current exchange key. See [API keys](https://inference.ai.exchange/docs/keys.md). | | `404 model_not_found` | The name after `openai/` is not on the exchange. | Pick a name from `GET /v1/models`. | | `400 unsupported_parameter`, `param` is a parameter you set | No offer for the model supports that parameter. | Remove the parameter, or select another model. See [Errors](https://inference.ai.exchange/docs/errors.md). | | `402 budget_exhausted` | The exchange key reached its spending limit. A LiteLLM budget does not cause this. | Raise the limit of the key. See [API keys](https://inference.ai.exchange/docs/keys.md#spending-limits). | | `503 model_unavailable` | No seller can serve the model now. The response has `Retry-After: 30`. | Try again later, or select another model. |