# Sell with an OpenAI-compatible API If your API already speaks the OpenAI dialect, you can sell with no agent and no delivery endpoint. The exchange calls your API directly. To start, [log in](#log-in) with `dispatch_dialect` set to `openai`. To run an agent instead, see [Serve and get paid](https://inference.ai.exchange/docs/provider-dispatch.md). ## When to use this Use this path when your API streams a chat completion in the OpenAI format. You still keep the session open to quote. [Quote over the session](https://inference.ai.exchange/docs/provider-quoting.md) has the frames. | The exchange sends | Your route | Answer | | ------------------ | ---------------------------------------- | --------------------------------------------- | | Chat | `POST {dispatch_url}/chat/completions` | Server-sent events, with a last `usage` chunk | | Embeddings | `POST {dispatch_url}/embeddings` | One JSON body | | Speech | `POST {dispatch_url}/audio/speech` | The audio file | | Images | `POST {dispatch_url}/images/generations` | One JSON body | These jobs need an agent, and the exchange does not send them to this path. The next offer gets the job. - Video jobs. See [Sell video](https://inference.ai.exchange/docs/provider-video.md). - Decision jobs. - A deployment that needs the queue of the studio engine. ## Set it up 1. Register the provider and its deployments. See [Register deployments](https://inference.ai.exchange/docs/provider-models.md). 2. Log in with `dispatch_dialect` set to `openai`. 3. Quote as usual. ### Log in `dispatch_url` is the base URL of your API. Its scheme, host and port must match the registered `declared_endpoint`. `dispatch_token` is the key of your API. The exchange sends it as `Authorization: Bearer ` and keeps it in memory for this session only. ```json { "type": "login", "slug": "acme", "api_key": "sk-liquid-...", "dispatch_url": "https://api.acme.example/v1", "dispatch_token": "your-upstream-api-key", "dispatch_dialect": "openai", "capabilities": [] } ``` ### Open the session and quote This script logs in, quotes each listed deployment and then sends a `heartbeat` every two seconds. The exchange closes a session that sends no frame for five seconds. See [Keep the session alive](https://inference.ai.exchange/docs/provider-quoting.md#keep-the-session-alive). The script uses the Python `websockets` package. A real provider also quotes again before the quote expires, and when its price changes. ```python import asyncio import json import websockets URL = "wss://router.inference.ai.exchange/v1/provider/ws" LOGIN = { "type": "login", "slug": "acme", "api_key": "sk-liquid-...", "dispatch_url": "https://api.acme.example/v1", "dispatch_token": "your-upstream-api-key", "dispatch_dialect": "openai", "capabilities": [], } def quote(model_id: str, revision: int) -> dict: return { "type": "quote", "target": {"kind": "registered", "model_id": model_id}, "pricing": [ {"billable": "input_text", "unit": "mtok", "usd": 0.12}, {"billable": "output_text", "unit": "mtok", "usd": 0.48} ], "service_class": {"ttft_ms": 1500, "completion_ms": 30000}, "supported_features": ["text"], "retention_policy_supported": ["none"], "revision": revision, } async def beat(socket) -> None: while True: await asyncio.sleep(2) await socket.send(json.dumps({"type": "heartbeat"})) async def main() -> None: async with websockets.connect(URL) as socket: await socket.send(json.dumps(LOGIN)) beating = asyncio.create_task(beat(socket)) try: async for raw in socket: frame = json.loads(raw) print(frame["type"]) if frame["type"] == "listed": for model in frame["models"]: await socket.send(json.dumps(quote(model["model_id"], 1))) elif frame["type"] in ("login_reject", "quote_reject"): raise SystemExit(frame) finally: beating.cancel() asyncio.run(main()) ``` ## What the exchange sends ### Chat The exchange posts a streamed completion to `{dispatch_url}/chat/completions`. The header `Idempotency-Key` is the award id. Your server may ignore it. The exchange never sends an award twice after the request leaves. ```bash curl https://api.acme.example/v1/chat/completions \ -H "Authorization: Bearer your-upstream-api-key" \ -H "Idempotency-Key: aw_8f3c…" \ -H "Content-Type: application/json" \ -d '{ "model": "acme-70b-instruct", "messages": [{ "role": "user", "content": "Name a prime number." }], "max_tokens": 64, "stream": true, "stream_options": { "include_usage": true } }' ``` `model` is the upstream identifier of the deployment. `max_tokens` is the cap of the award. Stream server-sent events. Send a last chunk with `usage`. The exchange settles on `prompt_tokens`, `completion_tokens`, `prompt_tokens_details.cached_tokens` and `completion_tokens_details.reasoning_tokens`. A stream with no usage block does not settle as a success. ```text data: {"choices":[{"index":0,"delta":{"role":"assistant","content":"Seven"}}]} data: {"choices":[{"index":0,"delta":{"content":" is prime."},"finish_reason":"stop"}]} data: {"choices":[],"usage":{"prompt_tokens":14,"completion_tokens":5,"total_tokens":19,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}} data: [DONE] ``` ### Embeddings ```bash curl https://api.acme.example/v1/embeddings \ -H "Authorization: Bearer your-upstream-api-key" \ -H "Content-Type: application/json" \ -d '{ "model": "acme-embed-1", "input": ["first text", "second text"], "encoding_format": "float", "dimensions": 256 }' ``` The answer must report `usage.prompt_tokens`, because the job settles on it. Without it the job fails and the buyer pays nothing. ```json { "data": [ { "index": 0, "embedding": [0.012, -0.044, 0.31] }, { "index": 1, "embedding": [0.101, 0.002, -0.27] } ], "usage": { "prompt_tokens": 6, "total_tokens": 6 } } ``` ### Speech ```bash curl https://api.acme.example/v1/audio/speech \ -H "Authorization: Bearer your-upstream-api-key" \ -H "Content-Type: application/json" \ -o answer.mp3 \ -d '{ "model": "acme-tts-1", "input": "The auction is open.", "voice": "alloy", "response_format": "mp3" }' ``` The answer is the audio file. The exchange counts the characters of the text itself, because speech has no token count. It types the file by your `Content-Type`, for example `audio/mpeg`. ### Images ```bash curl https://api.acme.example/v1/images/generations \ -H "Authorization: Bearer your-upstream-api-key" \ -H "Content-Type: application/json" \ -d '{ "model": "acme-image-1", "prompt": "A trading floor made of water", "n": 2, "size": "1024x1024" }' ``` ```json { "data": [{ "b64_json": "iVBORw0KGgo…" }, { "b64_json": "iVBORw0KGgo…" }] } ``` How an image job settles: - A deployment priced for each image settles on the number of images that you return. - A deployment priced in tokens settles on `usage` (`input_tokens` and `output_tokens`, or `prompt_tokens` and `completion_tokens`). It fails without `usage`. The exchange reads the whole answer of an embeddings, speech or images call before the buyer gets any of it, so a bad answer goes to the next offer. A call that sends no byte for two minutes fails. ## Refuse a job Answer an error with a status code of 400 or more before the first chunk, and the exchange offers the job to the next offer. ```http HTTP/1.1 429 Too Many Requests Content-Type: application/json { "error": { "message": "the deployment is at capacity", "type": "rate_limit" } } ``` After the first chunk of a chat answer, the exchange cannot fall back to another offer, because the buyer would get one answer from two providers. ## Limits - A media job that carries a reference input, such as an image to edit, fails on this path. The dialect has no field for it. Use an agent for those jobs. - A render has no queue and no keep-alive. The exchange does not use the studio engine for this path. - Quote as usual. The `quote` frames do not change.