Sell with an OpenAI-compatible API

If your API already speaks the OpenAI dialect, you can sell with no agent and no delivery endpoint. The exchange calls your API directly. To start, log in with dispatch_dialect set to openai. To run an agent instead, see Serve and get paid.

When to use this

Use this path when your API streams a chat completion in the OpenAI format. You still keep the session open to quote. Quote over the session has the frames.

The exchange sends Your route Answer
Chat POST {dispatch_url}/chat/completions Server-sent events, with a last usage chunk
Embeddings POST {dispatch_url}/embeddings One JSON body
Speech POST {dispatch_url}/audio/speech The audio file
Images POST {dispatch_url}/images/generations One JSON body

These jobs need an agent, and the exchange does not send them to this path. The next offer gets the job.

Set it up

  1. Register the provider and its deployments. See Register deployments.
  2. Log in with dispatch_dialect set to openai.
  3. Quote as usual.

Log in

dispatch_url is the base URL of your API. Its scheme, host and port must match the registered declared_endpoint. dispatch_token is the key of your API. The exchange sends it as Authorization: Bearer <key> and keeps it in memory for this session only.

{
  "type": "login",
  "slug": "acme",
  "api_key": "sk-liquid-...",
  "dispatch_url": "https://api.acme.example/v1",
  "dispatch_token": "your-upstream-api-key",
  "dispatch_dialect": "openai",
  "capabilities": []
}

Open the session and quote

This script logs in, quotes each listed deployment and then sends a heartbeat every two seconds. The exchange closes a session that sends no frame for five seconds. See Keep the session alive. The script uses the Python websockets package. A real provider also quotes again before the quote expires, and when its price changes.

import asyncio
import json
import websockets

URL = "wss://router.inference.ai.exchange/v1/provider/ws"

LOGIN = {
    "type": "login",
    "slug": "acme",
    "api_key": "sk-liquid-...",
    "dispatch_url": "https://api.acme.example/v1",
    "dispatch_token": "your-upstream-api-key",
    "dispatch_dialect": "openai",
    "capabilities": [],
}

def quote(model_id: str, revision: int) -> dict:
    return {
        "type": "quote",
        "target": {"kind": "registered", "model_id": model_id},
        "pricing": [
          {"billable": "input_text", "unit": "mtok", "usd": 0.12},
          {"billable": "output_text", "unit": "mtok", "usd": 0.48}
        ],
        "service_class": {"ttft_ms": 1500, "completion_ms": 30000},
        "supported_features": ["text"],
        "retention_policy_supported": ["none"],
        "revision": revision,
    }

async def beat(socket) -> None:
    while True:
        await asyncio.sleep(2)
        await socket.send(json.dumps({"type": "heartbeat"}))

async def main() -> None:
    async with websockets.connect(URL) as socket:
        await socket.send(json.dumps(LOGIN))
        beating = asyncio.create_task(beat(socket))
        try:
            async for raw in socket:
                frame = json.loads(raw)
                print(frame["type"])
                if frame["type"] == "listed":
                    for model in frame["models"]:
                        await socket.send(json.dumps(quote(model["model_id"], 1)))
                elif frame["type"] in ("login_reject", "quote_reject"):
                    raise SystemExit(frame)
        finally:
            beating.cancel()

asyncio.run(main())

What the exchange sends

Chat

The exchange posts a streamed completion to {dispatch_url}/chat/completions. The header Idempotency-Key is the award id. Your server may ignore it. The exchange never sends an award twice after the request leaves.

curl https://api.acme.example/v1/chat/completions \
  -H "Authorization: Bearer your-upstream-api-key" \
  -H "Idempotency-Key: aw_8f3c…" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "acme-70b-instruct",
    "messages": [{ "role": "user", "content": "Name a prime number." }],
    "max_tokens": 64,
    "stream": true,
    "stream_options": { "include_usage": true }
  }'

model is the upstream identifier of the deployment. max_tokens is the cap of the award.

Stream server-sent events. Send a last chunk with usage. The exchange settles on prompt_tokens, completion_tokens, prompt_tokens_details.cached_tokens and completion_tokens_details.reasoning_tokens. A stream with no usage block does not settle as a success.

data: {"choices":[{"index":0,"delta":{"role":"assistant","content":"Seven"}}]}

data: {"choices":[{"index":0,"delta":{"content":" is prime."},"finish_reason":"stop"}]}

data: {"choices":[],"usage":{"prompt_tokens":14,"completion_tokens":5,"total_tokens":19,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}}

data: [DONE]

Embeddings

curl https://api.acme.example/v1/embeddings \
  -H "Authorization: Bearer your-upstream-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "acme-embed-1",
    "input": ["first text", "second text"],
    "encoding_format": "float",
    "dimensions": 256
  }'

The answer must report usage.prompt_tokens, because the job settles on it. Without it the job fails and the buyer pays nothing.

{
  "data": [
    { "index": 0, "embedding": [0.012, -0.044, 0.31] },
    { "index": 1, "embedding": [0.101, 0.002, -0.27] }
  ],
  "usage": { "prompt_tokens": 6, "total_tokens": 6 }
}

Speech

curl https://api.acme.example/v1/audio/speech \
  -H "Authorization: Bearer your-upstream-api-key" \
  -H "Content-Type: application/json" \
  -o answer.mp3 \
  -d '{
    "model": "acme-tts-1",
    "input": "The auction is open.",
    "voice": "alloy",
    "response_format": "mp3"
  }'

The answer is the audio file. The exchange counts the characters of the text itself, because speech has no token count. It types the file by your Content-Type, for example audio/mpeg.

Images

curl https://api.acme.example/v1/images/generations \
  -H "Authorization: Bearer your-upstream-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "acme-image-1",
    "prompt": "A trading floor made of water",
    "n": 2,
    "size": "1024x1024"
  }'

{
  "data": [{ "b64_json": "iVBORw0KGgo…" }, { "b64_json": "iVBORw0KGgo…" }]
}

How an image job settles:

The exchange reads the whole answer of an embeddings, speech or images call before the buyer gets any of it, so a bad answer goes to the next offer. A call that sends no byte for two minutes fails.

Refuse a job

Answer an error with a status code of 400 or more before the first chunk, and the exchange offers the job to the next offer.

HTTP/1.1 429 Too Many Requests
Content-Type: application/json

{ "error": { "message": "the deployment is at capacity", "type": "rate_limit" } }

After the first chunk of a chat answer, the exchange cannot fall back to another offer, because the buyer would get one answer from two providers.

Limits