# Serve and get paid When your offer wins, the exchange announces the award on your session and sends the job to your dispatch endpoint over HTTP. You stream the answer back. The exchange pays for the usage that you report. ## Award notice ```json { "type": "award_notice", "award_id": "aw_…", "job_id": "job_…", "offer_id": "3f0c…", "quote_revision": 7, "activation_seq": 12, "served_model": "gpt-oss-120b", "cap_usd": 0.0184, "max_output_tokens": 1024 } ``` - `quote_revision` and `activation_seq` name the winning quote. `activation_seq` comes from its `quote_ack`. - `cap_usd` is the maximum charge. The exchange fixes it before the buyer receives output. - The exchange settles at the winning rates, even if you quote again. Keep the terms of each revision. - The notice and the dispatch request can arrive in any order, and twice. Join them on `award_id`. ## Dispatch request The exchange sends a `POST` with a JSON body to your `dispatch_url`. The `idempotency-key` header holds the award id. ```json { "award_id": "aw_…", "job_id": "job_…", "offer_id": "3f0c…", "served_model": "gpt-oss-120b", "messages": [ { "role": "user", "content": "What is in this picture?", "parts": [ { "type": "text", "text": "What is in this picture?" }, { "type": "image_url", "url": "https://example.com/photo.jpg", "detail": "auto" } ] } ], "max_output_tokens": 1024, "temperature": 0.2, "stream": true } ``` The exchange sends a parameter only to offers that declared it. | Field | Description | | -------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `pricing_unit` | The unit of the winning offer, if unusual for the kind. `tokens` on an image job means a token charge. | | `messages[].content` | The text of the turn. | | `messages[].parts` | Content that is not text. See [Attachments](#attachments). | | `messages[].tool_calls`, `messages[].tool_call_id` | The tool calls of an assistant turn, and the call that a `tool` turn answers. | | `messages[].reasoning` | Reasoning blocks from the buyer. See [Reasoning](#reasoning). | | `max_output_tokens` | The limit for the cap. The exchange charges output above it to you. | | `tools`, `tool_choice` | Tool definitions in the Anthropic `input_schema` form, and the buyer's `tool_choice`. Only with `tools`. | | `response_format` | The buyer's `response_format`. | | `top_p`, `top_k`, `min_p`, `repetition_penalty`, `stop`, `seed`, `logprobs`, `top_logprobs`, `logit_bias`, `frequency_penalty`, `presence_penalty` | Sampling parameters. Each needs the related `sampling.*` feature. | | `reasoning` | `{ "effort": … }` or `{ "budget_tokens": … }`. `effort` is always a string, such as `"80"`. | | `cache` | Cache breakpoints. See [Prompt caching](#prompt-caching). | | `audio_output` | `{ "voice": …, "format": … }`. The buyer asked for speech with the text. Only with `output.audio`. | | `image`, `audio`, `video` | The output to produce, only on a deployment of that kind. See [Images](#images), [Speech](#speech) and [Video](#video). | ### Attachments A turn with more than text has a `parts` array next to its `content`. `content` still holds the text. | Part | Fields | Sent when the offer declared | | ------------- | ----------------------------------- | ---------------------------- | | `text` | `text` | always | | `image_url` | `url`, optional `detail` | `input.image` | | `input_audio` | `data` (base64), `format` | `input.audio` | | `file` | `data`, `mime`, optional `filename` | `input.file` | | `video_url` | `url` | `input.video` | `url` is a link or a `data:` URL, as the buyer sent it. Only images can be links. You get an attachment only when your offer declared the feature and your registration admits it. You get `video_url` only if you declare `input.video`. Today only the OpenAI adapter does so, and only when video input is on or the model card route names video. ### Prompt caching `cache.breakpoints` lists up to four breakpoints in prompt order. Each has an `anchor` and a `ttl` of `5m` or `1h`: ```json { "breakpoints": [ { "anchor": { "at": "tools" }, "ttl": "1h" }, { "anchor": { "at": "message", "index": 5 }, "ttl": "5m" } ] } ``` `tools` is the end of the tool definitions. `message` is the end of `messages[index]`. For an Anthropic upstream, put `cache_control` on the last block before each breakpoint. - `cache.key` groups requests that share a prefix. Send it to an upstream that routes by it, such as `prompt_cache_key` of OpenAI. - `cache.scope` is unique to the buyer. If your upstream account serves more than one buyer, put it before all cached content. Otherwise one buyer can learn what another sent. Anthropic dialect: send it as the first system block, and move each breakpoint after the tools to the end of the system prompt. Automatic caching: send it as the first system message. Do not cache past the deepest breakpoint, because the cap covers writes only up to it. If your upstream charges for writes, quote `cache_write_short` and `cache_write_long` in `pricing`. A job that reports a write with no rate does not settle. ### Reasoning An assistant turn can have `reasoning` blocks: | `type` | Fields | | --------------------- | ----------------------------------------------- | | `reasoning.text` | `text`, optional `signature`, optional `format` | | `reasoning.encrypted` | `data`, optional `format` | `format` names the models that can read the `signature` or `data`, such as `anthropic-claude-v1`. With thinking on, Anthropic refuses the next turn of a tool-use loop unless the signed blocks come first and unchanged. Send back blocks in your upstream format, and drop the others. ## Response Respond with `200` and `Content-Type: application/x-ndjson`. Write one JSON event on each line. Always stream, whatever `stream` says. The exchange buffers the output for buyers who did not ask for streaming. The exchange closes a dispatch that is silent for 120 seconds, and limits a chat job to one hour by default. A streaming buyer pays for the output received before a limit. ``` {"type":"reasoning","text":"The image shows a bicycle. "} {"type":"delta","text":"There is a "} {"type":"delta","text":"red bicycle."} {"type":"done","finish_reason":"stop","usage":{"meters":{"uncached_input":412,"cache_read":0,"cache_write_short":0,"cache_write_long":0,"generation":6},"tokenizer_id":"o200k_base","reasoning_tokens":null,"raw":{}}} ``` | Event | Fields | Description | | --------------------- | --------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `delta` | `text` | Output text. The first `delta` stops the time-to-first-token clock. | | `logprobs` | `content` | Log probabilities for the next `delta`. See [Log probabilities](#log-probabilities). It does not stop the clock. | | `reasoning` | `text` | Model reasoning, if the upstream returns it. It does not stop the clock, but after it the job cannot change provider. | | `audio_delta` | `data` (base64), `transcript`, `id` | A piece of the spoken answer to an `audio_output` request. It has `data`, `transcript` or both. Only the first piece has the `id`. It stops the clock and ends any change of offer. Report `audio_tokens` in `completion_tokens_details`. | | `reasoning_signature` | `signature`, `format` | Ends the reasoning block with the upstream signature. The next `reasoning` starts a new block. | | `reasoning_redacted` | `data`, `format` | An encrypted reasoning block. The buyer cannot read it, but must send it back. | | `tool_call_start` | `index`, `id`, `name` | A tool call starts. `index` orders concurrent calls. | | `tool_call_delta` | `index`, `arguments` | A part of the argument JSON. Join the parts for each `index`. | | `image` | `index`, `b64_json` or `url`, `mime`, optional `revised_prompt` | One complete image. `index` is its position in the requested images. | | `audio` | `b64_audio`, `mime` | The complete clip. | | `video_ready` | `handle`, `token`, `mime`, optional `bytes`, `expires_at` | The finished video. It holds no bytes. | | `done` | `finish_reason`, `usage` | Success. Terminal. | | `failed` | `code`, `message` | Failure. Terminal. After you send output, the exchange does not try another provider. | ### Log probabilities If the dispatch has `logprobs: true`, send a `logprobs` event before each `delta` whose tokens have log probabilities. `content` lists the tokens of that `delta` in order: ```json { "token": "A", "logprob": -0.12, "bytes": [65], "top_logprobs": [{ "token": "B", "logprob": -2.4, "bytes": [66] }] } ``` `top_logprobs` has at most the number of entries that the dispatch asks for, and can be empty. `bytes` is optional. Send no entry for reasoning or tool-call tokens. If your offer declared `sampling.logprobs` and the answer has text, send the first `logprobs` event before the first `delta`. - If you do not, the attempt fails with `logprobs_missing` before the buyer gets output. The exchange tries the next offer and ranks the model out for a short time. - If the events stop after the first `delta`, the job settles and the exchange records the failure. - ### Usage - `meters`: token counts for `uncached_input`, `cache_read`, `cache_write_short`, `cache_write_long` and `generation`. Exactly one meter counts each token. - `tokenizer_id`: must equal the registered `tokenizer_id`. - `reasoning_tokens`: optional. The reasoning tokens in `generation`. - `raw`: the upstream usage object, unchanged. The exchange keeps it as evidence. The exchange settles on `meters`, at the winning rates. ## Images An image dispatch has an `image` object: ```json { "count": 2, "size": "1024x1024", "output_format": "png", "response_format": "b64_json" } ``` The exchange used `count`, `size` and `output_format` to match your offer and set the price. It does not pay for more images or a different size. `size` is the pixel size that the exchange picked when the buyer sent `resolution` or `aspect_ratio`. These fields are present only when the buyer set them and your offer declared the feature: | Field | Feature | What to do | | -------------------- | -------------------------- | ------------------------------------------------------------------------------------ | | `seed` | `image.seed` | Pass it to your upstream. | | `background` | `image.background` | `transparent` or `opaque`. The exchange drops `auto`. | | `output_compression` | `image.output_compression` | A whole number from 0 to 100, for a `jpeg` or `webp` image. | | `stream` | `image.stream` | `true`. The buyer asked for one image. Send drafts as `image_partial` events if any. | `quality` is present when the buyer set it. It sets the price and is not matched. To send a draft, send an `image_partial` event before the final `image`: ``` {"type":"image_partial","index":0,"partial_index":0,"b64_json":"iVBOR…","mime":"image/png"} ``` A draft is valid only if the buyer set `stream`. `index` is below `count`, `partial_index` is 0, 1 or 2, and you send each pair once. Do not send a draft after the final image of the same `index`. A draft with no bytes, or any other break of these rules, fails the job. A draft is not an image: `meters.images` counts the final images only. Send each image complete in one `image` event, with exactly one of `b64_json` and `url`. The first image stops the time-to-first-token clock. Then send `done` with `meters.images` set to the number sent. The charge is that count times the `output_image` price. If you meter more images than you sent, the job fails. When `pricing_unit` is `tokens`, also set `meters.uncached_input` and `meters.generation`. A job with no image fails. ## Speech A speech dispatch has an `audio` object: ```json { "characters": 18, "voice": "alloy", "response_format": "mp3", "speed": 1.0 } ``` The exchange used `voice` and `response_format` to match your offer. `characters` is the length of the text of all turns together, and it is the quantity charged. An empty `voice` means that the buyer named none: speak in the first voice that you declared. - `quality` sets the price and is not matched. - `instructions` is a style line for the clip. It is present only with `speech.instructions`. - `turns` is a list of `{voice?, instructions?}`, one for each turn, and turn n speaks user message n. It is present only for more than one turn, and only with `speech.turns`. Send one `audio` event with the full clip as base64. Then send `done` with `meters.characters` set to the count you received. The charge is that count times the `output_speech` price. The job fails if you meter more characters, send no clip, send invalid base64, or send two clips. ## Transcription A transcription dispatch has a `transcription` object. The audio is one `input_audio` part in the messages. ```json { "seconds": 4, "format": "mp3", "language": "en", "timestamps": ["segment"], "diarize": true, "keyterms": ["Liquid Inference"] } ``` `seconds` is the length that the exchange measured, rounded up. It is the cap. `language`, `timestamps`, `prompt` and `temperature` are present only when the buyer set them. `diarize` and `keyterms` are present only with `transcription.diarize` and `transcription.keyterms`. Send one `transcript` event with `text`, and `language`, `segments` and `words` if you have them. A segment has `id`, `start`, `end` and `text`, and a `speaker` such as `speaker_0` when `diarize` is true. Always return the full form. The exchange builds the format that the buyer asked for. Then send `done` with `meters.audio_seconds` set to the seconds that you transcribed. It must be more than 0. The charge is that count times your `input_audio` price, and not more than the cap. Times must be finite, and `end` must not be before `start`. ## Embeddings An embeddings dispatch has an `embedding` object: `inputs` (the number of texts), `dimensions`, `encoding_format` and `input_type`. `input_type` is present only with `embedding.input_type`, and it is one of `search_query`, `search_document`, `classification` and `clustering`. Send one or more `embeddings` events with the vectors, in order. Send exactly one vector for each input. Meter input tokens only. ## Video If your upstream has the OpenAI video routes, the reference agent does this for you. See [Sell video](https://inference.ai.exchange/docs/provider-video.md). A video dispatch has a `video` object: ```json { "seconds": 8, "size": "1280x720", "audio": true, "resolution": "720p", "seed": 7 } ``` The exchange used `seconds` and `size` to match your offer and set the price. `audio` and `resolution` are the buyer's choice of sound and resolution. `size` is the pixel size that the exchange picked when the buyer sent `aspect_ratio`. The other fields below are present only when the buyer set them and your offer declared the feature. | Field | Feature | What to do | | ----------------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | `seed` | `video.seed` | A whole number of 0 or more. Pass it to your upstream. | | `creativity` | `video.creativity` | A number from 0 to 1. | | `upscale_factor` | `video.upscale_factor` | A whole number of 2 or more. | | `previous_job_id` | `video.continue` | The job id of an earlier video of the same workspace that completed. See [Continue a video](https://inference.ai.exchange/docs/provider-video.md#continue-a-video). | `reference_images`, `reference_video_seconds` and `reference_audio_seconds` are the most that you can meter for references. The prompt is the last user turn. The buyer's references are content parts of that turn, in the order sent. A frame image comes first, with a `frame_type` of `first_frame` or `last_frame`. See [Frame images](https://inference.ai.exchange/docs/provider-models.md#frame-images). A render can take minutes. While it runs, send a blank line at least every 60 seconds. The exchange ignores blank lines, but closes a dispatch that is silent for 120 seconds. It limits the job to ten minutes by default. If it closes the dispatch, stop the work. The exchange closes the dispatch in the same way when the buyer cancels the job, and it sends no other message. See [Cancel and expiry](https://inference.ai.exchange/docs/provider-video.md#cancel-and-expiry). Do not send the video in the event stream. When your upstream has the file, send one `video_ready` event. Then send `done` with `meters.video_seconds`. The charge is that count times your `output_video` price. ``` {"type":"video_ready","handle":"task_1","token":"5f2c…","mime":"video/mp4","bytes":7195859} {"type":"done","finish_reason":"stop","usage":{"meters":{"video_seconds":8},"tokenizer_id":"provider-native","raw":{}}} ``` - `handle` is your name for the file. - `token` proves that a later request comes from the exchange. Make it from the handle with a key that only you know, such as an HMAC. Keep the key across restarts, or the buyer cannot download a paid video. - `bytes` and `expires_at` are optional. Send `expires_at` if your upstream deletes the file at a known time. The job fails and does not settle if you meter more seconds than asked, send no `video_ready` or two, name an empty file or one that is not `video/*`, or send an event after `done`. When the buyer downloads the video, the exchange sends a `POST` with `{"handle": …, "token": …}` to `content` next to your dispatch path (`/v1/dispatch` becomes `/v1/content`). Check the token, then send the file with its `Content-Type`. Answer any other token with `403`. Keep the file until `expires_at`. ## Idempotency The exchange can send a dispatch twice. Record award ids and refuse a repeat before upstream work starts. The reference agent answers `409 duplicate_award`. ## Failures ### What ends an attempt - A non-`2xx` status, or a connection that does not open, ends your attempt. The exchange tries the next offer. - If no other offer can take the job, the buyer gets `503 provider_failed`. A `4xx` status does not change this, except `tools_unsupported`. - If your endpoint accepts the dispatch and then stops with no event, the job fails with `503 provider_failed`. The exchange does not try another offer, because you may still run the job. ### Report an upstream error Send a `failed` event. Start its `message` with ` returned : `, then the upstream body. A client error is a `4xx` status other than `401`, `402`, `403`, `404` or `429`. The buyer gets `400` with one of these codes: - `tools_unsupported`: the upstream says the model cannot use tools. With no output sent, the exchange tries the next offer. - `request_rejected`: any other client error. The buyer must fix it, so the exchange does not try another offer. - `upstream_account_limited`: your upstream account reached a spend limit or has no funds. See below. Other failures give `503 provider_failed`: an unreachable provider, a provider fault, and the upstream statuses `401`, `402`, `403`, `404` and `429`. A `402` means your upstream account cannot pay. A `404` means your upstream lacks an offered model. With no output sent, the exchange first tries the next offer. ### Account limits The exchange also recognizes the usual words of an account-limit error, such as `spend limit`, at any status. The buyer never sees the `message`, and the receipt keeps only the code. Until you raise the limit or add funds, each attempt that you win for that upstream fails. ### What counts against your deployment A failure counts against your deployment health only when you caused it. A `request_rejected` failure and a fault of the exchange do not count. A late answer counts as slow, not as a failure. An attempt that moved to the next offer counts. ### Bad tool calls and bad JSON The exchange also checks your answers. A bad answer does not change the job or its charge. The rates show in your console. They change the ranking only if the buyer sets `exchange.rank.failure_risk_ranking` ([Purchasing rules](https://inference.ai.exchange/docs/rules.md)). - **Tool calls.** A call is bad when its arguments are not JSON, it names a tool that the request did not offer, or its arguments break the tool schema. - **JSON output.** For `json_object` or `json_schema`, the answer is bad when it is not one JSON value or does not match the schema. A `max_tokens` stop, a refusal and tool calls alone do not count. ### Reference agent behavior When an upstream refuses a job with `upstream_account_limited`, `401`, `402` or `403`, the reference agent marks the account as down for 5 minutes. It quotes each model through the next cheapest upstream that sells it, and withdraws models that no other upstream sells. After the cooldown, the next job tests the account. Each failure doubles the cooldown, up to 1 hour, and a success resets it. The state is in memory. ## Settlement The exchange settles each job on `done`. Your earnings are the reported meters at the winning rates, plus any fixed fee, minus buyer credits. The exchange fee is a separate line that the buyer pays. You then get a `settlement_notice`: ```json { "type": "settlement_notice", "award_id": "aw_…", "job_id": "job_…", "meters": { "uncached_input": 412, "cache_read": 0, "cache_write_short": 0, "cache_write_long": 0, "generation": 6 }, "supplier_charge_usd": 0.0000523, "credit_usd": 0, "status": "settled" } ``` `supplier_charge_usd` is your earnings before the exchange fee. `credit_usd` is the credit that the buyer gets when the provider fails after it sent output. | Condition | Effect | | --------------------------------------------------- | --------------------------------------------------------- | | Output above `max_output_tokens` | The exchange charges the overshoot to you. | | Missed `ttft_ms` or `completion_ms` deadline | The buyer pays. The exchange marks you slow. | | Usage in an unregistered tokenizer | The exchange does not settle it as the same unit. | | Failed job, or failure after output that you caused | Nobody pays and nobody earns. The buyer keeps any output. | | The buyer closes a stream after output | The buyer pays for the output relayed, and you earn it. | The **Work** page of the console shows the settled values. To copy them into your own systems, read the jobs feed. ### Read the jobs feed `GET /v1/provider/jobs` returns your settled jobs, oldest first. It takes the provider key of your agent: ```bash curl "https://router.inference.ai.exchange/v1/provider/jobs?limit=500" \ -H "Authorization: Bearer $LIQUID_PROVIDER_KEY" ``` ```json { "jobs": [ { "job_id": "job_…", "award_id": "aw_…", "served_model": "deepseek-v4.1", "outcome": "completed", "status": "settled", "supplier_charge_usd": 0.0000523, "buyer_charge_usd": 0.0000601, "fee_usd": 0.0000078, "credit_usd": 0, "meters": { "uncached_input": 412, "generation": 6 }, "ttft_ms": 412, "completion_ms": 1290 } ], "next_cursor": "1793628191204000000:job_…", "has_more": false } ``` 1. Read without a cursor. The first read starts 90 days ago. 2. Send `next_cursor` as `cursor` on the next read. 3. While `has_more` is `true`, read again at once. Otherwise wait at least one minute. - A job appears five minutes after it settles and never changes. Its fields match the `settlement_notice`, so join on `award_id`. The feed does not name the buyer. - A read during another read gets `429` with `Retry-After`. Wait that many seconds and read again. ### Time to first token The buyer's `ttft_ms` deadline counts from the start of the job, so it includes earlier failed offers. Your own time is `ttft_ms` less `attempt_started_ms` on the receipt. The exchange marks you slow only when your own attempt passed the deadline. ## Payout destination You set your payout destination through the hosted onboarding of the payout rail. Operators can read and clear a destination, but not set one. These endpoints use the browser session and need manage access. `PUT` and `POST` also need a recent identity check. ### Start onboarding ``` POST /v1/account/providers/{provider_id}/payout/onboarding ``` Returns a URL for Stripe Connect onboarding. You give your identity, business and bank details to Stripe, not to the exchange. The exchange continues the stored account if the rail opened it for this exchange. Otherwise it opens and stores a new account. The account is a company account for the `hq_country` of the provider. That country sets the agreement with Stripe: - The US, Canada, the UK, Switzerland and the EEA (except Iceland): **full** agreement. Transfers arrive on the normal Stripe schedule. - Any other country, including Iceland: **recipient** agreement. The account only receives transfers, and each one is available 24 hours later than usual. - Stripe cannot change the agreement after you accept it. - Outside the US, a full-agreement account also needs the card payments capability, so Stripe asks for card details. The exchange takes no card payments through it. - If Stripe refuses the account, the code is `rail_refused`. ### What the account is It is a Stripe Connect Express account under the exchange platform. It only receives transfers and sends them to your bank. It is separate from your own Stripe accounts, and can share their bank account. The exchange cannot connect a Stripe account that you opened yourself. ### Attach or clear an existing account ``` PUT /v1/account/providers/{provider_id}/payout ``` Attaches an account that the rail connects to this exchange. Send `rail` and `handle` together. A body with neither clears the destination. The exchange asks the rail about the account before it stores the handle: - A handle that is not an account ID, or an account that the rail does not connect to the exchange: `400` with `unknown_payout_account`. Nothing is stored. - The exchange cannot ask the rail: `503`. Nothing is stored. Send the request again. - The account is not finished yet: accepted. `GET` on the same path returns the destination with `payable`: `true` if the rail can pay the account now, `false` if the rail needs more from you or no longer connects the account (`blocked_reason` says why), and `null` with no destination or if the exchange could not ask the rail. ## Payouts Each hour, the exchange opens a payout for each provider whose balance is at least $25. It is one Stripe transfer to the destination. - It pays earnings that settled before today. It rounds down to whole cents and caps each transfer at an operator limit. The remainder stays in the balance. - A provider with no destination, or one that the rail cannot pay, gets no payout. - If Stripe refuses a transfer, the money returns to the balance. The run waits 24 hours before it tries that provider again. The **Earnings** page shows the amount owed and paid. ## Endpoint requirements - The exchange must reach the endpoint at the scheme, host and port of your `declared_endpoint`. You choose the path. - The endpoint must have a public address. The exchange rejects loopback, private and link-local addresses. An exchange can also limit dispatch hosts to an allowlist. - On shutdown, finish the current dispatches first.