# Deployments A deployment is a model that you serve at one precision in one region. Register it, state its limits, and move it from `UNLISTED` to the public book. A registration has no prices. Your session [quotes the deployment](https://inference.ai.exchange/docs/provider-quoting.md), and an operator decides whether to list it. Every endpoint here takes the [provider key](https://inference.ai.exchange/docs/provider.md) as a bearer token. ## Register a deployment ```bash curl https://router.inference.ai.exchange/v1/provider/models \ -H "Authorization: Bearer sk-liquid-..." \ -H "Content-Type: application/json" \ -d '{ "deployment_name": "gpt-oss-120b", "model": "gpt-oss-120b", "creator": "OpenAI", "precision": "fp8", "region": "eu-west", "max_input_tokens": 128000, "max_output_tokens": 4096, "tokenizer_id": "o200k_base", "input_modalities": [{ "type": "text" }, { "type": "image", "max_bytes": 20000000 }], "output_modalities": [{ "type": "text" }] }' ``` | Parameter | Type | Required | Description | | ---------------------------- | ------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `deployment_name` | string | Yes | Your name for the deployment. The book shows `/`. | | `model` | string | No | Legacy upstream identifier. Default: `deployment_name`. It does not approve a public name. | | `upstream_model` | string | No | Opaque handle that the exchange returns at dispatch. Not part of the identity. Default: `model`. | | `registry_model_id` | UUID | No | An approved registry identity of the same kind. See [Select an approved identity](#select-an-approved-identity). | | `creator` | string | No | The company that made the model. Buyers see it, and the `exchange.candidates.creators.allow` rule uses it. Ids are lowercase. Where [models.dev](https://models.dev) lists the company, use its lab id, such as `alibaba` for Qwen or `zhipuai` for GLM. | | `model_family`, `base_model` | string | No | The model that this one is a variant of, and the model that a quantization or fine-tune comes from. | | `precision` | string | Yes | Weight precision, for example `fp8`, `bf16`, `int4`. | | `region` | string | Yes | Serving region. The `exchange.privacy.regions` rule uses it. | | `max_input_tokens` | integer | Yes | Context limit in the deployment tokenizer. Input and output of one job must fit in it together. Must be positive. | | `max_output_tokens` | integer | No | Maximum tokens for each job. Must be positive. Each registration writes this field, so omitting it removes a limit that you set before. | | `tokenizer_id` | string | No | Tokenizer of the usage reports. Default: `provider-native`. | | `kind` | string | No | `chat` (per token), `image_generation` (per image), `audio_speech` (per character) or `video_generation` (per second). Default: `chat`. The first registration sets it. | | `input_modalities` | array | No | Input kinds with limits. See [What the deployment takes](#what-the-deployment-takes). An empty list states nothing. | | `output_modalities` | array | No | Output kinds with limits. See [What the deployment produces](#what-the-deployment-produces). An empty list means text only. | | `pricing_unit` | string | No | An `image_generation` deployment can state `tokens` when its upstream bills for tokens. Omit it to keep the current unit. | | `image_output_tokens_max` | integer | No | Most output tokens for one image. Required with `pricing_unit` `tokens`, and refused otherwise. The cap of a job holds this many tokens for each image. | | `description` | string | No | Text that buyers see. Omit it to keep the current text. Send `""` to clear it. | | `hardware` | object | No | See [Hardware](#hardware). Omit it to keep the current claim. | | `serving` | object | No | See [Serving](#serving). Omit it to keep the current claim. | | `byok` | boolean | No | `true` makes the exchange serve the deployment with your own cloud key. `upstream_model` must be a model id that a live key lists. See [Sell with bring your own key](https://inference.ai.exchange/docs/provider-byok.md). | The response is `201` with every field above, and also: | Field | Meaning | | -------------------------------------------------------------------- | -------------------------------------------------------------------------- | | `id` | The registration id. Buyers see it as the offer id. | | `model_deployment` | `/`. It follows a handle change. | | `canonical_model` | The public name. It is `null` without a registry link. | | `status`, `status_reason` | The [listing status](#listing-statuses). A new registration is `UNLISTED`. | | `markup_bps`, `paused_at`, `created_at`, `listed_at`, `suspended_at` | Margin, pause and status times. Unset values are `null`. | Repeat registrations: - Registration is idempotent on `(provider_id, deployment_name, precision, region)`. A repeat call returns the current registration and clears no suspension or pause. - A change to the deployment identity clears its registry link and public admission. - `kind` cannot change. A new kind returns `409 kind_changed`. Use a new deployment name. Errors: `400` with `invalid_registration`, `invalid_bands`, `invalid_modalities`, `invalid_output_modalities`, `invalid_pricing`, `invalid_hardware` or `invalid_serving_claims`; `409 kind_changed`; `401`; `503 storage_unavailable`. ## Hardware State the hardware of this deployment only, not your fleet. The exchange does not use the claim to rank offers, route a job or break a tie. | Field | Type | Required | Description | | ------------------------ | ------- | -------------------- | ------------------------------------------------------------------------------------------------- | | `accelerator` | string | Yes | For example `nvidia-h100-80` or `amd-mi300x`. | | `interconnect` | string | With more than 1 GPU | How devices in one node connect, for example `nvlink` or `infinity-fabric`. | | `gpus_per_node` | integer | Yes | Devices in one node, 1 to 72. A rack system, such as an NVL72, is one node. | | `nodes` | integer | Yes | Nodes that serve one replica, 1 to 1024. | | `network` | object | No | Link between nodes. Allowed only when `nodes` is more than 1. | | `host` | string | Yes | `bare_metal`, or `vm_passthrough`. | | `instance_type`, `notes` | string | No | Instance type, such as `p5.48xlarge`, and other facts that change speed (500 characters at most). | | `network` field | Type | Required | Description | | ---------------- | ------- | -------- | ------------------------------------------------------------------------- | | `fabric` | string | Yes | For example `infiniband`, `roce` or `efa`. | | `topology` | string | No | `rail_optimized`, `fat_tree`, `torus`, `full_mesh` or `other`. | | `bandwidth_gbps` | integer | No | Bandwidth of one node in Gbit/s, 1 to 100000. Add every card of the node. | | `placement` | string | No | `same_rack`, `same_cluster` or `same_zone`. | An operator claim stays when you register again. A count out of range, a missing accelerator, no interconnect for more than 1 device, or a bad `network` returns `400 invalid_hardware`. ## Serving State how you serve the deployment. The exchange cannot check the values and does not use them to rank or route. Every field is optional. | Field | Type | Description | | ----------------------- | ------- | -------------------------------------------------------------------------------------------- | | `engine` | string | `vllm`, `sglang`, `tensorrt_llm`, `dynamo`, `llama_cpp` or `other`. | | `engine_version` | string | For example `0.10.2`. It needs an `engine`. At most 32 characters. | | `speculative_decoding` | boolean | `true` for speculative decoding or multi-token prediction. | | `disaggregated_serving` | boolean | `true` when prefill and decode run on separate machines. | | `kv_cache_precision` | string | `bf16`, `fp16`, `fp8`, `int8` or `fp4`. Separate from the weight precision. | | `prompt_caching` | string | `none`, `automatic` (the engine finds a repeated prefix) or `explicit` (the buyer marks it). | An engine version with no engine, or one that is too long, returns `400 invalid_serving_claims`. ## What the deployment takes The `type` of each entry in `input_modalities` is `text`, `image`, `audio`, `video` or `file`. Each `type` can occur once. An absent limit means that you stated none. | Field | Applies to | Checked | Description | | ------------------------ | ------------------------- | ------- | ------------------------------------------------------------------------------------------------- | | `sources` | image, audio, video, file | Yes | `url`, `base64`. | | `formats` | image, audio, video, file | Yes | Accepted MIME types, for example `image/png`. | | `max_bytes` | image, audio, video, file | Yes | Largest single part, in bytes. | | `max_references` | image | Yes | Most images that one request can reference. | | `max_duration_seconds` | audio, video | Audio | Longest clip, in seconds. | | `supported_frame_images` | image | Yes | Frames that the deployment reads: `first_frame`, `last_frame`. See [Frame images](#frame-images). | | `detail_levels` | image | No | `image_url.detail` levels: `auto`, `low`, `high`. | The list has two effects: - **It limits what you can quote.** Your agent quotes a media feature only for a kind that the list names. The list cannot add to what your upstream accepts. - **It excludes your offer from a job that breaks a limit.** The reason is `media_unsupported`, and the error names the constraint, such as `image.max_bytes`. This happens before the award, so the job can go to another deployment. ### Frame images A buyer can fix the first or last frame of a video with `frame_images`. Declare each frame that your upstream reads in `supported_frame_images` on the `image` input. A deployment that declares no frame never gets an image that has one. ```json "input_modalities": [ { "type": "text" }, { "type": "image", "max_references": 2, "supported_frame_images": ["first_frame", "last_frame"] } ] ``` ## What the deployment produces The `type` of each entry in `output_modalities` is `text`, `image`, `audio` or `video`, once each. Image, speech and video deployments name `image`, `audio` and `video`, and must also name `text`. | Field | Applies to | Checked | Description | | ---------------------- | ------------------- | ---------- | ------------------------------------------------------------------------------------------------------- | | `formats` | image, audio, video | Yes | Formats it produces. Write `png` or `image/png`. | | `sizes` | image, video | Yes | Pixel sizes, for example `1024x1024`. | | `max_count` | image | Yes | Most images in one response. | | `response_formats` | image | Yes | `url`, `b64_json` or both. If you list none, a request for a form can still reach it. | | `voices` | audio | Yes | Voices it can speak in. | | `seconds` | video | Yes | Lengths it renders, for example `[4, 8]`. | | `max_duration_seconds` | audio, video | Video only | Longest clip. For audio, the exchange only records it, because it cannot measure a clip first. | | `audio` | video | Yes | Sound of the video: `none`, `always` or `optional`. See [Sound](#sound-of-a-video). | | `resolutions` | video | Yes | Resolutions it renders, such as `720p`. See [Sound, resolution and price](#sound-resolution-and-price). | A video request states its length before the job starts, from 1 to 30 seconds. List every length that the deployment renders in `seconds`, for example `[4, 5, 6, …, 30]`. A request for a length that you did not list does not reach you, and registration refuses a length above 30. If you set only `max_duration_seconds`, a request for a length below your real minimum can reach you and fail. A request with no `seconds` gets the shortest length that any offer lists, or 4 when none lists one. The exchange does not accept `seconds: -1`, because the buyer must state the length. ### Sound, resolution and price A buyer can send `generate_audio` (`true` or `false`) and `resolution` (such as `720p`). A request that sends one reaches only a deployment that states the option and can make it. A deployment that states nothing serves only a request that does not send it. - `audio` is `none` (every video is silent), `always` (every video has sound) or `optional` (the buyer chooses, and a request that says nothing gets sound). - `resolutions` lists the labels that the deployment renders, such as `["720p", "1080p"]`. If the price depends on the sound or the resolution, quote several `output_video` lines with a `variant`. [Quote a media model](https://inference.ai.exchange/docs/provider-media-quote.md#3-write-the-price-table) has the rules and an example. The exchange refuses a quote with a variant that names a sound without `audio` as `optional`, or a resolution that `resolutions` does not list. Send the choice to your upstream: the dispatch holds it in `audio` and `resolution` of the `video` object. A request for a format, size, count or voice that you did not list goes to another offer before the award. The charge is known before dispatch: | Kind | `pricing` line | Charge | | --------------------------------- | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Image, each image | `output_image`, no token line | Rate times the images that the buyer asked for. | | Image, `pricing_unit` `tokens` | `input_text` and `output_text`, no `output_image` | The tokens that you meter, up to the cap: the prompt at the input rate plus `image_output_tokens_max` per image at the generation rate. | | Speech | `output_speech` | Rate times the characters in the text. | | Video | `output_video` | Rate times the seconds that the buyer asked for. | | Reference images (image or video) | `input_reference_image` | Rate times the reference images that you metered, at most the number the exchange counted. Without the rate they are free. A token-priced image deployment does not quote it. Start and frame images do not count. | | Reference video | `input_reference_video` | Rate times the seconds of reference video that you metered, at most the seconds sent. | | Reference audio | `input_reference_audio` | Rate times the seconds of reference audio that you metered, at most the seconds sent (30 at most). | `operator_locked` names each field that an operator set. A later registration does not undo it. ## List deployments `GET /v1/provider/models` with your provider key returns each registration in all statuses. The `listed` frame of the session lists what you can quote now. ## From registration to the public book A listing puts a deployment on the public book, where buyers reach it by model name. A registration never lists itself. 1. **Register.** The deployment starts `UNLISTED`. 2. **Quote it.** It rests as an [unverified offer](#unverified-offers). 3. **The exchange measures it** with benchmarks and quality checks. 4. **An operator lists it, or the exchange does.** The exchange sets `listed_at`. See [Automatic listing](#automatic-listing). 5. **After the next quote**, it competes under the model name. ## Listing statuses Two independent fields control availability. A paused `LISTED` deployment keeps its listing, but you do not quote it. | Field | Who sets it | Meaning | | ----------- | -------------------------------------------------------- | --------------------------------------------- | | `status` | An operator, and you when you resume a paused deployment | Whether the deployment is on the public book. | | `paused_at` | You | Set while you do not quote it. | | Status | Quotable | Description | Operator transitions | | ----------- | --------------- | ------------------------------------------------------------------------------------------------ | ----------------------- | | `UNLISTED` | Yes, unverified | Not in the public book. Only your workspace and the exchange buy from it, by `model_deployment`. | `LISTED`, `SUSPENDED` | | `LISTED` | Yes | In the public book. Buyers can ask for it by model name. | `UNLISTED`, `SUSPENDED` | | `SUSPENDED` | No | No buyer can reach it. `status_reason` and `suspended_at` give the reason and time. | `UNLISTED` (reinstate) | Public admission requires an approved registry identity. `status_reason` and `suspended_at` are set only when the status is `SUSPENDED`. `listed_at` is set only when it is `LISTED`. The session rejects a quote for a `SUSPENDED` or paused deployment with `quote_reject`. A new registration does not end a suspension. Repair the cause and contact the operators. ## Automatic listing When automatic listing is on, the exchange checks every deployment every 60 seconds. An operator can still list by hand. It lists an `UNLISTED` deployment when all of these are true: - Your provider is admitted and public. - Your signed provider agreement is on record. Ask the operators to record it. - The deployment has an approved registry identity of its kind, and it names no other identity than its link. - You did not pause it, and no reservation commits it. - Your last decision was not an unlisting. - You list no other deployment of the same identity under the same name. Case and punctuation do not count. ## Unverified offers A quote for an `UNLISTED` deployment rests as an **unverified** offer. Only your workspace and the exchange can buy from it. - Send its `model_deployment`, such as `acme/gpt-oss-120b`, as the model, with an API key of the owning workspace. Another workspace gets `model_not_found`. - The exchange buys from it to measure it. It never chooses it for a model name, router model or preset, and its quotes stay out of the price history. - Use it to test end to end with a real auction and settlement. To change the owner workspace: 1. Open the provider in the console and go to **Access**. 2. Choose **Move to another workspace**. You must be an owner or admin of both workspaces. 3. Restart the agent. The exchange reads the owner when the agent connects. ## Select an approved identity A public deployment must have an approved identity from the model registry, of the same kind. 1. Send `GET /v1/provider/model-registry` with your provider key. Each record has a UUID, canonical name, creator, revision, kind and aliases. 2. Send the UUID as `registry_model_id` when you register. 3. Keep the dispatch handle in `upstream_model`. - `/v1/provider/models` and the session return the public name as `canonical_model`. Read the dispatch handle from `upstream_model`, not `model`. - If an operator retires an identity, the linked deployments are unlisted and their names blocked. Your workspace can still use each exact symbol. Without an approved identity, use a private deployment by its exact symbol. The model request panel in the console accepts evidence for review. ## Pause and resume A pause stops the quotes for one deployment and keeps its listing. ```bash curl -X POST https://router.inference.ai.exchange/v1/provider/models/3f0c.../pause \ -H "Authorization: Bearer sk-liquid-..." ``` This sets `paused_at` and removes the resting offer at once. Use `/resume` to clear the pause. Resume does not grant public admission, and it changes nothing for a `SUSPENDED` deployment. The account API does the same: ``` PATCH /v1/account/providers/{provider_id}/models/{model_id} { "paused": true } ``` Send `{ "paused": false }` to resume. An unknown id returns `404 no_model`. ## Set a margin The console stores a margin for each deployment, in basis points above your upstream cost. ``` PATCH /v1/account/providers/{provider_id}/models/{model_id} { "markup_bps": 1500 } ``` An integer from 0 to 100000 sets the margin (`1500` is 15%). `null` restores the agent default. An absent field changes nothing. A value out of range returns `400 invalid_markup`. The agent reads the margin on its next refresh.