Deployments: Decision

A decision deployment answers /v1/systemone. It reads a state and typed questions and returns typed answers. It is metered in input tokens.

Deployments has the fields that every kind shares, the listing statuses and the lifecycle. This page has what is special to this kind.

Register a deployment

Copy the request, set your provider key and change the names. Every endpoint here takes the provider key as a bearer token.

curl https://router.inference.ai.exchange/v1/provider/models \
  -H "Authorization: Bearer $LIQUID_PROVIDER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
      "kind": "decision",
      "deployment_name": "triage-1",
      "creator": "acme",
      "precision": "fp8",
      "region": "eu-west",
      "max_input_tokens": 32000,
      "output_modalities": [
        {
          "type": "decision",
          "max_options": 8
        }
      ]
    }'
import os
import requests

response = requests.post(
    "https://router.inference.ai.exchange/v1/provider/models",
    headers={"Authorization": f"Bearer {os.environ['LIQUID_PROVIDER_KEY']}"},
    json={
        "kind": "decision",
        "deployment_name": "triage-1",
        "creator": "acme",
        "precision": "fp8",
        "region": "eu-west",
        "max_input_tokens": 32000,
        "output_modalities": [
            {
                "type": "decision",
                "max_options": 8
            }
        ]
    },
)
response.raise_for_status()
print(response.json()["status"])
const response = await fetch('https://router.inference.ai.exchange/v1/provider/models', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.LIQUID_PROVIDER_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    kind: 'decision',
    deployment_name: 'triage-1',
    creator: 'acme',
    precision: 'fp8',
    region: 'eu-west',
    max_input_tokens: 32000,
    output_modalities: [
      {
        type: 'decision',
        max_options: 8,
      },
    ],
  }),
})
if (!response.ok) throw new Error(await response.text())
console.log((await response.json()).status)

The answer is 201 with the registration. A new registration is UNLISTED. kind cannot change: a new kind returns 409 kind_changed.

What you declare

Field Meaning
max_options The most options that it reads in one choice question. At least 2.
max_input_tokens The context limit for the state and the questions.

Quote and serve

Quote input.text only. A decision deployment generates no tokens. A job can read the prompt more than once, so the cap counts the prompt as many times as the job allows. Your answers must cover each question by id.

{ "billable": "input.text", "unit": "mtok", "usd": 0.1 }

Charge

The charge is the input tokens that you report, summed over every read, up to the cap.

Next