Deployments: Embedding

An embedding deployment answers /v1/embeddings. It reads text and returns vectors. It is metered in input tokens.

Deployments has the fields that every kind shares, the listing statuses and the lifecycle. This page has what is special to this kind.

Register a deployment

Copy the request, set your provider key and change the names. Every endpoint here takes the provider key as a bearer token.

curl https://router.inference.ai.exchange/v1/provider/models \
  -H "Authorization: Bearer $LIQUID_PROVIDER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
      "kind": "embedding",
      "deployment_name": "bge-large",
      "creator": "baai",
      "precision": "fp16",
      "region": "eu-west",
      "max_input_tokens": 8192,
      "output_modalities": [
        {
          "type": "embedding",
          "dimensions": 1024,
          "adjustable": false
        }
      ]
    }'
import os
import requests

response = requests.post(
    "https://router.inference.ai.exchange/v1/provider/models",
    headers={"Authorization": f"Bearer {os.environ['LIQUID_PROVIDER_KEY']}"},
    json={
        "kind": "embedding",
        "deployment_name": "bge-large",
        "creator": "baai",
        "precision": "fp16",
        "region": "eu-west",
        "max_input_tokens": 8192,
        "output_modalities": [
            {
                "type": "embedding",
                "dimensions": 1024,
                "adjustable": False
            }
        ]
    },
)
response.raise_for_status()
print(response.json()["status"])
const response = await fetch('https://router.inference.ai.exchange/v1/provider/models', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.LIQUID_PROVIDER_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    kind: 'embedding',
    deployment_name: 'bge-large',
    creator: 'baai',
    precision: 'fp16',
    region: 'eu-west',
    max_input_tokens: 8192,
    output_modalities: [
      {
        type: 'embedding',
        dimensions: 1024,
        adjustable: false,
      },
    ],
  }),
})
if (!response.ok) throw new Error(await response.text())
console.log((await response.json()).status)

The answer is 201 with the registration. A new registration is UNLISTED. kind cannot change: a new kind returns 409 kind_changed.

What you declare

Field Meaning
dimensions The native length of a vector. At least 1.
adjustable true when it shortens its vectors on request. Then dimensions is required.
max_input_tokens The context limit for one input, in the deployment tokenizer.

An embedding deployment generates no tokens, so it quotes an input rate and no output rate.

Quote and serve

Quote input.text only. A quote that prices what an embedding does not sell is refused. Declare embedding.input_type if your upstream honours it.

{ "billable": "input.text", "unit": "mtok", "usd": 0.1 }

Charge

The charge is the input tokens that you report, up to the cap. The cap is the estimate of the input tokens of all the texts. You must return exactly one vector for each text.

Next