Reference inputs

An image, speech, transcription or video job can start from reference inputs: images, audio, video and files. Send them in one list, input_references, with a type and an optional role for each entry. Each reference counts as input of the job.

Send the list on POST /v1/images/generations, POST /v1/images/edits, POST /v1/videos, POST /v1/audio/speech and POST /v1/audio/transcriptions. For the names that older clients send, see What changed in the API.

curl https://router.inference.ai.exchange/v1/images/generations \
  -H "Authorization: Bearer sk-liquid-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-image-1",
    "prompt": "a logo in the same style",
    "input_references": [
      { "type": "image", "url": "https://example.com/logo.png" },
      { "type": "audio", "data": "<base64>", "media_type": "audio/wav" }
    ]
  }'

Entries

• Send url or data, not both. Bare base64 in data needs media_type, such as image/png.
• media_type always means a MIME type, such as image/png.
• Send up to 16 entries in all, frames included.
• A link to audio is allowed only on /v1/audio/transcriptions. On the other endpoints, send audio as a data:audio/ URL or as base64.
• An endpoint refuses a field that it does not read, with 400 and the name of the field. It never drops it.

Roles

role is optional. It is reference by default.

An endpoint refuses a role that it does not read, with 400. POST /v1/audio/transcriptions takes exactly one audio entry, or a file in a form.

Frame images

On POST /v1/videos, send an image with the role first_frame or last_frame to fix that frame of the video.

"input_references": [
  { "type": "image", "url": "https://example.com/closed.png", "role": "first_frame" },
  { "type": "image", "url": "https://example.com/open.png", "role": "last_frame" }
]

• Send each frame one time at most. Two entries with the same frame and different images get 400. The same image twice is one frame.
• Only an offer that declared the frame can win. If none did, the exchange refuses the request before the auction. eligibility.excluded names the frame, for example image.supported_frame_images: last_frame.
• The start image must have the shape of the video. See Start image and frames in Video generation.
• Frame images ride the price of the video. They are not charged as reference images.

Guidance inputs

An entry with no role, or with the role reference, is guidance.

"input_references": [
  { "type": "image", "url": "https://example.com/character.png" },
  { "type": "video", "url": "https://example.com/motion.mp4" }
]

To continue or extend a finished video, do not send it as a reference. Send previous_job_id. See Video generation.

Voice samples

On POST /v1/audio/speech, send a sample of a voice and, if you have it, the words of the sample.

"input_references": [
  { "type": "audio", "data": "<base64>", "media_type": "audio/wav", "role": "voice_sample" },
  { "type": "text", "text": "The words of the sample.", "role": "voice_transcript" }
]

A voice sample belongs to one voice. Send it with a string or with one turn, not with many turns.

Which offers can win

Only an offer that declared each input kind can win. The kinds are input.image, input.audio, input.video and input.file. The exchange also checks the limits that the offer declared, such as image.max_references and audio.max_bytes.

If no offer takes all of your references:

• The exchange refuses the request before the auction with 400 unsupported_parameter. param names the kind, such as input.image.
• eligibility.excluded gives the reason for each offer.
• The exchange never sends the request to a chat model.

Charge

• A reference image is charged for each image at the rate of the offer. Without a rate it is free.
• A reference video and a reference audio are charged for each second, at most 30 seconds. A job with one goes only to an offer that prices it.
• A frame image and the start image are not charged as reference images.

See Modalities for the cap.

Edit images and video forms

• POST /v1/images/edits is the same purchase for the OpenAI SDK. It needs at least one reference, takes files in image, and refuses a mask.
• POST /v1/videos takes a start image as a file in a form, as the OpenAI SDK sends it.

If no image offer reads your references, the error says so. The Playground then asks a chat model to describe the images, sends the description to the image model, and tells you that it did. The API does not do this.

In the Playground

The Image, Speech, Video and Compare prompt boxes take files. Drag a file onto the box, paste it, or use the paperclip button.

What a box accepts

• An image, speech or video job takes images, WAV, FLAC or MP3 audio, and PDF files as references.
• A chat job takes the same files in the message.
• A speech run with a transcription model takes one audio clip instead.
• The Playground sends up to 8 files, with at most 8 MB for each file and 14 MB in total.
• The Playground refuses a file that the exchange cannot read, such as a video, and it says what the job takes.

Links in the prompt

The Playground sends the prompt as text, so a link to an image in the prompt does not become a reference. If the prompt has a link to a PNG, JPEG, WebP or GIF file, the Playground tells you to download the image and attach it as a file.

When no model reads a file

If no model of a lane reads a file, the Playground marks the file and stops the run. Remove the file, or choose a model that reads it.

The price estimate for an image counts only the offers that read your files. A speech or video job has no estimate.

Video start frame

In a video run, the first image is the start frame. The Size menu sets the size of the video.

If the start frame does not have the shape of the video, the Playground marks it and stops the run. Select Edit on the image to crop it to the shape, or to keep the whole image with black bars. The editor gives a JPEG that is no larger than the size of the video. It works in your browser and sends the image nowhere.

Model page Playground

The Playground on the page of an image model has a Reference images field. Drop, paste or upload images, up to the limit that the offers of the model declare. The field is absent when no offer reads images.

The form also shows each control that an offer declared: the number of images, format, quality, background, compression, seed and streaming. A control that no offer declared is not shown. The JSON view shows the request body.

Saved files

A run keeps its files in this browser. The run header lists them. Run again, Retry and a variation send them again.

Next steps

• Image generation
• Speech
• Video generation