# Reference inputs

An image, speech, transcription or video job can start from reference inputs: images, audio, video and files. Send them in one list, `input_references`, with a `type` and an optional `role` for each entry. Each reference counts as input of the job.

Send the list on `POST /v1/images/generations`, `POST /v1/images/edits`, `POST /v1/videos`, `POST /v1/audio/speech` and `POST /v1/audio/transcriptions`. For the names that older clients send, see [What changed in the API](https://inference.ai.exchange/docs/api-changes.md).

```bash
curl https://router.inference.ai.exchange/v1/images/generations \
  -H "Authorization: Bearer sk-liquid-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-image-1",
    "prompt": "a logo in the same style",
    "input_references": [
      { "type": "image", "url": "https://example.com/logo.png" },
      { "type": "audio", "data": "<base64>", "media_type": "audio/wav" }
    ]
  }'
```

## Entries

| `type`  | Fields                                    | Notes                                                           |
| ------- | ----------------------------------------- | --------------------------------------------------------------- |
| `image` | `url` or `data`, `media_type`             | An `https://` URL or a `data:image/` URL.                       |
| `audio` | `url` or `data`, `media_type`             | A `data:audio/` URL, or base64 bytes with `media_type`.         |
| `video` | `url` or `data`, `media_type`             | An `https://` URL or a `data:video/` URL.                       |
| `file`  | `url` or `data`, `media_type`, `filename` | A `data:` URL, such as a PDF.                                   |
| `text`  | `text`                                    | The words of a voice sample. Needs the role `voice_transcript`. |

- Send `url` or `data`, not both. Bare base64 in `data` needs `media_type`, such as `image/png`.
- `media_type` always means a MIME type, such as `image/png`.
- Send up to 16 entries in all, frames included.
- A link to audio is allowed only on `/v1/audio/transcriptions`. On the other endpoints, send audio as a `data:audio/` URL or as base64.
- An endpoint refuses a field that it does not read, with `400` and the name of the field. It never drops it.

## Roles

`role` is optional. It is `reference` by default.

| `role`             | Entry type | Endpoint                | Meaning                                 |
| ------------------ | ---------- | ----------------------- | --------------------------------------- |
| `reference`        | any        | all of them             | Guidance for the job.                   |
| `first_frame`      | `image`    | `POST /v1/videos`       | The first frame of the video.           |
| `last_frame`       | `image`    | `POST /v1/videos`       | The last frame of the video.            |
| `voice_sample`     | `audio`    | `POST /v1/audio/speech` | A sample of the voice to speak in.      |
| `voice_transcript` | `text`     | `POST /v1/audio/speech` | The words that the voice sample speaks. |

An endpoint refuses a role that it does not read, with `400`. `POST /v1/audio/transcriptions` takes exactly one `audio` entry, or a `file` in a form.

## Frame images

On `POST /v1/videos`, send an image with the role `first_frame` or `last_frame` to fix that frame of the video.

```json
"input_references": [
  { "type": "image", "url": "https://example.com/closed.png", "role": "first_frame" },
  { "type": "image", "url": "https://example.com/open.png", "role": "last_frame" }
]
```

- Send each frame one time at most. Two entries with the same frame and different images get `400`. The same image twice is one frame.
- Only an offer that declared the frame can win. If none did, the exchange refuses the request before the auction. `eligibility.excluded` names the frame, for example `image.supported_frame_images: last_frame`.
- The start image must have the shape of the video. See Start image and frames in [Video generation](https://inference.ai.exchange/docs/video-generation.md).
- Frame images ride the price of the video. They are not charged as reference images.

## Guidance inputs

An entry with no role, or with the role `reference`, is guidance.

```json
"input_references": [
  { "type": "image", "url": "https://example.com/character.png" },
  { "type": "video", "url": "https://example.com/motion.mp4" }
]
```

To continue or extend a finished video, do not send it as a reference. Send `previous_job_id`. See [Video generation](https://inference.ai.exchange/docs/video-generation.md#continue-a-video).

## Voice samples

On `POST /v1/audio/speech`, send a sample of a voice and, if you have it, the words of the sample.

```json
"input_references": [
  { "type": "audio", "data": "<base64>", "media_type": "audio/wav", "role": "voice_sample" },
  { "type": "text", "text": "The words of the sample.", "role": "voice_transcript" }
]
```

A voice sample belongs to one voice. Send it with a string or with one turn, not with many turns.

## Which offers can win

Only an offer that declared each input kind can win. The kinds are `input.image`, `input.audio`, `input.video` and `input.file`. The exchange also checks the limits that the offer declared, such as `image.max_references` and `audio.max_bytes`.

If no offer takes all of your references:

- The exchange refuses the request before the auction with `400 unsupported_parameter`. `param` names the kind, such as `input.image`.
- `eligibility.excluded` gives the reason for each offer.
- The exchange never sends the request to a chat model.

## Charge

- A reference image is charged for each image at the rate of the offer. Without a rate it is free.
- A reference video and a reference audio are charged for each second, at most 30 seconds. A job with one goes only to an offer that prices it.
- A frame image and the start image are not charged as reference images.

See [Modalities](https://inference.ai.exchange/docs/modalities.md) for the cap.

## Edit images and video forms

- `POST /v1/images/edits` is the same purchase for the OpenAI SDK. It needs at least one reference, takes files in `image`, and refuses a `mask`.
- `POST /v1/videos` takes a start image as a file in a form, as the OpenAI SDK sends it.

If no image offer reads your references, the error says so. The Playground then asks a chat model to describe the images, sends the description to the image model, and tells you that it did. The API does not do this.

## In the Playground

The Image, Speech, Video and Compare prompt boxes take files. Drag a file onto the box, paste it, or use the paperclip button.

### What a box accepts

- An image, speech or video job takes images, WAV, FLAC or MP3 audio, and PDF files as references.
- A chat job takes the same files in the message.
- A speech run with a transcription model takes one audio clip instead.
- The Playground sends up to 8 files, with at most 8 MB for each file and 14 MB in total.
- The Playground refuses a file that the exchange cannot read, such as a video, and it says what the job takes.

### Links in the prompt

The Playground sends the prompt as text, so a link to an image in the prompt does not become a reference. If the prompt has a link to a PNG, JPEG, WebP or GIF file, the Playground tells you to download the image and attach it as a file.

### When no model reads a file

If no model of a lane reads a file, the Playground marks the file and stops the run. Remove the file, or choose a model that reads it.

The price estimate for an image counts only the offers that read your files. A speech or video job has no estimate.

### Video start frame

In a video run, the first image is the start frame. The Size menu sets the size of the video.

If the start frame does not have the shape of the video, the Playground marks it and stops the run. Select **Edit** on the image to crop it to the shape, or to keep the whole image with black bars. The editor gives a JPEG that is no larger than the size of the video. It works in your browser and sends the image nowhere.

### Model page Playground

The Playground on the page of an image model has a **Reference images** field. Drop, paste or upload images, up to the limit that the offers of the model declare. The field is absent when no offer reads images.

The form also shows each control that an offer declared: the number of images, format, quality, background, compression, seed and streaming. A control that no offer declared is not shown. The JSON view shows the request body.

### Saved files

A run keeps its files in this browser. The run header lists them. Run again, Retry and a variation send them again.

## Next steps

- [Image generation](https://inference.ai.exchange/docs/image-generation.md)
- [Speech](https://inference.ai.exchange/docs/speech.md)
- [Video generation](https://inference.ai.exchange/docs/video-generation.md)