# Speech Buy spoken audio from text. The endpoint has the shape of the OpenAI speech API, so an unchanged OpenAI client works. The response body is the audio file. ## Quickstart ```bash curl https://router.inference.ai.exchange/v1/audio/speech \ -H "Authorization: Bearer sk-liquid-..." \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4o-mini-tts", "input": "a cat on a bicycle", "voice": "alloy", "response_format": "mp3" }' --output speech.mp3 ``` ```python from openai import OpenAI client = OpenAI(base_url="https://router.inference.ai.exchange/v1", api_key="sk-liquid-...") with client.audio.speech.with_streaming_response.create( model="gpt-4o-mini-tts", voice="alloy", input="a cat on a bicycle" ) as response: response.stream_to_file("speech.mp3") ``` ## Examples Operators of the exchange made example voice samples with each model. The page of a model in the console lists them. The app shows two here, each with its model and its prompt. Press play to hear one. ## Find a model A control works only when an offer for the model declares it. Read the model card before you send a control. - `GET /v1/models?kind=speech` lists the models as cards, with `kind` `audio_speech` and `modality` `speech`. - `parameters` lists `instructions` and `turns` when an offer declared them. A name that is not a key is not supported, so omit it. - `pricing` gives the price of one character. - `GET /v1/exchange/models?kind=speech` needs no key. The `audio` entry of `output_modalities` lists the `voices` and `formats` that you can send as `voice` and `response_format`, as the union across the offers. `rate_per_character_usd` is the price in USD for each character. See [Modalities](https://inference.ai.exchange/docs/modalities.md#read-a-model-card) for the card and the other filters. ## Controls A control works only when an offer declares it: `speech.instructions` for a style line and `speech.turns` for more than one speaker. `voice` is matched against the voices that an offer registered. ### Voice `voice` is optional. An offer that declared its voices must list the voice that you name. A request for a voice that no offer lists gets no offer. Leave `voice` out to get the first voice of an offer that declared its voices. An offer that declared no voices cannot win a request that names none. ### Style Send `instructions` to set the style of the clip. Only an offer that declared `speech.instructions` can win. ```json { "model": "", "input": "Welcome back.", "instructions": "warm, slow", "voice": "alloy" } ``` ### Many speakers Send `input` as an array of turns for a clip with more than one speaker. Each turn has its own `voice` and can have its own `instructions`. A turn with no `voice` uses the `voice` of the request. A turn with no `instructions` uses the `instructions` of the request. ```json { "model": "", "input": [ { "text": "Hello, Jane.", "voice": "Kore" }, { "text": "Hi.", "voice": "Puck", "instructions": "tired" } ], "instructions": "cheerful" } ``` - An array holds from 1 to 64 turns. An array with one turn is the same as a string. - An array with more than one turn reaches only an offer that declared `speech.turns`. - A voice sample in `input_references` belongs to one voice. Send it with a string or with one turn. ## Response The body is the audio. - The `exchange` block comes back as JSON in the `x-liquid-exchange` response header, next to `x-liquid-cap-usd` and `x-liquid-job-id`. - It reports `characters`, so you can compare the characters that you paid for with the characters that you sent. - There is no streaming form. The exchange sends each clip complete. ## Billing The price is for each character of the text of all the turns. The exchange counts characters, not bytes, so accented and non-Latin text does not cost more for each character. It does not count voice names or `instructions`. The cap is exactly the charge. ## Request parameters | Field | Type | Required | Description | | ------------------ | --------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | `model` | string | Yes | The model to buy, or a preset of the speech kind. | | `input` | string or array | Yes | The text to speak, or an array of turns `{text, voice?, instructions?}` for more than one speaker. At most 4,096 characters in all. | | `input_references` | array | No | A sample of a voice (role `voice_sample`) and its words (a `text` entry with the role `voice_transcript`). See [Reference inputs](https://inference.ai.exchange/docs/reference-inputs.md). | | `voice` | string | No | The voice. See Voice. | | `instructions` | string | No | A style line, such as `cheerful`. At most 1,000 characters. | | `response_format` | string | No | `mp3`, `opus`, `aac`, `flac`, `wav` or `pcm`. The default is `mp3`. | | `speed` | number | No | From 0.25 to 4.0. It changes the length of the audio, not the cost. | | `quality` | string | No | A quality name such as `hd`. A deployment that prices that name apart charges its rate. Otherwise the base rate applies. | | `exchange` | object | No | Purchasing rules, such as `cap_usd`. See [Purchasing rules](https://inference.ai.exchange/docs/rules.md). | ## Errors | Error | Cause | Fix | | --------------------------- | ------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | | `400 unsupported_model` | The model is not a speech model. | Read `/v1/models` for `kind: audio_speech`. | | `400 unsupported_parameter` | No offer serves your `voice` or format, or none declared `speech.instructions` or `speech.turns`. `param` names it. | Read `parameters` on the model card, or `eligibility.excluded`. | ## Speech in a chat answer A chat model that speaks can answer with text and speech in one request. This is a different purchase from `/v1/audio/speech`: it is a chat job priced in tokens. Send `modalities` with `"audio"` and an `audio` object. This is the OpenAI Chat Completions form. ```json { "model": "", "stream": true, "modalities": ["text", "audio"], "audio": { "voice": "alloy", "format": "pcm16" }, "messages": [{ "role": "user", "content": "Say hello" }] } ``` Each stream chunk can hold `choices[0].delta.audio`. The first chunk has an `id`. A chunk has `data` (base64 audio), `transcript` (the text that the audio says), or both. ### Which offers can win - Only an offer that declared `output.audio` can win. If no offer did, the exchange refuses the request and names `output.audio`. - The request needs a `voice` and a `format`. An offer that declared its voices or formats wins only a request for one of them. ### The answer - With `stream: true`, the audio comes as `delta.audio` chunks. - Without it, the answer is one message. `message.audio` holds `id`, `data`, `expires_at` and `transcript`, and `content` is `null`. The router holds at most 32 MiB of audio for one answer. ### Continue the talk Send the earlier assistant turn with `"audio": {"id": "…"}`. The exchange keeps no audio, so the upstream that made the clip resolves the `id`. - This works only on that upstream, and only until it forgets the clip. - The `expires_at` of a buffered answer is the time of the answer, not the upstream's limit. - If the upstream refuses the `id`, send the transcript as the text of the turn. ### Cost - The provider counts the audio tokens apart from the text tokens. Audio tokens are a part of `completion_tokens`. - The exchange charges audio tokens at the audio rate of the offer, and other output tokens at the text rate. - The price cap uses the higher of the two rates for every output token that the request allows. So the cap holds for any mix of text and audio. - The charge uses the mix that the provider reports. The unused part of the cap returns to your balance. ### Failure The exchange can change to the next offer only before the first chunk. After the first chunk, a failure ends the job, and you pay for the output that you received. ## Next steps - [Audio input](https://inference.ai.exchange/docs/audio-input.md): send audio to a chat model. - [Modalities](https://inference.ai.exchange/docs/modalities.md): the map of all modalities.