Audio input

Send an audio clip to a chat model as a content part. The model answers in text. The exchange does not transcribe the clip for a model that cannot hear. Only an offer that declared audio input can win the job.

{
  "role": "user",
  "content": [
    { "type": "text", "text": "what does the speaker ask for?" },
    { "type": "input_audio", "input_audio": { "data": "UklGR…", "format": "wav" } }
  ]
}

Audio is for Chat Completions only. The Messages API has no audio block.

Examples

Summarise a voice note. Send a short recording and ask for the key points.

{
  "role": "user",
  "content": [
    { "type": "text", "text": "Summarise this voice note in three bullet points." },
    { "type": "input_audio", "input_audio": { "data": "UklGR…", "format": "wav" } }
  ]
}

Answer a spoken question. Send the question as audio with no text. The model answers in text.

{
  "role": "user",
  "content": [{ "type": "input_audio", "input_audio": { "data": "//uQxAAA…", "format": "mp3" } }]
}

To get the answer as audio, send the answer text to Speech.

Send audio

data is bare base64 or a data: URL. The exchange refuses a remote URL with 400 unsupported_content.

Field Meaning
data The bytes of the clip, in base64.
format wav, flac or mp3.

An offer can accept fewer formats.

Cost

The exchange reads the duration from the container header. It converts the duration to input tokens at 50 tokens for each second, unless the offer states its own table. The charge is the token count that the provider meters.

An offer can limit the length with max_duration_seconds, and the size with max_bytes.

Troubleshooting

Error Cause Fix
400 unsupported_parameter No offer for the model hears audio. Pick a model with audio in input_modalities.
400 unsupported_content A URL, or a header that cannot be read. Send the bytes as base64 in a supported container.
413 request_too_large The body is over 20 MB. Send a shorter clip.

To get the words of a clip, buy a transcript. See Speech-to-text.

Next steps