Price and wait

By default, the exchange does not pick the offer with the lowest charge. It picks the offer with the lowest cost: the estimated charge plus the wait cost. The wait cost is the money value that you give to the time that you wait for the answer.

This applies to the price strategy, which is the default. The cheapest axis uses it too.

How the cost is made

The exchange adds two amounts for each offer that meets your limits:

cost = estimated charge + (predicted wait in ms × wait value)

• The estimated charge is the price that the exchange expects for your request.
• The predicted wait is the time from the start of the job to the end of the answer. It is based on measurements of the provider. See How the exchange measures performance.
• The wait value is the price of one millisecond of wait, in nano-USD (one billionth of a dollar). The default is 200. That is $0.0002 for each second, or $0.72 for each hour.

The offer with the lowest cost wins. The exchange then runs the normal steps: the price cap, the hold, and the charge for the tokens that the provider really used.

Example

Two offers answer the same request. The wait value is the default, 200 nano-USD for each millisecond.

Offer B wins, although its charge is higher. You pay the real charge of offer B, which is never more than its price cap. If you set the wait value to zero, offer A wins.

Change the wait value

Set exchange.rank.latency_nanos_per_ms in a request or in a preset. Use a whole number that is zero or more.

{ "exchange": { "rank": { "latency_nanos_per_ms": 0 } } }

The exchange refuses a value that is not a whole number or that is below zero.

What the wait value changes

The wait value changes only the order of the offers. These rules do not change:

• An offer must still pass every limit, including your price cap.
• The price cap, the hold and the settlement work as before.
• The exchange never shortens your answer or sets your output limit.
• The price_band_bps setting applies to the cost, which includes the wait cost.

When the exchange ranks on price alone

The exchange ranks on the estimated charge alone in these cases:

• The strategy is fastest, smartest or auto. These strategies have their own rules and ignore the wait value.
• The request is not a chat request.
• The wait value is zero.
• One eligible offer has no complete measurement. The exchange then ranks the whole auction on price, so that an offer with no data does not get an advantage.

To see why an offer won, read Explain the choice.

Next steps

• How the exchange measures performance
• Purchasing rules
• Axis routing