# Run your agent

This page gives the procedures to start to sell, to change what you sell, and to upgrade your agent without a gap in the book. For a first start, follow [Set up for the first time](#set-up-for-the-first-time). [Quote over the session](https://inference.ai.exchange/docs/provider-quoting.md) gives the frames and their fields.

## Set up for the first time

1. [Apply as a provider](https://inference.ai.exchange/docs/provider.md). Set `declared_endpoint` to the scheme, host and port of your dispatch server. Wait until the status is `ADMITTED`.
2. [Issue a provider key](https://inference.ai.exchange/docs/provider.md). Keep the secret in the secret store of your agent.
3. [Register each deployment](https://inference.ai.exchange/docs/provider-models.md). A new deployment is `UNLISTED`.
4. Configure the agent with these values:

   | Setting          | Value                                                                                                             |
   | ---------------- | ----------------------------------------------------------------------------------------------------------------- |
   | Exchange address | `wss://router.inference.ai.exchange/v1/provider/ws`                                                               |
   | `slug`           | Your current handle.                                                                                              |
   | `api_key`        | The provider key.                                                                                                 |
   | `dispatch_url`   | The address of your dispatch server. The scheme, host and port must match `declared_endpoint`.                    |
   | `capabilities`   | `["handover"]`, if your agent meets the [handover requirements](#enable-the-handover-capability). Otherwise `[]`. |

5. Start the agent. Make sure that these frames arrive on the session:
   1. `login_ack`. For `login_reject`, read its `reason` and repair the login.
   2. `listed`. It names each deployment that you can quote. An `UNLISTED` deployment has `"unverified": true`.
   3. `quote_ack` for each `quote`. For `quote_reject`, do the repair step for its `code`.
6. Keep the deployments reachable and quoted. The exchange measures each deployment, and an operator lists it. See [List a model](https://inference.ai.exchange/docs/provider-models.md).
7. [Set where you are paid](https://inference.ai.exchange/docs/provider-dispatch.md).

To see each step work before you write your own agent, run the reference provider.

## Serve each kind of job

A deployment has one kind. The agent calls a different route of your upstream for each kind. Register the kind with `kind` in [Register deployments](https://inference.ai.exchange/docs/provider-models.md).

| Kind          | `kind`                | The agent calls                       | Charge                | Read                                              |
| ------------- | --------------------- | ------------------------------------- | --------------------- | ------------------------------------------------- |
| Chat          | `chat`                | `/chat/completions`                   | Tokens                | [Serve and get paid](https://inference.ai.exchange/docs/provider-dispatch.md)      |
| Embeddings    | `embedding`           | `/embeddings`                         | Input tokens          | [Serve and get paid](https://inference.ai.exchange/docs/provider-dispatch.md)      |
| Images        | `image_generation`    | `/images/generations`                 | Each image, or tokens | [Sell images](https://inference.ai.exchange/docs/provider-images.md)               |
| Speech        | `audio_speech`        | `/audio/speech`                       | Each character        | [Sell speech](https://inference.ai.exchange/docs/provider-speech.md)               |
| Transcription | `audio_transcription` | `/audio/transcriptions`               | Each second of audio  | [Sell transcription](https://inference.ai.exchange/docs/provider-transcription.md) |
| Video         | `video_generation`    | `/videos`, then a poll and `/content` | Each second of video  | [Sell video](https://inference.ai.exchange/docs/provider-video.md)                 |

Images, speech and embeddings can also use the [OpenAI-compatible path](https://inference.ai.exchange/docs/provider-openai.md), with no agent. Video and transcription need an agent.

## Change what you sell

You do not have to restart the agent to add a deployment, change a price or pause a deployment. To move the dispatch server or rotate the key, [upgrade the agent](#upgrade-the-agent).

| To do this               | Do this                                              |
| ------------------------ | ---------------------------------------------------- |
| Add a deployment         | [Register it](#add-a-deployment) and quote it.       |
| Change a price           | [Send a new `quote`](#change-a-price).               |
| Pause a deployment       | [Pause it](#stop-to-sell-a-deployment).              |
| Move the dispatch server | [Follow the steps below](#move-the-dispatch-server). |
| Rotate the key           | [Follow the steps below](#rotate-the-key).           |

### Add a deployment

1. [Register the deployment](https://inference.ai.exchange/docs/provider-models.md).
2. Wait for a new `listed` frame. The exchange compares the listing of each session about once a minute, and sends `listed` again when it changed.
3. Quote the new deployment.

The exchange also sends `listed` again when an operator lists, suspends or reinstates a deployment. Quote only the deployments in the newest `listed` frame.

### Change a price

Send a new `quote` for the deployment with a higher `revision`. The new terms replace the offer on the book.

### Stop to sell a deployment

[Pause the deployment](https://inference.ai.exchange/docs/provider-models.md). The exchange removes its offer at once and keeps its registration and listing.

### Move the dispatch server

If only the path changes, change `dispatch_url` in the agent configuration. Then [upgrade the agent](#upgrade-the-agent).

If the scheme, host or port changes, do these steps:

1. Change `declared_endpoint` of the provider. See [Apply as a provider](https://inference.ai.exchange/docs/provider.md).
2. [Upgrade the agent](#upgrade-the-agent) to a configuration with the new `dispatch_url`.
3. Keep the old dispatch server until the old agent stops. The exchange sends jobs for the offers of the old session to the old `dispatch_url`.

### Rotate the key

1. Issue a new key.
2. [Upgrade the agent](#upgrade-the-agent) to a configuration with the new key.
3. Revoke the old key after the old agent stops.

Do not revoke the old key first. The exchange closes the session of a revoked key within 10 seconds and removes its offers.

## Upgrade the agent

Use this procedure for a new version of your agent and for a change to its configuration.

### With the handover capability

The running agent and the new agent must both log in with `handover`. If the running agent did not, the new login replaces its session and removes its offers. The first upgrade to an agent with `handover` therefore has a gap in the book.

1. Start the new agent beside the running agent. Use the same handle. Give it a `dispatch_url` that reaches the new agent. See [Two agents and one endpoint](#two-agents-and-one-endpoint).
2. Wait for `login_ack` on the new session. Make sure that it has `"capabilities": ["handover"]`.
3. Let the new agent quote each deployment in its `listed` frame. Each quote takes over the offer of the old session, at any revision.
4. Wait for `superseded` on the old session. The exchange sends it when the new session holds each offer, and then closes the old session.
5. Let the old agent finish its current dispatches.
6. Stop the old agent.

Until step 4, the old session keeps each offer that the new session did not quote. The old agent must serve awards on those offers.

If the old session quotes a deployment that the new session took, it gets `quote_reject` with the code `superseded`.

If the upgrade fails:

- If the new session closes before it takes each offer, the old session is the current session again. The exchange removes the offers that the new session took and sends `listed` on the old session. The old agent must quote each deployment again.
- The new session has no time limit before it takes its first offer. After that, if 30 seconds pass with no more offers taken, the exchange closes the old session and removes the offers that it still holds.

### Without the handover capability

1. Stop the old agent. The exchange removes its offers when the session closes.
2. Start the new agent. Its offers rest when it quotes.

Your offers are absent from the book between the two steps. Do not run two agents without `handover` for one provider. Each login replaces the session of the other agent and removes its offers.

### Two agents and one endpoint

The exchange sends each job to the `dispatch_url` of the session that quoted the offer. During an upgrade, jobs go to both agents.

Both `dispatch_url` values must match the scheme, host and port of `declared_endpoint`. The path can differ. If a job must reach the agent that quoted it, do these steps:

1. Give each agent its own path, for example `https://provider.example.com/blue/dispatch` and `https://provider.example.com/green/dispatch`.
2. Configure your proxy to send each path to its agent.

If any agent can serve any job, both agents can use one `dispatch_url`.

## When the exchange deploys a release

The exchange sends `handover` to each session that logged in with it. Your agent then moves to the new process with no gap in the book. See [Move to a new exchange process](https://inference.ai.exchange/docs/provider-quoting.md#move-to-a-new-exchange-process).

Without `handover`, your offers are absent from the book until the agent connects again and quotes. Connect again at once after the exchange closes the session.

## Enable the handover capability

[Requirements for the handover capability](https://inference.ai.exchange/docs/provider-quoting.md#requirements-for-the-handover-capability) lists what your agent must do before it sends `["handover"]`. The reference provider implements them. Its source shows one way to share the state between two sessions.

### Test the capability

Test a complete takeover:

1. Start your agent. Wait until each deployment has a `quote_ack`.
2. Start a second process of the agent, with its own `dispatch_url`.
3. Make sure that the second session gets `login_ack` with `"capabilities": ["handover"]` and a `quote_ack` for each deployment.
4. Make sure that the first session gets `superseded`, and that the first process does not log in again.

Test a failed takeover:

1. Start your agent. Wait until each deployment has a `quote_ack`.
2. Start a second process of the agent, with its own `dispatch_url`.
3. Stop the second process before the first session gets `superseded`.
4. Make sure that the first session gets `listed` and quotes each deployment again.

## Next steps

- [Quote over the session](https://inference.ai.exchange/docs/provider-quoting.md)
- [Serve the work](https://inference.ai.exchange/docs/provider-dispatch.md)