Run your agent

This page gives the procedures to start to sell, to change what you sell, and to upgrade your agent without a gap in the book. For a first start, follow Set up for the first time. Quote over the session gives the frames and their fields.

Set up for the first time

1. Apply as a provider. Set declared_endpoint to the scheme, host and port of your dispatch server. Wait until the status is ADMITTED.

2. Issue a provider key. Keep the secret in the secret store of your agent.

3. Register each deployment. A new deployment is UNLISTED.

4. Configure the agent with these values:

   Setting	Value
   Exchange address	wss://router.inference.ai.exchange/v1/provider/ws
   slug	Your current handle.
   api_key	The provider key.
   dispatch_url	The address of your dispatch server. The scheme, host and port must match declared_endpoint.
   capabilities	["handover"], if your agent meets the handover requirements. Otherwise [].

5. Start the agent. Make sure that these frames arrive on the session:

   1. login_ack. For login_reject, read its reason and repair the login.
   2. listed. It names each deployment that you can quote. An UNLISTED deployment has "unverified": true.
   3. quote_ack for each quote. For quote_reject, do the repair step for its code.

6. Keep the deployments reachable and quoted. The exchange measures each deployment, and an operator lists it. See List a model.

7. Set where you are paid.

To see each step work before you write your own agent, run the reference provider.

Serve each kind of job

A deployment has one kind. The agent calls a different route of your upstream for each kind. Register the kind with kind in Register deployments.

Images, speech and embeddings can also use the OpenAI-compatible path, with no agent. Video and transcription need an agent.

Change what you sell

You do not have to restart the agent to add a deployment, change a price or pause a deployment. To move the dispatch server or rotate the key, upgrade the agent.

Add a deployment

1. Register the deployment.
2. Wait for a new listed frame. The exchange compares the listing of each session about once a minute, and sends listed again when it changed.
3. Quote the new deployment.

The exchange also sends listed again when an operator lists, suspends or reinstates a deployment. Quote only the deployments in the newest listed frame.

Change a price

Send a new quote for the deployment with a higher revision. The new terms replace the offer on the book.

Stop to sell a deployment

Pause the deployment. The exchange removes its offer at once and keeps its registration and listing.

Move the dispatch server

If only the path changes, change dispatch_url in the agent configuration. Then upgrade the agent.

If the scheme, host or port changes, do these steps:

1. Change declared_endpoint of the provider. See Apply as a provider.
2. Upgrade the agent to a configuration with the new dispatch_url.
3. Keep the old dispatch server until the old agent stops. The exchange sends jobs for the offers of the old session to the old dispatch_url.

Rotate the key

1. Issue a new key.
2. Upgrade the agent to a configuration with the new key.
3. Revoke the old key after the old agent stops.

Do not revoke the old key first. The exchange closes the session of a revoked key within 10 seconds and removes its offers.

Upgrade the agent

Use this procedure for a new version of your agent and for a change to its configuration.

With the handover capability

The running agent and the new agent must both log in with handover. If the running agent did not, the new login replaces its session and removes its offers. The first upgrade to an agent with handover therefore has a gap in the book.

1. Start the new agent beside the running agent. Use the same handle. Give it a dispatch_url that reaches the new agent. See Two agents and one endpoint.
2. Wait for login_ack on the new session. Make sure that it has "capabilities": ["handover"].
3. Let the new agent quote each deployment in its listed frame. Each quote takes over the offer of the old session, at any revision.
4. Wait for superseded on the old session. The exchange sends it when the new session holds each offer, and then closes the old session.
5. Let the old agent finish its current dispatches.
6. Stop the old agent.

Until step 4, the old session keeps each offer that the new session did not quote. The old agent must serve awards on those offers.

If the old session quotes a deployment that the new session took, it gets quote_reject with the code superseded.

If the upgrade fails:

• If the new session closes before it takes each offer, the old session is the current session again. The exchange removes the offers that the new session took and sends listed on the old session. The old agent must quote each deployment again.
• The new session has no time limit before it takes its first offer. After that, if 30 seconds pass with no more offers taken, the exchange closes the old session and removes the offers that it still holds.

Without the handover capability

1. Stop the old agent. The exchange removes its offers when the session closes.
2. Start the new agent. Its offers rest when it quotes.

Your offers are absent from the book between the two steps. Do not run two agents without handover for one provider. Each login replaces the session of the other agent and removes its offers.

Two agents and one endpoint

The exchange sends each job to the dispatch_url of the session that quoted the offer. During an upgrade, jobs go to both agents.

Both dispatch_url values must match the scheme, host and port of declared_endpoint. The path can differ. If a job must reach the agent that quoted it, do these steps:

1. Give each agent its own path, for example https://provider.example.com/blue/dispatch and https://provider.example.com/green/dispatch.
2. Configure your proxy to send each path to its agent.

If any agent can serve any job, both agents can use one dispatch_url.

When the exchange deploys a release

The exchange sends handover to each session that logged in with it. Your agent then moves to the new process with no gap in the book. See Move to a new exchange process.

Without handover, your offers are absent from the book until the agent connects again and quotes. Connect again at once after the exchange closes the session.

Enable the handover capability

Requirements for the handover capability lists what your agent must do before it sends ["handover"]. The reference provider implements them. Its source shows one way to share the state between two sessions.

Test the capability

Test a complete takeover:

1. Start your agent. Wait until each deployment has a quote_ack.
2. Start a second process of the agent, with its own dispatch_url.
3. Make sure that the second session gets login_ack with "capabilities": ["handover"] and a quote_ack for each deployment.
4. Make sure that the first session gets superseded, and that the first process does not log in again.

Test a failed takeover:

1. Start your agent. Wait until each deployment has a quote_ack.
2. Start a second process of the agent, with its own dispatch_url.
3. Stop the second process before the first session gets superseded.
4. Make sure that the first session gets listed and quotes each deployment again.

Next steps

• Quote over the session
• Serve the work