> ## Documentation Index
> Fetch the complete documentation index at: https://docs.restate.dev/llms.txt
> Use this file to discover all available pages before exploring further.
# Racing agents
> Run agents in parallel, return the fastest response and cancel the others.
export const GitHubLink = ({url}) => ;
Execute multiple AI approaches or strategies simultaneously and return the result from whichever completes first successfully. This pattern is ideal when you have multiple ways to solve the same problem and want to minimize latency by racing them against each other.
Useful for:
* Querying multiple AI models (e.g., GPT-4, Claude, Gemini) and returning the fastest response
* Running different agents, prompts or strategies in parallel and using the first successful outcome
## How does Restate help?
The benefits of using Restate for competitive racing patterns are:
* **Durable coordination**: Restate turns Promises/Futures into durable, distributed constructs that persist across failures and process restarts. Race multiple approaches and return the first successful result.
* **Cancel slow tasks**: Failed or slower approaches can be cancelled, preventing resource waste
* **Serverless scaling**: Deploy racing strategies on serverless infrastructure for automatic scaling while the main process remains suspended
* Works with **any LLM SDK** (Vercel AI, LangChain, LiteLLM, etc.) and **any programming language** supported by Restate (TypeScript, Python, Go, etc.).
## Example
Select your preferred SDK:
When you need a quick response and have access to multiple AI models, race them against each other to get the fastest result:
```ts racing-agents.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/typescript-restate-only/tour-of-agents/src/racing-agents.ts#here"} theme={null}
async function run(
ctx: Context,
{ message }: { message: string },
): Promise {
// Start both service calls concurrently
const slowCall = ctx.serviceClient(racingAgent).thinkLonger({ message });
const slowResponse = slowCall.map((res) => ({ tag: "slow", res }));
const fastCall = ctx.serviceClient(racingAgent).respondQuickly({ message });
const fastResponse = fastCall.map((res) => ({ tag: "fast", res }));
const pending = [slowResponse, fastResponse];
// Wait for the first one to complete
const { tag, res } = await RestatePromise.any(pending);
if (tag === "fast") {
console.log("Quick response won the race!");
const slowInvocationId = await slowCall.invocationId;
ctx.cancel(slowInvocationId);
} else {
console.log("Deep analysis won the race!");
const quickInvocationId = await fastCall.invocationId;
ctx.cancel(quickInvocationId);
}
return res ?? "LLM gave no response";
}
```
[Install Restate](/installation) and launch it:
```bash theme={null}
restate-server
```
Get the example:
```bash theme={null}
restate example typescript-restate-tour-of-agents && cd typescript-restate-tour-of-agents
npm install
```
Export your API key:
```bash theme={null}
export OPENAI_API_KEY=sk-...
```
```bash theme={null}
npx tsx ./src/racing-agents.ts
```
Register the services with Restate:
```bash theme={null}
restate deployments register http://localhost:9080 --force --yes # dev only: overrides previous registrations
```
Send a request:
```bash theme={null}
curl localhost:8080/restate/call/RacingAgent/run \
--json '{
"message": "What is the best approach to learn machine learning?"
}'
```
```python racing_agents.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/python-restate-only/tour-of-agents/app/racing_agents.py#here"} theme={null}
racing_agent = Service("RacingAgent")
@racing_agent.handler()
async def run(ctx: Context, query: Question) -> str | None:
"""Run two approaches in parallel and return the fastest response."""
# Start both service calls concurrently
slow_response = ctx.service_call(think_longer, arg=query)
quick_response = ctx.service_call(respond_quickly, arg=query)
done, pending = await restate.wait_completed(slow_response, quick_response)
# cancel the pending calls
for f in pending:
call_future = typing.cast(RestateDurableCallFuture, f)
ctx.cancel_invocation(await call_future.invocation_id())
# return the fastest result
return await done[0]
```
[Install Restate](/installation) and launch it:
```bash theme={null}
restate-server
```
Get the example:
```bash theme={null}
restate example python-restate-tour-of-agents && cd python-restate-tour-of-agents
```
Export your API key:
```bash theme={null}
export OPENAI_API_KEY=sk-...
```
```bash theme={null}
uv run app/racing_agents.py
```
Register the services with Restate:
```bash theme={null}
restate deployments register http://localhost:9080 --force --yes # dev only: overrides previous registrations
```
Send a request:
```bash theme={null}
curl localhost:8080/restate/call/RacingAgent/run \
--json '{
"message": "What is the best approach to learn machine learning?"
}'
```
This pattern is implementable with any of our SDKs and any AI SDK.
If you need help with a specific SDK, please reach out to us via [Discord](https://discord.restate.dev) or [Slack](https://slack.restate.dev).