> ## Documentation Index
> Fetch the complete documentation index at: https://docs.restate.dev/llms.txt
> Use this file to discover all available pages before exploring further.
## Submitting Feedback
If you encounter incorrect, outdated, or confusing documentation on this page, submit feedback:
POST https://docs.restate.dev/feedback
```json
{
"path": "/ai/patterns/error-handling",
"feedback": "Description of the issue"
}
```
Only submit feedback when you have something specific and actionable to report.
# Retries & Error Handling
> Implement robust error handling and retry strategies for reliable agents.
export const GitHubLink = ({url}) =>
;
};
LLM calls are costly, so you want to configure retry behavior to avoid infinite loops and high costs while still recovering from transient failures.
Restate distinguishes between two types of errors:
* **Transient errors**: Temporary issues like network failures or rate limits. Restate automatically retries these until they succeed or the retry policy is exhausted.
* **Terminal errors**: Permanent failures like invalid input or business rule violations. Restate does not retry these. The invocation fails permanently. You can catch these errors and handle them gracefully.
## Retrying LLM calls
LLM API calls fail transiently (rate limits, network issues, provider outages). Configure retry limits to handle this automatically and prevent runaway costs.
In the Vercel AI SDK, set `maxRetries` on `generateText` (default: 2) to retry failed calls due to rate limits or transient errors.
After retries are exhausted, the agent throws an error.
Restate then retries the invocation with exponential backoff to handle longer outages or network issues.
You can limit Restate's retries with the `maxRetryAttempts` option in `durableCalls` middleware:
```typescript errorhandling/fail-on-terminal-tool-agent.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/vercel-ai/tour-of-agents/src/errorhandling/fail-on-terminal-tool-agent.ts#max_attempts_example"} theme={null}
const model = wrapLanguageModel({
model: openai("gpt-4o"),
middleware: durableCalls(ctx, { maxRetryAttempts: 3 }),
});
```
Each Restate retry triggers up to `maxRetries` SDK attempts.
For example, with `maxRetryAttempts`: 3 and `maxRetries`: 2, a call may be attempted 6 times.
Once Restate's retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further.
Restate's `DurableRunner` lets you specify the retry behavior for LLM calls:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/openai-agents/tour-of-agents/app/error_handling.py#handle"} theme={null}
try:
result = await DurableRunner.run(
agent,
req.message,
llm_retry_opts=LlmRetryOpts(
max_attempts=3, initial_retry_interval=timedelta(seconds=2)
),
)
except restate.TerminalError as e:
# Handle terminal errors gracefully
return f"The agent couldn't complete the request: {e.message}"
```
By default, the runner retries ten times with an initial interval of one second. Once Restate's retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further.
Configure the number of retries for LLM calls when activating the Restate plugin for your ADK App:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/google-adk/tour-of-agents/app/error_handling.py#retries"} theme={null}
app = App(
name=APP_NAME, root_agent=agent, plugins=[RestatePlugin(max_model_call_retries=3)]
)
```
By default, the runner retries ten times with an initial interval of one second. Once Restate's retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further.
Restate's `RestateAgent` lets you specify the retry behavior for LLM calls via `RunOptions`:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/pydantic-ai/tour-of-agents/app/error_handling.py#retries"} theme={null}
restate_agent = RestateAgent(
agent,
run_options=RunOptions(max_attempts=3, initial_retry_interval=timedelta(seconds=2)),
)
```
By default, the runner retries ten times with an initial interval of one second. Once Restate's retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further.
Wrap LLM calls in `ctx.run()` with a retry limit to handle transient failures automatically:
```typescript theme={null}
// Retries up to 3 times with exponential backoff
const result = await ctx.run(
"LLM call",
async () => llmCall(messages, tools),
{ maxRetryAttempts: 3 }
);
```
Without `maxRetryAttempts`, Restate retries indefinitely with exponential backoff. For LLM calls, setting a limit prevents runaway costs from persistent failures.
You can set [custom retry policies](/guides/error-handling#at-the-run-block-level) for `ctx.run` steps.
Wrap LLM calls in `ctx.run_typed()` with a retry limit to handle transient failures automatically:
```python theme={null}
# Retries up to 3 times with exponential backoff
result = await ctx.run_typed(
"LLM call",
llm_call,
RunOptions(max_attempts=3),
messages=messages,
tools=tools,
)
```
Without `max_attempts`, Restate retries indefinitely with exponential backoff. For LLM calls, setting a limit prevents runaway costs from persistent failures.
You can set [custom retry policies](/guides/error-handling#at-the-run-block-level) for `.run` actions.
## Tool execution errors
When agent tools use Restate Context actions like `ctx.run`, Restate automatically retries transient errors in these operations. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions.
For errors that should not be retried, throw a terminal error:
```typescript {"CODE_LOAD::ts/src/tour/agents/terminal_error.ts#terminal_error"} theme={null}
throw new TerminalError("This tool is not allowed to run for this input.");
```
By default, the Vercel AI will convert any errors in tool executions into a message to the LLM, and the agent will decide how to proceed.
This is often desirable, as the LLM can decide to use a different tool or provide a fallback answer.
However, if you use Restate Context actions like `ctx.run` in your tool execution, Restate will retry any transient errors in these actions until they succeed.
```typescript {"CODE_LOAD::ts/src/tour/agents/inline-tool-errors.ts#here"} theme={null}
// Without ctx.run - error goes straight to agent
async function myTool() {
const result = await fetch("/api/data"); // Might fail due to network
// If this fails, agent gets the error immediately
}
// With ctx.run - Restate handles retries
async function myToolWithRestate(ctx: restate.Context) {
const result = await ctx.run("fetch-data", () => fetch("/api/data"));
// Network failures get retried automatically
// Only terminal errors reach the AI
}
```
Terminal errors thrown from Restate Context actions are not retried by Restate, and get processed by the Vercel AI.
Also here, the Vercel AI will convert the error into a message to the LLM, and the agent will decide how to proceed.
In some cases, you might want to treat terminal tool execution errors as permanent failures and stop the agent instead of letting the LLM decide how to proceed.
The Restate middleware provides two utilities to help with this:
**To fail the agent on terminal tool errors**, rethrow the error in `onStepFinish`:
```typescript errorhandling/fail-on-terminal-tool-agent.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/vercel-ai/tour-of-agents/src/errorhandling/fail-on-terminal-tool-agent.ts#option2"} theme={null}
const { text } = await generateText({
model,
tools: {
getWeather: tool({
description: "Get the current weather for a given city.",
inputSchema: z.object({ city: z.string() }),
execute: async ({ city }) => {
return await ctx.run("get weather", () => fetchWeather(city));
},
}),
},
stopWhen: [stepCountIs(5)],
onStepFinish: rethrowTerminalToolError,
system: "You are a helpful agent that provides weather updates.",
messages: [{ role: "user", content: prompt }],
});
```
To stop the agent on terminal tool errors and handle it after the agent finishes, you can use `hasTerminalToolError` in `stopWhen` and then inspect the steps for errors:
```typescript errorhandling/stop-on-terminal-tool-agent.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/vercel-ai/tour-of-agents/src/errorhandling/stop-on-terminal-tool-agent.ts#option3"} theme={null}
const { steps, text } = await generateText({
model,
tools: {
getWeather: tool({
description: "Get the current weather for a given city.",
inputSchema: z.object({ city: z.string() }),
execute: async ({ city }) => {
return await ctx.run("get weather", () => fetchWeather(city));
},
}),
},
stopWhen: [stepCountIs(5), hasTerminalToolError],
system: "You are a helpful agent that provides weather updates.",
messages: [{ role: "user", content: prompt }],
});
const terminalSteps = getTerminalToolSteps(steps);
if (terminalSteps.length > 0) {
// Do something with the terminal tool error steps
}
```
When agent tools use Restate Context actions like `ctx.run`, Restate automatically retries transient errors in these operations. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions.
For errors that should not be retried, throw a terminal error:
```python theme={null}
from restate import TerminalError
raise TerminalError("This tool is not allowed to run for this input.")
```
By default, the Restate OpenAI integration will raise any terminal errors in tool executions and will let you handle them in your handler.
The OpenAI Agent SDK also allows setting `failure_error_function` to `None`, which will rethrow any error in the agent execution as-is.
Also for example invalid LLM responses (e.g. tool call with invalid arguments or to a tool that doesn't exist).
The error will then lead to Restate retries. Restate will recover the invocation by replaying the journal entries.
This can lead to infinite retries if the error is not transient.
Therefore, be careful when using this option and handle errors appropriately in your agent logic.
You also might want to set [a retry policy at the service or handler level](/services/configuration#how-to-configure) to avoid infinite retries.
When agent tools use Restate Context actions like `ctx.run`, Restate automatically retries transient errors in these operations. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions.
For errors that should not be retried, throw a terminal error:
```python theme={null}
from restate import TerminalError
raise TerminalError("This tool is not allowed to run for this input.")
```
Restate retries tool executions by default until they succeed.
For errors which should not be retried, raise terminal errors from within your tool implementations.
You can catch these terminal errors in your handler and handle them accordingly.
When agent tools use Restate Context actions like `ctx.run`, Restate automatically retries transient errors in these operations. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions.
For example, wrapping a tool call in `restate_context().run_typed()` makes it durable with automatic retries:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/pydantic-ai/tour-of-agents/app/error_handling.py#here"} theme={null}
async def get_weather(city: WeatherRequest) -> WeatherResponse:
"""Get the current weather for a given city."""
return await restate_context().run_typed(
f"Get weather {city}", fetch_weather, req=city
)
```
For errors that should not be retried, raise a terminal error:
```python theme={null}
from restate import TerminalError
raise TerminalError("This tool is not allowed to run for this input.")
```
Restate retries tool executions by default until they succeed.
For errors which should not be retried, raise terminal errors from within your tool implementations.
You can catch these terminal errors in your handler and handle them accordingly:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/pydantic-ai/tour-of-agents/app/error_handling.py#handle"} theme={null}
@agent_service.handler()
async def run(_ctx: restate.Context, req: WeatherPrompt) -> str:
try:
result = await restate_agent.run(req.message)
except TerminalError as e:
# Handle terminal errors gracefully
return f"The agent couldn't complete the request: {e.message}"
return result.output
```
Restate automatically retries transient errors. This makes your tools resilient to network failures, database hiccups, and other temporary issues.
When a tool encounters an unrecoverable error (e.g., resource not found, invalid input, business rule violation), throw a `TerminalError` to stop retries immediately:
```typescript {"CODE_LOAD::ts/src/tour/agents/terminal_error.ts#terminal_error"} theme={null}
throw new TerminalError("This tool is not allowed to run for this input.");
```
You can catch and handle terminal errors in your agent logic if needed.
Restate automatically retries transient errors. This makes your tools resilient to network failures, database hiccups, and other temporary issues.
When a tool encounters an unrecoverable error (e.g., resource not found, invalid input, business rule violation), raise a `TerminalError` to stop retries immediately:
```python theme={null}
from restate import TerminalError
raise TerminalError("This tool is not allowed to run for this input.")
```
You can catch and handle terminal errors in your agent logic if needed.
To learn more about error handling with Restate, consult the [error handling guide](/guides/error-handling).
## Combining with rollback
For multi-step agent workflows where steps have side effects (bookings, payments, emails), combine error handling with [compensation/rollback patterns](/ai/patterns/rollback) to undo completed work when later steps fail.