Skip to main content
Microservice orchestration is about coordinating multiple services to complete complex business workflows. Restate provides powerful primitives for building resilient, observable orchestration patterns. In this guide, you’ll learn how to:
  • Build durable, fault-tolerant service orchestrations with automatic failure recovery
  • Implement sagas for distributed transactions with resilient compensation
  • Use durable timers and external events for complex async patterns
  • Implement stateful entities with Virtual Objects

Getting Started

A Restate application is composed of two main components:
  • Restate Server: The core engine that manages durable execution and orchestrates services. It acts as a message broker or reverse proxy in front of your services.
  • Your Services: Your business logic, implemented as service handlers using the Restate SDK to perform durable operations.
Application Structure A basic subscription service orchestration looks like this:
src/getstarted/service.ts
export const subscriptionService = restate.service({
  name: "SubscriptionService",
  handlers: {
    add: async (ctx: Context, req: SubscriptionRequest) => {
      const paymentId = ctx.rand.uuidv4();

      const payRef = await ctx.run("pay", () =>
        createRecurringPayment(req.creditCard, paymentId),
      );

      for (const subscription of req.subscriptions) {
        await ctx.run(`add-${subscription}`, () =>
          createSubscription(req.userId, subscription, payRef),
        );
      }
    },
  },
});
A service has handlers that can be called over HTTP. Each handler receives a Context object that provides durable execution primitives. Any action performed with the Context is automatically recorded and can survive failures. You don’t need to run your services in any special way. Restate works with how you already deploy your code, whether that’s in Docker, on Kubernetes, or via AWS Lambda.
The endpoint that serves the services of this tour over HTTP is defined in src/app.ts.

Run the example

Install Restate and launch it:
restate-server
Get the example:
restate example typescript-tour-of-orchestration && cd typescript-tour-of-orchestration
npm install
Run the example:
npm run dev
Then, tell Restate where your services are running via the UI (http://localhost:9070) or CLI:
restate deployments register http://localhost:9080
This registers a set of services that we will be covering in this tutorial.To invoke a handler, send a request to restate-ingress/MyServiceName/handlerName:
curl localhost:8080/restate/call/SubscriptionService/add \
--json '{"userId": "user-123", "creditCard": "4111111111111111", "subscriptions": ["Hulu", "Prime"]}'
Click in the UI’s invocations tab on the inovcation ID of your request to see the execution trace of your request.
Invocation overview

Durable Execution

Restate uses Durable Execution to ensure your orchestration logic survives failures and restarts. Whenever a handler executes an action with the Restate Context, this gets send over to the Restate Server and persisted in a log. On a failure or a crash, the Restate Server sends a retry request that contains the log of the actions that were executed so far. The service then replays the log to restore state and continues executing the remaining actions. This process continues until the handler runs till completion. Context in Restate Key Benefits:
  • Context run actions make external calls or non-deterministic operations durable. They get replayed on failures.
  • If the service crashes after payment creation, it resumes at the subscription step
  • Deterministic IDs logged with the context ensure operations are idempotent
  • Full execution traces for debugging and monitoring
Try to add a subscription for Netflix:
curl localhost:8080/restate/call/SubscriptionService/add \
--json '{"userId": "user-123", "creditCard": "4111111111111111", "subscriptions": ["Hulu", "Prime", "Netflix"]}'
On the invocation page in the UI, you can see that your request is retrying because the Netflix API is down:
Invocation overview
To fix the problem, remove the line failOnNetflix from the createSubscription function in the utils.ts file:
export function createSubscription(
  userId: string,
  subscription: string,
  _paymentRef: string,
): string {
  failOnNetflix(subscription);
  terminalErrorOnDisney(subscription);
  console.log(`>>> Created subscription ${subscription} for user ${userId}`);
  return "SUCCESS";
}
Once you restart the service, the workflow finishes successfully:
Invocation overview

Error Handling

By default, Restate retries failures infinitely with an exponential backoff strategy. For some failures, you might not want to retry or only retry a limited number of times. For these cases, Restate distinguishes between two types of errors:
  • Transient Errors: These are temporary issues that can be retried, such as network timeouts or service unavailability. Restate automatically retries these errors.
  • Terminal Errors: These indicate a failure that will not be retried, such as invalid input or business logic violations. Restate stops execution and allows you to handle these errors gracefully.
Throw a terminal error in your handler to indicate a terminal failure:
throw new TerminalError("Invalid credit card");
Some actions let you configure their retry behavior, for example to limit the number of retries of a run block:
const retryPolicy = {
  maxRetryAttempts: 3,
  initialRetryIntervalMillis: 1000,
};

const payRef = await ctx.run(
  "pay",
  () => createRecurringPayment(req.creditCard, paymentId),
  retryPolicy
);
When the retries are exhausted, the run block will throw a TerminalError, that you can handle in your handler logic.
Learn more with the Error Handling Guide.

Sagas and Rollback

On a terminal failure, Restate stops the execution of the handler. You might, however, want to roll back the changes made by the workflow to keep your system in a consistent state. This is where Sagas come in. Sagas are a pattern for rolling back changes made by a handler when it fails. In Restate, you can implement a saga by building a list of compensating actions for each step of the workflow. On a terminal failure, you execute them in reverse order:
src/sagas/service.ts
export const subscriptionSaga = restate.service({
  name: "SubscriptionSaga",
  handlers: {
    add: async (ctx: Context, req: SubscriptionRequest) => {
      const compensations = [];

      try {
        const paymentId = ctx.rand.uuidv4();
        compensations.push(() =>
          ctx.run("undo-pay", () => removeRecurringPayment(paymentId)),
        );
        const payRef = await ctx.run("pay", () =>
          createRecurringPayment(req.creditCard, paymentId),
        );

        for (const subscription of req.subscriptions) {
          compensations.push(() =>
            ctx.run(`undo-${subscription}`, () =>
              removeSubscription(req.userId, subscription),
            ),
          );
          await ctx.run(`add-${subscription}`, () =>
            createSubscription(req.userId, subscription, payRef),
          );
        }
      } catch (e) {
        if (e instanceof restate.TerminalError) {
          for (const compensation of compensations.reverse()) {
            await compensation();
          }
        }
        throw e;
      }
    },
  },
});
Benefits with Restate:
  • The list of compensations can be recovered after a crash, and Restate knows which compensations still need to be run.
  • Sagas always run till completion (success or complete rollback)
  • Full trace of all operations and compensations
  • No complex state machines needed
Add a subscription for Disney:
curl localhost:8080/restate/call/SubscriptionSaga/add \
--json '{"userId": "user-123", "creditCard": "4111111111111111", "subscriptions": ["Hulu", "Prime", "Disney"]}'
The Disney subscription is not available, so the handler will fail and run compensations:
Sagas
Learn more with the Sagas Guide.

Virtual Objects

Until now, the services we looked at did not share any state between requests. To implement stateful entities like shopping carts, user profiles, or AI agents, Restate provides Virtual Objects. Each Virtual Object instance maintains isolated state and is identified by a unique key. Here is an example of a Virtual Object that tracks user subscriptions: Objects
src/objects/service.ts
export const userSubscriptions = restate.object({
  name: "UserSubscriptions",
  handlers: {
    add: async (ctx: ObjectContext, subscription: string) => {
      // Get current subscriptions
      const subscriptions = (await ctx.get<string[]>("subscriptions")) ?? [];

      // Add new subscription
      if (!subscriptions.includes(subscription)) {
        subscriptions.push(subscription);
      }
      ctx.set("subscriptions", subscriptions);

      // Update metrics
      ctx.set("lastUpdated", await ctx.date.toJSON());
    },

    getSubscriptions: restate.handlers.object.shared(
      async (ctx: ObjectSharedContext) => {
        return (await ctx.get<string[]>("subscriptions")) ?? [];
      },
    ),
  },
});
Virtual Objects are ideal for implementing any entity with mutable state:
  • Long-lived state: K/V state is stored permanently. It has no automatic expiry. Clear it via ctx.clear().
  • Durable state changes: State changes are logged with Durable Execution, so they survive failures and are consistent with code execution
  • State is queryable via the state tab in the UI:
State
  • Built-in concurrency control: Restate’s Virtual Objects have built-in queuing and consistency guarantees per object key. Handlers either have read-write access (ObjectContext) or read-only access (shared object context).
  • Only one handler with write access can run at a time per object key to prevent concurrent/lost writes or race conditions.
  • Handlers with read-only access can run concurrently to the write-access handlers.
Queue
Add a few subscriptions for some users. To call a Virtual Object, you specify the object key in the URL (here user-123 and user-456):
curl localhost:8080/restate/call/UserSubscriptions/user-123/add --json '"Hulu"'
curl localhost:8080/restate/call/UserSubscriptions/user-123/add --json '"Prime"'
curl localhost:8080/restate/call/UserSubscriptions/user-123/add --json '"Disney"'
curl localhost:8080/restate/call/UserSubscriptions/user-456/add --json '"Netflix"'
Get the subscriptions for user-123:
curl localhost:8080/restate/call/UserSubscriptions/user-123/getSubscriptions
Or use the UI’s state tab to explore the object state.

Resilient Communication

The Restate SDK includes clients to call other handlers reliably. You can call another handler in three ways:
  • Request-Response: Wait for a response
  • One-Way Messages: Fire-and-forget
  • Delayed Messages: Schedule for later
When you call another handler, the Restate Server acts as a message broker. All communication is proxied via the Restate Server where it gets durably logged and retried till completion. Imagine a handler which processes a concert ticket purchase, and calls multiple services to handle payment, ticket delivery, and reminders:
src/communication/service.ts
export const concertTicketingService = restate.service({
  name: "ConcertTicketingService",
  handlers: {
    buy: async (ctx: Context, req: PurchaseTicketRequest) => {
      // Request-response call - wait for payment to complete
      const payRef = await ctx.serviceClient(paymentService).charge(req);

      // One-way message - fire and forget ticket delivery
      ctx.serviceSendClient(emailService).emailTicket(req);

      // Delayed message - schedule reminder for day before concert
      ctx
        .serviceSendClient(emailService)
        .sendReminder(req, sendOpts({ delay: dayBefore(req.concertDate) }));

      return `Ticket purchased successfully with payment reference: ${payRef}`;
    },
  },
});
Each of these calls gets persisted in Restate’s log and will be retried upon failures. The handler can finish execution without waiting for the ticket delivery or reminder to complete. You can use Restate’s communication primitives to implement microservices that communicate reliably and scale independently.
Buy a concert ticket:
curl localhost:8080/restate/call/ConcertTicketingService/buy --json '{
"ticketId": "ticket-789",
"price": 100,
"customerEmail": "[email protected]",
"concertDate": "2026-10-01T20:00:00Z"
}'
See in the UI how the first call had the response logged, while the ticket delivery happened asynchronously and the reminder was scheduled for in 406 days:
Communication

Request Idempotency

Restate allows adding an idempotency header to your requests. It will then deduplicate requests with the same idempotency key, ensuring that they only execute once. This can help us prevent duplicate calls the concert ticketing service if the user accidentally clicks “buy” multiple times. Add an idempotency header to your request:
curl -X POST localhost:8080/restate/call/ConcertTicketingService/buy \
-H 'Idempotency-Key: unique-key-123' \
--json '{"ticketId": "ticket-789", "price": 100, "customerEmail": "[email protected]", "concertDate": "2023-10-01T20:00:00Z"}'
Notice how doing the same request with the same idempotency key will print the same payment reference. Instead of executing the handler again, Restate returns the result of the first execution.

External Events

Until now we showed either synchronous API calls via run or calls to other Restate services. Another common scenario is APIs that respond asynchronously via webhooks or callbacks. For this, you can use Restate’s awakeables. For example, some payment providers like Stripe require you to initiate a payment and then wait for their webhook to confirm the transaction.
src/events/service.ts
export const payments = restate.service({
  name: "Payments",
  handlers: {
    process: async (ctx: Context, req: PaymentRequest) => {
      // Create awakeable to wait for webhook payment confirmation
      const confirmation = ctx.awakeable<PaymentResult>();

      // Initiate payment with external provider (Stripe, PayPal, etc.)
      const paymentId = ctx.rand.uuidv4();
      await ctx.run("pay", () => initPayment(req, paymentId, confirmation.id));

      // Wait for external payment provider to call our webhook
      return confirmation.promise;
    },

    // Webhook handler called by external payment provider
    confirm: async (
      ctx: Context,
      confirmation: { id: string; result: PaymentResult },
    ) => {
      // Resolve the awakeable to continue the payment flow
      ctx.resolveAwakeable(confirmation.id, confirmation.result);
    },
  },
});
Awakeables are like promises or futures that can be recovered after a crash. Restate persists the awakeable in its log and can recover it on another process when needed. There is no limit to how long you can persist an awakeable, so you can wait for external events that may take hours, or even months to arrive. You can also use awakeables to implement human-in-the-loop interactions, such as waiting for user input or approvals.
Initiate a payment by calling the process handler. Use the send verb to call the handler without waiting for the response:
curl localhost:8080/restate/send/Payments/process \
--json '{"amount": 100, "currency": "USD", "customerId": "cust-123", "orderId": "order-456"}'
In the UI, you can see that the payment is waiting for confirmation.You can restart the service to see how Restate continues waiting for the payment confirmation.Simulate approving the payment by executing the curl request that was printed in the service logs, similar to:
curl localhost:8080/restate/call/Payments/confirm \
--json '{"id": "sign_1PrDkbECjgdsBmMfEUQyCnioCP-csLbd2AAAAEQ", "result": {"success": true, "transactionId": "txn-123"}}'
You can see in the UI that the payment was processed successfully and the awakeable was resolved:
Awakeables

Durable Timers

Waiting on external events might take a long time, and you might want to add timeouts to operations like this. The Restate SDK offers durable timer implementations that you can use to limit waiting for an action. Restate tracks these timers so they survive crashes and do not restart from the beginning. Let’s extend our payment service to automatically cancel payments that don’t complete within a reasonable time:
src/timers/service.ts
export const paymentsWithTimeout = restate.service({
  name: "PaymentsWithTimeout",
  handlers: {
    process: async (ctx: Context, req: PaymentRequest) => {
      const confirmation = ctx.awakeable<PaymentResult>();

      const paymentId = ctx.rand.uuidv4();
      const payRef = await ctx.run("pay", () =>
        initPayment(req, paymentId, confirmation.id),
      );

      // Race between payment confirmation and timeout
      try {
        return await confirmation.promise.orTimeout({ seconds: 30 });
      } catch (e) {
        if (e instanceof TimeoutError) {
          // Cancel the payment with external provider
          await ctx.run("cancel-payment", () => cancelPayment(payRef));
          return {
            success: false,
            errorMessage: "Payment timeout",
          };
        }
        throw e;
      }
    },

    confirm: async (
      ctx: Context,
      confirmation: { id: string; result: PaymentResult },
    ) => {
      ctx.resolveAwakeable(confirmation.id, confirmation.result);
    },
  },
});
You can also set timeouts for RPC calls or other asynchronous operations with the Restate SDK.
Initiate a payment by calling the process handler. Use the send verb to call the handler without waiting for the response:
curl localhost:8080/restate/send/PaymentsWithTimeout/process \
--json '{"amount": 100, "currency": "USD", "customerId": "cust-123", "orderId": "order-456"}'
Wait for 30 seconds without confirming the payment.Try restarting the service while the payment is waiting for confirmation to see how Restate continues waiting for the timer and the confirmation.In the UI, you can see that the payment times out and cancels the payment:
Timers

Concurrent Tasks

When you are waiting on an awakeable or a timer, you are effectively running concurrent tasks and waiting for one of them to complete. Restate allows more advanced concurrency patterns to run tasks in parallel and wait for their results. Let’s extend our subscription service to process all subscriptions concurrently and handle failures gracefully:
src/concurrenttasks/service.ts
export const parallelSubscriptionService = restate.service({
  name: "ParallelSubscriptionService",
  handlers: {
    add: async (ctx: Context, req: SubscriptionRequest) => {
      const paymentId = ctx.rand.uuidv4();
      const payRef = await ctx.run("pay", () =>
        createRecurringPayment(req.creditCard, paymentId),
      );

      // Start all subscriptions in parallel
      const subscriptionPromises = [];
      for (const subscription of req.subscriptions) {
        subscriptionPromises.push(
          ctx.run(`add-${subscription}`, () =>
            createSubscription(req.userId, subscription, payRef),
          ),
        );
      }

      // Wait for all subscriptions to complete
      await RestatePromise.all(subscriptionPromises);

      return { success: true, paymentRef: payRef };
    },
  },
});
Restate retries all parallel tasks until they all complete and can deterministically replay the order of completion.
Add a few subscriptions for some users.
curl localhost:8080/restate/call/ParallelSubscriptionService/add \
--json '{"userId": "user-123", "creditCard": "4111111111111111", "subscriptions": ["Hulu", "Prime", "YouTube"]}'
In the UI, you can see that all subscriptions are processed in parallel:
Concurrent Tasks
You can extend this to include the saga pattern and run all compensations in parallel as well. Have a look at the Concurrent Tasks docs for your SDK to learn more (TS / Java / Kotlin / Python / Go).

Summary

Restate simplifies microservice orchestration with:
  • Durable Execution: Automatic failure recovery without complex retry logic
  • Sagas: Distributed transactions with resilient compensation
  • Service Communication: Reliable RPC and messaging between services
  • Stateful Processing: Consistent state management without external stores
  • Advanced Patterns: Fault-tolerant timers, awakeables, and parallel execution
Build resilient distributed systems without the typical complexity.