Skip to main content

Tour of Restate

This tutorial guides you through the development of an end-to-end Restate application, and covers all the essential features. After this tutorial, you should have a firm understanding of how Restate can help you and feel comfortable to tackle your next application on your own.

This tutorial implements a ticket reservation application for a theatre. It allows users to add tickets for specific seats to their shopping cart. After the user adds a ticket, the seat gets reserved for 15 minutes. If the user doesn't pay for the ticket within that time interval, the reservation is released and the ticket becomes available to other users.

Restate applications are made up of services that expose handlers. Handlers are functions that implement business logic. Restate manages their invocation and execution. Services communicate with one another using Remote Procedure Calls (RPC). Our ticket example consists of three services:

As we go, you will discover how Restate can help you with some intricacies in this application.

Prerequisites

This guide is written for:

  • TypeScript SDK version: 1.4.0
  • Restate Server Docker image: docker.io/restatedev/restate:1.1

Getting Started​

1
Set up the services

Download the example and run locally with an IDE:

CLI
wget

restate example typescript-tour-of-restate && cd typescript-tour-of-restate

Install the dependencies and build the app:


npm install && npm run build

Run the services


npm run app-dev

This GitHub repository contains the basic skeleton of the TypeScript services that you develop in this tutorial.

2
Launch Restate

Restate is a single self-contained binary. No external dependencies needed. Check out our Local Dev page for instructions on how to install Restate Server, then do:


restate-server

3
Register the services with Restate

Now, we need to tell Restate where our services are running. You can register services by calling the Restate Admin API (default port 9070) and supplying it the service endpoint URI:

CLI
curl

restate deployments register http://localhost:9080

Output
CLI
curl

❯ SERVICES THAT WILL BE ADDED:
- CheckoutService
Type: Service
HANDLER INPUT TYPE OUTPUT TYPE
handle one of "empty or value of content-type */*" value with content-type "application/json"
- CartObject
Type: VirtualObject ⬅️ 🚶🚶🚶
HANDLER INPUT TYPE OUTPUT TYPE
addTicket one of "empty or value of content-type */*" value with content-type "application/json"
expireTicket one of "empty or value of content-type */*" value with content-type "application/json"
checkout one of "empty or value of content-type */*" value with content-type "application/json"
- TicketObject
Type: VirtualObject ⬅️ 🚶🚶🚶
HANDLER INPUT TYPE OUTPUT TYPE
reserve one of "empty or value of content-type */*" value with content-type "application/json"
markAsSold one of "empty or value of content-type */*" value with content-type "application/json"
unreserve one of "empty or value of content-type */*" value with content-type "application/json"
✔ Are you sure you want to apply those changes? · yes
✅ DEPLOYMENT:
SERVICE REV
TicketObject 1
CheckoutService 1
CartObject 1

If you run Restate with Docker, replace http://localhost:9080 by http://host.docker.internal:9080.

🚀
All set up!

In src/app you will find the skeletons of the various services to help you start implementing the app. For example:

checkout_service.ts

export const checkoutService = restate.service({
name: "CheckoutService",
handlers: {
async handle(ctx: restate.Context, request: { userId: string; tickets: string[] }){
return true;
},
}
});
export const CheckoutService: typeof checkoutService = { name: "CheckoutService"};

Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.

The app.ts file contains the definition of the endpoint that hosts the services.

Invoking Handlers​

Implement it yourself or follow along by looking at the code under part1

Handlers can be invoked in several ways: via HTTP requests, programmatically with the SDK, or via Kafka events.

Request-response calls over HTTP​

Let's start with invoking our handler over HTTP using curl.

For example, add a ticket seat2B to the cart of Mary by calling the addTicket handler of the CartObject:


curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2B"'

If this prints out true, then you have a working setup.

When Mary wants to proceed with the purchase, call the checkout handler of the CartObject:


curl -X POST localhost:8080/CartObject/Mary/checkout

We will use these two curl commands often when developing the code, so keep them handy.

Restate as proxy

Restate acts as a proxy for your services. It forwards the request to the correct service and handler. Therefore, the request is sent to Restate and not directly to the service.

Why does the path contain Mary?

Handlers are either a part of plain services or Virtual Objects. Virtual Objects are a special type of service that allows you to group handlers together, share state between them, and control concurrency. Each Virtual Object has a unique key. We will cover the difference in more detail later. For now, it's important to note that when invoking a handler within a Virtual Object, you need to specify its key. In our example, the CartObject and TicketObject are Virtual Objects, while the CheckoutService is a plain service. To add the ticket to Mary's cart, we need to specify the key Mary in the path to reach her Virtual Object.

We can do the same programmatically within a handler by using the SDK. Let's try this out!

Request-response calls between handlers​

You can also call other handlers programmatically by using the clients generated by the Restate SDK. Let's try this out!

When we add a ticket to the cart, the CartObject/addTicket handler first needs to reserve the ticket for the user. It does that by calling the TicketObject/reserve handler:

cart_object.ts

async addTicket(ctx: restate.ObjectContext, ticketId: string) {
const reservationSuccess = await ctx.objectClient(TicketObject, ticketId).reserve();
return true;
},

Service logs

[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.790Z] DEBUG: Invoking function.
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.792Z] DEBUG: Adding message to journal and sending to Restate ; InvokeEntryMessage
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.792Z] DEBUG: Scheduling suspension in 30000 ms
[restate] [TicketObject/reserve][inv_1k78Krj3GqrK6odGaRe866kHZilkVf1H4l][2024-03-18T16:30:28.796Z] DEBUG: Invoking function.
[restate] [TicketObject/reserve][inv_1k78Krj3GqrK6odGaRe866kHZilkVf1H4l][2024-03-18T16:30:28.796Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage
[restate] [TicketObject/reserve][inv_1k78Krj3GqrK6odGaRe866kHZilkVf1H4l][2024-03-18T16:30:28.796Z] DEBUG: Function completed successfully.
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.799Z] DEBUG: Received completion message from Restate, adding to journal. ; CompletionMessage
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.800Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.800Z] DEBUG: Function completed successfully.

  1. Create the client via ctx.serviceClient or ctx.objectClient (for Virtual Objects). Specify the service definition (TicketObject) and optionally the Virtual Object key (ticketId).
  2. Specify the handler you want to call and supply the request. Here reserve().
  3. Await the response of the call.

Send a request to CartObject/addTicket as we did previously, and have a look at the service logs.

Sending messages between handlers​

We can also let handlers send messages to other handlers without waiting for a response.

In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes. When a user didn't proceed with the payment before the timeout, the CartObject/expireTicket handler is triggered. Let the expireTicket handler call the TicketObject/unreserve handler.

cart_object.ts

async expireTicket(ctx: restate.ObjectContext, ticketId: string) {
ctx.objectSendClient(TicketObject, ticketId).unreserve();
},

Specify that you want to call the TicketObject by supplying ticketObjectApi to the send function. Then call the unreserve handler on the TicketObject.

Once you have added this to the code, call the CartObject/expireTicket handler:


curl localhost:8080/CartObject/Mary/expireTicket -H 'content-type: application/json' -d '"seat2B"'

Service logs

[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM942bjcDmb1c1khoaJe11Hbz][2024-03-18T16:14:24.671Z] DEBUG: Invoking function.
[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM942bjcDmb1c1khoaJe11Hbz][2024-03-18T16:14:24.672Z] DEBUG: Adding message to journal and sending to Restate ; BackgroundInvokeEntryMessage
[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM942bjcDmb1c1khoaJe11Hbz][2024-03-18T16:14:24.673Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage
[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM942bjcDmb1c1khoaJe11Hbz][2024-03-18T16:14:24.673Z] DEBUG: Function completed successfully.
[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK3GuJXkgaXBbg69R47TCeAN][2024-03-18T16:14:24.677Z] DEBUG: Invoking function.
[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK3GuJXkgaXBbg69R47TCeAN][2024-03-18T16:14:24.677Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage
[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK3GuJXkgaXBbg69R47TCeAN][2024-03-18T16:14:24.677Z] DEBUG: Function completed successfully.

The service logs show how the expireTicket handler gets executed and then the unreserve handler. The call to expireTicket finishes earlier than the unreserve handler because expireTicket didn't wait for the response of the unreserve handler.

Restate as message queue

Restate persists and retries failed one-way invocations. There is no need to set up message queues to ensure delivery!

Sending messages via `curl`

To send messages via curl, add /send to the handler path:


curl localhost:8080/CartObject/Mary/addTicket/send -H 'content-type: application/json' -d '"seat2B"'

Output

{"invocationId":"inv_1aiqX0vFEFNH1Umgre58JiCLgHfTtztYK5","status":"Accepted"}

This returns the invocation ID. This is a unique identifier for the invocation. You can use it to track the progress of the invocation via the CLI, and to correlate logs and metrics.

📝 Try it out​

Make the CartObject/checkout handler call the CheckoutService/handle handler.

For the request field, you can use a hard-coded string array for now: ["seat2B"]. You will fix this later on. Note that the CheckoutService is not a Virtual Object, so you don't need to specify a key.

Solution

Add the following code to the CartObject/checkout handler:

cart_object.ts

async checkout(ctx: restate.ObjectContext) {
const success = await ctx.serviceClient(CheckoutService)
.handle({userId: ctx.key, tickets: ["seat2B"]});
return success;
},

Call CartObject/checkout as you did earlier and have a look at the logs again to see what happened:


[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.018Z] DEBUG: Invoking function.
[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.019Z] DEBUG: Adding message to journal and sending to Restate ; InvokeEntryMessage
[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.020Z] DEBUG: Scheduling suspension in 30000 ms
[restate] [CheckoutService/handle][inv_16WnWCiCVV5G2gUUevDM4uIli4v7TN9k2d][2024-03-19T07:57:24.023Z] DEBUG: Invoking function.
[restate] [CheckoutService/handle][inv_16WnWCiCVV5G2gUUevDM4uIli4v7TN9k2d][2024-03-19T07:57:24.024Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage
[restate] [CheckoutService/handle][inv_16WnWCiCVV5G2gUUevDM4uIli4v7TN9k2d][2024-03-19T07:57:24.024Z] DEBUG: Function completed successfully.
[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.026Z] DEBUG: Received completion message from Restate, adding to journal. ; CompletionMessage
[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.027Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage
[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.027Z] DEBUG: Function completed successfully.

Durable Execution​

The calls we just did seem like regular RPC calls as you might know them from other service frameworks. But under the hood a lot more is happening.

Restate makes RPC calls resilient by letting the Restate Server and SDK cooperate. Restate tracks the execution of code in a journal and can replay it in case of a failure. This is called Durable Execution.

Have a look at the animation to understand what happened under-the-hood:

Journals

This animation shows you what happened under the hood when we did the reserve call from the `CartObject` to the `TicketObject`. The animation uses the TypeScript SDK.

CartObject[Mary]
suspended...
async function addTicket(ctx, ticketId){
  const success = await ctx
    .objectClient(ticketObject)
    .reserve(ticketId);

  return success;
}
TicketObject[seat2B]
suspended...
async function reserve(ctx, ticketId){
  ...
  return success;
}

Whenever a failure would happen, Restate would be able to recover the latest state of the handler by sending over the journal. The code would fast-forward to the point where it crashed, and continue executing from there on.

To see the recovery of partial progress in practice, let's make the CartObject/addTicket handler crash right after the call.

cart_object.ts

_10
async addTicket(ctx: restate.ObjectContext, ticketId: string) {
_10
const reservationSuccess = await ctx.objectClient(TicketObject, ticketId).reserve();
_10
_10
return true;
_10
},

Add the following code to line 4 of the snippet, to let the code throw an error after the call:


throw new Error("Failing");

Call CartObject/addTicket again and have a look at the service logs.

Service logs

[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.245Z] DEBUG: Adding message to journal and sending to Restate ; InvokeEntryMessage
[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.246Z] DEBUG: Scheduling suspension in 30000 ms
[restate] [TicketObject/reserve][inv_1iGFK6hGrtOf3jcD8PupmCOJz1SDzvfPi1][2024-04-16T13:28:20.296Z] DEBUG: Invoking function.
[restate] [TicketObject/reserve][inv_1iGFK6hGrtOf3jcD8PupmCOJz1SDzvfPi1][2024-04-16T13:28:20.296Z] DEBUG: Journaled and sent output message ; OutputEntryMessage
[restate] [TicketObject/reserve][inv_1iGFK6hGrtOf3jcD8PupmCOJz1SDzvfPi1][2024-04-16T13:28:20.296Z] DEBUG: Function completed successfully.
[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.362Z] DEBUG: Received completion message from Restate, adding to journal.
Trace: [restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.363Z] TRACE: Function completed with an error: Failing Error: Failing
... rest of trace ...
[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.372Z] DEBUG: Invocation ended with retryable error. ; ErrorMessage
[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.437Z] DEBUG: Resuming (replaying) function.
[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.437Z] DEBUG: Matched and replayed message from journal ; InvokeEntryMessage
Trace: [restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.437Z] TRACE: Function completed with an error: Failing Error: Failing
... rest of trace ...

You see the retries taking place. And you see that only the first time the call to the CheckoutService was made. The other times, the call was skipped and the journaled response was replayed.

By default, Restate will keep retrying failed invocations until they succeed. If you want to cancel an invocation in a retry loop, you can use the CLI to do this. Let's have a look at that next.

Debugging with the CLI​

Now that we have a failing invocation, let's take the opportunity to show you how you can get more information about what is going on with the CLI. The CLI is a management tool that lets you interact with the Restate Server. You can use it to boostrap a new project, but also to get information about the services and invocations.

Have a look at some useful commands and try them out yourself:

terminal
$ restate services list
     NAME             REVISION  FLAVOR    DEPLOYMENT TYPE  DEPLOYMENT ID
 🌎  CartObject       1         ⬅️ 🚶🚶🚶  HTTP 2           dp_11pXug0mWsff2NOoRBZbOcV
 🌎  CheckoutService  1                   HTTP 2           dp_11pXug0mWsff2NOoRBZbOcV
 🌎  TicketObject     1         ⬅️ 🚶🚶🚶  HTTP 2           dp_11pXug0mWsff2NOoRBZbOcV


note
Remove the throwing of the exception from your code before you continue.


info
🚩 Explore the intermediate solution in part1, and run it with:

npm run part1



Scheduling Async Tasks​


Implement it yourself or follow along by looking at the code under part2


When a handler calls another handler, Restate registers the call and makes sure it happens.
You can also ask Restate to execute the call at a later point in the future, by adding a delay parameter to the call.
Restate then registers the call and triggers it after the delay has passed.


In the application, a ticket gets reserved for 15 minutes.
If the user doesn't pay within that time interval, then it becomes available again to other users.


Let the CartObject/addTicket handler call the CartObject/expireTicket handler with a delay of 15 minutes:
cart_object.ts

async addTicket(ctx: restate.ObjectContext, ticketId: string) {
  const reservationSuccess = await ctx.objectClient(TicketObject, ticketId).reserve();
  if (reservationSuccess) {
    ctx.objectSendClient(CartObject, ctx.key, {delay: 15 * 60 * 1000})
        .expireTicket(ticketId);
  }
  return reservationSuccess;
},

To test it out, put the delay to a lower value, for example 5 seconds, call the addTicket function, and see in the logs how the call to CartObject/expireTicket is executed 5 seconds later.
Service logs

... logs from reserve call ...
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM90xbqbDEnOzNgilf2WmjZTP][2024-03-19T08:49:43.081Z] DEBUG: Received completion message from Restate, adding to journal. ; CompletionMessage
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM90xbqbDEnOzNgilf2WmjZTP][2024-03-19T08:49:43.081Z] DEBUG: Adding message to journal and sending to Restate ; BackgroundInvokeEntryMessage
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM90xbqbDEnOzNgilf2WmjZTP][2024-03-19T08:49:43.081Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM90xbqbDEnOzNgilf2WmjZTP][2024-03-19T08:49:43.081Z] DEBUG: Function completed successfully.
[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM93r8tce9IfwnbiAsk8lCevD][2024-03-19T08:49:48.092Z] DEBUG: Invoking function.
[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM93r8tce9IfwnbiAsk8lCevD][2024-03-19T08:49:48.093Z] DEBUG: Adding message to journal and sending to Restate ; BackgroundInvokeEntryMessage
[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM93r8tce9IfwnbiAsk8lCevD][2024-03-19T08:49:48.093Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage
[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM93r8tce9IfwnbiAsk8lCevD][2024-03-19T08:49:48.093Z] DEBUG: Function completed successfully.
[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK529L4BRmz8ntFtiw2DkahH][2024-03-19T08:49:48.141Z] DEBUG: Invoking function.
[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK529L4BRmz8ntFtiw2DkahH][2024-03-19T08:49:48.141Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage
[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK529L4BRmz8ntFtiw2DkahH][2024-03-19T08:49:48.141Z] DEBUG: Function completed successfully.



Don't forget to set the delay back to 15 minutes.


No workflow orchestrator or cron jobs needed!
Durable timers are a powerful feature that can be used to implement workflows, schedule async tasks, or plan background jobs.
Restate makes them resilient to failures and ensures that they get executed. No extra infrastructure needed!


Suspendable sleep
Another timer-like feature of the SDK is suspendable sleep.
Restate will make sure that the function gets resumed after the specified duration has passed.
When running on function-as-a-service platforms, your function can suspend in the meantime, so you don't pay for the wait time.

await ctx.sleep(15 * 60 * 1000);



info
🚩 Explore the intermediate solution in part2, and run it with:

npm run part2



Virtual Objects vs. Services​


Implement it yourself or follow along by looking at the code under part3


At the beginning of this tutorial, we mentioned that the TicketObject and CartObject services are Virtual Objects.


Virtual Objects are identified by a key and allow you to store K/V state in Restate.
For each Virtual Object (key), only one invocation can run at a time (across all the handlers of that Virtual Object).


Services, on the other hand, do not have access to K/V state, and handlers can run concurrently.


Virtual Objects simplify many use cases
With access to consistent K/V state and strong concurrency guarantees, implementing the TicketObject in a resilient and consistent way becomes straightforward.
When a user reserves a ticket, we want to be sure that no concurrent other requests are reserving the same ticket at the same time.
To get this behaviour, we key the TicketObject on ticket ID. We now have a single Virtual Object per ticket.


Long-running operations in Virtual Objects
If you do long-running operations in a Virtual Object, no other invocations are processed the meantime.
For example, if you would implement the expiration of the ticket in the CartObject service by sleeping for 15 minutes:

await ctx.sleep(15 * 60 * 1000);
ctx.objectSendClient(TicketObject, ticketId).unreserve();

The user wouldn't be able to add any other tickets, nor buy the tickets.
If you do a delayed call, the invocation isn't ongoing until the delay has passed, so the Virtual Object is not locked.


Consistent K/V state​


Restate offers a key-value store to store application state for Virtual Objects.


No need for session databases!
Restate's state is guaranteed to be consistent across retries and invocations.
This eliminates the need for a session database.


Getting and setting K/V state​


Adapt the CartObject/addTicket function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
cart_object.ts

async addTicket(ctx: restate.ObjectContext, ticketId: string) {
  const reservationSuccess = await ctx.objectClient(TicketObject, ticketId).reserve();
  if (reservationSuccess) {
    const tickets = (await ctx.get<string[]>("tickets")) ?? [];
    tickets.push(ticketId);
    ctx.set("tickets", tickets);
    ctx.objectSendClient(CartObject, ctx.key, {delay: 15 * 60 * 1000})
        .expireTicket(ticketId);
  }
  return reservationSuccess;
},

To retrieve the cart, you use ctx.get.
This returns null if the value has never been set.
After you added the ticket to the cart array, you set the state to the new value with ctx.set.


You can store multiple key-value pairs, by using different state keys.
Here, you get the value under the key "tickets".
Restate returns the cart belonging to the current Virtual Object (for example, user Mary).


Run the services and call the addTicket function, to see the interaction with state.
Service logs

... logs from reserve call ...
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM97yGYEG5NWYtbMlnSAGHGY9][2024-03-19T08:55:20.941Z] DEBUG: Adding message to journal and sending to Restate ; GetStateEntryMessage
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM97yGYEG5NWYtbMlnSAGHGY9][2024-03-19T08:55:20.941Z] DEBUG: Adding message to journal and sending to Restate ; SetStateEntryMessage
... logs from expireTicket call ...



Local state access
When starting the invocation, Restate attaches the application state to the request.
So when you operate on the state in your function, you get access to a local copy of the state for fast access.


Also adapt the CartObject/checkout function, to use the tickets:
cart_object.ts

async checkout(ctx: restate.ObjectContext) {
  const tickets = (await ctx.get<string[]>("tickets")) ?? [];
  if (tickets.length === 0) {
    return false;
  }
  const success = await ctx.serviceClient(CheckoutService)
      .handle({userId: ctx.key, tickets: tickets});
  if (success) {
    ctx.clear("tickets");
  }
  return success;
},

After the tickets are checked out, you clear the state with ctx.clear.


Inspecting K/V state​


Restate exposes information on invocations and application state.
You can watch the state of the CartObject service, via:


CLI
psql

restate kv get -w -n 1 CartObject Mary



Output

🤖 State:
―――――――――
Service  CartObject
Key      Mary
KEY      VALUE
tickets  [
           "seat2B"
         ]



Add some tickets to the state to see how the query result gets updated.


Then, send a checkout request as earlier, and notice that the state is now empty.


📝 Try it out​


Finishing CartObject/expireTicket​
You have almost fully implemented the CartObject. Let's finish CartObject/expireTicket.
Before you call unreserve, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/unreserve and remove it from the state.
Solution
cart_object.ts

async expireTicket(ctx: restate.ObjectContext, ticketId: string) {
  const tickets = (await ctx.get<string[]>("tickets")) ?? [];
  const ticketIndex = tickets.findIndex((ticket) => ticket === ticketId);
  if (ticketIndex != -1) {
    tickets.splice(ticketIndex, 1);
    ctx.set("tickets", tickets);
    ctx.objectSendClient(TicketObject, ticketId).unreserve();
  }
},

Call the expireTicket handler with:

curl localhost:8080/CartObject/Mary/expireTicket -H 'content-type: application/json' -d '"seat2B"'



Implementing the TicketObject​


Track the status of the tickets in the TicketObject by storing it in the state and transitioning from one state to another, like a state machine.
The possible states are available (default), reserved, and sold.
Implement the handlers in the TicketObject to reserve, unreserve, and mark a ticket as sold.


While you are developing them, monitor the state of the TicketObject via:


CLI
psql

restate kv get -w -n 1 TicketObject seat2B



1
TicketObject/reserve
    

  1. Retrieve the value for the "status" state key.
    2. 

  2. If the value is set to TicketStatus.Available, then change it to TicketStatus.Reserved and
return true (reservation successful).
    3. 

  3. If the status isn't set to TicketStatus.Available, then return false.
    4. 

Solution
ticket_object.ts

async reserve(ctx: restate.ObjectContext) {
  const status =
      (await ctx.get<TicketStatus>("status")) ?? TicketStatus.Available;
  if (status === TicketStatus.Available) {
    ctx.set("status", TicketStatus.Reserved);
    return true;
  } else {
    return false;
  }
},

Now, you can't reserve the same ticket multiple times anymore.
Call addTicket multiple times for the same ID. The first time it returns true, afterwards false.
2
TicketObject/unreserve
Clear the "status", if it's not equal to TicketStatus.Sold.
Solution
ticket_object.ts

async unreserve(ctx: restate.ObjectContext) {
  const status =
      (await ctx.get<TicketStatus>("status")) ?? TicketStatus.Available;
  if (status !== TicketStatus.Sold) {
    ctx.clear("status");
  }
},

Now, the ticket reservation status is cleared when the delayed expireTicket call triggers.
Play around with reducing the delay of the expireTicket call in the addTicket handler.
Try to reserve the same ticket ID multiple times, and see how you are able to reserve it again after the unreserve handler executed.
3
TicketObject/markAsSold
Set the "status" to TicketStatus.Sold if it's reserved.
Solution
ticket_object.ts

async markAsSold(ctx: restate.ObjectContext) {
  const status =
      (await ctx.get<TicketStatus>("status")) ?? TicketStatus.Available;
  if (status === TicketStatus.Reserved) {
    ctx.set("status", TicketStatus.Sold);
  }
},

In the next section, you implement the CheckoutService/handle function that calls markAsSold.
This ties the final parts together.


info
🚩 Explore the intermediate solution in part3, and run it with:

npm run part3



Journaling actions​


Implement it yourself or follow along by looking at the code under part4


Restate's Durable Execution mechanism tracks the progress of the code execution in a journal.
Once an action/result has made it to the journal, it will not be re-executed on retries.


You can store the return value of any function in the journal, by using ctx.run.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
The SDK also offers helper functions for creating UUIDs and generating random numbers.
Handler logic needs to be deterministic
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use ctx.run to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/handle:


1
Generate an idempotency token
Let's use the SDK helper functions to generate a unique payment identifier and store it in Restate.
Once the token is stored, it will be the same on retries.
Try it out by printing the idempotency key and then throwing an error:
checkout_service.ts

async handle(
  ctx: restate.Context,
  request: { userId: string; tickets: string[] }
) {
  const idempotencyKey = ctx.rand.uuidv4();
  console.info("My idempotency key: " + idempotencyKey);
  throw new Error("Something happened!");
  return true;
},

Call CartObject/checkout and have a look at the logs to see what happens.
Service logs

... logs of `CartObjectService/CheckoutService` ...
[restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.498Z] DEBUG: Invoking function.
My idempotency key: e25b747f-ecfb-443b-8939-1935392aab6b
Trace: [restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.499Z] TRACE: Function completed with an error: Something happened! Error: Something happened!
... rest of trace  ...
[restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.512Z] DEBUG: Invocation ended with retryable error. ; ErrorMessage
[restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.568Z] DEBUG: Invoking function.
My idempotency key: e25b747f-ecfb-443b-8939-1935392aab6b
Trace: [restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.568Z] TRACE: Function completed with an error: Something happened! Error: Something happened!
... rest of trace  ...
[restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.568Z] DEBUG: Invocation ended with retryable error. ; ErrorMessage
... retries continue ...



2
Trigger the payment
Execute the payment via an external payment provider via PaymentClient.get().call(idempotencyKey, amount).
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollars.
checkout_service.ts

async handle(
  ctx: restate.Context,
  request: { userId: string; tickets: string[] }
) {
  const totalPrice = request.tickets.length * 40;
  const idempotencyKey = ctx.rand.uuidv4();
  const success = await ctx.run(() =>
    PaymentClient.get().call(idempotencyKey, totalPrice)
  );
  return success;
},



📝 Try it out​


Let's finish the checkout flow by sending the email notifications and marking the tickets as sold.


1
Implement the email notifications
After the CheckoutService/handle handler has handled the payment, you need to notify the users of the payment status:
    

  • Payment success: notify the users via EmailClient.get().notifyUserOfPaymentSuccess(request.getUserId()).
    • 

  • Payment failure: notify the users via the EmailClient.get().notifyUserOfPaymentFailure(request.getUserId()).
    • 

Solution
checkout_service.ts

async handle(ctx: restate.Context, request: { userId: string; tickets: string[] }) {
  const totalPrice = request.tickets.length * 40;
  const idempotencyKey = ctx.rand.uuidv4();
  const success = await ctx.run(() => PaymentClient.get().call(idempotencyKey, totalPrice));
  if (success) {
    await ctx.run(() => EmailClient.get().notifyUserOfPaymentSuccess(request.userId));
  } else {
    await ctx.run(() => EmailClient.get().notifyUserOfPaymentFailure(request.userId));
  }
  return success;
},

2
Mark tickets as sold
Let the CartObject/checkout handler mark all tickets as sold by calling TicketObject/markAsSold for each ticket.
Solution
cart_object.ts

async checkout(ctx: restate.ObjectContext) {
  const tickets = (await ctx.get<string[]>("tickets")) ?? [];
  if (tickets.length === 0) {
    return false;
  }
  const success = await ctx.serviceClient(CheckoutService)
      .handle({userId: ctx.key, tickets: ["seat2B"]});
  if (success) {
    for (const ticketId of tickets) {
      ctx.objectSendClient(TicketObject, ticketId).markAsSold();
    }
    ctx.clear("tickets");
  }
  return success;
},



🥳 You have now fully implemented the ticket reservation system!
Try it out by reserving some new tickets and buying them by checking out the cart.


info
🚩 Explore the intermediate solution in part4, and run it with:

npm run part4



Idempotency for any request​


As you saw, generating idempotency keys inside your handlers and storing them in Restate is easy.
But this doesn't guard us yet against retries of the HTTP request to Restate.


For example, if the caller of the addTicket handler didn't receive the success response of its first request, it might retry the request.


The second request will return false because the ticket already got reserved the first time, but the caller won't know about this.


To cover this, you can add an idempotency-key header to the incoming request to let Restate deduplicate them.


In our example, when we call the CartObject/addTicket handler, the first time the response is true and the second time it's false.
However, if we use the same idempotency key, the second call will return true as well, because it will return the result of the first call:

curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' \
    -H 'idempotency-key: ad5472esg4dsg525dssdfa5loi'  \
    -d '"seat2C"'



You can also see from the service logs that the handler wasn't executed the second time.


End-to-end idempotency
Restate gives you idempotency for any service, handler and request for free.
No extra setup.


info
You only need this when invoking handlers over HTTP.
When a handler calls another handler, Restate automatically takes care of the idempotency.


Tracing​


Restate exposes OpenTelemetry traces of your invocations.


1
Run Jaeger

docker run -d --name jaeger -e COLLECTOR_OTLP_ENABLED=true \
    -p 4317:4317 -p 16686:16686 jaegertracing/all-in-one:1.46



2
Relaunch Restate with tracing enabled
binary
Docker

restate-server --tracing-endpoint http://localhost:4317



3
Send a few requests

curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2A"'
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2B"'
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2C"'
curl -X POST localhost:8080/CartObject/Mary/checkout



4
Go to the Jaeger UI
http://localhost:16686


5
Inspect the traces
Select the CartObject service from the service dropdown.
You should see the addTicket and checkout requests listed.
Have a look at the traces of the checkout call:
CheckoutService call traces
You can see the calls that were done to Restate, for example invoke, sleep, one way call, get state, etc., and their timings.
If you expand one of the traces, you can see tags describing some metadata of the context call, for example invocation ID and the request.


For more information, have a look at the tracing docs.


🏁 The end​


You reached the end of this tutorial!


Let's recap what you did! You have built a ticket reservation system that is resilient, consistent, and scalable.
We used Restate to provide us with durable, distributed building blocks to simplify the implementation of the system.
Let's list a few of them:


What you implementedWhat you didn't implement, as Restate handles it for you
✅ Request-response invocations❌ Handling retries, timeouts, etc.
✅ Sending messages❌ Deploy and operate message queues for async requests
✅ Idempotent HTTP calls❌ Write deduplication logic
✅ Durable Execution: retries, partial progress recovery, and suspensions❌ Manual retry logic and partial progress recovery
✅ Durable timers: sleeping and scheduling async tasks❌ Workflow orchestrators or cron jobs for scheduling tasks
✅ Virtual Objects: concurrency guarantees and shared state❌ Guards for keeping state consistent across retries, concurrent requests, and scaling out.
✅ K/V state: storing and inspecting❌ Session databases for state. State consistency guards.
✅ Storing computation results in the journal❌ Logic to make operations idempotent (e.g. generate idempotency keys)


You now know the essentials to start developing Restate services! Have a look at the next steps to explore further.


Next steps​