Tour of Restate
- TypeScript
- Java
- Go
- Python
This tutorial guides you through the development of an end-to-end Restate application, and covers all the essential features. After this tutorial, you should have a firm understanding of how Restate can help you and feel comfortable to tackle your next application on your own.
This tutorial implements a ticket reservation application for a theatre. It allows users to add tickets for specific seats to their shopping cart. After the user adds a ticket, the seat gets reserved for 15 minutes. If the user doesn't pay for the ticket within that time interval, the reservation is released and the ticket becomes available to other users.
Restate applications are made up of services that expose handlers. Handlers are functions that implement business logic. Restate manages their invocation and execution. Services communicate with one another using Remote Procedure Calls (RPC). Our ticket example consists of three services:
As we go, you will discover how Restate can help you with some intricacies in this application.
- TypeScript
- Java
- Go
- Python
- Latest stable version of NodeJS >= v18.17.1 and npm CLI >= 9.6.7 installed.
- Restate CLI: Installation instructions
- Optional: Docker Engine or Podman, if you want to run the Restate Server with Docker. And to run Jaeger.
This guide is written for:
- TypeScript SDK version:
1.3.0
- Restate runtime Docker image:
docker.io/restatedev/restate:1.1
- JDK >= 11
- Restate CLI: Installation instructions
- Optional: Docker Engine or Podman, if you want to run the Restate Server with Docker. And to run Jaeger.
This guide is written for:
- Java SDK version:
1.1.0
- Restate runtime Docker image:
docker.io/restatedev/restate:1.1
- Go >= 1.21.0
- Restate CLI: Installation instructions
- Optional: Docker Engine or Podman, if you want to run the Restate Server with Docker. And to run Jaeger.
This guide is written for:
- Go SDK version:
0.11.0
- Restate runtime Docker image:
docker.io/restatedev/restate:1.1
- Python >= 3.11
- Restate CLI: Installation instructions
- Optional: Docker Engine or Podman, if you want to run the Restate Server with Docker. And to run Jaeger.
This guide is written for:
- Python SDK version:
0.3.0
- Restate runtime Docker image:
docker.io/restatedev/restate:1.1
Getting Startedβ
Set up the services
- TypeScript
- Java
- Go
- Python
Download the example and run locally with an IDE:
Install the dependencies and build the app:
npm install && npm run build
Run the services
npm run app-dev
This GitHub repository contains the basic skeleton of the TypeScript services that you develop in this tutorial.
Run the services
./gradlew run
This GitHub repository contains the basic skeleton of the Java services that you develop in this tutorial.
Run the services
go run ./app
Download the example and run locally with an IDE:
Setup your virtual environment:
python3 -m venv .venvsource .venv/bin/activate
Install the requirements:
pip install -r requirements.txt
Run the services:
python -m hypercorn --config hypercorn-config.toml tour/app/app:app
This GitHub repository contains the basic skeleton of the Python services that you develop in this tutorial.
Launch Restate
Restate is a single self-contained binary. No external dependencies needed. Check out our Local Dev page for other ways to run Restate.
Register the services with Restate
Now, we need to tell Restate where our services are running.
You can register services by calling the Restate Admin API (default port 9070
) and supplying it the service endpoint URI:
Output
- Typescript
- Java
- Go
- Python
If you run Restate with Docker, replace http://localhost:9080
by http://host.docker.internal:9080
.
All set up!
- TypeScript
- Java
- Go
- Python
In src/app
you will find the skeletons of the various services to help you start implementing the app.
For example:
Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.
The app.ts
file contains the definition of the endpoint that hosts the services.
In src/main/java/dev/restate/tour/app
you will find the skeletons of the various services to help you start implementing the app.
For example:
Services are annotated by @Service
. A service consists of a set of handlers, and each handler is annotated by @Handler
.
Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.
The AppMain.java
file contains the definition of the endpoint that hosts the services.
In app
you will find the skeletons of the various services to help you start implementing the app.
For example:
Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.
The main.go
file contains the definition of the endpoint that hosts the services.
In tour/app
you will find the skeletons of the various services to help you start implementing the app.
For example:
Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.
The app.py
file contains the definition of the endpoint that hosts the services.
Invoking Handlersβ
Handlers can be invoked in several ways: via HTTP requests, programmatically with the SDK, or via Kafka events.
Request-response calls over HTTPβ
Let's start with invoking our handler over HTTP using curl
.
- TypeScript
- Java
- Go
- Python
For example, add a ticket seat2B
to the cart of Mary by calling the addTicket
handler of the CartObject
:
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2B"'
If this prints out true
, then you have a working setup.
When Mary wants to proceed with the purchase, call the checkout
handler of the CartObject
:
curl -X POST localhost:8080/CartObject/Mary/checkout
For example, add a ticket seat2B
to the cart of Mary by calling the addTicket
handler of the CartObject
:
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2B"'
If this prints out true
, then you have a working setup.
When Mary wants to proceed with the purchase, call the checkout
handler of the CartObject
:
curl -X POST localhost:8080/CartObject/Mary/checkout
For example, add a ticket seat2B
to the cart of Mary by calling the AddTicket
handler of the CartObject
:
curl localhost:8080/CartObject/Mary/AddTicket -H 'content-type: application/json' -d '"seat2B"'
If this prints out true
, then you have a working setup.
When Mary wants to proceed with the purchase, call the Checkout
handler of the CartObject
:
curl -X POST localhost:8080/CartObject/Mary/Checkout
For example, add a ticket seat2B
to the cart of Mary by calling the addTicket
handler of the CartObject
:
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2B"'
If this prints out true
, then you have a working setup.
When Mary wants to proceed with the purchase, call the checkout
handler of the CartObject
:
curl -X POST localhost:8080/CartObject/Mary/checkout
We will use these two curl
commands often when developing the code, so keep them handy.
Restate acts as a proxy for your services. It forwards the request to the correct service and handler. Therefore, the request is sent to Restate and not directly to the service.
Mary
?Handlers are either a part of plain services or Virtual Objects.
Virtual Objects are a special type of service that allows you to group handlers together, share state between them, and control concurrency.
Each Virtual Object has a unique key.
We will cover the difference in more detail later.
For now, it's important to note that when invoking a handler within a Virtual Object, you need to specify its key.
In our example, the CartObject
and TicketObject
are Virtual Objects, while the CheckoutService
is a plain service.
To add the ticket to Mary's cart, we need to specify the key Mary
in the path to reach her Virtual Object.
We can do the same programmatically within a handler by using the SDK. Let's try this out!
Request-response calls between handlersβ
You can also call other handlers programmatically by using the clients generated by the Restate SDK. Let's try this out!
- TypeScript
- Java
- Go
- Python
When we add a ticket to the cart, the CartObject/addTicket
handler first needs to reserve the ticket for the user.
It does that by calling the TicketObject/reserve
handler:
Service logs
[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.790Z] DEBUG: Invoking function.[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.792Z] DEBUG: Adding message to journal and sending to Restate ; InvokeEntryMessage[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.792Z] DEBUG: Scheduling suspension in 30000 ms[restate] [TicketObject/reserve][inv_1k78Krj3GqrK6odGaRe866kHZilkVf1H4l][2024-03-18T16:30:28.796Z] DEBUG: Invoking function.[restate] [TicketObject/reserve][inv_1k78Krj3GqrK6odGaRe866kHZilkVf1H4l][2024-03-18T16:30:28.796Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage[restate] [TicketObject/reserve][inv_1k78Krj3GqrK6odGaRe866kHZilkVf1H4l][2024-03-18T16:30:28.796Z] DEBUG: Function completed successfully.[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.799Z] DEBUG: Received completion message from Restate, adding to journal. ; CompletionMessage[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.800Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage[restate] [CartObject/addTicket][inv_1gdJBtdVEcM95mw1LLMMJY1Y0thJ9ugFGN][2024-03-18T16:30:28.800Z] DEBUG: Function completed successfully.
- Create the client via
ctx.serviceClient
orctx.objectClient
(for Virtual Objects). Specify the service definition (TicketObject
) and optionally the Virtual Object key (ticketId
). - Specify the handler you want to call and supply the request. Here
reserve()
. - Await the response of the call.
Send a request to CartObject/addTicket
as we did previously, and have a look at the service logs.
When we add a ticket to the cart, the CartObject/addTicket
handler first needs to reserve the ticket for the user.
It does that by calling the TicketObject/reserve
handler:
Service logs
2024-04-16 17:18:59 DEBUG [CartObject/addTicket] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CartObject/addTicket2024-04-16 17:18:59 INFO [CartObject/addTicket] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-16 17:19:00 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH5LkrvugCbFBq1VKcNrjuzn] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-16 17:19:00 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH5LkrvugCbFBq1VKcNrjuzn] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): InvokeEntryMessage2024-04-16 17:19:00 DEBUG [TicketObject/reserve] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to TicketObject/reserve2024-04-16 17:19:00 INFO [TicketObject/reserve] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-16 17:19:00 DEBUG [TicketObject/reserve][inv_1aAMfXkieWDz1fARH9n1r2H1YjjsTZxei5] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-16 17:19:00 DEBUG [TicketObject/reserve][inv_1aAMfXkieWDz1fARH9n1r2H1YjjsTZxei5] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): OutputEntryMessage2024-04-16 17:19:00 INFO [TicketObject/reserve][inv_1aAMfXkieWDz1fARH9n1r2H1YjjsTZxei5] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:19:00 DEBUG [TicketObject/reserve][inv_1aAMfXkieWDz1fARH9n1r2H1YjjsTZxei5] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-16 17:19:00 INFO [TicketObject/reserve][inv_1aAMfXkieWDz1fARH9n1r2H1YjjsTZxei5] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:19:00 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH5LkrvugCbFBq1VKcNrjuzn] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [2](): OutputEntryMessage2024-04-16 17:19:00 INFO [CartObject/addTicket][inv_1aiqX0vFEFNH5LkrvugCbFBq1VKcNrjuzn] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:19:00 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH5LkrvugCbFBq1VKcNrjuzn] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-16 17:19:00 INFO [CartObject/addTicket][inv_1aiqX0vFEFNH5LkrvugCbFBq1VKcNrjuzn] dev.restate.sdk.core.InvocationStateMachine - End invocation
- Use the pre-generated client (
TicketObject
): This gets generated when you compile the code for the first time. So if you haven't done that yet, run./gradlew build
to generate the client. - Supply the context (and specify the Virtual Object) key via
fromContext(ctx, ticketId)
. - Specify the handler you want to call and supply the request. Here
reserve()
. - Await the response of the call.
Once you have added this to the code, restart the service, call the CartObject/addTicket
method as we did previously, and have a look at the service logs.
When we add a ticket to the cart, the CartObject/AddTicket
handler first needs to reserve the ticket for the user.
It does that by calling the TicketObject/Reserve
handler:
Service logs
2024/08/16 13:40:47 INFO Handling invocation method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp0QGHxdcBOYZTbVbvHUMuQh2024/08/16 13:40:47 INFO Handling invocation method=TicketObject/Reserve invocationID=inv_19maBIcE9uRD2cN79SYc6Ef1KldA21IG1H2024/08/16 13:40:47 INFO Invocation completed successfully method=TicketObject/Reserve invocationID=inv_19maBIcE9uRD2cN79SYc6Ef1KldA21IG1H2024/08/16 13:40:47 INFO Invocation completed successfully method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp0QGHxdcBOYZTbVbvHUMuQh
- Create the client via
restate.Service
orrestate.Object
(for Virtual Objects). Specify the return type (bool
) as a type parameter, the service name (TicketObject
) and method (Reserve
) and where necessary the Virtual Object key (ticketId
). - Specify what type of call you want to make - in this case we want a synchronous request-response call so we use
Request()
, passingrestate.Void{}
as this handler needs no input type.
Send a request to CartObject/AddTicket
as we did previously, and have a look at the service logs.
When we add a ticket to the cart, the CartObject/addTicket
handler first needs to reserve the ticket for the user.
It does that by calling the TicketObject/reserve
handler:
- Use
ctx.service_call
(for Services) orctx.object_call
(for Virtual Objects). - Specify the handler you want to call and supply the request: Here, we supply the
reserve
method that we import from theTicketObject
. For Virtual Objects, you also need to specify the key of the Virtual Object that you want to call. Here, this is the ticket ID. - Await the response of the call.
Send a request to CartObject/addTicket
as we did previously.
You can see the calls to addTicket
and reserve
in the Restate Server logs.
Sending messages between handlersβ
We can also let handlers send messages to other handlers without waiting for a response.
- TypeScript
- Java
- Go
- Python
In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes.
When a user didn't proceed with the payment before the timeout, the CartObject/expireTicket
handler is triggered.
Let the expireTicket
handler call the TicketObject/unreserve
handler.
Specify that you want to call the TicketObject
by supplying ticketObjectApi
to the send
function.
Then call the unreserve
handler on the TicketObject
.
Once you have added this to the code, call the CartObject/expireTicket
handler:
curl localhost:8080/CartObject/Mary/expireTicket -H 'content-type: application/json' -d '"seat2B"'
Service logs
[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM942bjcDmb1c1khoaJe11Hbz][2024-03-18T16:14:24.671Z] DEBUG: Invoking function.[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM942bjcDmb1c1khoaJe11Hbz][2024-03-18T16:14:24.672Z] DEBUG: Adding message to journal and sending to Restate ; BackgroundInvokeEntryMessage[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM942bjcDmb1c1khoaJe11Hbz][2024-03-18T16:14:24.673Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM942bjcDmb1c1khoaJe11Hbz][2024-03-18T16:14:24.673Z] DEBUG: Function completed successfully.[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK3GuJXkgaXBbg69R47TCeAN][2024-03-18T16:14:24.677Z] DEBUG: Invoking function.[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK3GuJXkgaXBbg69R47TCeAN][2024-03-18T16:14:24.677Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK3GuJXkgaXBbg69R47TCeAN][2024-03-18T16:14:24.677Z] DEBUG: Function completed successfully.
The service logs show how the expireTicket
handler gets executed and then the unreserve
handler.
The call to expireTicket
finishes earlier than the unreserve
handler because expireTicket
didn't wait for the response of the unreserve
handler.
In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes.
When a user didn't proceed with the payment before the timeout, the CartObject/expireTicket
handler is triggered.
Let the expireTicket
handler call the TicketObject/unreserve
handler.
You now call send()
on the generated client to send a message instead of doing a request-response call.
You also don't need to await the response, as you don't expect one.
Call the handler via:
curl localhost:8080/CartObject/Mary/expireTicket -H 'content-type: application/json' -d '"seat2B"'
Service logs
2024-04-16 17:27:45 DEBUG [CartObject/expireTicket] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CartObject/expireTicket2024-04-16 17:27:45 INFO [CartObject/expireTicket] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-16 17:27:45 DEBUG [CartObject/expireTicket][inv_1aiqX0vFEFNH0T0mRlvCk7xTVSB5xQIaKR] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-16 17:27:45 DEBUG [CartObject/expireTicket][inv_1aiqX0vFEFNH0T0mRlvCk7xTVSB5xQIaKR] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): BackgroundInvokeEntryMessage2024-04-16 17:27:45 DEBUG [CartObject/expireTicket][inv_1aiqX0vFEFNH0T0mRlvCk7xTVSB5xQIaKR] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [2](): OutputEntryMessage2024-04-16 17:27:45 INFO [CartObject/expireTicket][inv_1aiqX0vFEFNH0T0mRlvCk7xTVSB5xQIaKR] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:27:45 DEBUG [CartObject/expireTicket][inv_1aiqX0vFEFNH0T0mRlvCk7xTVSB5xQIaKR] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-16 17:27:45 INFO [CartObject/expireTicket][inv_1aiqX0vFEFNH0T0mRlvCk7xTVSB5xQIaKR] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:27:45 DEBUG [TicketObject/unreserve] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to TicketObject/unreserve2024-04-16 17:27:45 INFO [TicketObject/unreserve] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-16 17:27:45 DEBUG [TicketObject/unreserve][inv_1aAMfXkieWDz6IcQAkXhOPoZ3T9A9KTC3D] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-16 17:27:45 DEBUG [TicketObject/unreserve][inv_1aAMfXkieWDz6IcQAkXhOPoZ3T9A9KTC3D] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): OutputEntryMessage2024-04-16 17:27:45 INFO [TicketObject/unreserve][inv_1aAMfXkieWDz6IcQAkXhOPoZ3T9A9KTC3D] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:27:45 DEBUG [TicketObject/unreserve][inv_1aAMfXkieWDz6IcQAkXhOPoZ3T9A9KTC3D] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-16 17:27:45 INFO [TicketObject/unreserve][inv_1aAMfXkieWDz6IcQAkXhOPoZ3T9A9KTC3D] dev.restate.sdk.core.InvocationStateMachine - End invocation
The service logs show how the expireTicket
handler gets executed and then the unreserve
handler.
The call to expireTicket
finishes earlier than the unreserve
handler because expireTicket
didn't wait for the response of the unreserve
handler.
In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes.
When a user didn't proceed with the payment before the timeout, the CartObject/ExpireTicket
handler is triggered.
Let the ExpireTicket
handler call the TicketObject/Unreserve
handler.
Specify that you want to call the TicketObject
by supplying the service and method names to the ObjectSend()
function.
This function is an alternative to Object()
which provides a client that can only send one-way messages
and therefore doesn't need an output type parameter. Finally call the Send
method on the returned client.
Once you have added this to the code, call the CartObject/ExpireTicket
handler:
curl localhost:8080/CartObject/Mary/ExpireTicket -H 'content-type: application/json' -d '"seat2B"'
Service logs
2024/08/16 13:55:31 INFO Handling invocation method=CartObject/ExpireTicket invocationID=inv_1fmRNvSNVxNp6GGSrDXviABHoq8paj5Bqp2024/08/16 13:55:31 INFO Invocation completed successfully method=CartObject/ExpireTicket invocationID=inv_1fmRNvSNVxNp6GGSrDXviABHoq8paj5Bqp2024/08/16 13:55:31 INFO Handling invocation method=TicketObject/Unreserve invocationID=inv_19maBIcE9uRD30z1j0kx3N3SOEPCngmSrL2024/08/16 13:55:31 INFO Invocation completed successfully method=TicketObject/Unreserve invocationID=inv_19maBIcE9uRD30z1j0kx3N3SOEPCngmSrL
The service logs show how the ExpireTicket
handler gets executed and then the Unreserve
handler.
The call to ExpireTicket
finishes earlier than the Unreserve
handler because ExpireTicket
didn't wait for the response of the Unreserve
handler.
In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes.
When a user didn't proceed with the payment before the timeout, the CartObject/expireTicket
handler is triggered.
Let the expireTicket
handler call the TicketObject/unreserve
handler.
Import the unreserve
handler from the TicketObject
and supply it to ctx.object_send
together with the ticket ID.
Once you have added this to the code, call the CartObject/expireTicket
handler:
curl localhost:8080/CartObject/Mary/expireTicket -H 'content-type: application/json' -d '"seat2B"'
The Restate Server logs show how the expireTicket
handler gets executed and then the unreserve
handler.
Restate persists and retries failed one-way invocations. There is no need to set up message queues to ensure delivery!
To send messages via curl
, add /send
to the handler path:
- TypeScript
- Java
- Go
- Python
curl localhost:8080/CartObject/Mary/addTicket/send -H 'content-type: application/json' -d '"seat2B"'
curl localhost:8080/CartObject/Mary/addTicket/send -H 'content-type: application/json' -d '"seat2B"'
curl localhost:8080/CartObject/Mary/AddTicket/send -H 'content-type: application/json' -d '"seat2B"'
curl localhost:8080/CartObject/Mary/addTicket/send -H 'content-type: application/json' -d '"seat2B"'
Output
{"invocationId":"inv_1aiqX0vFEFNH1Umgre58JiCLgHfTtztYK5","status":"Accepted"}
This returns the invocation ID. This is a unique identifier for the invocation. You can use it to track the progress of the invocation via the CLI, and to correlate logs and metrics.
π Try it outβ
- TypeScript
- Java
- Go
- Python
Make the CartObject/checkout
handler call the CheckoutService/handle
handler.
For the request field, you can use a hard-coded string array for now: ["seat2B"]
.
You will fix this later on. Note that the CheckoutService
is not a Virtual Object, so you don't need to specify a key.
Solution
Add the following code to the CartObject/checkout
handler:
Call CartObject/checkout
as you did earlier and have a look at the logs again to see what happened:
[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.018Z] DEBUG: Invoking function.[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.019Z] DEBUG: Adding message to journal and sending to Restate ; InvokeEntryMessage[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.020Z] DEBUG: Scheduling suspension in 30000 ms[restate] [CheckoutService/handle][inv_16WnWCiCVV5G2gUUevDM4uIli4v7TN9k2d][2024-03-19T07:57:24.023Z] DEBUG: Invoking function.[restate] [CheckoutService/handle][inv_16WnWCiCVV5G2gUUevDM4uIli4v7TN9k2d][2024-03-19T07:57:24.024Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage[restate] [CheckoutService/handle][inv_16WnWCiCVV5G2gUUevDM4uIli4v7TN9k2d][2024-03-19T07:57:24.024Z] DEBUG: Function completed successfully.[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.026Z] DEBUG: Received completion message from Restate, adding to journal. ; CompletionMessage[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.027Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage[restate] [CartObject/checkout][inv_1gdJBtdVEcM919dOBhoVBm3fUMlaIHnANr][2024-03-19T07:57:24.027Z] DEBUG: Function completed successfully.
Make the CartObject/checkout
handler call the CheckoutService/handle
handler.
For the request field, you can use a hard-coded string array for now: ["seat2B"]
.
You will fix this later on. Note that the CheckoutService
is not a Virtual Object, so you don't need to specify a key.
Solution
Add the following code to the CartObject/checkout
handler:
Restart the service and call the CartObject/checkout
handler as you did earlier and have a look at the logs again to see what happened.
2024-04-16 17:32:10 DEBUG [CartObject/checkout] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CartObject/checkout2024-04-16 17:32:10 INFO [CartObject/checkout] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-16 17:32:10 DEBUG [CartObject/checkout][inv_1aiqX0vFEFNH7u1kZjvyH4KJpuO9j4njCp] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-16 17:32:10 DEBUG [CartObject/checkout][inv_1aiqX0vFEFNH7u1kZjvyH4KJpuO9j4njCp] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): InvokeEntryMessage2024-04-16 17:32:10 DEBUG [CheckoutService/handle] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CheckoutService/handle2024-04-16 17:32:10 INFO [CheckoutService/handle] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-16 17:32:10 DEBUG [CheckoutService/handle][inv_12rkfeAcppNY3cI4F6DBZK8fir8uaIrIBP] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-16 17:32:11 DEBUG [CheckoutService/handle][inv_12rkfeAcppNY3cI4F6DBZK8fir8uaIrIBP] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): OutputEntryMessage2024-04-16 17:32:11 INFO [CheckoutService/handle][inv_12rkfeAcppNY3cI4F6DBZK8fir8uaIrIBP] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:32:11 DEBUG [CheckoutService/handle][inv_12rkfeAcppNY3cI4F6DBZK8fir8uaIrIBP] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-16 17:32:11 INFO [CheckoutService/handle][inv_12rkfeAcppNY3cI4F6DBZK8fir8uaIrIBP] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:32:11 DEBUG [CartObject/checkout][inv_1aiqX0vFEFNH7u1kZjvyH4KJpuO9j4njCp] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [2](): OutputEntryMessage2024-04-16 17:32:11 INFO [CartObject/checkout][inv_1aiqX0vFEFNH7u1kZjvyH4KJpuO9j4njCp] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:32:11 DEBUG [CartObject/checkout][inv_1aiqX0vFEFNH7u1kZjvyH4KJpuO9j4njCp] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-16 17:32:11 INFO [CartObject/checkout][inv_1aiqX0vFEFNH7u1kZjvyH4KJpuO9j4njCp] dev.restate.sdk.core.InvocationStateMachine - End invocation
Make the CartObject/Checkout
handler call the CheckoutService/Handle
handler.
For the request field, you can use a hard-coded string array for now: ["seat2B"]
.
You will fix this later on. Note that the CheckoutService
is not a Virtual Object, so you don't need to specify a key.
Solution
Add the following code to the CartObject/Checkout
handler:
Call CartObject/Checkout
as you did earlier and have a look at the logs again to see what happened:
2024/08/16 13:41:03 INFO Handling invocation method=CartObject/Checkout invocationID=inv_1fmRNvSNVxNp3lG8MTJNCop4OwmqZXVRi92024/08/16 13:41:03 INFO Handling invocation method=CheckoutService/Handle invocationID=inv_1keKt2264Zem3IoLhpf8KDmHMJnmbgUbsJ2024/08/16 13:41:03 INFO Invocation completed successfully method=CheckoutService/Handle invocationID=inv_1keKt2264Zem3IoLhpf8KDmHMJnmbgUbsJ2024/08/16 13:41:03 INFO Invocation completed successfully method=CartObject/Checkout invocationID=inv_1fmRNvSNVxNp3lG8MTJNCop4OwmqZXVRi9 ```
Make the CartObject/checkout
handler call the CheckoutService/handle
handler.
For the request field, you can use a hard-coded string array for now: ["seat2B"]
.
You will fix this later on. Note that the CheckoutService
is not a Virtual Object, so you don't need to specify a key.
Solution
Add the following code to the CartObject/checkout
handler:
Call CartObject/checkout
as you did earlier and have a look at the Restate Server logs again to see what happened:
Durable Executionβ
The calls we just did seem like regular RPC calls as you might know them from other service frameworks. But under the hood a lot more is happening.
Restate makes RPC calls resilient by letting the Restate Server and SDK cooperate. Restate tracks the execution of code in a journal and can replay it in case of a failure. This is called Durable Execution.
Have a look at the animation to understand what happened under-the-hood:
Journals
addTicket ( Mary, seat2B )
reserve ( seat2B )
This animation shows you what happened under the hood when we did the reserve call from the `CartObject` to the `TicketObject`. The animation uses the TypeScript SDK.
async function addTicket(ctx, ticketId){
const success = await ctx
.objectClient(ticketObject)
.reserve(ticketId);
return success;
}
Journal:
{ seat2B }
{ success }
{ success }
async function reserve(ctx, ticketId){
...
return success;
}
Journal:
{ success }
Whenever a failure would happen, Restate would be able to recover the latest state of the handler by sending over the journal. The code would fast-forward to the point where it crashed, and continue executing from there on.
- TypeScript
- Java
- Go
- Python
To see the recovery of partial progress in practice, let's make the CartObject/addTicket
handler crash right after the call.
Add the following code to line 4 of the snippet, to let the code throw an error after the call:
throw new Error("Failing");
Call CartObject/addTicket
again and have a look at the service logs.
Service logs
[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.245Z] DEBUG: Adding message to journal and sending to Restate ; InvokeEntryMessage[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.246Z] DEBUG: Scheduling suspension in 30000 ms[restate] [TicketObject/reserve][inv_1iGFK6hGrtOf3jcD8PupmCOJz1SDzvfPi1][2024-04-16T13:28:20.296Z] DEBUG: Invoking function.[restate] [TicketObject/reserve][inv_1iGFK6hGrtOf3jcD8PupmCOJz1SDzvfPi1][2024-04-16T13:28:20.296Z] DEBUG: Journaled and sent output message ; OutputEntryMessage[restate] [TicketObject/reserve][inv_1iGFK6hGrtOf3jcD8PupmCOJz1SDzvfPi1][2024-04-16T13:28:20.296Z] DEBUG: Function completed successfully.[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.362Z] DEBUG: Received completion message from Restate, adding to journal.Trace: [restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.363Z] TRACE: Function completed with an error: Failing Error: Failing... rest of trace ...[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.372Z] DEBUG: Invocation ended with retryable error. ; ErrorMessage[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.437Z] DEBUG: Resuming (replaying) function.[restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.437Z] DEBUG: Matched and replayed message from journal ; InvokeEntryMessageTrace: [restate] [CartObject/addTicket][inv_1aiqX0vFEFNH0TF1pLRFBDFosQCCTAN1M5][2024-04-16T13:28:20.437Z] TRACE: Function completed with an error: Failing Error: Failing... rest of trace ...
To see the recovery of partial progress in practice, let's make the CartObject/addTicket
handler crash right after the call.
Instead of returning true, let the code fail after the call:
throw new IllegalStateException("The handler failed");
Call CartObject/addTicket
again and have a look at the service logs.
Service logs
2024-04-16 17:33:59 DEBUG [CartObject/addTicket] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CartObject/addTicket2024-04-16 17:33:59 INFO [CartObject/addTicket] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-16 17:33:59 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH5uLBb8M6CjbRkVUcVScH1T] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-16 17:33:59 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH5uLBb8M6CjbRkVUcVScH1T] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): InvokeEntryMessage2024-04-16 17:33:59 DEBUG [TicketObject/reserve] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to TicketObject/reserve2024-04-16 17:33:59 INFO [TicketObject/reserve] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-16 17:33:59 DEBUG [TicketObject/reserve][inv_1aAMfXkieWDz6Dn3DPBWPXOWCarIhmgCSl] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-16 17:33:59 DEBUG [TicketObject/reserve][inv_1aAMfXkieWDz6Dn3DPBWPXOWCarIhmgCSl] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): OutputEntryMessage2024-04-16 17:33:59 INFO [TicketObject/reserve][inv_1aAMfXkieWDz6Dn3DPBWPXOWCarIhmgCSl] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:33:59 DEBUG [TicketObject/reserve][inv_1aAMfXkieWDz6Dn3DPBWPXOWCarIhmgCSl] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-16 17:33:59 INFO [TicketObject/reserve][inv_1aAMfXkieWDz6Dn3DPBWPXOWCarIhmgCSl] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:33:59 WARN [CartObject/addTicket][inv_1aiqX0vFEFNH5uLBb8M6CjbRkVUcVScH1T] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Error when processing the invocationjava.lang.IllegalStateException: The handler failed... rest of trace ...2024-04-16 17:33:59 WARN [CartObject/addTicket][inv_1aiqX0vFEFNH5uLBb8M6CjbRkVUcVScH1T] dev.restate.sdk.core.InvocationStateMachine - Invocation failedjava.lang.IllegalStateException: The handler failed... rest of trace ...2024-04-16 17:33:59 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH5uLBb8M6CjbRkVUcVScH1T] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-16 17:33:59 INFO [CartObject/addTicket][inv_1aiqX0vFEFNH5uLBb8M6CjbRkVUcVScH1T] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-16 17:33:59 DEBUG [CartObject/addTicket] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CartObject/addTicket2024-04-16 17:33:59 INFO [CartObject/addTicket] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-16 17:33:59 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH5uLBb8M6CjbRkVUcVScH1T] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-16 17:33:59 WARN [CartObject/addTicket][inv_1aiqX0vFEFNH5uLBb8M6CjbRkVUcVScH1T] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Error when processing the invocationjava.lang.IllegalStateException: The handler failed... rest of trace ...2024-04-16 17:33:59 WARN [CartObject/addTicket][inv_1aiqX0vFEFNH5uLBb8M6CjbRkVUcVScH1T] dev.restate.sdk.core.InvocationStateMachine - Invocation failedjava.lang.IllegalStateException: The handler failed... rest of trace ...
To see the recovery of partial progress in practice, let's make the CartObject/AddTicket
handler crash right after the call.
Add the following code to line 6 of the snippet, to let the code throw an error after the call:
return false, fmt.Errorf("Failing")
Call CartObject/AddTicket
again and have a look at the service logs.
Service logs
2024/08/16 15:22:01 INFO Handling invocation method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp5kQH2M8E6VUscJII4P9QjL2024/08/16 15:22:01 INFO Handling invocation method=TicketObject/Reserve invocationID=inv_19maBIcE9uRD0VSkEQjYYaGPjXrHpUjZ8l2024/08/16 15:22:01 INFO Invocation completed successfully method=TicketObject/Reserve invocationID=inv_19maBIcE9uRD0VSkEQjYYaGPjXrHpUjZ8l2024/08/16 15:22:01 ERROR Invocation returned a non-terminal failure method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp5kQH2M8E6VUscJII4P9QjL err=Failed2024/08/16 15:22:01 INFO Handling invocation method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp5kQH2M8E6VUscJII4P9QjL2024/08/16 15:22:01 ERROR Invocation returned a non-terminal failure method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp5kQH2M8E6VUscJII4P9QjL err=Failed2024/08/16 15:22:01 INFO Handling invocation method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp5kQH2M8E6VUscJII4P9QjL2024/08/16 15:22:01 ERROR Invocation returned a non-terminal failure method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp5kQH2M8E6VUscJII4P9QjL err=Failed... rest of trace ...
To see the recovery of partial progress in practice, let's make the CartObject/addTicket
handler crash right after the call.
Add the following code to line 4 of the snippet, to let the code throw an error after the call:
raise Exception("Failing")
Call CartObject/addTicket
again and have a look at the Restate Server logs to see what happens.
You see the retries taking place. And you see that only the first time the call to the CheckoutService
was made.
The other times, the call was skipped and the journaled response was replayed.
By default, Restate will keep retrying failed invocations until they succeed. If you want to cancel an invocation in a retry loop, you can use the CLI to do this. Let's have a look at that next.
Debugging with the CLIβ
Now that we have a failing invocation, let's take the opportunity to show you how you can get more information about what is going on with the CLI. The CLI is a management tool that lets you interact with the Restate Server. You can use it to boostrap a new project, but also to get information about the services and invocations.
Have a look at some useful commands and try them out yourself:
Remove the throwing of the exception from your code before you continue.
π© Explore the intermediate solution in part1
, and run it with:
- TypeScript
- Java
- Go
- Python
npm run part1
./gradlew -PmainClass=dev.restate.tour.part1.AppMain run
go run ./part1
python3 -m hypercorn -b localhost:9080 tour/part1/app:app
Scheduling Async Tasksβ
When a handler calls another handler, Restate registers the call and makes sure it happens. You can also ask Restate to execute the call at a later point in the future, by adding a delay parameter to the call. Restate then registers the call and triggers it after the delay has passed.
In the application, a ticket gets reserved for 15 minutes. If the user doesn't pay within that time interval, then it becomes available again to other users.
- TypeScript
- Java
- Go
- Python
Let the CartObject/addTicket
handler call the CartObject/expireTicket
handler with a delay of 15 minutes:
To test it out, put the delay to a lower value, for example 5 seconds, call the addTicket
function, and see in the logs how the call to CartObject/expireTicket
is executed 5 seconds later.
Service logs
... logs from reserve call ...[restate] [CartObject/addTicket][inv_1gdJBtdVEcM90xbqbDEnOzNgilf2WmjZTP][2024-03-19T08:49:43.081Z] DEBUG: Received completion message from Restate, adding to journal. ; CompletionMessage[restate] [CartObject/addTicket][inv_1gdJBtdVEcM90xbqbDEnOzNgilf2WmjZTP][2024-03-19T08:49:43.081Z] DEBUG: Adding message to journal and sending to Restate ; BackgroundInvokeEntryMessage[restate] [CartObject/addTicket][inv_1gdJBtdVEcM90xbqbDEnOzNgilf2WmjZTP][2024-03-19T08:49:43.081Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage[restate] [CartObject/addTicket][inv_1gdJBtdVEcM90xbqbDEnOzNgilf2WmjZTP][2024-03-19T08:49:43.081Z] DEBUG: Function completed successfully.[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM93r8tce9IfwnbiAsk8lCevD][2024-03-19T08:49:48.092Z] DEBUG: Invoking function.[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM93r8tce9IfwnbiAsk8lCevD][2024-03-19T08:49:48.093Z] DEBUG: Adding message to journal and sending to Restate ; BackgroundInvokeEntryMessage[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM93r8tce9IfwnbiAsk8lCevD][2024-03-19T08:49:48.093Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage[restate] [CartObject/expireTicket][inv_1gdJBtdVEcM93r8tce9IfwnbiAsk8lCevD][2024-03-19T08:49:48.093Z] DEBUG: Function completed successfully.[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK529L4BRmz8ntFtiw2DkahH][2024-03-19T08:49:48.141Z] DEBUG: Invoking function.[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK529L4BRmz8ntFtiw2DkahH][2024-03-19T08:49:48.141Z] DEBUG: Journaled and sent output message ; OutputStreamEntryMessage[restate] [TicketObject/unreserve][inv_1k78Krj3GqrK529L4BRmz8ntFtiw2DkahH][2024-03-19T08:49:48.141Z] DEBUG: Function completed successfully.
Let the CartObject/addTicket
handler call the CartObject/expireTicket
handler with a delay of 15 minutes:
To test it out, put the delay to a lower value, for example 5 seconds, call the addTicket
function, and see in the logs how the call to CartObject/expireTicket
is executed 5 seconds later.
Service logs
... logs from reserve call ...2024-04-17 08:08:10 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH3fRqvARAGmeIcbyLXImG3L] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-17 08:08:10 INFO [CartObject/addTicket][inv_1aiqX0vFEFNH3fRqvARAGmeIcbyLXImG3L] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-17 08:08:15 DEBUG [CartObject/expireTicket] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CartObject/expireTicket2024-04-17 08:08:15 INFO [CartObject/expireTicket] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-17 08:08:15 DEBUG [CartObject/expireTicket][inv_1aiqX0vFEFNH5R28lg9wg1c3CtOJOhHEM9] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-17 08:08:15 DEBUG [CartObject/expireTicket][inv_1aiqX0vFEFNH5R28lg9wg1c3CtOJOhHEM9] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): BackgroundInvokeEntryMessage2024-04-17 08:08:15 DEBUG [CartObject/expireTicket][inv_1aiqX0vFEFNH5R28lg9wg1c3CtOJOhHEM9] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [2](): OutputEntryMessage2024-04-17 08:08:15 INFO [CartObject/expireTicket][inv_1aiqX0vFEFNH5R28lg9wg1c3CtOJOhHEM9] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-17 08:08:15 DEBUG [CartObject/expireTicket][inv_1aiqX0vFEFNH5R28lg9wg1c3CtOJOhHEM9] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-17 08:08:15 INFO [CartObject/expireTicket][inv_1aiqX0vFEFNH5R28lg9wg1c3CtOJOhHEM9] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-17 08:08:15 DEBUG [TicketObject/unreserve] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to TicketObject/unreserve2024-04-17 08:08:15 INFO [TicketObject/unreserve] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-17 08:08:15 DEBUG [TicketObject/unreserve][inv_1aAMfXkieWDz0btTCuaF2NHgJEdX2tXHCF] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-17 08:08:15 DEBUG [TicketObject/unreserve][inv_1aAMfXkieWDz0btTCuaF2NHgJEdX2tXHCF] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): OutputEntryMessage2024-04-17 08:08:15 INFO [TicketObject/unreserve][inv_1aAMfXkieWDz0btTCuaF2NHgJEdX2tXHCF] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-17 08:08:15 DEBUG [TicketObject/unreserve][inv_1aAMfXkieWDz0btTCuaF2NHgJEdX2tXHCF] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-17 08:08:15 INFO [TicketObject/unreserve][inv_1aAMfXkieWDz0btTCuaF2NHgJEdX2tXHCF] dev.restate.sdk.core.InvocationStateMachine - End invocation
Let the CartObject/AddTicket
handler call the CartObject/ExpireTicket
handler with a delay of 15 minutes:
To test it out, put the delay to a lower value, for example 5 seconds, call the AddTicket
function, and see in the logs how the call to CartObject/ExpireTicket
is executed 5 seconds later.
Service logs
2024/08/16 15:33:41 INFO Handling invocation method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp6JdmgIQ7cfkYv1aUgCa3ER2024/08/16 15:33:41 INFO Handling invocation method=TicketObject/Reserve invocationID=inv_19maBIcE9uRD3LcEEnLudAidH5TXVNerfP2024/08/16 15:33:41 INFO Invocation completed successfully method=TicketObject/Reserve invocationID=inv_19maBIcE9uRD3LcEEnLudAidH5TXVNerfP2024/08/16 15:33:41 INFO Invocation completed successfully method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp6JdmgIQ7cfkYv1aUgCa3ER2024/08/16 15:33:46 INFO Handling invocation method=CartObject/ExpireTicket invocationID=inv_1fmRNvSNVxNp2Je39FKxZGCuaWYqw2OvyV2024/08/16 15:33:46 INFO Invocation completed successfully method=CartObject/ExpireTicket invocationID=inv_1fmRNvSNVxNp2Je39FKxZGCuaWYqw2OvyV2024/08/16 15:33:46 INFO Handling invocation method=TicketObject/Unreserve invocationID=inv_19maBIcE9uRD72CK05c2mkJvQZr352Qhvr2024/08/16 15:33:46 INFO Invocation completed successfully method=TicketObject/Unreserve invocationID=inv_19maBIcE9uRD72CK05c2mkJvQZr352Qhvr
Let the CartObject/addTicket
handler call the CartObject/expireTicket
handler with a delay of 15 minutes:
To test it out, put the delay to a lower value, for example 5 seconds, call the addTicket
function, and see in the Restate Server logs how the call to CartObject/expireTicket
is executed 5 seconds later.
Don't forget to set the delay back to 15 minutes.
Durable timers are a powerful feature that can be used to implement workflows, schedule async tasks, or plan background jobs. Restate makes them resilient to failures and ensures that they get executed. No extra infrastructure needed!
Another timer-like feature of the SDK is suspendable sleep. Restate will make sure that the function gets resumed after the specified duration has passed. When running on function-as-a-service platforms, your function can suspend in the meantime, so you don't pay for the wait time.
- TypeScript
- Java
- Go
- Python
await ctx.sleep(15 * 60 * 1000);
ctx.sleep(Duration.ofMinutes(15));
if err := restate.Sleep(ctx, 15*time.Minute); err != nil { return err}
await ctx.sleep(delta=timedelta(minutes=15))
π© Explore the intermediate solution in part2
, and run it with:
- TypeScript
- Java
- Go
- Python
npm run part2
./gradlew -PmainClass=dev.restate.tour.part2.AppMain run
go run ./part2
python3 -m hypercorn -b localhost:9080 tour/part2/app:app
Virtual Objects vs. Servicesβ
At the beginning of this tutorial, we mentioned that the TicketObject
and CartObject
services are Virtual Objects.
Virtual Objects are identified by a key and allow you to store K/V state in Restate. For each Virtual Object (key), only one invocation can run at a time (across all the handlers of that Virtual Object).
Services, on the other hand, do not have access to K/V state, and handlers can run concurrently.
With access to consistent K/V state and strong concurrency guarantees, implementing the TicketObject
in a resilient and consistent way becomes straightforward.
When a user reserves a ticket, we want to be sure that no concurrent other requests are reserving the same ticket at the same time.
To get this behaviour, we key the TicketObject
on ticket ID. We now have a single Virtual Object per ticket.
If you do long-running operations in a Virtual Object, no other invocations are processed the meantime.
For example, if you would implement the expiration of the ticket in the CartObject
service by sleeping for 15 minutes:
- TypeScript
- Java
- Go
- Python
await ctx.sleep(15 * 60 * 1000);ctx.objectSendClient(TicketObject, ticketId).unreserve();
ctx.sleep(Duration.ofMinutes(15));TicketObjectClient.fromContext(ctx, ticketId).send().unreserve();
if err := restate.Sleep(ctx, 15*time.Minute); err != nil { return err}restate.ObjectSend(ctx, "TicketObject", ticketId, "unreserve").Send(restate.Void{})
await ctx.sleep(delta=timedelta(minutes=15))ctx.object_send(unreserve, key=ticket_id, arg=None)
The user wouldn't be able to add any other tickets, nor buy the tickets. If you do a delayed call, the invocation isn't ongoing until the delay has passed, so the Virtual Object is not locked.
Consistent K/V stateβ
Restate offers a key-value store to store application state for Virtual Objects.
Restate's state is guaranteed to be consistent across retries and invocations. This eliminates the need for a session database.
Getting and setting K/V stateβ
- TypeScript
- Java
- Go
- Python
Adapt the CartObject/addTicket
function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
To retrieve the cart, you use ctx.get
.
This returns null
if the value has never been set.
After you added the ticket to the cart array, you set the state to the new value with ctx.set
.
Adapt the CartObject/addTicket
function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
To retrieve the cart, you use ctx.get
with a state key that describes the name and the (de)serializers to be used.
ctx.get
returns an Optional, only containing a value if one was set before.
After you added the ticket to the cart array, you set the state to the new value with ctx.set
.
Adapt the CartObject/AddTicket
function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
To retrieve the cart, you use restate.Get
.
This returns the zero value if there's no value for this key; a nil slice is a useful result in this case.
After you added the ticket to the cart array, you set the state to the new value with restate.Set
.
Adapt the CartObject/addTicket
function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
To retrieve the cart, you use ctx.get
.
This returns null
if the value has never been set.
After you added the ticket to the cart array, you set the state to the new value with ctx.set
.
You can store multiple key-value pairs, by using different state keys.
Here, you get the value under the key "tickets"
.
Restate returns the cart belonging to the current Virtual Object (for example, user Mary
).
- TypeScript
- Java
- Go
- Python
Run the services and call the addTicket
function, to see the interaction with state.
Service logs
... logs from reserve call ...[restate] [CartObject/addTicket][inv_1gdJBtdVEcM97yGYEG5NWYtbMlnSAGHGY9][2024-03-19T08:55:20.941Z] DEBUG: Adding message to journal and sending to Restate ; GetStateEntryMessage[restate] [CartObject/addTicket][inv_1gdJBtdVEcM97yGYEG5NWYtbMlnSAGHGY9][2024-03-19T08:55:20.941Z] DEBUG: Adding message to journal and sending to Restate ; SetStateEntryMessage... logs from expireTicket call ...
Run the services and call the addTicket
function, to see the interaction with state.
Service logs
... reserve call ...2024-04-17 08:13:23 DEBUG [CartObject/addTicket] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CartObject/addTicket2024-04-17 08:13:23 INFO [CartObject/addTicket] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-17 08:13:23 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH7nJWPQ0NFyGKHOyNmoE3hn] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYING2024-04-17 08:13:23 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH7nJWPQ0NFyGKHOyNmoE3hn] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [1](): InvokeEntryMessage2024-04-17 08:13:23 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH7nJWPQ0NFyGKHOyNmoE3hn] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [2](): GetStateEntryMessage2024-04-17 08:13:23 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH7nJWPQ0NFyGKHOyNmoE3hn] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [3](): SetStateEntryMessage2024-04-17 08:13:23 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH7nJWPQ0NFyGKHOyNmoE3hn] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [4](): BackgroundInvokeEntryMessage2024-04-17 08:13:23 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH7nJWPQ0NFyGKHOyNmoE3hn] dev.restate.sdk.core.InvocationStateMachine - Current journal entry [5](): OutputEntryMessage2024-04-17 08:13:23 INFO [CartObject/addTicket][inv_1aiqX0vFEFNH7nJWPQ0NFyGKHOyNmoE3hn] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-17 08:13:23 DEBUG [CartObject/addTicket][inv_1aiqX0vFEFNH7nJWPQ0NFyGKHOyNmoE3hn] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-17 08:13:23 INFO [CartObject/addTicket][inv_1aiqX0vFEFNH7nJWPQ0NFyGKHOyNmoE3hn] dev.restate.sdk.core.InvocationStateMachine - End invocation
Run the services with TRACE loglevel (slog.SetLogLoggerLevel(-8)
in Go >= 1.22) and call the AddTicket
function, to see the interaction with state.
Service logs
... logs from reserve call ...2024/08/16 16:25:37 DEBUG-4 Sending message to runtime method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp62ejaow32hBZDNwwSzc63v type=GetStateEntryMessage2024/08/16 16:25:37 DEBUG Processed completion method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp62ejaow32hBZDNwwSzc63v index=12024/08/16 16:25:37 DEBUG-4 Sending message to runtime method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp62ejaow32hBZDNwwSzc63v type=SetStateEntryMessage2024/08/16 16:25:37 DEBUG-4 Sending message to runtime method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp62ejaow32hBZDNwwSzc63v type=OneWayCallEntryMessage2024/08/16 16:25:37 INFO Invocation completed successfully method=CartObject/AddTicket invocationID=inv_1fmRNvSNVxNp62ejaow32hBZDNwwSzc63v
When starting the invocation, Restate attaches the application state to the request. So when you operate on the state in your function, you get access to a local copy of the state for fast access.
- TypeScript
- Java
- Go
- Python
Also adapt the CartObject/checkout
function, to use the tickets:
After the tickets are checked out, you clear the state with ctx.clear
.
Also adapt the CartObject/checkout
function, to use the tickets:
After the tickets are checked out, you clear the state with ctx.clear
.
Also adapt the CartObject/Checkout
function, to use the tickets:
After the tickets are checked out, you clear the state with restate.Clear
.
Also adapt the CartObject/checkout
function, to use the tickets:
After the tickets are checked out, you clear the state with ctx.clear
.
Inspecting K/V stateβ
Restate exposes information on invocations and application state.
You can watch the state of the CartObject
service, via:
Output
π€ State:βββββββββService CartObjectKey MaryKEY VALUEtickets [ "seat2B" ]
Add some tickets to the state to see how the query result gets updated.
Then, send a checkout request as earlier, and notice that the state is now empty.
π Try it outβ
- TypeScript
- Java
- Go
- Python
Finishing CartObject/expireTicket
β
You have almost fully implemented the CartObject
. Let's finish CartObject/expireTicket
.
Before you call unreserve
, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/unreserve
and remove it from the state.
Solution
Call the expireTicket
handler with:
curl localhost:8080/CartObject/Mary/expireTicket -H 'content-type: application/json' -d '"seat2B"'
Finishing CartObject/expireTicket
β
You have almost fully implemented the CartObject
. Let's finish CartObject/expireTicket
.
Before you call unreserve
, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/unreserve
and remove it from the state.
Solution
Call the expireTicket
handler with:
curl localhost:8080/CartObject/Mary/expireTicket -H 'content-type: application/json' -d '"seat2B"'
Finishing CartObject/ExpireTicket
β
You have almost fully implemented the CartObject
. Let's finish CartObject/ExpireTicket
.
Before you call Unreserve
, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/Unreserve
and remove it from the state.
Solution
Call the ExpireTicket
handler with:
curl localhost:8080/CartObject/Mary/ExpireTicket -H 'content-type: application/json' -d '"seat2B"'
Finishing CartObject/expireTicket
β
You have almost fully implemented the CartObject
. Let's finish CartObject/expireTicket
.
Before you call unreserve
, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/unreserve
and remove it from the state.
Solution
Call the expireTicket
handler with:
curl localhost:8080/CartObject/Mary/expireTicket -H 'content-type: application/json' -d '"seat2B"'
Implementing the TicketObject
β
Track the status of the tickets in the TicketObject
by storing it in the state and transitioning from one state to another, like a state machine.
The possible states are available (default), reserved, and sold.
Implement the handlers in the TicketObject
to reserve, unreserve, and mark a ticket as sold.
While you are developing them, monitor the state of the TicketObject
via:
- TypeScript
- Java
- Go
- Python
TicketObject/reserve
- Retrieve the value for the
"status"
state key. - If the value is set to
TicketStatus.Available
, then change it toTicketStatus.Reserved
and returntrue
(reservation successful). - If the status isn't set to
TicketStatus.Available
, then returnfalse
.
Solution
Now, you can't reserve the same ticket multiple times anymore.
Call addTicket
multiple times for the same ID. The first time it returns true
, afterwards false
.
TicketObject/unreserve
Clear the "status"
, if it's not equal to TicketStatus.Sold
.
Solution
Now, the ticket reservation status is cleared when the delayed expireTicket
call triggers.
Play around with reducing the delay of the expireTicket
call in the addTicket
handler.
Try to reserve the same ticket ID multiple times, and see how you are able to reserve it again after the unreserve
handler executed.
TicketObject/markAsSold
Set the "status"
to TicketStatus.Sold
if it's reserved.
Solution
In the next section, you implement the CheckoutService/handle
function that calls markAsSold
.
This ties the final parts together.
TicketObject/reserve
- Retrieve the value for the
"status"
state key. - If the value is set to
TicketStatus.Available
, then change it toTicketStatus.Reserved
and returntrue
(reservation successful). - If the status isn't set to
TicketStatus.Available
, then returnfalse
.
Solution
Now, you can't reserve the same ticket multiple times anymore.
Call addTicket
multiple times for the same ID. The first time it returns true
, afterwards false
.
TicketObject/unreserve
Clear the "status"
, if it's not equal to TicketStatus.Sold
.
Solution
Now, the ticket reservation status is cleared when the delayed expireTicket
call triggers.
Play around with reducing the delay of the expireTicket
call in the addTicket
handler.
Try to reserve the same ticket ID multiple times, and see how you are able to reserve it again after the unreserve
handler executed.
TicketObject/markAsSold
Set the "status"
to TicketStatus.Sold
if it's reserved.
Solution
In the next section, you implement the CheckoutService/handle
function that calls markAsSold
.
This ties the final parts together.
TicketObject/Reserve
- Retrieve the value for the
"status"
state key. - If the value is set to
auxiliary.TicketStatusAvailable
, then change it toTicketStatusReserved
and returntrue
(reservation successful). - If the status isn't set to
TicketStatusAvailable
, then returnfalse
.
Solution
Now, you can't reserve the same ticket multiple times anymore.
Call AddTicket
multiple times for the same ID. The first time it returns true
, afterwards false
.
TicketObject/Unreserve
Clear the "status"
, if it's not equal to auxiliary.TicketStatusSold
.
Solution
Now, the ticket reservation status is cleared when the delayed ExpireTicket
call triggers.
Play around with reducing the delay of the ExpireTicket
call in the AddTicket
handler.
Try to reserve the same ticket ID multiple times, and see how you are able to reserve it again after the Unreserve
handler executed.
TicketObject/MarkAsSold
Set the "status"
to auxiliary.TicketStatusSold
if it's reserved.
Solution
In the next section, you implement the CheckoutService/Handle
function that calls MarkAsSold
.
This ties the final parts together.
TicketObject/reserve
- Retrieve the value for the
"status"
state key. - If the value is set to
"AVAILABLE"
, then change it to"RESERVED"
and returntrue
(reservation successful). - If the status isn't set to
"AVAILABLE"
, then returnfalse
.
Solution
Now, you can't reserve the same ticket multiple times anymore.
Call addTicket
multiple times for the same ID. The first time it returns true
, afterwards false
.
TicketObject/unreserve
Clear the "status"
, if it's not equal to "SOLD"
.
Solution
Now, the ticket reservation status is cleared when the delayed expireTicket
call triggers.
Play around with reducing the delay of the expireTicket
call in the addTicket
handler.
Try to reserve the same ticket ID multiple times, and see how you are able to reserve it again after the unreserve
handler executed.
TicketObject/markAsSold
Set the "status"
to "SOLD"
if it's reserved.
Solution
In the next section, you implement the CheckoutService/handle
function that calls markAsSold
.
This ties the final parts together.
π© Explore the intermediate solution in part3
, and run it with:
- TypeScript
- Java
- Go
- Python
npm run part3
./gradlew -PmainClass=dev.restate.tour.part3.AppMain run
go run ./part3
python3 -m hypercorn -b localhost:9080 tour/part3/app:app
Journaling actionsβ
Restate's Durable Execution mechanism tracks the progress of the code execution in a journal. Once an action/result has made it to the journal, it will not be re-executed on retries.
- TypeScript
- Java
- Go
- Python
You can store the return value of any function in the journal, by using ctx.run
.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
The SDK also offers helper functions for creating UUIDs and generating random numbers.
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use ctx.run
to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/handle
:
You can store the return value of any function in the journal, by using ctx.run
.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
The SDK also offers helper functions for creating UUIDs and generating random numbers.
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use ctx.run
to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/handle
:
You can store the return value of any function in the journal, by using restate.Run
.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
The SDK also offers helper functions for creating UUIDs and generating random numbers.
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use restate.Run
to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/Handle
:
You can store the return value of any function in the journal, by using ctx.run
.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use ctx.run
to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/handle
:
Generate an idempotency token
- TypeScript
- Java
- Go
- Python
Let's use the SDK helper functions to generate a unique payment identifier and store it in Restate. Once the token is stored, it will be the same on retries. Try it out by printing the idempotency key and then throwing an error:
Call CartObject/checkout
and have a look at the logs to see what happens.
Service logs
... logs of `CartObjectService/CheckoutService` ...[restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.498Z] DEBUG: Invoking function.My idempotency key: e25b747f-ecfb-443b-8939-1935392aab6bTrace: [restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.499Z] TRACE: Function completed with an error: Something happened! Error: Something happened!... rest of trace ...[restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.512Z] DEBUG: Invocation ended with retryable error. ; ErrorMessage[restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.568Z] DEBUG: Invoking function.My idempotency key: e25b747f-ecfb-443b-8939-1935392aab6bTrace: [restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.568Z] TRACE: Function completed with an error: Something happened! Error: Something happened!... rest of trace ...[restate] [CheckoutService/handle][inv_1jhuapyO2Bpg3prqzrAJOFs99mt7jv5x3r][2024-03-19T09:15:30.568Z] DEBUG: Invocation ended with retryable error. ; ErrorMessage... retries continue ...
Let's use the SDK helper functions to generate a unique payment identifier and store it in Restate. Once the token is stored, it will be the same on retries. Try it out by printing the idempotency key and then throwing an error:
Call CartObject/checkout
and have a look at the logs to see what happens.
Service logs
2024-04-17 08:33:52 DEBUG [CheckoutService/handle] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CheckoutService/handle2024-04-17 08:33:52 INFO [CheckoutService/handle] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-17 08:33:52 DEBUG [CheckoutService/handle][inv_155UJNoky4WU38J6ReFTcsOP0S1XiFjWWl] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYINGMy idempotency key: e43f57b8-ab19-27f2-3693-2e7dd6bda3992024-04-17 08:33:52 WARN [CheckoutService/handle][inv_155UJNoky4WU38J6ReFTcsOP0S1XiFjWWl] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Error when processing the invocationjava.lang.IllegalStateException: The handler failed... rest of trace ...2024-04-17 08:33:52 DEBUG [CheckoutService/handle][inv_155UJNoky4WU38J6ReFTcsOP0S1XiFjWWl] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to CLOSED2024-04-17 08:33:52 INFO [CheckoutService/handle][inv_155UJNoky4WU38J6ReFTcsOP0S1XiFjWWl] dev.restate.sdk.core.InvocationStateMachine - End invocation2024-04-17 08:33:52 DEBUG [CheckoutService/handle] dev.restate.sdk.http.vertx.RequestHttpServerHandler - Handling request to CheckoutService/handle2024-04-17 08:33:52 INFO [CheckoutService/handle] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Start processing invocation2024-04-17 08:33:52 DEBUG [CheckoutService/handle][inv_155UJNoky4WU38J6ReFTcsOP0S1XiFjWWl] dev.restate.sdk.core.InvocationStateMachine - Transitioning state machine to REPLAYINGMy idempotency key: e43f57b8-ab19-27f2-3693-2e7dd6bda3992024-04-17 08:33:52 WARN [CheckoutService/handle][inv_155UJNoky4WU38J6ReFTcsOP0S1XiFjWWl] dev.restate.sdk.core.ResolvedEndpointHandlerImpl - Error when processing the invocationjava.lang.IllegalStateException: The handler failed... rest of trace ...
Let's use the SDK helper functions to generate a unique payment identifier and store it in Restate. Once the token is stored, it will be the same on retries. Try it out by printing the idempotency key and then throwing an error:
Call CartObject/Checkout
and have a look at the logs to see what happens.
Service logs
2024/08/16 17:14:53 INFO Handling invocation method=CheckoutService/Handle invocationID=inv_1lPHOWXVSzKi3YKaekWZqw8vVMjyli4utzGenerated idempotency key 9594910a-3612-4888-9e0a-c3ae58fc2dce2024/08/16 17:14:53 ERROR Invocation returned a non-terminal failure method=CheckoutService/Handle invocationID=inv_1lPHOWXVSzKi3YKaekWZqw8vVMjyli4utz err="Something happened!"2024/08/16 17:14:53 INFO Handling invocation method=CheckoutService/Handle invocationID=inv_1lPHOWXVSzKi3YKaekWZqw8vVMjyli4utzGenerated idempotency key 9594910a-3612-4888-9e0a-c3ae58fc2dce2024/08/16 17:14:53 ERROR Invocation returned a non-terminal failure method=CheckoutService/Handle invocationID=inv_1lPHOWXVSzKi3YKaekWZqw8vVMjyli4utz err="Something happened!"
Let's use ctx.run
to generate a unique payment identifier and store it in Restate.
Once the token is stored, it will be the same on retries.
Try it out by printing the idempotency key and then throwing an error:
Call CartObject/checkout
and have a look at the logs to see what happens.
Service logs
My idempotency key is: 84452572-5d8a-48ea-91a5-e3e6f011b4ebTraceback (most recent call last):... rest of trace ...raise Exception("Something happened!")Exception: Something happened!My idempotency key is: 84452572-5d8a-48ea-91a5-e3e6f011b4ebTraceback (most recent call last):... rest of trace ...raise Exception("Something happened!")Exception: Something happened!My idempotency key is: 84452572-5d8a-48ea-91a5-e3e6f011b4eb... retries continue ...
Trigger the payment
- TypeScript
- Java
- Go
- Python
Execute the payment via an external payment provider via PaymentClient.get().call(idempotencyKey, amount)
.
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollars.
Execute the payment via an external payment provider via PaymentClient.get().call(idempotencyKey, amount)
.
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollars.
Execute the payment via an external payment provider via auxiliary.PaymentClient{}.Call(idempotencyKey, amount)
.
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollars.
Execute the payment via an external payment provider via payment_client.call(idempotency_key, total_price)
.
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollars.
π Try it outβ
Let's finish the checkout flow by sending the email notifications and marking the tickets as sold.
- TypeScript
- Java
- Go
- Python
Implement the email notifications
After the CheckoutService/handle
handler has handled the payment, you need to notify the users of the payment status:
- Payment success: notify the users via
EmailClient.get().notifyUserOfPaymentSuccess(request.getUserId())
. - Payment failure: notify the users via the
EmailClient.get().notifyUserOfPaymentFailure(request.getUserId())
.
Solution
Mark tickets as sold
Let the CartObject/checkout
handler mark all tickets as sold by calling TicketObject/markAsSold
for each ticket.
Solution
Implement the email notifications
After the CheckoutService/handle
handler has handled the payment, you need to notify the users of the payment status:
- Payment success: notify the users via
EmailClient.get().notifyUserOfPaymentSuccess(request.getUserId())
. - Payment failure: notify the users via the
EmailClient.get().notifyUserOfPaymentFailure(request.getUserId())
.
Solution
Mark tickets as sold
Let the CartObject/checkout
handler mark all tickets as sold by calling TicketObject/markAsSold
for each ticket.
Solution
Implement the email notifications
After the CheckoutService/Handle
handler has handled the payment, you need to notify the users of the payment status:
- Payment success: notify the users via
EmailClient{}.NotifyUserOfPaymentSuccess(request.UserId)
. - Payment failure: notify the users via the
EmailClient{}.NotifyUserOfPaymentFailure(request.UserId)
.
Solution
Mark tickets as sold
Let the CartObject/checkout
handler mark all tickets as sold by calling TicketObject/markAsSold
for each ticket.
Solution
Implement the email notifications
After the CheckoutService/handle
handler has handled the payment, you need to notify the users of the payment status:
- Payment success: notify the users via
email_client.notify_user_of_payment_success(order['user_id'])
. - Payment failure: notify the users via the
email_client.notify_user_of_payment_failure(order['user_id'])
.
Solution
Mark tickets as sold
Let the CartObject/checkout
handler mark all tickets as sold by calling TicketObject/markAsSold
for each ticket.
Solution
π₯³ You have now fully implemented the ticket reservation system! Try it out by reserving some new tickets and buying them by checking out the cart.
π© Explore the intermediate solution in part4
, and run it with:
- TypeScript
- Java
- Go
- Python
npm run part4
./gradlew -PmainClass=dev.restate.tour.part4.AppMain run
go run ./part4
python3 -m hypercorn -b localhost:9080 tour/part4/app:app
Idempotency for any requestβ
As you saw, generating idempotency keys inside your handlers and storing them in Restate is easy. But this doesn't guard us yet against retries of the HTTP request to Restate.
- TypeScript
- Java
- Go
- Python
For example, if the caller of the addTicket
handler didn't receive the success response of its first request, it might retry the request.
For example, if the caller of the addTicket
handler didn't receive the success response of its first request, it might retry the request.
For example, if the caller of the AddTicket
handler didn't receive the success response of its first request, it might retry the request.
For example, if the caller of the addTicket
handler didn't receive the success response of its first request, it might retry the request.
The second request will return false
because the ticket already got reserved the first time, but the caller won't know about this.
To cover this, you can add an idempotency-key
header to the incoming request to let Restate deduplicate them.
- TypeScript
- Java
- Go
- Python
In our example, when we call the CartObject/addTicket
handler, the first time the response is true
and the second time it's false
.
However, if we use the same idempotency key, the second call will return true
as well, because it will return the result of the first call:
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' \ -H 'idempotency-key: ad5472esg4dsg525dssdfa5loi' \ -d '"seat2C"'
In our example, when we call the CartObject/addTicket
handler, the first time the response is true
and the second time it's false
.
However, if we use the same idempotency key, the second call will return true
as well, because it will return the result of the first call:
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' \ -H 'idempotency-key: ad5472esg4dsg525dssdfa5loi' \ -d '"seat2C"'
In our example, when we call the CartObject/AddTicket
handler, the first time the response is true
and the second time it's false
.
However, if we use the same idempotency key, the second call will return true
as well, because it will return the result of the first call:
curl localhost:8080/CartObject/Mary/AddTicket -H 'content-type: application/json' \-H 'idempotency-key: ad5472esg4dsg525dssdfa5loi' \-d '"seat2C"'
In our example, when we call the CartObject/addTicket
handler, the first time the response is true
and the second time it's false
.
However, if we use the same idempotency key, the second call will return true
as well, because it will return the result of the first call:
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' \-H 'idempotency-key: ad5472esg4dsg525dssdfa5loi' \-d '"seat2C"'
You can also see from the service logs that the handler wasn't executed the second time.
Restate gives you idempotency for any service, handler and request for free. No extra setup.
You only need this when invoking handlers over HTTP. When a handler calls another handler, Restate automatically takes care of the idempotency.
Tracingβ
Restate exposes OpenTelemetry traces of your invocations.
Run Jaeger
docker run -d --name jaeger -e COLLECTOR_OTLP_ENABLED=true \ -p 4317:4317 -p 16686:16686 jaegertracing/all-in-one:1.46
Relaunch Restate with tracing enabled
Send a few requests
- TypeScript
- Java
- Go
- Python
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2A"'curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2B"'curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2C"'curl -X POST localhost:8080/CartObject/Mary/checkout
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2A"'curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2B"'curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2C"'curl -X POST localhost:8080/CartObject/Mary/checkout
curl localhost:8080/CartObject/Mary/AddTicket -H 'content-type: application/json' -d '"seat2A"'curl localhost:8080/CartObject/Mary/AddTicket -H 'content-type: application/json' -d '"seat2B"'curl localhost:8080/CartObject/Mary/AddTicket -H 'content-type: application/json' -d '"seat2C"'curl -X POST localhost:8080/CartObject/Mary/Checkout
curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2A"'curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2B"'curl localhost:8080/CartObject/Mary/addTicket -H 'content-type: application/json' -d '"seat2C"'curl -X POST localhost:8080/CartObject/Mary/checkout
Go to the Jaeger UI
Inspect the traces
Select the CartObject
service from the service dropdown.
- TypeScript
- Java
- Go
- Python
You should see the addTicket
and checkout
requests listed.
Have a look at the traces of the checkout
call:
You should see the addTicket
and checkout
requests listed.
Have a look at the traces of the checkout
call:
You should see the AddTicket
and Checkout
requests listed.
Have a look at the traces of the Checkout
call:
You should see the addTicket
and checkout
requests listed.
Have a look at the traces of the checkout
call:
You can see the calls that were done to Restate, for example invoke, sleep, one way call, get state, etc., and their timings. If you expand one of the traces, you can see tags describing some metadata of the context call, for example invocation ID and the request.
For more information, have a look at the tracing docs.
π The endβ
You reached the end of this tutorial!
Let's recap what you did! You have built a ticket reservation system that is resilient, consistent, and scalable. We used Restate to provide us with durable, distributed building blocks to simplify the implementation of the system. Let's list a few of them:
What you implemented | What you didn't implement, as Restate handles it for you |
---|---|
β Request-response invocations | β Handling retries, timeouts, etc. |
β Sending messages | β Deploy and operate message queues for async requests |
β Idempotent HTTP calls | β Write deduplication logic |
β Durable Execution: retries, partial progress recovery, and suspensions | β Manual retry logic and partial progress recovery |
β Durable timers: sleeping and scheduling async tasks | β Workflow orchestrators or cron jobs for scheduling tasks |
β Virtual Objects: concurrency guarantees and shared state | β Guards for keeping state consistent across retries, concurrent requests, and scaling out. |
β K/V state: storing and inspecting | β Session databases for state. State consistency guards. |
β Storing computation results in the journal | β Logic to make operations idempotent (e.g. generate idempotency keys) |
You now know the essentials to start developing Restate services! Have a look at the next steps to explore further.
Next stepsβ
- Run the examples
- Read the Concepts: although most of this has been covered in this tutorial
- Check out the Java SDK documentation
- Check out the TypeScript SDK documentation
- Check out the Go SDK documentation