You can use the UI to debug your applications.
Have a look at the UI announcement blog post to get some inspiration on how you can use the UI for debugging and understanding your applications.
SQL over the data in Restate
Restate exposes the following SQL tables:sys_invocation
: to inspect invocationssys_inbox
: to inspect queue of pending invocationssys_keyed_service_status
: to inspect the status of a Virtual Objectsys_journal
: to inspect the invocations’ journalsys_service
: to inspect the registered servicessys_deployment
: to inspect service deploymentssys_idempotency
: to inspect idempotency keysstate
: to inspect application state
Inspecting invocations
You can execute SQL queries via the CLI or over HTTP.Listing ongoing invocations
Listing ongoing invocations
For each of the queries we will show the CLI command and the equivalent SQL query that you can execute via the CLI or over HTTP.Restate only retains the entries for active invocations, workflows or invocations that were invoked with an idempotency key.
Active invocations are invocations that haven’t completed yet and are either invoked or suspended.
For workflows and invocations that were invoked with an idempotency key, the entries are retained for their specified retention time.The CLI command only shows the active invocations, not the completed ones. Use
--all
to see completed ones as well.Retrieving the status of an invocation
Retrieving the status of an invocation
pending
: enqueued waiting for its turnready
: ready to be processed, but not yet runningrunning
: actively processingbacking-off
: retrying due to a failuresuspended
: waiting on some external input (e.g. request-response call, awakeable, sleep, …)completed
: completed (this is shown only for idempotent invocations)
Inspecting the invocation journal
Inspecting the invocation journal
Inspecting invocation retries
Inspecting invocation retries
To have a look at the invocations that are currently in a retry loop, you can execute:
Listing invocations that are blocking a Virtual Object
Listing invocations that are blocking a Virtual Object
You can retrieve the invocation ID that is currently blocking a Virtual Object via:With the CLI, you can also drill down and list only invocations that are blocking any Virtual Object:Add
--key my-key
to list only invocations that are blocking a specific Virtual Object.You can then use the invocation ID to cancel the invocation.Checking the last time an invocation was modified
Checking the last time an invocation was modified
invoked
to suspended
, or when the last journal entry was added.Checking how an invocation was triggered
Checking how an invocation was triggered
To find out if an invocation was triggered via the ingress or by another service:With the CLI, you see the caller at the root of the tree in the invocation progress:For the SQL queries, the
invoked_by
field contains either ingress
or service
.
If the invocation was triggered by another service, then the fields invoked_by_service_name
and invoked_by_id
will supply more information about the invoking service.Retrieving the trace ID of an invocation
Retrieving the trace ID of an invocation
Listing inactive invocations
Listing inactive invocations
To list the oldest invocations that are not making progress:
Listing zombie invocations
Listing zombie invocations
Zombie invocations are invocations that are pinned to a specific deployment but that deployment was forcefully removed. You can list them by executing:For the SQL queries, you need to know the deployment ID of the deployment that was forcefully removed.
Inspecting application state
With the CLI
To retrieve the state of a specific service and service key, do:--binary
flag,
and get the value as base64 encoded string.
With SQL queries
You can query the application state via thestate
table.
value_utf8
.
To retrieve the state of a specific service name, service key and state key, do:
ctx.set("count", 1)
stores 1
under the key count
.
Joining the sys_invocation and state table
Joining the sys_invocation and state table
To join the
sys_invocation
and state
table:Edit application state
You can edit the application state either via the state tab of the UI or via the CLI:cli env
).
It sends the new state values back to the runtime to be applied.
Use --binary
if the values are not JSON-encoded UTF-8 strings.
In this case, you need to decode the base64-encoded string, and encode it back to base64 after editing.
Use --plain
to retrieve the state as a JSON object.
This can be useful in combination with tools like jq
for example:
If during the editing of the state with the CLI, an invocation changed the state as well, then the edit of the CLI will not take affect.
If you want the CLI state edit to be applied even if the state has changed in the meantime, then use the
--force
flag.counter
for the key bob
: