Connect your Restate handlers to Kafka topics. Restate takes care of Kafka consumer management and pushes events to your Restate handlers. You get zero-overhead consumer management with automatic retries, durable execution, and stateful processing capabilities.
Each event leads to an invocation, meaning a request to execute a handler. Each invocation has its own unique ID and lifecycle. Have a look at managing invocations to learn how to manage the lifecycle of an invocation.

​
Invoking Handlers via Kafka Events

You can invoke handlers via Kafka events, by doing the following:
1

Develop and register an event handler

You can invoke any handler via Kafka events. The event payload will be (de)serialized as JSON.
  • When invoking Virtual Object or Workflow handlers via Kafka, the key of the Kafka record will be used to determine the Virtual Object/Workflow key. The key needs to be a valid UTF-8 string. The events are delivered to the subscribed handler in the order in which they arrived on the topic partition.
  • When invoking Virtual Object or Workflow shared handlers via Kafka, the key of the Kafka record will be used to determine the Virtual Object/Workflow key. The key needs to be a valid UTF-8 string. The events are delivered to the subscribed handler in parallel without ordering guarantees.
  • When invoking Service handlers over Kafka, events are delivered in parallel without ordering guarantees.
Since you can invoke any handler via Kafka events, a single handler can be invoked both by RPC and via Kafka.
2

Configure Restate to connect to a Kafka cluster

Define the Kafka cluster that Restate needs to connect to in the Restate configuration file:
restate.toml
[[ingress.kafka-clusters]]
name = "my-cluster"
brokers = ["PLAINTEXT://broker:9092"]
And make sure the Restate Server uses it via restate-server --config-file restate.toml.Check the configuration docs for more details.
3

Register the service you want to invoke.

4

Subscribe the event handler to the Kafka topic

Let Restate forward events from the Kafka topic to the event handler by creating a subscription:
curl localhost:9070/subscriptions --json '{
"source": "kafka://my-cluster/my-topic",
"sink": "service://MyService/handle",
"options": {"auto.offset.reset": "earliest"}
}'
Once you’ve created a subscription, Restate immediately starts consuming events from Kafka. The handler will be invoked for each event received from Kafka.The options field is optional and accepts any configuration parameter from librdkafka configuration.

​
Managing Kafka Subscriptions

Restate can trigger handlers via Kafka events.

​
Create Subscriptions

Subscribe a handler to a Kafka topic: 
curl localhost:9070/subscriptions --json '{
"source": "kafka://my-cluster/my-topic",
"sink": "service://MyService/Handle",
"options": {"auto.offset.reset": "earliest"}
}'
The options field is optional and accepts any librdkafka configuration parameter.

​
List Subscriptions

View current subscriptions: 
curl localhost:9070/subscriptions
Example response: 
{
"subscriptions": [
{
"id": "sub_11XHoawrCiWtv8kzhEyGtsR",
"source": "kafka://my-cluster/my-topic",
"sink": "service://Greeter/greet",
"options": {
"auto.offset.reset": "earliest",
"client.id": "restate",
"group.id": "sub_11XHoawrCiWtv8kzhEyGtsR"
}
}
]
}

​
Delete Subscriptions

Remove a subscription using its ID (starts with sub_): 
curl -X DELETE localhost:9070/subscriptions/sub_11XHoawrCiWtv8kzhEyGtsR
When you delete a subscription, Restate stops the associated consumer group. Messages already enqueued by Restate will still be processed.