Skip to main content
In this guide, you will learn how to connect your Restate service handler to a Kafka topic. Restate lets you connect any handler to a Kafka topic, and invoke the handler for each event that arrives on the topic. This way, you can use Restate to process events in a lightweight, flexible, transactional way.
1

Get the Greeter service template and run the service

You can choose any of the SDK languages for this quickstart. You do not need to adapt the handler code to be able to read from Kafka. Any handler can be connected to Kafka.
2

Start the Kafka cluster

Let’s start a Kafka cluster with a single broker, and create a topic named greetings.You can run your Kafka cluster in your preferred way. Here, we will use Docker Compose to start a Kafka cluster.Create a docker-compose.yaml file with the following content:
docker-compose.yaml
Start the Kafka broker:
3

Running Restate Server

Now, let’s start the Restate Server:
4

Register the Kafka cluster

Let the Restate Server know about the Kafka cluster by registering it via the CLI:
5

Register the service

Let the Restate Server know about the Greeter service by registering it:
6

Connect the handler to the topic

Now, we need to make Restate subscribe to the Kafka topics and tell it where it should push the events that arrive on the topic.Execute the following command to create a subscription, and invoke the handler for each event:
For Go, you need to capitalize the handler name: service://Greeter/Greet.This command tells Restate to invoke the greet handler of the Greeter service for each event that arrives on the greetings topic in the my-cluster Kafka cluster.
7

Invoke the handler by publishing an event

Create a Kafka producer and publish an event to the greetings topic:
Then type a message and press enter. The greeter takes a String name as an input:
For some greeter templates, you might need to send just the name as "Sarah".
8

We invoked a Restate service over Kafka

You now see your handler getting invoked.The way this worked is that Restate reads the message off the Kafka topic and durably persisted it, similar to what it does for HTTP invocations.It then pushed the message to the handler, as opposed to the pull mechanism that you would have with a Kafka consumer.If the handler fails, Restate will retry the request until it eventually succeeds.When invoking Basic Services, Restates ignores the key of the message.When invoking Virtual Objects, Restate uses the key of the Kafka message as the Virtual Object key. Whereas a Kafka partition contains multiple keys, Restate effectively keeps track of a queue per key.Have a look at the Event Processing use case page to learn about what you can do with Restate and event processing.
9

Cleanup: removing the subscription

You can see the subscriptions that are active via the CLI:
Example output:
As you can see, subscriptions have an ID that starts with sub_.Now you can use the subscription ID to delete the subscription: