- TypeScript
- Java
- Kotlin
- Go
- Python
Event Processing
Lightweight, transactional event processing.
Process Kafka events with flexible flows of transactional steps.
Restate takes care of the event plumbing and pushes events to your handler.
Lightweight Durable Functions
Queue per key
Push events to functions
Event processing with Restate
Connect functions to Kafka topics. Restate pushes the events to your function.
- TypeScript
- Java
- Kotlin
- Go
- Python
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
const userUpdates = restate.object({name: "userUpdates",handlers: {updateUserEvent: async (ctx: restate.ObjectContext, event: UserUpdate) => {const { profile, permissions, resources } = verifyEvent(event);let userId = await ctx.run(() => updateProfile(profile));while (userId === NOT_READY) {await ctx.sleep(5_000);userId = await ctx.run(() => updateProfile(profile));}const roleId = await ctx.run(() => setPermissions(userId, permissions));await ctx.run(() => provisionResources(userId, roleId, resources));},},});
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
@VirtualObjectpublic class UserUpdatesService {@Handlerpublic void updateUserEvent(ObjectContext ctx, UserUpdate event) {String userId = ctx.run(JsonSerdes.STRING, () -> updateUserProfile(event.getProfile()));while (userId.equals("NOT_READY")) {ctx.sleep(Duration.ofMillis(5000));userId = ctx.run(JsonSerdes.STRING, () -> updateUserProfile(event.getProfile()));}String finalUserId = userId;String roleId =ctx.run(JsonSerdes.STRING, () -> setUserPermissions(finalUserId, event.getPermissions()));ctx.run(() -> provisionResources(finalUserId, roleId, event.getResources()));}}
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
@VirtualObjectclass UserUpdatesService {@Handlersuspend fun updateUserEvent(ctx: ObjectContext, event: UserUpdate) {var userId = ctx.runBlock { updateUserProfile(event.profile) }while (userId == "NOT_READY") {ctx.sleep(5000.milliseconds)userId = ctx.runBlock { updateUserProfile(event.profile) }}val finalUserId = userIdval roleId = ctx.runBlock { setUserPermissions(finalUserId, event.permissions) }ctx.runBlock { provisionResources(finalUserId, roleId, event.resources) }}}
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
func (UserUpdates) UpdateUserEvent(ctx restate.ObjectContext, event UserUpdate) error {userId, err := restate.Run(ctx, func(ctx restate.RunContext) (string, error) {return updateProfile(ctx, event.Profile)})if err != nil {return err}for userId == NOT_READY {if err := restate.Sleep(ctx, 5*time.Second); err != nil {return err}userId, err = restate.Run(ctx, func(ctx restate.RunContext) (string, error) {return updateProfile(ctx, event.Profile)})if err != nil {return err}}roleId, err := restate.Run(ctx, func(ctx restate.RunContext) (string, error) {return setPermissions(ctx, userId, event.Permissions)})if err != nil {return err}if _, err := restate.Run(ctx, func(ctx restate.RunContext) (restate.Void, error) {return restate.Void{}, provisionResources(ctx, userId, roleId, event.Resources)}); err != nil {return err}return nil}
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Lightweight Durable Functions
Write functions that take Kafka events as inputs. Functions execute with Durable Execution: their progress is tracked and they can be retried from the exact point before the crash, as if you are taking micro-checkpoints throughout the function execution.
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Push events to functions
Let Restate subscribe to a Kafka topic and specify to which function to push the events. Restate will take care of the event plumbing: polling for records, committing offsets, recovering...
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Queue per key
Events get sent to objects based on the Kafka key. For each key, Restate ensures that events are processed sequentially and in order. Slow events on other keys do not block processing.
In the example, we process user updates in a queue per user. Slow updates for one user do not block updates for other users.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Postpone processing
Flexibly postpone processing of events until later. Restate tracks the timers and re-invokes. When sleeping, other events for that key are enqueued.
Here, we postpone processing for 5 seconds if the user profile is not ready yet.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Durable side effects
The results of interactions with external systems are tracked and recovered after failures. This simplifies writing flows that keep multiple systems in sync.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
Flexible control flow
As opposed to many stream processing systems, Restate does not put any restrictions on the control flow (e.g. DAG). Each event crafts its own path through the code and builds up its own recovery log.
user_updates = VirtualObject("userUpdates")@user_updates.handler()async def update(ctx, event: UserUpdateEvent) -> None:user_id = await ctx.run("update profile",lambda: update_profile(event.profile))while user_id == NOT_READY:await ctx.sleep(5000)user_id = await ctx.run("update profile",lambda: update_profile(event.profile))role_id = await ctx.run("set permissions",lambda: set_permissions(user_id, event.permissions))await ctx.run("provision resources",lambda: provision_resources(user_id, role_id, event.resources))
LOW-LATENCY
Restate’s event-driven foundation built in Rust lets you process events at high speed. Restate keeps a queue per key and pushes events to your functions to process them in parallel as fast as possible.
DURABLE EXECUTION
Restate manages the complexities of reading from Kafka to make sure each event gets processed exactly once. Restate handles retries and recovery of your event handlers to the exact point before the crash.
Stateful event processing with Restate
Implement stateful event handlers with Restate.
- TypeScript
- Java
- Kotlin
- Go
- Python
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
const eventEnricher = restate.object({name: "profile",handlers: {userEvent: async (ctx: ObjectContext, event: UserProfile) => {ctx.set("user", event);ctx.objectSendClient(EventEnricher, ctx.key).emit(restate.rpc.sendOpts({ delay: 1000 }));},featureEvent: async (ctx: ObjectContext, featureEvent: string) => {const userEvent = await ctx.get<UserProfile>("user");if(!userEvent) {throw new TerminalError("User not found");}userEvent.features.push(featureEvent);ctx.set("user", userEvent);},emit: async (ctx: ObjectContext) => {const user = await ctx.get<UserProfile>("user");send(ctx.key, user);ctx.clearAll();},},});
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
@VirtualObjectpublic class ProfileService {private static final StateKey<UserProfile> USER =StateKey.of("user", JacksonSerdes.of(UserProfile.class));@Handlerpublic void userEvent(ObjectContext ctx, String name) {UserProfile profile = new UserProfile(ctx.key(), name);ctx.set(USER, profile);ProfileServiceClient.fromContext(ctx, ctx.key()).send(Duration.ofSeconds(1)).emit();}@Handlerpublic void featureEvent(ObjectContext ctx, String email) {UserProfile user =ctx.get(USER).orElseThrow(() -> new TerminalException("No user found"));user.setEmail(email);ctx.set(USER, user);}@Handlerpublic void emit(ObjectContext ctx) {UserProfile user =ctx.get(USER).orElseThrow(() -> new TerminalException("No user found"));send(ctx.key(), user);ctx.clearAll();}}
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
@VirtualObjectclass ProfileService {companion object {private val USER = StateKey.of("user", KtSerdes.json<UserProfile>())}@Handlersuspend fun userEvent(ctx: ObjectContext, name: String) {val profile = UserProfile(ctx.key(), name)ctx.set(USER, profile)ProfileServiceClient.fromContext(ctx, ctx.key()).send(1.seconds).emit()}@Handlersuspend fun featureEvent(ctx: ObjectContext, email: String) {val user = ctx.get(USER) ?: throw TerminalException("No user found")user.email = emailctx.set(USER, user)}@Handlersuspend fun emit(ctx: ObjectContext) {val user = ctx.get(USER) ?: throw TerminalException("No user found")send(ctx.key(), user)ctx.clearAll()}}
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
func (Profile) UserEvent(ctx restate.ObjectContext, event UserProfile) {restate.Set(ctx, "user", event)restate.ObjectSend(ctx, "EventEnricher", restate.Key(ctx), "Emit").Send(restate.Void{}, restate.WithDelay(1*time.Second))}func (Profile) FeatureEvent(ctx restate.ObjectContext, featureEvent string) error {userEvent, err := restate.Get[UserProfile](ctx, "user")if err != nil {return err}userEvent.Features = append(userEvent.Features, featureEvent)restate.Set(ctx, "user", userEvent)return nil}func (Profile) Emit(ctx restate.ObjectContext) error {user, err := restate.Get[UserProfile](ctx, "user")if err != nil {return err}send(restate.Key(ctx), user)restate.ClearAll(ctx)return nil}
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
K/V State
Store state in Restate and access it from other handlers. Restate guarantees that it is consistent and persistent. The state gets delivered together with the request, so you operate on local state.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Event Enrichment
Enrich events with data from multiple sources by storing it in state and eventually exposing it via other functions.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Delayed actions
Schedule async follow-up tasks for now or for later. Restate tracks the timers and triggers them when the time comes.
Here, we wait one second for other user features to arrive before sending the event to downstream processing.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
Combine Kafka and RPC
Functions can be called over RPC or Kafka without changing the code. In the example, the registration can come over Kafka, while the email gets called via HTTP.
event_enricher = VirtualObject("profile")@event_enricher.handler()async def process_name(ctx: ObjectContext, name: str):ctx.set("user", {"name": name})# Schedule a delayed RPC call to emit the eventctx.object_send(emit, ctx.key(), arg=None, send_delay=timedelta(seconds=60))@event_enricher.handler()async def process_email(ctx: ObjectContext, email: str):user_event = await ctx.get("user")if user_event is None:raise TerminalError("No user found")user_event["email"] = emailctx.set("user", user_event)@event_enricher.handler()async def emit(ctx: ObjectContext):user = await ctx.get("user")send(ctx.key(), user)ctx.clear_all()
Event handlers as regular, lightweight functions
Let Restate subscribe to a Kafka topic and push events to your functions.
Your functions run as normal services in your existing infra.
What you can build with Event Processing and Restate
Kafka-triggered workflows
Handle the payment, request the order preparation, wait for driver acceptance callback, etc.
Digital twin pattern
A delivery service that responds to location updates arriving over Kafka. The order gets updated accordingly.