Skip to main content
This guide shows how to use the Restate to execute a list of tasks in parallel and then gather their result, also known as fan-out, fan-in.

​
How does Restate help?

  • Restate lets you schedule the tasks asynchronously and guarantees that all tasks will run, with retries and recovery on failures.
  • Restate turns Promises/Futures into durable, distributed constructs that are persisted in Restate and can be recovered and awaited on another process.
  • You can deploy the subtask executors on serverless infrastructure, like AWS Lambda, to let them scale automatically. The main task, that is idle while waiting on the subtasks, gets suspended until it can make progress.
Fan out: You can fan out tasks with Restate by creating a handler that processes a single subtask, and then scheduling it repeatedly from another handler. Fan in: You can fan in the results of the subtasks by using Restate’s Promise Combinators to wait for all promises to resolve.

​
Example

The example implements a worker service:
  1. It splits a task into subtasks.
  2. It schedules all the subtasks. Each subtask results in a promise that gets added to a list.
  3. The result is gathered by waiting for all promises to resolve.
You can run this on FaaS infrastructure, like AWS Lambda, and it will scale automatically. The run handler will then suspend while it waits for all subtasks to finish. Restate will then resume the handler when all subtasks are done. 
const fanOutWorker = restate.service({
  name: "worker",
  handlers: {
    run: async (ctx: Context, task: Task): Promise<Result> => {
      // Split the task in subtasks
      const subtasks: SubTask[] = await ctx.run("split task", () => split(task));

      // Fan out the subtasks - run them in parallel
      const resultPromises = [];
      for (const subtask of subtasks) {
        const subResultPromise = ctx.serviceClient(fanOutWorker).runSubtask(subtask);
        resultPromises.push(subResultPromise);
      }

      // Fan in - Aggregate the results
      const results = await RestatePromise.all(resultPromises);
      return aggregate(ctx, results);
    },

    // Can also run on FaaS
    runSubtask: async (ctx: Context, subtask: SubTask): Promise<SubTaskResult> => {
      // Processing logic goes here ...
      // Can be moved to a separate service to scale independently
      return executeSubtask(ctx, subtask);
    },
  },
});

restate.serve({
  services: [fanOutWorker],
  port: 9080,
});
In this example, we parallelize RPC calls, but this can also be used to parallelize ctx.run actions.
This pattern is implementable with any of our SDKs. We are still working on translating all patterns to all SDK languages. If you need help with a specific language, please reach out to us via Discord or Slack.

​
Running the example

1

Download the example

restate example typescript-patterns-use-cases && cd typescript-patterns-use-cases
2

Start the Restate Server

restate-server
3

Start the Service

npm install
npx tsx watch ./src/parallelizework/fan_out_worker.ts
4

Register the services

restate deployments register localhost:9080
5

Send a request

curl localhost:8080/worker/run \
    --json '{"description": "get out of bed,shower,make coffee,have breakfast"}'
6

Check the service logs

See how all tasks get spawned in parallel, finish at different times, and then get aggregated.
[restate] [worker/runSubtask][inv_17jBqoqRG0TN3msVqHEpZn2aQMOX5kSKrf][2025-01-17T08:51:44.993Z] INFO:  Started executing subtask: get out of bed
[restate] [worker/runSubtask][inv_1f8R1NuF0LF27EdQ0R6s7PR8hld245OM8h][2025-01-17T08:51:44.995Z] INFO:  Started executing subtask: shower
[restate] [worker/runSubtask][inv_101oPhGwxQqZ0sQebkQnpGyV9Rp3oj9CSJ][2025-01-17T08:51:44.997Z] INFO:  Started executing subtask: make coffee
[restate] [worker/runSubtask][inv_1eKDShaxMCEB6DXasrR5OtRXJEvA2je33X][2025-01-17T08:51:44.998Z] INFO:  Started executing subtask: have breakfast
[restate] [worker/runSubtask][inv_17jBqoqRG0TN3msVqHEpZn2aQMOX5kSKrf][2025-01-17T08:51:47.003Z] INFO:  Execution subtask finished: get out of bed
[restate] [worker/runSubtask][inv_101oPhGwxQqZ0sQebkQnpGyV9Rp3oj9CSJ][2025-01-17T08:51:48.007Z] INFO:  Execution subtask finished: make coffee
[restate] [worker/runSubtask][inv_1f8R1NuF0LF27EdQ0R6s7PR8hld245OM8h][2025-01-17T08:51:48.999Z] INFO:  Execution subtask finished: shower
[restate] [worker/runSubtask][inv_1eKDShaxMCEB6DXasrR5OtRXJEvA2je33X][2025-01-17T08:51:49.001Z] INFO:  Execution subtask finished: have breakfast
[restate] [worker/run][inv_18QHSeAYfvim1oNXRl9I5105veQcTW3BEl][2025-01-17T08:51:49.007Z] INFO:  Aggregated result: get out of bed: DONE,shower: DONE,make coffee: DONE,have breakfast: DONE