This page describes how you can deploy a distributed Restate cluster.
To understand the terminology used on this page, it might be helpful to read through the architecture reference.
To migrate an existing single-node deployment into a multi-node deployment without losing data, check out this guide.

​
Quickstart using Docker Compose

To run a Restate cluster locally, check out the Restate cluster guide for a Docker Compose ready-made example.

​
Configuration

To deploy a distributed Restate cluster without external dependencies, you need to configure the following settings in your server configuration:
restate.toml
# Every node needs to have a unique node name
node-name = "UNIQUE_NODE_NAME"
# All nodes need to have the same cluster name
cluster-name = "CLUSTER_NAME"
# Make sure it does not conflict with the other nodes
advertised-address = "ADVERTISED_ADDRESS"
# At most one node can be configured with auto-provision = true
auto-provision = false
# Default replication factor for both the logs and the partitions.
#
# Replicate the data to 2 nodes. This requires that the cluster has at least 2 nodes to
# become operational. If the cluster has at least 3 nodes, then it can tolerate 1 node failure.
#
# This also controls the default partition replication. A value of 2 means each partition
# will be running on (max) of 2 nodes, ensuring redundancy and fault tolerance.
default-replication = 2

[bifrost]
# Only the replicated Bifrost provider can be used in a distributed deployment
default-provider = "replicated"

[metadata-server]
# To tolerate node failures, use the replicated metadata server
type = "replicated"

[metadata-client]
# List all the advertised addresses of the nodes that run the metadata-server role
addresses = ["ADVERTISED_ADDRESS", "ADVERTISED_ADDRESS_NODE_X"]

[admin]
# Make sure it does not conflict with the other nodes
bind-address = "ADMIN_BIND_ADDRESS"

[ingress]
# Make sure it does not conflict with other nodes
bind-address = "INGRESS_BIND_ADDRESS"
It is important that every Restate node you start has a unique node-name specified. All nodes that are part of the cluster need to have the same cluster-name specified. At most one node can be configured with auto-provision = true. If no node is allowed to auto provision, then you have to manually provision the cluster. Refer to the Cluster provisioning section for more information. The log provider needs to be configured with default-provider = "replicated". The default-replication should be set to the number of nodes that the data should be replicated to. If you run at least 2 * default-replication-property - 1 nodes, then the cluster can tolerate default-replication-property - 1 node failures. The metadata server type should be set to replicated to tolerate node failures. Every node that runs the metadata-server role will join the metadata store cluster. To tolerate n metadata node failures, you need to run at least 2 * n + 1 Restate nodes with the metadata-server role configured. The metadata-client should be configured with the advertised addresses of all nodes that run the metadata-server role. Every Restate node that runs the worker role will also run the ingress server and accept incoming invocations. For those nodes that run on the same machine, make sure that the ports do not conflict.
Snapshots are essential to support safe log trimming and also allow you to set partition replication to a subset of all cluster nodes, while still allowing for fast partition fail-over to any live node. Snapshots are also necessary to add more nodes in the future.
If you plan to scale your cluster over time, we strongly recommend enabling snapshotting. Without it, newly added nodes may not be fully utilized by the system.

​
Cluster provisioning

Once you start the node that is configured with auto-provision = true, it will provision the cluster so that other nodes can join. The provision step initializes the metadata store and writes the initial NodesConfiguration with the initial cluster configuration to the metadata store. In case none of the nodes is allowed to auto-provision, then you need to provision the cluster manually via restatectl. 
restatectl provision --address <SERVER_TO_PROVISION> --yes
This provisions the cluster with default settings specified in the server configuration.