Restate Operator v2.4.0 Release Notes
Highlights
- Trusted CA certificates - RestateCluster now supports custom trusted CA certificates via
spec.security.trustedCaCerts, removing the need for custom Restate images when using internal CAs. - Configurable canary image - The canary job image is now configurable via Helm, supporting air-gapped and restricted registry environments.
- IPv6 support - The operator now binds to a dual-stack address, fixing readiness probe failures on IPv6-only clusters.
- Faster drain cleanup - Old deployment versions are now polled every 10 seconds during drain, instead of waiting up to 5 minutes.
New Features
Trusted CA certificates
You can now configure custom trusted CA certificates for RestateCluster viaspec.security.trustedCaCerts. This is useful when Restate needs to trust internal CAs, for example when
calling services behind an internal load balancer with a private certificate.The operator adds an init container that concatenates the system CA bundle with
your custom certificates into a single PEM file, and sets SSL_CERT_FILE on
the Restate container to point to the combined bundle.Changing the Secret references (name or key) triggers a pod rollout.Configurable canary image
The container image used for PIA and Workload Identity canary jobs is now configurable via thecanaryImage Helm value, CANARY_IMAGE environment
variable, or --canary-image CLI flag. Previously busybox:uclibc was
hardcoded, which fails in environments that cannot pull from Docker Hub.cat, grep, and wget.Related: Issue #94, PR #106Bug Fixes
IPv6 dual-stack support
The operator now binds its HTTP server to[::] instead of 0.0.0.0,
supporting both IPv4 and IPv6 clusters. Previously, the readiness probe
failed on IPv6-only clusters because the operator only listened on IPv4.Related: Issue #93, PR #107Faster drain cleanup polling
When old deployment versions still have active invocations (draining), the operator now requeues every 10 seconds instead of waiting for the default 5-minute reconcile interval. This means old versions are cleaned up within seconds of drain completion rather than up to 5 minutes.Related: PR #112Upgrading
CRD Update Required: Helm does not automatically upgrade CRDs. After upgrading the operator, you must manually apply the updated CRDs:Restate Operator v2.3.1 Release Notes
This contains an important fix for a bug introduced v2.3.0. If you’re using v2.3.0 you will need to upgrade to this version.Highlights
- Fix: GCP Workload Identity now requires explicit opt-in via
gcpWorkloadIdentity: trueHelm value, fixing a 403 error loop on non-GCP clusters introduced in v2.3.0.
Bug Fixes
IAMPolicyMember cleanup causes 403 on non-GCP clusters
In v2.3.0, the operator unconditionally attempted to delete IAMPolicyMember resources during reconciliation, even on non-GCP clusters where the RBAC rules were not granted. This caused a 403 Forbidden error loop on every reconcile.The operator now requires thegcpWorkloadIdentity Helm value to be explicitly
set before it will create or delete IAMPolicyMember resources. The
iam.gke.io/gcp-service-account annotation is ignored with a warning unless
the flag is enabled.Impact on Users:- Non-GCP clusters: The 403 reconcile loop is fixed. No action needed.
- GCP clusters using Workload Identity: You must now set
gcpWorkloadIdentity: truein your Helm values.
Upgrading
Upgrade the operator via Helm:Restate Operator v2.3.0
✨ New Features
-
GCP Workload Identity via Config Connector — The operator now automatically creates
IAMPolicyMemberresources to bind Kubernetes service accounts to GCP service accounts via Workload Identity. This is triggered when a RestateCluster hasiam.gke.io/gcp-service-accountinserviceAccountAnnotations. The GCP project ID is extracted from the service account email, so no additional configuration is needed beyond the annotation. A canary job validates that credentials are available before the StatefulSet proceeds. This mirrors the existing AWS Pod Identity Association pattern and requires Config Connector to be installed on the GKE cluster. -
Configurable cluster DNS suffix — The operator now supports configuring the Kubernetes cluster DNS suffix via the
--cluster-dnsCLI flag,CLUSTER_DNSenvironment variable, or HelmclusterDnsvalue. Previouslycluster.localwas hardcoded in all internal service URLs. This is needed for multi-cluster setups, federated environments, and clusters with custom DNS naming. -
Configurable drain delay — Added
drainDelaySecondsto the RestateDeployment CRD’sspec.restatesection. This controls how long the operator waits after a deployment is drained before removing the old version. Default remains 300 seconds (5 minutes). (https://github.com/restatedev/restate-operator/pull/96)
🐛 Bug Fixes
-
Improved admin API error messages — When a deployment registration is rejected by the admin API (e.g. breaking changes without
--force), the error message now includes the response from Restate and is logged and emitted as a Kubernetes event, making failures much easier to diagnose. (#100) -
Fixed canary job completion detection — Fixed a bug where a completed canary job was treated as still pending, causing the operator to loop indefinitely with a
NotReadystatus condition. (#102)
⚙️ Configuration Changes
- New Helm value
clusterDnsfor configuring the cluster DNS suffix (default:cluster.local) - Conditional RBAC for
IAMPolicyMemberCRDs when GCP Workload Identity is enabled
⚠️ Upgrading Notes
CRD Update Required: Helm does not automatically upgrade CRDs. After upgrading the operator, you must manually apply the updated CRDs:Full release notes:
release-notes/v2.3.0.mdView on GitHubRestate Operator v2.2.0
✨ New Features
-
Knative Serving deployment mode —
RestateDeploymentnow supports Knative Serving as an alternative to traditional ReplicaSets. This enables:- Scale-to-zero: Services automatically scale down when idle, saving resources
- Automatic scaling: Replicas scale based on concurrent request load
- In-place updates: Update service implementation without changing Restate deployment identity
- Tag-based identity: Control versioning behavior with the
tagfield — same tag means in-place update, changed tag means versioned update, no tag means auto-versioning
🐛 Bug Fixes
- Fix DNS network policy for NodeLocal DNSCache — The operator now creates DNS egress policies that work with both traditional kube-dns and NodeLocal DNSCache (
169.254.20.10). This fixes DNS resolution issues on GKE Autopilot and other Kubernetes environments using node-local DNS caching. (https://github.com/restatedev/restate-operator/pull/88)
⚙️ Configuration Changes
- Default partitions increased to 24 — The default number of partitions is now 24 (previously lower), providing better parallelism for most workloads. (https://github.com/restatedev/restate-operator/pull/84)
📝 Documentation
- Added dedicated Knative Serving mode section to README with examples and tag-based versioning guide
- Added troubleshooting section for DNS resolution issues
- Updated RocksDB memory documentation (https://github.com/restatedev/restate-operator/pull/82)
⚠️ Upgrading Notes
CRD Update Required: Helm does not automatically upgrade CRDs. After upgrading the operator, you must manually apply the updated CRDs:
Note: The restatedeployments CRD update is especially important for this release as it includes the new Knative Serving deployment mode fields.
Then upgrade the operator via Helm:New Contributors
- @AhmedSoliman made their first contribution in https://github.com/restatedev/restate-operator/pull/82
✨ New Features
-
Operator-driven cluster provisioning - Added support for automatic cluster provisioning via the new
spec.cluster.autoProvisionfield. When enabled, the operator will automatically provision the Restate cluster by calling the gRPCProvisionClusterAPI after pods are running. This is particularly useful for multi-node clusters where manual provisioning was previously required. The provisioning status is tracked instatus.provisionedto prevent repeated provisioning attempts. (#55)⚠️ Important: When using
🎯cluster.autoProvision: true, you must setauto-provision = falsein your Restate config to avoid split brain situations.cluster.autoProvision: trueis the recommended approach for provisioning Restate clusters.
🔧 Improvements
- Simplified example configurations by removing default values that are no longer needed (replicated loglet and replicated metadata server are now defaults)
🏗️ CRD Changes
- Added
spec.cluster.autoProvisionfield to enable operator-managed cluster provisioning - Added
status.provisionedfield to track provisioning state
⬆️ Upgrading
CRD Update Required: Helm does not automatically upgrade CRDs. After upgrading the operator, you must manually apply the new CRDs:What’s Changed
- Update default tunnel client version by @jackkleeman in https://github.com/restatedev/restate-operator/pull/81
What’s Changed
- Avoid reconcile loop in netpol peer list by @jackkleeman in https://github.com/restatedev/restate-operator/pull/73
What’s Changed
- Ignore completed invs when determining active invocations by @jackkleeman in https://github.com/restatedev/restate-operator/pull/71
What’s Changed
- [Security] Update assorted dependencies by @pcholakov in https://github.com/restatedev/restate-operator/pull/69
- Support existing namespaces by @jackkleeman in https://github.com/restatedev/restate-operator/pull/70
What’s Changed
- Add priorityClassName to restatecluster by @pcholakov in https://github.com/restatedev/restate-operator/pull/63
- Release v1.8.4 by @pcholakov in https://github.com/restatedev/restate-operator/pull/66
What’s Changed
- Add service path parameter by @krisztiansala in https://github.com/restatedev/restate-operator/pull/56
- feat: Add topology spread constraints support to RestateCluster by @EronWright in https://github.com/restatedev/restate-operator/pull/60
- Delete jobs on immutable conflict by @jackkleeman in https://github.com/restatedev/restate-operator/pull/61
- Release v1.8.3 by @pcholakov in https://github.com/restatedev/restate-operator/pull/62
New Contributors
- @EronWright made their first contribution in https://github.com/restatedev/restate-operator/pull/60
What’s Changed
- Allow cleaning up old replicasets while the new one is not ready by @jackkleeman in https://github.com/restatedev/restate-operator/pull/59
What’s Changed
- Support registering at a subpath by @jackkleeman in https://github.com/restatedev/restate-operator/pull/37
- Allow setting the http1.1 flag from the restate operator by @damianr13 in https://github.com/restatedev/restate-operator/pull/38
- Fix cleanup replicasets to only look at those in this ns by @jackkleeman in https://github.com/restatedev/restate-operator/pull/41
- Add RestateCluster compute spec imagePullSecrets property by @pcholakov in https://github.com/restatedev/restate-operator/pull/39
- Add volumeAttributesClassName by @jackkleeman in https://github.com/restatedev/restate-operator/pull/42
- Add support for custom restate entrypoint/args by @pcholakov in https://github.com/restatedev/restate-operator/pull/43
New Contributors
- @damianr13 made their first contribution in https://github.com/restatedev/restate-operator/pull/38
What’s Changed
- fix: Honor use_http11 flag by @krisztiansala in https://github.com/restatedev/restate-operator/pull/52
- fix: Add restatecloudenvironment crd to the helm chart
New Contributors
- @krisztiansala made their first contribution in https://github.com/restatedev/restate-operator/pull/52
This release introduces a new crd (restatecloudenvironment). Helm won’t install new crds when upgrading so if you update you’ll need to install the new crd, which is attached to this release.
What’s Changed
- Add initial readiness delay by @jackkleeman in https://github.com/restatedev/restate-operator/pull/44
- Add RestateCloudEnvironment by @jackkleeman in https://github.com/restatedev/restate-operator/pull/40
- Don’t propagate last applied configuration annotation by @jackkleeman in https://github.com/restatedev/restate-operator/pull/46
- Add warnings about minio by @jackkleeman in https://github.com/restatedev/restate-operator/pull/47
What’s Changed
- Fix: add missing patch events to RBAC config by @2snEM6 in https://github.com/restatedev/restate-operator/pull/35
New Contributors
- @2snEM6 made their first contribution in https://github.com/restatedev/restate-operator/pull/35
What’s Changed
- Document all of operator CRD by @haf in https://github.com/restatedev/restate-operator/pull/30
- Split out minio docs and document network isolation by @jackkleeman in https://github.com/restatedev/restate-operator/pull/33
New Contributors
- @haf made their first contribution in https://github.com/restatedev/restate-operator/pull/30
What’s Changed
- Netpol fixes by @jackkleeman in https://github.com/restatedev/restate-operator/pull/28
Fixes network policy enforcement for distributed clustersSee the release notes for 1.6.0 if you’re upgrading from a pre 1.6.0 version.
What’s Changed
- Allow restate clusters to talk to themselves by @jackkleeman in https://github.com/restatedev/restate-operator/pull/26
- Document cluster better by @jackkleeman in https://github.com/restatedev/restate-operator/pull/27
This release adds PodDisruptionBudgets for clusters, and a new CRD, RestateDeployment, which manages versioning for your Restate SDK services automatically.See the release notes for 1.6.0 if you’re upgrading from a pre 1.6.0 version.
What’s Changed
- Add RestateDeployment v1beta1 crd by @jackkleeman in https://github.com/restatedev/restate-operator/pull/20
- Create pdb for clusters by @jackkleeman in https://github.com/restatedev/restate-operator/pull/25
When upgrading to v1.6.0, you may need to delete the operator-managed StatefulSets so that the operator can recreate them with the new pod management policy, as Kubernetes enforces this as an immutable field. You can do this with
kubectl -n your-restate-namespace delete statefulset restate --cascade=orphan, without causing any interruption to your cluster.What’s Changed
- Set RESTATE_NODE_NAME to POD_NAME by default by @tillrohrmann in https://github.com/restatedev/restate-operator/pull/21
- Set pod management policy to parallel by @tillrohrmann in https://github.com/restatedev/restate-operator/pull/23
What’s Changed
- Add CLA automation by @tillrohrmann in https://github.com/restatedev/restate-operator/pull/18
- Make it easier to run a multinode cluster with the operator by @jackkleeman in https://github.com/restatedev/restate-operator/pull/19
New Contributors
- @tillrohrmann made their first contribution in https://github.com/restatedev/restate-operator/pull/18
What’s Changed
- Added affinity to restate operator by @kumorikarasu in https://github.com/restatedev/restate-operator/pull/17
New Contributors
- @kumorikarasu made their first contribution in https://github.com/restatedev/restate-operator/pull/17
What’s Changed
- Fix papercuts by @jackkleeman in https://github.com/restatedev/restate-operator/pull/16
What’s Changed
- Use predicates to dedupe changes to cm, svc, ss, job by @jackkleeman in https://github.com/restatedev/restate-operator/pull/13
Avoid spurious updates from namespaces and service accountsView on GitHub
What’s Changed
- Node selector support by @jackkleeman in https://github.com/restatedev/restate-operator/pull/12
What’s Changed
- Add support for specifying restate pod tolerations by @pcholakov in https://github.com/restatedev/restate-operator/pull/11
New Contributors
- @pcholakov made their first contribution in https://github.com/restatedev/restate-operator/pull/11
1.0 release coincides with the release of Restate 1.0View on GitHub
What’s Changed
- Add config file support by @jackkleeman in https://github.com/restatedev/restate-operator/pull/10
What’s Changed
- Bump rustls from 0.21.10 to 0.21.12 by @dependabot in https://github.com/restatedev/restate-operator/pull/9
- Bump h2 from 0.3.24 to 0.3.26 by @dependabot in https://github.com/restatedev/restate-operator/pull/8
- RUST_LOG is no longer set to debug; uses Restate default of info. Set the environment variable in the RestateCluster object to override.
No release notes provided.View on GitHub
No release notes provided.View on GitHub
What’s Changed
- Canary pods need to be created via Jobs by @jackkleeman in https://github.com/restatedev/restate-operator/pull/7
No release notes provided.View on GitHub
What’s Changed
- Signing key integration by @jackkleeman in https://github.com/restatedev/restate-operator/pull/6
What’s Changed
- Avoid apply loops with predicate filters by @jackkleeman in https://github.com/restatedev/restate-operator/pull/5
No release notes provided.View on GitHub
What’s Changed
- Bump mio from 0.8.10 to 0.8.11 by @dependabot in https://github.com/restatedev/restate-operator/pull/4
New Contributors
- @dependabot made their first contribution in https://github.com/restatedev/restate-operator/pull/4
No release notes provided.View on GitHub
What’s Changed
- Propagate labels and annotations into all created objects by @jackkleeman in https://github.com/restatedev/restate-operator/pull/3
What’s Changed
- SecurityGroupPolicy support by @jackkleeman in https://github.com/restatedev/restate-operator/pull/2
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub
No release notes provided.View on GitHub