Upgrades on Kubernetes

helm upgrade is the typical path. A few component-specific notes apply.

Standard upgrade

helm repo update
helm upgrade openzro openzro/openzro \
  --namespace openzro \
  -f my-openzro.yaml

The chart preserves operator-supplied secrets across upgrades via the Helm lookup function:

  • Relay inter-pod HMAC secret (auto-generated on first install, preserved on upgrade)
  • Management WireGuard private key (provisioned by a pre-install Helm hook, preserved across upgrades)
  • Database bootstrap credentials (idempotent provisioning Job)

You shouldn't need to back these up before an upgrade — they travel through the upgrade unchanged. Operators who use external secret managers (SealedSecrets, sops, vault-csi) feed values via existingSecret references and bypass the chart's auto-gen entirely.

Component restart behavior

ComponentUpgrade behavior
managementRollingUpdate. Peer connections continue talking to whichever replica is up.
signalRollingUpdate. Peers reconnect on disconnect — sub-second reconnect.
relayRollingUpdate. Peer TCP connections drop → reconnect via load balancer onto a (possibly different) pod. With multi-pod fabric on, peers on different pods continue communicating throughout.
dashboardRollingUpdate. UI is stateless.
dexRollingUpdate per upstream Dex chart.

Relay rolling updates are the most operationally-visible: every peer connection reconnects exactly once. Sub-second WireGuard handshake makes this invisible to most workloads, but latency-sensitive flows may notice the blip.

Migrating signal from Ingress to LoadBalancer (chart 2.1.0-alpha.31+)

Operators on community ingress-nginx hitting the gRPC streaming bug (#5366, project archived 2026-03 with no fix) can move signal off the Ingress. Same shape as the relay TLS pattern.

Steps:

  1. Set the new values block:

    signal:
      publicHostname: signal.example.com
      containerPort: 443
      service:
        type: LoadBalancer
        port: 443
      tls:
        enabled: true
        certManager:
          enabled: true
          issuerRef:
            kind: ClusterIssuer
            name: letsencrypt-prod
      ingress:
        enabled: false
    
  2. helm upgrade — chart creates Certificate CRD, signal pods restart with TLS, Service becomes LoadBalancer.

  3. Wait for cert-manager to issue the cert (Certificate Ready: True):

    kubectl get certificate -n openzro openzro-signal-tls
    
  4. Get the new external IP and update DNS:

    kubectl get svc -n openzro openzro-signal -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
    

    Point signal.example.com at that IP.

  5. Existing peers reconnect on next sync (within ~30s after the management broadcasts the new signal URL via Signal.URI).

For operators on Envoy / Traefik / NGINX-Plus controllers, the bug doesn't apply — keep signal.ingress.enabled: true.

CRD upgrades

Helm doesn't upgrade CRDs by default. After every chart bump, re-apply the CRDs:

helm pull openzro/openzro-operator --untar --untardir /tmp/op
kubectl apply -f /tmp/op/openzro-operator/crds/

The CRDs are additive — new fields don't break existing CRs. If a CRD changes shape in a breaking way, the release notes will say so explicitly.

Database migrations

The management daemon runs schema migrations automatically on first start after an upgrade. There's no separate migration step.

Migrations are forward-only — downgrading from a newer chart to an older one isn't supported when migrations have already applied. If you need to roll back, restore from a database backup taken before the upgrade.

Dex storage migration

When postgres.enabled: true (the recommended setup), the chart auto-routes Dex's storage to the dex database via the same Postgres credentials. No PVC, no SQLite, no manual values flip.

If you started in lab mode (sqlite) and want to migrate to Postgres later:

  1. Set postgres.enabled: true and configure connection details
  2. Pre-create the dex database (or let the provisioning Job do it on next upgrade)
  3. Export static passwords + connectors from the SQLite store (dex CLI or direct SQL dump)
  4. helm upgrade — Dex starts against the empty Postgres dex DB
  5. Re-import passwords + connectors

There's no automated SQLite → Postgres migration in upstream Dex, so the export/import is manual. Operators on the recommended postgres path skip this entirely.

Verifying a successful upgrade

kubectl -n openzro rollout status deployment/openzro-management
kubectl -n openzro rollout status deployment/openzro-signal
kubectl -n openzro rollout status deployment/openzro-relay
kubectl -n openzro rollout status deployment/openzro-dashboard

# Quick smoke
curl -s https://openzro.example.com/dex/.well-known/openid-configuration | jq .issuer

If a rollout hangs, kubectl describe pod and kubectl logs --previous are the first diagnostics. Probes:

  • Management / signal / dashboard — TCP probe on the service port. Stuck rollout usually means the new image can't reach its database (postgres) or its broker (NATS embedded peer connection refused) — check the management/signal logs first.
  • RelayhttpGet /metrics on port 9090 (HTTP plaintext). TLS handshake on the public port (33080) does not pollute logs.