Tighten persistence.deploymentStrategy guard to require existingClaim, not persistence.enabled

[eamonnmoloney]

Description

The persistence.deploymentStrategy guard admits a combination that can only ever hurt the user: Recreate together with chart-managed (ephemeral) persistence.

Every component that exposes the knob validates it like this:

{{- if and (eq $deploymentStrategy "Recreate") (not .Values.<component>.persistence.enabled) -}}
{{- fail "... Recreate requires <component>.persistence.enabled: true ..." -}}
{{- end -}}

That check was written when chart-managed persistence meant a single shared PersistentVolumeClaim, where Recreate was the mitigation for Multi-Attach deadlocks during helm upgrade (ADR 0092).

Since the chart-managed path moved to a per-pod generic ephemeral volume, that premise no longer holds. Each pod now gets its own PVC, so a surge pod never contends with the outgoing pod and RollingUpdate is always safe. The only remaining case where Recreate earns its downtime is existingClaim, where the chart does not control the access mode and the user-supplied volume may be ReadWriteOnce.

So the guard keys on the wrong field. It should key on existingClaim, not persistence.enabled.

Expected vs Actual Behavior

Given:

connectors:
  persistence:
    enabled: true
    deploymentStrategy: Recreate   # no existingClaim

Actual: renders successfully. The Deployment gets strategy.type: Recreate, so every helm upgrade tears the pod down before starting its replacement — downtime bought for nothing, since the volume is already per-pod.

Expected: the render fails with a message explaining that Recreate is only meaningful alongside existingClaim.

Motivation / Use Case

This is a silent, self-inflicted availability regression. Nothing warns the user; the manifest renders cleanly and the cost only shows up as unexplained downtime on every subsequent upgrade. The knob's own @param documentation already tells users the correct rule — the template just doesn't enforce it:

Set to "Recreate" only when using existingClaim with a ReadWriteOnce volume that cannot tolerate concurrent attach during a rollout (introduces brief downtime per upgrade).

Tightening the guard makes the template agree with its documentation.

Acceptance Criteria

• Recreate + existingClaim renders successfully (unchanged).
• Recreate without existingClaim fails the render with a message naming existingClaim as the precondition.
• RollingUpdate (the default) is unaffected in every combination.
• Applied consistently across all components exposing the knob, so no behavioural split is introduced.
• Unit tests cover the newly-rejected combination for each component.
• @param wording in values.yaml updated to state the existingClaim precondition as a requirement rather than a recommendation.
• ADR 0092 amended to record that the guard condition changed once the chart-managed path became ephemeral.

Additional Context

Affected surface

Charts and components currently carrying the guard:

Chart	Component	Chart-managed path	Tighten?
8.8	Web Modeler restapi	ephemeral	yes
8.9	Web Modeler restapi	shared PVC	no — see below
8.10	Web Modeler restapi	ephemeral	yes
8.10	Connectors	ephemeral	yes, once #6522 merges
8.10	Identity	ephemeral	yes, once #6522 merges

Optimize is out of scope: it hardcodes Recreate and does not expose the knob.

Precondition: do not tighten 8.9 yet

Chart 8.9 must be excluded until #6027 merges. Its Web Modeler restapi still mounts a shared chart-managed PVC:

charts/camunda-platform-8.9/templates/web-modeler/deployment-restapi.yaml
197:          persistentVolumeClaim:      # existingClaim
200:          persistentVolumeClaim:      # chart-managed — still shared

On that shared-PVC path Recreate + persistence.enabled is genuinely load-bearing — it is exactly the ADR 0092 Multi-Attach mitigation. Tightening 8.9 before the ephemeral change lands would remove a working mitigation and reintroduce the upgrade deadlock. Either sequence this after #6027, or scope the change to 8.8/8.10 and follow up.

Why this wasn't fixed in #6522

Raised during review of #6522, which converts Connectors and Identity /tmp to per-pod ephemeral volumes in 8.10. Deliberately left out of scope: that PR would have had to either tighten only its own two components — creating exactly the kind of cross-component behavioural split ADR 0092 set out to close — or edit Web Modeler across three charts, well outside its stated scope. Filing separately so the change lands uniformly.

References

• ADR 0092 — opt-in Recreate for chart-managed RWO persistence (defines the guard's original rationale)
• ADR 0043 — hardcoded deployment strategies (the prior state)
• fix(8.10): back Connectors and Identity /tmp with a per-pod ephemeral volume #6522 — Connectors/Identity per-pod ephemeral volume, 8.10 (where this was found)
• fix(8.8,8.9,8.10): allow Recreate strategy for Web Modeler persistence #6014 — first application of the knob, Web Modeler restapi
• fix(8.9): back Web Modeler restapi /tmp with a per-pod ephemeral volume #6027 — Web Modeler 8.9 ephemeral volume (blocks the 8.9 portion)
• fix(8.10): back Web Modeler restapi /tmp with a per-pod ephemeral volume #6406, fix(8.8): back Web Modeler restapi /tmp with a per-pod ephemeral volume #6408 — Web Modeler ephemeral volume, 8.10 and 8.8
• SUPPORT-30069 / [ISSUE] create a scenario values.yaml file and a guide for enabling persistence #4767 — original Multi-Attach upgrade deadlock report

Environment

• Chart version: 8.8, 8.9, 8.10
• Platform: any — this is a render-time validation issue, not runtime or platform specific

[distro-ci]

🧭 Triage Assessment

Field	Value
Work type	BUG
Kind	bug
Severity	⚠️ Mid
Likelihood	🔹 Low
Confidence	0.94

📝 Rationale

Within SM Platform scope: the team boundaries assign Helm Charts, camunda/camunda-platform-helm, and Self-Managed Helm configuration issues to Distribution. This is a BUG with kind=bug because the existing guard permits an invalid configuration that causes avoidable upgrade downtime; it is not merely documentation or maintenance work. The triage guidelines define bugs as incorrect existing behavior. Severity is mid: the impact is noticeable production availability loss during upgrades, but there is no data loss, security impact, or core-flow blockage, and users have a straightforward workaround by retaining the default RollingUpdate or using existingClaim. Under the bug rubric, mid covers noticeable impact where core flows remain functional, while high generally requires no simple workaround. Likelihood is low because the issue affects only users who explicitly select Recreate with chart-managed persistence; the default RollingUpdate behavior is unaffected. #6409 is the closest existing issue and may overlap for the Web Modeler portion because it tracks narrowing the Recreate rationale to existingClaim after the ephemeral-volume migration. #6268 and #6269 are related Connectors and Identity persistence issues involving the former RWO/Multi-Attach rationale, but are not exact duplicates. The issue provides sufficient scope, affected versions, rationale, acceptance criteria, and sequencing constraints, so no additional information is required.

🔍 Similar Issues

#6409, #6268, #6269, #4767

---

🤖 This issue was automatically triaged by an AI-based system.
✋ If you believe this assessment is incorrect, update the labels or add the triage:manual label for human review.