Skip to content

refactor: remove deprecated global elasticsearch and opensearch values from 8.10 chart - #6914

Open
bkenez wants to merge 6 commits into
mainfrom
6766-remove-deprecated-global-es-os
Open

refactor: remove deprecated global elasticsearch and opensearch values from 8.10 chart#6914
bkenez wants to merge 6 commits into
mainfrom
6766-remove-deprecated-global-es-os

Conversation

@bkenez

@bkenez bkenez commented Aug 20, 2026

Copy link
Copy Markdown
Member

Which problem does the PR fix?

Closes #6766.

What's in this PR?

Removes the deprecated global.elasticsearch and global.opensearch value trees from the 8.10 chart (deprecated in the 8.9 cycle via #5171, ADR-0084), completing the deprecation lifecycle before the first stable 15.x release.

  • values.yaml: both global trees deleted; the URL defaults move down into optimize.database.{elasticsearch,opensearch}.url so default rendering is unchanged.
  • Templates render only from the component-scoped keys (optimize.database.*, orchestration.data.secondaryStorage.*). All global fallback arms are stripped and helpers that became pure proxies are inlined and deleted (camundaPlatform.elasticsearchPort/opensearchPort, optimize.effectiveEs/OsTlsConfig, optimize.effectiveEs/OsAuthConfig, optimize.effectiveEs/OsUsername, optimize.effectiveOsAwsEnabled, orchestration.legacyElasticsearchExporterAuthenticationEnabled).
  • The JKS truststore-password injection (_resolve_tls_jks_config + TRUSTSTORE_PASSWORD) is deleted: its only producer keys lived under the removed global trees and the mechanism was already unreachable on main (all callers pass component TLS configs, which have no jks block). Migration: global.tls.caBundle.secret (PEM), or an explicit -Djavax.net.ssl.trustStorePassword in <component>.javaOpts.
  • cluster-name is no longer rendered: the unified camunda.data.secondary-storage.<engine>.cluster-name property defaults to the same values the chart set, and nothing in the 8.10 application consumes it beyond config-bean copy-downs. Escape hatch: orchestration.extraConfiguration.
  • Root-level camundaPlatform.keyRemoved guards for global.elasticsearch and global.opensearch replace the six stale per-leaf guards (which would nil-deref once the trees are gone). keyRemoved gains an optional, backwards-compatible migration field so the error names the replacement keys.
  • The legacy exporter gates simplify to optimize.database.<engine>.enabled AND optimizeEnabled AND (exporters.zeebe.enabled OR regions < 2). The rdbms term was subsumed by the Optimize term. A second dropped term, global.<engine>.enabled AND exporters.zeebe.enabled (no Optimize requirement), traces to the original 8.10 chart scaffold (3812a38a1e) and was never covered by any test — post-fix: align legacy exporter with its component datastore source #6840 the exporter's contract is Optimize-owned (ADR-0084), so this untested, Optimize-independent path is not preserved. Documented in the migration guide as a disclosed narrowing, not a silent one.
  • The caBundle "trust, not encryption" warning for optimize.database.<db>.url.protocol: http is now gated on that database being enabled — otherwise the moved-down http default would fire it for every caBundle user regardless of Optimize usage.
  • Legacy Operate/Tasklist opensearch/zeebeOpensearch config blocks are removed — they were reachable only via global.opensearch.enabled; the canonical camunda.data.secondary-storage.opensearch.aws-enabled and CamundaExporter connect.awsEnabled paths still render.
  • CI scenario values migrate to component-scoped keys; backend enablement moves from base.yaml into the persistence layers. opensearch-self-signed stays distinct from -os-trust as the component BYO-JKS coverage (a JKS truststore is readable without its integrity password; TRUSTSTORE_PASSWORD was already not rendered on main).
  • Unit tests: no-op global.elasticsearch.enabled boilerplate deleted; web-modeler service tests satisfy the storage constraint via orchestration.data.secondaryStorage.type; the TLS no-password contract is folded into the shared requireTLSCustomKey helper; new TestGlobalDatastoreTreesRemovedGate covers both guards plus a clean component-scoped render.
  • Residuals: contributor test-instructions example, helm-values-mcp deprecation test (retargeted to orchestration.extraInitContainers), docs/tls-coverage-810.md. Also repairs a pre-existing list_components test failure (asserted 8.10 bundled ES/Keycloak chart dependencies that were removed in 15.0.0-alpha2; reproduced on pristine main) by pinning those assertions to 8.9.
  • README.md, values.schema.json, and goldens regenerated via repo tooling. Golden churn is exactly the cluster-name line in 4 orchestration configmap goldens.
  • Rendered-parity verified across 16 configurations (component-scoped ES/OS/rdbms/caBundle/JKS setups + every 8.10 persistence layer stack): only intended diffs (cluster-name, removed deprecation warnings, removed global-gated legacy blocks, whitespace from the collapsed TLS emitter).

The full leaf-by-leaf migration mapping (including the two items above with no direct replacement) is documented in camunda-docs#9653, which also updates the 8.9→8.10 upgrade guide, secret-management, TLS, and production-install pages: camunda/camunda-docs#9653

Note for reviewers grepping the repo: charts/elasticsearch-21 has its own unrelated global.elasticsearch.service.* keys — a different chart with no dependency relationship to camunda-platform-8.10.

scripts/deploy-camunda/deploy/merge_test.go keeps its global.elasticsearch fixture on purpose: it models the 8.7 chart, where the key is still a supported surface.

Checklist

Please make sure to follow our Contributing Guide.

Before opening the PR:

  • In the repo's root dir, run make go.update-golden-only.
  • There is no other open pull request for the same update/change.
  • Tests for charts are added (if needed).
  • In-repo documentation are updated (if needed).

After opening the PR:

  • Did you sign our CLA (Contributor License Agreement)? It will show once you open the PR.
  • Did all checks/tests pass in the PR?

@bkenez
bkenez force-pushed the 6766-remove-deprecated-global-es-os branch from 673d840 to 2294d05 Compare August 20, 2026 06:27
@github-actions

github-actions Bot commented Aug 20, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://camunda.github.io/camunda-platform-helm/camunda-platform-helm/pr-preview/pr-6914/

Built to branch gh-pages at 2026-08-21 05:56 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR completes the 8.10 chart deprecation lifecycle by removing the deprecated global.elasticsearch and global.opensearch value trees and their template fallbacks, shifting all datastore configuration to component-scoped keys (Optimize + Orchestration secondary storage) and enforcing removal via root-level keyRemoved constraints.

Changes:

  • Remove global.elasticsearch/global.opensearch from charts/camunda-platform-8.10/values.yaml and regenerate the 8.10 schema/docs outputs.
  • Update 8.10 templates to render only from component-scoped datastore keys and remove now-unreachable legacy branches (incl. truststore-password injection and cluster-name rendering).
  • Migrate unit/integration tests, CI scenario values layers, and helm-values-mcp tests to match the new configuration surface and regenerated goldens.

Reviewed changes

Copilot reviewed 41 out of 41 changed files in this pull request and generated no comments.

Show a summary per file
File Description
helm-values-mcp/test/helm-values.test.ts Adjusts MCP tests for 8.10 dependency assertions and deprecated-path detection.
docs/tls-coverage-810.md Updates TLS coverage notes to reflect removal of global datastore trees.
charts/camunda-platform-8.10/values.yaml Removes deprecated global datastore trees; moves datastore URL defaults into component-scoped keys.
charts/camunda-platform-8.10/values.schema.json Regenerates schema to drop removed global datastore trees and update descriptions/defaults.
charts/camunda-platform-8.10/test/unit/web-modeler/service_test.go Migrates tests off global.elasticsearch.* to secondary-storage type selection.
charts/camunda-platform-8.10/test/unit/orchestration/statefulset_test.go Removes obsolete global.elasticsearch.enabled value from golden test setup.
charts/camunda-platform-8.10/test/unit/orchestration/golden/configmap.golden.yaml Golden update removing cluster-name output.
charts/camunda-platform-8.10/test/unit/orchestration/golden/configmap-retention.golden.yaml Golden update removing cluster-name output.
charts/camunda-platform-8.10/test/unit/orchestration/golden/configmap-log4j2.golden.yaml Golden update removing cluster-name output.
charts/camunda-platform-8.10/test/unit/orchestration/golden/configmap-authorizations.golden.yaml Golden update removing cluster-name output.
charts/camunda-platform-8.10/test/unit/orchestration/configmap_unified_test.go Removes obsolete global.elasticsearch.enabled setup from RDBMS test case.
charts/camunda-platform-8.10/test/unit/orchestration/configmap_test.go Removes obsolete global.elasticsearch.enabled setup from golden RDBMS configmap test.
charts/camunda-platform-8.10/test/unit/common/multi_optimize_reference_test.go Removes legacy global.elasticsearch.enabled boilerplate from multi-optimize reference tests.
charts/camunda-platform-8.10/test/unit/common/constraints_test.go Adds coverage for keyRemoved guards on removed global datastore trees.
charts/camunda-platform-8.10/test/unit/common/component_tls_custom_key_test.go Updates TLS assertions to ensure truststore-password injection is absent.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/persistence/rdbms.yaml Migrates scenario values to component-scoped Optimize datastore keys.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/persistence/rdbms-self-signed.yaml Migrates scenario values to component-scoped Optimize datastore keys.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/persistence/rdbms-external.yaml Migrates scenario values to component-scoped Optimize datastore keys.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/persistence/opensearch-self-signed.yaml Removes global-tree setup; updates notes around JKS password non-usage.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/persistence/opensearch-self-signed-os-trust.yaml Drops obsolete global Elasticsearch disablement.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/persistence/opensearch-embedded.yaml Drops obsolete global Elasticsearch disablement.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/persistence/no-elasticsearch.yaml Renames/retargets to “none” backend for 8.10 and removes global-tree usage.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/persistence/elasticsearch.yaml Migrates scenario values to component-scoped Optimize datastore keys.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/persistence/elasticsearch-external.yaml Migrates scenario values to component-scoped Optimize datastore keys and updates notes.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/identity/oidc.yaml Removes obsolete global Elasticsearch enablement.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/identity/auth0.yaml Removes obsolete global Elasticsearch enablement.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/features/multinamespace-hub.yaml Removes obsolete global Elasticsearch enablement toggle.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/features/multinamespace-2orch-hub.yaml Removes obsolete global Elasticsearch enablement toggle.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/features/multi-optimize/values-platform.yaml Removes obsolete global Elasticsearch enablement toggle.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/features/multi-optimize/values-optimize-only.yaml Removes obsolete global Elasticsearch enablement toggle.
charts/camunda-platform-8.10/test/integration/scenarios/chart-full-setup/values/base.yaml Removes global datastore enablement from base layer per new persistence layering.
charts/camunda-platform-8.10/templates/orchestration/statefulset.yaml Removes global datastore secret env injection; aligns legacy exporter auth wiring to component-scoped keys.
charts/camunda-platform-8.10/templates/orchestration/files/_application.yaml Drops global fallbacks/legacy blocks; removes cluster-name; simplifies exporter auth/prefix defaults.
charts/camunda-platform-8.10/templates/orchestration/_helpers.tpl Removes global datastore fallbacks from secondary storage selection and legacy exporter helpers.
charts/camunda-platform-8.10/templates/optimize/files/_environment-config.yaml Removes global fallbacks; uses Optimize component-scoped datastore keys exclusively.
charts/camunda-platform-8.10/templates/optimize/deployment.yaml Removes global fallbacks and inlines effective auth references to component-scoped keys.
charts/camunda-platform-8.10/templates/optimize/_helpers.tpl Simplifies effective TLS/URL/prefix helpers after removal of global datastore trees.
charts/camunda-platform-8.10/templates/common/constraints.tpl Replaces per-leaf guards with root keyRemoved checks; adds optional migration text; updates caBundle warnings gating.
charts/camunda-platform-8.10/templates/common/_helpers.tpl Removes global datastore host/port fallbacks and deletes truststore-password injection path.
charts/camunda-platform-8.10/README.md Regenerates docs to remove deprecated global datastore trees and reflect updated references.
.github/instructions/go-tests.instructions.md Updates example test instructions to use secondary-storage type instead of global datastore flags.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

pull Bot pushed a commit to Mu-L/camunda that referenced this pull request Aug 20, 2026
…atform global datastore values (camunda#60601)

## Description

camunda-platform chart 15.x (Camunda 8.10) removes the deprecated
`global.elasticsearch.*` and `global.opensearch.*` value trees
(camunda/camunda-platform-helm#6766, removal in
camunda/camunda-platform-helm#6914). Once the chart dependency moves
past `15.0.0-alpha4`, any values file still carrying those keys fails
`helm install`/`helm upgrade` with a `keyRemoved` error.

This PR migrates the two consumers in this repository to the
component-scoped replacement keys ahead of the chart bump. The values
are valid both before and after the removal, so this is safe to merge
independently:

- `.ci/preview-environments/charts/c8sm`:
`global.elasticsearch.{enabled,url.host}` moves to
`camunda-platform.optimize.database.elasticsearch.{enabled,url.host}`
(Optimize's datastore and the legacy Zeebe exporter resolve from that
block; the Orchestration Cluster already uses
`orchestration.data.secondaryStorage`). Also retargets a stale comment
in `templates/elasticsearch.yml`.
- `load-tests/setup/main/values`:
`camunda-platform-values-elasticsearch.yaml` and
`camunda-platform-values-opensearch.yaml` move to
`orchestration.data.secondaryStorage.<engine>.url` plus
`optimize.database.<engine>.*`. The `-optimize-*` variants already use
component-scoped keys.
- Load-test golden files regenerated (`make warm-chart-cache
update-golden PATTERN='GoldenFiles/c8-golden-main'`, 16/16 pass). The
diff is strictly removals: the chart's deprecation warnings for the
legacy keys, and legacy Operate/Tasklist OpenSearch config blocks that
only rendered when `global.opensearch.enabled` was set (ignored by the
unified 8.10 application configuration).

## Checklist

- [ ] Enable backports when necessary (fex. [for bug
fixes](https://github.com/camunda/camunda/blob/main/CONTRIBUTING.md#backporting-changes),
[for CI
changes](https://camunda.github.io/camunda/ci/#when-to-backport-ci-changes),
or [for documentation
changes](https://camunda.github.io/camunda/ci/#documentation-specific-backporting-monorepo-docs-folders)).

## Related issues

closes camunda/camunda-platform-helm#6917
@bkenez
bkenez force-pushed the 6766-remove-deprecated-global-es-os branch from f2daae4 to d36bbf4 Compare August 21, 2026 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[TASK] Remove deprecated global Elasticsearch and OpenSearch values from 8.10 before GA

2 participants