release-0.22.1

Added

• New 'Aborted' status for CHI is set when reconcile is aborted by an operator

Changed

• Allow shard weight to be zero (#1192 by maxistua)
• Removed excessive logging for pod update events
• Removed 30s delay after creating a service
• Allow empty values for CRD status and some other fields in order to facilitate migration from old operator versions that were upgraded without upgrading CRD first. Fixes #842, #890 and similar issues.

Full Changelog: release-0.22.0...release-0.22.1

release-0.22.0

Added

• Support volume re-provisioning. If volume is broken and PVC detects it as lost, operator re-provisions the volume
• When new CHI is created, all hosts are created in parallel
• Allow to turn off waiting for running queries to complete. This can be done both in operator configuration or in CHI itself:
  In operator configuration:

spec:
  reconcile:
    host:
      wait:
        queries: "false"

In CHI:

spec:
  reconciling:
    policy: nowait

• When changes are applied to clusters with a lot of shards, the change is probed on a first node only. Is successul, it is applied on 50% of shards. This can be configured in operator configuration:

reconcile:
  # Reconcile runtime settings
  runtime:
    # Max number of concurrent CHI reconciles in progress
    reconcileCHIsThreadsNumber: 10

    # The operator reconciles shards concurrently in each CHI with the following limitations:
    #   1. Number of shards being reconciled (and thus having hosts down) in each CHI concurrently
    #      can not be greater than 'reconcileShardsThreadsNumber'.
    #   2. Percentage of shards being reconciled (and thus having hosts down) in each CHI concurrently
    #      can not be greater than 'reconcileShardsMaxConcurrencyPercent'.
    #   3. The first shard is always reconciled alone. Concurrency starts from the second shard and onward.
    # Thus limiting number of shards being reconciled (and thus having hosts down) in each CHI by both number and percentage

    # Max number of concurrent shard reconciles within one CHI in progress
    reconcileShardsThreadsNumber: 5
    # Max percentage of concurrent shard reconciles within one CHI in progress
    reconcileShardsMaxConcurrencyPercent: 50

• Operator-related metrics are exposed to Prometheus now:

clickhouse_operator_chi_reconciles_started
clickhouse_operator_chi_reconciles_completed
clickhouse_operator_chi_reconciles_timings

clickhouse_operator_host_reconciles_started
clickhouse_operator_host_reconciles_completed
clickhouse_operator_host_reconciles_restarts
clickhouse_operator_host_reconciles_errors
clickhouse_operator_host_reconciles_timings

clickhouse_operator_pod_add_events
clickhouse_operator_pod_update_events
clickhouse_operator_pod_delete_events

Changed

• fix typo in operator_installation_details.md by @seeekr in #1219
• Set operator release date fot createdAt CSV field by @dmvolod in #1223
• Fix type for exclude and include fields in 70-chop-config.yaml example by @dmvolod in #1222
• change dashboard refresh rate 1m and add min_duration_ms, max_duration_ms dashboard variables, rename query_type to query_kind by @Slach in #1235
• add securityContext to helm chart by @farodin91 in #1255
• metrics-exporter collects all hosts and queries in parallel

Fixed

• Fixed a bug when operator could break multiple nodes if incorrect configuration has been deployed several times in a row
• Fixed a bug when schema could not be created on new nodes, if nodes took too long to start
• Fixed a bug when services were not reconciled in rare cases

New Contributors

• @seeekr made their first contribution in #1219
• @dmvolod made their first contribution in #1223
• @farodin91 made their first contribution in #1255

Full Changelog: release-0.21.3...release-0.22.0

release-0.21.3

Added

• Added selectors to CHITemplates. Now it is possible to define a template that is applied to certain CHI. Example here: https://github.com/Altinity/clickhouse-operator/blob/0.21.3/docs/chi-examples/50-CHIT-04-auto-template-volume-with-selector.yaml
• Added '.status.useTemplates' to reflect all templates used in CHI manually or auto

Changed

• CHITemplates are now re-loaded automatically without a need to restart operator. Changes in CHITemplates are not applied automatically to affected CHI.

Fixed

• Fix nil pointer deref in metrics exporter (#1187) by @zcross in #1188
• Migrate piechart plugin on Grafana Dashboard by @MiguelNdeCarvalho in #1190
• Permission error when deleting Pod sometimes

New Contributors

• @MiguelNdeCarvalho made their first contribution in #1190

Full Changelog: release-0.21.2...release-0.21.3

release-0.21.2

Added

• Added support of clickhouse-log via podTemplates by @dmmarkov in #1012
• Added SQL UDFs replication when adding shards/replicas. Closes #1174

Changed

• Operator and metrics-exporter now automatically select 'http' or 'https' for connecting to cluster based on cluster configuration
• Changed statefulSet update behaviour to abort the update on failure. That protects the rest of the cluster from a breaking changes
• Removed unneeded operations on persistent volumes

Fixed

• fix crash when reconcilePVC() failed by @jewelzqiu in #1168
• fix reconcile threads number by @jewelzqiu in #1170

Other

• Run tests in parallel by @antip00 in #1171
• fix build script to adapt to macos m1. by @xiedeyantu in #1169
• Improvements to Grafana and Prometheus manifests

New Contributors

• @jewelzqiu made their first contribution in #1168
• @xiedeyantu made their first contribution in #1169
• @dmmarkov made their first contribution in #1012

Full Changelog: release-0.21.1...release-0.21.2

release-0.21.1

Added

• Added configurable shard-level concurrent reconciliation by @zcross in #1124. It is controlled by the following operator configuration settings:

reconcile:
  runtime:
    # Max number of concurrent CHI reconciles in progress
    reconcileCHIsThreadsNumber: 10
    # Max number of concurrent shard reconciles in progress
    reconcileShardsThreadsNumber: 1
    # The maximum percentage of cluster shards that may be reconciled in parallel
    reconcileShardsMaxConcurrencyPercent: 50

• Added default configuration for ClickHouse system.trace_log table with 30 days TTL

Changed

• ZooKeeper manifests were rewritten to store configuration separately

Fixed

• Fixed a bug in metrics-exporter that might stop working on ClickHouse nodes with certain types of system.errors. Closes #1161
• Fixed a bug with ClickHouse major version detection for Altinity.Stable builds

Full Changelog: release-0.21.0...release-0.21.1

release-0.21.0

What's Changed

• Changed the way Operator applies ClickHouse settings. In the previous version, every change in settings required a restart via re-creating a StatefulSet. In this version it does not re-create StatefulSet anymore, but maintains a logic that decide if ClickHouse needs to be restarted in order to pickup a change or not. In case of restart it is performed using scaling stateful set down and up. The restart logic is controlled by configurationRestartPolicy configuration setting. Here is the default configuration:

    rules:
      - version: "*"
        rules:
          - settings/*: "yes"
          - settings/dictionaries_config: "no"
          - settings/logger: "no"
          - settings/macros/*: "no"
          - settings/max_server_memory_*: "no"
          - settings/max_*_to_drop: "no"
          - settings/max_concurrent_queries: "no"
          - settings/models_config: "no"
          - settings/user_defined_executable_functions_config: "no"

          - zookeeper/*: "yes"

          - files/config.d/*.xml: "yes"
          - files/config.d/*dict*.xml: "no"

          - profiles/default/background_*_pool_size: "yes"
          - profiles/default/max_*_for_server: "yes"
      - version: "21.*"
        rules:
          - settings/logger: "yes" 

• Improved reliability of schema creation for new shards/replicas
• Added "secure" option for Zookeeper connections by @Tvion in #1114
• Added "insecure" flag that closes insecure TCP/HTTP ClickHouse ports. See security_hardening.md for more detail
• Added an option to disable metrics exporter by @roimor in #1131
• Added system.errors scrapping into metrics-exporter
• Refactor Registry internals to make concurrent access safe by @zcross in #1115

Fixed

• Fixed a bug that might result in PDB being deleted. Closes #1139
• Fixed propagation of podTemplate environment variables from ClickHouseInstallationTemplate to ClickHouseInstallation
• Fixed propagation of startup probe from ClickHouseInstallationTemplate to ClickHouseInstallation

Non-operator changes

• Changed Grafana deployment to allow persisting custom dashboards
• Changed ZooKeeper version to 3.8.1

New Contributors

• @roimor made their first contribution in #1131
• @Tvion made their first contribution in #1114
• @zcross made their first contribution in #1115

Full Changelog: release-0.20.3...release-0.21.0

release-0.20.3

What's Changed

• Use alpine base image instead of UBI
• Fix error handling when table already exists in ZooKeeper for new ClickHouse versions

Full Changelog: release-0.20.2...release-0.20.3

release-0.20.2

What's Changed

• Added 'hostsCompleted' to the CHI status and events
• Changed some 'default' profile settings:
  • Enabled 'do_not_merge_across_partitions_select_final'
  • Set 'load_balancing' to 'nearest_hostname'
  • Set niceness ('os_thread_priority') to 2.
• Improved stability of metric-exporter when some ClickHouse nodes are responding slowly
• Changed sequence of LB service creation to avoid a situation when service exists with no available endpoints
• Added 'secure' flag at cluster level for enabling distributed queries over TLS
• Addressed CVEs in dependent packages

Note: datatype for 'secure' flag has been changed from boolean to String (accepting, 'true', 'yes', '1' etc.)

release-0.20.1

What's Changed

• Improvements to Grafana dashboards by @Slach
• Generate operator helm chart by @slamdev in #1049
• Added cluster to PodDesruptionBudget selector. Closes #996. Note: PDB requires Kubernetes version 1.21 or above.
• Added status.hostsCompleted to track reconcile progress
• Do not restart CHI with restart attribute when operator is restarted
• Address https://nvd.nist.gov/vuln/detail/CVE-2022-27664. Closes #1039
• Fixed RBAC permissions for secrets. Closes #1051

New Contributors

• @slamdev made their first contribution in #1049

Full Changelog: release-0.20.0...release-0.20.1

release-0.20.0

What's Changed

• Add support for setting secure communication between clickhouse instances. (Inspried by @chancez in #938). See examples:
  • auto secret
  • plain text secret
  • k8s secret reference
• Operator managed PV provisioning. (Inspired by @chhan-coupang in #947). Allows to re-scale volumes w/o a downtime. In order to enable, use:

  defaults:
    storageManagement:
      provisioner: Operator

• Use secret for clickhouse_operator credentials as default
• Improve zookeeper scaleout by @Slach in #984
• add loadaverage + network + disk panel to clickhouse dashboard by @Slach in #1015
• comment out chopusername check by @SuzyWangIBMer in #1009
• Reduced a size of status field
• Improved logging

Fixed

• Fixed a bug with cluster definitions being reset when CHI has been updated via k8s API POST method.
• Fixed a problem of metrics exporter not being able to scrape any managed cluster when one of clickhouse nodes is slow
• Fix zookeeper advanced setup with empty dir by @luluz66 in #993
• Delete duplicate return by @Abirdcfly in #987
• Fixes #991 - fix watch namespace regex by @mcgrawia in #992
• Address CVE-2022-32149 by @bkuschel in #1035

New Contributors

• @Abirdcfly made their first contribution in #987
• @mcgrawia made their first contribution in #992
• @chhan-coupang made their first contribution in #947
• @luluz66 made their first contribution in #993
• @chancez made their first contribution in #938
• @antip00 made their first contribution in #1034

Full Changelog: release-0.19.2...release-0.20.0