Skip to content

Commit c71c26a

Browse files
Deprecating and removing tablet throttler CLI flags and tests (#13246)
* Table throttler: --throttler-config-via-topo now defaults to 'true' Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * add deprecation message Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * endtoend tests: remove '--enable-lag-throttler' and use 'UpdateThrottlerConfig' everywhere Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * always use vtctldclient Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * use cluster.VtctldClientProcess Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * disable --throttler-config-via-topo in old throttler tests Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Remove --throttler-config-via-topo where used, since it now defaults 'true' Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix vreplication cluster setup, waiting for throttler config to apply Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * changelog Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * extend throttler threshold Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * a bit more verbose Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fixed CLI test Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * remove old '--enable-lag-throttler' flag, introduce '--heartbeat_on_demand_duration' Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * more log info in throttler.Open() Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * more logging Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Revert to --heartbeat_enable Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Protect throttler config change application with initMutex And in e2e test update the throttler config on the keyspace when it's created. Only wait for the new tablets in a shard to have the throttler enabled when adding a Shard. Signed-off-by: Matt Lord <mattalord@gmail.com> * More CI testing Signed-off-by: Matt Lord <mattalord@gmail.com> * CI testing cont Signed-off-by: Matt Lord <mattalord@gmail.com> * Yes... Signed-off-by: Matt Lord <mattalord@gmail.com> * Somebody doesn't like force pushes so msg here Signed-off-by: Matt Lord <mattalord@gmail.com> * Increase on-demand heartbeat duration from 10s to 1m Signed-off-by: Matt Lord <mattalord@gmail.com> * Use only on-demand heartbeats everywhere Signed-off-by: Matt Lord <mattalord@gmail.com> * Use same throttler config everywhere Signed-off-by: Matt Lord <mattalord@gmail.com> * Update all keyspaces and don't fail test on missing JSON keys Signed-off-by: Matt Lord <mattalord@gmail.com> * Use constant heartbeats in vrepl e2e tests Until #13175 is fixed. Signed-off-by: Matt Lord <mattalord@gmail.com> * Increase workflow command timeout Signed-off-by: Matt Lord <mattalord@gmail.com> * Don't wait for throttler on non-serving primaries Signed-off-by: Matt Lord <mattalord@gmail.com> * #13175 is fixed, therefore re-instating on-deman heartbeats Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Added ToC Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Tweak comment and kick CI Signed-off-by: Matt Lord <mattalord@gmail.com> * Treat isOpen as the ready/running signal. Also align all initMutex usage. Signed-off-by: Matt Lord <mattalord@gmail.com> * Re-adjust comment Signed-off-by: Matt Lord <mattalord@gmail.com> * Adjust CheckIsReady() to match OnlineDDL's expectation/usage This was only using IsReady() before, now it's using IsOpen() and IsReady(). Signed-off-by: Matt Lord <mattalord@gmail.com> * Get rid of log messages from SrvKeyspaceWatcher when no node/key Signed-off-by: Matt Lord <mattalord@gmail.com> * More corrections/tweaks Signed-off-by: Matt Lord <mattalord@gmail.com> * Use more convenient/clear new IsRunning function Signed-off-by: Matt Lord <mattalord@gmail.com> * Revert "Use more convenient/clear new IsRunning function" This reverts commit 9aef276 as this change was not correct. Signed-off-by: Matt Lord <mattalord@gmail.com> * Further fix correct use of IsOpen(), IsRunning(), IsEnabled() Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * throttler.throttledApps cannot be nil Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Remove --enable_lag_throttler flag Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Deprecate --throttler_config_via_topo Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * remove throttler mitigation code, as the problem was solved in #13195 Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * deperecate throttler config flags Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Removed tabletmanager_throttler and tabletmanager_throttler_custom_config tests Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * changelog Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * remove EnableThrottler() call Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * restore default value Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * update threshold Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * update flags desc Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * using atomic.Bool Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Update changelog/18.0/18.0.0/summary.md Co-authored-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * use MarkDeprecated Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * do not expect flags in vttablet --help Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * remove --throttler-config-via-topo from examples scripts Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> --------- Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Signed-off-by: Matt Lord <mattalord@gmail.com> Co-authored-by: Matt Lord <mattalord@gmail.com>
1 parent a960b91 commit c71c26a

17 files changed

Lines changed: 67 additions & 1003 deletions

File tree

.github/workflows/cluster_endtoend_tabletmanager_throttler.yml

Lines changed: 0 additions & 139 deletions
This file was deleted.

.github/workflows/cluster_endtoend_tabletmanager_throttler_custom_config.yml

Lines changed: 0 additions & 139 deletions
This file was deleted.

changelog/18.0/18.0.0/summary.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
- **[VTAdmin](#vtadmin)**
1010
- [Updated to node v18.16.0](#update-node)
1111
- **[Deprecations and Deletions](#deprecations-and-deletions)**
12+
- [Deprecated Flags](#deprecated-flags)
1213
- [Deleted `k8stopo`](#deleted-k8stopo)
1314
- [Deleted `vtgr`](#deleted-vtgr)
1415
- **[New stats](#new-stats)**
@@ -39,6 +40,17 @@ here https://nodejs.org/en/blog/release/v18.16.0.
3940

4041
### <a id="deprecations-and-deletions"/>Deprecations and Deletions
4142

43+
#### <a id="deprecated-flags"/>Deprecated Command Line Flags
44+
45+
Throttler related `vttablet` flags:
46+
47+
- `--enable-lag-throttler` is now removed after being deprecated in `v17.0`
48+
- `--throttle_threshold` is deprecated and will be removed in `v19.0`
49+
- `--throttle_metrics_query` is deprecated and will be removed in `v19.0`
50+
- `--throttle_metrics_threshold` is deprecated and will be removed in `v19.0`
51+
- `--throttle_check_as_check_self` is deprecated and will be removed in `v19.0`
52+
- `--throttler-config-via-topo` is deprecated after asummed `true` in `v17.0`. It will be removed in a future version.
53+
4254
#### <a id="deleted-k8stopo"/>Deleted `k8stopo`
4355

4456
The `k8stopo` has been deprecated in Vitess 17, also see https://github.com/vitessio/vitess/issues/13298. With Vitess 18

examples/common/scripts/vttablet-up.sh

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,10 @@ vttablet \
5454
--service_map 'grpc-queryservice,grpc-tabletmanager,grpc-updatestream' \
5555
--pid_file $VTDATAROOT/$tablet_dir/vttablet.pid \
5656
--vtctld_addr http://$hostname:$vtctld_web_port/ \
57-
--throttler-config-via-topo --heartbeat_enable --heartbeat_interval=250ms --heartbeat_on_demand_duration=5s \
57+
--disable_active_reparents \
58+
--heartbeat_enable \
59+
--heartbeat_interval=250ms \
60+
--heartbeat_on_demand_duration=5s \
5861
> $VTDATAROOT/$tablet_dir/vttablet.out 2>&1 &
5962

6063
# Block waiting for the tablet to be listening

go/flags/endtoend/vttablet.txt

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -99,14 +99,12 @@ Usage of vttablet:
9999
--emit_stats If set, emit stats to push-based monitoring and stats backends
100100
--enable-consolidator Synonym to -enable_consolidator (default true)
101101
--enable-consolidator-replicas Synonym to -enable_consolidator_replicas
102-
--enable-lag-throttler Synonym to -enable_lag_throttler
103102
--enable-per-workload-table-metrics If true, query counts and query error metrics include a label that identifies the workload
104103
--enable-tx-throttler Synonym to -enable_tx_throttler
105104
--enable_consolidator This option enables the query consolidator. (default true)
106105
--enable_consolidator_replicas This option enables the query consolidator only on replicas.
107106
--enable_hot_row_protection If true, incoming transactions for the same row (range) will be queued and cannot consume all txpool slots.
108107
--enable_hot_row_protection_dry_run If true, hot row protection is not enforced but logs if transactions would have been queued.
109-
--enable_lag_throttler If true, vttablet will run a throttler service, and will implicitly enable heartbeats
110108
--enable_replication_reporter Use polling to track replication lag.
111109
--enable_transaction_limit If true, limit on number of transactions open at the same time will be enforced for all users. User trying to open a new transaction after exhausting their limit will receive an error immediately, regardless of whether there are available slots or not.
112110
--enable_transaction_limit_dry_run If true, limit on number of transactions open at the same time will be tracked for all users, but not enforced.
@@ -313,12 +311,7 @@ Usage of vttablet:
313311
--tablet_manager_grpc_server_name string the server name to use to validate server certificate
314312
--tablet_manager_protocol string Protocol to use to make tabletmanager RPCs to vttablets. (default "grpc")
315313
--tablet_protocol string Protocol to use to make queryservice RPCs to vttablets. (default "grpc")
316-
--throttle_check_as_check_self Should throttler/check return a throttler/check-self result (changes throttler behavior for writes)
317-
--throttle_metrics_query SELECT Override default heartbeat/lag metric. Use either SELECT (must return single row, single value) or `SHOW GLOBAL ... LIKE ...` queries. Set -throttle_metrics_threshold respectively.
318-
--throttle_metrics_threshold float Override default throttle threshold, respective to --throttle_metrics_query (default 1.7976931348623157e+308)
319314
--throttle_tablet_types string Comma separated VTTablet types to be considered by the throttler. default: 'replica'. example: 'replica,rdonly'. 'replica' aways implicitly included (default "replica")
320-
--throttle_threshold duration Replication lag threshold for default lag throttling (default 1s)
321-
--throttler-config-via-topo When 'true', read config from topo service and ignore throttle_threshold, throttle_metrics_threshold, throttle_metrics_query, throttle_check_as_check_self (default true)
322315
--topo_consul_lock_delay duration LockDelay for consul session. (default 15s)
323316
--topo_consul_lock_session_checks string List of checks for consul session. (default "serfHealth")
324317
--topo_consul_lock_session_ttl string TTL for consul session.

0 commit comments

Comments
 (0)