VTOrc can now detect and recover from MySQL primaries that are wedged on an InnoDB latch. When InnoDB has a thread stalled waiting on a semaphore, mysqld logs warning MY-012985 to performance_schema.error_log. VTOrc monitors this signal and triggers Emergency Reparent Shard (ERS) to promote a replica before mysqld self-kills at innodb_fatal_semaphore_wait_threshold.
This addresses a scenario where mysqld is alive and accepting connections (replicas still report Slave_IO_Running: Yes / Slave_SQL_Running: Yes), but writes cannot commit due to the stalled latch. Without this analyzer, the shard remains in a zero-write outage for the full kill threshold plus restart time — potentially 10+ minutes with MySQL's default settings.
Requirements:
- MySQL 8.0+ (the
MY-012985error code andperformance_schema.error_logtable do not exist on MariaDB or MySQL < 8.0) - At least one valid replica available for promotion
Configuration:
Vitess's bundled MySQL 8.0+ configuration files (mysql80.cnf, mysql8026.cnf, mysql84.cnf, mysql90.cnf) now set innodb_fatal_semaphore_wait_threshold = 240 (down from MySQL's default of 600). This means:
MY-012985warning fires at T=120s (half the threshold)mysqldself-kills at T=240s
The analyzer detects the stall around T=120s, giving ERS time to complete before mysqld would self-kill. Operators can override this setting via EXTRA_MY_CNF.
See #20169 for details.
When calling cancel or complete on an auto-generated _reverse workflow without explicitly providing --keep-data=false, the system now defaults to keeping data and returns a warning. This prevents accidental deletion of production tables on the original source side, where the _reverse workflow's target is actually your production keyspace.
Behavior change:
| Workflow type | --keep-data flag |
Effective keep_data |
Warning emitted |
|---|---|---|---|
| Normal | omitted | false |
No |
_reverse |
omitted | true |
Yes |
_reverse |
--keep-data=false |
false |
No |
The --keep-data flag help text has been updated to note this default explicitly. This change applies to MoveTables, Reshard, and other VReplication workflow types that use the shared cancel/complete paths.
See #19906 for details.
VTGate now supports preventing cross-keyspace reads (joins and UNIONs), preventing queries that would combine data from different keyspaces. This can be configured at two levels:
VTGate flag (applies to all queries):
--prevent-cross-keyspace-reads
Per-keyspace VSchema setting (applies to specific keyspaces):
vtctldclient ApplyVSchema --vschema='{"prevent_cross_keyspace_reads": true}' my_keyspaceWhen enabled, the planner will reject queries that require joining or combining (via UNION) tables from different keyspaces. This can be overridden on a per-query basis using the ALLOW_CROSS_KEYSPACE_READS comment directive:
/*vt+ ALLOW_CROSS_KEYSPACE_READS */ SELECT * FROM ks1.t1 JOIN ks2.t2 ON t1.id = t2.id;The VTGate flag prevents cross-keyspace reads globally, regardless of per-keyspace VSchema settings.
A new --consolidator-reject-on-cap flag (default false) has been added to VTTablet. When enabled alongside a non-zero --consolidator-query-waiter-cap, queries that would join a consolidated result but exceed the global consolidator waiter cap are rejected with a RESOURCE_EXHAUSTED error instead of silently falling back to independent MySQL execution.
Important: The cap is enforced against the consolidator's global totalWaiterCount across all queries, not a per-query waiter count. This means a duplicate for query B can be rejected because query A has already consumed most of the global waiter budget. This provides backpressure when the consolidator as a whole is saturated, rather than when any single query has too many waiters.
See #19836 for details.
Previously the schema engine had a hardcoded cap of 10,000 tables: a vttablet whose underlying MySQL had more than 10,000 tables would fail to load its schema and could not serve queries. This made recovery from EmergencyReparentShard impossible without dropping tables directly on MySQL.
Two changes:
- The schema engine no longer enforces a row cap on its reload queries. A vttablet with any number of tables will load successfully.
- A new flag,
--queryserver-config-schema-max-table-count(default10000), governs new schema object creation for tables and views.CREATE TABLEandCREATE VIEWstatements that would push the engine's tracked schema-object count above this limit are rejected at vttablet with a clear error before they reach MySQL. The flag is dynamic: changes are observed without restart.
Tablets that already have more tracked schema objects than the configured limit will reload fine — only new creations are gated. Operators who need to support more tables and views should increase the flag and ensure both vttablet and mysqld have enough memory to comfortably hold the larger schema.
See #19978 for details.