perf: reduce query path overhead by ~12% (vtgate) and ~3% (gRPC)#20148
Draft
schlubbi wants to merge 3 commits into
Draft
perf: reduce query path overhead by ~12% (vtgate) and ~3% (gRPC)#20148schlubbi wants to merge 3 commits into
schlubbi wants to merge 3 commits into
Conversation
Reduce per-query CPU overhead across the vtgate execution hot path and gRPC serialization layer through allocation reduction, lazy initialization, and fast-path optimizations. vtgate Execute path (hot-cache, point select): - Before: 11,404 ns/op, 201 allocs/op, 11.4 KB/op - After: ~10,000 ns/op, 164 allocs/op, ~9.6 KB/op - Improvement: -12% latency, -18% allocations gRPC Execute round-trip: - Before: 91,991 ns/op, 193 allocs/op - After: 89,585 ns/op, 185 allocs/op - Improvement: -2.6% latency, -4% allocations (range_select: -17% allocs) Key optimizations: - Pool LogStats via sync.Pool, release after query logging - Skip trace span annotations when noop tracer is active - Skip noteAliasedExprName for ColName (never rewritten) - Pool TrackedBuffer in sqlparser.String() - Fixed [4]string array in stats key construction - Skip CopyBindVariables when no log subscribers - Lazy time.Now() and callerID in ExpressionEnv - Inline single-shard path in multiGoTransaction - Embed BindVarNeeds value in normalizer struct - Pool ExecuteRequest in gRPC client - Batch-allocate Row/Value arrays in proto3 conversion - Combined callInfoContext (embed CallInfo in context type) - ResolveDestinations single-destination fast path - Single-shard Result fast path (skip AppendResult) - Cache RoutingIndexes on Plan at build time - Pool AllErrorRecorder in ExecuteMultiShard - DestinationKeyspaceID type-assert bypass - Stack-allocate PlanKey hash Digest Signed-off-by: Stefan Jöst <stefan@joest.dev> Signed-off-by: Stefan Jöst <schlubbi@github.com>
vitess-bot
Bot
added
NeedsWebsiteDocsUpdate
Contributor
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
Tests
Documentation
New flags
If a workflow is added or modified:
Backward compatibility
|
github-actions
Bot
added
Component: Evalengine
arthurschreiber
added
Benchmark me
Contributor
|
Hello! 👋 This Pull Request is now handled by arewefastyet. The current HEAD and future commits will be benchmarked. You can find the performance comparison on the arewefastyet website. |
…Key hash The previous commit added Init256() usage in engine/plan.go but the vthash/hash.go and highway/highwayhash.go changes were not included in the patch. Signed-off-by: Stefan Jöst <stefan@joest.dev> Signed-off-by: Stefan Jöst <schlubbi@github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR targets per-query CPU/allocation reductions in vtgate’s execution hot path and the gRPC serialization/client layers, primarily via pooling, lazy initialization, and single-shard fast paths.
Changes:
- Introduces pooling for frequently-allocated request/logging/formatting objects (e.g. gRPC ExecuteRequest, vtgate LogStats, sqlparser TrackedBuffer) and adds selective/lazy computation for hot-path fields (e.g. trace annotations, evalengine time/user).
- Adds vtgate hot-path fast paths (single-shard result handling, stats-key construction, routing resolution) and caches some plan-derived metadata.
- Optimizes proto3 row/value conversions via batch allocations.
Show a summary per file
| File | Description |
|---|---|
| go/vt/vttablet/grpctabletconn/conn.go | Pools ExecuteRequest objects for gRPC Execute to reduce allocations. |
| go/vt/vtgate/scatter_conn.go | Allocation reductions and fast paths in scatter execution; pools AllErrorRecorder on success path. |
| go/vt/vtgate/plan_execute.go | Uses cached routing-index info when populating LogStats. |
| go/vt/vtgate/logstats/logstats.go | Pools LogStats objects; adds Release() for reuse. |
| go/vt/vtgate/executor.go | Skips trace annotations for noop tracer; conditional bindvar copying; releases pooled LogStats when not delivered. |
| go/vt/vtgate/evalengine/expr_env.go | Lazily initializes NOW()/current user information in ExpressionEnv. |
| go/vt/vtgate/engine/routing.go | Adds a bindvar numeric fast path to avoid eval machinery for simple cases. |
| go/vt/vtgate/engine/route.go | Adds a single-bindvar fast path in query construction. |
| go/vt/vtgate/engine/plan.go | Caches routing-index metadata on Plan; stack-allocates PlanKey hash hasher. |
| go/vt/sqlparser/tracked_buffer.go | Pools TrackedBuffer usage in sqlparser.String(). |
| go/vt/sqlparser/normalizer.go | Reduces normalizer allocations via lazy map init and embedded BindVarNeeds. |
| go/vt/callinfo/plugin_grpc.go | Uses a combined context+callinfo wrapper to reduce allocations in gRPC contexts. |
| go/vt/callerid/callerid.go | Avoids context wrapping when caller IDs are nil. |
| go/trace/trace.go | Adds trace.IsNoop() to allow callers to skip work when tracing is disabled. |
| go/streamlog/streamlog.go | Adds HasSubscribers(); changes Send to return delivery count. |
| go/sqltypes/proto3.go | Batch allocation optimizations for proto3 row/value conversions. |
Copilot's findings
- Files reviewed: 18/18 changed files
- Comments generated: 4
Comment on lines
+74
to
110
| // Batch-allocate all Row structs in a single backing array to reduce | ||
| // per-row heap allocations from N to 1. | ||
| backing := make([]querypb.Row, len(rows)) | ||
| result := make([]*querypb.Row, len(rows)) | ||
|
|
||
| // Pre-allocate a single Lengths backing array for all rows. | ||
| nCols := len(rows[0]) | ||
| allLengths := make([]int64, 0, nCols*len(rows)) | ||
|
|
||
| // First pass: compute lengths and accumulate total value size. | ||
| totalValueBytes := 0 | ||
| for i, r := range rows { | ||
| result[i] = RowToProto3(r) | ||
| result[i] = &backing[i] | ||
| start := len(allLengths) | ||
| for _, c := range r { | ||
| if c.IsNull() { | ||
| allLengths = append(allLengths, -1) | ||
| } else { | ||
| l := c.Len() | ||
| allLengths = append(allLengths, int64(l)) | ||
| totalValueBytes += l | ||
| } | ||
| } | ||
| backing[i].Lengths = allLengths[start:] | ||
| } | ||
|
|
||
| // Second pass: batch-allocate all Values into a single buffer. | ||
| allValues := make([]byte, 0, totalValueBytes) | ||
| for i, r := range rows { | ||
| start := len(allValues) | ||
| for _, c := range r { | ||
| if !c.IsNull() { | ||
| allValues = append(allValues, c.Raw()...) | ||
| } | ||
| } | ||
| backing[i].Values = allValues[start:] | ||
| } |
Comment on lines
+283
to
+289
| Type: getPlanType(primitive), | ||
| QueryType: sqlparser.ASTToStatementType(stmt), | ||
| Original: query, | ||
| Instructions: primitive, | ||
| BindVarNeeds: bindVarNeeds, | ||
| TablesUsed: tablesUsed, | ||
| RoutingIndexes: GetRoutingIndexes(primitive), |
Comment on lines
499
to
504
| logStats.RowsAffected = qr.RowsAffected | ||
| logStats.RowsReturned = uint64(len(qr.Rows)) | ||
| // log the tables and routing indexes used in the plan for successful query execution. | ||
| logStats.TablesUsed = plan.TablesUsed | ||
| executedRoot := vcursor.ExecutedPrimitive() | ||
| if executedRoot == nil { | ||
| executedRoot = plan.Instructions | ||
| } | ||
| logStats.RoutingIndexesUsed = engine.GetRoutingIndexes(executedRoot) | ||
| logStats.RoutingIndexesUsed = plan.RoutingIndexes | ||
| } |
Comment on lines
+1196
to
+1198
| if e.queryLogger.HasSubscribers() { | ||
| logStats.BindVariables = sqltypes.CopyBindVariables(bindVars) | ||
| } |
…ched_size - Revert lazy time.Now() and callerID in ExpressionEnv — caused nil pointer panic in TestCompilerReference/FnUnixTimestamp because compiled expressions evaluate time functions without the eager init the test infrastructure expects. - Update callinfo/plugin_grpc_test.go for renamed callInfoContext type. - Regenerate engine/cached_size.go for new RoutingIndexes field. Signed-off-by: Stefan Jöst <stefan@joest.dev> Signed-off-by: Stefan Jöst <schlubbi@github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the result of a pi-autoresearch session that was tasked to improve performance. All changes are AI generated and haven't been verified yet
Generated Content
Reduce per-query CPU overhead across the vtgate execution hot path and gRPC serialization layer through allocation reduction, lazy initialization, and fast-path optimizations.
vtgate Execute path (hot-cache, point select):
gRPC Execute round-trip:
Key optimizations:
Description
Related Issue(s)
Checklist
Deployment Notes
AI Disclosure