Skip to content

ESQL: Skip unused STATS groups by adding a Top N BlockHash implementation #127148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 51 commits into from
Jun 11, 2025
Merged

ESQL: Skip unused STATS groups by adding a Top N BlockHash implementation #127148

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
3322f34
Add initial blockhash with TreeSet and basic operator test
ivancea Apr 16, 2025
fcdfbd4
Fix dedupe for nulls and multivalues
ivancea Apr 16, 2025
7282e98
Added tests for null order and sort order
ivancea Apr 16, 2025
b6d7265
Merge branch 'main' into esql-top-n-agg-grouping
ivancea Apr 21, 2025
4950f5f
Move generic BlockHash code to base TestCase
ivancea Apr 21, 2025
a425c5d
Initial TopNBlockHash test structure, WIP
ivancea Apr 21, 2025
665ace3
Added TopNBlockHash test without nulls, and fixed bug in ords
ivancea Apr 22, 2025
6616fdf
Fix BlockHashTestCase keys assertion early return not checking all keys
ivancea Apr 22, 2025
d838057
Added nulls tests, fixed dedupe lookup nulls handling and nulls error…
ivancea Apr 22, 2025
e78fac5
Added multivalue tests and fixed dedupe with multivalues and no matches
ivancea Apr 22, 2025
143d86f
Update docs/changelog/127148.yaml
ivancea Apr 22, 2025
fa3a779
[CI] Auto commit changes from spotless
Apr 22, 2025
4b212cd
Remove TopNBlockHash abstract class
ivancea Apr 22, 2025
87eb135
Remove unused class
ivancea Apr 23, 2025
c9635f9
Merge branch 'main' into esql-top-n-agg-grouping
ivancea Apr 23, 2025
29fa4d1
Fix SeenGroupIds and added extra tests for it
ivancea Apr 23, 2025
7ca3ce3
Included TopNLongBlockHash into BlockHash.ubuild() and GroupSpec logi…
ivancea Apr 24, 2025
13900f6
[CI] Auto commit changes from spotless
Apr 24, 2025
c778b3b
Keep the last value in the top while using TreeSet, for performance
ivancea Apr 25, 2025
270eb6b
Merge branch 'main' into esql-top-n-agg-grouping
ivancea Apr 25, 2025
5483dc2
Use LongBucketedSort and LongHash for unique seen values
ivancea Apr 25, 2025
3a775ac
Add field recording amount on values in top structure
ivancea Apr 25, 2025
2dde5c1
Fixed to have both last top value and value count
ivancea Apr 29, 2025
a5855c1
Specialize block parameters on AddInput
ivancea Apr 30, 2025
5176663
Call the specific add() methods for eacj block type
ivancea Apr 30, 2025
fb670bd
Implement custom add in HashAggregationOperator
ivancea Apr 30, 2025
ea433a2
Merge branch 'main' into esql-top-n-agg-grouping-tests
ivancea May 5, 2025
fd97a8c
Updated new aggs
ivancea May 5, 2025
ddb6735
Added custom sorted structure
ivancea May 5, 2025
3d83784
Improve limit migration logic
ivancea May 5, 2025
23195a8
Merge branch 'main' into esql-top-n-agg-grouping-tests
ivancea May 7, 2025
afff00e
Merge branch 'main' into esql-top-n-agg-grouping
ivancea May 20, 2025
a54ba97
Spotless
ivancea May 20, 2025
20f75c1
Merge branch 'main' into esql-top-n-agg-grouping-tests
ivancea May 21, 2025
8d101fb
Remove count var
ivancea May 21, 2025
c27faa0
Removed lastValue field
ivancea May 22, 2025
9cff744
Added LonghHash for early detection of existing values
ivancea May 22, 2025
22da89e
Revert "Added LonghHash for early detection of existing values"
ivancea May 22, 2025
9c12c35
Merge branch 'main' into esql-top-n-agg-grouping-tests
ivancea May 27, 2025
714aed9
Merge branch 'main' into esql-top-n-agg-grouping
ivancea May 28, 2025
9d8f749
Merge branch 'esql-top-n-agg-grouping' into esql-top-n-agg-grouping-t…
ivancea May 28, 2025
ed18e7c
Make operator tests not depend directly on the BlockHash implementation
ivancea May 28, 2025
16c88eb
Restore original BucketedSort
ivancea May 28, 2025
2b1f868
Improve migration and remove unused BucketedSort test
ivancea May 28, 2025
f0349f4
Remove fixed TO-DOs
ivancea May 28, 2025
e7ebd05
Fix potential memory leak on circuit breaker error
ivancea May 28, 2025
40b7682
Merge branch 'main' into esql-top-n-agg-grouping
ivancea May 29, 2025
76291a6
Renamed topN set and added tests
ivancea May 29, 2025
dd48d97
Remove unused code from TopNMultivalueDedupeLong
ivancea May 29, 2025
e63b73a
Fix missing success=true in constructor
ivancea May 29, 2025
b2c42e4
Merge branch 'main' into esql-top-n-agg-grouping
ivancea Jun 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog/127148.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 127148
summary: Skip unused STATS groups by adding a Top N `BlockHash` implementation
area: ES|QL
type: enhancement
issues: []
9 changes: 9 additions & 0 deletions ...main/generated-src/org/elasticsearch/compute/aggregation/blockhash/BytesRefBlockHash.java
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions ...c/main/generated-src/org/elasticsearch/compute/aggregation/blockhash/DoubleBlockHash.java
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions .../src/main/generated-src/org/elasticsearch/compute/aggregation/blockhash/IntBlockHash.java
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions ...src/main/generated-src/org/elasticsearch/compute/aggregation/blockhash/LongBlockHash.java
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading