Commit df68805
Support sorted shuffle write in Sapphire-Velox stack (#24739)
Summary:
Support sorted shuffle write in Sapphire-Velox stack:
- Copy the key value serializer: `fbcode/spark_cpp/src/main/serializer/CoscoKeyValueSerializer.h` from Spruce to `github/presto-trunk/presto-native-execution/presto_cpp/main/operators/KeyValueSerializer.h`
This is so that we don't have dependency on the internal implementation.
- Allow PartitionAndSerialize Node to take `sortingOrder and `sortingKeys` as optional parameters.
- If both `sortingOrder and `sortingKeys` are provided, serialize the Keys and invoke Cosco sorted shuffle by providing an non-empty key.
Differential Revision: D712365191 parent 21bdd48 commit df68805
14 files changed
Lines changed: 1729 additions & 38 deletions
File tree
- presto-native-execution/presto_cpp/main/operators
- tests
Lines changed: 2 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
0 commit comments