Skip to content

Commit df68805

Browse files
emilysun201309facebook-github-bot
authored andcommitted
Support sorted shuffle write in Sapphire-Velox stack (#24739)
Summary: Support sorted shuffle write in Sapphire-Velox stack: - Copy the key value serializer: `fbcode/spark_cpp/src/main/serializer/CoscoKeyValueSerializer.h` from Spruce to `github/presto-trunk/presto-native-execution/presto_cpp/main/operators/KeyValueSerializer.h` This is so that we don't have dependency on the internal implementation. - Allow PartitionAndSerialize Node to take `sortingOrder and `sortingKeys` as optional parameters. - If both `sortingOrder and `sortingKeys` are provided, serialize the Keys and invoke Cosco sorted shuffle by providing an non-empty key. Differential Revision: D71236519
1 parent 21bdd48 commit df68805

14 files changed

Lines changed: 1729 additions & 38 deletions

presto-native-execution/presto_cpp/main/operators/CMakeLists.txt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,8 @@ add_library(
2020
LocalPersistentShuffle.cpp
2121
BroadcastWrite.cpp
2222
BroadcastFactory.cpp
23-
BroadcastExchangeSource.cpp)
23+
BroadcastExchangeSource.cpp
24+
KeyValueSerializer.cpp)
2425

2526
target_link_libraries(
2627
presto_operators

0 commit comments

Comments
 (0)