Releases: apache/kvrocks
2.14.0
Highlights
Added WAIT — Blocks after a write until N replicas acknowledge (or timeout), returning the ack count. Enables synchronous-style replication for stronger durability and consistency in Kvrocks.
Added T-Digest — Probabilistic data type for fast, memory-efficient approximate percentiles on streams and large datasets. Ideal for latency histograms, telemetry, and anomaly detection. See supported commands.
Lua scripting: strict key-access mode — Enforces access only to keys declared via EVAL/FCALL. In this mode the global lock is removed, delivering much higher concurrency and throughput. Recommended for workloads that heavily use Lua scripts.
Dep Updates — RocksDB is upgraded to version 10.6.2 in this release.
Other notable updates — Added RISC-V support; significant replication performance/stability improvements; and Search enhancements, including a compaction filter and skipping expired keys during queries.
New features
- feat(replication): implement WAIT without timeout support by @zhixinwen in #3047
- feat(tdigest): add
TDIGEST.MERGEcommand implementation by @LindaSummer in #3054 - feat(replication): make replication delay configurable by @zhixinwen in #3087
- feat(namespace): add a subcommand to show the current ns name by @PragmaTwice in #3088
- feat(replication): clean up all wait_contexts in CleanupWaitConnection by @zhixinwen in #3104
- feat(replication): replication support group sync by @zhixinwen in #3092
- feat(function): support subcommand FUNCTION FLUSH by @PragmaTwice in #3119
- feat: Add support for RISC-V architecture by @wanghan-sanechips in #3053
- feat(replication): WAIT cmd supports timeout by @zhixinwen in #3117
- feat(scripting): support strict key-accessing mode for lua scripting by @PragmaTwice in #3139
- feat(config): add RocksDB periodic compaction controls by @sryanyuan in #3170
- feat(info): expose RocksDB level file count and pending compaction bytes via info by @sryanyuan in #3181
- feat(replication): add replication-no-slowdown config by @zhixinwen in #3219
Improvements
- feat(replication): implement WAIT based on replica acks by @zhixinwen in #3061
- feat(replication): set low watermark when reading replication data by @zhixinwen in #3098
- feat(function): propagate FUNCTION FLUSH/DELETE to replicas by @PragmaTwice in #3121
- feat(Dockerfile): use a numeric UID instead of a username by @SpecLad in #3138
- feat(script): make EVAL & FCALL exclusive if lua-strict-key-accessing is disabled by @PragmaTwice in #3155
- feat(command): return all flags in COMMAND INFO by @PragmaTwice in #3160
- feat(replication): optimize wait and replication locks by @zhixinwen in #3090
- feat(search): support to skip expired keys in query by @PragmaTwice in #3086
- feat(replication): implement _getack for quick WAIT response by @zhixinwen in #3075
- feat(search): add a compaction filter for search column family by @PragmaTwice in #3084
- chore(common): refactor parse utils via std::from_chars by @PragmaTwice in #3052
- build: disable LTO in debug mode by @PragmaTwice in #3069
- chore(metadata): optimize RedisTypeNames & make timeseries an emptyable type by @PragmaTwice in #3083
- chore(function): make LUA_GC_CYCLE_PERIOD a global constant by @PragmaTwice in #3116
- chore: remove unused
autoResizeBlockAndSSTmethod and config by @jonahgao in #3136 - chore(config): enable
level_compaction_dynamic_level_bytesby default by @jonahgao in #3144 - perf(storage): eliminate unnecessary
rocksdb::DB::ListColumnFamilies()by @jonahgao in #3145 - build: fix build with Homebrew Clang on macOS by @sryanyuan in #3154
- perf(scan): optimise scan performance when scanning keys with a tag by @sryanyuan in #3156
- perf(repl): only calculate crc of fetched file when needed by @jonahgao in #3161
- refactor(storage): optimize MDel with MultiGet options by @cyningsun in #3195
- chore(cluster): check and send the migration batches when the buffer is full by @hll1213181368 in #3227
Bug fixes
- fix(config): the default log-dir should include stdout by @PragmaTwice in #3085
- fix(migration): send migration batch while full to avoid oom by @greatsharp in #3094
- fix(search): support number literal for tag fields by @zhenghaoz in #3096
- fix(key): check status returned from SETEX and PSETEX by @hll1213181368 in #3108
- fix(hscan): HSCAN support NOVALUES by @LiuQhahah in #3109
- fix(search): support escaped character in tags by @zhenghaoz in #3110
- fix(function): reset lua states from all workers in FUNCTION DELETE by @PragmaTwice in #3115
- fix(search): convert query tag to lowercase if case insensitive by @zhenghaoz in #3118
- fix(search): wrong upper bound in reverse scan for infinity by @zhenghaoz in #3124
- fix(replication): Fix Seg Fault On Signal When Replication is Enabled by @zhixinwen in #3131
- fix(scan): pattern-based SCAN iterations may skip remaining keys by @sryanyuan in #3146
- fix(namespace): set token can delete another namespace by @jonahgao in #3149
- fix(test): flaky test case in KMETADTA command by @git-hulk in #3150
- fix(repl): check the status of listing checkpoint files by @jonahgao in #3158
- fix(list): check the status of rocksdb iterator by @jonahgao in #3168
- fix(stream): XPENDING won't return the idle and deliver count when reading with id '<' by @git-hulk in #3185
- fix(ts): Disable
TS.MGET/MRANGEin cluster mode by @yezhizi in #3182 - fix(json): allow to insert at t...
2.13.0
Highlights
This new release introduces significant features, improvements, and fixes.
A critical fix is that the server will crash when using ZRANDMEMBER for non-existent keys, please see #2982.
We introduce a few new commands and enhance some old commands, including:
- CLIENT REPLY to allow clients to control server reply behavior
- KPROFILE to enable/disable/dump the memory profiling
- FLUSHMEMTABLE to flush memtables
- FLUSHBLOCKCACHE to explicitly release the block cache
Another highlight is that the community is working to support the TDIGEST data structure, but we don't bring it to this release for stable consideration.
New Features
- feat(cmds): Add support for CLIENT REPLY subcommand and related tests by @DCjanus in #2943
- feat(tdigest): Add support for QUANTILE command by @SharonIV0x86 in #2849
- feat(command): add a new command to enable/disable/dump the memory profiling by @git-hulk in #3007
- feat(command): Add redis command to flush memtables by @Ryan4253 in #3022
- feat(command): add new command to release block cache by @sryanyuan in #3049
Improvements
- feat(search): add debug mode for FT.EXPLAINSQL by @PragmaTwice in #2886
- refactor: replace glog with spdlog by @PragmaTwice in #2881
- feat(logs): enable saving the slow logs into the file by @greatsharp in #2903
- chore(search): rename TopNSort to TopN by @xiedeyantu in #2930
- feat(event): add logging for blob file creation and deletion by @Ryan4253 in #3014
- feat(config): allow to configure the rocksdb.min_write_buffer_number_to_merge by @nathanlo-hrt in #3016
- feat(storage): allow to enable the blob cache by @Ryan4253 in #3027
- feat(search): accept DIALECT option in FT.SEARCH by @PragmaTwice in #3037
- feat(config): allow to configure the rocksdb.sst_file_delete_rate_bytes_per_sec by @sryanyuan in #3040
- chore(search): use c++ attributes gnu::may_alias by @PragmaTwice in #3045
- chore(log): replace logging calls in
cli/*andcommon/*by @PragmaTwice in #2888 - chore(log): replace logging calls in
utils/kvrocks2redisby @PragmaTwice in #2890 - chore(log): replace logging calls in
config/*by @sharatthk in #2891 - chore(log): add unreachable() based on the current logging infra by @PragmaTwice in #2893
- chore(log): replace logging calls in
types/*by @yezhizi in #2892 - chore(log): remove the DCHECK macro by @PragmaTwice in #2896
- chore(log): replace logging calls in
cluster/*by @SharonIV0x86 in #2894 - chore: fix & cleanup RESP error messages by @PragmaTwice in #2902
- chore(log): replace logging calls in commands/* by @XueSongTap in #2905
- chore(util): add prefix/suffix string util functions by @PragmaTwice in #2911
- chore(log): replace logging calls in server/* by @xiedeyantu in #2928
- chore(log): replace logging calls in storage/* by @SharonIV0x86 in #2909
- chore(log): replace logging calls in storage/rdb by @PragmaTwice in #2936
Bug Fixes
- fix(protocol): inline mode should allow the quoted string by @git-hulk in #2873
- fix: typo in compaction checker by @geonove in #2879
- fix(tdigest): fix centroid encoding overwrite for same mean by @LindaSummer in #2878
- feat(storage): support for sideloading SSTs by @ltagliamonte-dd in #2845
- fix(stream): should return an empty string instead of nil in XREAD/XRANGE by @git-hulk in #2897
- feat(script): pass storage context through scripting by @PragmaTwice in #2901
- fix(core): potential data race between blocking command and transaction by @git-hulk in #2910
- fix(zset): wrong RESP reply in ZRANDMEMBER and ZMSCORE command by @weim0000 in #2937
- fix(zset): crash in ZRANDMEMBER for non-existent keys by @weim0000 in #2982
- fix(zset): wrong RESP reply in ZMSCORE command for non-existent key by @weim0000 in #2983
- fix(list): dst-key's version is 0 in rpoplpush cmd for non-existent dst-key by @weim0000 in #2985
- fix(stream): ensure XINFO STREAM command has enough arguments by @rmsacks in #2994
- fix(search): missing the indexing phase for scripting by @PragmaTwice in #2998
- fix(txn): wrong RESP reply for EXEC if error occurred in commit by @PragmaTwice in #2996
- fix(event): should log reason string instead of id in EventListener::OnFlushCompleted by @Ryan4253 in #3012
- fix(storage): don't try to commit empty write batches by @nathanlo-hrt in #3015
- fix: Unset change_level if level_compaction_dynamic_level_bytes is enabled by @hotpxl in #3020
- fix(core): remove extraneous RocksDB::CancelAllBackgroundWork calls (#3017) by @Ryan4253 in #3019
- fix(core): Avoid hardcoding RocksDB property strings by @Ryan4253 in #3029
- fix(config): Parse rocksdb.max_bytes_for_level_base as uint64_t by @Ryan4253 in #3032
- fix(config): Setting rocksdb.level0_slowdown_writes_trigger to 0 disables it by @Ryan4253 in #3034
- fix(config): the default log-dir should include stdout by @PragmaTwice in #3085
Misc
- chore(deps): bump rocksdb to v10.0.1 by @aleksraiden in #2864
- chore: clean up cgo flags and conditionally compile grocksdb by @PragmaTwice in #2885
- chore(ci): mark the branch 2.12 as a protected branch by @git-hulk in #2895
- fix(ci): use CMAKE 3.31 instead of 4.0 in macOS by @git-hulk in #2899
- fix(ci): add workaround for rocksdb compiling on arch linux gcc 15 by @LindaSummer in #2929
- chore(deps): bump Google test to v1.17.0 by @aleksraiden in #2913
- chore(ci): update crate-ci/typos action to v1.32.0 by @aleksraiden in #2926
- chore(deps): bump fmtlib to 11.2.0 by @aleksraiden in #2925
- chore(tests): remove SST load command test and grocksdb by @git-hulk in #2942
- chore(deps): bump rocksdb to v10.2.1 by @aleksraiden in #2912
- chore(deps): bump spdlog to 1.15.3 by @aleksraiden in #2951
- ci: add a clang build job in ubuntu ARM runner by @PragmaTwice in #2952
- chore(README): refine the wording for supported platform, cluster and license by @PragmaTwice in #2953
- chore(config): mark the configuration
rocksdb.row_cache_sizeas deprecated by @git-hulk in #2989 - chore(config): change
raw-key-valueas the default cluster migration type by @git-hulk in #2990 - chore: remove ghproxy option in x.py by @PragmaTwice in #2991
- chore(deps): update go deps by @aleksraiden in #2999
- build(jemalloc): enable the profiling option when compiling the jemalloc by @git-hulk in #3001
- build: replace zlib with zlib-ng for rocksdb compression by @PragmaTwice in #3003
- chore(Dockerfile): enable the jemalloc option but no active by @git-hulk in #3011
- chore(deps): Bump cpptrace to v1.0.0 by @aleksraiden in #3025
- chore(deps): bump cpptrace to v1.0.2 by @aleksraiden in #3039
- chore(deps): bump oneTBB to v2022.2.0 by @aleksraiden in #3043
- chore(ci): bump crate-ci/typos to 1.34.0 by @aleksraiden in #3042
- chore(deps): bump rocksdb v10.4.2 by @aleksraiden in #3050
New Contributors
- @geonove made their first contribution in #2879
- @sharatthk made their first contribution in #2891
- @XueSongTap made their first contribution in #2905
- @xiedeyantu made thei...
2.12.1
This release is a patch release that addresses several important bug fixes. Users of old versions are strongly encouraged to upgrade to this new version.
Bug Fixes
2.12.0
Highlights
This new release introduces significant features, improvements, and fixes. A critical fix is that the server will crash when passing a negative offset in SETRANGE command, please see #2783.
We introduce a few new commands and enhance some old commands, including:
- HSETEXPIRE to allow setting a hash field with expiration time
- KMETADATA to fetch the metadata of a key
- POLLUPDATES supports using RESP output format
- LASTSAVE supports iso8601 time format
Another highlight is that the community is working to support the TDIGEST data structure, but we don't bring it to this release for stable consideration.
New Features
- feat(command): add support of RESP output format for the POLLUPDATES command by @git-hulk in #2745
- feat(hash): add the support of HSETEXPIRE command (HSET + EXPIRE) by @ltagliamonte-dd in #2750
- feat(server): add rocksdb_version, server_time_usec, executable and config_file to INFO by @PragmaTwice in #2781
- feat(info): support to specify multiple sections for INFO by @PragmaTwice in #2772
- feat(tdigest): add TDIGEST.CREATE and TDIGEST.INFO command by @LindaSummer in #2799
- feat(tdigest): add tdigest algorithm and storage encoding implementations by @LindaSummer in #2741
- feat(config): allow to configure the rocksdb's max compaction bytes by @git-hulk in #2802
- feat(tdigest): add TDIGEST.ADD command by @wyxxxcat in #2803
- feat(tdigest): add TDIGEST.MAX/MIN command by @wyxxxcat in #2811
- feat(cmd): add iso8601 time format parameter to LASTSAVE command by @SharonIV0x86 in #2822
- feat(tdigest): add support of TDIGEST.RESET command by @SharonIV0x86 in #2826
- feat(cmd): add new command KMETADATA by @ltagliamonte-dd in #2827
Improvements
- feat(script, function): remove global lock for EVAL, EVALSHA and FCALL by @PragmaTwice in #2731
- feat(main): refine shutdown message to show signal name by @PragmaTwice in #2733
- chore(command): add admin flag for permission-required commands by @PragmaTwice in #274
- feat(info): move the last scan time as field to the info keyspace section by @git-hulk in #2775
- refactor: adding stream elements to response by @torwig in #2836
- chore(metadata): wrap RedisTypeNames as a method by @PragmaTwice in #2838
- feat(conn): optimize RESP reply functions by @PragmaTwice in #2847
- feat(tdigest): refactor buffer merge's min max updating logic by @LindaSummer in #2814
Bug Fixes
- fix(script): disable exclusive commands to be executed in scripting by @PragmaTwice in #2732
- chore(x.py): fix several typing issue in x.py by @PragmaTwice in #2739
- feat(script): allow blocking commands in scripting by @PragmaTwice in #2740
- chore(config): enable the RESP3 potocol by default by @git-hulk in #2746
- fix(tests): flaky test case in BLMPOP command by @git-hulk in #2754
- fix(tls): remove SSL_sendfile path to avoid errors in replication fullsync by @PragmaTwice in #2757
- ci: fix build-push-action by supplying context argument by @PragmaTwice in #2780
- fix(string): the server will crash when passing a negative offset in SETRANGE command by @git-hulk in #2783
- fix(zset): The ZREMRANGEBYRANK command is missing LogData of the zset data type by @sryanyuan in #2808
- fix: access nullptr issue in LoadClusterNodes by @RiversJin in #2820
- fix(cluster): Resolve forbidden slot range cleanup bug during slot mgration by @RiversJin in #2829
- fix(bloom): crash due to pinnable slice in bloom filter by @mapleFU in #2843
- fix(bloom): try to fix GetSelf invalid access by @mapleFU in #2867
Misc
- chore(dockerfile): switch compiler to clang by @PragmaTwice in #2737
- chore(server): check the arity should happen before the flag generate by @git-hulk in #2736
- chore(cmake): add c to language list of
project()by @PragmaTwice in #2738 - chore: bump fmtlib to 11.1.3 by @aleksraiden in #2744
- chore: update copyright year in NOTICE by @PragmaTwice in #2748
7 - ci: add arm64 job for check-docker by @PragmaTwice in #2749
- ci: refactor nightly image workflow and utilize arm64 runners by @PragmaTwice in #2751
- chore: replace DCHECK_NOTNULL to suppress warnings by @PragmaTwice in #2753
- chore(ci): bump crate-ci/typos to 1.29.5 by @aleksraiden in #2755
- build: remove glibc-specific dependencies by @err931 in #2759
- ci: add an alpine-based build & test workflow by @PragmaTwice in #2760
- chore(test): update godeps by @aleksraiden in #2762
- ci: release nightly images for version tags by @PragmaTwice in #2769
- ci: isolate geoadd and georange test and run it in parallel by @lyang24 in #2768
- chore(server): improve code style of
Server::Get..Info()by @PragmaTwice in #2774 - chore: bump googletest to 1.16.0 by @aleksraiden in #2776
- ci: keep git history while building nightly images by @PragmaTwice in #2779
- chore(server): return InfoEntries instead of string in Get..Info() by @PragmaTwice in #2778
- chore: bump jsoncons to v1.2.0 by @aleksraiden in #2782
- chore(test): update godeps by @aleksraiden in #2785
- chore(ci): bump minica to v1.1.0 by @aleksraiden in #2787
- chore(ci): bump action apache/skywalking-eyes/header to 0.7.0 by @aleksraiden in #2788
- ci: execute long running go tests in parallel by @lyang24 in #2763
- chore(build): bump golang-lint to 1.64.3 by @aleksraiden in #2786
- chore: bump Zstandard (zstd) to v1.5.7 by @aleksraiden in #2797
- chore(test): update godeps by @aleksraiden in #2798
- chore(ci): bump crate-ci/typo to v1.2.7 by @aleksraiden in #2790
- chore: remove the duplicate const error message by @git-hulk in #2800
- chore: remove redundant code in the method
Executeofslaveofcmd by @fukua95 in #2801 - chore: bump cpptrace to v0.8.0 by @aleksraiden in #2796
- chore: bump fmtlib to 11.1.4 by @aleksraiden in #2809
- chore: remove need for the intermediate variable in Size function by @SharonIV0x86 in #2810
- chore(ci): bump crate-ci/typos to v1.30.0 by @aleksraiden in #2812
- chore: bump jsoncons v1.3.0 by @aleksraiden in #2816
- chore(test): update go deps by @aleksraiden in #2824
- chore: bump rocksdb to v9.11.1 by @aleksraiden in #2818
- build(deps): bump go-redis/v9 from 9.7.1 to 9.7.3 by @dependabot in #2840
- chore(deps): bump Google Snappy to v1.2.2 by @aleksraiden in #2854
- chore(deps): bump oneTBB to 2022.1.0 by @aleksraiden in #2855
- chore(ci): bump crate-ci/typos to v1.31.0 by @aleksraiden in #2857
- chore: set CMAKE_POLICY_VERSION_MINIMUM & pin the version of dorny/paths-filter by @PragmaTwice in #2870
- ci: migrate macos-13 build to macos-14 by @PragmaTwice in #2876
- chore(deps): bump jsoncons to v1.3.1 by @aleksraiden in #2875
- chore(deps): bump cpptrace to v0.8.3 by @aleksraiden in #2872
New Contributors
2.11.1
This release is a patch release which includes several important bug fixes (including the security fix for CVE-2025-25069).
Users of old versions are strongly encouraged to upgrade to this new version.
Bug Fixes:
- fix(tls): remove SSL_sendfile path to avoid errors in replication fullsync by @PragmaTwice in #2757
- fix(conn): detect and drop HTTP requests in connection by @PragmaTwice in #2765
Misc:
- ci: refactor nightly image workflow and utilize arm64 runners by @PragmaTwice in #2751
- ci: release nightly images for version tags by @PragmaTwice in #2769
Full Changelog: v2.11.0...v2.11.1-rc1
2.11.0
Highlights
Note that RocksDB has been upgraded from v9.3.1 to v9.10.0 in this version.
We are excited to announce the release of Kvrocks Search in this version, which gives Kvrocks the ability of secondary indexing and structured querying. You can find detailed information on the website. Please note that this feature is still experimental, and you can provide feedback through GitHub Issues if you encounter any problems.
New Features
- feat: support histograms for command latency statistics by @rabunkosar-dd in #2721
- feat(search): support the FT.TAGVALS command by @jonathanc-n in #2564
- feat(worker): Allow passing a socket FD to dup and listen on by @nathanlo-hrt in #2598
- feat(config): support retrieving configuration fields using glob patterns by @sryanyuan in #2592
- feat(config): add
rocksdb.wal_compressionto allow enable wal compression by @paragor in #2607 - feat(config): add
rocksdb.compression_start_levelto allow configure levels without compression by @paragor in #2605 - feat(keys, scan): Support arbitrary glob patterns by @nathanlo-hrt in #2608
- feat(cluster): add support of the JSON type in cluster migration by @git-hulk in #2654
- feat(config): make rocksdb.dump_malloc_stats configurable by @nathanlo-hrt in #2658
- feat(cmd, config): support REPLICAOF command and replicaof config option by @PragmaTwice in #2678
- feat: skip block cache deallocation to make shutdown fast by @wanghenshui in #2683
- feat(config): add the
partition_filtersoption to disable the partitioned filters by @fukua95 in #2688
Improvements
- feat(command): add slow flag to more commands by @PragmaTwice in #2570
- feat(json): change default max_nesting_depth to 1024 by @PragmaTwice in #2572
- feat(search): add field options and definition part to FT.INFO by @PragmaTwice in #2571
- feat(conn): move time-consuming ops out of ExecuteCommand for context by @PragmaTwice in #2611
- feat(config): Minor update comment for compression_start_level error code by @mapleFU in #2615
- feat(cmd): avoid to use CommandAttributes::flags directly by @PragmaTwice in #2619
- feat(cmd): make key range fields private in CommandAttributes by @PragmaTwice in #2621
- feat(resp): optimize simple string "OK" construction by @RiversJin in #2627
- feat(status): support rocksdb::Status fowarding in GET_OR_RET by @PragmaTwice in #2630
- feat(cmd): add blocking flag and remove useless flags by @PragmaTwice in #2637
- feat(search): add mutex to HNSW index updating by @PragmaTwice in #2649
- feat(search): print properties for vector field in FT.INFO by @PragmaTwice in #2655
- feat(cmd): all blocking commands should be no-script by @PragmaTwice in #2666
- feat(cmd): add the auth flag for authentication checking by @PragmaTwice in #2669
- feat(conn): avoid to check the command name directly by @PragmaTwice in #2668
- feat(cmd): add more information to shutdown log by @PragmaTwice in #2672
- feat(conn): allow CONFIG in stale-data replica by @PragmaTwice in #2677
- feat(search): add TAGGED as an alternative syntax to HASTAG by @PragmaTwice in #2692
- feat(search): trim whitespaces in tag values by @PragmaTwice in #2702
- refactor: promote the
engine::Contextin theExecuteby @PokIsemaine in #2586 - refactor: hoist key mutexes to ExecuteCommands by @PokIsemaine in #2620
- refactor: initialize google logger after parsing options by @aleksraiden in #2664
- refactor: reduce memory copying during incremental synchronization by @RiversJin in #2689
- refactor: move rdb related code to
storage/rdbfolder by @LindaSummer in #2595 - perf(cluster): add upper bound for scan during migration by @RiversJin in #2724
Bug Fixes
- fix: Better checking for prefix matches by @nathanlo-hrt in #2599
- fix(command): wrong key range in commands: DISK/MEMORY/DUMP by @git-hulk in #2623
- fix(cmd): add exclusive flag to FLUSHDB/FLUSHALL by @PragmaTwice in #2624
- fix(script): avoid SetCurrentConnection on read-only scripting by @PragmaTwice in #2640
- fix(cluster): should remove the master if it's not node in the cluster by @git-hulk in #2643
- fix(bugfix): for txn_write_batch creation by @mapleFU in #2648
- fix(log): fix rocksdb stall condition string output by @mapleFU in #2651
- fix(cmd): args should be parsed before retrieving keys in COMMAND GETKEYS by @PragmaTwice in #2661
- fix(stream): add
KeyRangeGenforXReadandXReadGroupby @LindaSummer in #2657 - fix(cmd): always print ok log when get full data file info failed by @sryanyuan in #2663
- fix(replication): slave blocks until keepalive timer is reached when master is gone without fin/rst notification by @sryanyuan in #2662
- fix(string): reset the value of expired key for incrby cmd by @FishYoung in #2667
- fix(string): reset the value of expired key for SETRANGE cmd by @weim0000 in #2686
- fix(cluster): should stop the migration if it's changed to the slave role by @RiversJin in #2716
Misc
- chore: bump jsoncons to v0.177.0 by @aleksraiden in #2493
- chore(search): update README to add some references for KQIR by @PragmaTwice in #2581
- chore: Bump cpptrace to v0.7.2 by @aleksraiden in #2585
- chore: add branch 2.10 as a protected branch in .asf.yaml by @PragmaTwice in #2591
- chore: Bump rocksdb to v9.7.2 by @aleksraiden in #2600
- chore: bump oneTBB to v2022.0.0 by @aleksraiden in #2632
- chore: bump jsoncons to v0.178.0 by @aleksraiden in #2631
- chore: Add devcontainer config for Kvrocks by @what-is-me in #2634
- chore: bump rocksdb to v9.7.4 by @aleksraiden in #2639
- chore(cmd): add comments for command flags by @PragmaTwice in #2641
- chore(search): improve code structure of IndexManager by @PragmaTwice in #2650
- chore(enhancement): rename is_txn_mode to txn_context_enabled by @mapleFU in #2644
- chore: bump cpptrace to v0.7.3 by @aleksraiden in #2670
- chore: bump glog to 0.7.1 by @aleksraiden in #2106
- chore: bump rocksdb to v9.8.4 by @aleksraiden in #2687
- chore: add move assign operator for MultiLockGuard by @PragmaTwice in #2693
- chore: bump rocksdb to v9.9.3 by @aleksraiden in #2694
- chore: bump cpptrace to v0.7.4 by @aleksraiden in #2696
- chore: bump fmtlib to 11.1.0 by @aleksraiden in #2698
- chore: bump fmtlib to v11.1.1 by @aleksraiden in #2699
- chore: bump xxhash to v0.8.3 by @aleksraiden in #2700
- chore: bump jsoncons to v1.0.0 by @aleksraiden in #2704
- chore: bump rocksdb to v9.10.0 by @aleksraiden in #2705
- chore: bump cpptrace to v0.7.5 by @aleksraiden in #2709
- chore(tests): use built-in min and max by @fukua95 in #2718
- chore: bump fmtlib to 11.1.2 by @aleksraiden in #2720
- chore: bump jsoncons to v1.1.0 by @aleksraiden in #2719
- docs: Add Kvrocks' Zulip URL to README by @PragmaTwice in #2588
- docs: remove the Slack invitation link from README.md by @git-hulk in #2593
- test(integration): add integration t...
2.10.1
This release addresses some critical bugs found in 2.10.0.
Bug Fixes
- fix(stream): change XREADGROUP response for empty read response #2569 by @jonathanc-n
- fix(stream): Fix XPENDING serialization issue #2566 by @nathanlo-hrt
- fix(stream): fix
XINFOgroup metadata key overlapping. #2576 by @LindaSummer - fix: some rocksdb properties are counted multiple times #2589 by @sryanyuan
- fix: should remove the db lock from the context to prevent blocking user requests #2597 by @git-hulk
- fix(json): JSON.MSET may update on the old json value #2579 by @poipoiPIO
- fix(rocksdb): downgrade RocksDB to v9.3.1 due to a blobdb issue #2614 by @git-hulk
2.10.0
Important Notice
This new release introduces significant features, improvements, and fixes.
Dependency Updates: RocksDB version bumps from 8.11.4 to 9.6.1.
Highlights
The new data structure HyperLogLog and its commands are supported now, users can use PF* commands in Kvrocks. Thanks to contributors: @mapleFU @tutububug.
All stream group commands are supported in this release which makes Kvrocks totally compatible with Redis stream.
We allow to enhance the transaction consistency by introducing the db context. It will always use the same snapshot in the transaction while the configuration txn_context_enabled is enabled. Thanks to contributor: @PokIsemaine
Kvrocks Search commands and HNSW index are supported in Kvrocks, but we haven't released them because they're not ready for production at the moment.
New Features
- feat(config): add compaction-checker-cron to extend original compaction-checker-range by @PragmaTwice in #2383
- feat(search): Add HNSW encoding index & insertion/deletion algorithm by @Beihao-Zhou in #2368
- feat(cluster): support migrate slot range by @PokIsemaine in #2389
- feat(config): change redis-cursor-compatible default value to yes by @jihuayu in #2429
- feat(json): add support of JSON.RESP command by @jackyyyyyssss in #2390
- feat(search): support query parameters in SQL and RediSearch query by @PragmaTwice in #2443
- feat(search): Hnsw Vector Search Plan Operator & Executor by @Beihao-Zhou in #2434
- feat(config): allow to change the max length of the bulk string by @git-hulk in #2444
- feat(config): implement integer config option with the unit by @PragmaTwice in #2448
- feat(hyperloglog): add support of the Hyperloglog data structure by @tutububug in #2142
- feat(search): implement vector query for sql/redisearch parser & transformer by @Beihao-Zhou in #2450
- feat(hyperloglog): Add support of PFMERGE command by @mapleFU in #2457
- feat(search): add support of HNSW vector fields for FT.CREATE by @Beihao-Zhou in #2477
- feat(command): Implement the new command POLLUPDATES for polling updates by sequence by @git-hulk in #2472
- feat(conn): add arguments for unknown command message by @PragmaTwice in #2483
- feat(search): allow to omit
ON HASH|JSONin FT.CREATE by @PragmaTwice in #2484 - feat(script): Support script flags of Eval script and Function by @PokIsemaine in #2446
- feat: Add slow flags for commands with heavy disk I/O by @jonathanc-n in #2494
- refactor: Improve consistency and isolation semantics by adding Context parameter to DB API by @PokIsemaine in #2332
- feat(config): add
txn_context_enabledto allow to enable the transaction feature by @PokIsemaine in #2506 - feat: support limit WriteBatch size by @AntiTopQuark in #2508
- feat: Bitmap type DUMP/RESTORE command support by @poipoiPIO in #2535
Bug Fixes
- fix(stream): NOGROUP/BUSYGROUP error message should return without 'ERR' prefix by @git-hulk in #2397
- fix: Add timeouts to go-redis client and ping on client creation by @torwig in #2404
- fix(search): misidentify DESC token in SQL parser by @PragmaTwice in #2439
- fix: monitor dumps 0xffffff prefix for signed char by @PragmaTwice in #2482
- fix(replication): potential deadlock when switching master frequently by @git-hulk in #2516
- fix(namespace): didn't reload namespaces from DB after the full sync by @git-hulk in #2527
- fix(replication): didn't resume the db status after restarting full sync by @git-hulk in #2549
- fix(rdb): incorrect type conversion during RDB loads the int8 encoding by @fstd2 in #2547
- fix: add more check in cluster node parsing by @jonathanc-n in #2538
- fix(test): use fixed seed to improve reproducibility. by @LindaSummer in #2557
- fix(config): wrong rocksdb config name max_sub_compactions by @ChrisZMF in #2499
Code Improvements
- refactor: rename variable names for clarity in LCS method by @shoothzj in #2392
- feat: add
iter->status()check for loop iterators by @LindaSummer in #2395 - perf(conn): skip index matching when index map is empty by @PragmaTwice in #2405
- chore(resp): extract RESP functions from Connection by @PragmaTwice in #2406
- refactor(stream): change the encoding of stream consumer group by @Yangsx-1 in #2384
- chore: fix compiler warning in cluster and HNSW indexing by @jjz921024 in #2436
- feat(stream): add support of XPENDING command by @Yangsx-1 in #2387
- chore(hyperloglog): add go test cases and minor optimizations/bugfixes by @mapleFU in #2463
- feat(search): Hnsw Vector Search Optimizaton Pass by @Beihao-Zhou in #2466
- feat(command): add category enum for every command by @PragmaTwice in #2479
- chore: use
StringJoininEnumField::Setby @PragmaTwice in #2485 - chore: improve the style and document for SlotMigration by @mapleFU in #2465
- fix(test): remove meaningless
list-max-ziplist-sizeby @PokIsemaine in #2517 - fix(typo):
filed -> fieldby @AntiTopQuark in #2519 - test: add go test cases for
txn-context-enabledconfig by @PokIsemaine in #2530 - chore(tests): add basic tests for the stream consumer group by @jonathanc-n in #2533
- chore(tests): enhance the namespace replication to prevent flaky test by @git-hulk in #2540
- chore(tests): refactor and enhance test cases to prevent flaky test failure by @git-hulk in #2544
Build & CI Changes
- ci: Bump build-push-action to v6 by @aleksraiden in #2367
- chore: protect release branches in .asf.yaml by @PragmaTwice in #2385
- fix(.asf.yaml): branch protection cannot be created by @PragmaTwice in #2386
- ci: use official typos github action by @shoothzj in #2388
- chore(ci): drop CentOS7 in CI since it's EOL now by @git-hulk in #2393
- chore: Bump fmtlib to 11.0.0. by @aleksraiden in #2391
- chore: Bump fmtlib to 11.0.1 by @aleksraiden in #2394
- chore: Bump rocksdb to v9.3.1 by @aleksraiden in #2380
- chore: Bump oneTBB to 2021.13.0 by @aleksraiden in #2375
- chore(.asf.yaml): add release branch 2.9 to protected branches by @PragmaTwice in #2396
- chore(test): bump dependencies of go test by @aleksraiden in #2400
- chore(test): bump dependencies of go-redis by @aleksraiden in #2407
- chore: bump Rocksdb to v9.4.0 by @aleksraiden in #2408
- ci: Update MacOS runner to macos13 by @aleksraiden in #2324
- ci: update runner os from ubuntu 20 to 22 by @PragmaTwice in #2409
- chore(x.py): improve style of commit message for release by @PragmaTwice in #2410
- chore(x.py): revert gotest timeout to 1800sec by @aleksraiden in #2413
- chore: bump googletest to v1.15.0 by @aleksraiden in #2430
- chore: bump cpptrace to v0.6.3 by @aleksraiden in #2412
- chore: bump fmtlib to v11.0.2 by @aleksraiden in #2440
- chore: bump lz4 to v1.10.0 by @aleksraiden in #2441
- feat(ci): integrate Rocky Linux environment into the CI workflow by @SpiffyEight77 in #2451
- build(cmake): update compiler version requirement by @PragmaTwice in #2455
- chore: bump googletest to v1.15.2 by @aleksraiden in #2460
- chore(storage): Remove speedb due to lack of active by @git-hulk in #2476
- feat: use non-root use...
2.9.0
Important Notice
This new release introduces significant features, improvements, and fixes.
Numerous new commands are now supported, such as DUMP, SORT, LCS, COPY, ASKING, READONLY, and READWRITE. These additions enhance Kvrocks' capabilities by improving data migration and cluster features. A big thank you to all contributors for their valuable input to Kvrocks.
The default value of CMake option ENABLE_NEW_ENCODING is set to ON, which means the new data type encoding (specified here) is enabled by default while all old data is still compatible. (Note: After you upgrade to Kvrocks 2.9.0 and write new data, if you roll back Kvrocks to 2.3.0 or earlier versions, these newly-written data will not be available.)
Async IO in RocksDB is enabled by default since this release which might bring the performance benefit for the scan behavior if the io uring is supported.
In addition, the config option compaction-checker-range is still available but is marked as deprecated. We recommend using compaction-checker-cron instead. Also, the cron syntax in the config file has been extended. For example, you can now use cron expressions like * */2 1,3-6,8 * *.
Kvrocks Search (compatible with both SQL and RediSearch queries) and stream groups are two experimental new features that are already available in the unstable branch, but they are not enabled in this release.
RocksDB is still in version 8.11 (8.11.4) for this version. However, in the next feature release, we plan to bump RocksDB to version 9.
New Features
- Add the support of the LCS command by @JoverZhang in #2116
- Add compression level option by @Beihao-Zhou in #2171
- Add support of READONLY and READWRITE by @LiuYuHui in #2173
- Support value setting for nonexistent paths in JSON.SET by @PragmaTwice in #2178
- Add BIT support to BITPOS by @sheharyaar in #2170
- Add support of the command MOVEX by @Chiro11 in #2225
- Add support of the command DUMP by @AntiTopQuark in #2227
- Add support of the command JSON.MSET by @zjregee in #2228
- Add support of the SORT command by @PokIsemaine in #2262
- Support ASKING command by @jjz921024 in #2273
- Add support of the command JSON.DEBUG MEMORY by @jackyyyyyssss in #2323
- Enable the rocksdb async io by default by @xiaobiaozhao #2308
Bug Fixes
- Fix glog may buffer the stdout's output by @i18n-now in #2131
- Fix duplicate flag for building rocksdb by @aleksraiden in #2139
- Fix build warnings: local variable 's' will be copied despite being returned by name by @aleksraiden in #2148
- Fix missing fields in HELLO command response by @git-hulk in #2150
- Fix XREADGROUP command didn't fetch the latest metadata after creating a consumer by @Yangsx-1 in #2153
- Fix the pidfile and backup_dir will be rewritten even though they weren't modified by @git-hulk in #2186
- Fix wrongly append the ERR prefix in no script error by @lloydzhou in #2201
- Fix should forbid importing the slot which belongs to itself in cluster mode by @git-hulk in #2209
- Fix GetOptions in C++17 build by @mapleFU in #2213
- Fix wrongly try to rewrite the namespace in the cluster mode by @git-hulk in #2221
- Fix should use the minimum compatible RDB version when dumping the payload by @git-hulk in #2252
- Fix the replica should remove the master link when receiving the CLUSTER RESET command by @git-hulk in #2259
- Fix unused-variable error for lower gcc version due to structure bindings by @13015517713 in #2265
- Fix the error logging when the importer is not in a started state by @git-hulk in #2280
- Fix JSON.[STRLEN|OBJLEN] response inconsistently with Redis when the path doesn't exist by @jackyyyyyssss in #2336
- Fix minica version in CI workflow to prevent TLS test failure by @PragmaTwice in #2307
- Fix compiler warnings in KQIR and RDB by @raffertyyu in #2303
- Fix plan operator child iterator in KQIR by @PragmaTwice in #2282
- Fix JSON.ARRTRIM key no exists by @jackyyyyyssss in #2374
- Fix config: avoid rewriting the config file if it's unnecessary by @git-hulk in #2347
- Fix stream: make consumer decrement pending number when message is acknowledged by @LindaSummer in #2352
Code Improvements
- Extract common/port.h and optimize dbstats by @mapleFU in #2145
- Add macos arm64 runner to actions workflow by @PragmaTwice in #2155
- Optimize numeric comparison via interval analysis in KQIR by @PragmaTwice in #2257
- Refactor TTL processing in string type by @PragmaTwice in #2250
- Improve code style in cluster by @jihuayu in #2272
- Refactor Column Family Handling by @mapleFU in #2296
- Improve logging message for retryable background IO errors by @VasuDevrani in #2317
- Optimize the implementation of IntervalSet intersection by @13015517713 in #2300
- Improve RESP handling code in replication by @PragmaTwice in #2334
- Refactor cron: add CronPattern and implement more cron syntax by @PragmaTwice in #2377
Build & CI Changes
- Uniform image update/upgrade in Dockerfile by @aleksraiden in #2138
- Try to workaround the broken ubuntu image by @git-hulk in #2161
- Revert broken ubuntu image temporary fix by @enjoy-binbin in #2189
- Bump zstd to v1.5.6 by @aleksraiden in #2204
- Add semantics checker for KQIR by @PragmaTwice in #2207
- Add a simple pass manager for KQIR by @PragmaTwice in #2226
- Bump Snappy to 1.2.0 by @aleksraiden in #2224
- (ci) Bump apache/skywalking-eyes to 0.6.0 by @aleksraiden in #2241
- Enable CI for kvrocks2redis by @Zakelly in #2175
- ci(pr-lint): force the PR title to follow the conventional commits by @PragmaTwice in #2342
- Upload build artifact for the sonar scan by @git-hulk in #2321
- Fix missing temp dir while running sonar scan by @git-hulk in #2322
- Bump Snappy v1.2.1 by @aleksraiden in #2325
- Set ENABLE_NEW_ENCODING=FALSE in CI workflow by @PragmaTwice in #2333
- (ci) Remove 'ps aux' command from kvrocks2redis test by @aleksraiden in #2337
- chore: bump rocksdb to v9.2.1 by @aleksraiden in #2327
- chore: bump jsoncons to v176.0 by @aleksraiden in #2328
- test: fix listpack decoding in kvrocks2redis integration tests by @PragmaTwice in #2345
- build: set ENABLE_NEW_ENCODING=FALSE when building kvrocks2redis by @PragmaTwice in #2350
- chore: bump rocksdb to v9.2.2 by @aleksraiden in #2351
Documentation Changes
- Add more documents in KQIR by @PragmaTwice in #2196
- Fix invalid url in docs by @mapleFU in #2192
- Update "Type Conversion" doc by @PragmaTwice in #2235
- Fix doc typo by @Yangsx-1 in #2240
- Add more docs for the dynamic column family by @mapleFU in #2298
2.8.0
Important Notice
Dependency Updates: RocksDB is bumped from 8.8.1 to 8.11.3 in this version.
Highlights:
Set the default value of redis-cursor-compatible to yes, so from this version, Kvrocks' SCAN will be compatible with Redis by default. We also proudly announce that RESP3 is fully supported now, you can enable the RESP3 feature via set resp3-enabled yes.
In addition, the cluster now can use the rocksdb write-batch to migrate slot data between instances via the APPLYBATCH command, which is faster and consumes less CPU compared to the replay command way.
It is worth mentioning that the community is working on the RedisSearch module, and we look forward to seeing RedisSearch in a future release.
New Features
- Add the support of JSON.MGET command by @skyitachi in #1930
- Add support of new command: ssubscribe and sunsubscribe by @raffertyyu in #2003
- Add support of new command: RESET by @chrisxu333 in #1999
- Add the ApplyBatch command for data migration scenario by @caipengbo in #2010
- Add support of new command: ZRANDMEMBER by @JxLi0921 in #2016
- Add support of the command ZDIFF and ZDIFFSTORE by @MaheshMadushan in #2021
- Add support of the command RENAME&RENAMENX by @jihuayu in #2026
- Add support of using hyper clock cache as the block cache by @JxLi0921 in #2031
- Implement the unify WAL iterator by @caipengbo in #2040
- Initialize metadata and tag indexing encoding for RediSearch by @PragmaTwice in #2066
- Allow to migrate cluster slot by raw write batch by @caipengbo in #2067
- Add search metadata encoding for numeric fields by @PragmaTwice in #2086
- Support BIT and BYTE options in the BITCOUNT command by @kay011 in #2087
- Add a noindex flag to search metadata by @PragmaTwice in #2094
- Set the redis-cursor-compatible field to yes in the initial configuration by @jihuayu in #2103
- Implement index updating for numeric and tag field by @PragmaTwice in #2115
- Fallback to use the redis command migration type if the target don't support the ApplyBatch command by @caipengbo in #2117
- Implement the RESP3 boolean type by @git-hulk in #1991
- Implement the RESP3 null for the nil string and array by @git-hulk in #2017
- Implement RESP3 set type for the set data structure by @git-hulk in #2024
- Add the support of RESP3 map type by @git-hulk in #2028
- Add RESP3 big number by @git-hulk in #2034
- Add the support of RESP3 double type by @git-hulk in #2053
- Add support of RESP3 attribute type by @git-hulk in #2088
- Add support of RESP3 verbatim string by @wsehjk in #2102
- Add support of RESP3 in Lua by @git-hulk in #2119
Bug Fixes
- Fix redundant ERR prefix in cluster redirect error message by @git-hulk in #1928
- Avoid to fetch old value in SET command if NX/XX/GET/KEEPTTL is not set by @git-hulk in #1968
- Fix parse BITCOUNT by @tisonkun in #1979
- Fix RESET arity and command flags by @enjoy-binbin in #2018
- Fix crash in zset store getkeys, fix zdiff/bzmpop range, add tests by @enjoy-binbin in #2051
- Bugfix: BITCOUNT/BITPOS negative handling fixing by @mapleFU in #2069
- Fix incorrent bytes to calculate length of ziplist/listpack 32bit string by @git-hulk in #2124
Improvements
- Add redis_mode/kvrocks_mode fields in INFO SERVER by @enjoy-binbin in #1927
- Add the rocksdb's block cache hit/miss count to the INFO command by @git-hulk in #1938
- Add SET command KEEPTLE and GET options supported by @jihuayu in #1935
- Add no-multi no-script flags to SHUTDOWN command by @enjoy-binbin in #1957
- Add support of EXPIRETIME and PEXPIRETIME by @kay011 in #1965
- Record and export the keyspace hit/miss count to INFO command by @git-hulk in #1971
- Add the support of ANALYZE command to inspect the performance of RocksDB by @chrisxu333 in #1943
- Add INFO comment when DB not scaned by @jihuayu in #1956
- Allow to get the latest sequence number when creating the backup by @git-hulk in #1987
- Add support of BITFIELD_RO by @jyf111 in #1990
- Add the support of ZINTER and ZINTERCARD by @kay011 in #1992
- Replace manual RedisType check by calling IsSingleKVType by @PragmaTwice in #2013
- Make the result of SRANDMEMBER random by @wsehjk in #2032
- Ignore max-db-size limit when deleting data or writing aux informations by @caipengbo in #2047
- Use human readable string instead of the integer when printing flush/compaction reason by @git-hulk in #1962
- Add usage of std::move where compiler suggested by @torwig in #2099
- Check if the type is an entry while iterating the stream subkeys by @jihuayu in #2112
- Make BITPOS in Bitmap handling
stop_givenby @mapleFU in #2085 - Make COMMAND command consistent with redis when the renamed command exists by @caipengbo in #2123
- Bump jsoncons to 0.173.2 by @aleksraiden in #2082
- Bump speedb to 2.8.0 by @aleksraiden in #2080
- Bump fmtlib to 10.2.1 by @aleksraiden in #2081
- Bump jsoncons to 0.173.4 by @aleksraiden in #2089
- Bump span-lite to 0.11.0 by @aleksraiden in #2092
- Bump typo checker in CI to 1.18.1 by @aleksraiden in #2093
- Bump rocksdb to v8.9.1 by @aleksraiden in #1933
- Bump rocksdb to 8.10.0 by @aleksraiden in #2005
- Bump rocksdb to 8.10.2 by @aleksraiden in #2110
- Bump rocksdb to 8.11.3 by @aleksraiden in #2122
Misc
- Unify the way of parsing metadata by @git-hulk in #1964
- Optimize ParseMetadata by introducing RedisTypes by @PragmaTwice in #1967
- Update copyright date in NOTICE by @PragmaTwice in #1975
- Refactor kvrocks2redis via rocksdb secondary instance by @maochongxin in #1963
- The storage engine type awareness in Kvrocks by @MaheshMadushan in #1973
- Implement an unify key-value iterator for Kvrocks by @git-hulk in #2004
- Minor refactor the implementation of the command ZRANDMEMBER by @mapleFU in #2025
- Add type check before metadata decode by @jihuayu in #2015
- Minor: improve the style of zdiff by @mapleFU in #2027
- Clarification in kvrocks.conf that disable_wal will break replication by @pleaseshutup in #2029
- Run SonarQube scan in Github Actions to support coverage analysis by @PragmaTwice in #2030
- Fix build file missing in SonarCloud CI by @PragmaTwice in #2033
- Fix shell command in SonarCloud CI by @PragmaTwice in #2037
- Only runs sonar action on the main repo by @jihuayu in #2038
- Minor: avoid the implicit conversion from PinnableSlice to Slice by @mapleFU in #2039
- Remove the second parameter of MultiBulkString by @PragmaTwice in #2042
- Use condition argument of cv.wait instead of loop by @PragmaTwice in #2043
- fix(debug command): update err msg
int->integerby @AntiTopQuark in #2046 - Enhance error handling and types in BITFIELD by @mapleFU in #2041
- Dump github event information in CI by @PragmaTwice in #2049
- Fix sonar ...