Feat/optimize mem use#134
Merged
yanyiwu merged 5 commits intoyanyiwu:masterfrom Nov 27, 2025
Merged
Conversation
kevinmatthe
force-pushed
the
feat/optimize_mem_use
branch
2 times, most recently
from
7b4bf8f to
37361c7
Compare
- 修改分词函数返回类型为 Word* 替代 char** 以提高效率 - 新增内存整理函数 Trim() 用于释放未使用的内存 - 添加 FreeWithTrim() 和 WithTrim() 方法优化 Go 接口内存管理 - 实现零拷贝的 convertCWordToSlice 和 convertCWordToStructs 转换函数 - 新增基准测试文件 jieba_benchmark_test.go 用于性能评估 ⚡️ perf(jieba): 改进分词结果处理 - 使用 Word 结构体替代字符串数组存储分词结果 - 优化 C/Go 边界数据转换效率 - 减少内存分配和拷贝操作 ✅ test(jieba): 添加基准测试 - 新增 Cut/CutAll/CutForSearch 性能测试 - 添加内存占用对比测试 - 实现 RSS 内存统计功能
kevinmatthe
force-pushed
the
feat/optimize_mem_use
branch
from
37361c7 to
8e5b074
Compare
- 修复注释格式，增加空格提高可读性 - 统一变量声明方式，使用短变量声明 - 简化结构体初始化语法，移除冗余类型声明 - 重命名示例函数以更清晰地表达用途 ✅ test(jieba): 更新测试用例 - 保持测试覆盖率不变 - 优化测试数据格式 - 确保测试断言清晰明确
- 包含<stdlib.h>头文件以确保内存分配函数可用
- 新增Trim函数用于手动触发malloc_trim内存回收 - 移除旧的性能测试代码 - 添加新的性能测试文件jieba_trim_test.go ✅ test(jieba): 添加malloc_trim相关测试用例 - 添加BenchmarkTrimCost_Amortized测试不同负载下的Trim性能 - 添加TestTrimLatencyImpact测试Trim对延迟的影响 - 添加TestTrimLatency测试Trim的实际耗时 - 添加TestRSSRecovery验证Trim对内存回收的效果
- 将getRSS函数从jieba_benchmark_test.go移动到jieba_trim_test.go - 删除benchmark测试文件中未使用的导入包
Contributor
Author
|
Part1. 内存拷贝优化： Base分支的Benchmark： 待合入分支的Benchmark： 可见在高频调用的耗时上可以缩减接近30%的水平。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
预提交PR，TODO: 补充测试用例、性能报告
关联issue/问题：
#133 增加RSS提前释放
Cut、CutAll、CutForSearch存在过多的跨CGO内存拷贝，利用vector返回优化。