Skip to content

Commit c265805

Browse files
n24q02mclaude
andcommitted
fix: disable ORT memory pattern to prevent RAM growth with varying sequences
ONNX Runtime's enable_mem_pattern retains peak-sized activation buffers across inferences with different sequence lengths, causing steady RAM growth in long-running processes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 32ba3f5 commit c265805

1 file changed

Lines changed: 3 additions & 0 deletions

File tree

qwen3_embed/common/onnx_model.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,9 @@ def _load_onnx_model(
108108

109109
so = ort.SessionOptions() # type: ignore[possibly-missing-attribute]
110110
so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL # type: ignore[possibly-missing-attribute]
111+
# Disable memory pattern optimization to prevent ORT from retaining
112+
# peak-sized buffers across inferences with varying sequence lengths.
113+
so.enable_mem_pattern = False
111114

112115
if threads is not None:
113116
so.intra_op_num_threads = threads

0 commit comments

Comments
 (0)