Commit 1e42244
Introduce JK to Disable Memory Reallocation Optimization
Summary: When implementing the stream chunker, we anticipated that stream buffers after chunking will end up growing to the size that previously triggered chunking. As a tradeoff between minimizing reallocations (for performance) and actually releasing memory (to relieve memory pressure), we heuristically determine the new buffer capacity for each stream. The issue with this optimization is that it conflicts with the rest of our memory tracking logic since we now have retained memory in the memory pool that is not accounted for. We now through local testing that disabling this optimization leads to better memory pressure relief. In this diff, we introduce a JK, [dwio/nimble_chunking:disable_memory_reallocation_optimization](https://www.internalfb.com/intern/justknobs/?name=dwio%2Fnimble_chunking#disable_memory_reallocation_optimization), that will be enabled just for DISCO experiments. This will help us understand the full impact of this optimization and whether it should be retained.
Differential Revision: D874944271 parent 93a536d commit 1e42244
1 file changed
+7
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
| |||
32 | 33 | | |
33 | 34 | | |
34 | 35 | | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
35 | 42 | | |
36 | 43 | | |
37 | 44 | | |
| |||
0 commit comments