diff --git a/docs/advanced/index.md b/docs/advanced/index.md new file mode 100644 index 0000000..34dfdbc --- /dev/null +++ b/docs/advanced/index.md @@ -0,0 +1,9 @@ +--- +slug: /advanced +title: 🛰 Advanced +hide_title: true +--- + +# 🔌 Advanced + +Advanced topic. \ No newline at end of file diff --git a/docs/advanced/memory.md b/docs/advanced/memory.md new file mode 100644 index 0000000..4df6d2b --- /dev/null +++ b/docs/advanced/memory.md @@ -0,0 +1,17 @@ +--- +slug: /advanced/memory +title: 💽 Memory management +--- + +## Memory management +There are situations where you might need to limit the memory usage of Chroma. By default, Chroma loads the entire queried collection into its in-memory cache. To address this, we have implemented a mechanism to set a memory limit, which Chroma endeavors to adhere to as closely as possible through various cache eviction strategies. Here are some important points to note: + - Loading a collection into the cache can introduce some latency, as this process consumes resources. + - This approach is based on best effort, considering that real-time monitoring of RAM usage per collection is an estimation. + +##### Available Cache Strategies +- **Default**: Under this policy, there is no eviction process. Chroma loads all queried collections into memory and retains them until the process is restarted. +- **LRU** (Least Recently Used): In this strategy, Chroma evicts the least recently used collection to free up capacity for the newly queried collection. However,if a collection is larger than the set limit, Chroma will still attempt to load it, evicting other collections if necessary. + + +#### Setup +Set the environment variable `CHROMA_SEGMENT_CACHE_POLICY=LRU` to enable the LRU caching strategy. Additionally, define the memory limit by setting `CHROMA_MEMORY_LIMIT_BYTES=1000000000`, which restricts the cache to a maximum of 1GB. \ No newline at end of file diff --git a/sidebars.js b/sidebars.js index 62c6fd8..0ffc3a6 100644 --- a/sidebars.js +++ b/sidebars.js @@ -49,6 +49,12 @@ const sidebars = { label: '🔧 API Docs', className: 'category-link', }, + { + type: 'link', + href: '/advanced', + label: '🛰 Advanced', + className: 'category-link', + }, ], integrations: [ { @@ -72,6 +78,23 @@ const sidebars = { ], }, ], + advanced: [ + { + type: 'link', + label: '← Home', + href: '/' + }, + 'advanced/index', + { + type: 'category', + label: 'Advanced', + collapsed: false, + className: 'category-header', + items: [ + 'advanced/memory', + ], + }, + ], embeddings: [ { type: 'link',