5.2.2 Harbor Boost Configuration
Harbor Boost is configured using environment variables. Following options are available:
Type:
StrList
Default: ``
An alias for HARBOR_BOOST_OPENAI_URLS.
Type:
StrList
Default: ``
An alias for HARBOR_BOOST_OPENAI_KEYS.
Type:
StrList
Default: ``
A semicolon-separated list of URLs to the OpenAI APIs to boost.
Prefer using named APIs via HARBOR_BOOST_OPENAI_URL_* and HARBOR_BOOST_OPENAI_KEY_* instead.
Must index-match contents of HARBOR_BOOST_OPENAI_KEYS.
Example:
HARBOR_OPENAI_URLS=https://localhost:11434/v1;https://localhost:8080/v1Type:
StrList
Default: ``
A semicolon-separated list of API keys to use for the OpenAI APIs to boost.
Prefer using named APIs via HARBOR_BOOST_OPENAI_URL_* and HARBOR_BOOST_OPENAI_KEY_* instead.
Must index-match contents of HARBOR_BOOST_OPENAI_URLS.
Example:
HARBOR_OPENAI_KEYS=sk-abc123;sk-def456Type:
str
Default: ``
Named OpenAI-compatible API URLs to boost.
* means multiple variables can be defined with arbitrary postfix.
Example:
HARBOR_BOOST_OPENAI_URL_OLLAMA=https://localhost:11434/v1
HARBOR_BOOST_OPENAI_KEY_OLLAMA=sk-ollama123
HARBOR_BOOST_OPENAI_URL_HF=https://api-inference.huggingface.co/v1
HARBOR_BOOST_OPENAI_KEY_HF=sk-hf456Type:
str
Default: ``
Example:
HARBOR_BOOST_OPENAI_URL_OLLAMA=https://localhost:11434/v1
HARBOR_BOOST_OPENAI_KEY_OLLAMA=sk-ollama123
HARBOR_BOOST_OPENAI_URL_HF=https://api-inference.huggingface.co/v1
HARBOR_BOOST_OPENAI_KEY_HF=sk-hf456Type:
ConfigDict
Default:temperature=0.35
Allows to specify extra payload for /chat/completions endpoints for all downstream services at once.
Format is key=value,key2=value2,....
Example:
HARBOR_BOOST_EXTRA_LLM_PARAMS=temperature=0.35,top_p=0.9
# Be careful using provider-specific parameters
HARBOR_BOOST_EXTRA_LLM_PARAMS=temperature=0.12,max_ctx=8192Type:
StrList
Default:all
A list of boost modules that will be advertised by /v1/models endpoint.
All loaded modules can still be used directly, this configuration only affects
which modules are advertised in the API.
Supports all value to enable all modules.
Example:
# Serve all modules
HARBOR_BOOST_MODULES=all
# Only serve klmbr and rcn modules
HARBOR_BOOST_MODULES=klmbr;rcnWhen using with Harbor, you can configure this via Harbor CLI:
# Enable the module
harbor boost modules add <module>
# Disable the module
harbor boost modules rm <module>
# List enabled modules
harbor boost modules lsNote that new Harbor releases might introduce new modules, so the default value of this setting could change in the future. Check out Harbor Profiles for a way to save and restore your configuration.
Type:
StrList
Default:modules;custom_modules
A list of folders to load boost modules from.
You can mount custom modules to the /boost/custom_modules or a custom location and use this configuration to load them.
Example:
# Load from default locations
HARBOR_BOOST_MODULE_FOLDERS=modules;custom_modules
# Disable all built-in modules and load only custom ones
HARBOR_BOOST_MODULE_FOLDERS=/some/custom/pathType:
bool
Default:true
When set to true, the boost output the intermediate steps of the module, not only the final result, providing more dynamic feedback to the user.
Intermediate output includes status messages, internal monologue, and other non-final completions. Note that it doesn't mean "all output" from the module, as the module source can still decide to not emit specific things at all, or inverse - emit them even if this setting is off.
Example of the intermediate output from the g1 module - underlying reasoning steps:
Type:
str
Default:md:codeblock
A module can call llm.emit_status during its processing, which will be streamed as a "status" or "progress" message to the user. This setting controls the format of this message, which will be dependent on what's supported by the frontend where boost response is displayed.
Options:
md:codeblock "
```boost
{status}
```
",
md:h1 "
# {status}
",
md:h2 "
## {status}
",
md:h3 "
### {status}
",
plain "
{status}
",
none ""The default is md:codeblock and looks like this in the WebUI:
Type:
bool
Default:false
Depending on the configuration of your setup, your LLM backend might or might not be connected to the UI directly. If not (or using boost as a standalone service), you can toggle this option on for the boost to serve them as is.
# Now "unboosted" models will also be available
# via the boost API
harbor config boost.base_models trueType:
ConfigDict
Default: ``
When specified, boost will only serve models matching the filter. The filter is a key/value expression that'll be matched against the model metadata. See examples below:
# Only boost models with the "llama" in the name
harbor config set boost.model_filter id.contains=llama
# Only boost models matching the regex
harbor config set boost.model_filter id.regex=.+q8_0$
# Boost by regex matching multiple IDs
harbor config set boost.model_filter "id.regex=.*(llama3.1:8b|llama3.2:3b|qwen2.5:7b)"
# Only boost a model with the exact ID
harbor config set boost.model_filter id=llama3.1:8bThis filter runs after the boosted models (per module) are added, so you can filter them out as well.
Type:
str
Default: ``
By default, boost will accept and serve any request, but you can configure one or more API keys to restrict access to the service.
Example:
# Configure the API key
HARBOR_BOOST_API_KEY=sk-boost
# Send the API key in the header
# Authorization: sk-boostType:
StrList
Default: ``
A colon-separated list of API keys to use for the boost API. Counterpart to HARBOR_BOOST_API_KEY.
Example:
# Configure the API keys
HARBOR_BOOST_API_KEYS=sk-user1;sk-user2;sk-user3Type:
str
Default: ``
Allows specifying additional, "named" API keys that will be accepted by the boost API.
Example:
# Configure the API keys
HARBOR_BOOST_API_KEY_MAIN=sk-main
# Temporary API key for testing
HARBOR_BOOST_API_KEY_TEST=sk-testType:
str
Default:http://localhost:34131
URL which boost artifacts should use to access the boost API
Type:
int
Default:15
The percentage of text to modify with the klmbr module
Type:
StrList
Default:all
The list of modifications klmbr will apply
Type:
str
Default:all
The strategy that selects messages to modify for the klmbr module
Type:
ConfigDict
Default: ``
The parameters for the strategy that selects messages to modify for the klmbr module
Type:
str
Default:match
The strategy that selects messages to modify for the rcn module
Type:
ConfigDict
Default:role=user,index=-1
Parameters for rcn message selection
Type:
str
Default:match
The strategy that selects messages to modify for the g1 module
Type:
ConfigDict
Default:role=user,index=-1
Parameters for g1 message selection
Type:
int
Default:15
The maximum number of reasoning steps to generate
Type:
str
Default:match
The strategy that selects messages to target for the mcts module
Type:
ConfigDict
Default:role=user,index=-1
Parameters for mcts message selection
Type:
int
Default:2
The maximum number of simulations to run (per iteration)
Type:
int
Default:2
The maximum number of iterations to run
Type:
int
Default:2
The amount of thoughts (node expansions) to generate per simulation
Type:
float
Default:1.414
The exploration constant for the MCTS algorithm
Type:
str
Default:match
The strategy that selects messages to target for the eli5 module
Type:
ConfigDict
Default:role=user,index=-1
Parameters for eli5 message selection
Type:
str
Default:match
The strategy that selects messages to target for the supersummer module
Type:
ConfigDict
Default:role=user,index=-1
Parameters for supersummer message selection
Type:
int
Default:5
The number of questions to generate for the summarisation
Type:
str
Default:few paragraphs
Desired length of the summary
Type:
int
Default:5
The amount of thoughts to generate for the r0 module