fix: always include max_completion_tokens for OpenAI compatible providers #6267
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes an issue where the response context limit was randomly ignored when using OpenAI Compatible API providers like koboldcpp.
Problem
When using OpenAI Compatible API providers, the
max_completion_tokens
parameter was only included whenincludeMaxTokens
was explicitly set totrue
. When undefined, the parameter was omitted, causing providers like koboldcpp to fall back to their default limit (512 tokens), which is insufficient for generating complete diffs.Solution
max_completion_tokens
whenincludeMaxTokens
is undefined (treating undefined as true for backward compatibility)Testing
includeMaxTokens: false
(explicitly disabled) and undefined (default enabled)Fixes #6265
Important
Ensure
max_completion_tokens
is always included for OpenAI compatible providers unless explicitly disabled, with updated tests inopenai.spec.ts
.max_completion_tokens
for OpenAI compatible providers unlessincludeMaxTokens
is explicitly false inopenai.ts
.includeMaxTokens
as true when undefined for backward compatibility.max_completion_tokens
values.openai.spec.ts
to reflect new default behavior.includeMaxTokens
is false or undefined.addMaxTokensIfNeeded()
inopenai.ts
.This description was created by
for e83f5cc. You can customize this summary. It will automatically update as commits are pushed.