Skip to content

Change providers to accept extra parameter#5080

Draft
glGarg wants to merge 1 commit intomicrosoft:mainfrom
glGarg:spgarg/provider_change_thinking
Draft

Change providers to accept extra parameter#5080
glGarg wants to merge 1 commit intomicrosoft:mainfrom
glGarg:spgarg/provider_change_thinking

Conversation

@glGarg
Copy link
Copy Markdown
Member

@glGarg glGarg commented Apr 20, 2026

Adds support for passing custom fields via an extraBody property throughout the BYOK (Bring Your Own Key) model configuration and request pipeline. This enables providers to specify additional request parameters (such as special options for certain endpoints) in a flexible way. The changes ensure that these custom fields are available in model metadata, configuration, and are included in outgoing requests.

Copilot AI review requested due to automatic review settings April 20, 2026 19:39
@glGarg glGarg marked this pull request as draft April 20, 2026 19:41
@glGarg
Copy link
Copy Markdown
Member Author

glGarg commented Apr 20, 2026

This is how I set the thinking parameter in the config:

customModels:
  customoai:
    accounts/msft/deployments/wxbsoe65:
      name: 'wxbsoe65'
      url: 'https://api.fireworks.ai/inference/v1/chat/completions'
      toolCalling: true
      vision: false
      thinking: false
      maxInputTokens: 128000
      maxOutputTokens: 16384
      requiresAPIKey: true
      extraBody:
        thinking:
          type: disabled

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for propagating a provider-specified extraBody object through BYOK model metadata/config so OpenAI-compatible endpoints can inject additional request fields when forming outbound requests.

Changes:

  • Extend model metadata/types (IChatModelInformation, BYOKModelCapabilities) to include extraBody.
  • Plumb extraBody from Custom OAI model configuration into resolved model info.
  • Merge extraBody into the outgoing OpenAI request body in OpenAIEndpoint.
Show a summary per file
File Description
src/platform/endpoint/common/endpointProvider.ts Adds extraBody to chat model metadata shape.
src/extension/byok/vscode-node/customOAIProvider.ts Allows custom OAI model configs to specify extraBody and forwards it into model metadata.
src/extension/byok/node/openAIEndpoint.ts Merges extraBody into request payloads before sending.
src/extension/byok/common/byokProvider.ts Carries extraBody through BYOK capability resolution into IChatModelInformation.

Copilot's findings

  • Files reviewed: 4/4 changed files
  • Comments generated: 2

Comment on lines +294 to +297
// Merge extraBody fields (e.g. thinking: {type: "disabled"} for Fireworks)
if (this.modelMetadata.extraBody) {
Object.assign(body, this.modelMetadata.extraBody);
}
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Object.assign(body, this.modelMetadata.extraBody) merges unvalidated, user-configurable data into the request body. This can lead to prototype pollution via keys like __proto__, constructor, or prototype, and also produces odd results if extraBody is not a plain object at runtime. Consider validating extraBody is a non-null plain object and filtering out dangerous keys before merging (or doing a safe merge helper).

Copilot uses AI. Check for mistakes.
Comment on lines +294 to +297
// Merge extraBody fields (e.g. thinking: {type: "disabled"} for Fireworks)
if (this.modelMetadata.extraBody) {
Object.assign(body, this.modelMetadata.extraBody);
}
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New behavior merges modelMetadata.extraBody into outgoing requests, but there are existing unit tests for OpenAIEndpoint and none appear to assert this merge behavior. Please add coverage that verifies extraBody fields are applied (and, if you add validation/sanitization, that unsafe keys are ignored).

Copilot uses AI. Check for mistakes.
@battaglia01
Copy link
Copy Markdown

I was just looking for this feature and am glad it is being added, as we can then use reasoning in OpenRouter.

@glGarg Would we put this in chatLanguageModels.json or somewhere else? The config you have above doesn't look like json.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants