Skip to content

Commit 4010165

Browse files
authored
[NA] [BE][FE][SDK] Rename OPIK_BUILTIN to OPIK_FREE and improve free model UX (#4443)
* [NA] [BE][FE][SDK] Rename OPIK_BUILTIN to OPIK_FREE and improve free model UX Rename internal terminology from OPIK_BUILTIN to OPIK_FREE and enhance the user experience for the free model in the Playground. - Display free model directly in model selector (not in a dropdown/submenu) - Show model as "gpt-4o-mini (free)" with Opik icon for clear identification - Dynamic model label from backend config via `model_label` field - Provider label changed from "Opik Built-in" to "Opik" - Description updated to "Free model provided by Opik - no API key required" - API key name changed to "OPIK_FREE_MODEL_API_KEY" in configuration page - Free model included in search/filter results - LlmProvider enum: OPIK_BUILTIN → OPIK_FREE (value: "opik-free") - Package rename: opikbuiltin → freemodel - Class renames: OpikBuiltin* → FreeModel* - Config: opikbuiltinProviderEnabled → opikfreeProviderEnabled - Environment variable: TOGGLE_OPIKBUILTIN_PROVIDER_ENABLED → TOGGLE_OPIKFREE_PROVIDER_ENABLED - Added model_label to provider configuration for dynamic UI display - PROVIDER_TYPE.OPIK_BUILTIN → PROVIDER_TYPE.OPIK_FREE - PROVIDER_MODEL_TYPE.OPIK_BUILTIN_MODEL → PROVIDER_MODEL_TYPE.OPIK_FREE_MODEL - Feature toggle: OPIKBUILTIN_PROVIDER_ENABLED → OPIKFREE_PROVIDER_ENABLED - Refactored PromptModelSelect for improved free model presentation - TypeScript SDK provider types updated - OpenAPI specs updated * Address PR feedback: add validation and improve robustness - FreeModelConfig: Add JavaDoc explaining required fields - FreeModelConfig: isEnabled() now validates that actualModel, spanProvider, and baseUrl are set, ensuring fail-fast on misconfiguration - PromptModelSelect: Handle empty string for model_label with .trim() check * Fix camelCase naming for opikFreeProviderEnabled Rename opikfreeProviderEnabled to opikFreeProviderEnabled to follow consistent camelCase naming convention in configuration properties. * Remove opikFreeProviderEnabled feature toggle The free model visibility is already controlled by freeModel.enabled backend config, making the feature toggle redundant. This simplifies the codebase by removing unnecessary toggle checks. Changes: - Remove toggle from backend config and ServiceTogglesConfig - Remove toggle from frontend feature-toggles and DEFAULT_STATE - Update ProviderSelect and ProviderGrid to explicitly exclude OPIK_FREE - Update PROVIDER_FEATURE_TOGGLE_MAP type to use Exclude<> - Regenerate OpenAPI specs and SDKs * [NA] [FE] Add opik_free_model metadata flag for BI tracking Add a boolean flag to span metadata when users use the free model, enabling the BI team to identify and track free model usage in the analytics database. * Revision 3: Refactor PromptModelSelect - extract useModelOptions hook - Extract data transformation logic into useModelOptions hook - Simplify renderSelectTrigger with getSelectedModelInfo callback - Fix vertical alignment with py-1 padding on options container - Reduce component from 484 to ~350 lines * Revision 4: Fix useModelOptions type errors - Change ProviderKey to ProviderObject (correct exported type) - Use ModelOption[] instead of inline type for consistency * Revision 5: Fix free model row vertical alignment in PromptModelSelect
1 parent e6b0563 commit 4010165

File tree

42 files changed

+507
-381
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+507
-381
lines changed

apps/opik-backend/config.yml

Lines changed: 15 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -600,9 +600,6 @@ serviceToggles:
600600
# Default: true
601601
# Description: Whether or not Custom LLM provider is enabled
602602
customllmProviderEnabled: ${TOGGLE_CUSTOMLLM_PROVIDER_ENABLED:-"true"}
603-
# Default: true
604-
# Description: Whether or not Opik Built-in provider is enabled
605-
opikbuiltinProviderEnabled: ${TOGGLE_OPIKBUILTIN_PROVIDER_ENABLED:-"true"}
606603
# Default: false
607604
# Description: Whether or not Collaborators tab feature is enabled
608605
collaboratorsTabEnabled: ${TOGGLE_COLLABORATORS_TAB_ENABLED:-"false"}
@@ -753,22 +750,23 @@ webhook:
753750
# Description: Maximum time to wait for acquiring metrics alert job lock
754751
metricsAlertJobLockWaitTimeout: ${WEBHOOK_METRICS_ALERT_JOB_LOCK_WAIT_TIMEOUT:-1s}
755752

756-
# Built-in LLM Provider configuration
757-
# Note: The model name "opik-builtin-model" is hardcoded and cannot be changed.
753+
# Free Model configuration (admin-configured server-side LLM provider)
754+
# Note: The model name "opik-free-model" is hardcoded and cannot be changed.
758755
# This ensures backward compatibility with automation rules that store the model name.
759-
builtinLlmProvider:
756+
freeModel:
757+
# Description: Whether or not to enable the free model provider
760758
# Default: false
761-
# Description: Whether or not to enable the built-in LLM provider
762-
enabled: ${BUILTIN_LLM_PROVIDER_ENABLED:-false}
763-
# Default: gpt-4o-mini
759+
enabled: ${OPIK_FREE_MODEL_ENABLED:-false}
764760
# Description: Actual model name sent to the AI server (the underlying LLM)
765-
actualModel: ${BUILTIN_LLM_PROVIDER_ACTUAL_MODEL:-gpt-4o-mini}
766-
# Default: openai
767-
# Description: Provider name stored in spans for cost tracking
768-
spanProvider: ${BUILTIN_LLM_PROVIDER_SPAN_PROVIDER:-openai}
769761
# Default: empty
770-
# Description: API key for the OpenAI endpoint
771-
apiKey: ${BUILTIN_LLM_PROVIDER_API_KEY:-}
762+
actualModel: ${OPIK_FREE_MODEL_ACTUAL_MODEL:-}
763+
# Description: Provider name stored in spans for cost tracking (e.g., "openai")
764+
# Default: empty
765+
spanProvider: ${OPIK_FREE_MODEL_SPAN_PROVIDER:-}
766+
# Description: Base URL for OpenAI-compatible endpoint (e.g., "https://api.openai.com/v1")
772767
# Default: empty
773-
# Description: Base URL for OpenAI-compatible endpoint (empty for built-in OpenAI)
774-
baseUrl: ${BUILTIN_LLM_PROVIDER_BASE_URL:-}
768+
baseUrl: ${OPIK_FREE_MODEL_BASE_URL:-}
769+
# Description: API key for the endpoint (optional for auth-less endpoints)
770+
# Default: empty
771+
apiKey: ${OPIK_FREE_MODEL_API_KEY:-}
772+

apps/opik-backend/src/main/java/com/comet/opik/OpikApplication.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,10 @@
2121
import com.comet.opik.infrastructure.llm.LlmModule;
2222
import com.comet.opik.infrastructure.llm.antropic.AnthropicModule;
2323
import com.comet.opik.infrastructure.llm.customllm.CustomLlmModule;
24+
import com.comet.opik.infrastructure.llm.freemodel.FreeModelModule;
2425
import com.comet.opik.infrastructure.llm.gemini.GeminiModule;
2526
import com.comet.opik.infrastructure.llm.openai.OpenAIModule;
2627
import com.comet.opik.infrastructure.llm.openrouter.OpenRouterModule;
27-
import com.comet.opik.infrastructure.llm.opikbuiltin.OpikBuiltinModule;
2828
import com.comet.opik.infrastructure.llm.vertexai.VertexAIModule;
2929
import com.comet.opik.infrastructure.ratelimit.RateLimitModule;
3030
import com.comet.opik.infrastructure.redis.RedisModule;
@@ -98,7 +98,7 @@ public void initialize(Bootstrap<OpikConfiguration> bootstrap) {
9898
new ConfigurationModule(), new CacheModule(), new JobModule(), new AnthropicModule(),
9999
new GeminiModule(), new OpenAIModule(), new OpenRouterModule(), new LlmModule(),
100100
new AwsModule(), new UsageLimitModule(), new VertexAIModule(), new CustomLlmModule(),
101-
new OpikBuiltinModule())
101+
new FreeModelModule())
102102
.installers(JobGuiceyInstaller.class)
103103
.listen(new OpikGuiceyLifecycleEventListener(), new EventListenerRegistrar())
104104
.enableAutoConfig()

apps/opik-backend/src/main/java/com/comet/opik/api/LlmProvider.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ public enum LlmProvider {
1616
OPEN_ROUTER("openrouter"),
1717
VERTEX_AI("vertex-ai"),
1818
CUSTOM_LLM("custom-llm"),
19-
OPIK_BUILTIN("opik-builtin"),
19+
OPIK_FREE("opik-free"),
2020
;
2121

2222
@JsonValue

apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/priv/ChatCompletionsResource.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ public Response create(
8686
}
8787

8888
// Include actual model and provider in response headers for frontend span tracking.
89-
// For OPIK_BUILTIN provider, the user-facing model name (e.g., "opik-builtin-model") differs
89+
// For OPIK_FREE provider, the user-facing model name (e.g., "opik-free-model") differs
9090
// from the actual model used (e.g., "gpt-4o-mini"), and these headers allow correct cost calculation.
9191
var response = Response.ok()
9292
.type(type)

apps/opik-backend/src/main/java/com/comet/opik/domain/LlmProviderApiKeyService.java

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
import java.util.UUID;
2626

2727
import static com.comet.opik.api.LlmProvider.CUSTOM_LLM;
28-
import static com.comet.opik.infrastructure.BuiltinLlmProviderConfig.BUILTIN_PROVIDER_ID;
28+
import static com.comet.opik.infrastructure.FreeModelConfig.FREE_MODEL_PROVIDER_ID;
2929
import static com.comet.opik.infrastructure.db.TransactionTemplateAsync.READ_ONLY;
3030
import static com.comet.opik.infrastructure.db.TransactionTemplateAsync.WRITE;
3131

@@ -73,13 +73,17 @@ public ProviderApiKey.ProviderApiKeyPage find(@NonNull String workspaceId) {
7373

7474
var content = new ArrayList<>(providerApiKeys);
7575

76-
// Inject virtual built-in provider if enabled
77-
var builtinConfig = configuration.getBuiltinLlmProvider();
78-
if (builtinConfig.isEnabled()) {
76+
// Inject virtual free model provider if enabled
77+
var freeModelConfig = configuration.getFreeModel();
78+
if (freeModelConfig.isEnabled()) {
79+
// Model label shows actual model name (e.g., "gpt-4o-mini")
80+
// Frontend will append "(free)" suffix
7981
var virtualProvider = ProviderApiKey.builder()
80-
.id(BUILTIN_PROVIDER_ID)
81-
.provider(LlmProvider.OPIK_BUILTIN)
82-
.configuration(Map.of("models", builtinConfig.getModel()))
82+
.id(FREE_MODEL_PROVIDER_ID)
83+
.provider(LlmProvider.OPIK_FREE)
84+
.configuration(Map.of(
85+
"models", freeModelConfig.getModel(),
86+
"model_label", freeModelConfig.getActualModel()))
8387
.readOnly(true)
8488
.build();
8589
// Add at the end so user-configured providers are auto-selected by default in the Playground

apps/opik-backend/src/main/java/com/comet/opik/domain/llm/structuredoutput/StructuredOutputStrategy.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ static StructuredOutputStrategy getStrategy(LlmProvider provider, String modelNa
4343
.map(StructuredOutputSupported::isStructuredOutputSupported).orElse(false);
4444
case VERTEX_AI -> VertexAIModelName.byQualifiedName(modelName)
4545
.map(StructuredOutputSupported::isStructuredOutputSupported).orElse(false);
46-
case ANTHROPIC, CUSTOM_LLM, OPIK_BUILTIN -> false; // TODO: Should we pick a model that supports structured output?
46+
case ANTHROPIC, CUSTOM_LLM, OPIK_FREE -> false; // TODO: Should we pick a model that supports structured output?
4747
};
4848

4949
return isStructuredOutputSupported ? new ToolCallingStrategy() : new InstructionStrategy();

apps/opik-backend/src/main/java/com/comet/opik/infrastructure/BuiltinLlmProviderConfig.java

Lines changed: 0 additions & 49 deletions
This file was deleted.
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
package com.comet.opik.infrastructure;
2+
3+
import com.fasterxml.jackson.annotation.JsonProperty;
4+
import lombok.Getter;
5+
import lombok.Setter;
6+
import lombok.ToString;
7+
8+
import java.util.UUID;
9+
10+
@Getter
11+
@Setter
12+
@ToString
13+
public class FreeModelConfig {
14+
15+
/**
16+
* Fixed UUID for the free model provider. This is a virtual provider that doesn't exist in the database.
17+
*/
18+
public static final UUID FREE_MODEL_PROVIDER_ID = UUID.fromString("00000000-0000-0000-0000-000000000001");
19+
20+
/**
21+
* Fixed model name for the free model provider. This must remain constant because automation rules
22+
* store this model name in their configuration. Changing it would break existing rules.
23+
*/
24+
public static final String FREE_MODEL = "opik-free-model";
25+
26+
/**
27+
* Whether the free model provider is enabled in configuration.
28+
* Note: Use {@link #isEnabled()} to check if the provider is actually usable,
29+
* as it also validates that required fields are configured.
30+
*/
31+
@JsonProperty
32+
private boolean enabled = false;
33+
34+
/**
35+
* The actual model name sent to the AI server (e.g., "gpt-4o-mini").
36+
* Required when enabled=true. If empty, the provider will not be registered.
37+
*/
38+
@JsonProperty
39+
private String actualModel = "";
40+
41+
/**
42+
* Provider name stored in spans for cost tracking (e.g., "openai").
43+
* Required when enabled=true. If empty, the provider will not be registered.
44+
*/
45+
@JsonProperty
46+
private String spanProvider = "";
47+
48+
/**
49+
* Base URL for the OpenAI-compatible endpoint (e.g., "https://api.openai.com/v1").
50+
* Required when enabled=true. If empty, the provider will not be registered.
51+
*/
52+
@JsonProperty
53+
private String baseUrl = "";
54+
55+
/**
56+
* API key for the endpoint. Optional for auth-less endpoints.
57+
*/
58+
@JsonProperty
59+
@ToString.Exclude
60+
private String apiKey = "";
61+
62+
/**
63+
* Returns true if the free model provider is enabled AND all required fields are configured.
64+
* This ensures fail-fast behavior - the provider won't register with missing configuration.
65+
*/
66+
public boolean isEnabled() {
67+
return enabled
68+
&& actualModel != null && !actualModel.isBlank()
69+
&& spanProvider != null && !spanProvider.isBlank()
70+
&& baseUrl != null && !baseUrl.isBlank();
71+
}
72+
73+
/**
74+
* Returns the fixed model name. This is not configurable to ensure backward compatibility
75+
* with stored automation rules.
76+
*/
77+
public String getModel() {
78+
return FREE_MODEL;
79+
}
80+
}

apps/opik-backend/src/main/java/com/comet/opik/infrastructure/OpikConfiguration.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,5 +102,5 @@ public class OpikConfiguration extends JobConfiguration {
102102
private AttachmentsConfig attachmentsConfig = new AttachmentsConfig();
103103

104104
@Valid @NotNull @JsonProperty
105-
private BuiltinLlmProviderConfig builtinLlmProvider = new BuiltinLlmProviderConfig();
105+
private FreeModelConfig freeModel = new FreeModelConfig();
106106
}

apps/opik-backend/src/main/java/com/comet/opik/infrastructure/ServiceTogglesConfig.java

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,5 @@ public class ServiceTogglesConfig {
4747
@JsonProperty
4848
@NotNull boolean customllmProviderEnabled;
4949
@JsonProperty
50-
@NotNull boolean opikbuiltinProviderEnabled;
51-
@JsonProperty
5250
@NotNull boolean collaboratorsTabEnabled;
5351
}

0 commit comments

Comments
 (0)