Skip to content

feat: production-grade bring-your-own-model (BYO) support - #733

Merged
n24q02m merged 8 commits into
mainfrom
feat/byo-model
Jun 12, 2026
Merged

feat: production-grade bring-your-own-model (BYO) support#733
n24q02m merged 8 commits into
mainfrom
feat/byo-model

Conversation

@n24q02m

@n24q02m n24q02m commented Jun 12, 2026

Copy link
Copy Markdown
Owner

Spec B Part A — make add_custom_model production-grade so any ONNX-able embedding model can be plugged in. Qwen3 stays the only built-in.

Bug fixes (verified file:line)

  • dim/MRL swallowed on custom + pooled paths (CustomTextEmbedding, PooledEmbedding, PooledNormalizedEmbedding ignored **kwargs) — embed(..., dim=256) now truncates (pool → truncate → normalize, matching Qwen3).
  • custom models broken under parallel>1 — registration lived in class vars (POSTPROCESSING_MAPPING/SUPPORTED_MODELS) that don't survive a spawned worker → KeyError. Folded pooling/normalization onto a new frozen CustomDenseModelDescription, keyed the registry case-insensitively, and propagated it to workers via a polymorphic _extra_worker_params() hook.
  • case-sensitivity in the registry — fixed by the same refactor (lowercased keys).

Features

  • CustomModelSpec — one-call BYO registration (CustomModelSpec(...).register()).
  • qwen3_embed.export.export_to_onnx() — optional HF-id→ONNX helper (lazy imports torch/optimum; NOT a packaged extra because the export deps don't co-resolve with the lean tokenizers/huggingface-hub pins — documented manual install).
  • NOTICE — retains fastembed (Qdrant) Apache-2.0 attribution (was missing).

Tests

  • 496 unit passed (was 491), ruff/format/ty clean.
  • Worker-registry propagation covered by test_custom_registry_survives_serialization. The end-to-end parallel=2 integration test is skipif(win32) (Windows multiprocessing+onnxruntime spawn deadlocks under pytest — runs on Linux).

Notes: custom INT8 models score per-row (batched INT8 isn't batch-invariant — Spec A Part 2 finding).

🤖 Generated with Claude Code

n24q02m and others added 8 commits June 11, 2026 21:20
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-insensitive

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dlock

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@n24q02m
n24q02m merged commit 5220b34 into main Jun 12, 2026
25 checks passed
@n24q02m
n24q02m deleted the feat/byo-model branch June 12, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant