fix: optimize FP16 to FP32 Casting

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

Author: n24q02m

.jules/bolt.md

@@ -49,3 +49,6 @@
 ## 2025-02-14 - Optimize regex substitution loops with search fast-path
 **Learning:** Using `re.search` as a fast-path condition before executing a `re.subn` loop significantly improves performance (e.g., ~50% faster for clean text) because `re.search` is highly optimized in C and avoids the overhead of substitution checks when no matches exist.
 **Action:** Always implement an initial `search` or string-matching fast-path before performing iterative regex substitutions or replacements, especially on hot paths like text sanitization.
+## 2024-05-24 - Defer Tensor Casting
+**Learning:** Casting massive full-vocab output tensors from FP16 to FP32 before slicing causes huge memory allocation overhead.
+**Action:** Extract the needed scalar logits first, then let np.subtract handle the cast on the small slice.

qwen3_embed/rerank/cross_encoder/qwen3_cross_encoder.py

@@ -289,9 +289,6 @@ def _onnx_embed_texts(self, texts: list[str], **kwargs: Any) -> OnnxOutputContex
             outputs = self.model.run(self.ONNX_OUTPUT_NAMES, onnx_input)
             model_output = outputs[0]
 
-            if getattr(model_output, "dtype", None) == np.float16:
-                model_output = model_output.astype(np.float32)  # type: ignore[unresolved-attribute]
-
             # batch=1 with no padding: the yes/no token is the last position. The mask
             # (all ones) is passed for the full-vocab variant; the YesNo variant ignores
             # it (output already collapsed to (1, 2)).