fix: resolve vector dimension mismatch error when switching embedding models (#5616)

[roomote-bot]

Description

Fixes #5616

This PR resolves the vector dimension error that occurs when switching between embedding models with different vector dimensions (e.g., from 2048-dimension models to Google Gemini's 768-dimension model).

Problem

When users switched from a high-dimensional embedding model (2048 dimensions) to a lower-dimensional model like Google Gemini (768 dimensions), the Qdrant vector store would fail to recreate the collection, resulting in:

• "Vector dimension error: expected dim: 2048, got 768"
• Red error status in the indexing interface
• Inability to complete re-indexing

Solution

Enhanced the QdrantVectorStore with robust dimension mismatch handling:

Key Changes Made

1. Enhanced Collection Recreation Logic

   • Added atomic collection deletion with verification
   • Implemented step-by-step error handling for better debugging
   • Added proper type checking for vector configuration extraction

2. Improved Error Reporting

   • Detailed error context based on failure stage (deletion, verification, or creation)
   • User-friendly error messages with actionable information
   • Preserved original error context using the cause property

3. Robust Verification Process

   • Verification that collection is actually deleted before attempting recreation
   • Brief delay to ensure deletion is processed by Qdrant
   • Comprehensive logging for better debugging

Files Changed

• src/services/code-index/vector-store/qdrant-client.ts - Enhanced collection recreation logic
• src/services/code-index/vector-store/__tests__/qdrant-client.spec.ts - Added comprehensive test coverage

Testing

• ✅ All existing tests pass (59/59 QdrantVectorStore tests)
• ✅ All code-index tests pass (329/329 total tests)
• ✅ Added 5 new comprehensive test cases for dimension mismatch scenarios:
  • Successful collection recreation with new dimensions
  • Deletion failure handling
  • Creation failure handling
  • Verification failure handling
  • Specific 2048→768 dimension scenario from the issue
• ✅ No linting errors
• ✅ TypeScript compilation successful

Verification of Acceptance Criteria

• ✅ Issue reproduction: The exact scenario from issue Indexing fails after switching vector model dimensions due to dimension mismatch error #5616 (2048→768 dimensions) is now handled gracefully
• ✅ Error handling: Users receive clear, actionable error messages instead of cryptic database errors
• ✅ System stability: Atomic operations prevent the system from getting stuck in inconsistent states
• ✅ Backward compatibility: All existing functionality continues to work as expected

Impact

• Better user experience: Clear error messages instead of cryptic database errors
• Improved reliability: Atomic operations prevent inconsistent states
• Enhanced maintainability: Well-tested code with comprehensive error scenarios
• Robust error handling: Graceful handling of edge cases like partial deletion failures

Checklist

• Code follows project style guidelines
• Self-review completed
• Comments added for complex logic
• No breaking changes
• All tests passing
• TypeScript compilation successful
• Comprehensive test coverage added

---

Important

Fixes vector dimension mismatch error in QdrantVectorStore by enhancing collection recreation logic and error handling.

• Behavior:
  • Fixes vector dimension mismatch error when switching embedding models in QdrantVectorStore.
  • Enhances collection recreation logic with atomic deletion and verification in qdrant-client.ts.
  • Improves error reporting with detailed context and user-friendly messages.
• Testing:
  • Adds 5 new test cases in qdrant-client.spec.ts for dimension mismatch scenarios.
  • Ensures all existing tests pass, maintaining backward compatibility.
• Functions:
  • Adds _recreateCollectionWithNewDimension() and _createPayloadIndexes() to handle collection recreation and index creation.
  • Updates initialize() to handle dimension mismatches and improve error handling.

^{This description was created by for 5da7606. You can customize this summary. It will automatically update as commits are pushed.}

[delve-auditor]

✅ No security or compliance issues detected. Reviewed everything up to 5da7606.

Security Overview

• 🔎 Scanned files: 2 changed file(s)

Detected Code Changes

Change Type	Relevant files
Bug Fix	► qdrant-client.ts     Enhanced vector dimension mismatch handling     Added atomic collection recreation     Improved error handling and reporting ► qdrant-client.spec.ts     Added comprehensive test coverage for dimension mismatch scenarios

Reply to this PR with @delve-auditor followed by a description of what change you want and we'll auto-submit a change to this PR to implement it.