Skip to content

test: Convert test_workspace_isolation.py to pytest style#2371

Merged
danielaskdd merged 4 commits intoHKUDS:workspace-isolationfrom
BukeLy:pytest-style-conversion
Nov 17, 2025
Merged

test: Convert test_workspace_isolation.py to pytest style#2371
danielaskdd merged 4 commits intoHKUDS:workspace-isolationfrom
BukeLy:pytest-style-conversion

Conversation

@BukeLy
Copy link
Contributor

@BukeLy BukeLy commented Nov 17, 2025

Summary

Convert workspace isolation test suite to pytest style and fix critical test issues.

Changes

1. Convert to pytest framework

  • Replace custom TestResults class with standard pytest assertions
  • Add @pytest.mark.asyncio decorators to async test functions
  • Add pytest fixture for test setup
  • Remove custom test runner
  • Reduce code from 1262 to 869 lines (31% reduction)

2. Fix Mock LLM output format

Problem: Mock LLM returned JSON format, causing extraction failures:

  • WARNING: "Complete delimiter can not be found in extraction result"
  • Extracted 0 entities and 0 relations
  • Empty graph files (310 bytes)

Solution: Use LightRAG's tuple-delimited format:

# Before
return """{"entities": [{"name": "Test Entity"}], "relationships": []}"""

# After
return """entity<|#|>Artificial Intelligence<|#|>concept<|#|>description
entity<|#|>Machine Learning<|#|>concept<|#|>description
relation<|#|>Machine Learning<|#|>Artificial Intelligence<|#|>keywords<|#|>description
<|COMPLETE|>"""

Result: Correctly extracts 2 entities and 1 relation

3. Enhance E2E test validation

Add content-specific assertions to verify data isolation:

  • Check each workspace contains only its own data
  • Check no cross-workspace data leakage
  • Detect data contamination between workspaces

4. Update documentation

Update file header docstring to accurately describe all 11 test scenarios.

Testing

uv run pytest tests/test_workspace_isolation.py -v

All 11 tests pass in 2.43s:

  • test_pipeline_status_isolation
  • test_lock_mechanism
  • test_backward_compatibility
  • test_multi_workspace_concurrency
  • test_namespace_lock_reentrance
  • test_different_namespace_lock_isolation
  • test_error_handling
  • test_update_flags_workspace_isolation
  • test_empty_workspace_standardization
  • test_json_kv_storage_workspace_isolation
  • test_lightrag_end_to_end_workspace_isolation

Impact

  • Improved test maintainability and CI/CD integration
  • Fixed Mock LLM to produce realistic test data
  • Enhanced test detection capability for data isolation
  • Better documentation
  • Single file changed: +407 -803 lines

Related

Based on workspace-isolation branch for PR #2366.

Why this change is needed:
The test file was using a custom TestResults class for tracking test
execution and results, which is not standard practice for pytest-based
test suites. This makes the tests harder to integrate with CI/CD pipelines
and reduces compatibility with pytest plugins and tooling.

How it solves it:
- Removed custom TestResults class and manual result tracking
- Added @pytest.mark.asyncio decorator to all async test functions
- Converted all results.add() calls to standard pytest assert statements
- Added pytest fixture (setup_shared_data) for common test setup
- Removed custom main() runner (pytest handles test discovery/execution)
- Kept all test logic, assertions, and debugging print statements intact

Impact:
- All 11 test functions maintain identical behavior and coverage
- Tests now follow pytest conventions and integrate with pytest ecosystem
- Test output is cleaner and more informative with pytest's reporting
- Easier to run selective tests using pytest's filtering options

Testing:
Verified by running: uv run pytest tests/test_workspace_isolation.py -v
Result: All 11 tests passed in 2.41s
Why this change is needed:
The mock LLM function was returning JSON format, which is incorrect
for LightRAG's entity extraction. This caused "Complete delimiter
can not be found" warnings and resulted in 0 entities/relations
being extracted during tests.

How it solves it:
- Updated mock_llm_func to return correct tuple-delimited format
- Format: entity<|#|>name<|#|>type<|#|>description
- Format: relation<|#|>source<|#|>target<|#|>keywords<|#|>description
- Added proper completion delimiter: <|COMPLETE|>
- Now correctly extracts 2 entities and 1 relation

Impact:
- E2E test now properly validates entity/relation extraction
- No more "Complete delimiter" warnings
- Tests can now detect extraction-related bugs
- Graph files contain actual data (2 nodes, 1 edge) instead of empty graphs

Testing:
All 11 tests pass in 2.42s with proper entity extraction:
- Chunk 1 of 1 extracted 2 Ent + 1 Rel (previously 0 Ent + 0 Rel)
- Graph files now 2564 bytes (previously 310 bytes)
…tion

Add specific content assertions to detect cross-contamination between workspaces.
Previously only checked that workspaces had different data, now verifies:

- Each workspace contains only its own text content
- Each workspace does NOT contain the other workspace's content
- Cross-contamination would be immediately detected

This ensures the test can find problems, not just pass.

Changes:
- Add assertions for "Artificial Intelligence" and "Machine Learning" in project_a
- Add assertions for "Deep Learning" and "Neural Networks" in project_b
- Add negative assertions to verify data leakage doesn't occur
- Add detailed output messages showing what was verified

Testing:
- pytest tests/test_workspace_isolation.py::test_lightrag_end_to_end_workspace_isolation
- Test passes with proper content isolation verified
Previous docstring mentioned only 4 scenarios but the file actually contains
11 comprehensive test cases. Updated to list all scenarios:

1. Pipeline Status Isolation
2. Lock Mechanism (Parallel/Serial)
3. Backward Compatibility
4. Multi-Workspace Concurrency
5. NamespaceLock Re-entrance Protection
6. Different Namespace Lock Isolation
7. Error Handling
8. Update Flags Workspace Isolation
9. Empty Workspace Standardization
10. JsonKVStorage Workspace Isolation
11. LightRAG End-to-End Workspace Isolation

This makes the file header accurately describe its contents.
@danielaskdd danielaskdd merged commit 3806892 into HKUDS:workspace-isolation Nov 17, 2025
@BukeLy BukeLy deleted the pytest-style-conversion branch November 18, 2025 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants