Add testing

2025-09-26 15:26:13 +00:00
parent a91188e83f
commit 17fcd3596b
7 changed files with 1615 additions and 0 deletions
--- a/tests/test_engine/README.md
+++ b/tests/test_engine/README.md
@@ -0,0 +1,195 @@
+# Engine Module Test Suite
+
+This directory contains comprehensive tests for the vector search engine module, covering all components from the abstract base engine to concrete implementations and the factory pattern.
+
+## Test Structure
+
+### Core Test Files
+
+- **`test_base_engine.py`** - Tests for the abstract `BaseEngine` class
+- **`test_qdrant_engine.py`** - Tests for the `QdrantEngine` implementation
+- **`test_factory.py`** - Tests for the engine factory and `EngineType` enum
+- **`test_integration.py`** - End-to-end integration tests
+- **`conftest.py`** - Shared fixtures and test configuration
+
+## Test Coverage
+
+### BaseEngine Tests (`test_base_engine.py`)
+- ✅ Abstract class enforcement - ensures BaseEngine cannot be instantiated
+- ✅ Abstract method verification - validates required method signatures
+- ✅ Generic typing constraints - tests TypeVar functionality
+- ✅ Semantic search workflow - tests the complete search flow
+- ✅ Condition transformation - tests with various condition types
+- ✅ Response transformation - tests data structure conversion
+- ✅ Parameter validation - tests default values and edge cases
+
+### QdrantEngine Tests (`test_qdrant_engine.py`)
+- ✅ Inheritance verification - ensures proper BaseEngine inheritance
+- ✅ Generic type parameters - validates `BaseEngine[list[ScoredPoint], Filter]`
+- ✅ Condition transformation - tests conversion to Qdrant Filter objects
+  - Match conditions → `MatchValue`
+  - MatchAny conditions → `MatchAny`
+  - MatchExclude conditions → `MatchExcept`
+- ✅ Response transformation - tests `ScoredPoint` to `SearchRow` conversion
+- ✅ Null payload filtering - ensures entries with null payloads are excluded
+- ✅ Client interaction - mocks and verifies Qdrant client calls
+- ✅ Error propagation - ensures client exceptions bubble up correctly
+- ✅ Initialization - tests Settings and AsyncQdrantClient setup
+
+### Factory Tests (`test_factory.py`)
+- ✅ Backend type enumeration - tests `Backend` enum values and behavior
+- ✅ Factory function typing - validates overload signatures for type safety
+- ✅ Engine instantiation - tests creation of concrete engine instances
+- ✅ Error handling - validates behavior with invalid inputs
+- ✅ Caching behavior - ensures `@cache` decorator works correctly (same instances)
+- ✅ COSMOS engine handling - tests NotImplementedError for unimplemented engines
+- ✅ String enum behavior - tests StrEnum functionality and JSON serialization
+
+### Integration Tests (`test_integration.py`)
+- ✅ Complete workflow - factory → conditions → search → response transformation
+- ✅ Parameter passing - verifies correct parameter flow through all layers
+- ✅ Complex conditions - tests multiple condition types together
+- ✅ Large result sets - tests handling of 100+ search results
+- ✅ Edge cases - empty conditions, null payloads, error scenarios
+- ✅ Named vectors - tests support for Qdrant NamedVector objects
+- ✅ Multiple engine instances - tests independence and concurrent usage
+
+## Test Fixtures (`conftest.py`)
+
+### Automatic Fixtures
+- `clear_engine_cache` - Auto-clears `@cache` decorator before/after each test
+
+### Mock Objects
+- `mock_qdrant_client` - AsyncMock for Qdrant client with default responses
+- `mock_settings` - Mock Settings object with test configuration
+- `mock_qdrant_engine_dependencies` - Complete mocked environment
+
+### Sample Data
+- `sample_embedding` - Standard test embedding vector
+- `sample_conditions` - Common condition objects for testing
+- `sample_scored_points` - Realistic ScoredPoint objects
+- `sample_search_rows` - Expected SearchRow outputs
+- `qdrant_filter_single/multiple` - Pre-built Qdrant Filter objects
+
+## Running Tests
+
+### Run All Engine Tests
+```bash
+uv run pytest tests/test_engine/ -v
+```
+
+### Run Specific Test File
+```bash
+uv run pytest tests/test_engine/test_base_engine.py -v
+uv run pytest tests/test_engine/test_qdrant_engine.py -v
+uv run pytest tests/test_engine/test_factory.py -v
+uv run pytest tests/test_engine/test_integration.py -v
+```
+
+### Run Specific Test Class or Method
+```bash
+uv run pytest tests/test_engine/test_factory.py::TestEngineFactory::test_get_engine_qdrant -v
+uv run pytest tests/test_engine/test_integration.py::TestEngineIntegration -v
+```
+
+### Coverage Report
+```bash
+uv run pytest tests/test_engine/ --cov=src/vector_search_mcp/engine --cov-report=html
+```
+
+## Test Patterns Used
+
+### Mocking Strategy
+- **External dependencies** - All external services (Qdrant client, Settings) are mocked
+- **Dependency injection** - Tests inject mocks through constructor parameters
+- **Return value control** - Mocks return predictable test data for assertions
+
+### Async Testing
+- Uses `@pytest.mark.asyncio` for async method testing
+- `AsyncMock` objects for async client methods
+- Proper await/async syntax throughout
+
+### Type Testing
+- Generic type parameter validation
+- Overload signature verification  
+- Runtime type checking where applicable
+
+### Error Testing
+- Exception propagation validation
+- Invalid input handling
+- Boundary condition testing
+
+## Key Test Insights
+
+### Generic Typing Validation
+The tests verify that the generic `BaseEngine[ResponseType, ConditionType]` pattern works correctly:
+- `QdrantEngine` is typed as `BaseEngine[list[ScoredPoint], Filter]`
+- Type checkers can verify correct usage at compile time
+- Runtime behavior matches type declarations
+
+### Factory Pattern Testing
+The overload tests ensure proper type inference:
+```python
+# Type checker knows this returns QdrantEngine
+engine = get_engine(Backend.QDRANT)
+
+# Type checker knows this returns BaseEngine (generic)
+backend: Backend = some_variable
+engine = get_engine(backend)
+```
+
+### Caching Behavior
+Tests verify that the `@cache` decorator works correctly:
+```python
+# Both calls return the same instance
+engine1 = get_engine(Backend.QDRANT)
+engine2 = get_engine(Backend.QDRANT)
+assert engine1 is engine2  # Same instance due to caching
+```
+
+### Integration Flow Validation
+Integration tests verify the complete data flow:
+1. `get_engine()` creates proper engine instance
+2. `semantic_search()` calls `transform_conditions()`
+3. Transformed conditions passed to `run_similarity_query()`
+4. Query response processed by `transform_response()`
+5. Final `SearchRow` objects returned to caller
+
+## Maintenance Notes
+
+### Adding New Engine Types
+When adding new engines:
+1. Add enum value to `Backend`
+2. Add overload signature to `get_engine()`
+3. Update factory tests for new count and behavior
+4. Create engine-specific test file following `test_qdrant_engine.py` pattern
+5. Remember that `@cache` decorator will cache instances per backend type
+
+### Mock Updates
+If engine interfaces change:
+1. Update fixture return types in `conftest.py`
+2. Verify mock method signatures match real implementations
+3. Update integration tests for new parameter flows
+4. Ensure cache clearing fixture handles any new caching behavior
+
+### Performance Testing
+Current tests focus on correctness. For performance testing:
+- Use `pytest-benchmark` for timing critical paths
+- Test with realistic data sizes (1000+ embeddings)
+- Mock network I/O but measure transformation logic
+- Consider cache warming effects when benchmarking
+
+## Recent Fixes Applied
+
+### After Formatter Changes
+The following issues were resolved after code formatting:
+
+1. **Enum Rename**: `EngineType` → `Backend` - All tests updated
+2. **Caching Addition**: `@cache` decorator added to `get_engine()` 
+   - Tests updated to expect same instances (not different ones)
+   - Auto-cache clearing fixture added to `conftest.py`
+3. **Mock Isolation**: Improved mock setup to prevent real network calls
+   - Proper patch contexts in all integration tests
+   - Cache clearing ensures clean test state
+
+All 62 tests now pass successfully! 🎉