searchbox/tests/test_engine/README.md

# Engine Module Test Suite

This directory contains comprehensive tests for the vector search engine module, covering all components from the abstract base engine to concrete implementations and the factory pattern.

## Test Structure

### Core Test Files

- **`test_base_engine.py`** - Tests for the abstract `BaseEngine` class
- **`test_qdrant_engine.py`** - Tests for the `QdrantEngine` implementation
- **`test_factory.py`** - Tests for the engine factory and `EngineType` enum
- **`test_integration.py`** - End-to-end integration tests
- **`conftest.py`** - Shared fixtures and test configuration

## Test Coverage

### BaseEngine Tests (`test_base_engine.py`)
- ✅ Abstract class enforcement - ensures BaseEngine cannot be instantiated
- ✅ Abstract method verification - validates required method signatures
- ✅ Generic typing constraints - tests TypeVar functionality
- ✅ Semantic search workflow - tests the complete search flow
- ✅ Condition transformation - tests with various condition types
- ✅ Response transformation - tests data structure conversion
- ✅ Parameter validation - tests default values and edge cases

### QdrantEngine Tests (`test_qdrant_engine.py`)
- ✅ Inheritance verification - ensures proper BaseEngine inheritance
- ✅ Generic type parameters - validates `BaseEngine[list[ScoredPoint], Filter]`
- ✅ Condition transformation - tests conversion to Qdrant Filter objects
  - Match conditions → `MatchValue`
  - MatchAny conditions → `MatchAny`
  - MatchExclude conditions → `MatchExcept`
- ✅ Response transformation - tests `ScoredPoint` to `SearchRow` conversion
- ✅ Null payload filtering - ensures entries with null payloads are excluded
- ✅ Client interaction - mocks and verifies Qdrant client calls
- ✅ Error propagation - ensures client exceptions bubble up correctly
- ✅ Initialization - tests Settings and AsyncQdrantClient setup

### Factory Tests (`test_factory.py`)
- ✅ Backend type enumeration - tests `Backend` enum values and behavior
- ✅ Factory function typing - validates overload signatures for type safety
- ✅ Engine instantiation - tests creation of concrete engine instances
- ✅ Error handling - validates behavior with invalid inputs
- ✅ Caching behavior - ensures `@cache` decorator works correctly (same instances)
- ✅ COSMOS engine handling - tests NotImplementedError for unimplemented engines
- ✅ String enum behavior - tests StrEnum functionality and JSON serialization

### Integration Tests (`test_integration.py`)
- ✅ Complete workflow - factory → conditions → search → response transformation
- ✅ Parameter passing - verifies correct parameter flow through all layers
- ✅ Complex conditions - tests multiple condition types together
- ✅ Large result sets - tests handling of 100+ search results
- ✅ Edge cases - empty conditions, null payloads, error scenarios
- ✅ Named vectors - tests support for Qdrant NamedVector objects
- ✅ Multiple engine instances - tests independence and concurrent usage

## Test Fixtures (`conftest.py`)

### Automatic Fixtures
- `clear_engine_cache` - Auto-clears `@cache` decorator before/after each test

### Mock Objects
- `mock_qdrant_client` - AsyncMock for Qdrant client with default responses
- `mock_settings` - Mock Settings object with test configuration
- `mock_qdrant_engine_dependencies` - Complete mocked environment

### Sample Data
- `sample_embedding` - Standard test embedding vector
- `sample_conditions` - Common condition objects for testing
- `sample_scored_points` - Realistic ScoredPoint objects
- `sample_search_rows` - Expected SearchRow outputs
- `qdrant_filter_single/multiple` - Pre-built Qdrant Filter objects

## Running Tests

### Run All Engine Tests
```bash
uv run pytest tests/test_engine/ -v
```

### Run Specific Test File
```bash
uv run pytest tests/test_engine/test_base_engine.py -v
uv run pytest tests/test_engine/test_qdrant_engine.py -v
uv run pytest tests/test_engine/test_factory.py -v
uv run pytest tests/test_engine/test_integration.py -v
```

### Run Specific Test Class or Method
```bash
uv run pytest tests/test_engine/test_factory.py::TestEngineFactory::test_get_engine_qdrant -v
uv run pytest tests/test_engine/test_integration.py::TestEngineIntegration -v
```

### Coverage Report
```bash
uv run pytest tests/test_engine/ --cov=src/vector_search_mcp/engine --cov-report=html
```

## Test Patterns Used

### Mocking Strategy
- **External dependencies** - All external services (Qdrant client, Settings) are mocked
- **Dependency injection** - Tests inject mocks through constructor parameters
- **Return value control** - Mocks return predictable test data for assertions

### Async Testing
- Uses `@pytest.mark.asyncio` for async method testing
- `AsyncMock` objects for async client methods
- Proper await/async syntax throughout

### Type Testing
- Generic type parameter validation
- Overload signature verification  
- Runtime type checking where applicable

### Error Testing
- Exception propagation validation
- Invalid input handling
- Boundary condition testing

## Key Test Insights

### Generic Typing Validation
The tests verify that the generic `BaseEngine[ResponseType, ConditionType]` pattern works correctly:
- `QdrantEngine` is typed as `BaseEngine[list[ScoredPoint], Filter]`
- Type checkers can verify correct usage at compile time
- Runtime behavior matches type declarations

### Factory Pattern Testing
The overload tests ensure proper type inference:
```python
# Type checker knows this returns QdrantEngine
engine = get_engine(Backend.QDRANT)

# Type checker knows this returns BaseEngine (generic)
backend: Backend = some_variable
engine = get_engine(backend)
```

### Caching Behavior
Tests verify that the `@cache` decorator works correctly:
```python
# Both calls return the same instance
engine1 = get_engine(Backend.QDRANT)
engine2 = get_engine(Backend.QDRANT)
assert engine1 is engine2  # Same instance due to caching
```

### Integration Flow Validation
Integration tests verify the complete data flow:
1. `get_engine()` creates proper engine instance
2. `semantic_search()` calls `transform_conditions()`
3. Transformed conditions passed to `run_similarity_query()`
4. Query response processed by `transform_response()`
5. Final `SearchRow` objects returned to caller

## Maintenance Notes

### Adding New Engine Types
When adding new engines:
1. Add enum value to `Backend`
2. Add overload signature to `get_engine()`
3. Update factory tests for new count and behavior
4. Create engine-specific test file following `test_qdrant_engine.py` pattern
5. Remember that `@cache` decorator will cache instances per backend type

### Mock Updates
If engine interfaces change:
1. Update fixture return types in `conftest.py`
2. Verify mock method signatures match real implementations
3. Update integration tests for new parameter flows
4. Ensure cache clearing fixture handles any new caching behavior

### Performance Testing
Current tests focus on correctness. For performance testing:
- Use `pytest-benchmark` for timing critical paths
- Test with realistic data sizes (1000+ embeddings)
- Mock network I/O but measure transformation logic
- Consider cache warming effects when benchmarking

## Recent Fixes Applied

### After Formatter Changes
The following issues were resolved after code formatting:

1. **Enum Rename**: `EngineType` → `Backend` - All tests updated
2. **Caching Addition**: `@cache` decorator added to `get_engine()` 
   - Tests updated to expect same instances (not different ones)
   - Auto-cache clearing fixture added to `conftest.py`
3. **Mock Isolation**: Improved mock setup to prevent real network calls
   - Proper patch contexts in all integration tests
   - Cache clearing ensures clean test state

All 62 tests now pass successfully! 🎉