searchbox/tests/test_engine

Engine Module Test Suite

This directory contains comprehensive tests for the vector search engine module, covering all components from the abstract base engine to concrete implementations and the factory pattern.

Test Structure

Core Test Files

• test_base_engine.py - Tests for the abstract BaseEngine class
• test_qdrant_engine.py - Tests for the QdrantEngine implementation
• test_factory.py - Tests for the engine factory and EngineType enum
• test_integration.py - End-to-end integration tests
• conftest.py - Shared fixtures and test configuration

Test Coverage

BaseEngine Tests (test_base_engine.py)

• ✅ Abstract class enforcement - ensures BaseEngine cannot be instantiated
• ✅ Abstract method verification - validates required method signatures
• ✅ Generic typing constraints - tests TypeVar functionality
• ✅ Semantic search workflow - tests the complete search flow
• ✅ Condition transformation - tests with various condition types
• ✅ Response transformation - tests data structure conversion
• ✅ Parameter validation - tests default values and edge cases

QdrantEngine Tests (test_qdrant_engine.py)

• ✅ Inheritance verification - ensures proper BaseEngine inheritance
• ✅ Generic type parameters - validates BaseEngine[list[ScoredPoint], Filter]
• ✅ Condition transformation - tests conversion to Qdrant Filter objects
  • Match conditions → MatchValue
  • MatchAny conditions → MatchAny
  • MatchExclude conditions → MatchExcept
• ✅ Response transformation - tests ScoredPoint to SearchRow conversion
• ✅ Null payload filtering - ensures entries with null payloads are excluded
• ✅ Client interaction - mocks and verifies Qdrant client calls
• ✅ Error propagation - ensures client exceptions bubble up correctly
• ✅ Initialization - tests Settings and AsyncQdrantClient setup

Factory Tests (test_factory.py)

• ✅ Backend type enumeration - tests Backend enum values and behavior
• ✅ Factory function typing - validates overload signatures for type safety
• ✅ Engine instantiation - tests creation of concrete engine instances
• ✅ Error handling - validates behavior with invalid inputs
• ✅ Caching behavior - ensures @cache decorator works correctly (same instances)
• ✅ COSMOS engine handling - tests NotImplementedError for unimplemented engines
• ✅ String enum behavior - tests StrEnum functionality and JSON serialization

Integration Tests (test_integration.py)

• ✅ Complete workflow - factory → conditions → search → response transformation
• ✅ Parameter passing - verifies correct parameter flow through all layers
• ✅ Complex conditions - tests multiple condition types together
• ✅ Large result sets - tests handling of 100+ search results
• ✅ Edge cases - empty conditions, null payloads, error scenarios
• ✅ Named vectors - tests support for Qdrant NamedVector objects
• ✅ Multiple engine instances - tests independence and concurrent usage

Test Fixtures (conftest.py)

Automatic Fixtures

• clear_engine_cache - Auto-clears @cache decorator before/after each test

Mock Objects

• mock_qdrant_client - AsyncMock for Qdrant client with default responses
• mock_settings - Mock Settings object with test configuration
• mock_qdrant_engine_dependencies - Complete mocked environment

Sample Data

• sample_embedding - Standard test embedding vector
• sample_conditions - Common condition objects for testing
• sample_scored_points - Realistic ScoredPoint objects
• sample_search_rows - Expected SearchRow outputs
• qdrant_filter_single/multiple - Pre-built Qdrant Filter objects

Running Tests

Run All Engine Tests

uv run pytest tests/test_engine/ -v

Run Specific Test File

uv run pytest tests/test_engine/test_base_engine.py -v
uv run pytest tests/test_engine/test_qdrant_engine.py -v
uv run pytest tests/test_engine/test_factory.py -v
uv run pytest tests/test_engine/test_integration.py -v

Run Specific Test Class or Method

uv run pytest tests/test_engine/test_factory.py::TestEngineFactory::test_get_engine_qdrant -v
uv run pytest tests/test_engine/test_integration.py::TestEngineIntegration -v

Coverage Report

uv run pytest tests/test_engine/ --cov=src/vector_search_mcp/engine --cov-report=html

Test Patterns Used

Mocking Strategy

• External dependencies - All external services (Qdrant client, Settings) are mocked
• Dependency injection - Tests inject mocks through constructor parameters
• Return value control - Mocks return predictable test data for assertions

Async Testing

• Uses @pytest.mark.asyncio for async method testing
• AsyncMock objects for async client methods
• Proper await/async syntax throughout

Type Testing

• Generic type parameter validation
• Overload signature verification
• Runtime type checking where applicable

Error Testing

• Exception propagation validation
• Invalid input handling
• Boundary condition testing

Key Test Insights

Generic Typing Validation

The tests verify that the generic BaseEngine[ResponseType, ConditionType] pattern works correctly:

• QdrantEngine is typed as BaseEngine[list[ScoredPoint], Filter]
• Type checkers can verify correct usage at compile time
• Runtime behavior matches type declarations

Factory Pattern Testing

The overload tests ensure proper type inference:

# Type checker knows this returns QdrantEngine
engine = get_engine(Backend.QDRANT)

# Type checker knows this returns BaseEngine (generic)
backend: Backend = some_variable
engine = get_engine(backend)

Caching Behavior

Tests verify that the @cache decorator works correctly:

# Both calls return the same instance
engine1 = get_engine(Backend.QDRANT)
engine2 = get_engine(Backend.QDRANT)
assert engine1 is engine2  # Same instance due to caching

Integration Flow Validation

Integration tests verify the complete data flow:

1. get_engine() creates proper engine instance
2. semantic_search() calls transform_conditions()
3. Transformed conditions passed to run_similarity_query()
4. Query response processed by transform_response()
5. Final SearchRow objects returned to caller

Maintenance Notes

Adding New Engine Types

When adding new engines:

1. Add enum value to Backend
2. Add overload signature to get_engine()
3. Update factory tests for new count and behavior
4. Create engine-specific test file following test_qdrant_engine.py pattern
5. Remember that @cache decorator will cache instances per backend type

Mock Updates

If engine interfaces change:

1. Update fixture return types in conftest.py
2. Verify mock method signatures match real implementations
3. Update integration tests for new parameter flows
4. Ensure cache clearing fixture handles any new caching behavior

Performance Testing

Current tests focus on correctness. For performance testing:

• Use pytest-benchmark for timing critical paths
• Test with realistic data sizes (1000+ embeddings)
• Mock network I/O but measure transformation logic
• Consider cache warming effects when benchmarking

Recent Fixes Applied

After Formatter Changes

The following issues were resolved after code formatting:

1. Enum Rename: EngineType → Backend - All tests updated
2. Caching Addition: @cache decorator added to get_engine()
   • Tests updated to expect same instances (not different ones)
   • Auto-cache clearing fixture added to conftest.py
3. Mock Isolation: Improved mock setup to prevent real network calls
   • Proper patch contexts in all integration tests
   • Cache clearing ensures clean test state

All 62 tests now pass successfully! 🎉