Add deployment guides

2026-03-06 16:26:25 +00:00
parent 205974c351
commit 610b6c3367
5 changed files with 2051 additions and 404 deletions
--- a/docs/DOCKER_DEPLOYMENT.md
+++ b/docs/DOCKER_DEPLOYMENT.md
@@ -0,0 +1,471 @@
+# Docker Deployment Guide
+
+> Deploy the LLM Gateway using pre-built Docker images or build your own.
+
+## Table of Contents
+
+- [Quick Start](#quick-start)
+- [Using Pre-Built Images](#using-pre-built-images)
+- [Configuration](#configuration)
+- [Docker Compose](#docker-compose)
+- [Building from Source](#building-from-source)
+- [Production Considerations](#production-considerations)
+- [Troubleshooting](#troubleshooting)
+
+## Quick Start
+
+Pull and run the latest image:
+
+```bash
+docker run -d \
+  --name llm-gateway \
+  -p 8080:8080 \
+  -e OPENAI_API_KEY="sk-your-key" \
+  -e ANTHROPIC_API_KEY="sk-ant-your-key" \
+  -e GOOGLE_API_KEY="your-key" \
+  ghcr.io/yourusername/llm-gateway:latest
+
+# Verify it's running
+curl http://localhost:8080/health
+```
+
+## Using Pre-Built Images
+
+Images are automatically built and published via GitHub Actions on every release.
+
+### Available Tags
+
+- `latest` - Latest stable release
+- `v1.2.3` - Specific version tags
+- `main` - Latest commit on main branch (unstable)
+- `sha-abc1234` - Specific commit SHA
+
+### Pull from Registry
+
+```bash
+# Pull latest stable
+docker pull ghcr.io/yourusername/llm-gateway:latest
+
+# Pull specific version
+docker pull ghcr.io/yourusername/llm-gateway:v1.2.3
+
+# List local images
+docker images | grep llm-gateway
+```
+
+### Basic Usage
+
+```bash
+docker run -d \
+  --name llm-gateway \
+  -p 8080:8080 \
+  --env-file .env \
+  ghcr.io/yourusername/llm-gateway:latest
+```
+
+## Configuration
+
+### Environment Variables
+
+Create a `.env` file with your API keys:
+
+```bash
+# Required: At least one provider
+OPENAI_API_KEY=sk-your-openai-key
+ANTHROPIC_API_KEY=sk-ant-your-anthropic-key
+GOOGLE_API_KEY=your-google-key
+
+# Optional: Server settings
+SERVER_ADDRESS=:8080
+LOGGING_LEVEL=info
+LOGGING_FORMAT=json
+
+# Optional: Features
+ADMIN_ENABLED=true
+RATE_LIMIT_ENABLED=true
+RATE_LIMIT_REQUESTS_PER_SECOND=10
+RATE_LIMIT_BURST=20
+
+# Optional: Auth
+AUTH_ENABLED=false
+AUTH_ISSUER=https://accounts.google.com
+AUTH_AUDIENCE=your-client-id.apps.googleusercontent.com
+
+# Optional: Observability
+OBSERVABILITY_ENABLED=false
+OBSERVABILITY_METRICS_ENABLED=false
+OBSERVABILITY_TRACING_ENABLED=false
+```
+
+Run with environment file:
+
+```bash
+docker run -d \
+  --name llm-gateway \
+  -p 8080:8080 \
+  --env-file .env \
+  ghcr.io/yourusername/llm-gateway:latest
+```
+
+### Using Config File
+
+For more complex configurations, use a YAML config file:
+
+```bash
+# Create config from example
+cp config.example.yaml config.yaml
+# Edit config.yaml with your settings
+
+# Mount config file into container
+docker run -d \
+  --name llm-gateway \
+  -p 8080:8080 \
+  -v $(pwd)/config.yaml:/app/config.yaml:ro \
+  ghcr.io/yourusername/llm-gateway:latest \
+  --config /app/config.yaml
+```
+
+### Persistent Storage
+
+For persistent conversation storage with SQLite:
+
+```bash
+docker run -d \
+  --name llm-gateway \
+  -p 8080:8080 \
+  -v llm-gateway-data:/app/data \
+  -e OPENAI_API_KEY="your-key" \
+  -e CONVERSATIONS_STORE=sql \
+  -e CONVERSATIONS_DRIVER=sqlite3 \
+  -e CONVERSATIONS_DSN=/app/data/conversations.db \
+  ghcr.io/yourusername/llm-gateway:latest
+```
+
+## Docker Compose
+
+The project includes a production-ready `docker-compose.yaml` file.
+
+### Basic Setup
+
+```bash
+# Create .env file with API keys
+cat > .env <<EOF
+GOOGLE_API_KEY=your-google-key
+ANTHROPIC_API_KEY=sk-ant-your-key
+OPENAI_API_KEY=sk-your-key
+EOF
+
+# Start gateway + Redis
+docker-compose up -d
+
+# Check status
+docker-compose ps
+
+# View logs
+docker-compose logs -f gateway
+```
+
+### With Monitoring
+
+Enable Prometheus and Grafana:
+
+```bash
+docker-compose --profile monitoring up -d
+```
+
+Access services:
+- Gateway: http://localhost:8080
+- Admin UI: http://localhost:8080/admin/
+- Prometheus: http://localhost:9090
+- Grafana: http://localhost:3000 (admin/admin)
+
+### Managing Services
+
+```bash
+# Stop all services
+docker-compose down
+
+# Stop and remove volumes (deletes data!)
+docker-compose down -v
+
+# Restart specific service
+docker-compose restart gateway
+
+# View logs
+docker-compose logs -f gateway
+
+# Update to latest image
+docker-compose pull
+docker-compose up -d
+```
+
+## Building from Source
+
+If you need to build your own image:
+
+```bash
+# Clone repository
+git clone https://github.com/yourusername/latticelm.git
+cd latticelm
+
+# Build image (includes frontend automatically)
+docker build -t llm-gateway:local .
+
+# Run your build
+docker run -d \
+  --name llm-gateway \
+  -p 8080:8080 \
+  --env-file .env \
+  llm-gateway:local
+```
+
+### Multi-Platform Builds
+
+Build for multiple architectures:
+
+```bash
+# Setup buildx
+docker buildx create --use
+
+# Build and push multi-platform
+docker buildx build \
+  --platform linux/amd64,linux/arm64 \
+  -t ghcr.io/yourusername/llm-gateway:latest \
+  --push .
+```
+
+## Production Considerations
+
+### Security
+
+**Use secrets management:**
+```bash
+# Docker secrets (Swarm)
+echo "sk-your-key" | docker secret create openai_key -
+
+docker service create \
+  --name llm-gateway \
+  --secret openai_key \
+  -e OPENAI_API_KEY_FILE=/run/secrets/openai_key \
+  ghcr.io/yourusername/llm-gateway:latest
+```
+
+**Run as non-root:**
+The image already runs as UID 1000 (non-root) by default.
+
+**Read-only filesystem:**
+```bash
+docker run -d \
+  --name llm-gateway \
+  --read-only \
+  --tmpfs /tmp \
+  -v llm-gateway-data:/app/data \
+  -p 8080:8080 \
+  --env-file .env \
+  ghcr.io/yourusername/llm-gateway:latest
+```
+
+### Resource Limits
+
+Set memory and CPU limits:
+
+```bash
+docker run -d \
+  --name llm-gateway \
+  -p 8080:8080 \
+  --memory="512m" \
+  --cpus="1.0" \
+  --env-file .env \
+  ghcr.io/yourusername/llm-gateway:latest
+```
+
+### Health Checks
+
+The image includes built-in health checks:
+
+```bash
+# Check health status
+docker inspect --format='{{.State.Health.Status}}' llm-gateway
+
+# Manual health check
+curl http://localhost:8080/health
+curl http://localhost:8080/ready
+```
+
+### Logging
+
+Configure structured JSON logging:
+
+```bash
+docker run -d \
+  --name llm-gateway \
+  -p 8080:8080 \
+  -e LOGGING_FORMAT=json \
+  -e LOGGING_LEVEL=info \
+  --log-driver=json-file \
+  --log-opt max-size=10m \
+  --log-opt max-file=3 \
+  ghcr.io/yourusername/llm-gateway:latest
+```
+
+### Networking
+
+**Custom network:**
+```bash
+# Create network
+docker network create llm-network
+
+# Run gateway on network
+docker run -d \
+  --name llm-gateway \
+  --network llm-network \
+  -p 8080:8080 \
+  --env-file .env \
+  ghcr.io/yourusername/llm-gateway:latest
+
+# Run Redis on same network
+docker run -d \
+  --name redis \
+  --network llm-network \
+  redis:7-alpine
+```
+
+## Troubleshooting
+
+### Container Won't Start
+
+Check logs:
+```bash
+docker logs llm-gateway
+docker logs --tail 50 llm-gateway
+```
+
+Common issues:
+- Missing required API keys
+- Port 8080 already in use (use `-p 9000:8080`)
+- Invalid configuration file syntax
+
+### High Memory Usage
+
+Monitor resources:
+```bash
+docker stats llm-gateway
+```
+
+Set limits:
+```bash
+docker update --memory="512m" llm-gateway
+```
+
+### Connection Issues
+
+**Test from inside container:**
+```bash
+docker exec -it llm-gateway wget -O- http://localhost:8080/health
+```
+
+**Check port bindings:**
+```bash
+docker port llm-gateway
+```
+
+**Test provider connectivity:**
+```bash
+docker exec llm-gateway wget -O- https://api.openai.com
+```
+
+### Database Locked (SQLite)
+
+If using SQLite with multiple containers:
+```bash
+# SQLite doesn't support concurrent writes
+# Use Redis or PostgreSQL instead:
+
+docker run -d \
+  --name redis \
+  redis:7-alpine
+
+docker run -d \
+  --name llm-gateway \
+  -p 8080:8080 \
+  -e CONVERSATIONS_STORE=redis \
+  -e CONVERSATIONS_DSN=redis://redis:6379/0 \
+  --link redis \
+  ghcr.io/yourusername/llm-gateway:latest
+```
+
+### Image Pull Failures
+
+**Authentication:**
+```bash
+# Login to GitHub Container Registry
+echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin
+
+# Pull image
+docker pull ghcr.io/yourusername/llm-gateway:latest
+```
+
+**Rate limiting:**
+Images are public but may be rate-limited. Use Docker Hub mirror or cache.
+
+### Debugging
+
+**Interactive shell:**
+```bash
+docker exec -it llm-gateway sh
+```
+
+**Inspect configuration:**
+```bash
+# Check environment variables
+docker exec llm-gateway env
+
+# Check config file
+docker exec llm-gateway cat /app/config.yaml
+```
+
+**Network debugging:**
+```bash
+docker exec llm-gateway wget --spider http://localhost:8080/health
+docker exec llm-gateway ping google.com
+```
+
+## Useful Commands
+
+```bash
+# Container lifecycle
+docker stop llm-gateway
+docker start llm-gateway
+docker restart llm-gateway
+docker rm -f llm-gateway
+
+# Logs
+docker logs -f llm-gateway
+docker logs --tail 100 llm-gateway
+docker logs --since 30m llm-gateway
+
+# Cleanup
+docker system prune -a
+docker volume prune
+docker image prune -a
+
+# Updates
+docker pull ghcr.io/yourusername/llm-gateway:latest
+docker stop llm-gateway
+docker rm llm-gateway
+docker run -d --name llm-gateway ... ghcr.io/yourusername/llm-gateway:latest
+```
+
+## Next Steps
+
+- **Production deployment**: See [Kubernetes guide](../k8s/README.md) for orchestration
+- **Monitoring**: Enable Prometheus metrics and set up Grafana dashboards
+- **Security**: Configure OAuth2/OIDC authentication
+- **Scaling**: Use Kubernetes HPA or Docker Swarm for auto-scaling
+
+## Additional Resources
+
+- [Main README](../README.md) - Full documentation
+- [Kubernetes Deployment](../k8s/README.md) - Production orchestration
+- [Configuration Reference](../config.example.yaml) - All config options
+- [GitHub Container Registry](https://github.com/yourusername/latticelm/pkgs/container/llm-gateway) - Published images
--- a/docs/README.md
+++ b/docs/README.md
@@ -0,0 +1,74 @@
+# Documentation
+
+Welcome to the latticelm documentation. This directory contains detailed guides and documentation for various aspects of the LLM Gateway.
+
+## User Guides
+
+### [Docker Deployment Guide](./DOCKER_DEPLOYMENT.md)
+Complete guide to deploying the LLM Gateway using Docker with pre-built images or building from source.
+
+**Topics covered:**
+- Using pre-built container images from CI/CD
+- Configuration with environment variables and config files
+- Docker Compose setup with Redis and monitoring
+- Production considerations (security, resources, networking)
+- Multi-platform builds
+- Troubleshooting and debugging
+
+### [Admin Web UI](./ADMIN_UI.md)
+Documentation for the built-in admin dashboard.
+
+**Topics covered:**
+- Accessing the Admin UI
+- Features and capabilities
+- System information dashboard
+- Provider status monitoring
+- Configuration management
+
+## Developer Documentation
+
+### [Admin UI Specification](./admin-ui-spec.md)
+Technical specification and design document for the Admin UI component.
+
+**Topics covered:**
+- Component architecture
+- API endpoints
+- UI mockups and wireframes
+- Implementation details
+
+### [Implementation Summary](./IMPLEMENTATION_SUMMARY.md)
+Overview of the implementation details and architecture decisions.
+
+**Topics covered:**
+- System architecture
+- Provider implementations
+- Key features and their implementations
+- Technology stack
+
+## Additional Resources
+
+## Deployment Guides
+
+### [Kubernetes Deployment Guide](../k8s/README.md)
+Production-grade Kubernetes deployment with high availability, monitoring, and security.
+
+**Topics covered:**
+- Deploying with Kustomize and kubectl
+- Secrets management (External Secrets Operator, Sealed Secrets)
+- Monitoring with Prometheus and OpenTelemetry
+- Horizontal Pod Autoscaling and PodDisruptionBudgets
+- Security best practices (RBAC, NetworkPolicies, Pod Security)
+- Cloud-specific guides (AWS EKS, GCP GKE, Azure AKS)
+- Storage options (Redis, PostgreSQL, managed services)
+- Rolling updates and rollback strategies
+
+For more documentation, see:
+
+- **[Main README](../README.md)** - Overview, quick start, and feature documentation
+- **[Configuration Example](../config.example.yaml)** - Detailed configuration options with comments
+
+## Need Help?
+
+- **Issues**: Check the [GitHub Issues](https://github.com/yourusername/latticelm/issues)
+- **Discussions**: Use [GitHub Discussions](https://github.com/yourusername/latticelm/discussions) for questions
+- **Contributing**: See [Contributing Guidelines](../README.md#contributing) in the main README