This diagram showcases the performance-optimized architecture of MCP Gateway (ContextForge), highlighting Rust-powered components, async patterns, and scaling capabilities.
Architecture Overview Rust-Powered Components (GIL Bypass) Component Technology Performance Gain Use Case Pydantic v2 Rust core (pydantic-core) 5-50x faster validation Request/response schemas (5,463 lines) orjson Rust JSON library 3x faster serialization All JSON encoding/decoding Granian Rust HTTP server +20-50% throughput HTTP request handling hiredis C-based Redis parser Up to 83x faster Large Redis response parsing uvloop Cython/libuv event loop 2-4x faster async I/O Async event loop
Optimization Before After Improvement psycopg v3 prepared statementsParsed each query Auto-prepared 2-3x faster repeated queries COPY protocol INSERT statements Binary COPY 5-10x faster bulk inserts Pipeline mode Sequential queries Pipelined 2-5x batch improvements PgBouncer pooling 1600+ connections 200 connections 8x connection reduction
Cache Layer Hit Rate Latency Reduction DB Query Reduction JWT Cache >80% 5-12ms → <1ms Per-request auth overhead Auth Cache >90% 8-15ms → 1-3ms 3-4 → 0-1 queries/request Registry Cache 95%+ Variable 50-200 → 0-1 queries GlobalConfig Cache 99%+ 1ms → 0.00001ms 42K+ queries eliminated
Compression & Bandwidth Algorithm Compression Ratio Best For Brotli 15-25% smaller than Gzip Production, CDNs Zstd Very good, fastest High-throughput APIs Gzip Good, universal Legacy compatibility
Scaling Capacity Configuration Capacity Single pod (16 workers) ~1,600 RPS 3 pods (default) ~4,800 RPS 10 pods (HPA scaled) ~16,000 RPS 50 pods (max) ~80,000 RPS
Issue # Feature Impact #1695 Granian HTTP server migration +20-50% throughput #1696, #1692 orjson throughout codebase 3x JSON performance #1699 uvicorn[standard] with uvloop/httptools 15-30% faster async #1702 hiredis Redis parser Up to 83x Redis parsing #1740 psycopg v3 migration Auto-prepared, COPY, pipeline #1750, #1753 PgBouncer connection pooling 8x connection reduction #1715 GlobalConfig in-memory cache 42K queries eliminated #1773 get_user_teams() caching Reduced idle-in-transaction #1809-1814 Schema/template/filter caching Compilation overhead removed #1816, #1819, #1830 Precompiled regex patterns CPU reduction in hot paths #1828, #1837 SSE/logging micro-optimizations Reduced allocation overhead #1844 Monitoring profile Production observability #2025 Startup resilience (exponential backoff) Prevents crash-loop CPU storms
Startup Resilience The gateway implements exponential backoff with jitter for database and Redis connection retries at startup. This prevents CPU-intensive crash-respawn loops when dependencies are temporarily unavailable.
Problem Solved Without exponential backoff, a dependency outage would cause:
Solution: Exponential Backoff with Jitter Metric Before After Retry attempts 3 (~6 seconds) 30 (~5 minutes) CPU during outage ~500% (crash loop) ~0% (sleeping) Recovery pattern Thundering herd Staggered with jitter System stability Cascading failures Graceful degradation
Configuration # Database Startup Resilience
DB_MAX_RETRIES = 30 # Max attempts before worker exits (default: 30)
DB_RETRY_INTERVAL_MS = 2000 # Base interval in ms (doubles each attempt)
# Redis Startup Resilience
REDIS_MAX_RETRIES = 30 # Max attempts before worker exits (default: 30)
REDIS_RETRY_INTERVAL_MS = 2000 # Base interval in ms (doubles each attempt)
Retry Progression Example With default settings (2s base interval):
Attempt Base Delay With Jitter (±25%) Cumulative Time 1 2s 1.5s - 2.5s ~2s 2 4s 3s - 5s ~6s 3 8s 6s - 10s ~14s 4 16s 12s - 20s ~30s 5+ 30s (cap) 22.5s - 37.5s ~60s+
After 30 retries: approximately 5 minutes total wait time before the worker gives up, providing ample time for dependencies to recover during maintenance windows or transient outages.
Future: Python 3.14 Free-Threading (GIL Removal) Quick Reference Commands # Start full performance stack
up -d
# Access via caching proxy (production)
http://localhost:8080/health
# Start with monitoring
--profile monitoring up -d
# View cache hit rates
-I http://localhost:8080/tools | grep X-Cache-Status
# Run load test
-n 10000 -c 200 -H "Authorization: Bearer $TOKEN " http://localhost:8080/
# Check HPA status (Kubernetes)
get hpa -n mcp-gateway