Gateway Tuning Guide¶

This page collects practical levers for squeezing the most performance, reliability, and observability out of ContextForge-no matter where you run the container (Code Engine, Kubernetes, Docker Compose, Nomad, etc.).

TL;DR

1. Tune the runtime environment via .env and configure mcpgateway to use PostgreSQL and Redis.
2. Adjust Gunicorn workers & time-outs in gunicorn.conf.py.
3. Right-size CPU/RAM for the container or spin up more instances (with shared Redis state) and change the database settings (ex: connection limits).
4. Benchmark with hey (or your favourite load-generator) before & after. See also: performance testing guide

1 - Environment variables (.env)¶

Redis Connection Pool Tuning¶

When using CACHE_TYPE=redis, tune the connection pool for your workload:

High-concurrency production settings:

REDIS_MAX_CONNECTIONS=100
REDIS_SOCKET_TIMEOUT=1.0
REDIS_SOCKET_CONNECT_TIMEOUT=1.0
REDIS_HEALTH_CHECK_INTERVAL=15

Tip Any change here requires rebuilding or restarting the container if you pass the file with --env-file.

2 - Gunicorn settings (gunicorn.conf.py)¶

Edit the file before building the image, then redeploy.

2b - Uvicorn Performance Extras¶

ContextForge uses uvicorn[standard] which includes high-performance components that are automatically detected and used:

Automatic Detection¶

When Gunicorn spawns Uvicorn workers, these components are automatically detected:

# Verify extras are installed
pip list | grep -E "uvloop|httptools|websockets|watchfiles"

# Expected output (Linux/macOS):
# httptools    0.6.x
# uvloop       0.21.x
# websockets   15.x.x
# watchfiles   1.x.x

Platform Notes¶

• Linux/macOS: Full performance benefits (uvloop + httptools)
• Windows: httptools provides benefits; uvloop unavailable (graceful fallback to asyncio)

Performance Impact¶

Combined improvements from uvloop and httptools:

Note: These optimizations are transparent - no code or configuration changes needed.

2c - Granian (Alternative HTTP Server)¶

ContextForge supports two HTTP servers:

• Gunicorn + Uvicorn (default) - Battle-tested, mature, excellent stability
• Granian (alternative) - Rust-based, native HTTP/2, lower memory

Usage¶

# Local development
make serve                    # Gunicorn + Uvicorn (default)
make serve-granian            # Granian (alternative)
make serve-granian-http2      # Granian with HTTP/2 + TLS

# Container with Gunicorn (default)
make container-run
make container-run-gunicorn-ssl

# Container with Granian (alternative)
make container-run-granian
make container-run-granian-ssl

# Docker Compose (default uses Gunicorn)
docker compose up

Switching HTTP Servers¶

The HTTP_SERVER environment variable controls which server to use:

# Docker/Podman - use Gunicorn (default)
docker run mcpgateway/mcpgateway

# Docker/Podman - use Granian
docker run -e HTTP_SERVER=granian mcpgateway/mcpgateway

# Docker Compose - set in environment section
environment:
  - HTTP_SERVER=gunicorn  # default
  # - HTTP_SERVER=granian # alternative

Configuration¶

Performance tuning profiles:

# High-throughput (fewer workers, more threads per worker)
GRANIAN_WORKERS=4 GRANIAN_RUNTIME_THREADS=4 make serve

# High-concurrency (more workers, max backpressure)
GRANIAN_WORKERS=16 GRANIAN_BACKPRESSURE=1024 GRANIAN_BACKLOG=4096 make serve

# Memory-constrained (fewer workers)
GRANIAN_WORKERS=2 make serve

# Force HTTP/1 only (avoids HTTP/2 overhead)
GRANIAN_HTTP=1 make serve

Notes:

• On Python 3.12+, the Rust task implementation is unavailable; asyncio is used automatically
• uvloop provides best performance on Linux/macOS
• Increase GRANIAN_BACKLOG and GRANIAN_BACKPRESSURE for high-concurrency workloads

Backpressure for Overload Protection¶

Granian's native backpressure prevents unbounded request queuing during overload. When the server reaches capacity, excess requests receive immediate 503 responses instead of waiting in a queue (which can cause memory exhaustion or cascading timeouts).

How it works:

Incoming Request
       │
       ▼
┌──────────────────────────────────┐
│  Granian Worker (1 of N)         │
│                                  │
│  current_requests < BACKPRESSURE?│
│      │                           │
│      ├── YES → Process request   │
│      │                           │
│      └── NO  → Immediate 503     │
│               (no queuing)       │
└──────────────────────────────────┘

Capacity calculation:

Total capacity = GRANIAN_WORKERS × GRANIAN_BACKPRESSURE

Example with recommended settings:
  Workers: 16
  Backpressure: 64
  Total: 16 × 64 = 1024 concurrent requests

  Requests 1-1024: Processed normally
  Request 1025+: Immediate 503 Service Unavailable

Recommended production settings:

# docker-compose.yml or Kubernetes
environment:
  - HTTP_SERVER=granian
  - GRANIAN_WORKERS=16
  - GRANIAN_BACKLOG=4096        # OS socket queue for pending connections
  - GRANIAN_BACKPRESSURE=64     # Per-worker limit (16×64=1024 total)

Benefits over unbounded queuing:

When to Use Granian¶

Performance Comparison¶

Note: Always benchmark with your specific workload before switching servers.

Real-World Performance (Database-Bound Workload)¶

Under load testing with 2500 concurrent users against PostgreSQL:

Key Finding: When the database is the bottleneck, both servers achieve similar throughput. The main differences are:

• Memory: Gunicorn uses 32% less RAM (fork-based model with copy-on-write)
• CPU: Granian uses 8% less CPU (more efficient HTTP parsing in Rust)
• Stability: Granian handles overload gracefully (backpressure), Gunicorn queues indefinitely

Recommendation:

3 - Container resources¶

Always test with your workload; JSON-RPC payload size and backend model latency change the equation.

To change your database connection settings, see the respective documentation for your selected database or managed service. For example, when using IBM Cloud Databases for PostgreSQL - you can raise the maximum number of connections.

4 - Performance testing¶

4.1 Tooling: hey¶

Install one of:

brew install hey            # macOS
sudo apt install hey         # Debian/Ubuntu
# or build from source
go install github.com/rakyll/hey@latest  # $GOPATH/bin must be in PATH

4.2 Sample load-test script (tests/hey.sh)¶

#!/usr/bin/env bash
# Run 10 000 requests with 200 concurrent workers.
JWT="$(cat jwt.txt)"   # <- place a valid token here
hey -n 10000 -c 200 \
    -m POST \
    -T application/json \
    -H "Authorization: Bearer ${JWT}" \
    -D tests/hey/payload.json \
    http://localhost:4444/rpc

Payload (tests/hey/payload.json)

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "convert_time",
  "params": {
    "source_timezone": "Europe/Berlin",
    "target_timezone": "Europe/Dublin",
    "time": "09:00"
  }
}

4.3 Reading the output¶

hey prints latency distribution, requests/second, and error counts. Focus on:

• 99^th percentile latency - adjust timeout if it clips.
• Errors - 5xx under load often mean too few workers or DB connections.
• Throughput (RPS) - compare before/after tuning.

4.4 Common bottlenecks & fixes¶

5 - Logging & observability¶

• Set loglevel = "debug" in gunicorn.conf.py during tests; revert to info in prod.
• Forward stdout/stderr from the container to your platform's log stack (e.g. kubectl logs, docker logs).
• Expose /metrics via a Prometheus exporter (planned) for request timing & queue depth; track enablement in the project roadmap.

6 - MCP Session Pool Tuning¶

The MCP session pool maintains persistent connections to upstream MCP servers, providing 10-20x latency improvement for repeated tool calls from the same user.

MCP_SESSION_POOL_ENABLED=true

When to Enable Pooling¶

Configuration Variables¶

Recommended Production Settings¶

# Baseline settings for authenticated deployments (low-to-moderate traffic)
MCP_SESSION_POOL_ENABLED=true
MCP_SESSION_POOL_MAX_PER_KEY=10
MCP_SESSION_POOL_TTL=300
MCP_SESSION_POOL_TRANSPORT_TIMEOUT=30

# High-concurrency settings (1000+ concurrent users)
MCP_SESSION_POOL_ENABLED=true
MCP_SESSION_POOL_MAX_PER_KEY=200          # 50-200 for high concurrency
MCP_SESSION_POOL_TTL=300
MCP_SESSION_POOL_TRANSPORT_TIMEOUT=30
MCP_SESSION_POOL_ACQUIRE_TIMEOUT=60       # Longer timeout under load

# Ensure identity headers are present
ENABLE_HEADER_PASSTHROUGH=true
DEFAULT_PASSTHROUGH_HEADERS="Authorization,X-Tenant-Id,X-User-Id,X-API-Key"

Session Isolation¶

Sessions are isolated by a composite key: (URL, identity_hash, transport_type). Identity is derived from authentication headers (Authorization, X-Tenant-ID, X-User-ID, X-API-Key, Cookie).

Key security considerations:

1. Anonymous Pooling: When no identity headers are present, identity collapses to "anonymous" and all such requests share sessions. This is safe only if upstream MCP servers are stateless.
2. Shared Credentials: With OAuth Client Credentials or static API keys, all users share the same identity hash. Only safe if the upstream MCP server has no per-user state.
3. Header Passthrough: If gateway auth is disabled (AUTH_REQUIRED=false), enable header passthrough to preserve user identity:

   ENABLE_HEADER_PASSTHROUGH=true
   DEFAULT_PASSTHROUGH_HEADERS="Authorization,X-Tenant-Id,X-User-Id"
   

Long-Running Tools¶

The transport timeout applies to all HTTP operations, not just connection establishment. For tools that take longer than 30 seconds:

# Increase for long-running tools
MCP_SESSION_POOL_TRANSPORT_TIMEOUT=120

Health Check Timeout Trade-offs¶

Pool staleness checks use MCP_SESSION_POOL_TRANSPORT_TIMEOUT (default 30s) for session acquisition. When MCP_SESSION_POOL_EXPLICIT_HEALTH_RPC=true, the explicit RPC call uses HEALTH_CHECK_TIMEOUT (default 5s).

Behavior summary:

Trade-off: The 30s transport timeout allows long-running tools to complete but means unhealthy sessions may take longer to detect. If you need faster failure detection:

# Stricter health checks (5s timeout for explicit RPC)
MCP_SESSION_POOL_EXPLICIT_HEALTH_RPC=true
HEALTH_CHECK_TIMEOUT=5

# Or reduce transport timeout (affects all operations)
MCP_SESSION_POOL_TRANSPORT_TIMEOUT=10

Circuit Breaker Behavior¶

The circuit breaker is keyed by URL only (not per-identity). After MCP_SESSION_POOL_CIRCUIT_BREAKER_THRESHOLD consecutive session creation failures for a URL, the circuit opens and all requests fail fast for MCP_SESSION_POOL_CIRCUIT_BREAKER_RESET seconds.

Note: Only session creation failures (connection refused, SSL errors) trip the circuit. Tool call failures do not affect the circuit breaker.

Monitoring¶

Monitor pool performance via the metrics endpoint:

curl -u admin:changeme http://localhost:4444/admin/mcp-pool/metrics

Response includes:

• total_sessions_created / total_sessions_reused: Pool hit ratio
• pool_hits / pool_misses: Cache effectiveness
• active_sessions: Current utilization
• circuit_breaker_states: Per-URL circuit status

Operational Checklist¶

Before enabling pooling in production:

• Confirm upstream MCP servers are stateless for any shared/anonymous access
• Verify identity headers are present and stable
• Validate tool call durations vs MCP_SESSION_POOL_TRANSPORT_TIMEOUT
• Ensure tracing headers are not relied upon in pooled sessions

After enabling pooling:

• Monitor pool metrics at /admin/mcp-pool/metrics
• Watch for increased tool timeouts or unexpected auth failures
• Verify correlation IDs in upstream logs (note: per-request headers are stripped from pooled sessions)

7 - Nginx Reverse Proxy Tuning¶

When deploying ContextForge behind nginx (as in the default docker-compose.yml), several optimizations can significantly improve performance under load.

Admin UI Caching¶

Admin pages use Jinja2 template rendering which is CPU-intensive under high concurrency. The default nginx configuration enables short-TTL caching with multi-tenant isolation:

Performance impact (4000 concurrent users):

Multi-Tenant Cache Safety¶

The cache key includes all authentication credentials to ensure user isolation:

proxy_cache_key "$scheme$request_method$host$request_uri$is_args$args$http_authorization$cookie_jwt_token$cookie_access_token";

This ensures:

• Bearer token auth ($http_authorization): API clients get isolated caches
• Primary session cookie ($cookie_jwt_token): Browser users get isolated caches
• Alternative auth cookie ($cookie_access_token): Fallback auth method also isolated

Verifying Cache Behavior¶

Check the X-Cache-Status header to verify caching is working:

curl -I http://localhost:8080/admin/ -b "jwt_token=..." | grep X-Cache
# X-Cache-Status: MISS  (first request)
# X-Cache-Status: HIT   (subsequent requests within 5s)
# X-Cache-Status: STALE (background refresh in progress)

Disabling Admin Caching¶

If you need real-time admin data or have concerns about caching, modify infra/nginx/nginx.conf:

location /admin {
    proxy_cache off;
    add_header Cache-Control "no-cache, no-store, must-revalidate" always;
    # ... rest of config
}

8 - High-Concurrency Production Tuning¶

This section covers comprehensive tuning for deployments handling 1000+ concurrent users. These settings have been tested under load with 6500 concurrent users.

8.1 Database Connection Pool (SQLAlchemy)¶

The gateway's internal connection pool manages connections between the application and PgBouncer (or PostgreSQL directly).

Example high-concurrency configuration:

# With PgBouncer (recommended)
DB_POOL_CLASS=queue
DB_POOL_PRE_PING=true
DB_POOL_SIZE=20
DB_MAX_OVERFLOW=10
DB_POOL_TIMEOUT=60
DB_POOL_RECYCLE=60    # Half of PgBouncer CLIENT_IDLE_TIMEOUT (120s)

Common errors and solutions:

8.2 PgBouncer Connection Pooler¶

PgBouncer multiplexes many application connections into fewer PostgreSQL connections, dramatically reducing database overhead.

Client-Side Settings (from gateway workers)¶

Server-Side Settings (to PostgreSQL)¶

Timeout Settings¶

Transaction Reset Settings¶

Example PgBouncer configuration (docker-compose.yml):

pgbouncer:
  image: edoburu/pgbouncer:latest
  environment:
    - DATABASE_URL=postgres://postgres:password@postgres:5432/mcp
    - POOL_MODE=transaction
    # Client limits
    - MAX_CLIENT_CONN=5000
    - DEFAULT_POOL_SIZE=600
    - MIN_POOL_SIZE=100
    - RESERVE_POOL_SIZE=150
    # Server limits
    - MAX_DB_CONNECTIONS=700
    - SERVER_LIFETIME=1800
    - SERVER_IDLE_TIMEOUT=600
    # Timeouts
    - QUERY_WAIT_TIMEOUT=60
    - CLIENT_IDLE_TIMEOUT=120
    - IDLE_TRANSACTION_TIMEOUT=60
    # Reset
    - SERVER_RESET_QUERY=DISCARD ALL
    - SERVER_RESET_QUERY_ALWAYS=1
  ulimits:
    nofile:
      soft: 65536
      hard: 65536

8.3 Container File Descriptor Limits (ulimits)¶

Each network connection requires a file descriptor. Containers default to 1024 soft limit, which is insufficient for high concurrency.

docker-compose.yml example:

services:
  pgbouncer:
    ulimits:
      nofile:
        soft: 65536
        hard: 65536

  postgres:
    ulimits:
      nofile:
        soft: 8192
        hard: 8192

  redis:
    ulimits:
      nofile:
        soft: 65536
        hard: 65536

  gateway:
    ulimits:
      nofile:
        soft: 65535
        hard: 65535

Verification:

# Check container limits
docker exec <container> cat /proc/1/limits | grep "open files"

# Count current open FDs
docker exec <container> ls /proc/1/fd | wc -l

Common error: accept() failed: No file descriptors available - Increase ulimits.nofile.

8.4 Host System Tuning (sysctl)¶

The Docker host kernel settings affect all containers. These must be set on the host, not in containers.

Apply temporarily:

sudo sysctl -w \
  net.core.somaxconn=65535 \
  net.core.netdev_max_backlog=65535 \
  net.ipv4.tcp_max_syn_backlog=65535 \
  net.ipv4.tcp_fin_timeout=15 \
  net.ipv4.tcp_tw_reuse=1 \
  net.ipv4.ip_local_port_range="1024 65535"

Apply permanently (/etc/sysctl.d/99-mcp-loadtest.conf):

# High-concurrency TCP tuning for ContextForge load testing
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 1024 65535
fs.file-max = 2097152

Then apply: sudo sysctl -p /etc/sysctl.d/99-mcp-loadtest.conf

8.5 HTTPX Client Pool (Outbound HTTP)¶

The gateway uses a shared HTTPX client pool for all outbound requests (federation, health checks, A2A, MCP tool calls).

Example:

HTTPX_MAX_CONNECTIONS=200
HTTPX_MAX_KEEPALIVE_CONNECTIONS=100
HTTPX_KEEPALIVE_EXPIRY=30.0
HTTPX_READ_TIMEOUT=120.0
HTTPX_POOL_TIMEOUT=10.0

8.6 Complete High-Concurrency Configuration¶

Here's a complete configuration for 3000-6500 concurrent users:

# docker-compose.yml gateway environment
environment:
  # Database pool (via PgBouncer)
  - DATABASE_URL=postgresql+psycopg://postgres:password@pgbouncer:6432/mcp
  - DB_POOL_CLASS=queue
  - DB_POOL_PRE_PING=true
  - DB_POOL_SIZE=20
  - DB_MAX_OVERFLOW=10
  - DB_POOL_TIMEOUT=60
  - DB_POOL_RECYCLE=60

  # MCP Session Pool
  - MCP_SESSION_POOL_ENABLED=true
  - MCP_SESSION_POOL_MAX_PER_KEY=200
  - MCP_SESSION_POOL_ACQUIRE_TIMEOUT=60

  # HTTPX Client Pool
  - HTTPX_MAX_CONNECTIONS=200
  - HTTPX_MAX_KEEPALIVE_CONNECTIONS=100
  - HTTPX_READ_TIMEOUT=120.0

  # Redis
  - REDIS_MAX_CONNECTIONS=150

  # Performance
  - LOG_LEVEL=ERROR
  - DISABLE_ACCESS_LOG=true
  - AUDIT_TRAIL_ENABLED=false

9 - MCP Streamable HTTP Transport Tuning¶

The MCP Streamable HTTP endpoint (/servers/{id}/mcp) has its own performance characteristics distinct from the REST API (/rpc). This section covers settings that specifically affect MCP protocol performance.

Critical Settings¶

These settings have the largest measured impact on MCP throughput:

Settings with Negligible Impact¶

These settings were tested and found to have less than ~3% effect on MCP throughput. Tune them for correctness, not performance:

MCP SDK / FastMCP Tunables¶

When running MCP servers behind the gateway:

MCP Worker Tuning¶

The gateway worker count (GUNICORN_WORKERS) directly affects MCP throughput:

• Too few workers: CPU underutilized, low RPS
• Too many workers: Excessive context switching, DB connection pressure
• Rule of thumb: Match GUNICORN_WORKERS to the available CPU cores per container. 24 workers per 8-CPU container is well-tuned for MCP workloads.

Auth Cache TTL Guidance¶

The auth cache prevents repeated database lookups for the same JWT token. Higher TTLs reduce DB pressure but delay permission changes:

10 - Disable unused features¶

ContextForge has many optional features that are enabled by default for completeness but consume CPU, memory, or database I/O even when not used. Disabling features you do not need is a free performance win that does not require additional resources.

High-impact features to disable¶

These features have measurable per-request or background overhead. Disable any you are not actively using:

Medium-impact features¶

These add some overhead but are useful in most deployments. Disable only if you are certain you do not need them:

Low-impact features (safe to leave enabled)¶

These have negligible runtime cost and are generally worth keeping:

Deployment profiles¶

MCP-only deployment (maximum MCP protocol throughput):

# Disable everything not needed for MCP tool serving
MCPGATEWAY_UI_ENABLED=false
MCPGATEWAY_ADMIN_API_ENABLED=false
MCPGATEWAY_A2A_ENABLED=false
MCPGATEWAY_CATALOG_ENABLED=false
LLMCHAT_ENABLED=false
PLUGINS_ENABLED=false
OBSERVABILITY_ENABLED=false
ENABLE_METRICS=false
AUDIT_TRAIL_ENABLED=false
SECURITY_LOGGING_ENABLED=false
STRUCTURED_LOGGING_DATABASE_ENABLED=false
CORRELATION_ID_ENABLED=false
DB_METRICS_RECORDING_ENABLED=false
MCPGATEWAY_PERFORMANCE_NET_CONNECTIONS_ENABLED=false
COMPRESSION_ENABLED=false          # Let nginx handle compression
DISABLE_ACCESS_LOG=true

# Keep these enabled
AUTH_CACHE_ENABLED=true
REGISTRY_CACHE_ENABLED=true
MCP_SESSION_POOL_ENABLED=true
CACHE_TYPE=redis

Full-featured production (all features, tuned for performance):

# Features enabled
MCPGATEWAY_UI_ENABLED=true
MCPGATEWAY_ADMIN_API_ENABLED=true
MCPGATEWAY_A2A_ENABLED=true
PLUGINS_ENABLED=true

# Expensive features disabled unless needed
OBSERVABILITY_ENABLED=false         # Enable only when tracing needed
ENABLE_METRICS=false                # Enable with Prometheus
AUDIT_TRAIL_ENABLED=false           # Enable for compliance
STRUCTURED_LOGGING_DATABASE_ENABLED=false
SECURITY_LOGGING_ENABLED=false
DB_METRICS_RECORDING_ENABLED=true   # Keep for built-in analytics

# Caching maximized
AUTH_CACHE_ENABLED=true
AUTH_CACHE_BATCH_QUERIES=true
REGISTRY_CACHE_ENABLED=true
MCP_SESSION_POOL_ENABLED=true
CACHE_TYPE=redis

Development / debugging (full observability, relaxed performance):

# Everything on for debugging
MCPGATEWAY_UI_ENABLED=true
MCPGATEWAY_ADMIN_API_ENABLED=true
OBSERVABILITY_ENABLED=true
ENABLE_METRICS=true
DB_METRICS_RECORDING_ENABLED=true
DB_QUERY_LOG_ENABLED=true           # N+1 detection
STRUCTURED_LOGGING_DATABASE_ENABLED=true
LOG_LEVEL=INFO
CORRELATION_ID_ENABLED=true
PERFORMANCE_TRACKING_ENABLED=true

11 - Security tips while tuning¶

• Never commit real JWT_SECRET_KEY, DB passwords, or tokens-use .env.example as a template.
• Prefer platform secrets (K8s Secrets, Code Engine secrets) over baking creds into the image.
• If you enable gevent/eventlet, pin their versions and run bandit or trivy scans.

See Also¶

• Performance Profiling Guide - py-spy, memray, PostgreSQL profiling, MCP bottleneck triage
• Database Performance Guide - N+1 detection, query logging, query counting
• Performance Architecture - MCP request path, caching layers, scaling capacity
• Scaling Guide - Production scaling configuration