Gateway Tuning Guide¶

This page collects practical levers for squeezing the most performance, reliability, and observability out of MCP Gateway-no matter where you run the container (Code Engine, Kubernetes, Docker Compose, Nomad, etc.).

TL;DR

1. Tune the runtime environment via .env and configure mcpgateway to use PostgreSQL and Redis.
2. Adjust Gunicorn workers & time-outs in gunicorn.conf.py.
3. Right-size CPU/RAM for the container or spin up more instances (with shared Redis state) and change the database settings (ex: connection limits).
4. Benchmark with hey (or your favourite load-generator) before & after. See also: performance testing guide

1 - Environment variables (.env)¶

Redis Connection Pool Tuning¶

When using CACHE_TYPE=redis, tune the connection pool for your workload:

High-concurrency production settings:

REDIS_MAX_CONNECTIONS=100
REDIS_SOCKET_TIMEOUT=1.0
REDIS_SOCKET_CONNECT_TIMEOUT=1.0
REDIS_HEALTH_CHECK_INTERVAL=15

Tip Any change here requires rebuilding or restarting the container if you pass the file with --env-file.

2 - Gunicorn settings (gunicorn.conf.py)¶

Edit the file before building the image, then redeploy.

2b - Uvicorn Performance Extras¶

MCP Gateway uses uvicorn[standard] which includes high-performance components that are automatically detected and used:

Automatic Detection¶

When Gunicorn spawns Uvicorn workers, these components are automatically detected:

# Verify extras are installed
pip list | grep -E "uvloop|httptools|websockets|watchfiles"

# Expected output (Linux/macOS):
# httptools    0.6.x
# uvloop       0.21.x
# websockets   15.x.x
# watchfiles   1.x.x

Platform Notes¶

• Linux/macOS: Full performance benefits (uvloop + httptools)
• Windows: httptools provides benefits; uvloop unavailable (graceful fallback to asyncio)

Performance Impact¶

Combined improvements from uvloop and httptools:

Note: These optimizations are transparent - no code or configuration changes needed.

2c - Granian (Alternative HTTP Server)¶

MCP Gateway supports two HTTP servers: - Gunicorn + Uvicorn (default) - Battle-tested, mature, excellent stability - Granian (alternative) - Rust-based, native HTTP/2, lower memory

Usage¶

# Local development
make serve                    # Gunicorn + Uvicorn (default)
make serve-granian            # Granian (alternative)
make serve-granian-http2      # Granian with HTTP/2 + TLS

# Container with Gunicorn (default)
make container-run
make container-run-gunicorn-ssl

# Container with Granian (alternative)
make container-run-granian
make container-run-granian-ssl

# Docker Compose (default uses Gunicorn)
docker compose up

Switching HTTP Servers¶

The HTTP_SERVER environment variable controls which server to use:

# Docker/Podman - use Gunicorn (default)
docker run mcpgateway/mcpgateway

# Docker/Podman - use Granian
docker run -e HTTP_SERVER=granian mcpgateway/mcpgateway

# Docker Compose - set in environment section
environment:
  - HTTP_SERVER=gunicorn  # default
  # - HTTP_SERVER=granian # alternative

Configuration¶

Performance tuning profiles:

# High-throughput (fewer workers, more threads per worker)
GRANIAN_WORKERS=4 GRANIAN_RUNTIME_THREADS=4 make serve

# High-concurrency (more workers, max backpressure)
GRANIAN_WORKERS=16 GRANIAN_BACKPRESSURE=1024 GRANIAN_BACKLOG=4096 make serve

# Memory-constrained (fewer workers)
GRANIAN_WORKERS=2 make serve

# Force HTTP/1 only (avoids HTTP/2 overhead)
GRANIAN_HTTP=1 make serve

Notes: - On Python 3.12+, the Rust task implementation is unavailable; asyncio is used automatically - uvloop provides best performance on Linux/macOS - Increase GRANIAN_BACKLOG and GRANIAN_BACKPRESSURE for high-concurrency workloads

When to Use Granian¶

Performance Comparison¶

Note: Always benchmark with your specific workload before switching servers.

Real-World Performance (Database-Bound Workload)¶

Under load testing with 2500 concurrent users against PostgreSQL:

Key Finding: When the database is the bottleneck, both servers achieve similar throughput. The main differences are:

• Memory: Gunicorn uses 32% less RAM (fork-based model with copy-on-write)
• CPU: Granian uses 8% less CPU (more efficient HTTP parsing in Rust)
• Stability: Granian handles overload gracefully (backpressure), Gunicorn queues indefinitely

Recommendation:

3 - Container resources¶

Always test with your workload; JSON-RPC payload size and backend model latency change the equation.

To change your database connection settings, see the respective documentation for your selected database or managed service. For example, when using IBM Cloud Databases for PostgreSQL - you can raise the maximum number of connections.

4 - Performance testing¶

4.1 Tooling: hey¶

Install one of:

brew install hey            # macOS
sudo apt install hey         # Debian/Ubuntu
# or build from source
go install github.com/rakyll/hey@latest  # $GOPATH/bin must be in PATH

4.2 Sample load-test script (tests/hey.sh)¶

#!/usr/bin/env bash
# Run 10 000 requests with 200 concurrent workers.
JWT="$(cat jwt.txt)"   # <- place a valid token here
hey -n 10000 -c 200 \
    -m POST \
    -T application/json \
    -H "Authorization: Bearer ${JWT}" \
    -D tests/hey/payload.json \
    http://localhost:4444/rpc

Payload (tests/hey/payload.json)

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "convert_time",
  "params": {
    "source_timezone": "Europe/Berlin",
    "target_timezone": "Europe/Dublin",
    "time": "09:00"
  }
}

4.3 Reading the output¶

hey prints latency distribution, requests/second, and error counts. Focus on:

• 99^th percentile latency - adjust timeout if it clips.
• Errors - 5xx under load often mean too few workers or DB connections.
• Throughput (RPS) - compare before/after tuning.

4.4 Common bottlenecks & fixes¶

5 - Logging & observability¶

• Set loglevel = "debug" in gunicorn.conf.py during tests; revert to info in prod.
• Forward stdout/stderr from the container to your platform's log stack (e.g. kubectl logs, docker logs).
• Expose /metrics via a Prometheus exporter (planned) for request timing & queue depth; track enablement in the project roadmap.

6 - Security tips while tuning¶

• Never commit real JWT_SECRET_KEY, DB passwords, or tokens-use .env.example as a template.
• Prefer platform secrets (K8s Secrets, Code Engine secrets) over baking creds into the image.
• If you enable gevent/eventlet, pin their versions and run bandit or trivy scans.