Plugin Framework¶
Production Ready
The plugin framework is production ready with comprehensive hook coverage, robust error handling, and battle-tested implementations. Supports both self-contained and external service plugins.
Overview¶
The MCP Context Forge Plugin Framework provides a comprehensive, production-grade system for extending gateway functionality through pre/post processing hooks at various points in the MCP request lifecycle. The framework supports both high-performance self-contained plugins and sophisticated external AI service integrations.
Key Capabilities¶
- AI Safety Middleware - Integration with LlamaGuard, OpenAI Moderation, custom ML models
- Content Security - PII detection and masking, input validation, output sanitization
- Policy Enforcement - Business rules, compliance checking, audit trails
- Performance Protection - Timeout handling, resource limits, graceful degradation
- Operational Excellence - Hot configuration reload, health monitoring, detailed metrics
- Enterprise Features - Multi-tenant isolation, conditional execution, sophisticated context management
Architecture¶
The plugin framework implements a hybrid architecture supporting both self-contained and external service integrations:
Self-Contained Plugins¶
- In-Process Execution: Written in Python, run directly within the gateway process
- High Performance: Sub-millisecond latency, no network overhead
- Direct Access: Full access to gateway internals and context
- Use Cases: PII filtering, regex transformations, input validation, simple business rules
- Examples:
PIIFilterPlugin
,SearchReplacePlugin
,DenyListPlugin
External Service Plugins¶
- Microservice Integration: Call external AI safety services via HTTP/gRPC/MCP
- Enterprise AI Support: LlamaGuard, OpenAI Moderation, custom ML models
- Authentication Support: Bearer tokens, API keys, mutual TLS, custom headers
- Scalable Architecture: Services can be deployed independently, auto-scaled
- Use Cases: Advanced AI safety, complex ML inference, enterprise policy engines
- Examples: LlamaGuard integration, OpenAI Moderation, HashiCorp Vault, OPA
Unified Plugin Interface¶
Both plugin types implement the same interface, enabling seamless switching between deployment models:
class Plugin:
async def prompt_pre_fetch(self, payload, context) -> PluginResult
async def tool_pre_invoke(self, payload, context) -> PluginResult
# ... unified interface for all hook points
Enabling Plugins¶
1. Environment Configuration¶
Enable the plugin framework in your .env
file:
# Enable plugin framework
PLUGINS_ENABLED=true
# Optional: Custom plugin config path
PLUGIN_CONFIG_FILE=plugins/config.yaml
2. Plugin Configuration¶
The plugin configuration file is used to configure a set of plugins to run a set of hook points throughout the MCP Context Forge. An example configuration is below. It contains two main sections: plugins
and plugin_settings
.
Create or modify plugins/config.yaml
:
# Main plugin configuration
plugins:
- name: "ContentFilter"
kind: "plugins.native.content_filter.ContentFilterPlugin"
description: "Filters inappropriate content"
version: "1.0"
author: "Your Team"
hooks: ["prompt_pre_fetch", "prompt_post_fetch"]
tags: ["security", "filter"]
mode: "enforce" # enforce | permissive | disabled
priority: 100 # Lower number = higher priority
conditions:
- prompts: ["customer_chat", "support_bot"]
server_ids: [] # Apply to all servers
tenant_ids: [] # Apply to all tenants
config:
# Plugin-specific configuration
block_patterns: ["ssn", "credit_card"]
mask_char: "*"
# Global plugin settings
plugin_settings:
parallel_execution_within_band: false
plugin_timeout: 30
fail_on_plugin_error: false
enable_plugin_api: true
plugin_health_check_interval: 60
The plugins
section lists the set of configured plugins that will be loaded by the Context Forge at startup. Each plugin contains a set of standard configurations, and then a config
section designed for plugin specific configurations. The attributes are defined as follows:
Attribute | Description | Example Value |
---|---|---|
name | A unique name for the plugin. | MyFirstPlugin |
kind | A fully qualified string representing the plugin python object. | plugins.native.content_filter.ContentFilterPlugin |
description | The description of the plugin configuration. | A plugin for replacing bad words. |
version | The version of the plugin configuration. | 0.1 |
author | The team that wrote the plugin. | MCP Context Forge |
hooks | A list of hooks for which the plugin will be executed. Supported hooks: "prompt_pre_fetch", "prompt_post_fetch", "tool_pre_invoke", "tool_post_invoke", "resource_pre_fetch", "resource_post_fetch" | ["prompt_pre_fetch", "prompt_post_fetch", "tool_pre_invoke", "tool_post_invoke", "resource_pre_fetch", "resource_post_fetch"] |
tags | Descriptive keywords that make the configuration searchable. | ["security", "filter"] |
mode | Mode of operation of the plugin. - enforce (stops during a violation), permissive (audits a violation but doesn't stop), disabled (disabled) | permissive |
priority | The priority in which the plugin will run - 0 is higher priority | 100 |
conditions | A list of conditions under which a plugin is run. See section on conditions. | |
config | Plugin specific configuration. This is a dictionary and is passed to the plugin on initialization. |
The plugin_settings
are as follows:
Attribute | Description | Example Value |
---|---|---|
parallel_execution_within_band | Plugins in the same band are run in parallel (currently not implemented). | true or false |
plugin_timeout | The time in seconds before stopping plugin execution (not implemented). | 30 |
fail_on_plugin_error | Cause the execution of the task to fail if the plugin errors. | true or false |
plugin_health_check_interval | Health check interval in seconds (not implemented). | 60 |
3. Execution Modes¶
Each plugin can operate in one of three modes:
Mode | Description | Use Case |
---|---|---|
enforce | Blocks requests on policy violations | Production guardrails |
permissive | Logs violations but allows requests | Testing and monitoring |
disabled | Plugin loaded but not executed | Temporary deactivation |
4. Priority and Execution Order¶
Plugins execute in priority order (ascending):
# Execution order example
plugins:
- name: "Authentication"
priority: 10 # Runs first
- name: "RateLimiter"
priority: 50 # Runs second
- name: "ContentFilter"
priority: 100 # Runs third
- name: "Logger"
priority: 200 # Runs last
Plugins with the same priority may execute in parallel if parallel_execution_within_band
is enabled.
5. Conditions of Execution¶
Users may only want plugins to be invoked on specific servers, tools, and prompts. To address this, a set of conditionals can be applied to a plugin. The attributes in a conditional combine together in as a set of and
operations, while each attribute list item is ored
with other items in the list. The attributes are defined as follows:
Attribute | Description |
---|---|
server_ids | The list of MCP servers on which the plugin will trigger |
tools | The list of tools on which the plugin will be applied. |
prompts | The list of prompts on which the plugin will be applied. |
resources | The list of resource URIs on which the plugin will be applied. |
user_patterns | The list of users on which the plugin will be applied. |
content_types | The list of content types on which the plugin will trigger. |
Available Hooks¶
The plugin framework provides comprehensive hook coverage across the entire MCP request lifecycle:
Production Hooks (Implemented)¶
Hook | Execution Point | Use Cases | Payload Type |
---|---|---|---|
prompt_pre_fetch | Before prompt template retrieval | Argument validation, PII scanning, input sanitization | PromptPrehookPayload |
prompt_post_fetch | After prompt template rendering | Content filtering, output transformation, safety checks | PromptPosthookPayload |
tool_pre_invoke | Before tool execution | Authorization, argument validation, dangerous operation blocking | ToolPreInvokePayload |
tool_post_invoke | After tool execution | Result filtering, PII masking, audit logging, response transformation | ToolPostInvokePayload |
resource_pre_fetch | Before resource fetching | URI validation, protocol checking, metadata injection | ResourcePreFetchPayload |
resource_post_fetch | After resource content retrieval | Content filtering, size validation, sensitive data redaction | ResourcePostFetchPayload |
Planned Hooks (Roadmap)¶
Hook | Purpose | Expected Release |
---|---|---|
server_pre_register | Server attestation and validation before admission | v0.7.0 |
server_post_register | Post-registration processing and setup | v0.7.0 |
auth_pre_check | Custom authentication logic integration | v0.7.0 |
auth_post_check | Post-authentication processing and enrichment | v0.7.0 |
federation_pre_sync | Gateway federation validation and filtering | v0.8.0 |
federation_post_sync | Post-federation data processing and reconciliation | v0.8.0 |
Tool Hooks Details¶
The tool hooks enable plugins to intercept and modify tool invocations:
tool_pre_invoke
: Receives the tool name and arguments before execution. Can modify arguments or block the invocation entirely.tool_post_invoke
: Receives the tool result after execution. Can modify the result or block it from being returned.
Example use cases: - PII detection and masking in tool inputs/outputs - Rate limiting specific tools - Audit logging of tool usage - Input validation and sanitization - Output filtering and transformation
Resource Hooks Details¶
The resource hooks enable plugins to intercept and modify resource fetching:
resource_pre_fetch
: Receives the resource URI and metadata before fetching. Can modify the URI, add metadata, or block the fetch entirely.resource_post_fetch
: Receives the resource content after fetching. Can modify the content, redact sensitive information, or block it from being returned.
Example use cases: - Protocol validation (block non-HTTPS resources) - Domain blocklisting/allowlisting - Content size limiting - Sensitive data redaction - Content transformation and filtering - Resource caching metadata
Planned hooks (not yet implemented):
server_pre_register
/server_post_register
- Server validationauth_pre_check
/auth_post_check
- Custom authenticationfederation_pre_sync
/federation_post_sync
- Gateway federation
Writing Plugins¶
Plugin Structure¶
from mcpgateway.plugins.framework import (
Plugin,
PluginConfig,
PluginContext,
PromptPrehookPayload,
PromptPrehookResult,
PromptPosthookPayload,
PromptPosthookResult,
ToolPreInvokePayload,
ToolPreInvokeResult,
ToolPostInvokePayload,
ToolPostInvokeResult,
ResourcePreFetchPayload,
ResourcePreFetchResult,
ResourcePostFetchPayload,
ResourcePostFetchResult
)
class MyPlugin(Plugin):
"""Example plugin implementation."""
def __init__(self, config: PluginConfig):
super().__init__(config)
# Initialize plugin-specific configuration
self.my_setting = config.config.get("my_setting", "default")
async def prompt_pre_fetch(
self,
payload: PromptPrehookPayload,
context: PluginContext
) -> PromptPrehookResult:
"""Process prompt before retrieval."""
# Access prompt name and arguments
prompt_name = payload.name
args = payload.args
# Example: Block requests with forbidden words
if "forbidden" in str(args.values()).lower():
return PromptPrehookResult(
continue_processing=False,
violation=PluginViolation(
plugin_name=self.name,
description="Forbidden content detected",
violation_code="FORBIDDEN_CONTENT",
details={"found_in": "arguments"}
)
)
# Example: Modify arguments
if "transform_me" in args:
args["transform_me"] = args["transform_me"].upper()
return PromptPrehookResult(
modified_payload=PromptPrehookPayload(prompt_name, args)
)
# Allow request to continue unmodified
return PromptPrehookResult()
async def prompt_post_fetch(
self,
payload: PromptPosthookPayload,
context: PluginContext
) -> PromptPosthookResult:
"""Process prompt after rendering."""
# Access rendered prompt
prompt_result = payload.result
# Example: Add metadata to context
context.metadata["processed_by"] = self.name
# Example: Modify response
for message in prompt_result.messages:
message.content.text = message.content.text.replace(
"old_text", "new_text"
)
return PromptPosthookResult(
modified_payload=payload
)
async def tool_pre_invoke(
self,
payload: ToolPreInvokePayload,
context: PluginContext
) -> ToolPreInvokeResult:
"""Process tool before invocation."""
# Access tool name and arguments
tool_name = payload.name
args = payload.args
# Example: Block dangerous operations
if tool_name == "file_delete" and "system" in str(args):
return ToolPreInvokeResult(
continue_processing=False,
violation=PluginViolation(
plugin_name=self.name,
description="Dangerous operation blocked",
violation_code="DANGEROUS_OP",
details={"tool": tool_name}
)
)
# Example: Modify arguments
if "sanitize_me" in args:
args["sanitize_me"] = self.sanitize_input(args["sanitize_me"])
return ToolPreInvokeResult(
modified_payload=ToolPreInvokePayload(tool_name, args)
)
return ToolPreInvokeResult()
async def tool_post_invoke(
self,
payload: ToolPostInvokePayload,
context: PluginContext
) -> ToolPostInvokeResult:
"""Process tool after invocation."""
# Access tool result
tool_name = payload.name
result = payload.result
# Example: Filter sensitive data from results
if isinstance(result, dict) and "sensitive_data" in result:
result["sensitive_data"] = "[REDACTED]"
return ToolPostInvokeResult(
modified_payload=ToolPostInvokePayload(tool_name, result)
)
# Example: Add audit metadata
context.metadata["tool_executed"] = tool_name
context.metadata["execution_time"] = time.time()
return ToolPostInvokeResult()
async def resource_pre_fetch(
self,
payload: ResourcePreFetchPayload,
context: PluginContext
) -> ResourcePreFetchResult:
"""Process resource before fetching."""
# Access resource URI and metadata
uri = payload.uri
metadata = payload.metadata
# Example: Block certain protocols
from urllib.parse import urlparse
parsed = urlparse(uri)
if parsed.scheme not in ["http", "https", "file"]:
return ResourcePreFetchResult(
continue_processing=False,
violation=PluginViolation(
plugin_name=self.name,
description=f"Protocol {parsed.scheme} not allowed",
violation_code="PROTOCOL_BLOCKED",
details={"uri": uri, "protocol": parsed.scheme}
)
)
# Example: Add metadata
metadata["validated_by"] = self.name
return ResourcePreFetchResult(
modified_payload=ResourcePreFetchPayload(uri, metadata)
)
async def resource_post_fetch(
self,
payload: ResourcePostFetchPayload,
context: PluginContext
) -> ResourcePostFetchResult:
"""Process resource after fetching."""
# Access resource content
uri = payload.uri
content = payload.content
# Example: Redact sensitive patterns from text content
if hasattr(content, 'text') and content.text:
# Redact email addresses
import re
content.text = re.sub(
r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
'[EMAIL_REDACTED]',
content.text
)
return ResourcePostFetchResult(
modified_payload=ResourcePostFetchPayload(uri, content)
)
async def shutdown(self):
"""Cleanup when plugin shuts down."""
# Close connections, save state, etc.
pass
Plugin Context and State¶
Plugins can maintain state between pre/post hooks:
async def prompt_pre_fetch(self, payload, context):
# Store state for later use
context.set_state("request_time", time.time())
context.set_state("original_args", payload.args.copy())
return PromptPrehookResult()
async def prompt_post_fetch(self, payload, context):
# Retrieve state from pre-hook
elapsed = time.time() - context.get_state("request_time", 0)
original = context.get_state("original_args", {})
# Add timing metadata
context.metadata["processing_time_ms"] = elapsed * 1000
return PromptPosthookResult()
External Service Plugin Example¶
class LLMGuardPlugin(Plugin):
"""Example external service integration."""
def __init__(self, config: PluginConfig):
super().__init__(config)
self.service_url = config.config.get("service_url")
self.api_key = config.config.get("api_key")
self.timeout = config.config.get("timeout", 30)
async def prompt_pre_fetch(self, payload, context):
# Call external service
async with httpx.AsyncClient() as client:
try:
response = await client.post(
f"{self.service_url}/analyze",
json={
"text": str(payload.args),
"policy": "strict"
},
headers={
"Authorization": f"Bearer {self.api_key}"
},
timeout=self.timeout
)
result = response.json()
if result.get("blocked", False):
return PromptPrehookResult(
continue_processing=False,
violation=PluginViolation(
plugin_name=self.name,
description=result.get("reason", "Content blocked"),
violation_code="LLMGUARD_BLOCKED",
details=result
)
)
except Exception as e:
# Handle errors based on plugin settings
if self.config.mode == PluginMode.ENFORCE:
return PromptPrehookResult(
continue_processing=False,
violation=PluginViolation(
plugin_name=self.name,
description=f"Service error: {str(e)}",
violation_code="SERVICE_ERROR",
details={"error": str(e)}
)
)
return PromptPrehookResult()
Plugin Development Guide¶
1. Create Plugin Directory¶
mkdir -p plugins/my_plugin
touch plugins/my_plugin/__init__.py
touch plugins/my_plugin/plugin.py
touch plugins/my_plugin/plugin-manifest.yaml
2. Write Plugin Manifest¶
# plugins/my_plugin/plugin-manifest.yaml
description: "My custom plugin for X"
author: "Your Name"
version: "1.0.0"
tags: ["custom", "filter"]
available_hooks:
- "prompt_pre_fetch"
- "prompt_post_fetch"
default_config:
setting_one: "default_value"
setting_two: 123
3. Implement Plugin Class¶
# plugins/my_plugin/plugin.py
from mcpgateway.plugins.framework import Plugin
class MyPlugin(Plugin):
# Implementation here
pass
4. Register in Configuration¶
# plugins/config.yaml
plugins:
- name: "MyCustomPlugin"
kind: "plugins.my_plugin.plugin.MyPlugin"
hooks: ["prompt_pre_fetch"]
# ... other configuration
5. Test Your Plugin¶
# tests/test_my_plugin.py
import pytest
from plugins.my_plugin.plugin import MyPlugin
from mcpgateway.plugins.framework import PluginConfig
@pytest.mark.asyncio
async def test_my_plugin():
config = PluginConfig(
name="test",
kind="plugins.my_plugin.plugin.MyPlugin",
hooks=["prompt_pre_fetch"],
config={"setting_one": "test_value"}
)
plugin = MyPlugin(config)
# Test your plugin logic
result = await plugin.prompt_pre_fetch(payload, context)
assert result.continue_processing
Best Practices¶
1. Error Handling¶
Always handle errors gracefully:
async def prompt_pre_fetch(self, payload, context):
try:
# Plugin logic
pass
except Exception as e:
logger.error(f"Plugin {self.name} error: {e}")
# In permissive mode, log and continue
if self.mode == PluginMode.PERMISSIVE:
return PromptPrehookResult()
# In enforce mode, block the request
return PromptPrehookResult(
continue_processing=False,
violation=PluginViolation(
plugin_name=self.name,
description="Plugin error occurred",
violation_code="PLUGIN_ERROR",
details={"error": str(e)}
)
)
2. Performance Considerations¶
- Keep plugin operations lightweight
- Use caching for expensive operations
- Respect the configured timeout
- Consider async operations for I/O
class CachedPlugin(Plugin):
def __init__(self, config):
super().__init__(config)
self._cache = {}
self._cache_ttl = config.config.get("cache_ttl", 300)
async def expensive_operation(self, key):
# Check cache first
if key in self._cache:
cached_value, timestamp = self._cache[key]
if time.time() - timestamp < self._cache_ttl:
return cached_value
# Perform expensive operation
result = await self._do_expensive_work(key)
# Cache result
self._cache[key] = (result, time.time())
return result
3. Conditional Execution¶
Use conditions to limit plugin scope:
conditions:
- prompts: ["sensitive_prompt"]
server_ids: ["prod-server-1", "prod-server-2"]
tenant_ids: ["enterprise-tenant"]
user_patterns: ["admin-*", "support-*"]
4. Logging and Monitoring¶
Use appropriate log levels:
logger.debug(f"Plugin {self.name} processing prompt: {payload.name}")
logger.info(f"Plugin {self.name} blocked request: {violation_code}")
logger.warning(f"Plugin {self.name} timeout approaching")
logger.error(f"Plugin {self.name} failed: {error}")
API Reference¶
Plugin Management Endpoints¶
Endpoint | Method | Description |
---|---|---|
/plugins | GET | List all loaded plugins |
/plugins/stats | GET | Get plugin execution statistics |
/plugins/reload/{name} | POST | Reload a specific plugin |
/plugins/stats/reset | POST | Reset plugin statistics |
Example API Usage¶
# List plugins
curl http://localhost:8000/plugins
# Response
[
{
"name": "ContentFilter",
"priority": 100,
"mode": "enforce",
"hooks": ["prompt_pre_fetch", "prompt_post_fetch"],
"tags": ["security", "filter"],
"conditions": {
"prompts": ["customer_chat"]
}
}
]
Troubleshooting¶
Plugin Not Loading¶
- Check server logs for initialization errors
- Verify plugin class path in configuration
- Ensure all dependencies are installed
- Check Python import path includes plugin directory
Plugin Not Executing¶
- Verify plugin is enabled (
mode
!= "disabled") - Check conditions match your request
- Review priority ordering
- Enable debug logging to see execution flow
Performance Issues¶
- Monitor plugin execution time in logs
- Check for blocking I/O operations
- Review timeout settings
- Consider caching expensive operations
Production Deployment Examples¶
Enterprise AI Safety Pipeline¶
# Production-grade AI safety configuration
plugins:
# Step 1: PII Detection and Masking (Highest Priority)
- name: "PIIFilter"
kind: "plugins.pii_filter.pii_filter.PIIFilterPlugin"
hooks: ["prompt_pre_fetch", "prompt_post_fetch", "tool_pre_invoke", "tool_post_invoke"]
mode: "enforce"
priority: 10
config:
detect_ssn: true
detect_credit_card: true
detect_email: true
mask_strategy: "partial"
block_on_detection: false
# Step 2: External AI Safety Service (LlamaGuard)
- name: "LlamaGuardSafety"
kind: "external"
hooks: ["prompt_pre_fetch", "tool_pre_invoke"]
mode: "enforce"
priority: 20
mcp:
proto: STREAMABLEHTTP
url: "https://ai-safety.internal.corp/llamaguard/v1"
conditions:
- server_ids: ["production-chat", "customer-support"]
# Step 3: OpenAI Moderation for Final Check
- name: "OpenAIMod"
kind: "external"
hooks: ["prompt_post_fetch", "tool_post_invoke"]
mode: "permissive" # Log violations but don't block
priority: 30
mcp:
proto: STREAMABLEHTTP
url: "https://api.openai.com/v1/moderations"
# Step 4: Audit Logging (Lowest Priority)
- name: "AuditLogger"
kind: "plugins.audit.audit_logger.AuditLoggerPlugin"
hooks: ["prompt_pre_fetch", "tool_pre_invoke", "tool_post_invoke"]
mode: "permissive"
priority: 100
config:
log_level: "INFO"
include_payloads: false # For privacy
audit_endpoints: ["https://audit.internal.corp/api/v1/logs"]
Multi-Tenant Security Configuration¶
plugins:
# Enterprise tenant gets strict filtering
- name: "EnterpriseSecurityFilter"
kind: "plugins.security.enterprise_filter.EnterpriseFilterPlugin"
hooks: ["prompt_pre_fetch", "tool_pre_invoke"]
mode: "enforce"
priority: 50
conditions:
- tenant_ids: ["enterprise-corp", "banking-client"]
tools: ["database-query", "file-access", "system-command"]
config:
sql_injection_protection: true
command_injection_protection: true
file_system_restrictions: true
# Free tier gets basic content filtering
- name: "BasicContentFilter"
kind: "plugins.content.basic_filter.BasicFilterPlugin"
hooks: ["prompt_pre_fetch", "prompt_post_fetch"]
mode: "permissive"
priority: 75
conditions:
- tenant_ids: ["free-tier"]
config:
profanity_filter: true
spam_detection: true
rate_limit_warnings: true
Development vs Production Configurations¶
# Development Environment
plugins:
- name: "DevPIIFilter"
kind: "plugins.pii_filter.pii_filter.PIIFilterPlugin"
hooks: ["prompt_pre_fetch", "tool_pre_invoke"]
mode: "permissive" # Don't block in dev
priority: 50
config:
detect_ssn: true
log_detections: true
mask_strategy: "partial"
whitelist_patterns:
- "test@example.com"
- "555-555-5555"
- "123-45-6789" # Test SSN
# Production Environment
plugins:
- name: "ProdPIIFilter"
kind: "plugins.pii_filter.pii_filter.PIIFilterPlugin"
hooks: ["prompt_pre_fetch", "prompt_post_fetch", "tool_pre_invoke", "tool_post_invoke"]
mode: "enforce" # Block in production
priority: 10
config:
detect_ssn: true
detect_credit_card: true
detect_phone: true
detect_email: true
detect_api_keys: true
block_on_detection: true
audit_detections: true
compliance_mode: "strict"
Performance and Scalability¶
Benchmark Results¶
- Self-Contained Plugins: <1ms latency overhead per hook
- External Service Plugins: 10-100ms depending on service (cached responses: <5ms)
- Memory Usage: ~5MB base overhead + ~1MB per active plugin
- Throughput: Tested to 1,000+ req/s with 5 active plugins
Performance Optimization Tips¶
# Optimize plugin configuration for high-throughput environments
plugin_settings:
plugin_timeout: 5000 # 5 second timeout for external services
parallel_execution_within_band: true # Enable when available
fail_on_plugin_error: false # Continue processing on plugin failures
plugins:
- name: "CachedAIService"
kind: "external"
priority: 50
config:
cache_ttl_seconds: 300 # Cache responses for 5 minutes
cache_max_entries: 10000 # LRU cache with 10K entries
timeout_ms: 2000 # Fast timeout for high-throughput
retry_attempts: 1 # Single retry only
Monitoring and Observability¶
Plugin Metrics¶
The framework exposes comprehensive metrics for monitoring:
# Plugin execution metrics
mcpgateway_plugin_executions_total{plugin="PIIFilter",hook="prompt_pre_fetch",status="success"}
mcpgateway_plugin_duration_seconds{plugin="PIIFilter",hook="prompt_pre_fetch"}
mcpgateway_plugin_violations_total{plugin="PIIFilter",violation_code="PII_DETECTED"}
mcpgateway_plugin_errors_total{plugin="LlamaGuard",error_type="timeout"}
# Context and memory metrics
mcpgateway_plugin_contexts_active
mcpgateway_plugin_contexts_cleaned_total
mcpgateway_plugin_memory_usage_bytes
Health Check Integration¶
plugins:
- name: "ExternalAIService"
kind: "external"
mcp:
proto: STREAMABLEHTTP
url: "https://ai-service.corp/api/v1"
health_check_endpoint: "/health"
health_check_interval: 30
config:
circuit_breaker_enabled: true
circuit_breaker_failure_threshold: 5
circuit_breaker_timeout: 60
Security Considerations¶
Plugin Isolation and Security¶
- Input Validation: All plugin configurations validated against JSON schemas
- Timeout Protection: Configurable timeouts prevent plugin hangs
- Resource Limits: Memory and payload size limits prevent resource exhaustion
- Error Isolation: Plugin failures don't affect gateway stability
- Audit Logging: Complete audit trail of plugin executions and violations
External Service Security¶
# Secure external service configuration
plugins:
- name: "SecureExternalService"
kind: "external"
mcp:
proto: STREAMABLEHTTP
url: "https://secure-ai-service.corp/api/v1"
tls_verify: true
tls_client_cert: "/etc/ssl/certs/client.crt"
tls_client_key: "/etc/ssl/private/client.key"
auth:
type: "bearer"
token: "${AI_SERVICE_TOKEN}" # Environment variable
config:
allowed_response_codes: [200, 201]
max_response_size_mb: 10
connection_pool_size: 20
Future Roadmap¶
Near-term Enhancements (v0.7.0)¶
- Server Attestation Hooks:
server_pre_register
with TPM/TEE support - Authentication Hooks:
auth_pre_check
/auth_post_check
for custom auth - Admin UI: Visual plugin management and monitoring dashboard
- Hot Configuration Reload: Update plugin configs without restart
- Advanced Caching: Redis-backed caching for external service calls
Long-term Vision (v0.8.0+)¶
- Plugin Marketplace: Community plugin sharing and discovery
- Advanced Analytics: Plugin performance analytics and optimization recommendations
- A/B Testing Framework: Split traffic between plugin configurations
- Policy as Code: Integration with Open Policy Agent (OPA) for complex rule evaluation
- Machine Learning Pipeline: Built-in support for custom ML model deployment
Contributing¶
To contribute a plugin:
- Follow the plugin structure guidelines
- Include comprehensive tests
- Document configuration options
- Submit a pull request with examples
For framework improvements, please open an issue to discuss proposed changes.