CLI Reference

This section provides a comprehensive reference for the ARES command-line interface (CLI).

Overview

The ARES CLI provides commands for running evaluations, viewing available modules, and managing configurations.

Main Commands

evaluate

Execute a red-teaming evaluation using a configuration file.

Syntax:

ares evaluate <config_file> [OPTIONS]

Arguments:

config_file: Path to the YAML configuration file (required)

Options:

Option	Type	Description
`--limit`, `-l`	Flag	Run ARES in test mode with limited number of seeds (typically 5). Useful for quick validation.
`--first N`, `-n N`	Integer	Use only the first N seeds from the provided source. Allows precise control over evaluation size.
`--dashboard`	Flag	Launch the interactive dashboard after evaluation completes. Opens automatically in browser.
`-v, --verbose`	Flag	Show detailed configuration information during execution. Displays resolved configs and parameters.

Examples:

Basic evaluation:

ares evaluate example_configs/minimal.yaml

Test mode with limited seeds:

ares evaluate example_configs/minimal.yaml --limit
ares evaluate example_configs/minimal.yaml -l

Use first 10 seeds only:

ares evaluate example_configs/minimal.yaml --limit --first 10
ares evaluate example_configs/minimal.yaml -l -n 10

Run with dashboard:

ares evaluate example_configs/minimal.yaml --dashboard

Verbose mode for debugging:

ares evaluate example_configs/minimal.yaml --verbose
ares evaluate example_configs/minimal.yaml -v

Combined options:

ares evaluate example_configs/minimal.yaml -l -n 5 --dashboard --verbose
ares evaluate example_configs/minimal.yaml -l -n 5 --dashboard -v

show-chat

Display evaluation results in chat-like HTML format.

Syntax:

ares show-chat --file <evaluation_file> [OPTIONS]

Arguments:

--file, -f: Path to the evaluation JSON file (required)

Options:

Option	Type	Description
`--output`, `-o`	Path	Output HTML file path. Default: `<input>_visualization.html`
`--max-items`, `-m`	Integer	Maximum number of items to display. Useful for large evaluation files.
`--evaluator`, `-e`	String	Name of the evaluator (e.g., ‘keyword’, ‘conversation_eval’). Provides better context.
`--open`	Flag	Open the generated HTML file in the default browser automatically.

Examples:

Basic chat view:

ares show-chat --file results/evaluation.json
ares show-chat -f results/evaluation.json

Custom output path:

ares show-chat -f results/evaluation.json -o results/chat.html

Limit to first 10 items:

ares show-chat -f results/evaluation.json --max-items 10
ares show-chat -f results/evaluation.json -m 10

Open in browser automatically:

ares show-chat -f results/evaluation.json --open

Specify evaluator name:

ares show-chat -f results/llm_eval_evaluation.json -e llm_eval

Combined options:

ares show-chat -f results/evaluation.json -o chat.html -m 20 --open

Output:

The show-chat command creates an interactive HTML page displaying:

Conversations in a chat-like format
Evaluation metrics and scores
Attack success indicators
Goal and strategy information
Turn-by-turn analysis for multi-turn conversations

Supported Evaluation Types:

Single-turn responses
Multi-turn conversations
Goal-level aggregated evaluations
Conversation-level evaluations

show

Display information about available ARES modules and configurations.

Subcommands:

show modules

List all available ARES modules (goals, strategies, evaluators, connectors).

ares show modules

Output: Displays a comprehensive list of all registered modules in ARES.

show connectors

List all available connector implementations.

ares show connectors

With name filter:

ares show connectors -n huggingface

Output: Shows connector details including type, configuration parameters, and usage examples.

show goals

List all available goal implementations.

ares show goals

With name filter:

ares show goals -n generic_attack_goal

Output: Displays goal types, their purposes, and configuration templates.

show strategies

List all available attack strategy implementations.

ares show strategies

With name filter:

ares show strategies -n direct_requests
ares show strategies -n crescendo

Output: Shows strategy details including attack methods, parameters, and example configurations.

show evals

List all available evaluator implementations.

ares show evals

With name filter:

ares show evals -n keyword
ares show evals -n harmbench_eval

Output: Displays evaluator types, evaluation methods, and configuration examples.

Common Usage Patterns

Quick Validation

Test your configuration with a small subset of data:

ares evaluate my_config.yaml --limit --verbose

This runs with ~5 seeds and shows detailed output for debugging.

Production Evaluation

Run a full evaluation with results dashboard:

ares evaluate my_config.yaml --dashboard

Controlled Testing

Test with specific number of seeds:

ares evaluate my_config.yaml --limit --first 20 --verbose
ares evaluate my_config.yaml -l -n 20 -v

Exploring Available Options

Before creating a configuration, explore available modules:

ares show modules
ares show strategies
ares show evals
ares show connectors

Get specific module details:

ares show strategies -n crescendo
ares show connectors -n huggingface

Viewing Results in Chat Format

After running an evaluation, view results in chat format:

# Quick chat view
ares show-chat -f results/evaluation.json --open

# For large datasets, limit items
ares show-chat -f results/evaluation.json -m 50 --open

# Custom output location
ares show-chat -f results/evaluation.json -o reports/chat.html

Environment Variables

ARES_HOME

Set the base directory for ARES file resolution.

Usage:

export ARES_HOME=/path/to/ares/directory
ares evaluate config.yaml

Behavior:

ARES searches for configuration files relative to ARES_HOME
connectors.yaml is loaded from the same directory as the config file
If not found, searches up to ARES_HOME directory
Default: Current working directory

Example:

# Set ARES_HOME to project root
export ARES_HOME=/home/user/ares-project

# Run evaluation - files resolved relative to ARES_HOME
ares evaluate configs/my_eval.yaml

Connector-Specific Variables

Different connectors require specific environment variables. These should be set in a .env file or exported in your shell.

HuggingFace:

# Login via CLI (for gated models)
huggingface-cli login

WatsonX:

export WATSONX_URL=https://your-watsonx-instance.com
export WATSONX_API_KEY=your-api-key
export WATSONX_PROJECT_ID=your-project-id

WatsonX AgentLab:

export WATSONX_AGENTLAB_API_KEY=your-agent-api-key

OpenAI (via LiteLLM):

export OPENAI_API_KEY=your-openai-key

Exit Codes

The ARES CLI uses standard exit codes:

0: Success
1: General error (configuration error, file not found, etc.)
2: Invalid command or arguments

Troubleshooting

Configuration Not Found

Error: FileNotFoundError: config.yaml not found

Solution:

Check file path is correct
Use absolute path or set ARES_HOME
Verify file exists: ls -la config.yaml

Module Not Found

Error: ModuleNotFoundError: No module named 'ares_plugin'

Solution:

Install the required plugin: pip install ares-plugin
Verify installation: pip list | grep ares
Check plugin is in correct location

Invalid Configuration

Error: KeyError: 'target' or Invalid configuration

Solution:

Use --verbose to see full config
Validate YAML syntax
Check required fields are present
Use ares show modules to verify module names

Connector Errors

Error: OSError: API key not found

Solution:

Check environment variables are set
Verify .env file is in correct location
For HuggingFace: run huggingface-cli login
Check API key permissions

Getting Help

For additional help:

ares --help
ares evaluate --help
ares show --help

Visit the ARES Documentation for detailed guides and examples.

Report issues on GitHub.