CLI Reference

This section provides a comprehensive reference for the ARES command-line interface (CLI).

Overview

The ARES CLI provides commands for running evaluations, viewing available modules, and managing configurations.

Main Commands

evaluate

Execute a red-teaming evaluation using a configuration file.

Syntax:

ares evaluate <config_file> [OPTIONS]

Arguments:

config_file: Path to the YAML configuration file (required)

Options:

Option	Type	Description
`--limit`, `-l`	Flag	Run ARES in test mode with limited number of seeds (typically 5). Useful for quick validation.
`--first N`, `-n N`	Integer	Use only the first N seeds from the provided source. Allows precise control over evaluation size.
`--run-tagged`, `-r`	Flag	Save all output files in a tagged subfolder under `report_path`. Prevents results from being overwritten across runs.
`--target JSON`, `-t JSON`	String	Override the target connector at runtime with a JSON dict.
`--test`, `-c`	Flag	Print the resolved configuration and exit without running.
`--generate-only`	Flag	Stop after the generate step — builds crafted attack prompts and saves them to disk, but does not send them to the target.
`--evaluate-only`	Flag	Skip prompt generation and target attacks; run evaluations against existing attack result files.
`--csv`	Flag	Write a CSV report alongside each evaluator’s JSON output.
`--dashboard`	Flag	Launch the interactive dashboard after evaluation completes. Opens automatically in browser.
`-v, --verbose`	Flag	Show detailed configuration information during execution. Displays resolved configs and parameters.

Examples:

Basic evaluation:

ares evaluate example_configs/minimal.yaml

Test mode with limited seeds:

ares evaluate example_configs/minimal.yaml --limit
ares evaluate example_configs/minimal.yaml -l

Use first 10 seeds only:

ares evaluate example_configs/minimal.yaml --limit --first 10
ares evaluate example_configs/minimal.yaml -l -n 10

Run with dashboard:

ares evaluate example_configs/minimal.yaml --dashboard

Verbose mode for debugging:

ares evaluate example_configs/minimal.yaml --verbose
ares evaluate example_configs/minimal.yaml -v

Combined options:

ares evaluate example_configs/minimal.yaml -l -n 5 --dashboard --verbose
ares evaluate example_configs/minimal.yaml -l -n 5 --dashboard -v

Inspect generated prompts before attacking:

ares evaluate example_configs/minimal.yaml --generate-only

Re-evaluate existing attack results:

ares evaluate example_configs/minimal.yaml --evaluate-only
ares evaluate example_configs/minimal.yaml --evaluate-only -r

Two-Phase Workflow

ARES runs a three-phase pipeline for each strategy:

generate() — craft attack prompts → saved to strategy-<name>_generate.json
attack() — send prompts to the target → saved to strategy-<name>.json
evaluate() — score responses → saved to evaluation-<eval>.json

--generate-only stops after phase 1 (useful for auditing prompts before they reach the target). --evaluate-only starts at phase 3 (useful for re-running evaluations against already-collected attack results, e.g. after adding a new evaluator).

When --evaluate-only is combined with -r, ARES automatically locates the most recently created numbered run folder and evaluates the strategy files inside it.

Note

--evaluate-only and --generate-only are mutually exclusive. If both are passed, --evaluate-only takes precedence and a warning is logged.

show-chat

Display evaluation results in chat-like HTML format.

Syntax:

ares show-chat --file <evaluation_file> [OPTIONS]

Arguments:

--file, -f: Path to the evaluation JSON file (required)

Options:

Option	Type	Description
`--output`, `-o`	Path	Output HTML file path. Default: `<input>_visualization.html`
`--max-items`, `-m`	Integer	Maximum number of items to display. Useful for large evaluation files.
`--evaluator`, `-e`	String	Name of the evaluator (e.g., ‘keyword’, ‘conversation_eval’). Provides better context.
`--open`	Flag	Open the generated HTML file in the default browser automatically.

Examples:

Basic chat view:

ares show-chat --file results/evaluation.json
ares show-chat -f results/evaluation.json

Custom output path:

ares show-chat -f results/evaluation.json -o results/chat.html

Limit to first 10 items:

ares show-chat -f results/evaluation.json --max-items 10
ares show-chat -f results/evaluation.json -m 10

Open in browser automatically:

ares show-chat -f results/evaluation.json --open

Specify evaluator name:

ares show-chat -f results/llm_eval_evaluation.json -e llm_eval

Combined options:

ares show-chat -f results/evaluation.json -o chat.html -m 20 --open

Output:

The show-chat command creates an interactive HTML page displaying:

Conversations in a chat-like format
Evaluation metrics and scores
Attack success indicators
Goal and strategy information
Turn-by-turn analysis for multi-turn conversations

Supported Evaluation Types:

Single-turn responses
Multi-turn conversations
Goal-level aggregated evaluations
Conversation-level evaluations

show-report

Display a report for a given configuration without re-running the evaluation.

Syntax:

ares show-report <config_file> [OPTIONS]

Arguments:

config_file: Path to the YAML configuration file (required)

Options:

Option	Type	Description
`--dashboard`	Flag	Launch the interactive dashboard instead of printing the tabular report to the console.

Examples:

Print report to console:

ares show-report example_configs/minimal.yaml

Open interactive dashboard:

ares show-report example_configs/minimal.yaml --dashboard

install-plugin

Install an ARES plugin directly from the repository.

Syntax:

ares install-plugin <plugin_name>

Arguments:

plugin_name: Name of the plugin to install (e.g. ares-litellm-connector, ares-autodan, ares-pyrit, ares-human-jailbreak)

Examples:

ares install-plugin ares-litellm-connector
ares install-plugin ares-autodan
ares install-plugin ares-pyrit
ares install-plugin ares-human-jailbreak

gui

Launch the ARES graphical user interface in the default browser.

Syntax:

ares gui [OPTIONS]

Options:

Option	Type	Description
`--port`, `-p`	Integer	Port number for the web server (default: `8081`).

Examples:

ares gui
ares gui --port 8080
ares gui -p 9090

evaluate-only

Run evaluations against existing attack result files, skipping goal generation and target attacks. Reads strategy-<name>.json files from the configured (or specified) output folder, runs all configured evaluators, and writes the report.

Syntax:

ares evaluate-only --config <config_file> [OPTIONS]

Options:

Option	Type	Description
`--config`, `-c`	Path	Path to the ARES YAML configuration file (required).
`--output`, `-o`	Path	Path to a folder containing existing strategy result files. When omitted, paths from the YAML are used.
`--verbose`, `-v`	Flag	Show detailed configuration information during execution.

Examples:

Re-evaluate using paths from YAML:

ares evaluate-only --config example_configs/minimal.yaml

Re-evaluate against a specific results folder:

ares evaluate-only --config example_configs/minimal.yaml --output results/my-run_00

Note

evaluate-only is a standalone command. It is distinct from the --evaluate-only flag of ares evaluate, which skips goal/strategy steps but still runs within the normal evaluate flow.

show

Display information about available ARES modules and configurations.

Subcommands:

show modules

List all available ARES modules (goals, strategies, evaluators, connectors).

ares show modules

Output: Displays a comprehensive list of all registered modules in ARES.

show connectors

List all available connector implementations.

ares show connectors

With name filter:

ares show connectors -n huggingface

Output: Shows connector details including type, configuration parameters, and usage examples.

show goals

List all available goal implementations.

ares show goals

With name filter:

ares show goals -n generic_attack_goal

Output: Displays goal types, their purposes, and configuration templates.

show strategies

List all available attack strategy implementations.

ares show strategies

With name filter:

ares show strategies -n direct_requests
ares show strategies -n crescendo

Output: Shows strategy details including attack methods, parameters, and example configurations.

show evals

List all available evaluator implementations.

ares show evals

With name filter:

ares show evals -n keyword
ares show evals -n harmbench_eval

Output: Displays evaluator types, evaluation methods, and configuration examples.

Common Usage Patterns

Quick Validation

Test your configuration with a small subset of data:

ares evaluate my_config.yaml --limit --verbose

This runs with ~5 seeds and shows detailed output for debugging.

Production Evaluation

Run a full evaluation with results dashboard:

ares evaluate my_config.yaml --dashboard

Controlled Testing

Test with specific number of seeds:

ares evaluate my_config.yaml --limit --first 20 --verbose
ares evaluate my_config.yaml -l -n 20 -v

Exploring Available Options

Before creating a configuration, explore available modules:

ares show modules
ares show strategies
ares show evals
ares show connectors

Get specific module details:

ares show strategies -n crescendo
ares show connectors -n huggingface

Viewing Results in Chat Format

After running an evaluation, view results in chat format:

# Quick chat view
ares show-chat -f results/evaluation.json --open

# For large datasets, limit items
ares show-chat -f results/evaluation.json -m 50 --open

# Custom output location
ares show-chat -f results/evaluation.json -o reports/chat.html

Environment Variables

ARES_HOME

Set the base directory for ARES file resolution.

Usage:

export ARES_HOME=/path/to/ares/directory
ares evaluate config.yaml

Behavior:

ARES searches for configuration files relative to ARES_HOME
connectors.yaml is loaded from the same directory as the config file
If not found, searches up to ARES_HOME directory
Default: Current working directory

Example:

# Set ARES_HOME to project root
export ARES_HOME=/home/user/ares-project

# Run evaluation - files resolved relative to ARES_HOME
ares evaluate configs/my_eval.yaml

Connector-Specific Variables

Different connectors require specific environment variables. These should be set in a .env file or exported in your shell.

HuggingFace:

# Login via CLI (for gated models)
huggingface-cli login

WatsonX:

export WATSONX_URL=https://your-watsonx-instance.com
export WATSONX_API_KEY=your-api-key
export WATSONX_PROJECT_ID=your-project-id

WatsonX AgentLab:

export WATSONX_AGENTLAB_API_KEY=your-agent-api-key

OpenAI (via LiteLLM):

export OPENAI_API_KEY=your-openai-key

Exit Codes

The ARES CLI uses the following exit codes:

0: Success — evaluation completed normally.
1: Error — configuration error, file not found, invalid JSON target override, or unhandled runtime exception.

Note

Exit code 2 is not used by ARES. Commands such as evaluate-only exit silently (code 0) when a recoverable error occurs (e.g. missing config file). Check the log output for ERROR entries in those cases.

Troubleshooting

Configuration Not Found

Error: FileNotFoundError: config.yaml not found

Solution:

Check file path is correct
Use absolute path or set ARES_HOME
Verify file exists: ls -la config.yaml

Module Not Found

Error: ModuleNotFoundError: No module named 'ares_plugin'

Solution:

Install the required plugin: pip install ares-plugin
Verify installation: pip list | grep ares
Check plugin is in correct location

Invalid Configuration

Error: KeyError: 'target' or Invalid configuration

Solution:

Use --verbose to see full config
Validate YAML syntax
Check required fields are present
Use ares show modules to verify module names

Connector Errors

Error: OSError: API key not found

Solution:

Check environment variables are set
Verify .env file is in correct location
For HuggingFace: run huggingface-cli login
Check API key permissions

Getting Help

For additional help:

ares --help
ares evaluate --help
ares show --help

Visit the ARES Documentation for detailed guides and examples.

Report issues on GitHub.