CLI Reference

This section provides a comprehensive reference for the ARES command-line interface (CLI).

Overview

The ARES CLI provides commands for running evaluations, viewing available modules, and managing configurations.

Main Commands

evaluate

Execute a red-teaming evaluation using a configuration file.

Syntax:

ares evaluate <config_file> [OPTIONS]

Arguments:

  • config_file: Path to the YAML configuration file (required)

Options:

Option

Type

Description

--limit

Flag

Run ARES in test mode with limited number of seeds (typically 5). Useful for quick validation.

--first N

Integer

Use only the first N seeds from the provided source. Allows precise control over evaluation size.

--dashboard

Flag

Launch the interactive dashboard after evaluation completes. Opens automatically in browser.

-v, --verbose

Flag

Show detailed configuration information during execution. Displays resolved configs and parameters.

Examples:

Basic evaluation:

ares evaluate example_configs/minimal.yaml

Test mode with limited seeds:

ares evaluate example_configs/minimal.yaml --limit

Use first 10 seeds only:

ares evaluate example_configs/minimal.yaml --first 10

Run with dashboard:

ares evaluate example_configs/minimal.yaml --dashboard

Verbose mode for debugging:

ares evaluate example_configs/minimal.yaml --verbose
ares evaluate example_configs/minimal.yaml -v

Combined options:

ares evaluate example_configs/minimal.yaml --first 5 --dashboard --verbose

show

Display information about available ARES modules and configurations.

Subcommands:

show modules

List all available ARES modules (goals, strategies, evaluators, connectors).

ares show modules

Output: Displays a comprehensive list of all registered modules in ARES.

show connectors

List all available connector implementations.

ares show connectors

With name filter:

ares show connectors -n huggingface

Output: Shows connector details including type, configuration parameters, and usage examples.

show goals

List all available goal implementations.

ares show goals

With name filter:

ares show goals -n generic_attack_goal

Output: Displays goal types, their purposes, and configuration templates.

show strategies

List all available attack strategy implementations.

ares show strategies

With name filter:

ares show strategies -n direct_requests
ares show strategies -n crescendo

Output: Shows strategy details including attack methods, parameters, and example configurations.

show evals

List all available evaluator implementations.

ares show evals

With name filter:

ares show evals -n keyword
ares show evals -n harmbench_eval

Output: Displays evaluator types, evaluation methods, and configuration examples.

Common Usage Patterns

Quick Validation

Test your configuration with a small subset of data:

ares evaluate my_config.yaml --limit --verbose

This runs with ~5 seeds and shows detailed output for debugging.

Production Evaluation

Run a full evaluation with results dashboard:

ares evaluate my_config.yaml --dashboard

Controlled Testing

Test with specific number of seeds:

ares evaluate my_config.yaml --first 20 --verbose

Exploring Available Options

Before creating a configuration, explore available modules:

ares show modules
ares show strategies
ares show evals
ares show connectors

Get specific module details:

ares show strategies -n crescendo
ares show connectors -n huggingface

Environment Variables

ARES_HOME

Set the base directory for ARES file resolution.

Usage:

export ARES_HOME=/path/to/ares/directory
ares evaluate config.yaml

Behavior:

  • ARES searches for configuration files relative to ARES_HOME

  • connectors.yaml is loaded from the same directory as the config file

  • If not found, searches up to ARES_HOME directory

  • Default: Current working directory

Example:

# Set ARES_HOME to project root
export ARES_HOME=/home/user/ares-project

# Run evaluation - files resolved relative to ARES_HOME
ares evaluate configs/my_eval.yaml

Connector-Specific Variables

Different connectors require specific environment variables. These should be set in a .env file or exported in your shell.

HuggingFace:

# Login via CLI (for gated models)
huggingface-cli login

WatsonX:

export WATSONX_URL=https://your-watsonx-instance.com
export WATSONX_API_KEY=your-api-key
export WATSONX_PROJECT_ID=your-project-id

WatsonX AgentLab:

export WATSONX_AGENTLAB_API_KEY=your-agent-api-key

OpenAI (via LiteLLM):

export OPENAI_API_KEY=your-openai-key

Exit Codes

The ARES CLI uses standard exit codes:

  • 0: Success

  • 1: General error (configuration error, file not found, etc.)

  • 2: Invalid command or arguments

Troubleshooting

Configuration Not Found

Error: FileNotFoundError: config.yaml not found

Solution:

  1. Check file path is correct

  2. Use absolute path or set ARES_HOME

  3. Verify file exists: ls -la config.yaml

Module Not Found

Error: ModuleNotFoundError: No module named 'ares_plugin'

Solution:

  1. Install the required plugin: pip install ares-plugin

  2. Verify installation: pip list | grep ares

  3. Check plugin is in correct location

Invalid Configuration

Error: KeyError: 'target' or Invalid configuration

Solution:

  1. Use --verbose to see full config

  2. Validate YAML syntax

  3. Check required fields are present

  4. Use ares show modules to verify module names

Connector Errors

Error: OSError: API key not found

Solution:

  1. Check environment variables are set

  2. Verify .env file is in correct location

  3. For HuggingFace: run huggingface-cli login

  4. Check API key permissions

Getting Help

For additional help:

ares --help
ares evaluate --help
ares show --help

Visit the ARES Documentation for detailed guides and examples.

Report issues on GitHub.