FAQ

Frequently Asked Questions about ARES – AI Robustness Evaluation System.

🧩 General Questions

What is ARES? ARES is a plugin-based framework developed by IBM Research for automated red-teaming of AI systems. It helps evaluate model robustness using configurable scenarios and extensible plugins.

What types of models can ARES evaluate? ARES supports standard model providers (HuggingFace, LiteLLM, etc), REST API endpoints, etc via connector plugins. These include HuggingFace transformers, OpenAI APIs, and custom endpoints.

What are plugins in ARES? Plugins extend ARES functionality. Types include:

• 🎯 Attack Plugins

• 🔗 Connector Plugins

• 🎓 Goals Plugins

• 📊 Evaluation Plugins

🔧 Installation and Setup

How do I install ARES? You can install ARES in several ways:

• Clone the repository:

  git clone https://github.com/IBM/ares.git
  cd ares
  pip install .
  

• Using pip or uv (if available):

  pip install ares-redteamer
  

Which Python versions are supported? ARES requires Python 3.10+.

How do I run ARES after installation? Use the following command to run minimal example: .. code-block:: bash

ares evaluate example_configs/minimal.yaml

Where can I find examples? Check the example_configs/ folder in the repository, a walkthrough notebook/, or explore the documentation.

🧰 Configuration and Usage

How do I craft `example_config.yaml` files for ARES? Start with examples in the example_configs/ folder. See the ARES Configuration for details on how to populate each node (target/red-teaming/strategy/goals/evaluation/connector)

How do connectors work and how can I run one? Connectors interface with models or AI endpoints. Define them in connectors.yaml, use in your example_config.yaml and ARES will initialize them. Developers can use _build_connector() internally to instantiate connectors.

How do I use different intents and implement them? Intents define the goal of a red-teaming scenario (e.g., privacy leak, prompt injection). Choose intents aligned with your evaluation goals and map them to OWASP/NIST categories. Check intents.json for OWAPS intents.

How do I use `example_config` to run a test? Use a sample from example_configs/, modify paths and parameters, then run:

ares evaluate path/to/example_config.yaml

🛠️ Troubleshooting and Development

Why did my custom evaluation framework default to Keywords Evaluation? This fallback occurs when ARES fails to load your custom plugin. Check for: - Incorrect interface implementation - Typo in type value (full class name of the plugin module) in your YAML config - Plugin not installed - Incorrect YAML config structure - check more example YAMLs in the example_configs/

How do I implement a custom evaluation plugin? Create a folder under plugins/ using a template folder provided there, use ARES AttackEval() as a base class and implement the evaluate() method. Reference this class in your config file.

How do I run a connector manually? If developing or debugging, use _build_connector() to instantiate a connector class with its config. Normally, ARES handles this automatically. You can also use connector: key on YAML config to acces specific connector from your plugin/module.

Does ARES support GPU acceleration? Yes, if endpoint supports acceleration, the connector class will have this option to configure (e.g. check HuggingFace examples)

📚 Learn More

Visit the full documentation for: - Plugin development - Configuration options - Contribution guidelines

ARES is open-source and welcomes community contributions.