FAQ
===

Frequently Asked Questions about **ARES** – AI Robustness Evaluation System.

🧩 General Questions
--------------------

**What is ARES?**
ARES is a plugin-based framework developed by IBM Research for automated red-teaming of AI systems. It helps evaluate model robustness using configurable scenarios and extensible plugins.

**What types of models can ARES evaluate?**
ARES supports standard model providers (HuggingFace, LiteLLM, etc), REST API endpoints, etc via connector plugins. These include HuggingFace transformers, OpenAI APIs, and custom endpoints.

**What are plugins in ARES?**
Plugins extend ARES functionality. Types include:

- 🎯 Attack Plugins
- 🔗 Connector Plugins
- 🎓 Goals Plugins
- 📊 Evaluation Plugins


🔧 Installation and Setup
-------------------------

**How do I install ARES?**
You can install ARES in several ways:

- Clone the repository:

  .. code-block:: bash

     git clone https://github.com/IBM/ares.git
     cd ares
     pip install .

- Using pip or uv (if available):

  .. code-block:: bash

     pip install ares-redteamer

**Which Python versions are supported?**
ARES requires **Python 3.10+**.

**How do I run ARES after installation?**
Use the following command to run minimal example:
.. code-block:: bash

    ares evaluate example_configs/minimal.yaml

**Where can I find examples?**
Check the `example_configs/ <https://github.com/IBM/ares/blob/main/example_configs/>`_ folder in the repository, a `walkthrough notebook/ <https://github.com/IBM/ares/blob/main/notebooks/>`_, or explore the documentation.

🧰 Configuration and Usage
--------------------------

**How do I craft `example_config.yaml` files for ARES?**
Start with examples in the `example_configs/ <https://github.com/IBM/ares/blob/main/example_configs/>`_ folder.
See the ARES Configuration for details on how to populate each node (target/red-teaming/strategy/goals/evaluation/connector)

**How do connectors work and how can I run one?**
Connectors interface with models or AI endpoints. Define them in `connectors.yaml <https://github.com/IBM/ares/blob/main/example_configs/connectors.yaml>`_, use in your `example_config.yaml` and ARES will initialize them. Developers can use `_build_connector()` internally to instantiate connectors.

**How do I use different intents and implement them?**
Intents define the goal of a red-teaming scenario (e.g., privacy leak, prompt injection). Choose intents aligned with your evaluation goals and map them to OWASP/NIST categories.
Check `intents.json` for OWAPS intents.

**How do I use `example_config` to run a test?**
Use a sample from `example_configs/ <https://github.com/IBM/ares/blob/main/example_configs/>`_, modify paths and parameters, then run:

.. code-block:: bash

    ares evaluate path/to/example_config.yaml

🛠️ Troubleshooting and Development
----------------------------------

**Why did my custom evaluation framework default to Keywords Evaluation?**
This fallback occurs when ARES fails to load your custom plugin. Check for:
- Incorrect interface implementation
- Typo in ``type`` value (full class name of the plugin module) in your YAML config
- Plugin not installed
- Incorrect YAML config structure - check more example YAMLs in the `example_configs/ <https://github.com/IBM/ares/blob/main/example_configs/>`_

**How do I implement a custom evaluation plugin?**
Create a folder under `plugins/` using a template folder provided there, use ARES AttackEval() as a base class and implement the `evaluate()` method. Reference this class in your config file.

**How do I run a connector manually?**
If developing or debugging, use `_build_connector()` to instantiate a connector class with its config. Normally, ARES handles this automatically.
You can also use ``connector:`` key on YAML config to acces specific connector from your plugin/module.


**Does ARES support GPU acceleration?**
Yes, if endpoint supports acceleration, the connector class will have this option to configure (e.g. check `HuggingFace examples <https://github.com/IBM/ares/blob/main/example_configs/connectors.yaml>`_)


📚 Learn More
-------------

Visit the full documentation for:
- Plugin development
- Configuration options
- Contribution guidelines

ARES is open-source and welcomes community contributions.