ICX360

Overview

This toolkit provides in-context explanations for LLMs - explanations of the output of an LLM in terms of parts of the input context given to the LLM. It will be useful for both researchers and practitioners who want to understand the reason for a particular generation with respect to the context.

The toolkit features explanation methods, quick start and elaborate example notebooks, tests, and documentation. The quick start notebooks can also be run in Google Colab.

Methods

1. Multi-Level Explanations for Generative Language Models: Explains generated text by attributing to parts of the input context and quantifying the importance of these parts to the generation.

2. CELL your Model: Contrastive Explanations for Large Language Models: Explains text generation by generating contrastive prompts (i.e., edited version of the input prompt) that elicit responses that differ from the original response according to a pre-defined score.

3. Token Highlighter: Inspecting and Mitigating Jailbreak Prompts for Large Language Models: Explains potential jailbreak threats by highlighting important prompt tokens based on model gradients.

Prerequisites

The toolkit can be installed locally using the instructions below. Please ensure sufficient resources (such as GPUs) are available for running the methods.

The toolkit uses uv as the package manager (Python 3.11). Make sure that uv is installed via either:

curl -Ls https://astral.sh/uv/install.sh | sh

or using Homebrew:

brew install astral-sh/uv/uv

or using pip (use this if in Windows):

pip install uv

Installation

Once uv is installed, in Linux or Mac, clone the repo:

git clone git@github.com:IBM/ICX360.git icx360
cd icx360

Ensure that you are inside the icx360 directory (where README.md is located) and run:

uv venv --python 3.12
source .venv/bin/activate
uv pip install .

Or in Windows, run:

uv venv --python 3.12
.venv/bin/activate
uv pip install .

The package has been tested on Red Hat Enterprise Linux 9.

Quickstart Examples

1. MExGen Quick Start

2. CELL Quick Start

3. Token Highlighter Jailbreak Inspector Quick Start

Find many more examples here.

Tests

We have included a collection of tests that can be used for checking if the installation worked properly. This can be achieved by running the "non-slow" tests:

pytest -m "not slow"

Note that these only test the plumbing and not any realistic functionality.

Those interested in testing the realistic functionality can run the notebooks in the ./examples/ folder or run the "slow and not vllm" tests. These use Hugging Face models that are large enough to meaningfully perform their tasks. Please ensure you have sufficient infrastructure (e.g. GPUs) before running these. Run the "slow and not vllm" tests using:

pytest -m "slow and not vllm"

Finally, if you also have a VLLM model that you would like to test, first enter its parameters in model_catalog in tests/conftest.py and then run:

pytest -m "vllm"

License

ICX360 is provided under Apache 2.0 license.

Contributing

• Get started by checking our contribution guidelines.
• If you have any questions, just ask!

Get involved

Lets form a community around this toolkit! Ask a question, raise an issue, or express interest to contribute in the discussions page.

IBM ❤️ Open Source AI

The first release of ICX360 has been brought to you by IBM in the hope of building a larger community around this topic.