Contributing a taxonomy or CoT templates

This document will describe how to contribute custom taxonomy files and CoT templates to the Risk Atlas Nexus project.

1. Overview
   1. What are the taxonomy files?
   2. How are mappings created between taxonomies?
2. Add your own taxonomies/risks/actions
   1. An example addition of a risk
3. Add your own mappings
4. Add your own Chain of Thought templates
   1. Contribute the templates

Overview

You may wish to contribute your own taxonomy definitions, entity mappings or Chain of Thought templates to the Risk Atlas Nexus project. We welcome any contributions.

Please familiarise yourself with the general contribution guidelines and set up your own fork of the project.

What are the taxonomy files?

The Risk Atlas Nexus ontology combines the AI risk view (taxonomies, risks, actions) with an AI model view (AI systems, AI models, model evaluations) into one schema. These are loaded into the system with LinkML, through yaml representations, stored in the knowledge graph data folder.

How are mappings maintained between taxonomies?

To express some semantically meaningful mapping between risks from different taxonomies, we have used the Simple Standard for Sharing Ontological Mappings (SSSOM) . The mappings are maintained in SSOM TSV files and are converted to LinkML data YAML using Python helper scripts.

Add your own taxonomies/risks/actions

You can add your own risk definitions by adding yaml to knowledge graph data folder. Ensure the entries comply with the schema

An example addition of a risk

If you would like to contribute your own taxonomy files to the project for others to use, add one or more yaml files to the knowledge graph data directory .

For example, to add a new risk, create a file with the following sample content, and add it to the knowledge graph data directory.

- id: my-own-risk
  name: A very risky AI behaviour
  description: An LLM-based system is often very risky
  isDefinedByTaxonomy: my-taxonomy

Add your own mappings

1. Prepare a TSV file either
   1. manually, or
   2. semi-automatically, with aid of the python notebook Prepare taxonomy mappings.
2. Verify the TSV mappings are correct.
3. Lift all the TSV files to yaml mapping entries by executing make lift_mappings_from_tsv.
4. Ensure the changes were saved to the knowledge graph data mapping folder .

Save your changes

• Save your new taxonomy files in the knowledge graph data directory . Push your changes and make a PR to the main project.

Add your own Chain of Thought templates

The template used in risk_atlas_nexus/data/templates/cot_examples.json defines the few-shot examples which are use for the auto-assist functionality.

Customization: To adapt this auto-assist functionality to custom questionnaires, users need to provide their own set of questions, example intents, and corresponding answers in a json file (e.g.,risk_atlas_nexus/data/templates/cot_examples.json). This will enable the LLM to learn from these few-shot examples and generate responses for unseen queries.

Template Structure:

• Each question is associated with a list of example intents and corresponding answers.
• The format is:
  • Question
  • - intents: (list of example intents)
  • - answers: (list of corresponding answers for the respective intents above)

Contribute the templates

If you would like to contribute your own templates to the project for others to use:

• Save your new template in the risk_atlas_nexus/data/templates/ folder, push your changes and make a PR to the main project.