Risk identification

This notebook explains how to uncover risks related to your usecase based on a given taxonomy.¶

Import libraries¶

In [ ]:

from risk_atlas_nexus.blocks.inference import (
    RITSInferenceEngine,
    WMLInferenceEngine,
    OllamaInferenceEngine,
    VLLMInferenceEngine,
)
from risk_atlas_nexus.blocks.inference.params import (
    InferenceEngineCredentials,
    RITSInferenceEngineParams,
    WMLInferenceEngineParams,
    OllamaInferenceEngineParams,
    VLLMInferenceEngineParams,
)
from risk_atlas_nexus.library import RiskAtlasNexus

from risk_atlas_nexus.blocks.inference import ( RITSInferenceEngine, WMLInferenceEngine, OllamaInferenceEngine, VLLMInferenceEngine, ) from risk_atlas_nexus.blocks.inference.params import ( InferenceEngineCredentials, RITSInferenceEngineParams, WMLInferenceEngineParams, OllamaInferenceEngineParams, VLLMInferenceEngineParams, ) from risk_atlas_nexus.library import RiskAtlasNexus

Risk Atlas Nexus uses Large Language Models (LLMs) to infer risks dimensions. Therefore requires access to LLMs to inference or call the model.¶

Available Inference Engines: WML, Ollama, vLLM, RITS. Please follow the Inference APIs guide before going ahead.

Note: RITS is intended solely for internal IBM use and requires TUNNELALL VPN for access.

In [ ]:

inference_engine = OllamaInferenceEngine(
    model_name_or_path="granite3.2:8b",
    credentials=InferenceEngineCredentials(api_url="OLLAMA_API_URL"),
    parameters=OllamaInferenceEngineParams(
        num_predict=1000, num_ctx=8192, temperature=0, repeat_penalty=1
    ),
)

# inference_engine = WMLInferenceEngine(
#     model_name_or_path="ibm/granite-20b-code-instruct",
#     credentials={
#         "api_key": "WML_API_KEY",
#         "api_url": "WML_API_URL",
#         "project_id": "WML_PROJECT_ID",
#     },
#     parameters=WMLInferenceEngineParams(
#         max_new_tokens=1000, decoding_method="greedy", repetition_penalty=1
#     ),
# )

# inference_engine = VLLMInferenceEngine(
#     model_name_or_path="ibm-granite/granite-3.1-8b-instruct",
#     credentials=InferenceEngineCredentials(
#         api_url="VLLM_API_URL", api_key="VLLM_API_KEY"
#     ),
#     parameters=VLLMInferenceEngineParams(max_tokens=1000, temperature=0.7),
# )

# inference_engine = RITSInferenceEngine(
#     model_name_or_path="ibm/granite-20b-code-instruct",
#     credentials={
#         "api_key": "RITS_API_KEY",
#         "api_url": "RITS_API_URL",
#     },
#     parameters=RITSInferenceEngineParams(max_completion_tokens=1000, temperature=0.7),
# )

inference_engine = OllamaInferenceEngine( model_name_or_path="granite3.2:8b", credentials=InferenceEngineCredentials(api_url="OLLAMA_API_URL"), parameters=OllamaInferenceEngineParams( num_predict=1000, num_ctx=8192, temperature=0, repeat_penalty=1 ), ) # inference_engine = WMLInferenceEngine( # model_name_or_path="ibm/granite-20b-code-instruct", # credentials={ # "api_key": "WML_API_KEY", # "api_url": "WML_API_URL", # "project_id": "WML_PROJECT_ID", # }, # parameters=WMLInferenceEngineParams( # max_new_tokens=1000, decoding_method="greedy", repetition_penalty=1 # ), # ) # inference_engine = VLLMInferenceEngine( # model_name_or_path="ibm-granite/granite-3.1-8b-instruct", # credentials=InferenceEngineCredentials( # api_url="VLLM_API_URL", api_key="VLLM_API_KEY" # ), # parameters=VLLMInferenceEngineParams(max_tokens=1000, temperature=0.7), # ) # inference_engine = RITSInferenceEngine( # model_name_or_path="ibm/granite-20b-code-instruct", # credentials={ # "api_key": "RITS_API_KEY", # "api_url": "RITS_API_URL", # }, # parameters=RITSInferenceEngineParams(max_completion_tokens=1000, temperature=0.7), # )

Create an instance of RiskAtlasNexus¶

Note: (Optional) You can specify your own directory in RiskAtlasNexus(base_dir=<PATH>) to utilize custom AI ontologies. If left blank, the system will use the provided AI ontologies.

In [ ]:

risk_atlas_nexus = RiskAtlasNexus()

risk_atlas_nexus = RiskAtlasNexus()

Risk Identification API¶

RiskAtlasNexus.identify_risks_from_usecases()

Params:

• usecases (List[str]): A List of strings describing AI usecases
• inference_engine (InferenceEngine): An LLM inference engine to infer risks from the usecases.
• taxonomy (str, optional): The string label for a taxonomy. Default to None.
• cot_examples (Dict[str, List], optional): The Chain of Thought (CoT) examples to use in the risk identification. The example template is available at src/risk_atlas_nexus/data/templates/risk_generation_cot.json. Assign the ID of the taxonomy you wish to use as the key for CoT examples. Providing this value will override the CoT examples present in the template master. Default to None.
• max_risk (int, optional): The maximum number of risks to extract. Pass None to allow the inference engine to determine the number of risks. Defaults to None.

Risk Identification using IBM AI Risk taxonomy¶

In [ ]:

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers."

risks = risk_atlas_nexus.identify_risks_from_usecases(
    usecases=[usecase],
    inference_engine=inference_engine,
    taxonomy="ibm-risk-atlas",
    max_risk=5,
)

risks

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers." risks = risk_atlas_nexus.identify_risks_from_usecases( usecases=[usecase], inference_engine=inference_engine, taxonomy="ibm-risk-atlas", max_risk=5, ) risks

Risk Identification using IBM AI Risk taxonomy with Custom CoT examples¶

Note: To use custom risk cot_examples for a new taxonomy or an existing taxonomy, users must provide their own set of example usecase and associated risks in a JSON file such as in risk_generation_cot.json. This will enable the LLM to learn from these few-shot examples and generate better responses. Please follow the guide here.

In [ ]:

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers."

risks = risk_atlas_nexus.identify_risks_from_usecases(
    usecases=[usecase],
    inference_engine=inference_engine,
    taxonomy="ibm-risk-atlas",
    cot_examples={
        "ibm-risk-atlas": [
            {
                "Usecase": "In a medical chatbot, generative AI can be employed to create a triage system that assesses patients' symptoms and provides immediate, contextually relevant advice based on their medical history and current condition. The chatbot can analyze the patient's input, identify potential medical issues, and offer tailored recommendations or insights to the patient or healthcare provider. This can help streamline the triage process, ensuring that patients receive the appropriate level of care and attention, and ultimately improving patient outcomes.",
                "Risks": [
                    "Improper usage",
                    "Incomplete advice",
                    "Lack of model transparency",
                    "Lack of system transparency",
                    "Lack of training data transparency",
                    "Data bias",
                    "Uncertain data provenance",
                    "Lack of data transparency",
                    "Impact on human agency",
                    "Impact on affected communities",
                    "Improper retraining",
                    "Inaccessible training data",
                ],
            }
        ]
    },
    max_risk=5,
)

risks

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers." risks = risk_atlas_nexus.identify_risks_from_usecases( usecases=[usecase], inference_engine=inference_engine, taxonomy="ibm-risk-atlas", cot_examples={ "ibm-risk-atlas": [ { "Usecase": "In a medical chatbot, generative AI can be employed to create a triage system that assesses patients' symptoms and provides immediate, contextually relevant advice based on their medical history and current condition. The chatbot can analyze the patient's input, identify potential medical issues, and offer tailored recommendations or insights to the patient or healthcare provider. This can help streamline the triage process, ensuring that patients receive the appropriate level of care and attention, and ultimately improving patient outcomes.", "Risks": [ "Improper usage", "Incomplete advice", "Lack of model transparency", "Lack of system transparency", "Lack of training data transparency", "Data bias", "Uncertain data provenance", "Lack of data transparency", "Impact on human agency", "Impact on affected communities", "Improper retraining", "Inaccessible training data", ], } ] }, max_risk=5, ) risks

Risk Identification using NIST AI taxonomy¶

In [ ]:

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers."

risks = risk_atlas_nexus.identify_risks_from_usecases(
    usecases=[usecase],
    inference_engine=inference_engine,
    taxonomy="nist-ai-rmf",
)

risks

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers." risks = risk_atlas_nexus.identify_risks_from_usecases( usecases=[usecase], inference_engine=inference_engine, taxonomy="nist-ai-rmf", ) risks

Risk Identification using MIT AI taxonomy¶

In [ ]:

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers."

risks = risk_atlas_nexus.identify_risks_from_usecases(
    usecases=[usecase],
    inference_engine=inference_engine,
    taxonomy="mit-ai-risk-repository",
)

risks

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers." risks = risk_atlas_nexus.identify_risks_from_usecases( usecases=[usecase], inference_engine=inference_engine, taxonomy="mit-ai-risk-repository", ) risks

Risk Identification using Granite Guardian taxonomy¶

In [ ]:

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers."

risks = risk_atlas_nexus.identify_risks_from_usecases(
    usecases=[usecase],
    inference_engine=inference_engine,
    taxonomy="ibm-granite-guardian",
)

risks

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers." risks = risk_atlas_nexus.identify_risks_from_usecases( usecases=[usecase], inference_engine=inference_engine, taxonomy="ibm-granite-guardian", ) risks

Risk Identification using Credo Unified Control Framework taxonomy¶

In [ ]:

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers."

risks = risk_atlas_nexus.identify_risks_from_usecases(
    usecases=[usecase],
    inference_engine=inference_engine,
    taxonomy="credo-ucf",
)

risks

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers." risks = risk_atlas_nexus.identify_risks_from_usecases( usecases=[usecase], inference_engine=inference_engine, taxonomy="credo-ucf", ) risks