Auto assist questionnaire

Auto-fill Questionnaire using Chain of Thought or Few-Shot Examples¶

This notebook showcases the application of few-shot examples in autofilling questionnaires. It utilizes a json file (risk_questionnaire_cot.json) to provide the LLM with example responses for some use-cases.

By leveraging these few-shot examples, we can enable seamless completion of lengthy questionnaires, minimizing manual effort and improving overall efficiency.

In [2]:

from risk_atlas_nexus.blocks.inference import (
    RITSInferenceEngine,
    WMLInferenceEngine,
    OllamaInferenceEngine,
    VLLMInferenceEngine,
)
from risk_atlas_nexus.blocks.inference.params import (
    InferenceEngineCredentials,
    RITSInferenceEngineParams,
    WMLInferenceEngineParams,
    OllamaInferenceEngineParams,
    VLLMInferenceEngineParams,
)

from risk_atlas_nexus.data import load_resource
from risk_atlas_nexus.library import RiskAtlasNexus

from risk_atlas_nexus.blocks.inference import ( RITSInferenceEngine, WMLInferenceEngine, OllamaInferenceEngine, VLLMInferenceEngine, ) from risk_atlas_nexus.blocks.inference.params import ( InferenceEngineCredentials, RITSInferenceEngineParams, WMLInferenceEngineParams, OllamaInferenceEngineParams, VLLMInferenceEngineParams, ) from risk_atlas_nexus.data import load_resource from risk_atlas_nexus.library import RiskAtlasNexus

Risk Atlas Nexus uses Large Language Models (LLMs) to infer risks dimensions. Therefore requires access to LLMs to inference or call the model.¶

Available Inference Engines: WML, Ollama, vLLM, RITS. Please follow the Inference APIs guide before going ahead.

Note: RITS is intended solely for internal IBM use and requires TUNNELALL VPN for access.

In [2]:

inference_engine = OllamaInferenceEngine(
    model_name_or_path="granite3.2:8b",
    credentials=InferenceEngineCredentials(api_url="OLLAMA_API_URL"),
    parameters=OllamaInferenceEngineParams(
        num_predict=1000, temperature=0, repeat_penalty=1, num_ctx=8192
    ),
)

# inference_engine = WMLInferenceEngine(
#     model_name_or_path="ibm/granite-20b-code-instruct",
#     credentials={
#         "api_key": "WML_API_KEY",
#         "api_url": "WML_API_URL",
#         "project_id": "WML_PROJECT_ID",
#     },
#     parameters=WMLInferenceEngineParams(
#         max_new_tokens=1000, decoding_method="greedy", repetition_penalty=1
#     ),
# )

# inference_engine = VLLMInferenceEngine(
#     model_name_or_path="ibm-granite/granite-3.1-8b-instruct",
#     credentials=InferenceEngineCredentials(
#         api_url="VLLM_API_URL", api_key="VLLM_API_KEY"
#     ),
#     parameters=VLLMInferenceEngineParams(max_tokens=1000, temperature=0.7),
# )

# inference_engine = RITSInferenceEngine(
#     model_name_or_path="ibm-granite/granite-3.1-8b-instruct",
#     credentials={
#         "api_key": "RITS_API_KEY",
#         "api_url": "RITS_API_URL",
#     },
#     parameters=RITSInferenceEngineParams(max_tokens=1000, temperature=0.7),
# )

inference_engine = OllamaInferenceEngine( model_name_or_path="granite3.2:8b", credentials=InferenceEngineCredentials(api_url="OLLAMA_API_URL"), parameters=OllamaInferenceEngineParams( num_predict=1000, temperature=0, repeat_penalty=1, num_ctx=8192 ), ) # inference_engine = WMLInferenceEngine( # model_name_or_path="ibm/granite-20b-code-instruct", # credentials={ # "api_key": "WML_API_KEY", # "api_url": "WML_API_URL", # "project_id": "WML_PROJECT_ID", # }, # parameters=WMLInferenceEngineParams( # max_new_tokens=1000, decoding_method="greedy", repetition_penalty=1 # ), # ) # inference_engine = VLLMInferenceEngine( # model_name_or_path="ibm-granite/granite-3.1-8b-instruct", # credentials=InferenceEngineCredentials( # api_url="VLLM_API_URL", api_key="VLLM_API_KEY" # ), # parameters=VLLMInferenceEngineParams(max_tokens=1000, temperature=0.7), # ) # inference_engine = RITSInferenceEngine( # model_name_or_path="ibm-granite/granite-3.1-8b-instruct", # credentials={ # "api_key": "RITS_API_KEY", # "api_url": "RITS_API_URL", # }, # parameters=RITSInferenceEngineParams(max_tokens=1000, temperature=0.7), # )

[2025-05-26 18:30:21:729] - INFO - RiskAtlasNexus - OLLAMA inference engine will execute requests on the server at http://localhost:11434.
[2025-05-26 18:30:21:803] - INFO - RiskAtlasNexus - Created OLLAMA inference engine.

Create an instance of RiskAtlasNexus¶

Note: (Optional) You can specify your own directory in RiskAtlasNexus(base_dir=<PATH>) to utilize custom AI ontologies. If left blank, the system will use the provided AI ontologies.

In [3]:

risk_atlas_nexus = RiskAtlasNexus()

risk_atlas_nexus = RiskAtlasNexus()

[2025-05-26 18:30:21:880] - INFO - RiskAtlasNexus - Created RiskAtlasNexus instance. Base_dir: None

Defining Examples for Auto-Assist Functionality¶

The auto-assist feature utilizes few-shot examples defined in the file risk_atlas_nexus/data/templates/risk_questionnaire_cot.json to predict the output of the risk questionnaire.

Customization:

To adapt this auto-assist functionality to custom risk questionnaire, users need to provide their own set of questions, example intents, and corresponding answers in a json file such as in risk_questionnaire_cot.json. This will enable the LLM to learn from these few-shot examples and generate responses for unseen queries.

CoT Template - Zero Shot method

Each question is accompanied by corresponding examples provided as an empty list.

  [
      {
          "question": "In which environment is the system used?",
          "cot_examples": []
      }
      ...
  ]

CoT Template - Few Shot method

Each question is associated with a list of examples, each containing intent, answer, and optional explanation.

  [
      {
          "question": "In which environment is the system used?",
          "cot_examples": [
            {
              "intent": "Find patterns in healthcare insurance claims",
              "answer": "Insurance Claims Processing or Risk Management or Data Analytics",
              "explanation": "The system might be used by an insurance company's claims processing department to analyze and identify patterns in healthcare insurance claims."
            },
            {
                "intent": "optimize supply chain management in Investment banks",
                "answer": "Treasury Departments or Asset Management Divisions or Private Banking Units",
                "explanation": null
            },
            ...
          ]
      }
      ...
  ]

In this notebook, we're using a simplified template to cover 7 questions from the Airo questionnaire:

1. AI Domain
2. System environment
3. Utilized techniques
4. Intended User
5. Intended Purpose
6. System Application
7. AI Subject

Load Risk Questionnaire¶

Note: The cell below loads examples of risk questionnaires from Risk Atlas Master. To load your custom questionnaire, create it according to the specified format and load it instead.

In [7]:

risk_questionnaire = load_resource("risk_questionnaire_cot.json")

risk_questionnaire[0]

risk_questionnaire = load_resource("risk_questionnaire_cot.json") risk_questionnaire[0]

Out[7]:

{'question': 'What domain does your use request fall under? Customer service/support, Technical, Information retrieval, Strategy, Code/software engineering, Communications, IT/business automation, Writing assistant, Financial, Talent and Organization including HR, Product, Marketing, Cybersecurity, Healthcare, User Research, Sales, Risk and Compliance, Design, Other',
 'cot_examples': [{'intent': 'Optimize supply chain management in Investment banks',
   'answer': 'Strategy',
   'explanation': 'Since the task is involved in improving the processes to ensure better performance. It is not finance since the task is on supply chain optimization and not on financial aspects even though the application domain is banks.'},
  {'intent': 'Ability to create dialog flows and integrations from natural language instructions.',
   'answer': 'Customer service/support',
   'explanation': 'Since the task relates to human conversations or generating human converstations or support.'},
  {'intent': 'Check if a document has grammatical mistakes.',
   'answer': 'Writing assitant',
   'explanation': 'Since this helps in improving the quality of text. It is not customer service since this on on the quality of text rather than helping in human conversations.'},
  {'intent': 'Optimize supply chain management in Investment banks',
   'answer': 'Strategy',
   'explanation': 'Since the task is involved in improving the processes to ensure better performance. It is not finance since the task is on supply chain optimization and not on financial aspects even though the application domain is banks.'},
  {'intent': "In the context of drug repurposing, generative AI can be employed to analyze vast databases of existing drugs and their clinical trials data. By identifying patterns and similarities, the AI can suggest potential new therapeutic indications for existing drugs, based on the chemical structure and pharmacological properties of the APIs. This process can help streamline the drug development pipeline, as it would reduce the need for time-consuming and expensive clinical trials for new indications. For instance, a drug like Atorvastatin, which is currently used to lower cholesterol, could be repurposed for the treatment of diabetic nephropathy, a kidney disease, based on the AI's analysis of similar drugs and their clinical data. This would not only save resources but also provide new treatment options for patients suffering from this debilitating condition. ",
   'answer': 'Healthcare and strategy',
   'explanation': 'Since the task is related to healthcare and drug repurposing, which involves analyzing data related to drugs and their clinical trials, this falls under the healthcare domain. It also involves Strategy it talks about using patterns to create new treatment options.'}]}

There are two ways to use the inference engine to get the LLM outputs. generate_zero_shot_risk_questionnaire_output which gives the zero-shot output for the question and generate_few_shot_risk_questionnaire_output which gives the output using few-shot examples defined above.

Auto Assist Questionnaire - Zero Shot¶

In [5]:

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers."

results = risk_atlas_nexus.generate_zero_shot_risk_questionnaire_output(
    usecase, risk_questionnaire, inference_engine
)

# Display Results
for index, (question_data, result) in enumerate(
    zip(risk_questionnaire, results), start=1
):
    print(
        f"\n{index}: "
        + question_data["question"]
        + "\nA: "
        + result.prediction["answer"]
    )

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers." results = risk_atlas_nexus.generate_zero_shot_risk_questionnaire_output( usecase, risk_questionnaire, inference_engine ) # Display Results for index, (question_data, result) in enumerate( zip(risk_questionnaire, results), start=1 ): print( f"\n{index}: " + question_data["question"] + "\nA: " + result.prediction["answer"] )

Inferring with OLLAMA: 100%|██████████| 7/7 [00:24<00:00,  3.46s/it]

1: What domain does your use request fall under? Customer service/support, Technical, Information retrieval, Strategy, Code/software engineering, Communications, IT/business automation, Writing assistant, Financial, Talent and Organization including HR, Product, Marketing, Cybersecurity, Healthcare, User Research, Sales, Risk and Compliance, Design, Other
A: Customer service/support

2: In which environment is the system used?
A: The system is used in a digital environment, specifically designed for online interactions between customers and support agents.

3: What techniques are utilised in the system? Multi-modal: {Document Question/Answering, Image and text to text, Video and text to text, visual question answering}, Natural language processing: {feature extraction, fill mask, question answering, sentence similarity, summarization, table question answering, text classification, text generation, token classification, translation, zero shot classification}, computer vision: {image classification, image segmentation, text to image, object detection}, audio:{audio classification, audio to audio, text to speech}, tabular: {tabular classification, tabular regression}, reinforcement learning
A: The system employs a variety of techniques, including multi-modal approaches such as Document Question/Answering, Image and text to text, Video and text to text, and visual question answering. In terms of Natural Language Processing, it utilizes methods like feature extraction, fill mask, question answering, sentence similarity, summarization, table question answering, text classification, text generation, token classification, translation, and zero shot classification. For computer vision, it uses image classification, image segmentation, text to image, and object detection. In the audio domain, it applies audio classification, audio to audio, and text to speech. Additionally, it incorporates tabular methods such as tabular classification and tabular regression. Lastly, it leverages reinforcement learning.

4: Who is the intended user of the system?
A: The intended user of the system is the support agent.

5: What is the intended purpose of the system?
A: The intended purpose of the system is to assist compliance officers in generating personalized, relevant responses, recommendations, and summaries of claims for customers. This is done to enhance interactions between support agents and customers, ensuring accurate and efficient communication.

6: What is the application of the system?
A: The application of the system is to assist compliance officers in generating personalized, relevant responses, recommendations, and summaries of claims for support agents. This is achieved by understanding the intent of the customer's inquiry and providing tailored information to enhance the interaction between the support agent and the customer.

7: Who is the subject as per the intent?
A: The subject in this context is the compliance officer.

Auto Assist Questionnaire - Few Shot¶

In [8]:

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers."

results = risk_atlas_nexus.generate_few_shot_risk_questionnaire_output(
    usecase,
    risk_questionnaire,
    inference_engine,
)

# Display Results
for index, (question_data, result) in enumerate(
    zip(risk_questionnaire, results), start=1
):
    print(
        f"\n{index}: "
        + question_data["question"]
        + "\nA: "
        + result.prediction["answer"]
    )

usecase = "Generate personalized, relevant responses, recommendations, and summaries of claims for customers to support agents to enhance their interactions with customers." results = risk_atlas_nexus.generate_few_shot_risk_questionnaire_output( usecase, risk_questionnaire, inference_engine, ) # Display Results for index, (question_data, result) in enumerate( zip(risk_questionnaire, results), start=1 ): print( f"\n{index}: " + question_data["question"] + "\nA: " + result.prediction["answer"] )

Inferring with OLLAMA: 100%|██████████| 7/7 [00:36<00:00,  5.23s/it]

1: What domain does your use request fall under? Customer service/support, Technical, Information retrieval, Strategy, Code/software engineering, Communications, IT/business automation, Writing assistant, Financial, Talent and Organization including HR, Product, Marketing, Cybersecurity, Healthcare, User Research, Sales, Risk and Compliance, Design, Other
A: Customer service/support

2: In which environment is the system used?
A: Customer Service or Claims Support Departments

3: What techniques are utilised in the system? Multi-modal: {Document Question/Answering, Image and text to text, Video and text to text, visual question answering}, Natural language processing: {feature extraction, fill mask, question answering, sentence similarity, summarization, table question answering, text classification, text generation, token classification, translation, zero shot classification}, computer vision: {image classification, image segmentation, text to image, object detection}, audio:{audio classification, audio to audio, text to speech}, tabular: {tabular classification, tabular regression}, reinforcement learning
A: Natural language processing: text generation and summarization

4: Who is the intended user of the system?
A: Customer Support Agents

5: What is the intended purpose of the system?
A: To enhance customer service by providing support agents with personalized, contextually relevant information and recommendations, enabling them to address customer inquiries and claims more effectively and efficiently.

6: What is the application of the system?
A: Natural Language Generation (NLG): Develop AI models to generate personalized, context-aware responses and summaries for customer interactions. 
Sentiment Analysis: Use NLP to understand customer sentiment and tailor responses accordingly. 
Recommendation Engine: Analyze customer data to suggest relevant products, services, or solutions based on individual preferences and needs. 
Chatbot Integration: Integrate with customer support platforms to provide real-time, automated assistance to support agents.

7: Who is the subject as per the intent?
A: Claims and Customers