Model Insights

class ibm_watsonx_gov.visualizations.model_insights.ModelInsights(configuration: GenAIConfiguration, metrics: list[GenAIMetric])

Bases: object

Class to display venn diagrams using metric violations NOTE: For venn diagram interactivity, ipympl (jupyter-matplotlib) Jupyter extension needs to be installed

COLORS = ['#8A3FFC', '#1192E8', '#009D9A']
CYAN = '#1192E8'
DEFAULT_SELECTED_METRICS_COUNT = 9
MAX_METRIC_GROUP_SIZE = 3
PURPLE = '#8A3FFC'
TEAL = '#009D9A'
display_metrics(metrics_result: DataFrame)

Method to display ModelInsights

Parameters:

metrics_result (pd.DataFrame) – _description_

print_violation_summary(metric_ids_violation_count)
Helper method to format and display the violated records summary. This will highlight this information:

  • metric id

  • configured threshold

  • number of violated records

render_faithfulness_attributions(selected_violation)

This function will render a table of each faithfulness attribution of the answer with its score. When a row is selected, the contexts will be listed with each attribution highlighted and color coded based on its score.

render_question_and_answer_faithfulness(selected_violation)

Function to parse the faithfulness attributions, build html code, and display it

render_venn_diagrams(group_index=None, filters=None)

Function to render multiple interactive venn diagrams

show_all_metrics_dropdown()
Function to render the widget UI. This will render the following:

  • Dropdown component to select metrics

  • Default selected metrics based on the top metrics with violated records

  • Venn diagrams of the selected metrics

Note: For the venn diagrams to be interactive ipympl backend should by enabled, this can be done by:

  • installing ipympl Jupyter extension

  • explicitly enable ipympl backend by adding this line to the notebook %matplotlib ipympl

show_checkboxes_with_venn(metric_group_index: int)

Display venn diagram for the selected metric group along with checkboxes to select which metrics should be shown

show_violations_table_by_violation_ids(violation_ids: list[int])

Function to display records by in ids list