Prompt Safety Risk Metric¶
- pydantic model ibm_watsonx_gov.metrics.prompt_safety_risk.prompt_safety_risk_metric.PromptSafetyRiskMetric¶
Bases:
GenAIMetricDefines the PromptSafetyRisk metric class.
The PromptSafetyRisk metric evaluates how likely an AI is to respond with harmful, unsafe, or inappropriate content.
Note : system_prompt is mandatory
Examples
- Create PromptSafetyRisk metric with default parameters and compute using metrics evaluator.
metric = PromptSafetyRiskMetric(system_prompt="...") result = MetricsEvaluator().evaluate(data={"input_text": "...", metrics=[metric])
- Create PromptSafetyRisk metric with a custom threshold.
threshold = MetricThreshold(type="lower_limit", value=0.5) metric = PromptSafetyRiskMetric(threshold=threshold, system_prompt="...")
Show JSON schema
{ "title": "PromptSafetyRiskMetric", "description": "Defines the PromptSafetyRisk metric class.\n\nThe PromptSafetyRisk metric evaluates how likely an AI is to respond with harmful, unsafe, or inappropriate content.\n\nNote : system_prompt is mandatory \n\nExamples:\n 1. Create PromptSafetyRisk metric with default parameters and compute using metrics evaluator.\n .. code-block:: python\n\n metric = PromptSafetyRiskMetric(system_prompt=\"...\")\n result = MetricsEvaluator().evaluate(data={\"input_text\": \"...\", metrics=[metric])\n\n 2. Create PromptSafetyRisk metric with a custom threshold.\n .. code-block:: python\n\n threshold = MetricThreshold(type=\"lower_limit\", value=0.5)\n metric = PromptSafetyRiskMetric(threshold=threshold, system_prompt=\"...\")", "type": "object", "properties": { "name": { "const": "prompt_safety_risk", "default": "prompt_safety_risk", "description": "The prompt safety risk metric name.", "title": "Name", "type": "string" }, "thresholds": { "default": [ { "type": "lower_limit", "value": 0.7 } ], "description": "The metric thresholds.", "items": { "$ref": "#/$defs/MetricThreshold" }, "title": "Thresholds", "type": "array" }, "tasks": { "default": [ "question_answering", "classification", "summarization", "generation", "extraction", "retrieval_augmented_generation" ], "description": "The list of supported tasks.", "items": { "$ref": "#/$defs/TaskType" }, "title": "Tasks", "type": "array" }, "group": { "anyOf": [ { "$ref": "#/$defs/MetricGroup" }, { "type": "null" } ], "default": null, "description": "The metric group this metric belongs to." }, "is_reference_free": { "default": true, "description": "Decides whether this metric needs a reference for computation", "title": "Is Reference Free", "type": "boolean" }, "method": { "const": "prompt_injection_125m_0.7_en", "default": "prompt_injection_125m_0.7_en", "description": "The method used to compute the prompt safety risk metric.", "title": "Method", "type": "string" }, "metric_dependencies": { "default": [], "description": "Metrics that needs to be evaluated first", "items": { "$ref": "#/$defs/GenAIMetric" }, "title": "Metric Dependencies", "type": "array" }, "system_prompt": { "description": "The AI model system prompt which contains instructions to define its overall behavior.", "title": "System Prompt", "type": "string" } }, "$defs": { "GenAIMetric": { "description": "Defines the Generative AI metric interface", "properties": { "name": { "description": "The name of the metric", "title": "Metric Name", "type": "string" }, "thresholds": { "default": [], "description": "The list of thresholds", "items": { "$ref": "#/$defs/MetricThreshold" }, "title": "Thresholds", "type": "array" }, "tasks": { "description": "The task types this metric is associated with.", "items": { "$ref": "#/$defs/TaskType" }, "title": "Tasks", "type": "array" }, "group": { "anyOf": [ { "$ref": "#/$defs/MetricGroup" }, { "type": "null" } ], "default": null, "description": "The metric group this metric belongs to." }, "is_reference_free": { "default": true, "description": "Decides whether this metric needs a reference for computation", "title": "Is Reference Free", "type": "boolean" }, "method": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "The method used to compute the metric.", "title": "Method" }, "metric_dependencies": { "default": [], "description": "Metrics that needs to be evaluated first", "items": { "$ref": "#/$defs/GenAIMetric" }, "title": "Metric Dependencies", "type": "array" } }, "required": [ "name", "tasks" ], "title": "GenAIMetric", "type": "object" }, "MetricGroup": { "enum": [ "retrieval_quality", "answer_quality", "content_safety", "performance", "usage", "tool_call_quality", "readability" ], "title": "MetricGroup", "type": "string" }, "MetricThreshold": { "description": "The class that defines the threshold for a metric.", "properties": { "type": { "description": "Threshold type. One of 'lower_limit', 'upper_limit'", "enum": [ "lower_limit", "upper_limit" ], "title": "Type", "type": "string" }, "value": { "default": 0, "description": "The value of metric threshold", "title": "Threshold value", "type": "number" } }, "required": [ "type" ], "title": "MetricThreshold", "type": "object" }, "TaskType": { "description": "Supported task types for generative AI models", "enum": [ "question_answering", "classification", "summarization", "generation", "extraction", "retrieval_augmented_generation" ], "title": "TaskType", "type": "string" } }, "required": [ "system_prompt" ] }
- Fields:
- field method: Annotated[Literal['prompt_injection_125m_0.7_en'], FieldInfo(annotation=NoneType, required=False, default='prompt_injection_125m_0.7_en', title='Method', description='The method used to compute the prompt safety risk metric.')] = 'prompt_injection_125m_0.7_en'¶
The method used to compute the prompt safety risk metric.
- field name: Annotated[Literal['prompt_safety_risk'], FieldInfo(annotation=NoneType, required=False, default='prompt_safety_risk', title='Name', description='The prompt safety risk metric name.', frozen=True)] = 'prompt_safety_risk'¶
The prompt safety risk metric name.
- field system_prompt: Annotated[str, FieldInfo(annotation=NoneType, required=True, title='System Prompt', description='The AI model system prompt which contains instructions to define its overall behavior.')] [Required]¶
The AI model system prompt which contains instructions to define its overall behavior.
- field tasks: Annotated[list[TaskType], FieldInfo(annotation=NoneType, required=False, default=['question_answering', 'classification', 'summarization', 'generation', 'extraction', 'retrieval_augmented_generation'], title='Tasks', description='The list of supported tasks.', frozen=True)] = ['question_answering', 'classification', 'summarization', 'generation', 'extraction', 'retrieval_augmented_generation']¶
The list of supported tasks.
- field thresholds: Annotated[list[MetricThreshold], FieldInfo(annotation=NoneType, required=False, default=[MetricThreshold(type='lower_limit', value=0.7)], title='Thresholds', description='The metric thresholds.')] = [MetricThreshold(type='lower_limit', value=0.7)]¶
The metric thresholds.
- evaluate(data: DataFrame | dict, configuration: GenAIConfiguration, **kwargs)¶
- async evaluate_async(data: DataFrame | dict, configuration: GenAIConfiguration, **kwargs) list[AggregateMetricResult]¶
- model_post_init(context: Any, /) None¶
We need to both initialize private attributes and call the user-defined model_post_init method.
- pydantic model ibm_watsonx_gov.metrics.prompt_safety_risk.prompt_safety_risk_metric.PromptSafetyRiskResult¶
Bases:
RecordMetricResultShow JSON schema
{ "title": "PromptSafetyRiskResult", "type": "object", "properties": { "name": { "default": "prompt_safety_risk", "title": "Name", "type": "string" }, "method": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "The method used to compute this metric result.", "examples": [ "token_recall" ], "title": "Method" }, "provider": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "The provider used to compute this metric result.", "title": "Provider" }, "value": { "anyOf": [ { "type": "number" }, { "type": "string" }, { "type": "boolean" }, { "type": "null" } ], "description": "The metric value.", "title": "Value" }, "errors": { "anyOf": [ { "items": { "$ref": "#/$defs/Error" }, "type": "array" }, { "type": "null" } ], "default": null, "description": "The list of error messages", "title": "Errors" }, "additional_info": { "anyOf": [ { "type": "object" }, { "type": "null" } ], "default": null, "description": "The additional information about the metric result.", "title": "Additional Info" }, "group": { "$ref": "#/$defs/MetricGroup", "default": "content_safety" }, "thresholds": { "default": [], "description": "The metric thresholds", "items": { "$ref": "#/$defs/MetricThreshold" }, "title": "Thresholds", "type": "array" }, "record_id": { "description": "The record identifier.", "examples": [ "record1" ], "title": "Record Id", "type": "string" }, "record_timestamp": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "The record timestamp.", "examples": [ "2025-01-01T00:00:00.000000Z" ], "title": "Record Timestamp" } }, "$defs": { "Error": { "properties": { "code": { "description": "The error code", "title": "Code", "type": "string" }, "message_en": { "description": "The error message in English.", "title": "Message En", "type": "string" }, "parameters": { "default": [], "description": "The list of parameters to construct the message in a different locale.", "items": {}, "title": "Parameters", "type": "array" } }, "required": [ "code", "message_en" ], "title": "Error", "type": "object" }, "MetricGroup": { "enum": [ "retrieval_quality", "answer_quality", "content_safety", "performance", "usage", "tool_call_quality", "readability" ], "title": "MetricGroup", "type": "string" }, "MetricThreshold": { "description": "The class that defines the threshold for a metric.", "properties": { "type": { "description": "Threshold type. One of 'lower_limit', 'upper_limit'", "enum": [ "lower_limit", "upper_limit" ], "title": "Type", "type": "string" }, "value": { "default": 0, "description": "The value of metric threshold", "title": "Threshold value", "type": "number" } }, "required": [ "type" ], "title": "MetricThreshold", "type": "object" } }, "required": [ "value", "record_id" ] }
- Config:
arbitrary_types_allowed: bool = True
use_enum_values: bool = True
- Fields:
- field group: MetricGroup = MetricGroup.CONTENT_SAFETY¶
- field name: str = 'prompt_safety_risk'¶