Class: BenchmarkMetadataCard
Benchmark metadata cards offer a standardized way to document LLM benchmarks clearly and transparently. Inspired by Model Cards and Datasheets, Benchmark metadata cards help researchers and practitioners understand exactly what benchmarks test, how they relate to real-world risks, and how to interpret their results responsibly. This is an implementation of the design set out in BenchmarkCards: Large Language Model and Risk Reporting (https://doi.org/10.48550/arXiv.2410.12974)
URI: [nexus:benchmarkmetadatacard](https://w3id.org/ai-atlas-nexus/benchmarkmetadatacard)
[AiEval](AiEval.md) | A relationship where a BenchmarkMetadataCard describes an AI evaluation (benc... | direct | | [hasDataType](hasDataType.md) | \*
[String](String.md) | The type of data used in the benchmark (e | direct | | [hasDomains](hasDomains.md) | \*
[String](String.md) | The specific domains or areas where the benchmark is applied (e | direct | | [hasLanguages](hasLanguages.md) | \*
[String](String.md) | The languages included in the dataset used by the benchmark (e | direct | | [hasSimilarBenchmarks](hasSimilarBenchmarks.md) | \*
[String](String.md) | Benchmarks that are closely related in terms of goals or data type | direct | | [hasResources](hasResources.md) | \*
[String](String.md) | Links to relevant resources, such as repositories or papers related to the be... | direct | | [hasGoal](hasGoal.md) | 0..1
[String](String.md) | The specific goal or primary use case the benchmark is designed for | direct | | [hasAudience](hasAudience.md) | \*
[String](String.md) | The intended audience, such as researchers, developers, policymakers, etc | direct | | [hasTasks](hasTasks.md) | \*
[String](String.md) | The tasks or evaluations the benchmark is intended to assess | direct | | [hasLimitations](hasLimitations.md) | \*
[String](String.md) | Limitations in evaluating or addressing risks, such as gaps in demographic co... | direct | | [hasOutOfScopeUses](hasOutOfScopeUses.md) | \*
[String](String.md) | Use cases where the benchmark is not designed to be applied and could give mi... | direct | | [hasDataSource](hasDataSource.md) | \*
[String](String.md) | The origin or source of the data used in the benchmark (e | direct | | [hasDataSize](hasDataSize.md) | 0..1
[String](String.md) | The size of the dataset, including the number of data points or examples | direct | | [hasDataFormat](hasDataFormat.md) | \*
[String](String.md) | The structure and modality of the data (e | direct | | [hasAnnotation](hasAnnotation.md) | \*
[String](String.md) | The process used to annotate or label the dataset, including who or what perf... | direct | | [hasMethods](hasMethods.md) | \*
[String](String.md) | The evaluation techniques applied within the benchmark | direct | | [hasMetrics](hasMetrics.md) | \*
[String](String.md) | The specific performance metrics used to assess models (e | direct | | [hasCalculation](hasCalculation.md) | \*
[String](String.md) | The way metrics are computed based on model outputs and the benchmark data | direct | | [hasInterpretation](hasInterpretation.md) | \*
[String](String.md) | How users should interpret the scores or results from the metrics | direct | | [hasBaselineResults](hasBaselineResults.md) | \*
[String](String.md) | The results of well-known or widely used models to give context to new perfor... | direct | | [hasValidation](hasValidation.md) | \*
[String](String.md) | Measures taken to ensure that the benchmark provides valid and reliable evalu... | direct | | [hasRelatedRisk](hasRelatedRisk.md) | \*
[RiskConcept](RiskConcept.md) | A relationship where an entity relates to a risk | direct | | [hasDemographicAnalysis](hasDemographicAnalysis.md) | \*
[String](String.md) | How the benchmark evaluates performance across different demographic groups (... | direct | | [hasConsiderationPrivacyAndAnonymity](hasConsiderationPrivacyAndAnonymity.md) | \*
[String](String.md) | How any personal or sensitive data is handled and whether any anonymization t... | direct | | [hasLicense](hasLicense.md) | 0..1
[License](License.md) | Indicates licenses associated with a resource | direct | | [hasConsiderationConsentProcedures](hasConsiderationConsentProcedures.md) | \*
[String](String.md) | Information on how consent was obtained (if applicable), especially for datas... | direct | | [hasConsiderationComplianceWithRegulations](hasConsiderationComplianceWithRegulations.md) | \*
[String](String.md) | Compliance with relevant legal or ethical regulations (if applicable) | direct | | [hasDocumentation](hasDocumentation.md) | \*
[Documentation](Documentation.md) | Indicates documentation associated with an entity | direct | | [name](name.md) | 0..1
[String](String.md) | The official name of the benchmark | direct | | [overview](overview.md) | 0..1
[String](String.md) | A brief description of the benchmark's main goals and scope | direct | | [type](type.md) | 0..1
[String](String.md) | | direct | | [id](id.md) | 1
[String](String.md) | A unique identifier to this instance of the model element | [Entity](Entity.md) | | [description](description.md) | 0..1
[String](String.md) | The description of an entity | [Entity](Entity.md) | | [url](url.md) | 0..1
[Uri](Uri.md) | An optional URL associated with this instance | [Entity](Entity.md) | | [dateCreated](dateCreated.md) | 0..1
[Date](Date.md) | The date on which the entity was created | [Entity](Entity.md) | | [dateModified](dateModified.md) | 0..1
[Date](Date.md) | The date on which the entity was most recently modified | [Entity](Entity.md) | | [exact_mappings](exact_mappings.md) | \*
[Any](Any.md) | The property is used to link two concepts, indicating a high degree of confid... | [Entity](Entity.md) | | [close_mappings](close_mappings.md) | \*
[Any](Any.md) | The property is used to link two concepts that are sufficiently similar that ... | [Entity](Entity.md) | | [related_mappings](related_mappings.md) | \*
[Any](Any.md) | The property skos:relatedMatch is used to state an associative mapping link b... | [Entity](Entity.md) | | [narrow_mappings](narrow_mappings.md) | \*
[Any](Any.md) | The property is used to state a hierarchical mapping link between two concept... | [Entity](Entity.md) | | [broad_mappings](broad_mappings.md) | \*
[Any](Any.md) | The property is used to state a hierarchical mapping link between two concept... | [Entity](Entity.md) | | [isCategorizedAs](isCategorizedAs.md) | \*
[Any](Any.md) | A relationship where an entity has been deemed to be categorized | [Entity](Entity.md) | | [hasLifecycleStatus](hasLifecycleStatus.md) | 0..1
[LifecycleStatus](LifecycleStatus.md) | The editorial / publication lifecycle state of this entity | [Entity](Entity.md) | | [notes](notes.md) | \*
[String](String.md) | Free-text editorial notes, source breadcrumbs, or build-time provenance that ... | [Entity](Entity.md) | ## Usages | used by | used in | type | used | | ------------------------------------------------- | --------------------------------------------------- | ------ | ------------------------------------------------- | | [Container](Container.md) | [benchmarkmetadatacards](benchmarkmetadatacards.md) | range | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [AiEval](AiEval.md) | [hasBenchmarkMetadata](hasBenchmarkMetadata.md) | range | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | [describesAiEval](describesAiEval.md) | domain | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | [hasRelatedRisk](hasRelatedRisk.md) | domain | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [Question](Question.md) | [hasBenchmarkMetadata](hasBenchmarkMetadata.md) | range | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [Questionnaire](Questionnaire.md) | [hasBenchmarkMetadata](hasBenchmarkMetadata.md) | range | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | ## Identifier and Mapping Information ### Schema Source - from schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology ## Mappings | Mapping Type | Mapped Value | | ------------ | --------------------------- | | self | nexus:benchmarkmetadatacard | | native | nexus:BenchmarkMetadataCard | ## LinkML Source ### Direct
classDiagram
class BenchmarkMetadataCard
click BenchmarkMetadataCard href "../BenchmarkMetadataCard/"
Entity <|-- BenchmarkMetadataCard
click Entity href "../Entity/"
BenchmarkMetadataCard : broad_mappings
BenchmarkMetadataCard --> "*" Any : broad_mappings
click Any href "../Any/"
BenchmarkMetadataCard : close_mappings
BenchmarkMetadataCard --> "*" Any : close_mappings
click Any href "../Any/"
BenchmarkMetadataCard : dateCreated
BenchmarkMetadataCard : dateModified
BenchmarkMetadataCard : describesAiEval
BenchmarkMetadataCard --> "*" AiEval : describesAiEval
click AiEval href "../AiEval/"
BenchmarkMetadataCard : description
BenchmarkMetadataCard : exact_mappings
BenchmarkMetadataCard --> "*" Any : exact_mappings
click Any href "../Any/"
BenchmarkMetadataCard : hasAnnotation
BenchmarkMetadataCard : hasAudience
BenchmarkMetadataCard : hasBaselineResults
BenchmarkMetadataCard : hasCalculation
BenchmarkMetadataCard : hasConsiderationComplianceWithRegulations
BenchmarkMetadataCard : hasConsiderationConsentProcedures
BenchmarkMetadataCard : hasConsiderationPrivacyAndAnonymity
BenchmarkMetadataCard : hasDataFormat
BenchmarkMetadataCard : hasDataSize
BenchmarkMetadataCard : hasDataSource
BenchmarkMetadataCard : hasDataType
BenchmarkMetadataCard : hasDemographicAnalysis
BenchmarkMetadataCard : hasDocumentation
BenchmarkMetadataCard --> "*" Documentation : hasDocumentation
click Documentation href "../Documentation/"
BenchmarkMetadataCard : hasDomains
BenchmarkMetadataCard : hasGoal
BenchmarkMetadataCard : hasInterpretation
BenchmarkMetadataCard : hasLanguages
BenchmarkMetadataCard : hasLicense
BenchmarkMetadataCard --> "0..1" License : hasLicense
click License href "../License/"
BenchmarkMetadataCard : hasLifecycleStatus
BenchmarkMetadataCard --> "0..1" LifecycleStatus : hasLifecycleStatus
click LifecycleStatus href "../LifecycleStatus/"
BenchmarkMetadataCard : hasLimitations
BenchmarkMetadataCard : hasMethods
BenchmarkMetadataCard : hasMetrics
BenchmarkMetadataCard : hasOutOfScopeUses
BenchmarkMetadataCard : hasRelatedRisk
BenchmarkMetadataCard --> "*" RiskConcept : hasRelatedRisk
click RiskConcept href "../RiskConcept/"
BenchmarkMetadataCard : hasResources
BenchmarkMetadataCard : hasSimilarBenchmarks
BenchmarkMetadataCard : hasTasks
BenchmarkMetadataCard : hasValidation
BenchmarkMetadataCard : id
BenchmarkMetadataCard : isCategorizedAs
BenchmarkMetadataCard --> "*" Any : isCategorizedAs
click Any href "../Any/"
BenchmarkMetadataCard : name
BenchmarkMetadataCard : narrow_mappings
BenchmarkMetadataCard --> "*" Any : narrow_mappings
click Any href "../Any/"
BenchmarkMetadataCard : notes
BenchmarkMetadataCard : overview
BenchmarkMetadataCard : related_mappings
BenchmarkMetadataCard --> "*" Any : related_mappings
click Any href "../Any/"
BenchmarkMetadataCard : type
BenchmarkMetadataCard : url
[AiEval](AiEval.md) | A relationship where a BenchmarkMetadataCard describes an AI evaluation (benc... | direct | | [hasDataType](hasDataType.md) | \*
[String](String.md) | The type of data used in the benchmark (e | direct | | [hasDomains](hasDomains.md) | \*
[String](String.md) | The specific domains or areas where the benchmark is applied (e | direct | | [hasLanguages](hasLanguages.md) | \*
[String](String.md) | The languages included in the dataset used by the benchmark (e | direct | | [hasSimilarBenchmarks](hasSimilarBenchmarks.md) | \*
[String](String.md) | Benchmarks that are closely related in terms of goals or data type | direct | | [hasResources](hasResources.md) | \*
[String](String.md) | Links to relevant resources, such as repositories or papers related to the be... | direct | | [hasGoal](hasGoal.md) | 0..1
[String](String.md) | The specific goal or primary use case the benchmark is designed for | direct | | [hasAudience](hasAudience.md) | \*
[String](String.md) | The intended audience, such as researchers, developers, policymakers, etc | direct | | [hasTasks](hasTasks.md) | \*
[String](String.md) | The tasks or evaluations the benchmark is intended to assess | direct | | [hasLimitations](hasLimitations.md) | \*
[String](String.md) | Limitations in evaluating or addressing risks, such as gaps in demographic co... | direct | | [hasOutOfScopeUses](hasOutOfScopeUses.md) | \*
[String](String.md) | Use cases where the benchmark is not designed to be applied and could give mi... | direct | | [hasDataSource](hasDataSource.md) | \*
[String](String.md) | The origin or source of the data used in the benchmark (e | direct | | [hasDataSize](hasDataSize.md) | 0..1
[String](String.md) | The size of the dataset, including the number of data points or examples | direct | | [hasDataFormat](hasDataFormat.md) | \*
[String](String.md) | The structure and modality of the data (e | direct | | [hasAnnotation](hasAnnotation.md) | \*
[String](String.md) | The process used to annotate or label the dataset, including who or what perf... | direct | | [hasMethods](hasMethods.md) | \*
[String](String.md) | The evaluation techniques applied within the benchmark | direct | | [hasMetrics](hasMetrics.md) | \*
[String](String.md) | The specific performance metrics used to assess models (e | direct | | [hasCalculation](hasCalculation.md) | \*
[String](String.md) | The way metrics are computed based on model outputs and the benchmark data | direct | | [hasInterpretation](hasInterpretation.md) | \*
[String](String.md) | How users should interpret the scores or results from the metrics | direct | | [hasBaselineResults](hasBaselineResults.md) | \*
[String](String.md) | The results of well-known or widely used models to give context to new perfor... | direct | | [hasValidation](hasValidation.md) | \*
[String](String.md) | Measures taken to ensure that the benchmark provides valid and reliable evalu... | direct | | [hasRelatedRisk](hasRelatedRisk.md) | \*
[RiskConcept](RiskConcept.md) | A relationship where an entity relates to a risk | direct | | [hasDemographicAnalysis](hasDemographicAnalysis.md) | \*
[String](String.md) | How the benchmark evaluates performance across different demographic groups (... | direct | | [hasConsiderationPrivacyAndAnonymity](hasConsiderationPrivacyAndAnonymity.md) | \*
[String](String.md) | How any personal or sensitive data is handled and whether any anonymization t... | direct | | [hasLicense](hasLicense.md) | 0..1
[License](License.md) | Indicates licenses associated with a resource | direct | | [hasConsiderationConsentProcedures](hasConsiderationConsentProcedures.md) | \*
[String](String.md) | Information on how consent was obtained (if applicable), especially for datas... | direct | | [hasConsiderationComplianceWithRegulations](hasConsiderationComplianceWithRegulations.md) | \*
[String](String.md) | Compliance with relevant legal or ethical regulations (if applicable) | direct | | [hasDocumentation](hasDocumentation.md) | \*
[Documentation](Documentation.md) | Indicates documentation associated with an entity | direct | | [name](name.md) | 0..1
[String](String.md) | The official name of the benchmark | direct | | [overview](overview.md) | 0..1
[String](String.md) | A brief description of the benchmark's main goals and scope | direct | | [type](type.md) | 0..1
[String](String.md) | | direct | | [id](id.md) | 1
[String](String.md) | A unique identifier to this instance of the model element | [Entity](Entity.md) | | [description](description.md) | 0..1
[String](String.md) | The description of an entity | [Entity](Entity.md) | | [url](url.md) | 0..1
[Uri](Uri.md) | An optional URL associated with this instance | [Entity](Entity.md) | | [dateCreated](dateCreated.md) | 0..1
[Date](Date.md) | The date on which the entity was created | [Entity](Entity.md) | | [dateModified](dateModified.md) | 0..1
[Date](Date.md) | The date on which the entity was most recently modified | [Entity](Entity.md) | | [exact_mappings](exact_mappings.md) | \*
[Any](Any.md) | The property is used to link two concepts, indicating a high degree of confid... | [Entity](Entity.md) | | [close_mappings](close_mappings.md) | \*
[Any](Any.md) | The property is used to link two concepts that are sufficiently similar that ... | [Entity](Entity.md) | | [related_mappings](related_mappings.md) | \*
[Any](Any.md) | The property skos:relatedMatch is used to state an associative mapping link b... | [Entity](Entity.md) | | [narrow_mappings](narrow_mappings.md) | \*
[Any](Any.md) | The property is used to state a hierarchical mapping link between two concept... | [Entity](Entity.md) | | [broad_mappings](broad_mappings.md) | \*
[Any](Any.md) | The property is used to state a hierarchical mapping link between two concept... | [Entity](Entity.md) | | [isCategorizedAs](isCategorizedAs.md) | \*
[Any](Any.md) | A relationship where an entity has been deemed to be categorized | [Entity](Entity.md) | | [hasLifecycleStatus](hasLifecycleStatus.md) | 0..1
[LifecycleStatus](LifecycleStatus.md) | The editorial / publication lifecycle state of this entity | [Entity](Entity.md) | | [notes](notes.md) | \*
[String](String.md) | Free-text editorial notes, source breadcrumbs, or build-time provenance that ... | [Entity](Entity.md) | ## Usages | used by | used in | type | used | | ------------------------------------------------- | --------------------------------------------------- | ------ | ------------------------------------------------- | | [Container](Container.md) | [benchmarkmetadatacards](benchmarkmetadatacards.md) | range | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [AiEval](AiEval.md) | [hasBenchmarkMetadata](hasBenchmarkMetadata.md) | range | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | [describesAiEval](describesAiEval.md) | domain | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | [hasRelatedRisk](hasRelatedRisk.md) | domain | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [Question](Question.md) | [hasBenchmarkMetadata](hasBenchmarkMetadata.md) | range | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | | [Questionnaire](Questionnaire.md) | [hasBenchmarkMetadata](hasBenchmarkMetadata.md) | range | [BenchmarkMetadataCard](BenchmarkMetadataCard.md) | ## Identifier and Mapping Information ### Schema Source - from schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology ## Mappings | Mapping Type | Mapped Value | | ------------ | --------------------------- | | self | nexus:benchmarkmetadatacard | | native | nexus:BenchmarkMetadataCard | ## LinkML Source ### Direct
```yaml
name: BenchmarkMetadataCard
description: 'Benchmark metadata cards offer a standardized way to document LLM benchmarks
clearly and transparently. Inspired by Model Cards and Datasheets, Benchmark metadata
cards help researchers and practitioners understand exactly what benchmarks test,
how they relate to real-world risks, and how to interpret their results responsibly.
This is an implementation of the design set out in BenchmarkCards: Large Language
Model and Risk Reporting (https://doi.org/10.48550/arXiv.2410.12974)'
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
is_a: Entity
slots:
- describesAiEval
- hasDataType
- hasDomains
- hasLanguages
- hasSimilarBenchmarks
- hasResources
- hasGoal
- hasAudience
- hasTasks
- hasLimitations
- hasOutOfScopeUses
- hasDataSource
- hasDataSize
- hasDataFormat
- hasAnnotation
- hasMethods
- hasMetrics
- hasCalculation
- hasInterpretation
- hasBaselineResults
- hasValidation
- hasRelatedRisk
- hasDemographicAnalysis
- hasConsiderationPrivacyAndAnonymity
- hasLicense
- hasConsiderationConsentProcedures
- hasConsiderationComplianceWithRegulations
- hasDocumentation
slot_usage:
hasRelatedRisk:
name: hasRelatedRisk
domain: BenchmarkMetadataCard
range: RiskConcept
attributes:
name:
name: name
description: The official name of the benchmark.
from_schema: https://w3id.org/ai-atlas-nexus/ai_eval
domain_of:
- Entity
- BenchmarkMetadataCard
overview:
name: overview
description: A brief description of the benchmark's main goals and scope.
from_schema: https://w3id.org/ai-atlas-nexus/ai_eval
rank: 1000
domain_of:
- BenchmarkMetadataCard
type:
name: type
from_schema: https://w3id.org/ai-atlas-nexus/ai_eval
designates_type: true
domain_of:
- Organization
- Vocabulary
- Taxonomy
- Concept
- Control
- Group
- Entry
- Policy
- Rule
- Permission
- Prohibition
- Obligation
- Recommendation
- Certification
- EvaluationStandard
- BenchmarkMetadataCard
- ControlActivity
- ControlActivityPermission
- ControlActivityProhibition
- ControlActivityObligation
- ControlActivityRecommendation
- Requirement
range: string
class_uri: nexus:benchmarkmetadatacard
````
### Induced
```yaml
name: BenchmarkMetadataCard
description: 'Benchmark metadata cards offer a standardized way to document LLM benchmarks
clearly and transparently. Inspired by Model Cards and Datasheets, Benchmark metadata
cards help researchers and practitioners understand exactly what benchmarks test,
how they relate to real-world risks, and how to interpret their results responsibly.
This is an implementation of the design set out in BenchmarkCards: Large Language
Model and Risk Reporting (https://doi.org/10.48550/arXiv.2410.12974)'
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
is_a: Entity
slot_usage:
hasRelatedRisk:
name: hasRelatedRisk
domain: BenchmarkMetadataCard
range: RiskConcept
attributes:
name:
name: name
description: The official name of the benchmark.
from_schema: https://w3id.org/ai-atlas-nexus/ai_eval
owner: BenchmarkMetadataCard
domain_of:
- Entity
- BenchmarkMetadataCard
range: string
overview:
name: overview
description: A brief description of the benchmark's main goals and scope.
from_schema: https://w3id.org/ai-atlas-nexus/ai_eval
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
type:
name: type
from_schema: https://w3id.org/ai-atlas-nexus/ai_eval
designates_type: true
owner: BenchmarkMetadataCard
domain_of:
- Organization
- Vocabulary
- Taxonomy
- Concept
- Control
- Group
- Entry
- Policy
- Rule
- Permission
- Prohibition
- Obligation
- Recommendation
- Certification
- EvaluationStandard
- BenchmarkMetadataCard
- ControlActivity
- ControlActivityPermission
- ControlActivityProhibition
- ControlActivityObligation
- ControlActivityRecommendation
- Requirement
range: string
describesAiEval:
name: describesAiEval
description: A relationship where a BenchmarkMetadataCard describes an AI evaluation
(benchmark).
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
domain: BenchmarkMetadataCard
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
inverse: hasBenchmarkMetadata
range: AiEval
multivalued: true
inlined: false
hasDataType:
name: hasDataType
description: The type of data used in the benchmark (e.g., text, images, or multi-modal)
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasDomains:
name: hasDomains
description: The specific domains or areas where the benchmark is applied (e.g.,
natural language processing, computer vision).
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasLanguages:
name: hasLanguages
description: The languages included in the dataset used by the benchmark (e.g.,
English, multilingual).
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasSimilarBenchmarks:
name: hasSimilarBenchmarks
description: Benchmarks that are closely related in terms of goals or data type.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasResources:
name: hasResources
description: Links to relevant resources, such as repositories or papers related
to the benchmark.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasGoal:
name: hasGoal
description: The specific goal or primary use case the benchmark is designed for.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
hasAudience:
name: hasAudience
description: The intended audience, such as researchers, developers, policymakers,
etc.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasTasks:
name: hasTasks
description: The tasks or evaluations the benchmark is intended to assess.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- AiEval
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
inlined: false
hasLimitations:
name: hasLimitations
description: Limitations in evaluating or addressing risks, such as gaps in demographic
coverage or specific domains.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasOutOfScopeUses:
name: hasOutOfScopeUses
description: Use cases where the benchmark is not designed to be applied and could
give misleading results.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasDataSource:
name: hasDataSource
description: The origin or source of the data used in the benchmark (e.g., curated
datasets, user submissions).
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasDataSize:
name: hasDataSize
description: The size of the dataset, including the number of data points or examples.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
hasDataFormat:
name: hasDataFormat
description: The structure and modality of the data (e.g., sentence pairs, question-answer
format, tabular data).
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasAnnotation:
name: hasAnnotation
description: The process used to annotate or label the dataset, including who
or what performed the annotations (e.g., human annotators, automated processes).
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasMethods:
name: hasMethods
description: The evaluation techniques applied within the benchmark.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasMetrics:
name: hasMetrics
description: The specific performance metrics used to assess models (e.g., accuracy,
F1 score, precision, recall).
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- EveryEvalAIResult
- BenchmarkMetadataCard
range: string
multivalued: true
hasCalculation:
name: hasCalculation
description: The way metrics are computed based on model outputs and the benchmark
data.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasInterpretation:
name: hasInterpretation
description: How users should interpret the scores or results from the metrics.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasBaselineResults:
name: hasBaselineResults
description: The results of well-known or widely used models to give context to
new performance scores.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasValidation:
name: hasValidation
description: Measures taken to ensure that the benchmark provides valid and reliable
evaluations.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasRelatedRisk:
name: hasRelatedRisk
description: A relationship where an entity relates to a risk
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
domain: BenchmarkMetadataCard
owner: BenchmarkMetadataCard
domain_of:
- Term
- LLMQuestionPolicy
- Action
- AiSystem
- AiEval
- EveryEvalAIResult
- BenchmarkMetadataCard
- Adapter
- LLMIntrinsic
range: RiskConcept
multivalued: true
inlined: false
hasDemographicAnalysis:
name: hasDemographicAnalysis
description: How the benchmark evaluates performance across different demographic
groups (e.g., gender, race).
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasConsiderationPrivacyAndAnonymity:
name: hasConsiderationPrivacyAndAnonymity
description: How any personal or sensitive data is handled and whether any anonymization
techniques are applied.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasLicense:
name: hasLicense
description: Indicates licenses associated with a resource
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: airo:hasLicense
owner: BenchmarkMetadataCard
domain_of:
- Dataset
- Documentation
- Vocabulary
- Taxonomy
- RiskTaxonomy
- RiskControlGroupTaxonomy
- BaseAi
- AiTaskTaxonomy
- AiEval
- BenchmarkMetadataCard
- Adapter
range: License
hasConsiderationConsentProcedures:
name: hasConsiderationConsentProcedures
description: Information on how consent was obtained (if applicable), especially
for datasets involving personal data.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasConsiderationComplianceWithRegulations:
name: hasConsiderationComplianceWithRegulations
description: Compliance with relevant legal or ethical regulations (if applicable).
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
owner: BenchmarkMetadataCard
domain_of:
- BenchmarkMetadataCard
range: string
multivalued: true
hasDocumentation:
name: hasDocumentation
description: Indicates documentation associated with an entity.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: airo:hasDocumentation
owner: BenchmarkMetadataCard
domain_of:
- Dataset
- Vocabulary
- Taxonomy
- Concept
- Group
- Entry
- Term
- Principle
- Rule
- RiskTaxonomy
- RiskControlGroupTaxonomy
- Action
- BaseAi
- LargeLanguageModelFamily
- AiTaskTaxonomy
- AiEval
- EveryEvalAIResult
- BenchmarkMetadataCard
- ThirdPartyEvaluationEngagement
- Adapter
- LLMIntrinsic
range: Documentation
multivalued: true
inlined: false
id:
name: id
description: A unique identifier to this instance of the model element. Example
identifiers include UUID, URI, URN, etc.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: schema:identifier
identifier: true
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: string
required: true
description:
name: description
description: The description of an entity
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: schema:description
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: string
url:
name: url
description: An optional URL associated with this instance.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: schema:url
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: uri
dateCreated:
name: dateCreated
description: The date on which the entity was created.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: schema:dateCreated
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: date
required: false
dateModified:
name: dateModified
description: The date on which the entity was most recently modified.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: schema:dateModified
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: date
required: false
exact_mappings:
name: exact_mappings
description: The property is used to link two concepts, indicating a high degree
of confidence that the concepts can be used interchangeably across a wide range
of information retrieval applications
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: skos:exactMatch
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: Any
multivalued: true
inlined: false
close_mappings:
name: close_mappings
description: The property is used to link two concepts that are sufficiently similar
that they can be used interchangeably in some information retrieval applications.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: skos:closeMatch
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: Any
multivalued: true
inlined: false
related_mappings:
name: related_mappings
description: The property skos:relatedMatch is used to state an associative mapping
link between two concepts.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: skos:relatedMatch
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: Any
multivalued: true
inlined: false
narrow_mappings:
name: narrow_mappings
description: The property is used to state a hierarchical mapping link between
two concepts, indicating that the concept linked to, is a narrower concept than
the originating concept.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: skos:narrowMatch
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: Any
multivalued: true
inlined: false
broad_mappings:
name: broad_mappings
description: The property is used to state a hierarchical mapping link between
two concepts, indicating that the concept linked to, is a broader concept than
the originating concept.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: skos:broadMatch
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: Any
multivalued: true
inlined: false
isCategorizedAs:
name: isCategorizedAs
description: A relationship where an entity has been deemed to be categorized
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: nexus:isCategorizedAs
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: Any
multivalued: true
inlined: false
hasLifecycleStatus:
name: hasLifecycleStatus
description: The editorial / publication lifecycle state of this entity. Distinct
from AiLifecyclePhase, which describes an AI system's runtime evolution rather
than the editorial workflow of a catalogued entry.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
aliases:
- lifecycle_status
- doc_status
rank: 1000
slot_uri: adms:status
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: LifecycleStatus
notes:
name: notes
description: Free-text editorial notes, source breadcrumbs, or build-time provenance
that do not belong in the user-facing description. Opaque to consumers.
from_schema: https://w3id.org/ai-atlas-nexus/ai-risk-ontology
rank: 1000
slot_uri: skos:note
owner: BenchmarkMetadataCard
domain_of:
- Entity
range: string
recommended: false
multivalued: true
class_uri: nexus:benchmarkmetadatacard
````