AEF-1: Minimum Operating Conditions¶

AEF-1 is the AI Evaluator Forum's voluntary standard for independent third-party AI evaluations: 5 principles and 26 conditions, each either a requirement (ControlActivityObligation / ControlActivityProhibition) or a recommendation (ControlActivityRecommendation). Conditions 1.1 (technical access) and 2.4 (CoI disclosure) also have checklist sub-elements. Modelled here via the EvaluationStandard class. Source: https://aievaluatorforum.org/AEF_1_Minimum_Operating_Conditions_for_Independent_Third_Party_AI_Evaluations.pdf

In [1]:

from ai_atlas_nexus import AIAtlasNexus

ran = AIAtlasNexus()

from ai_atlas_nexus import AIAtlasNexus ran = AIAtlasNexus()

[2026-09-28 13:47:58:814] - INFO - AIAtlasNexus - Created AIAtlasNexus instance. Base_dir: None

Load AEF-1¶

In [2]:

CONDITION_CLASSES = [
    "controlactivityobligation",
    "controlactivityprohibition",
    "controlactivityrecommendation",
]

standard = ran.get_all(class_name="evaluationstandard", taxonomy="aef-1")
principles = ran.get_all(class_name="principle", taxonomy="aef-1")
conditions = {c.id: c for cls in CONDITION_CLASSES for c in ran.get_all(class_name=cls, taxonomy="aef-1")}
top_level = [conditions[rule_id] for p in principles for rule_id in p.hasRule]

print(f"standards: {len(standard)}, principles: {len(principles)}, conditions: {len(top_level)}")
print(f"requirements: {sum(c.type != 'ControlActivityRecommendation' for c in top_level)}, recommendations: {sum(c.type == 'ControlActivityRecommendation' for c in top_level)}")
print(f"hasPrinciple: {standard[0].hasPrinciple}")

CONDITION_CLASSES = [ "controlactivityobligation", "controlactivityprohibition", "controlactivityrecommendation", ] standard = ran.get_all(class_name="evaluationstandard", taxonomy="aef-1") principles = ran.get_all(class_name="principle", taxonomy="aef-1") conditions = {c.id: c for cls in CONDITION_CLASSES for c in ran.get_all(class_name=cls, taxonomy="aef-1")} top_level = [conditions[rule_id] for p in principles for rule_id in p.hasRule] print(f"standards: {len(standard)}, principles: {len(principles)}, conditions: {len(top_level)}") print(f"requirements: {sum(c.type != 'ControlActivityRecommendation' for c in top_level)}, recommendations: {sum(c.type == 'ControlActivityRecommendation' for c in top_level)}") print(f"hasPrinciple: {standard[0].hasPrinciple}")

standards: 1, principles: 5, conditions: 26
requirements: 15, recommendations: 11
hasPrinciple: ['aef1-principle-access', 'aef1-principle-coi', 'aef1-principle-autonomy', 'aef1-principle-transparency', 'aef1-principle-sensitive-info']

Conditions grouped by principle¶

Each principle lists its conditions via hasRule; conditions with checklist sub-elements list those via hasRule too.

In [3]:

for p in principles:
    print(p.name)
    for rule_id in p.hasRule:
        condition = conditions[rule_id]
        kind = "Recommendation" if condition.type == "ControlActivityRecommendation" else "Requirement"
        print(f"  {condition.name} [{kind}]")
        for sub_id in condition.hasRule or []:
            print(f"    {conditions[sub_id].name}")

for p in principles: print(p.name) for rule_id in p.hasRule: condition = conditions[rule_id] kind = "Recommendation" if condition.type == "ControlActivityRecommendation" else "Requirement" print(f" {condition.name} [{kind}]") for sub_id in condition.hasRule or []: print(f" {conditions[sub_id].name}")

Principle 1: Sufficient Access and Resources
  1.1 Technical access [Requirement]
    1.1.1 Query access
    1.1.2 Scaffolding
    1.1.3 Safeguard exemptions
    1.1.4 Intermediate system states
    1.1.5 Finetuning
    1.1.6 Model weights
    1.1.7 Other tools
    1.1.8 User data
  1.2 Information [Recommendation]
  1.3 Computational resources [Requirement]
  1.4 Time [Requirement]
  1.5 Safe harbor [Recommendation]
Principle 2: Minimized Conflicts of Interest
  2.1 Contingent compensation [Requirement]
  2.2 Organizational control [Requirement]
  2.3 CoI policy [Requirement]
  2.4 CoI disclosure [Requirement]
    2.4.1 Paid by provider or competitor
    2.4.2 Funding from provider or competitor
    2.4.3 Provider equity in evaluator
    2.4.4 Evaluator staff equity in provider
    2.4.5 Dual employment
    2.4.6 Other conflicts of interest
  2.5 Recusals [Requirement]
  2.6 Agreements [Recommendation]
Principle 3: Analytic Autonomy
  3.1 Scoping [Recommendation]
  3.2 Evaluation autonomy [Requirement]
  3.3 Direct access [Recommendation]
  3.4 Editorial control [Requirement]
Principle 4: Transparent Methods and Results
  4.1 Methodological transparency [Requirement]
  4.2 Disclosure rights [Recommendation]
  4.3 No contingent release [Recommendation]
  4.4 No misrepresentation [Recommendation]
  4.5 Timely disclosure [Recommendation]
  4.6 Redactions [Requirement]
  4.7 Redaction disclaimer [Recommendation]
Principle 5: Protection of Sensitive Information
  5.1 Publication terms [Requirement]
  5.2 Evaluation integrity [Recommendation]
  5.3 Protecting confidential information [Requirement]
  5.4 Responsible disclosure policy [Requirement]

Look up a single condition¶

In [4]:

ran.get_by_id(class_name="controlactivityobligation", identifier="aef1-ctrl-access-1")

ran.get_by_id(class_name="controlactivityobligation", identifier="aef1-ctrl-access-1")

Out[4]:

ControlActivityObligation(id='aef1-ctrl-access-1', name='1.1 Technical access', description='Requirement: The evaluator secured sufficient technical access to assess the specific system characteristics being evaluated. Of the sub-elements, only query access (1.1.1) is required; the rest may be important to a valid evaluation but depend on the nature of the evaluation methods.', url=None, dateCreated=None, dateModified=None, exact_mappings=None, close_mappings=None, related_mappings=['eu-gpai-cop-ss-appendix-3-4'], narrow_mappings=None, broad_mappings=None, isCategorizedAs=None, hasLifecycleStatus=None, notes=None, isDefinedByTaxonomy='aef-1', hasRule=['aef1-ctrl-access-1-1', 'aef1-ctrl-access-1-2', 'aef1-ctrl-access-1-3', 'aef1-ctrl-access-1-4', 'aef1-ctrl-access-1-5', 'aef1-ctrl-access-1-6', 'aef1-ctrl-access-1-7', 'aef1-ctrl-access-1-8'], hasDocumentation=['AEF-1-Dec-2025'], type='ControlActivityObligation', hasControlApplication=None, hasEvidenceCategory=None, hasTypicalLocation=None, isApplicableToCapability=None, hasRequirement=None, hasRequirementType=None, hasTypicalEvidence=None)

Motivating use case¶

A financial services firm (the system provider) engages an independent evaluator to evaluate its credit-risk LLM. The ThirdPartyEvaluationEngagement records the evaluator, the system provider, the AI evaluated, and the benchmarks run (usesEvaluation, here a fairness benchmark and a prompt injection benchmark) with their results (hasEvaluation).

The engagement also carries the completed AEF-1 checklist (hasStandardConformance): one ConditionAssessment per condition with an outcome (FULFILLED, NOT_FULFILLED, ALTERNATIVE_MEANS, NOT_APPLICABLE, NOT_ASSESSED), evidence, a justification where the condition was not fulfilled literally, and for 2.4.x the disclosure answer. An answer can be scoped to specific benchmarks with appliesToEvaluation. Sample data: sample_data/aef1_usecase_data.yaml.

In [5]:

from pathlib import Path

usecase_ran = AIAtlasNexus(base_dir=str(Path("sample_data")))

# Who evaluated whose system
[engagement] = usecase_ran.get_evaluation_engagements(system_provider="aef1-usecase-provider")
evaluator = usecase_ran.get_by_id(class_name="organizations", identifier=engagement.hasEvaluator)
provider = usecase_ran.get_by_id(class_name="organizations", identifier=engagement.hasSystemProvider)
print(f"{engagement.name} ({engagement.startDate} to {engagement.endDate})")
print(f"evaluator: {evaluator.name}, system provider: {provider.name} ({type(provider).__name__})")

for result_id in engagement.hasEvaluation:
    result = usecase_ran.get_by_id(class_name="aievalresults", identifier=result_id)
    benchmark = usecase_ran.get_by_id(class_name="evaluations", identifier=result.isResultOf)
    print(f"  {benchmark.name} = {result.value} ({result.evidence})")

from pathlib import Path usecase_ran = AIAtlasNexus(base_dir=str(Path("sample_data"))) # Who evaluated whose system [engagement] = usecase_ran.get_evaluation_engagements(system_provider="aef1-usecase-provider") evaluator = usecase_ran.get_by_id(class_name="organizations", identifier=engagement.hasEvaluator) provider = usecase_ran.get_by_id(class_name="organizations", identifier=engagement.hasSystemProvider) print(f"{engagement.name} ({engagement.startDate} to {engagement.endDate})") print(f"evaluator: {evaluator.name}, system provider: {provider.name} ({type(provider).__name__})") for result_id in engagement.hasEvaluation: result = usecase_ran.get_by_id(class_name="aievalresults", identifier=result_id) benchmark = usecase_ran.get_by_id(class_name="evaluations", identifier=result.isResultOf) print(f" {benchmark.name} = {result.value} ({result.evidence})")

[2026-09-28 13:47:59:486] - INFO - AIAtlasNexus - Created AIAtlasNexus instance. Base_dir: sample_data

Independent evaluation of the Credit Risk Assessment LLM (2026-03-02 to 2026-04-10)
evaluator: Example Evaluation Lab, system provider: Example Bank (AiProvider)
  Independent fairness benchmark = 0.03 (Maximum demographic parity difference across protected groups, below the firm's 0.05 threshold.)
  Independent prompt injection benchmark = 0.12 (Attack success rate across 500 injected applicant documents.)

In [6]:

[conformance] = engagement.hasStandardConformance
print(f"{conformance.conformsToStandard}: satisfies all requirements = {conformance.satisfiesAllRequirements}")

for a in conformance.hasConditionAssessment:
    if a.hasConformanceOutcome in ("NOT_FULFILLED", "ALTERNATIVE_MEANS"):
        scope = ", ".join(a.appliesToEvaluation or ["whole engagement"])
        print(f"  {conditions[a.assessesCondition].name}: {a.hasConformanceOutcome} [{scope}]")
        print(f"    {a.justification}")

[conformance] = engagement.hasStandardConformance print(f"{conformance.conformsToStandard}: satisfies all requirements = {conformance.satisfiesAllRequirements}") for a in conformance.hasConditionAssessment: if a.hasConformanceOutcome in ("NOT_FULFILLED", "ALTERNATIVE_MEANS"): scope = ", ".join(a.appliesToEvaluation or ["whole engagement"]) print(f" {conditions[a.assessesCondition].name}: {a.hasConformanceOutcome} [{scope}]") print(f" {a.justification}")

aef1-standard: satisfies all requirements = False
  1.4 Time: NOT_FULFILLED [aef1-usecase-injection-benchmark]
    Added to the scope with 8 business days remaining; exploratory probing of novel injection vectors was not completed.
  3.3 Direct access: ALTERNATIVE_MEANS [aef1-usecase-fairness-benchmark]
    Provider staff ran the queries over production loan data on the evaluator's behalf, under a contractual guarantee not to use the evaluator's methods for other purposes.

Checking the checklist¶

check_standard_conformance derives the overall answer from the rows: every requirement (including required sub-elements 1.1.1 and 2.4.x) must be FULFILLED or ALTERNATIVE_MEANS, with answers covering every benchmark of the engagement. It also reports missing justifications, and checks that the evaluator is a third party to the system provider.

In [7]:

check = usecase_ran.check_standard_conformance("aef1-usecase-engagement")
print(f"valid: {check.is_valid}, satisfies all requirements: {check.satisfies_all_requirements}")
print(f"independence issues: {check.independence_issues}")

for standard_check in check.standards:
    print(
        f"{standard_check.standard_id}: {len(standard_check.required_conditions)} requirements, "
        f"stated {standard_check.stated_satisfies_all_requirements}, derived {standard_check.derived_satisfies_all_requirements}"
    )
    for issue in standard_check.issues:
        print(f"  {conditions[issue.condition_id].name}: {issue.issue} ({issue.outcome}) {issue.evaluations}")

check = usecase_ran.check_standard_conformance("aef1-usecase-engagement") print(f"valid: {check.is_valid}, satisfies all requirements: {check.satisfies_all_requirements}") print(f"independence issues: {check.independence_issues}") for standard_check in check.standards: print( f"{standard_check.standard_id}: {len(standard_check.required_conditions)} requirements, " f"stated {standard_check.stated_satisfies_all_requirements}, derived {standard_check.derived_satisfies_all_requirements}" ) for issue in standard_check.issues: print(f" {conditions[issue.condition_id].name}: {issue.issue} ({issue.outcome}) {issue.evaluations}")

valid: True, satisfies all requirements: False
independence issues: []
aef1-standard: 22 requirements, stated False, derived False
  1.4 Time: not_met (NOT_FULFILLED) ['aef1-usecase-injection-benchmark']

Legal cross-references¶

AEF-1 conditions are mapped (related_mappings) to the EU General-Purpose AI Code of Practice (Safety and Security Chapter) and California SB 53. Mappings are in src/ai_atlas_nexus/data/mappings/aef1_to_legal_provisions.tsv.

In [8]:

items = standard + principles + list(conditions.values())
for item in items:
    for document_id in item.related_mappings or []:
        document = ran.get_by_id(class_name="documents", identifier=document_id)
        print(f"{item.name} -> {document.name}")

items = standard + principles + list(conditions.values()) for item in items: for document_id in item.related_mappings or []: document = ran.get_by_id(class_name="documents", identifier=document_id) print(f"{item.name} -> {document.name}")

AEF-1 -> California Business and Professions Code § 22757.12(c)(2)(C) (SB 53): Third-party evaluator involvement
AEF-1 -> EU GPAI Code of Practice, Safety and Security, Measure 7.3: Documentation of systemic risk identification, analysis, and mitigation
Principle 1: Sufficient Access and Resources -> EU GPAI Code of Practice, Safety and Security, Appendix 3.5: Independent external model evaluations
Principle 1: Sufficient Access and Resources -> EU GPAI Code of Practice, Safety and Security, Measure 7.3: Documentation of systemic risk identification, analysis, and mitigation
1.1 Technical access -> EU GPAI Code of Practice, Safety and Security, Appendix 3.4: Qualified model evaluation teams and adequate resources
1.3 Computational resources -> EU GPAI Code of Practice, Safety and Security, Appendix 3.4: Qualified model evaluation teams and adequate resources
1.4 Time -> EU GPAI Code of Practice, Safety and Security, Appendix 3.4: Qualified model evaluation teams and adequate resources
5.3 Protecting confidential information -> EU GPAI Code of Practice, Safety and Security, Appendix 3.5: Independent external model evaluations
1.1.3 Safeguard exemptions -> EU GPAI Code of Practice, Safety and Security, Appendix 3.4: Qualified model evaluation teams and adequate resources
1.1.4 Intermediate system states -> EU GPAI Code of Practice, Safety and Security, Appendix 3.4: Qualified model evaluation teams and adequate resources
1.2 Information -> EU GPAI Code of Practice, Safety and Security, Appendix 3.4: Qualified model evaluation teams and adequate resources
5.2 Evaluation integrity -> EU GPAI Code of Practice, Safety and Security, Appendix 3.5: Independent external model evaluations