AIRMF-GV-1.1 | Legal and regulatory requirements involving AI are understood, managed, and documented | 4 |
AIRMF-GV-1.2 | The characteristics of trustworthy AI are integrated into organizational policies, processes, and procedures | 3 |
AIRMF-GV-1.3 | Processes and procedures are in place to determine the needed level of risk management activities based on the organization's risk tolerance | 1 |
AIRMF-GV-1.4 | The risk management process and its outcomes are established through transparent policies, procedures, and other controls based on organizational risk priorities | 2 |
AIRMF-GV-1.5 | Ongoing monitoring and periodic review of the risk management process and its outcomes are planned, organizational roles and responsibilities are clearly defined, including determi | 2 |
AIRMF-GV-1.6 | Mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities | 2 |
AIRMF-GV-1.7 | Processes and procedures are in place for decommissioning and phasing out of AI systems safely and in a manner that does not increase risks or decrease the organization's trustwort | 1 |
AIRMF-GV-2.1 | Roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are documented and are clear to individuals and teams throughout the orga | 5 |
AIRMF-GV-2.2 | The organization's personnel and partners receive AI risk management training to enable them to perform their duties and responsibilities consistent with related policies, procedur | 2 |
AIRMF-GV-2.3 | Executive leadership of the organization takes responsibility for decisions about risks associated with AI system development and deployment | 2 |
AIRMF-GV-3.1 | Decision-making related to mapping, measuring, and managing AI risks throughout the lifecycle is informed by a diverse team | 0 |
AIRMF-GV-3.2 | Policies and procedures are in place to define and differentiate roles and responsibilities for human-AI configurations and oversight of AI systems | 2 |
AIRMF-GV-4.1 | Organizational policies and practices are in place to foster a critical thinking and safety-first mindset in the design, development, deployment, and uses of AI systems to minimize | 3 |
AIRMF-GV-4.2 | Organizational teams document the risks and potential impacts of the AI technology they design, develop, deploy, evaluate and use, and communicate about the impacts more broadly | 2 |
AIRMF-GV-4.3 | Organizational practices are in place to enable AI testing, identification of incidents, and information sharing | 2 |
AIRMF-GV-5.1 | Organizational policies and practices are in place to collect, consider, prioritize, and integrate feedback from those external to the team that developed or deployed the AI system | 2 |
AIRMF-GV-5.2 | Mechanisms are established to enable AI actors to regularly incorporate adjudicated feedback from relevant AI actors into system design and implementation | 2 |
AIRMF-GV-6.1 | Policies and procedures are in place that address AI risks associated with third-party entities, including risks of infringement of a third party's intellectual property or other r | 2 |
AIRMF-GV-6.2 | Contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be high-risk | 1 |
AIRMF-MN-1.1 | A determination is made as to whether the AI system achieves its intended purpose and stated objectives and whether its development or deployment should proceed | 5 |
AIRMF-MN-1.2 | Treatment of documented AI risks is prioritized based on impact, likelihood, or available resources or methods | 1 |
AIRMF-MN-1.3 | Responses to the AI risks deemed high priority as identified by the MAP function are developed, planned, and documented, and risk response options can include mitigating, transferr | 2 |
AIRMF-MN-1.4 | Negative residual risks, defined as the sum of all unmitigated risks, to both downstream acquirers of AI systems and end users are documented | 2 |
AIRMF-MN-2.1 | Resources required to manage AI risks are taken into account, along with viable non-AI alternative systems, approaches, or methods, to reduce the magnitude or likelihood of potenti | 4 |
AIRMF-MN-2.2 | Mechanisms are in place and applied to sustain the value of deployed AI systems | 0 |
AIRMF-MN-2.3 | Procedures are followed to respond to and recover from a previously unknown risk when it is identified | 2 |
AIRMF-MN-2.4 | Mechanisms are in place and applied, and responsibilities are assigned and understood, to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes in | 1 |
AIRMF-MN-3.1 | AI risks and benefits from third-party resources are regularly monitored, and risk controls are applied and documented | 4 |
AIRMF-MN-3.2 | Pre-trained models which are used for development are monitored as part of AI system regular monitoring and maintenance | 0 |
AIRMF-MN-4.1 | Post-deployment AI system monitoring plans are implemented, including mechanisms for capturing and evaluating input from users and other relevant AI actors, appeal and override, de | 4 |
AIRMF-MN-4.2 | Measurable activities for continual improvements are integrated into AI system updates and include regular engagement with interested parties, including relevant AI actors | 0 |
AIRMF-MN-4.3 | Incidents and errors are communicated to relevant AI actors including affected communities, and processes for tracking, responding to, and recovering from incidents and errors are | 2 |
AIRMF-MP-1.1 | Intended purpose, potentially beneficial uses, context-specific laws, norms and expectations, and prospective settings in which the AI system will be deployed are understood and do | 5 |
AIRMF-MP-1.2 | Inter-disciplinary AI actors, competencies, skills and capacities for establishing context reflect demographic diversity and broad domain and user experience expertise, and their p | 1 |
AIRMF-MP-1.3 | The organization's mission and relevant goals for the AI technology are understood and documented | 1 |
AIRMF-MP-1.4 | The business value or context of business use has been clearly defined or, in the case of assessing existing AI systems, re-evaluated | 1 |
AIRMF-MP-1.5 | Organizational risk tolerances are determined and documented | 1 |
AIRMF-MP-1.6 | System requirements are elicited from and understood by relevant AI actors, and design decisions take socio-technical implications into account to address AI risks | 2 |
AIRMF-MP-2.1 | The specific task, and methods used to implement the task, that the AI system will support is defined | 4 |
AIRMF-MP-2.2 | Information about the AI system's knowledge limits and how system output may be utilized and overseen by humans is documented | 2 |
AIRMF-MP-2.3 | Scientific integrity and TEVV considerations are identified and documented, including those related to experimental design, data collection and selection, system trustworthiness, a | 2 |
AIRMF-MP-3.1 | Potential benefits of intended AI system functionality and performance are examined and documented | 0 |
AIRMF-MP-3.2 | Potential costs, including non-monetary costs, which result from expected or realized AI errors or system functionality and trustworthiness are examined and documented, as connecte | 2 |
AIRMF-MP-3.3 | Targeted application scope is specified and documented based on the system's capability, established context, and AI system categorization | 2 |
AIRMF-MP-3.4 | Processes for operator and practitioner proficiency with AI system performance and trustworthiness, and relevant technical standards and certifications, are defined, assessed and d | 2 |
AIRMF-MP-3.5 | Processes for human oversight are defined, assessed, and documented in accordance with organizational policies from the GOVERN function | 2 |
AIRMF-MP-4.1 | Approaches for mapping AI technology and legal risks of its components, including the use of third-party data or software, are in place, followed, and documented, as are risks of i | 2 |
AIRMF-MP-4.2 | Internal risk controls for components of the AI system including third-party AI technologies are identified and documented | 2 |
AIRMF-MP-5.1 | Likelihood and magnitude of each identified impact are identified and documented, based on expected use, past uses of AI systems in similar contexts, public incident reports, feedb | 2 |
AIRMF-MP-5.2 | Practices and personnel for supporting regular engagement with relevant AI actors and integrating feedback about positive, negative, and unanticipated impacts are in place and docu | 2 |
AIRMF-MS-1.1 | Approaches and metrics for measurement of AI risks enumerated during the MAP function are selected for implementation starting with the most significant AI risks, and the risks or | 4 |
AIRMF-MS-1.2 | Appropriateness of AI metrics and effectiveness of existing controls is regularly assessed and updated, including reports of errors and impacts on affected communities | 2 |
AIRMF-MS-1.3 | Internal experts who did not serve as front-line developers for the system and independent assessors are involved in regular assessments and updates, and domain experts, users, AI | 1 |
AIRMF-MS-2.1 | Test sets, metrics, and details about the tools used during test, evaluation, validation, and verification are documented | 3 |
AIRMF-MS-2.10 | Privacy risk of the AI system as identified in the MAP function is examined and documented | 2 |
AIRMF-MS-2.11 | Fairness and bias as identified in the MAP function is evaluated and results are documented | 3 |
AIRMF-MS-2.12 | Environmental impact and sustainability of AI model training and management activities as identified in the MAP function are assessed and documented | 1 |
AIRMF-MS-2.13 | Effectiveness of the employed TEVV metrics and processes in the MEASURE function are evaluated and documented | 1 |
AIRMF-MS-2.2 | Evaluations involving human subjects meet applicable requirements including human subject protection and are representative of the relevant population | 1 |
AIRMF-MS-2.3 | AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment settings, and measures are documented | 2 |
AIRMF-MS-2.4 | The functionality and behavior of the AI system and its components, as identified in the MAP function, are monitored when in production | 2 |
AIRMF-MS-2.5 | The AI system to be deployed is demonstrated to be valid and reliable, and limitations of the generalizability beyond the conditions under which the technology was developed are do | 2 |
AIRMF-MS-2.6 | AI system is evaluated regularly for safety risks as identified in the MAP function, is demonstrated to be safe, its residual negative risk does not exceed the risk tolerance, and | 1 |
AIRMF-MS-2.7 | AI system security and resilience as identified in the MAP function are evaluated and documented | 5 |
AIRMF-MS-2.8 | Risks associated with transparency and accountability as identified in the MAP function are examined and documented | 3 |
AIRMF-MS-2.9 | The AI model is explained, validated, and documented, and AI system output is interpreted within its context as identified in the MAP function and to inform responsible use and gov | 2 |
AIRMF-MS-3.1 | Approaches, personnel, and documentation are in place to regularly identify and track existing, unanticipated, and emergent AI risks based on factors such as intended and actual pe | 2 |
AIRMF-MS-3.2 | Risk tracking approaches are considered for settings where AI risks are difficult to assess using currently available measurement techniques or where metrics are not yet available | 0 |
AIRMF-MS-3.3 | Feedback processes for end users and impacted communities to report problems and appeal system outcomes are established and integrated into AI system evaluation metrics | 2 |
AIRMF-MS-4.1 | Measurement approaches for identifying AI risks are connected to deployment contexts and informed through consultation with domain experts and other end users, and approaches are d | 2 |
AIRMF-MS-4.2 | Measurement results regarding AI system trustworthiness in deployment contexts and across the AI lifecycle are informed by input from domain experts and other relevant AI actors to | 1 |
AIRMF-MS-4.3 | Measurable performance improvements or declines based on consultations with relevant AI actors including affected communities, and field data about context-relevant risks and trust | 2 |