Artificial Intelligence Systems Risk Assessment
DS&A identifies the inherent risk of the Bank’s AI systems using the AI risk categories described in NIST’s AI Risk Management Framework.
We evaluate AI systems controls using online surveys that are based on the Suggested Actions and documentation recommendations in NIST’s AI Risk Management Framework Playbook.
We also use controls questions in our surveys that are based on the ISO/IEC’s Information Technology -Artificial Intelligence – Management System standard (ISO/IEC 42001).
Surveys are organized around the roles and responsibilities of the Bank’s “AI actors” (a term defined in the Framework) who are involved in the Bank’s AI systems’ life cycles (e.g., the business owners, CIO, Model Data Scientist, User Experience /User Interface Designer, Pretrained AI model vendor).
EXAMPLE AI SYSTEM RISK MANAGEMENT CONTROLS SURVEY
This example survey contains twenty-six questions which can be used to assess the Bank’s AI systems risk management practices. The questions are based on Suggested Actions in the NIST Playbook, and cover the following topics:
- AI Systems Risk Management Policies, Standards, and Procedures
- Board / Board-appointed Committee Oversight
- AI Actor Roles and Responsibilities
- AI Systems Inventory
- AI Systems Risk Factors and Risk Tolerance
- Decommissioning and exit strategies for AI systems.