Artificial intelligence offers considerable potential for the financial sector: processes become more efficient, decisions are supported by data, and new products and services become possible. At the same time, it presents institutions with the challenge of ensuring the long-term security, robustness, transparency and governance of their AI systems. Particularly in a highly regulated and sensitive environment, it is therefore essential to be able to test AI systems systematically and in a traceable manner.
Against this backdrop, the Federal Office for Information Security (BSI) has published the ‘Test Criteria Catalogue for AI Systems in Finance’, together with an accompanying study, as part of the AICRIV-Finanz project.
Almost 100 test criteria for practical application
The test catalogue comprises almost 100 practice-oriented criteria across ten thematic areas. These enable a structured assessment of AI systems with regard to, amongst other things:
- Security and robustness
- Transparency and traceability
- Fairness
- Data quality
- Governance
- Evaluation requirements
- Suitable testing methods
- Supporting tools
- Technical and organisational risks
- Regulatory requirements such as the EU AI Act and DORA
For each individual criterion, the catalogue contains specific evaluation requirements, suitable testing methods, supporting guidance, possible tools, and references to existing regulatory requirements and standards.
Development based on real-world use cases
The accompanying study describes how the criteria catalogue was developed and validated as part of the AICRIV-Finanz project. It was based on real-world use cases from the financial sector, regulatory requirements, existing standards and technical risk assessments.
Furthermore, the study demonstrates how the individual test criteria can be applied using suitable methods and tools and integrated throughout the entire lifecycle of an AI system. This ranges from development and validation through to live operation, and on to ongoing monitoring and adaptation.
Guidance for financial and insurance companies
The catalogue and study are aimed in particular at developers, providers, operators and testing organisations of AI systems in the finance and insurance sectors. They can also serve as a guide for other sectors in the structured testing of AI systems.
The criteria catalogue provides a practical basis for harnessing the potential of AI in the financial sector whilst ensuring that such systems are robust, compliant with regulations and used responsibly.
The full study and the criteria catalogue are available for download on the BSI website.
