Supervisors do not wait for an AI system to be labelled high-risk before asking how well it performs. Across the EU, Switzerland, the U.S. and international standard-setters, the expectation is the same: measure it, document it and keep measuring it, with rigour proportionate to the risk.
Financial institutions are under pressure from two directions. Boards and business lines want AI in production faster: research synthesis, knowledge retrieval, client communications, agentic workflows. Supervisors, meanwhile, increasingly treat compliance and technical performance as one conversation.
FINMA Guidance 08/2024, the FSB's Sound Practices consultation, the EBA's analysis of the AI Act and the U.S. Treasury's Financial Services AI Risk Management Framework all point the same way. Evidence of performance is becoming a supervisory expectation for AI in general, not only for the narrow set of systems the EU AI Act classifies as high-risk.
Supervisors expect measured performance, proportionate to risk
FINMA's Guidance 08/2024 (December 2024) addresses supervised institutions' use of AI broadly, not a specific risk class. It names "lack of robustness, correctness, explainability or bias" as core AI model risks that institutions must "identify, evaluate, oversee and track." For a Swiss bank, that already covers a generative AI assistant used by research or advisory staff.
The Financial Stability Board's June 2026 consultation report, "Sound Practices for Responsible Adoption of AI", takes the same view globally. Sound Practice 9 calls for "evaluating the performance of AI use cases proportionately to their materiality and risk, including through performance assessment, testing and ongoing monitoring." A credit-decisioning model needs deeper evaluation than an internal drafting assistant, but both need evaluation.
Sound Practice 7 adds that AI models perpetuate biases and inaccuracies in their input data unless that data is governed.
The European Banking Authority's November 2025 report on the AI Act's implications for banking and payments found no significant contradictions between the AI Act and existing banking legislation. The reason: banking regulation already assumes rigorous model risk management. Validation, ongoing monitoring and documented performance are disciplines banks already owe their supervisors, and AI extends them to a new class of models.
The U.S. Treasury's Financial Services AI Risk Management Framework (19 February 2026), adapting the NIST AI RMF, sets the same expectation for U.S. institutions. The BIS Financial Stability Institute's Insight No. 63 (December 2024) and its June 2025 report on the financial stability implications of AI both conclude that AI amplifies familiar risks: model risk, data quality and third-party dependency. Supervisors will assess them through the same measurement lens they already apply to other models.
Where the EU AI Act fits
The EU AI Act is often cited as the driver of AI testing, but its strictest requirements are narrow in scope. Articles 9, 10 and 15 (risk management, data governance, and declared accuracy, robustness and cybersecurity) apply to high-risk AI systems. In financial services, Annex III lists systems that evaluate creditworthiness or establish credit scores, and systems for risk assessment and pricing in life and health insurance.
Most generative AI use in banks falls outside that list: research synthesis, document analysis, knowledge retrieval, drafting. Classification follows the actual use case, so an institution that applies AI to credit decisions should assess that deployment separately.
The high-risk deadline has also moved. The Digital Omnibus on AI, published in the Official Journal on 24 July 2026, postponed the obligations for Annex III high-risk systems from 2 August 2026 to 2 December 2027.
Obligations for general-purpose AI models have applied since 2 August 2025. Article 55 requires providers of models with systemic risk to evaluate them with standardised, state-of-the-art protocols, including adversarial testing, and the GPAI Code of Practice (July 2025) adds a Safety and Security Model Report before release. These duties bind foundation model providers, not the institutions building on their models. They still set a reference point: if model providers must evaluate against standardised protocols, institutions using those models in their own workflows will be asked how they evaluated the result.
Article 15 also directs the Commission to encourage common benchmarks and measurement methodologies, a signal of where practice is heading. For institutions outside the high-risk category, though, the AI Act does not create the benchmarking expectation. Their supervisors already have.
The gap is already visible
ESMA's 20 February 2026 TRV Risk Analysis on AI adoption in EU securities markets quantifies the problem. Model hallucinations remain a concern for 51% of respondent firms. Firms rate their own AI transparency at 3.3 out of 5 on average, lower still for generative AI, and "the use of explainability methods was not widely reported."
Only 17% of firms reported complete board-level understanding of the AI they had deployed, and 45% named regulatory compliance itself as a challenge, while 76% said the AI Act was already affecting operations. Much of this AI is not high-risk under the Act. It is still deployed faster than it is characterised, and that is the gap benchmarking closes.
How Unique AI turns these insights into practice: Benchmark Pro
At Unique AI, we have built this measurement discipline into the platform itself. Benchmark Pro is the benchmarking and optimization module of the Unique AI platform, designed for banks, asset managers that need to show, not just state, how well their AI agents perform, and to keep improving them. It is built around three capabilities.
Compare Models Side by Side
Run scheduled model benchmarks on real financial use cases and compare quality, speed and cost. Whenever a new model or prompt version ships, the agent is re-benchmarked against the same test set, so every change is backed by numbers rather than a one-off spot check, and teams can see which model meets the quality bar at the lowest cost.
Score Agents with Finance-Trained Judges
Finance-trained judges score accuracy, compliance and coverage, with actionable recommendations, turning week-long reviews into hours. Each run covers hundreds of realistic test questions, including adversarial cases, against quality thresholds set per agent and tiered to the materiality of each use case. Scheduled runs track agent quality over time and send alerts when it drops below threshold, while every score and judge rationale is recorded, giving model risk and compliance teams a documented performance record to support their dialogue with supervisors such as FINMA.
Agent Optimization
Benchmark Pro turns benchmarking results into actionable improvements by identifying quality gaps, cost inefficiencies, and model/prompt adjustments. Each recommendation traces back to the judges' reasoning, so teams know exactly what to fix and why, then re-run the benchmark to confirm the change improved the agent. Measurement becomes a continuous improvement loop rather than a periodic audit.
Benchmark Pro is available as an add-on to the Unique AI platform, activated per space. To see how it would work on your own agents, get in touch with your Unique Ai contact or request a demo.
Sources
· EU AI Act (Regulation (EU) 2024/1689), Articles 9, 10, 15, 55. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32024R1689
· European Commission, AI Act overview. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
· EU AI Act, Article 55 (GPAI systemic-risk obligations). https://artificialintelligenceact.eu/article/55/
· GPAI Code of Practice, July 2025. https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai
· EDPS, Guidance for Risk Management of AI Systems, 11 November 2025. https://www.edps.europa.eu/system/files/2025-11/2025-11-11_ai_risks_management_guidance_en.pdf
· EDPB, Bias evaluation (D1), January 2025. https://www.edpb.europa.eu/system/files/2025-01/d1-ai-bias-evaluation_en.pdf
· EBA, AI Act implications for the EU banking and payments sector, 21 November 2025. https://www.eba.europa.eu/sites/default/files/2025-11/d8b999ce-a1d9-4964-9606-971bbc2aaf89/AI%20Act%20implications%20for%20the%20EU%20banking%20sector.pdf
· Financial Stability Board, Sound Practices for Responsible Adoption of AI (Consultation Report), 10 June 2026. https://www.fsb.org/2026/06/sound-practices-for-responsible-adoption-of-artificial-intelligence-ai-consultation-report/
· FINMA Guidance 08/2024, Governance and risk management when using AI, 18 December 2024. https://www.finma.ch/en/news/2024/12/20241218-mm-finma-am-08-24/
· U.S. Treasury, Financial Services AI Risk Management Framework, 19 February 2026. https://home.treasury.gov/news/press-releases/sb0401
· NIST AI Risk Management Framework (AI 100-1), 2023, and AI 600-1 Generative AI Profile, July 2024. https://www.nist.gov/itl/ai-risk-management-framework
· BIS FSI Insights No. 63, Regulating AI in the financial sector, December 2024. https://www.bis.org/publications/fsi-insight-63-regulating-ai-financial-sector-recent-developments-and-main-challenges
· BIS FSI, Financial stability implications of AI, 26 June 2025. https://www.bis.org/publications/fsi-summary-financial-stability-implications-artificial-intelligence-executive-summary
Why AI Benchmarking Is the Missing Link Between Compliance and Accuracy in Financial Services
Supervisors do not wait for an AI system to be labelled high-risk before asking how well it performs. Across the EU, Switzerland, the U.S. and international standard-setters, the expectation is the same: measure it, document it and keep measuring it, with rigour proportionate to the risk.
Financial institutions are under pressure from two directions. Boards and business lines want AI in production faster: research synthesis, knowledge retrieval, client communications, agentic workflows. Supervisors, meanwhile, increasingly treat compliance and technical performance as one conversation.
FINMA Guidance 08/2024, the FSB's Sound Practices consultation, the EBA's analysis of the AI Act and the U.S. Treasury's Financial Services AI Risk Management Framework all point the same way. Evidence of performance is becoming a supervisory expectation for AI in general, not only for the narrow set of systems the EU AI Act classifies as high-risk.
Supervisors expect measured performance, proportionate to risk
FINMA's Guidance 08/2024 (December 2024) addresses supervised institutions' use of AI broadly, not a specific risk class. It names "lack of robustness, correctness, explainability or bias" as core AI model risks that institutions must "identify, evaluate, oversee and track." For a Swiss bank, that already covers a generative AI assistant used by research or advisory staff.
The Financial Stability Board's June 2026 consultation report, "Sound Practices for Responsible Adoption of AI", takes the same view globally. Sound Practice 9 calls for "evaluating the performance of AI use cases proportionately to their materiality and risk, including through performance assessment, testing and ongoing monitoring." A credit-decisioning model needs deeper evaluation than an internal drafting assistant, but both need evaluation.
Sound Practice 7 adds that AI models perpetuate biases and inaccuracies in their input data unless that data is governed.
The European Banking Authority's November 2025 report on the AI Act's implications for banking and payments found no significant contradictions between the AI Act and existing banking legislation. The reason: banking regulation already assumes rigorous model risk management. Validation, ongoing monitoring and documented performance are disciplines banks already owe their supervisors, and AI extends them to a new class of models.
The U.S. Treasury's Financial Services AI Risk Management Framework (19 February 2026), adapting the NIST AI RMF, sets the same expectation for U.S. institutions. The BIS Financial Stability Institute's Insight No. 63 (December 2024) and its June 2025 report on the financial stability implications of AI both conclude that AI amplifies familiar risks: model risk, data quality and third-party dependency. Supervisors will assess them through the same measurement lens they already apply to other models.
Where the EU AI Act fits
The EU AI Act is often cited as the driver of AI testing, but its strictest requirements are narrow in scope. Articles 9, 10 and 15 (risk management, data governance, and declared accuracy, robustness and cybersecurity) apply to high-risk AI systems. In financial services, Annex III lists systems that evaluate creditworthiness or establish credit scores, and systems for risk assessment and pricing in life and health insurance.
Most generative AI use in banks falls outside that list: research synthesis, document analysis, knowledge retrieval, drafting. Classification follows the actual use case, so an institution that applies AI to credit decisions should assess that deployment separately.
The high-risk deadline has also moved. The Digital Omnibus on AI, published in the Official Journal on 24 July 2026, postponed the obligations for Annex III high-risk systems from 2 August 2026 to 2 December 2027.
Obligations for general-purpose AI models have applied since 2 August 2025. Article 55 requires providers of models with systemic risk to evaluate them with standardised, state-of-the-art protocols, including adversarial testing, and the GPAI Code of Practice (July 2025) adds a Safety and Security Model Report before release. These duties bind foundation model providers, not the institutions building on their models. They still set a reference point: if model providers must evaluate against standardised protocols, institutions using those models in their own workflows will be asked how they evaluated the result.
Article 15 also directs the Commission to encourage common benchmarks and measurement methodologies, a signal of where practice is heading. For institutions outside the high-risk category, though, the AI Act does not create the benchmarking expectation. Their supervisors already have.
The gap is already visible
ESMA's 20 February 2026 TRV Risk Analysis on AI adoption in EU securities markets quantifies the problem. Model hallucinations remain a concern for 51% of respondent firms. Firms rate their own AI transparency at 3.3 out of 5 on average, lower still for generative AI, and "the use of explainability methods was not widely reported."
Only 17% of firms reported complete board-level understanding of the AI they had deployed, and 45% named regulatory compliance itself as a challenge, while 76% said the AI Act was already affecting operations. Much of this AI is not high-risk under the Act. It is still deployed faster than it is characterised, and that is the gap benchmarking closes.
How Unique AI turns these insights into practice: Benchmark Pro
At Unique AI, we have built this measurement discipline into the platform itself. Benchmark Pro is the benchmarking and optimization module of the Unique AI platform, designed for banks, asset managers that need to show, not just state, how well their AI agents perform, and to keep improving them. It is built around three capabilities.
Compare Models Side by Side
Run scheduled model benchmarks on real financial use cases and compare quality, speed and cost. Whenever a new model or prompt version ships, the agent is re-benchmarked against the same test set, so every change is backed by numbers rather than a one-off spot check, and teams can see which model meets the quality bar at the lowest cost.
Score Agents with Finance-Trained Judges
Finance-trained judges score accuracy, compliance and coverage, with actionable recommendations, turning week-long reviews into hours. Each run covers hundreds of realistic test questions, including adversarial cases, against quality thresholds set per agent and tiered to the materiality of each use case. Scheduled runs track agent quality over time and send alerts when it drops below threshold, while every score and judge rationale is recorded, giving model risk and compliance teams a documented performance record to support their dialogue with supervisors such as FINMA.
Agent Optimization
Benchmark Pro turns benchmarking results into actionable improvements by identifying quality gaps, cost inefficiencies, and model/prompt adjustments. Each recommendation traces back to the judges' reasoning, so teams know exactly what to fix and why, then re-run the benchmark to confirm the change improved the agent. Measurement becomes a continuous improvement loop rather than a periodic audit.
Benchmark Pro is available as an add-on to the Unique AI platform, activated per space. To see how it would work on your own agents, get in touch with your Unique Ai contact or request a demo.
Sources
· EU AI Act (Regulation (EU) 2024/1689), Articles 9, 10, 15, 55. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32024R1689
· European Commission, AI Act overview. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
· EU AI Act, Article 55 (GPAI systemic-risk obligations). https://artificialintelligenceact.eu/article/55/
· GPAI Code of Practice, July 2025. https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai
· EDPS, Guidance for Risk Management of AI Systems, 11 November 2025. https://www.edps.europa.eu/system/files/2025-11/2025-11-11_ai_risks_management_guidance_en.pdf
· EDPB, Bias evaluation (D1), January 2025. https://www.edpb.europa.eu/system/files/2025-01/d1-ai-bias-evaluation_en.pdf
· EBA, AI Act implications for the EU banking and payments sector, 21 November 2025. https://www.eba.europa.eu/sites/default/files/2025-11/d8b999ce-a1d9-4964-9606-971bbc2aaf89/AI%20Act%20implications%20for%20the%20EU%20banking%20sector.pdf
· Financial Stability Board, Sound Practices for Responsible Adoption of AI (Consultation Report), 10 June 2026. https://www.fsb.org/2026/06/sound-practices-for-responsible-adoption-of-artificial-intelligence-ai-consultation-report/
· FINMA Guidance 08/2024, Governance and risk management when using AI, 18 December 2024. https://www.finma.ch/en/news/2024/12/20241218-mm-finma-am-08-24/
· U.S. Treasury, Financial Services AI Risk Management Framework, 19 February 2026. https://home.treasury.gov/news/press-releases/sb0401
· NIST AI Risk Management Framework (AI 100-1), 2023, and AI 600-1 Generative AI Profile, July 2024. https://www.nist.gov/itl/ai-risk-management-framework
· BIS FSI Insights No. 63, Regulating AI in the financial sector, December 2024. https://www.bis.org/publications/fsi-insight-63-regulating-ai-financial-sector-recent-developments-and-main-challenges
· BIS FSI, Financial stability implications of AI, 26 June 2025. https://www.bis.org/publications/fsi-summary-financial-stability-implications-artificial-intelligence-executive-summary