The European Commission’s long-awaited guidance on high-risk AI systems may appear at first glance to be a legal and regulatory document.
For banks and financial services firms, however, the real impact could be felt by software testing, quality engineering, digital resilience and AI governance teams tasked with proving that AI systems are safe, controlled and compliant.
Published as part of the EU AI Act framework, the 167-page document aims to help organisations determine whether an AI system falls within the Act’s high-risk category.
While much of the attention has focused on legal interpretation, the guidance also signals a broader shift in how organisations will need to test, validate, monitor and document AI systems throughout their lifecycle.
According to a legal team by international law firm Hogan Lovells, “the guidelines set out the key concepts and provide practical examples to assist stakeholders in the classification of high-risk AI systems.”
That classification exercise is likely to become a critical issue for financial institutions deploying AI across lending, fraud detection, customer onboarding, financial crime monitoring, employee recruitment and a growing number of operational processes.
Regulatory assurance
For QA teams, the most significant implication is that determining whether a system is high-risk is no longer simply a compliance question.
Once a bank concludes that an AI system falls within a high-risk category, a wide range of obligations around governance, documentation, monitoring, oversight and risk management can follow.
As a result, testing teams may increasingly find themselves responsible not only for validating functionality, but also for generating the evidence needed to support compliance.
The Hogan Lovells team, led by London partner Eduardo Ustaran, noted that “a number of key overarching points are made within the guidelines, concerning how providers should document and analyse the applicability of the high-risk rules to their systems.”
That requirement for documentation and analysis is likely to resonate with teams already grappling with AI governance frameworks, model validation programmes, DORA implementation projects and broader operational resilience initiatives.
Increasingly, regulators are asking organisations not merely whether an AI system works, but whether they can demonstrate how it works, how risks are identified, how decisions are monitored and how human oversight is maintained.
Fresh governance questions
The guidance also raises important questions for firms experimenting with large language models, copilots and agentic AI systems.
According to Hogan Lovells, “GPAI systems may come into scope of the high-risk AI system rules unless documentation consistently excludes high-risk AI use cases.”
For banks investing heavily in enterprise AI deployments, that observation may prove particularly significant.
Many organisations have focused on testing performance, accuracy and business outcomes. The Commission’s guidance suggests that documentation, governance controls and evidence trails could become equally important in determining whether a system falls within a regulated category.
For quality engineering teams, this creates a new challenge: proving not only what a system does, but also demonstrating the boundaries of how it is intended to be used.
Continuous validation
The banking sector is expected to scrutinise sections of the guidance dealing with creditworthiness assessments and other decision-making applications that could directly affect customers.
The Commission includes examples covering “biometric, employment/recruitment and creditworthiness categories,” areas where financial institutions are increasingly deploying AI-driven systems.
Such applications are likely to require ongoing validation, monitoring and governance controls long after initial deployment.
This is where AI governance increasingly intersects with traditional software testing disciplines. Model drift, changing customer behaviour, evolving datasets and emerging risks mean that validation cannot be treated as a one-time exercise.
Instead, organisations may need to establish continuous testing and monitoring frameworks capable of demonstrating ongoing compliance and operational resilience.
New workload for QA teams
The publication arrives as regulators across Europe continue to increase their focus on operational resilience, AI governance and technology risk management.
For banks already investing heavily in DORA programmes, AI assurance frameworks and model risk management controls, the guidance provides another indication that testing is expanding beyond traditional software quality metrics.
The challenge is no longer confined to identifying defects or validating functionality. Increasingly, institutions must also demonstrate governance, traceability, accountability and effective oversight.
Hogan Lovells pointed out that “the scope of the guidelines is limited to determining whether an AI system is high-risk or not.” Yet for testing and resilience teams, that determination may prove to be only the beginning.
The firm also stressed that “the guidelines will be complemented at a later date by guidelines on the obligations for providers and deployers of high-risk AI systems,” adding that “harmonised standards or implementing acts may also be applicable, once developed.”
For financial institutions, the direction of travel is becoming increasingly clear. The classification of an AI system may start as a compliance exercise, but the ability to prove that the system is controlled, monitored, documented and resilient is rapidly becoming a testing challenge.
16 SEPTEMBER IN LONDON
REGISTER TODAY – SIMPLY CLICK HERE
Why not become a QA Financial subscriber?
It’s entirely FREE
* Receive our weekly newsletter every Wednesday * Get priority invitations to our Forum events *
REGULATION & COMPLIANCE
Looking for more news on regulations and compliance requirements driving developments in software quality engineering at financial firms? Visit our dedicated Regulation & Compliance page here.
READ MORE
- Inside Rabobank: Engineering resilience by design
- Can AI agents finally automate data testing?
- Continuous testing drives DORA compliance
- Why software testing may face a major rethink
- Buy or build? AI rewrites software testing for banks
WATCH NOW
QA FINANCIAL PODCASTS
CLICK HERE TO LISTEN TO OUR EXCLUSIVE CONVERSATIONS
