Australia’s main regulator for the financial sector, The Australian Prudential Regulation Authority, has warned banks, insurers and superannuation trustees that governance, quality assurance and operational resilience practices are failing to keep pace with the rapid adoption of artificial intelligence.
The move is likely to intensify scrutiny on AI testing, software assurance and cyber resilience programmes across financial services firms down under.
In a recent industry letter and accompanying supervisory findings, APRA said it is seeking a “step-change” in how regulated entities manage AI-related risk as advanced AI systems become more deeply embedded into critical operations, customer-facing applications and software engineering environments.
“The AI revolution presents tremendous opportunities for banks, insurers and superannuation trustees to deliver improved efficiency and enhanced customer services,” explained APRA Member Therese McCarthy Hockey.
“But we cannot be blind to the risks of such powerful technology, whether in our own hands or the hands of those with malign intent,” she stressed.
She said that APRA’s most recent supervisory review across major financial institutions found that “governance, risk management, assurance and operational resilience practices are not keeping pace with the scale, speed, and complexity of AI adoption.”
“The systems and processes required to safely govern AI use aren’t keeping up.”
– Therese McCarthy Hockey
The regulator’s concerns land as financial institutions globally accelerate the deployment of AI-enabled software development, agentic AI systems, automated decision-making tools and AI-assisted cyber defence capabilities.
APRA warned that frontier AI models, including Anthropic’s Claude Mythos, “could enhance the discovery of vulnerabilities by bad actors” and “are expected to further increase the probability, speed and scale of cyber attacks.”
“What we’ve observed from our supervisory engagement is that while AI adoption is continuing apace, the systems and processes required to safely govern its use aren’t keeping up,” McCarthy Hockey shared.
“Likewise, the speed at which entities can identify and patch vulnerabilities needs to operate much faster, commensurate with the AI-accelerated threat,” she added.
Continuous AI validation and testing
For QA and quality engineering teams at banks and insurers, some of APRA’s sharpest observations focused on weaknesses in software assurance, change management and testing programmes for AI systems.
The regulator said “the volume and speed of AI assisted software development is placing strain on the effectiveness of change and release management controls,” while also identifying “gaps in the scope and coverage of security testing programmes for both AI implementation and responding to the AI augmented threat environment.”
APRA also warned that many firms are still relying on outdated assurance models ill-suited to adaptive and probabilistic AI systems.
“APRA also observed reliance on point in time and sample based assurance methods, despite these methods being ill suited to probabilistic models that learn, adapt and degrade over time,” McCarthy Hockey stressed.
In a finding likely to resonate strongly with QE and software testing leaders, APRA said: “Few entities had continuous validation or monitoring in place to detect issues such as model drift, bias, failure modes, or control breakdowns in a timely manner.”
The regulator added that “assurance activities often lagged AI deployment,” particularly where “agentic behaviour, automated decision making or AI assisted code generation were involved.”
McCarthy Hockey warned financial entities should implement “robust security testing across AI-generated code, software components and libraries” and strengthen “continuous and proportionate” lifecycle monitoring for AI systems based on risk and criticality.
The regulator also called for “integrated assurance across cyber security, data governance, model performance risk, operational resilience, privacy, and conduct risks,” alongside stronger technical capability inside second-line risk management and internal audit teams.
Oversight under scrutiny
The regulator said many boards remain insufficiently prepared to oversee AI-related risks effectively, despite strong executive interest in AI adoption.
“Boards have strong interest for AI’s potential benefits but many lack the technical literacy required to provide effective challenge to management on AI related risks and oversight,” APRA said.
The regulator noted that many organisations showed “an overreliance on vendor presentations and summaries without sufficient examination of key AI risks such as unpredictable model behaviour and the impact on critical operations.”
APRA also warned that AI functionality is increasingly embedded inside software platforms, developer tooling and enterprise systems in ways that reduce transparency around how models are trained, updated and constrained.
“AI functionality is often embedded within broader software platforms or developer tooling, reducing transparency over where and how models are trained, updated or constrained and limiting entities’ ability to completely assess and manage risks,” the regulator said.
The findings also highlighted concentration risk concerns, with some institutions heavily dependent on a single AI provider across multiple use cases without credible contingency or substitution plans.
AI supplier risks
APRA said entities must reassess operational resilience, supplier risk and cyber defence frameworks as AI systems become more deeply integrated into banking and insurance technology stacks.
The regulator warned that “identity and access management capabilities have not yet adjusted to nonhuman actors such as AI agents,” while implementation timelines for remediation activities including “patching and configuration management” were “not consistently aligned to the accelerated threat environment.”
APRA said firms should maintain “credible fallback processes” where AI supports critical operations and ensure stronger controls around “agentic and autonomous workflows.”
The regulator also stressed the importance of visibility into AI supply chains and dependencies.
“Few entities had demonstrated robust contingency planning or tested exit and substitution strategies for critical AI providers,” the watchdog stated.
It added that “upstream dependencies such as foundation models, training data sources and fourth party service providers are opaque,” limiting institutions’ ability to “independently assess model performance, bias, resilience and security.”
‘Findings are candid’
In response to APRA’s letter, digital consultancy firm Canbury said APRA’s findings reflected mounting pressure on governance and assurance functions as AI adoption accelerates across regulated financial institutions.
“The findings are candid: AI adoption is accelerating, but governance, assurance and security practices aren’t keeping pace,” the consultancy said in a statement.
Canbury noted that APRA had “explicitly” identified board-level technical literacy gaps and warned that “assurance functions, internal audit and risk teams, often don’t have the skills or tools to assess AI systems, particularly where agentic behaviour or automated decision-making is involved.”
The consultancy also highlighted APRA’s concerns around supplier concentration and transparency, saying “supplier risk management exists on paper, but concentration risk and supply chain opacity are real and largely unaddressed.”
APRA said it is continuing to develop its supervisory roadmap for AI oversight and warned that institutions failing to manage AI risks appropriately could face escalating regulatory action.
“Where entities fail to adequately identify, manage or control AI risks in a manner proportionate to their size, scale and complexity, we will take stronger supervisory action and, where appropriate, pursue enforcement,” the regulator concluded.
Why not become a QA Financial subscriber?
It’s entirely FREE
* Receive our weekly newsletter every Wednesday * Get priority invitations to our Forum events *
REGULATION & COMPLIANCE
Looking for more news on regulations and compliance requirements driving developments in software quality engineering at financial firms? Visit our dedicated Regulation & Compliance page here.
READ MORE
- Trust, not speed: Why AI governance is now a testing battleground for banks
- NatWest’s AI trade finance overhaul opens new chapter for QA teams
- Banking UAT moves beyond sign-off as QA takes centre stage in system rollouts
- Citi ramps up AI-driven testing in race to modernise legacy systems
- Lloyds, HSBC and NatWest get OpenAI access amid mounting concerns
WATCH NOW
QA FINANCIAL PODCASTS
CLICK HERE TO LISTEN TO OUR EXCLUSIVE CONVERSATIONS
