As banks accelerate digital transformation and AI tools begin to generate code at scale, the definition of software quality is shifting.
Traditional testing approaches, built around verifying whether applications behave as expected, are struggling to keep pace with modern architectures that are distributed, constantly evolving and increasingly dependent on third-party services.
For QA and software testing teams in financial services, this shift is not theoretical. It is already playing out in production environments where systems that pass functional tests can still fail under real-world conditions.
Infrastructure dependencies break, traffic patterns spike unpredictably, and microservices interact in ways that are difficult to model in controlled environments.
At the same time, regulatory pressure around operational resilience is intensifying. Banks are being pushed to prove not just that their systems work, but that they can withstand disruption, recover quickly and continue delivering critical services.
Against that backdrop, a growing number of banks and financial firms are rethinking how they define and validate software quality.
“Software testing is a broad and fairly stable discipline for a large part of the industry. But what has changed is the growth of software systems themselves and the nature of architecture,” explained Umasankar Mukkara, founder of ChaosNative, later acquired by Harness, and now VP of Product at Harness.
“With cloud-native systems, everything is broken into microservices. Instead of having ten VMs running an application, you might now have a hundred microservices,” Mukkara said.
As architectures evolve, so too does the scale and speed of software delivery. “Instead of shipping changes every six months or every quarter, you are doing daily builds. That creates a huge increase in dynamism.”
For banks operating across regions and relying on complex ecosystems of services, this dynamism introduces new failure modes that traditional QA processes do not always capture.
“Sometimes, a small change somewhere, often outside your control, can end up affecting services,” Mukkara noted.
This is driving a shift away from purely functional validation towards a broader focus on how systems behave under stress, failure and unexpected conditions.
“What matters is expecting things to change or fail in small ways, and then validating the resilience of your systems, your deployment model, and your architecture by deliberately introducing those failures.”
Risk of resilience debt
As development accelerates, particularly with the support of AI, the volume of code entering production is increasing rapidly. Without a corresponding evolution in testing, this creates a growing backlog of untested risk.
“If you do not do resilience testing, you can think of it as leaving some resilience debt on the table. Just like tech debt, you have resilience debt, and yet you still ship the product,” Mukkara stressed.
For financial institutions, this “resilience debt” can manifest in outages, failed failovers and degraded customer experiences, all of which carry regulatory and financial consequences.
“The challenge is that while AI and modern development are putting more code on the table to be tested, if you do not cover resilience testing to a level you are comfortable with, you are effectively leaving unknowns as unknowns,” Mukkara pointed out.
“If you do not do resilience testing, you can think of it as leaving some resilience debt on the table.”
– Umasankar Mukkara
While chaos engineering has helped organisations surface weaknesses by injecting controlled failures, it is no longer sufficient on its own.
“There can be small failures, which is what we broadly call chaos engineering. Then there are other scenarios such as introducing load and seeing how the system behaves, or simulating disasters,” he said.
In banking environments, this means testing recovery capabilities as rigorously as functionality.
“You also need to test things like taking down zones or regions and seeing whether the software that is supposed to go into disaster mode actually recovers,” Mukkara said in a recent DataQuest analysis.
“Are your RTO (Recovery Time Objective) and RPO (Recovery Point Objective) targets really being met per service or not?”
Together, these practices form a more comprehensive resilience testing strategy that aligns with the industry’s increasing focus on operational resilience.
Shifting resilience left
The concept of shifting testing earlier in the development lifecycle is not new, but its application is changing.
“What shift left means for resilience testing is this: do more of it before you ship the product. Put resilience testing into your SDLC as a gate and then ship,” Mukkara said.
For QA teams, this requires embedding resilience validation into pipelines rather than relying on production-based testing alone.
“If you are introducing ten new services as part of a feature release, you can say that these services must go through a defined level of resilience testing… before we even ship the product.”
Over time, this approach helps reduce uncertainty and improve preparedness.
“Over time, the known unknowns reduce, the risks are documented, and you are better prepared.”
Breaking down testing silos
One of the biggest barriers to effective resilience testing is organisational fragmentation.
“The performance testing team is different, the chaos team is different, the disaster recovery team is different, and the game day teams are different.”
“That fragmentation has become another challenge over the last two years.”
For banks, where testing functions are often distributed across internal teams and external partners, this separation can limit visibility and coordination.
“Load testing, chaos testing and DR testing were all part of a larger resilience conversation.”
“Load testing was not just about scale any more. Chaos testing was about accelerating the discovery of weaknesses. DR testing was about compliance and recovery assurance.”
Bringing these disciplines together is increasingly seen as essential to building a coherent resilience strategy.
“Sometimes, a small change somewhere, often outside your control, can end up affecting services.”
– Umasankar Mukkara
AI is accelerating software development, but it is also amplifying the burden on testing teams. “In fact, more testing is needed as AI gets to write code faster.”
“When a much larger volume of code is being generated, somebody still needs to test it.”
For financial institutions, this creates a tension between speed and stability.
“The purpose of writing code faster is to accelerate end-to-end software delivery, but that acceleration cannot come at the cost of resilience and reliability.”
“If you ship code ten times faster and it breaks, and if outages double in reality, then it becomes a business loss in terms of reputation and finance.”
AI is also becoming part of the testing solution itself. “One human will increasingly use AI to write tests better.”
But the core challenge remains. “There is going to be much more innovation around how you test, especially around resilience.”
Rethinking the outer loop
As AI transforms how code is written, attention is shifting to how it is delivered, tested and operated.
“The real question then is: how do you get that code out with the same or better speed, quality and efficiency?”
“How do you test more reliably without leaving resilience debt on the table?”
For banks, insurers and other financial institutions, answering these questions is becoming central to maintaining trust and meeting regulatory expectations.
“If the industry does not transform the outer loop with the same seriousness as the inner loop, the gains from faster coding will be lost in weak testing, poor resilience and inefficient deployment.”
In that environment, resilience testing is emerging as a defining capability for QA teams, shaping how software quality is measured in an increasingly complex and AI-driven financial ecosystem.
Why not become a QA Financial subscriber?
It’s entirely FREE
* Receive our weekly newsletter every Wednesday * Get priority invitations to our Forum events *
REGULATION & COMPLIANCE
Looking for more news on regulations and compliance requirements driving developments in software quality engineering at financial firms? Visit our dedicated Regulation & Compliance page here.
READ MORE
- Trust, not speed: Why AI governance is now a testing battleground for banks
- NatWest’s AI trade finance overhaul opens new chapter for QA teams
- Banking UAT moves beyond sign-off as QA takes centre stage in system rollouts
- Citi ramps up AI-driven testing in race to modernise legacy systems
- Lloyds, HSBC and NatWest get OpenAI access amid mounting concerns
WATCH NOW
QA FINANCIAL PODCASTS
CLICK HERE TO LISTEN TO OUR EXCLUSIVE CONVERSATIONS
