Test automation continues to sit at the centre of most QA transformation strategies in banking and financial services, yet many teams remain frustrated by the gap between ambition and impact.
According to industry insider Rohit Raghuvansi, Global VP of Engineering at Leapwork, the problem is rarely a lack of effort.
“For QA and software engineering teams, test automation is a little like new year’s resolutions,” Raghuvansi pointed out. “It’s easy to set lofty goals, and even to pursue them nominally. But it’s much harder to achieve meaningful results.”
He observed that teams often define success in narrow, activity-based terms. “QA engineers might say, for example, that they want to automate a certain percentage of their tests, and they might even make nominal progress toward that goal by writing more test cases or executing automated tests more frequently.”
The issue, Raghuvansi argued, is that these surface-level gains frequently fail to translate into better delivery outcomes.
“When you drill down into what’s actually happening, it’s often the case that the nominal gains translate to very little in the way of true value creation,” he explained.
“More frequent automated tests don’t usually improve software delivery frequently or application quality if they are not paired with other modifications, like changes to a team’s cultural norms surrounding software testing, that increase test effectiveness and confidence.”
“More frequent automated tests don’t usually improve software delivery.”
– Rohit Raghuvansi
To illustrate the point, Raghuvansi likened test automation programmes to fitness goals pursued without real change.
“This is a bit like what happens if you promise that, starting January 1, you’ll go to the gym more frequently and you actually do, but without scaling up the intensity of your workouts or investing in new types of training,” he said.
“Simply increasing the number of hours you spend in the gym each week is not likely to translate to an improvement in overall health,” Raghuvansi added.
Despite this, the expected benefits of automation remain widely understood. “For most, recognizing the benefits of test automation is easy enough,” Raghuvansi continued.
“They expect it to lead to faster delivery cycles, earlier detection of bugs and fewer defects in deployed applications.”
Targets and wire tests
In pursuit of those goals, many organisations set automation targets and wire tests into CI/CD pipelines. “Hoping to achieve these goals, teams often set a target for the percentage of tests that they want to automate, with something in the range of 20–30 percent being a common goal,” Raghuvansi said. “In theory, this should be great.”
In practice, however, automation often undermines confidence rather than strengthening it.
“The problem, though, is that merely automating tests is no guarantee of real progress,” he warned, pointing to environments where “automated tests deliver little real value” due to instability, unreliable results and limited understanding across teams.
The consequence is what Raghuvansi described as a “crisis of confidence.”
“They don’t trust automated tests to be reliable and so, even if the tests suggest that code is bug-free and ready for release, no one is actually confident pressing the ‘go’ button.”
That lack of trust frequently pushes teams back to manual processes. “This can also lead to scenarios where engineers end up testing everything manually, even if they already tested it automatically, because they won’t trust test results until they obtain them by hand,” Raghuvansi said. “At this point, test automation has achieved no real value at all.”
While teams may be tempted to blame tools or application complexity, he believes the deeper causes lie elsewhere.
“On the surface, it can be tempting to blame technical factors alone,” adding that “but the root causes of test automation shortcomings usually boil down to cultural and organizational challenges at least as much as technical barriers.”
Those challenges include limited familiarity with automation tools, fear of adopting new technologies and an over-reliance on a small group of specialists.
“A heavy reliance on ‘heroes’ to drive test automation strategies” and “the expectation that test automation will result in fast, easy wins” often undermine long-term commitment, Raghuvansi argued.
“Increasing automation for its own sake is of no value if it doesn’t improve overall outcomes.”
– Rohit Raghuvansi
Addressing these issues requires a shift in mindset rather than another tooling overhaul. “Given that the root causes of test automation shortcomings tend to be cultural and organizational, the solutions must also focus on changing organizational culture,” he added.
Instead of chasing coverage metrics, Raghuvansi urges teams to rethink how success is measured.
“Instead of thinking in terms of how many tests the team automates, focus on outcome-centric metrics, like the frequency of application deployments and regression cycles,” he said.
Ownership also needs to be made explicit. “Clearly define who ‘owns’ automated testing,” Raghuvansi went on to say, adding that this clarity is essential to prevent automation from becoming the domain of a few individuals rather than a shared responsibility.
For regulated environments such as banking, automation must also be formalised within delivery processes. “Teams should also define formally which role test automation plays in the software ‘delivery contract,’” he said.
“Setting clear, consistent expectations in this regard helps to build confidence that automated tests are a routine, reliable part of the overall development lifecycle.”
QA ‘needs to evolve’
Raghuvansi also argued that QA itself needs to evolve. “Rather than thinking of the role of QA engineers as being limited to deploying and executing tests, adopt an organisational mindset that treats QA as an architectural pursuit,” he shared.
“The purpose of QA should be defining the overall processes that optimize software quality and development efficiency.”
AI-driven automation, meanwhile, should be positioned carefully. “Teams are much likelier to trust AI-driven automated tests when the role of the tests is to reduce toil, not remove humans from feedback loops,” Raghuvansi remarked.
Used this way, “AI helps to reduce noise and effort, but without undercutting confidence in automated testing because humans remain in charge.”
Ultimately, he believes the industry needs to reset its expectations. “Unlocking the full power of test automation requires rethinking the role of test automation in software engineering,” Raghuvansi concluded.
“For too long, the focus has been on counting automated test coverage or speed. The real goal should be on analyzing the relationship between automated testing and software quality outcomes.”
COMING IN 2026
Why not become a QA Financial subscriber?
It’s entirely FREE
* Receive our weekly newsletter every Wednesday * Get priority invitations to our Forum events *
REGULATION & COMPLIANCE
Looking for more news on regulations and compliance requirements driving developments in software quality engineering at financial firms? Visit our dedicated Regulation & Compliance page here.
READ MORE
- Goldman puts AI coding to the test
- How to test AI models that banks do not control
- OpenAI, Filigran and SunTec: the latest vendor and product news
- Sygnum: Testing AI is ‘a measurement problem’
- Banks’ ‘code for all’ push raises testing risks
WATCH NOW
QA FINANCIAL PODCASTS
