Stephen Feloney has spent over 25 years developing and bringing software testing products to market.
During this time, he witnessed significant changes, such as automated testing, continuous testing, low-code/no-code platforms, Centers of Excellence, and the shift-left approach.
Yet, despite these advancements, one thing has remained constant, according to the VP of products, continuous testing at Perforce: “Testing remains a time-consuming, an inefficient bottleneck in the software development lifecycle.”
Although Feloney is the first to acknowledge that testing has become less of a bottleneck over the years, “we have never truly achieved seamless validation.”
The former Broadcom staffer pointed out that “today, AI is at the forefront of the evolution of our testing products, as with many other vendors, but the key question remains, such as will it make testing more efficient? What about the human element? Will AI reduce mundane tasks, or will it simply add another layer of complexity, as past ‘advancements’ have done?”
Automated testing
Practically all major vendors have started incorporating AI into their testing platforms.
Many have opted for AI copilots to assist in test creation, but this is only part of the solution, Feloney argued.
“With AI advancing rapidly, we must look beyond copilots and focus on more meaningful accuracy, efficiency, and speed improvements,” he stressed.
While copilots can accelerate test creation, they don’t solve some of testing’s more significant challenges, the industry insider continued.
“Feedback from users has been mixed. Some developers find copilots helpful, but many report that they sometimes generate incorrect tests or create more tests that need ongoing maintenance.”
Instead of addressing core problems, copilots risk exacerbating them, simply layering AI over an already broken process, Feloney said.
Testing is still ‘broken’
To understand why Feloney thinks that, he urged to look at the evolution of automated software testing. “The industry has developed various techniques for identifying objects on screen, enabling tests to run,” he said.
“While some methods work better than others, the fact remains that changes to the UI or application break tests necessitate constant maintenance, an estimated 30-40% of a tester’s time,” Feloney shared.
He takes regression testing as an example: regression suites must be rerun whenever a change is made to detect new issues.
In large applications, these tests can number in the thousands, yet it’s estimated that only 20% of tests in regression suites are valid, he said.
“Many of these tests produce errors that require time-consuming analysis to verify their authenticity.”
“It’s no wonder some enterprises still rely on manual testing, even though they know they can’t achieve full test coverage.”
– Stephen Feloney
When discussing how to evolve web, mobile, and performance test suites, Feloney said it was realised that while copilots assist in writing test scripts, they don’t address users’ most significant pain points: test maintenance and analysis.
“That’s why it’s best to focus on areas where value can be delivered, starting with test validation,” he added.
“We need to rethink how to approach testing. Can the need for scripting and object-based locators be eliminated?”
He said the next evolution in automated testing is not simply visualising what’s on the screen but using AI to understand it contextually and dynamically.
“By leveraging natural language processing, users can ask simple questions, and the system can analyse and validate tests accordingly without relying on objects or code or even needing to understand the underlying technologies of the applications.”
Feloney continued: “The result? Real-time validation that works across any platform and is usable by both technical and non-technical team members. It also survives massive changes to the code without manual intervention.”
While the immediate benefits of AI-driven test validation are clear to Feloney, the real value comes from hands-on experience and feedback from real users.
“Financial institutions, for example, have used this approach to validate data they could never validate before, such as checking whether on-screen graphs match the underlying tables,” he pointed out.
Moving beyond validation
But test validation is just the beginning, Feloney believes.
“To truly transform testing, we must look at how AI can free testers and developers from routine tasks, allowing them to focus on more valuable work.”
The goal is to democratize testing — making it accessible and efficient for everyone.
“I am not a fan of the word ‘democratize’, but when used correctly, this is genuinely what AI will enable,” he said.
To Feloney, the future of software testing is not about applying AI for the sake of it or adopting short-term fixes.
“It is about harnessing the most significant advancements in computing to solve real-world problems and finally fixing the broken processes plaguing software testing for years,” he concluded.
Why not become a QA Financial subscriber?
It’s entirely FREE
* Receive our weekly newsletter every Wednesday * Get priority invitations to our Forum events *
REGULATION & COMPLIANCE
Looking for more news on regulations and compliance requirements driving developments in software quality engineering at financial firms? Visit our dedicated Regulation & Compliance page here.
WATCH NOW
READ MORE
- Xceptor joins forces with OnCorps AI to target banks in automation drive
- QA Forum NY: Varo Bank’s Nicole Nunziata on secure coding
- Why QA teams may need to fear OpenAI’s bold AI agent plans
- Banks warned not to embed too much GenAI in digital systems
- UiPath turns to the cloud with its latest AI agent solution
