Agentic AI is poised to fundamentally reshape the software development and quality assurance landscape, particularly in highly regulated industries such as banking and financial services.
According to Rob Mason, chief technology officer at U.S.-based software testing firm Applause, this shift demands new mental models, design patterns, and testing strategies from both developers and QA professionals.
“Agentic AI represents a major evolution in how software systems operate, and it’s already changing the way we build and test applications,” said Mason.
“Unlike traditional AI models or rule-based automation, agentic AI systems are designed to pursue goals autonomously, making real-time decisions using planning, memory, tool use, and feedback loops.”
Mason explained how the autonomy of agentic systems introduces flexibility, but also unpredictability. “They don’t just execute; they act.”
He added: “This autonomy introduces flexibility, agents can solve problems in novel ways, but it also introduces non-determinism. The actions agents take depend on current context, prior memory, available tools, and how well they understand the task.”
“Unlike traditional AI models or rule-based automation, agentic AI systems are designed to pursue goals autonomously.”
– Rob Mason
From a development perspective, this means a shift from deterministic code execution to building dynamic environments that agents can navigate. “Developers still write code, but now much of that code is exposed as tools that agents can choose to use,” Mason noted. “You’re essentially setting up an autonomous playground with guardrails, not a locked-down assembly line.”
Prompt engineering, too, becomes a core part of the development lifecycle. “With agentic AI, the prompt centers around higher-level goals and parameters to guide the agent to reach the goal,” said Mason.
“These prompts become part of your interface surface, a new kind of API contract, but written in language instead of code.”
Debugging agentic systems is also a departure from traditional approaches. “Failures often stem from poor reasoning, ambiguous planning, or misused tools. That means developers must shift from debugging code execution to debugging agent behavior,” Mason said.
“You’ll inspect decision traces: Why did the agent choose this path? Was the plan coherent?”
This evolution impacts QA teams just as significantly. “Traditional QA assumes a known set of inputs and expected outputs. With agentic AI, behavior is less predictable,” Mason warned.
“From a QA perspective, we still need human creativity. We rely on our community of testers to introduce unexpected variables and uncover edge cases.”
New metrics
New metrics and infrastructures are required to assess performance and ensure reliability. “Agentic AI calls for new evaluation metrics. In addition to traditional QA KPIs, teams must assess metrics like accuracy, relevance, and hallucination rates,” Mason explained.
“It can be difficult for something with autonomy to evaluate whether it did a good job; that’s subjective. And that’s where humans are good: at evaluating subjective versus objective results.”
Purpose-built infrastructure is essential to test the autonomous and non-deterministic behaviors of agentic systems. “Replay systems for analysing past decisions… Synthetic goal generators to test diverse behaviors… Observability tools to detect drift and anomalies,” Mason listed. “These tools uncover anomalies that indicate when agent behavior is degrading or deviating from acceptable bounds.”
Importantly, Mason sees collaboration between development, QA, and AI/ML teams as critical. “Logs, test failures, and user corrections can feed back into the system to improve performance,” he stated.
“We’re moving from a release-and-forget model to a continuous tuning loop, more akin to MLOps than traditional software delivery.”
Ultimately, Mason believes embracing agentic AI will be a key differentiator for teams working in complex environments such as financial services.
“Agentic AI does not just add intelligence to software, it demands new mental models, new design patterns, and new quality criteria,” he concluded.
“Teams that build the right tooling, observability, and collaboration around agent behavior will be best positioned to deliver intelligent, adaptive software that scales,” Mason said.
NEW EVENT!
Why not become a QA Financial subscriber?
It’s entirely FREE
* Receive our weekly newsletter every Wednesday * Get priority invitations to our Forum events *
REGULATION & COMPLIANCE
Looking for more news on regulations and compliance requirements driving developments in software quality engineering at financial firms? Visit our dedicated Regulation & Compliance page here.
READ MORE
- Why real-time monitoring and scenario testing are becoming core QA disciplines
- BankDhofar takes an automated approach to strengthen QA
- Banks warned AI still fails on real-world software testing tasks
- SEC’s AI emphasis drives new QA and testing imperatives for US banks
- Inside the chaos: The new reliability discipline reshaping banking QA
WATCH NOW
