Deep Dive: why do most AI testing projects fail to scale?

London-based Anand Rao
London-based Anand Rao

Many industries are keen to leverage artificial intelligence to boost productivity or enhance creative output. The financial services industry is no exception, especially when it comes to software testing and taking a financial institution’s QA strategy to the next level.

However, several recent studies have shown that nearly three-quarters of AI scaling efforts simply fail to make a real impact.

So, why do so many of these initiatives fall short? According to Anand Rao, there are several reasons for that.

“The few primary reasons are legacy infrastructure and higher than expected costs and ‘disengagement’ by unexpected stakeholders,” explained the LTIMindtree Consulting vice president and portfolio head UK, Ireland & Europe at the global firm.

Many banks and financial institutions businesses start with AI projects designed to augment decision-making and automate testing tasks, Rao said.

“While pilots often show promising results, boosting efficiency, reduce workloads or speed of response, scaling them to manage millions of interactions daily requires substantial infrastructure upgrades, effective AI system integration, cost management, and stakeholder collaboration,” he analysed.

So, why does scaling from a limited pilot to an enterprise-wide solution pose such challenges?

“The answer lies largely in the infrastructure,” London-based Rao continued.

He added that plots usually involve simple systems with limited data and processing needs, but expanding these systems to enterprise level – encompassing customer service, supply chains, and financial systems – requires significant technological upgrades.

“Picture a small, efficient assistant evolving into a data-intensive system managing millions of interactions,” Rao noted.

“This demands powerful processors, vast storage, and sophisticated data pipelines to handle the increased load.”


“Business transformation with AI entails a holistic approach that combines strategic goals and upgrades, ensuring that AI adds value enterprise-wide.”

– Anand Rao

For example, imagine a company scaling AI to handle a million daily interactions, he continued.

“This shift, from hundreds of interactions to millions, needs robust processing power, rapid data storage, and real-time monitoring. Specialised chips and cloud-based graphics processing unit clusters are often necessary to manage the demands of Large Language Models,” Rao explained.

Scaling AI could cost anywhere from $380,000 to $630,000 per year, depending on infrastructure complexity, he estimated.

“For AI to scale effectively, financial organisations must establish AI system integration that supports fast data processing, secure data storage, and a strong legal and governance framework for accountability,” Rao said.

“Scaling can quickly become expensive,” he pointed out.

“If these needs aren’t met, the system may slow down, user satisfaction might decline, and operational savings may disappear. So, what does it take to ensure an AI system scales smoothly?”

For AI to deliver measurable value, businesses must have a clear plan to scale their AI initiatives.

Success requires aligning AI initiatives with business goals, choosing strategic use cases, monitoring return on investment, and preparing for future advancements, Rao wrote in a recent analysis.

“Business transformation with AI entails a holistic approach that combines strategic goals and technological upgrades, ensuring that AI adds value enterprise-wide,” he shared.

Rao is convinced that “blending business and technology transformations is key to helping organisations scale AI effectively.”

This approach allows enterprises to mature their AI capabilities while balancing technology with business outcomes, he added.

Compliance challenges

Like many banks and other financial services firms have experienced in recent years, a significant hurdle in scaling AI is ensuring AI solutions are compliant with regulations.

Both the UK and EU mandate that AI systems be robust, accurate, reliable, and trustworthy, with respect for privacy and human rights.

The EU AI Act and the General Data Protection Regulation (GDPR) require transparency, fairness, accountability, and robustness in AI.

How do businesses navigate these stringent standards? “For example, if a company collects sensitive customer data, it must comply with GDPR to avoid fines up to $22 million or 4% of annual global turnover,” Rao said.

The EU AI Act further imposes fines for violations, reaching US $38 million or 7% of annual turnover. Non-compliance not only risks severe fines but can also erode customer trust and dilute the financial benefits of AI, he pointed out.

“Building trustworthy AI involves more than technology, it’s also about human oversight,” Rao stressed. “Effective governance frameworks that ensure your AI models are explainable, auditable and unbiased.

He said that financial services firms should not sit still. In fact, they can implement several key actions, such as regular auditing of models to spot changes in the data distribution or illegalities in the model, bias checks to ensure fair treatment across demographics, as well as secure data management practices.

Although building these systems can be costly, Rao thinks the long-term benefits like lower compliance expenses and stronger customer trust outweigh the initial investment.

Hidden costs

As AI systems scale, token usage costs can accumulate quickly, Rao pointed out. “Each time a large language model processes data, it consumes tokens,” he explained.

“For instance, handling 100,000 tests daily, with each interaction using 500 tokens at US $0.0006 per token, would cost $30,000 daily or $600,000 annually. Without careful cost management, token expenses alone can wipe out the financial benefits of scaling AI.”

So how can firms reduce these scaling costs? Rao thinks implementing feedback loops for continuous AI fine-tuning and evolutionary algorithms to optimise energy usage and reduce token consumption are both effective strategies. For instance, a 30% reduction in token usage could save approximately $420,000 annually.

Additionally, “choosing cloud services that offer cost-efficient storage can also aid in scaling AI without excessive costs,” he shared.

“Profitability is maintained by inexpensive measures, and therefore, AI transformation of technology to activities allow businesses to reduce operating costs by balancing AI innovation with cost reduction for positive ROI,” Rao continued.

“Further, AI usage and performance monitoring enables firms to keep track of operating costs while reducing them on part of executive industry,” he said.

The balance between AI innovation and cost reduction ensures that AI systems deliver positive ROI over time, Rao explained, despite higher initial investments, with operational monitoring helping keep costs manageable while increasing efficiency.


“It is not just the technology that is important for AI scaling to succeed; it is also the people you engage with.”

– Anand Rao

Rao was keen to stress that a lack of buy-in from key business functions is a common reason for failed AI projects.

“Without early stakeholder involvement, AI initiatives may face resistance or be seen as threats to core business objectives,” Rao said.

If stakeholders don’t fully support AI, systems may be underutilised, leading to lower impact and reduced ROI, he noted.

“Early engagement, AI literacy programs, and positioning AI as a partner that enhances human work can foster company-wide support and drive meaningful AI outcomes.”

So, in summary, Rao stressed that scaling AI involves more than just technological advancements; it requires a fundamental shift in business culture.

“A truly successful AI journey bridges the gap between teams, fostering collaboration, and ensuring that every part of the organisation derives meaningful value from the transformation,” he concluded.


THIS MONTH


Why not become a QA Financial subscriber?

It’s entirely FREE

* Receive our weekly newsletter every Wednesday * Get priority invitations to our Forum events *

REGISTER HERE TODAY


COMING UP IN MARCH

QA Financial is delighted to announce that Tal Barmeir will join us as a speaker at the QA Financial Forum Toronto 2025 Places are limited – register today.


DO NOT MISS


READ MORE