The integration of AI into quality assurance offers both exciting possibilities and significant risks.
In one corner, we have quality assurance (QA), the bedrock of safety and precision in industries ranging from aerospace to healthcare.
In the other corner, artificial intelligence (AI), a powerful yet unproven contender in the realm of QA.
The question isn’t just who will win, but how can they coexist?
The integration of AI into QA processes brings both significant opportunities and serious risks, particularly in safety-critical environments where lives and reputations are on the line, according to Joseph Sorrentino, president of Lean Quality Systems, a quality assurance consulting firm.
The context of AI and QA
As the software development life cycle (SDLC) and product development life cycle are essential frameworks in QA, they guide the development, testing, and deployment of products and software, Sorrentino argues.
“Each phase—from requirements gathering to design, implementation, testing, and maintenance—requires meticulous attention to detail and adherence to safety standards,” he wrote in a recent QM analysis.
Therefore, AI presents both opportunities and challenges at each stage.
“AI can help identify and prioritize requirements by analysing large datasets,” Sorrentino pointed out.
“However, the inherent complexity and unpredictability of AI models raise concerns about whether these requirements can be met consistently, especially in safety-critical applications,” he was quick to add.
In terms of design, AI tools can optimize design processes, but the lack of transparency in AI decision-making poses a significant risk.
“Designers must ensure that AI-generated solutions are fully understood and meet all safety and regulatory requirements,” Sorrentino warned.
When it comes to implementation, integrating AI into QA processes can improve efficiency, but it also introduces the risk of errors.
“AI models, particularly large language models (LLMs), are prone to ‘hallucinations’—generating plausible but incorrect information—which could compromise the integrity of the product,” Sorrentino explained.
And then there is obviously the vital process of testing.
“AI can assist in automating and accelerating testing processes, but its lack of reliability in producing consistent, repeatable results is a major drawback,” he pointed out.
“In QA, where Six Sigma standards demand near-perfect accuracy, AI’s current error rate of approximately 10% is unacceptable,” Sorrentino stressed.
An extension of this process is maintenance.
“AI can help monitor systems in real-time, predicting potential failures before they occur,” he continued.
“However, the ability to trace and validate AI-driven decisions remains a challenge, making it difficult to ensure ongoing compliance with safety standards,” Sorrentino noted.
Role-based impact
Sorrentino argued that different roles within an organisation view AI through various lenses, each with unique concerns and responsibilities.
Firstly, there are the CEOs and COOs. “For senior executives, AI is often seen as a tool for increasing efficiency and profit margins,” he said.
However, the reputational risks associated with AI failures, particularly in safety-critical industries, must be carefully weighed against potential gains, Sorrentino highlighted.
Meanwhile, QA managers are tasked with maintaining the highest standards of safety and compliance.
“They must navigate the challenges of integrating AI into existing QA processes while ensuring that these systems do not undermine the rigorous standards that their industries demand,” he pointed out.
In addition, there is the engineers and technicians. “Those on the ground implementing AI in QA processes face practical challenges, such as ensuring that AI tools are correctly configured and do not introduce errors,” Sorrentino explained.
“Human oversight remains critical, especially when AI is deployed in environments where safety is paramount.”
Case studies
AI has shown promise in certain aspects of QA, according to Sorrentino, particularly in data analysis and pattern recognition.
For example, Palantir, a big data analytics platform, leverages AI to fuse vast amounts of sensor data and identify patterns that might otherwise go unnoticed.
However, AI’s limitations are starkly evident in other areas.
For instance, large language models like ChatGPT and Google’s Bard are built on billions of parameters, making them prone to inaccuracies when applied to specific vertical industries like QA.
“The challenge lies in their inability to consistently produce accurate, repeatable results—a critical requirement in QA,” Sorrentino said.
Therefore, best practices for implementing AI in QA are paramount, he stressed.
“To successfully integrate AI into QA processes, organizations must adopt a cautious and methodical approach,” Sorrentino noted, such as developing clear standards.
“Establish clear guidelines for AI use in QA, ensuring that all safety and compliance requirements are met,” he explained.
“This includes setting strict parameters for AI decision-making and requiring human oversight for critical tasks.”
Sorrentino also emphasised human oversight.
“AI should augment, not replace, human judgment in QA processes,” he added. “Ensure that skilled professionals are involved in all stages of the SDLC/PDLC to validate AI-driven decisions and maintain accountability.”
And thorough testing should not be underestimated.
“Rigorously test AI systems before deployment, focusing on their ability to produce consistent, repeatable results,” Sorrentino said. “Use additional tools and methods to validate AI outputs, particularly in safety-critical applications.”
Finally, he believes that safety and compliance should be an absolute priority. “Never compromise on safety standards, even if AI promises efficiency gains.”
‘Seasoned catcher’
“As a seasoned catcher, every pitch is a test, and every game is a high-stakes QA process,” Sorrentino continued.
“My job is to ensure that every throw, every catch, and every decision on the field meets the highest standards of precision and timing. Just like in QA, it’s about minimizing errors and maximizing success,” he continued.
Sorrentino added: “Imagine if AI were my coach. It could analyse countless game scenarios and predict the best strategies.”
But would it understand the nuances, the human elements, like reading the pitcher’s mood or gauging the batter’s intent? Could it guarantee a perfect throw to second base, every time?
“In my world, there’s no room for a 10% error rate. One bad decision can cost us the game,” he continued.
“That’s why, despite all the tech and analytics, nothing replaces the experience, intuition, and critical thinking of a player on the field. The stakes are just too high for anything less than perfection,” Sorrentino noted.
NEXT MONTH IN SINGAPORE
REGISTRATION IS NOW OPEN FOR THE QA FINANCIAL FORUM SINGAPORE 2024
Test automation, data and software risk management in the era of AI
The QA Financial Forum launches in Singapore on November 6th, 2024, at the Tanglin club.
An invited audience of DevOps, testing and quality engineering leaders from financial firms will hear presentations from expert speakers.
Delegate places are free for employees of banks, insurance companies, capital market firms and trading venues.
READ MORE
- Cognizant drags rival Infosys to court over trade secrets
- Testaify claims tool is ‘100x faster than seasoned QA architect’
- Fast-growing Newgen sets sights on banks in Middle East
- ABN Amro hires nCino and CBA for digital upgrade
- QAFF London: Lloyds’ Richard Bishop on the rise of ‘green software’
Become a QA Financial subscriber – for FREE
News and interviews * Receive our weekly newsletter * Get priority invitations to our Forum events
