The advent of generative AI (GenAI) is ushering in a new era in software testing. Acting as an AI test engineer, GenAI uses a test recorder designed to perform equivalent work to that of a human test automation engineer.
It interprets plain-language instructions to autonomously generate test automation code.
Looking at the rise of AI within the QA space, Tal Barmeir, the co-founder and chief executive officer of BlinqIO, called the journey from human automation tester to AI test automation engineer “transformative.”
“Traditionally, transitioning to test automation required significant time and resources, including learning to code and understanding automation frameworks,” Barmeir explained.
“AI removes these barriers and accelerates development cycles, dramatically reducing time-to-market and improving accuracy, all while decreasing the level of admin tasks for software testers,” she added.
Less coding complexities
AI-powered tools can interpret test scenarios written in plain language, automatically generate the necessary code for test automation, and execute tests across various platforms and languages.
“This dramatically reduces the enablement time, allowing QA professionals to focus on strategic tasks instead of coding complexities,” Barmeir wrote in a recent analysis.
“This also addresses the severe shortage in test automation, helping companies achieve quality across their digital offerings in no time, for any number of scenarios in parallel, and in a matter of minutes,” she continued.
“It creates test automation for multilingual websites and apps, as well as for any screen size, including computers, tablets and mobiles.”
Barmeir highlighted that the productivity gains from AI in testing are substantial.
“While traditional test automation might be limited to a single platform or language and the capacity of one person, AI-enhanced testing breaks these limitations,” she noted.
“Testers can now create and execute tests on any platform, web, mobile, desktop, using multiple languages, with the capacity of multiple testers. This not only amplifies testing capabilities but also introduces a new level of flexibility and efficiency,” the industry veteran wrote.
The AI advantage
Barmeir said that upskilling QA teams with AI offers the significant advantages of multilingual testing and 24/7 operation.
“In today’s global market, software products often need to cater to diverse users, necessitating testing in multiple languages,” she explained.
“AI facilitates this process without requiring testers to know each language, thereby expanding the reach and usability of software products.”
The ability to test around the clock ensures that testing does not become a bottleneck in fast-paced development, she elaborated.
Additionally, AI’s autonomous maintenance capabilities further reduce the time and effort needed for updating test cases, ensuring tests remain relevant and effective.
Challenges
While GenAI offers significant advantages, Barmeir was quick to point out it also presents operational and security challenges that banks and financial services firms must address.
Firstly, hallucinations: AI may generate inaccurate or fabricated outputs during testing, leading to incorrect results and potentially overlooking critical issues, she said.
Secondly, bias. “AI systems can inherit biases from their training data, impacting testing outcomes and resulting in unfair testing scenarios and overlooked edge cases.”
Then there are data privacy issues. “The risk of mishandling or leaking sensitive data used during testing raises significant privacy concerns,” Barmeir said.
Also, there is a lack of transparency. “The ‘black box’ nature of many AI systems makes it difficult to trace decision-making processes, hindering debugging and trust in the system,” she noted.
Moreover, there are security vulnerabilities, as GenAI systems are susceptible to adversarial attacks that could exploit system weaknesses, potentially compromising the entire testing process.
Finally, inconsistent outputs. “AI might produce erratic or irrelevant results, affecting test reliability and making it challenging to maintain consistent testing standards,” Barmeir remarked.
Risk mitigation
To harness the power of GenAI while mitigating these risks, Barmeir thinks financial services firms should implement a host of different strategies, primarily to limit any risks.
Firstly, Human-in-the-Loop (HITL) supervision is vital, she argued. “ “Incorporating human oversight ensures that AI-generated outputs are rigorously validated for accuracy and reliability,” Barmeir said.
“Human supervisors can review and approve AI-generated test cases, ensuring they meet necessary standards before implementation.”
Also, restricting AI autonomy should be preferred. “Limiting the AI’s creative freedom prevents the system from making unwarranted assumptions or actions,” she said.
“This means setting clear boundaries and guidelines ensures that the AI operates within acceptable parameters, maintaining a predictable and reliable testing process.”
In addition, requiring reasoning for actions is something firms need to keep into account.
“Enforcing a policy where AI must explain its decisions promotes transparency and builds trust in AI-generated results,” Barmeir stressed.
“By demanding reasoning for each action, developers can gain valuable insights into the AI’s thought process and make informed adjustments.”
“The ‘black box’ nature of many AI systems makes it difficult to trace decision-making processes.”
– Tal Barmeir
Data management is another priority, or it should be, she said. “Implementing robust data management policies safeguards sensitive information from being misused during AI training. Encryption, anonymization and access controls are critical measures in protecting data privacy.”
Finally, utilizing diverse training data helps financial firms, Barmeir argued. “Training AI on a wide-ranging dataset minimizes biases and enhances the AI system’s robustness.”
She added that diverse data exposure helps the AI generalise better and reduce the risk of biased outcomes.
“Regularly updating the training data to reflect current and comprehensive scenarios ensures that the AI remains effective and fair.”
In summary, Barmeir said that, as GenAI becomes increasingly integrated into software development life cycles, understanding its capabilities and limitations is paramount.
“By effectively managing these dynamics, development teams can leverage GenAI’s potential to enhance their testing practices while ensuring the integrity of their software products,” she noted.
“With careful consideration of the outlined challenges and mitigation strategies, organisations can harness the full power of GenAI to drive innovation in software testing and deliver high-quality software products,” Barmeir concluded.
UPCOMING EVENTS
QA FINANCIAL FORUM LONDON: RECAP
Last month, on September 11, QA Financial held the London conference of the QA Financial Forum, a global series of conference and networking meetings for software risk managers.
The agenda was designed to meet the needs of software testers working for banks and other financial firms working in regulated, complex markets.
Please check our special post-conference flipbook by clicking here.
READ MORE
- Cognizant drags rival Infosys to court over trade secrets
- Testaify claims tool is ‘100x faster than seasoned QA architect’
- Fast-growing Newgen sets sights on banks in Middle East
- ABN Amro hires nCino and CBA for digital upgrade
- QAFF London: Lloyds’ Richard Bishop on the rise of ‘green software’
Become a QA Financial subscriber – for FREE
* Receive our weekly newsletter * Priority invitations to our Forum events
