No matter how big the buzz gets around generative AI, the industry is still in the very early stages of adoption in the software quality/software testing industry.
There has been a lot of noise created around the promise of AI in testing, and QA professionals should expect that noise to grow only louder and louder.
However, when you ask most software testers or QA managers about the real impact of AI today, many are saying they haven’t seen a significant impact yet – at least not in the form of dollars and cents.
This is according to Prashant Mohan, senior director of product management at SmartBear, leading the company’s strategy and product direction for its testing portfolio.
“Amidst all the buzz and excitement that many feel about the potential for generative AI in our industry, big uncertainties and anxieties continue to loom,” Mohan said.
“It seems like only yesterday that we finally put the debate to rest around, ‘No, test automation will not replace every manual tester’s job,’ and now we’re already mired in debating whether AI is going to take all of our jobs, including those held by automation engineers who previously felt irreplaceable,” he said.
To Mohan, it’s clear that testers’ reception of AI-driven changes in test automation tooling and processes will continue to reflect both scepticism and promise for the foreseeable future.
Scepticism and concern
There’s valid scepticism of AI stemming from two notable concerns, argues Mohan.
First, testers have been promised “revolutionary” AI solutions that will solve all their problems, and autonomously complete all of their testing with only the push of a button.
“Sounds familiar? Many vendors have made the same claims about automated testing for more than a decade,” said the engineer with an MBA who joined SmartBear over eight years ago.
“Yet, many of these promises haven’t quite delivered, leaving practitioners understandably doubtful,” he continued.
“AI will never be a silver bullet solution for all testing challenges, and every testing and QA team knows this.”
– Prashant Mohan
Often, the reality of AI in testing has not matched the hype, making it challenging to see its practical value in everyday testing scenarios.
“AI will never be a silver bullet solution for all testing challenges, and every testing and QA team knows this,” Mohan stated.
Second, it is not hard to understand why many testers today are apprehensive about AI and the potential threat it poses to their job.
He pointed out that manual testers have been down this road before, but now automation engineers are also seeing demos, reading research, and hearing the rumours of AI completing tasks at unprecedented speeds that could render ‘technical expertise’ redundant.
“It’s unsettling for anyone to feel like their relevance is in jeopardy,” Mohan stressed. “And, while the reality is that generative AI is going to disrupt many industries, there’s another reality that bears stating and repeating until it’s accepted by all as just as true: AI should be seen as a tool that significantly enhances the tester’s role rather than replaces it.”
Zooming in on generative AI, not unlike test automation in many regards, is a powerful tool to be leveraged by testers.
It has the ability to significantly accelerate the completion of tasks that provide no extra value or ROI when performed by a human, so that humans have more time, or, in some cases, any time at all) to do the tasks that are best suited for humans and are performed worse by machines.
“Lastly, the most innovative solutions to complex problems in testing will always intersect at the intelligent use of modern technologies and the indispensable human touch,” Mohan said.
“As both these notions swirl, let’s talk about where exactly AI is having a positive impact and delivering immediate value for testers right now.”
Test automation
The increasing demands placed on development and testing teams to deliver high quality software at unprecedented speeds continue to fuel a drive toward identifying areas for workflow improvement.
“Test automation plays a pivotal role in this pursuit,” Mohan noted.
“Testing broadly is highly pattern-based at scale. Once you identify a set of regression test cases, you perform the same actions over and over,” he added.
“Any pattern-based work where you are executing repetitive actions consistently and can be performed by a robot is a natural fit for AI.”
Mohan explained: “Imagine a test case written in natural language within Jira, where many testers spend their time. With a simple click, AI can fix grammar, think, Grammarly for testing, make the test case legible, and break it into manageable steps, enabling new testers who have never seen your application or test case before to execute it.”
After review and any needed adjustments, the test can be automated with another click, removing the need for scripting or recording.
“You’ve eliminated the task of creating an automation test! Even better, all this works seamlessly within a tester’s existing workflow, and there is no need to learn new processes or tools,” Mohan stressed.
“This realisation was my own personal “a-ha” moment of seeing AI’s potential in testing, and exploring where it can further assist, not replace, the talented, valued, users I speak with all the time and strive to support.”
Another notable advancement for AI in testing is in reducing test flakiness, effectively addressing the instability of automated tests when there’s changes in the application under test or environment. “AI is helping ensure that tests remain intact and effective in these scenarios,” he said.
Through “self-healing” tests, there are teams today maintaining large test sets effectively, even as software systems undergo continuous, iterative development and updates.
Additionally, AI-powered visual testing allows testers to quickly compare images and identify discrepancies following code changes.
“With AI, testers can automate this manual, time-consuming task, and eliminate issues that are easy to go unnoticed when only a human eye is hunting for them,” Mohan said.
While testers are beginning to utilise these AI-powered capabilities, no company can claim to have legitimately ‘perfected’ an AI solution, he added.
“Despite this, testers are experiencing incremental benefits, as AI accelerates the execution of tedious tasks, highlighting its increasing role in expediting workflows and driving efficiency.”
While ‘larger’ problems across the software development lifecycle might not be magically solved through AI, cutting down the time it takes to complete repetitive, often mundane tasks is certainly a noteworthy step forward in the right direction, Mohan shared.
Years ahead
While nobody, outside of the companies building them, knows ‘what is next?’ when it comes to how AI tools will evolve from a capability standpoint, Mohan anticipates.
“There is an emerging sentiment in the testing industry that is gaining steam. Testers and other QA professionals who see this uncertainty as a great time to evolve alongside AI will be rewarded for doing so,” he said.
Testers can grow skill sets in areas immune to AI disruption such as understanding user behaviour within their domain and maintaining a focus on customers’ problems and needs.
Also, Mohan singled out embracing exploratory testing and critical thinking, developing strengths in adjacent areas like UI design, fostering seamless collaboration with development and product teams, as well as initiating conversations with engineering, product teams, customer support, and other business leaders.
“AI’s inability to fully comprehend user needs and foster cross-functional alignment aren’t knocks against it,” he said.
“They’re simply areas that AI, or any other automation engine, weren’t designed to do. And again, just like we know with test automation, understanding which tasks aren’t suited for automation is just as important as knowing which tasks are.”
Balancing the nuanced interplay between AI-driven efficiency gains and the human touch necessary for comprehensive understanding and collaboration will likely always be an important challenge worth solving, but it’s also a tremendous opportunity for those who view it as such, Mohan concluded.
NEXT MONTH IN SINGAPORE
REGISTRATION IS NOW OPEN FOR THE QA FINANCIAL FORUM SINGAPORE 2024
Test automation, data and software risk management in the era of AI
The QA Financial Forum launches in Singapore on November 6th, 2024, at the Tanglin club.
An invited audience of DevOps, testing and quality engineering leaders from financial firms will hear presentations from expert speakers.
Delegate places are free for employees of banks, insurance companies, capital market firms and trading venues.
QA FINANCIAL FORUM LONDON: RECAP
Last month, on September 11, QA Financial held the London conference of the QA Financial Forum, a global series of conference and networking meetings for software risk managers.
The agenda was designed to meet the needs of software testers working for banks and other financial firms working in regulated, complex markets.
Please check our special post-conference flipbook by clicking here.
READ MORE
- Cognizant drags rival Infosys to court over trade secrets
- Testaify claims tool is ‘100x faster than seasoned QA architect’
- Fast-growing Newgen sets sights on banks in Middle East
- ABN Amro hires nCino and CBA for digital upgrade
- QAFF London: Lloyds’ Richard Bishop on the rise of ‘green software’
Become a QA Financial subscriber – for FREE
News and interviews * Receive our weekly newsletter * Get priority invitations to our Forum events
