Rates of automation for software quality assurance and testing are increasing rapidly, but the outcome in terms of quality remains uncertain. While 68% of organisations responding to the annual survey behind the 2020 World Quality Report, produced by the Capgemini consultancy, say they have the required automation tools and 63% say they have enough time to automate, only 37% of firms believe they get a return from that same investment in automation.
The 2020 survey is based on responses from 1750 firms, 19% of which – the largest single group, are financial services firms. Spending on software quality assurance and testing represents 22% of firms’ IT budgets, and 31% of QA and testing spend is on software tools. Test automation has shown promising progress this year; 68% of organisations say they have the required automation tools while 63% have enough time to automate
“There is a level of disappointment in the results of automation,” said Mark Buenen, Global Leader, Digital Assurance and Testing for the Capgemini Group. “And what is really behind that is the lack of strategy in automation. As firms move increasingly to Agile and DevOps they are achieving more velocity [in app delivery] which good, but there are few guidelines for achieving better quality.” While there are other measures of improved efficiency as well as velocity, such as code coverage and defects caught before and after production, the challenge lies deeper: 42% of respondents reported a lack of professional test expertise in their agile teams, for example.
And the report uncovered some surprising geographic differences. While 47% of US-based respondents use agile methods for 30% or more of their test efforts, the proportion is only 11% for Italian firms and 4% for UK firms. And one in five respondents across the survey say they are still using Waterfall methods for most of their testing efforts, highlighting the skills shortages in agile. But there is little doubt that software quality assurance is steadily becoming more important.
The objective of ‘contributing to business growth and business outcomes’ was the highest rated target for testing and QA in this year’s survey, at 74% – up 6% from 2018. “This shows progress from two years prior, at which time detecting software defects and ensuring end-user satisfaction were rated higher,” according to the report. The report also found that COVID-19 has also impacted spending on software quality assurance. Those surveyed expected a 13% average drop in post-pandemic testing and QA budgets.
The pandemic has also highlighted enterprise areas for improvement, with 52% of respondents stating their QA goals had changed a great deal or that they needed a major overhaul. The pandemic is likely to accelerate the trend to increased automation of QA, the report concluded.
