Machine learning and QA: using data to super-charge testing

Artificial intelligence could be a game-changer for quality assurance. Experts like Accenture’s Shalini Chaudhari (pictured) say it will first be used for requirement capture, test strategy and test design. The downside is that machine learning takes time.

Artificial intelligence – the term that loosely applies to a range of technologies that mirror human cognitive functions – could be a valuable tool for quality assurance. But if AI is going to lead to a paradigm shift in the way software is tested, that shift is still some time away, according to Andreas Golze, vice president for quality assurance and engineering for Europe at Cognizant, the IT service provider.

The problem is, according to Golze, that the very nature of intelligent systems means that there is an element of learning – the system will make mistakes that need to be corrected so that it will not repeat them. It is hard to say how long it will take before they are really effective.

“If we imagine the progress of AI technology as a curve, we do not know where we are. We might be at the period where the technology rapidly accelerates in sophistication and usefulness, or we might be a few years or more from that. Time will tell,” said Golze.

Machine-learning, the most common technique that powers AI, is nothing new – it has been around since the earliest days of AI research in the 1960s. It is a statistical method that uses historical data to predict the future and optimise processes.

The technology has received a surge of interest recently as big data and the exponential growth of processing power have unlocked the number-crunching capabilities that make it a reality.

Still, it is still early days for practical applications of the technique and experts are divided on how mature the technology is. Golze believes that there is a lot of hype currently associated with the term AI:  “It is not a revolution, it is a new name for things we have used before.”

Shalini Chaudhari, managing director and testing services lead for Europe, Africa and Latin America at global consultancy Accenture, is more optimistic about the technology: “In the past two to four years AI has really taken off. Two key factors have contributed to this: an explosion in the availability of data through the internet of things, and the sheer computing power to process that data in a way that is not just confined to a few supercomputers in high-tech research facilities.”

Chaudhari said that AI is already making an impact, for example helping doctors diagnose diseases. And Accenture is a pioneer in using AI in software testing.

Accenture has been running a number of pilot projects over the past two years, said Chaudhari, and they have already attracted the interest of a number of financial firms: “In terms of quality assurance and testing, banks have been ahead of the curve. The same is true of AI. They are in talks with us about using the technology and they are pursuing their own projects in-house as well.”

Chaudhari said her firm has created an intelligent automation platform called myWizard. The platform understands natural language, responds to commands, and can give results when addressed. In practice it functions as a virtual coworker that is able to learn.

myWizard can help in quality assurance by functioning as a virtual tester –  starting at the business requirement stage of software development, said Chaudhari. This virtual tester can sit in conversation between a business user and a developer, making sure that any business requirements are correctly captured. If any ambiguity is introduced, the virtual tester will pick up on it and press the business user for clarification.

Detecting these potential errors earlier in the lifecycle saves effort and money. “In an AI world, we can leverage the years and years of data we have to prevent errors at the source,” said Chaudhari. “Contrast the limited memory, and availability, of a person, with the resources available to an AI advisor that can work tirelessly and can be present for whoever needs it.”

Sarat Addanki, head of testing and business assurance practice at Prolifics, the US IT solution provider, is using machine learning to automate test design: “The basic principle of test design is identifying a variety of business processes and then creating a test that executes them using certain inputs.”

The first step to using machine-learning to automate test design involves is having a human capture an application’s business processes. A programme then combines those business processes with test data to create a test case.

Addanki sees fruitful applications of the technology in areas of the financial sector like the mortgage industry. “The complex requirements that govern, for example, how to determine an interest rate lead to applications with equally complex business rules,” said Addanki. “The more of these business rules, the more chance a mistake can happen. AI can help you keep on top of that.”

And the advantages are numerous: “A machine-learning algorithm that generates test design can be much more thorough than any human tester. If a functionality has 100 different conditions it is impossible to prove that a manual tester is hitting every single one. The system can also verify that you are not repeating tests that look different, but are actually testing the same conditions. This maximises test coverage and minimises redundancies.”

Ideally, the system will be constantly learning new business processes as functionalities are added – running regression tests to make sure that no application-critical defects are introduced.

Cognizant’s Andreas Golze sees the most promising applications of AI in test strategy. He compares finding defects to fishing in the sea: “You never have the time to scan whole ocean, but defects come like fish, in groups. There are some areas where there are many, and some where there are none.”

The idea is to use past test data and feed it into a machine-learning system, which will them predict what the most at-risk components of the application are. Testers can then target their resources to the places where they will see the most return. “You never have the time to test everything, and AI can help you reduce risk while keeping that edge in time-to-market,” said Golze.

All of this requires comprehensive documentation of interdependencies between software components because, Golze said, if it is missing then the machine-learning programme cannot accurately predict what areas of an application will be impacted by any change. “And for many older applications the documentation is missing, or not up to scratch,” he added. It might be the case then that AI is useful for test-strategy but only for newer applications that are built with these requirements in mind.

Shalini Chaudhari will be speaking at the QA Financial Forum London on April 5th organised by See the agenda here. If you are interested in attending please contact [email protected]. If you work for a financial firm, you may be eligible for one of our VIP guest passes.

Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>