How quickly can a bank test its app on hundreds of combinations of devices and operating systems? Perfecto, a provider of cloud-based mobile testing environments, offers a solution for a highly fragmented market. Bank customers are unforgiving when it comes to mobile apps. Bugs mean lost market share. But every new phone release — indeed every new software update — creates a new challenge for testers. Will every bank send someone to wait in line for an iPhone 8, for example? Perfecto’s answer is a test lab of more than a thousand real mobile devices hosted in the cloud, numbering Lloyds Banking Group among its clients. Anish Patel, head of mobile delivery for the CIO at Lloyds, said that the implementation came in two phases. “We started using Perfecto for production testing, which is testing the finished app right before it is officially released. Perfecto gave us access to a wider range of devices then we normally have,” said Patel. The successfil use Perfecto for production testing prompted Lloyds to use the product in the development phase. The idea was to shift testing left in order to find faults sooner, moving testing to a nightly cadence for release. “We run automated tests every night, so that in the morning our engineers can work on any bugs that they find,” said Patel. “Discovering defects early on lowered development costs. Automation also has allowed us to minimise that unpredictable human factor, particularly important in a world where physical devices keep changing.” According to Carlo Cadet, head of product marketing at Perfecto, the test lab helps to accelerate feedback from customers into the app development process. “As soon as there’s a problem, you can get to work before customers are tweeting about it,” he said. “The goal is continuous testing throughout the development lifecycle.”
The core testing methodology revolves around the personas created to replicate those users. To adjust a test, a line of code is added according to the chosen persona. The beauty of the methodology is that Perfecto has found that a limited number of personas can cover the vast majority of users, and that a typical app has a target audience that can be represented by two or maybe three personas.
Devices on the Perfecto cloud can be tested using automation tools like Appium and Selenium, and they integrate with common open-source continuous delivery tools such as Jenkins, Travis, Circle CI, visual studio, as well as HP’s Quality Center. Perfecto also extends those frameworks and makes them enterprise grade. It allows testers to recreate real-world conditions during testing, such as wireless signal increasing and decreasing, memory constraints, different apps on the device, or geographical location. And, yes, a key benefit for quality assurance teams is not having to queue overnight for the latest iPhone. “We will have the iPhone 8 within an hour of it hitting the market,” said Cadet. “We think it’s critical that brands have access to operating systems and devices on the same day that they’re released.”
