San Francisco-based Kolena has launched a new web application that is designed to enable “rapid and accurate testing and validation of AI systems”, QA Financial has been told.
The startup, founded only three years ago, said its new AI Quality Platform is able to significantly enhance the testing of various artificial intelligence-powered applications and platforms.
According to Mohamed Elgendy, co-founder and CEO of Kolena, rigorous and frequent testing becomes more and more important as the number of AI applications within the financial services space and elsewhere is experiencing a record increase.
Banks, insurance firms and other financial services firms are indeed rolling out AI tools and platforms at an unprecedented speed, as industry insiders discussed during a QA Financial webinar last week.
The new tool focuses largely on scrutinising data quality, model testing and A/B testing, as well as monitoring for data drift and model degradation, as well as debugging.
The solution enables close scrutiny of AI and ML models in a range of different scenarios that are based on real world-models, thereby “moving beyond aggregate statistical metrics that can obscure a model’s performance on critical tasks,” Elgendy said.
Kolena’s clients then link the relevant models to their API, whereby the company provides proprietary dataset for their AI integrations.
“We move beyond aggregate statistical metrics that can obscure a model’s performance on critical tasks.”
– Mohamed Elgendy
Elgendy said that while he worked at Japanese e-com giant Rakuten he experienced first-hand the challenges and difficulties that come with deploying AI solutions, as well as during his role as a senior engineering manager at Amazon later on.
He stressed it is important to note that each customer can opt to measure for factors such as bias and diversity of age, race, ethnicity, and lists of dozens of metrics.
By conducting tests on specific models, namely by running hundreds of thousands of engagements, it can pick up unwanted and illogical outcomes and, if needed, alter the conditions and thereby the different outcomes.
“It will run tests and tell you exactly where your model has degraded,” Elgendy explained. “We kind of take the guessing part out of the equation, and turns it into a true engineering discipline like software.”
Perhaps unsurprisingly, Kolena ran extensive tests on its own platform ahead of lunch. The platform has been offered in a closed beta to a limited group over the last two years.
Based on that feedback, the system was altered and improved and the company now felt confident enough to open up the platform and go-to-market.
“We intentionally worked with a select set of customers that helped us define the list of unknowns, and unknown-unknowns,” Elgendy concluded.
Want to stay up to date and receive our news, features and interviews for free?
Our e-newsletter lands in your inbox every Friday. Sign up HERE in one simple step.
ALSO READ
