As global markets trade around the clock, the demands on testing and quality assurance have never been greater. Few organisations feel this pressure as acutely as Nasdaq, which underpins trading and post-trade systems worldwide.
Following his appearance at the QA Financial & E-Commerce Forum London on September 18th, Sudeepta Guchhait, Senior Director of Product Framework and Quality Engineering at Nasdaq, spoke with QA Financial about evolving strategies for reliability, the launch of Nasdaq’s Mimic testing platform, and what AI really means for capital markets.
Operating in a 24/7 environment is about more than technical uptime, Guchhait emphasised: “Well, it is not thinking about only for 24 by 7 production. It’s rather at NASDAQ, I would say as an organization is trying to become a better customer experience mindset or bring in more customer experience mindset.”
That shift has required investment in new frameworks and approaches across the business. “So what we have done is to transform and enhance the way we sell our product, we deliver our product and also to support our customer during the production upgrades and during the maintenance time period,” he explained.
The goal, Gucchhait said, is “developing a high standard quality assurance frameworks and expertise to provide a consistent and transparent standards becoming a QA investment program for our core products.”
Zero critical incidents is the standard Nasdaq aims for, though it is rarely attainable in absolute terms. “However, it is very difficult to achieve zero critical incidents in production. Of course, there will be some glitch. It could be either at the customer end or at our product end or even at our infrastructure.”
To mitigate this, Nasdaq has reorganised around what Guchhait calls “teams of teams” monitoring quality, integrated CI/CD pipelines across business lines, and leaned into AI.
“And now we are fostering towards bringing in more artificial intelligence, making sure how do we read our logs, how do we expedite our feedback loop, et cetera.”
Inside Mimic
At the London Forum, Guchhait unveiled Mimic, Nasdaq’s in-house AI-driven performance testing tool. The platform is designed to cope with the unique requirements of Nasdaq’s proprietary systems.
“Since we have our constraints on our some of the internal patented protocols like itch, ouch, and drop, which is not available in the market. It’s only within the NASDAQ system. It becomes more difficult for us to use any kind of market standard performance testing tool. Instead, what we have done is we have built our in-house performance testing tool,” he explained.
The results are powerful: “…which actually helps us to load, let’s say, for example, 5,000 order book changes in a minute. Or you can run some of the IPOs in fractions of seconds.”
The innovation comes from AI’s ability to learn from test logs and propose new scenarios. “So MIML provides us an opportunity to define corner cases, test cases, or you can say corner case use cases, which can be helpful for our performance test engineer to generate the test cases.”
“It becomes more difficult for us to use any kind of market standard performance testing tool.”
– Sudeepta Guchhait
Traditionally, test cases were based on customer requests, say, 10,000 order book changes in two minutes. “But now with MIMR it is trying to read the logs of our performance test results and trying to provide us different situations… So memory provides us one step ahead to think about different combinations of our test parameters based on reading the performance test logs, whatever we have actually executed.”
The scope of client needs makes testing even more demanding. “If we consider some of our clients, we have a few of our clients wherein we sell just one product. For example, let’s say either a trading system or a post-trade system. But there are some customers wherein we sell a bunch of products. Another example, trading plus post-trade plus depository. So what happens is the integration of all the three platforms becomes challenging for us.”
Integration challenges are compounded by test data, infrastructure, and business-critical scenarios. “Everybody knows that each of the product… works very well in silo. Of course, there won’t be any kind of zero critical defects. But at the end of day, when we try to integrate these products, it becomes challenging… So that’s where we are building our framework to first develop flaky tests so that we can try to break the system. And then also we try to build a robust regression, which is basically end to end.”
Lessons for financial firms
For smaller QA teams in banking and financial services, there are still lessons to be drawn. “One of the, I would say the most practical thing which we have done recently is to kind of reproduce or replicate or replay our customer production scenarios,” said Guchhait.
The approach involves clients providing anonymised logs of daily trading activity, which Nasdaq can replay in regression packs. “That gives us a lot of information about how customers running their business in a daily basis… So we get to see all those logs and based on that log, we try to build our replay scenarios… and kind of provide a confidence certificate to our customer saying that, yes, we have tested this and when you run this business scenarios in your test infrastructure, it won’t fail or it will obviously give you the correct results.”
Looking ahead, Guchhait sees AI as the defining issue. “I would say we are in capital market, we are definitely evolving in AI. There are a lot of challenges right now, setting up the governance mechanism within the AI. It’s very easy. I mean, AI is a, I would say it’s kind of success story for some of the organization, but at the same time, it is kind of just a buzzword for most of the organization as well.”
The key, he argued, is targeted adoption. “Will it help in our monitoring system? Will it help to generate more corner cases to make sure that our product doesn’t break in production? And also to understand how can we customise our performance test scenarios based on the results we are getting.”
As Nasdaq pushes forward with Mimic and beyond, the answers to those questions will shape how capital markets worldwide test, assure, and deliver the systems that keep global trading flowing.
QA FINANCIAL PODCASTS
Listen to Sudeepta Guchhait on Nasdaq’s new Mimic AI testing platform
QA Financial sits down with Sudeepta Guchhait, Senior Director of Product Framework & Quality Engineering at Nasdaq
——–
Listen to Wesley Scheffel and Robin Rain on Schroders’ DevOps strategy
We catch up with Wesley Scheffel, Head of Cloud Platform and Product Engineering at Schroders, and Robin Rain, Head of Cloud Platform Architecture
——–
Listen to Citi’s Jason Morris on Lightspeed and the future of continuous delivery
Jason Morris, Head of Developer Pipelines for Securities Markets and Banking at Citi, talks about Lightspeed
THIS NOVEMBER
Why not become a QA Financial subscriber?
It’s entirely FREE
* Receive our weekly newsletter every Wednesday * Get priority invitations to our Forum events *
REGULATION & COMPLIANCE
Looking for more news on regulations and compliance requirements driving developments in software quality engineering at financial firms? Visit our dedicated Regulation & Compliance page here.
READ MORE
- Why real-time monitoring and scenario testing are becoming core QA disciplines
- BankDhofar takes an automated approach to strengthen QA
- Banks warned AI still fails on real-world software testing tasks
- SEC’s AI emphasis drives new QA and testing imperatives for US banks
- Inside the chaos: The new reliability discipline reshaping banking QA
WATCH NOW
