Nasdaq AI Team Supports the Index Business
Hi, I’m Doug Hamilton – VP and Head of AI Engineering & Research at Nasdaq. Nasdaq was an early adopter of AI within financial services, establishing the team in 2016. Since then, we have launched numerous products and services aligned with Nasdaq’s core goal of promoting transparency, liquidity, insight and fairness into global capital markets.
Our AI team is structured to deliver high quality research into production, rapidly respond to client needs, and provide timely support for those production systems. Our research team, comprised of data scientists, focuses on identifying high value data and developing intelligent algorithms and models for deployment. Our AI Engineering team, composed of data and software engineers, focuses on building the platforms and pipelines needed to support research and maintain our products and services. As part of Nasdaq, we partner closely with product teams to ensure our clients’ needs are met.
The issue that caused my team to step in and support the Nasdaq Index business:
On September 30, 2022, Nasdaq was informed by the index contributor on a number of Nasdaq Disruptive Technology Indexes called Yewno that it was no longer going to be able to meet its obligations. In an effort to ensure a smooth transition away from Yewno, the Nasdaq AI team looked to step in and replace them as a new index contributor for the impacted Nasdaq Indexes. For the next six-plus months, the Nasdaq AI team worked tirelessly along with the Nasdaq index team and the impacted clients.
Here is what the Nasdaq AI team did to create its own patent data set that will ultimately go into effect on July 24, 2023, across several Nasdaq Disruptive Technology Indexes:
We investigated multiple sources of USPTO (US Patent and Trademark Office) and WIPO (World Intellectual Property Organization) patent data, ultimately settling on a long-standing consolidator in the space. We built robust data pipelines that access the data and associated parent and subsidiary entities. We leveraged and tuned a modern transformer-based Language model to understand the patent data so that it could effectively tag them with the appropriate technology labels. We validated our results with Nasdaq’s Index team and our clients. Finally, we deployed the model and pipeline through our tried and tested model management platform so the data can be calculated and consumed.
While we are very excited to for this data to be the new input into the suite of Nasdaq Indexes, this sort of data is very familiar to us. Since 2018 the group has published over 12 patents in the field of Artificial Intelligence. In 2019, we created an optimizer that has been used by three Nasdaq indexes that leverage proprietary AI to minimize their volatility exposure and have done so without issue. We have also delivered a slew of Natural Language Processing solutions related to analyzing transcripts, assessing corporate governance, extracting ESG information, etc. supporting other business units across Nasdaq. While not patents per se, all these projects did involve computers reading text and pulling out key themes and information just like we do here.
Nasdaq’s Approach
When looking to replace Yewno (the previous organization that supplied patent data for input into the Nasdaq Disruptive Technology Indexes) with an in-house solution we had two primary goals. First, develop a suitable replacement to scoring any company’s importance to an area of technology and the technologies of importance to any company. When it comes to the top technologies important to companies, Yewno and Nasdaq agreed more than 95% of the time. Second, we sought to improve on the original results. We did this in a few ways:
1) By using a more modern transformer-based architecture unavailable to the vendor when the calculations began. These are the same architectures underpinning systems like ChatGPT.
2) By tirelessly validating our results and applying mixed method fixes.
3) By using more encompassing data (more on this later).
Our more modern language model and other mixed-method fixes (e.g., rules, keywords, etc.) help resolve a number of historic challenges in allocating the appropriate technology to the appropriate patent. For example, confusing semiconductors for batteries, autonomous driving for electric cars, or general ambiguity associated with cybersecurity patents.
The Data
To produce the new scores, Nasdaq has partnered with IFI Claims, a subsidiary of ScienceDirect, to source patent data. IFI Claims has been doing the sturdy work of collecting patents, associating subsidiaries to parent firms, and assessing the ‘importance’ of patents through citations for over 50 years from the USPTO and WIPO. When we began working with the source data, we noticed immediately that it provides superior global coverage to the previous provider. Case in point: IFI Claims shows firms like Baidu, Tencent and Alibaba as key innovators while they are almost invisible in the historical Yewno data.
Final thought
With more inclusive data, modern Natural Language Processing Systems and the transparency, expertise and responsiveness of Nasdaq’s AI staff, we are excited to provide this data as the key selection input in the Nasdaq Disruptive Technology Indexes
Disclaimer:
Nasdaq® is a registered trademark of Nasdaq, Inc. The information contained above is provided for informational and educational purposes only, and nothing contained herein should be construed as investment advice, either on behalf of a particular security or an overall investment strategy. Neither Nasdaq, Inc. nor any of its affiliates makes any recommendation to buy or sell any security or any representation about the financial condition of any company. Statements regarding Nasdaq-listed companies or Nasdaq proprietary indexes are not guarantees of future performance. Actual results may differ materially from those expressed or implied. Past performance is not indicative of future results. Investors should undertake their own due diligence and carefully evaluate companies before investing. ADVICE FROM A SECURITIES PROFESSIONAL IS STRONGLY ADVISED.
© 2023. Nasdaq, Inc. All Rights Reserved.
The views and opinions expressed herein are the views and opinions of the author and do not necessarily reflect those of Nasdaq, Inc.
