The exterior of the U.S. Department of Treasury in Washington, DC.
New York CNN —
Top federal regulators are warning for the first time that the use of artificial intelligence poses a risk to the financial system.
The Financial Stability Oversight Council, a team of leading regulators across the US government, formally classified AI on Thursday as an “emerging vulnerability.”
Sophisticated AI models have exploded in popularity in recent years, even as many prominent names within the field say the nascent technology could have disastrous effects if it grows out of control.
“AI has the potential to spur innovation and drive efficiency, but its use in financial services requires thoughtful implementation and supervision to manage potential risks,” FSOC warned in its annual report Thursday.
FSOC, created in the aftermath of the 2008 financial crisis and led by US Treasury Secretary Janet Yellen, cautioned that AI can “introduce certain risks,” including cybersecurity concerns, compliance risks and privacy issues.
Regulators also expressed concern about “complicating factors” related to generative AI models like ChatGPT.
For example, the council flagged data security, consumer protection and privacy risks posed by financial firms using generative AI. And they flagged that generative AI models can produce flawed results known as “hallucinations.”
An impenetrable black box
Another worry for regulators is that some AI models operate as “black boxes” – meaning their inner workings are impenetrable to outsiders.
FSOC said this “lack of ‘explainability’ can make it difficult to assess the system’s conceptual soundness, increasing uncertainty about their suitability and reliability.”
In other words, if banks are relying on mysterious AI models, that makes it hard to understand how sturdy their underlying systems truly are.
Relatedly, regulators expressed concern about how these AI systems could “produce and possibly mask biased or inaccurate results.” That in turn would raise worries about fair lending and other consumer protection issues, FSOC said.
The move comes just two years after regulators first classified climate change as an “emerging threat to US financial stability.”
AI investment and adoption has boomed, even as some experts warn of serious dangers with the technology. President Joe Biden recently signed an executive order instructing federal agencies to take steps to safeguard the development of AI, which is a booming area of adoption and investment.
“Errors and biases can become even more difficult to identify and correct as AI approaches increase in complexity, underscoring the need for vigilance by developers of the technology, the financial sector firms using it, and the regulators overseeing such firms,” FSOC said.
The popularity of ChatGPT and other generative AI tools, which use large language models to identify patterns in data and create text and images, has only accelerated the interest in AI.
