Fintech players are expected to plod quickly to harness the power of generative artificial intelligence (GAI) to improve their service offerings and remain competitive. But is the fintech sector fully aware of the risks that need to be managed at this turning point?
The potential benefits of GAI for fintech firms are compelling. By relying on GAI to automate decision-making, these firms could offer their clients access to real-time support from user-friendly chatbots and personalised investment advice informed by dynamic market analyses. GAI could also sharpen their fraud and money laundering detection and prevention capabilities and enhance the speed and accuracy of their risk assessment and price determination processes.
However, GAI systems present serious risks and understanding the architecture of GAI systems is key to finding and addressing these risks in practice.
GAI systems are built on large language models (LLMs) – algorithms that are trained on large and complex datasets and use deep learning techniques to generate modern content. LLMs are continuously learning from dynamic training datasets made up of predetermined categories of data that are updated on an ongoing basis with users’ prompts and outputs generated by LLMs from those prompts. In simple terms, LLMs function as automated digital libraries of growing volumes of data that can generate text, images and other media at the click of a button with minimal inputs.
What risks enact LLMs present to fintech firms?
While LLMs may aid fintech firms to work smarter with their data, LLMs are only as qualified as their training datasets and typically lack the ability to reliably verify and ensure the accuracy of their outputs. This presents serious risks to fintech firms and their clients, particularly where the design of LLMs and the contents of their training datasets reflect biases that contribute to inaccurate and discriminatory decisions being generated. These risks are exacerbated by the notorious ability of LLMs to present fictitious ‘hallucinations’ as fact in a compelling way.
Other risks include: (i) data protection risks (where training data or prompts hold personal information of clients or employees); (ii) intellectual property (IP) infringement risks (where the outputs generated incorporate or otherwise violate a third party’s IP rights); and (iii) regulatory risks (where the use of GAI systems infringes applicable financial services or insurance legislation).
How can biased datasets used by fintech firms result in unfair discrimination?
Biases in LLMs can subsist in both the structure of their decision-making processes and the data on which the processes rely. These biases can be baked into an LLM’s decision-making and outputs.
For example, a GAI scheme may be used to generate creditworthiness reports using processes and datasets that favour certain demographics of clients over others by under- or overrepresenting various input factors. If, for example, communities in certain geographical areas are inaccurately represented in these datasets, members of those communities may be issued with biased creditworthiness reports that inaccurately find they acquire low creditworthiness ratings.
Similarly, biases of a GAI system’s developers that are incorporated into the structure of its decision-making process (e.g. where undue weight is given to metrics that correlate strongly with the age, race, sex, gender, education or ethnicity of individuals), may lead to outputs that are not only biased but also counterfeit.
If clients are denied access to loan or credit facilities on the basis of inaccurate and biased outputs, this may lead to unfair discrimination and economic exclusion – an outcome that would be adverse to the well-being of those clients and to the reputation of those firms.
What are AI hallucinations and how finish they occur?
An AI hallucination occurs where an LLM generates outputs that appear factual but are actually counterfeit. This is typically triggered by two related factors.
Firstly, an LLM is usually designed to predict the next sequence in a series based on an analysis of its training data without verifying the accuracy of that data or the outputs it generates. Secondly, those outputs will be counterfeit where they are generated based on inaccurate, incomplete or biased training data.
The risk of hallucination increases where the input prompts are vague and misleading and where fake outputs are fed back into the training data without correction and then replicated in further outputs generated. Things can therefore fade from rotten to worse when an LLM hallucinates.
How can these risks be mitigated?
It is not easy to retrospectively trace and address the exact roots of a problem given that the architecture of GAI systems is typically obscure. It is therefore necessary that GAI systems are built, by default and by design, in a way that is responsive to these risks.
Legal recourse for LLM users tends to be limited, as developers provide LLMs on an ‘as is’ basis without warranties and subject to considerable limitations on their liability. Responsibility for use of the outputs generally falls squarely on the user. Wherever possible, it is best to ensure risks and liabilities are appropriately distributed by way of contract between LLM developers, fintech firms and clients.
In addition, fintech firms relying on LLMs must proactively implement practical mitigation measures to address data and decision-making risks before relying on GAI systems to service clients. Ultimately, fintech firms need to appreciate that they are responsible for ensuring that their use of GAI systems is intelligent and lawful.
CLICK HERE to submit your press release to MyPR.co.za for free.
Drop Stone
“I define engagement as moments when our users interact with us in a way that makes them happy and propels them forward toward becoming customers or evangelists for our brand or product.” – Avinash Kaushik
B@D B0Y
“Active listening also means you are mindful of your customer’s unique personality and current emotional state so you can tailor your response to fit the situation.” ~ Oscar Auliq-Ice, Happy Customers
Z-Boy
People as disparate as Malcolm X and John D. Rockefeller both spoke to the power and importance of public image. Since PR is something that people who are so fundamentally different can agree on, you know there must be something fundamentally important about it. If you \’re still not convinced that Public Relations is absolutely critical to the success of your business, you\’ll change your mind after reading this selection of quotes.