Leader's Opinion: LLMs Ride the Overconfidence Wave with Mukundan Rengaswamy

In the world of machine learning, developers often grapple with the enigmatic quirks of Language Model Models (LLMs). Jonathan Whitaker and Jeremy Howard from fast.ai embarked on an intriguing experiment, unearthing a subtle yet pervasive issue with these models: overconfidence, a distinct phenomenon from the notorious LLM hallucination.

Mukundan Rengaswamy, Head of Data Engineering, Innovation & Architecture of Webster Bank weighed in on the matter, stating, “LLMs (Large Learning Models) and Gen AI have been in the news ever since ChatGpt was introduced to the public. Generative AI is powered by very large machine learning models that are pre-trained on vast amounts of data. A lot of research is being done on these models to better understand the behavior and refine them for broader usage.” 

Overconfidence, they found, occurs when a model tenaciously clings to information from its training data, even when it's blatantly incorrect for a given question. The culprits behind this phenomenon? The familiar duo of overfitting and underfitting. Additionally he said that, “The “overconfidence” highlighted in the paper could be due to overfitting of models. Ideally, one would want to select a model at the sweet spot between underfitting and overfitting. This is the goal, but is very difficult to do in practice. There are several techniques that could be used to mitigate the challenges that may also arise with such fast-learning models.”

Overfitting is when a model becomes too intricate, mirroring its training data too closely, while underfitting arises when the model lacks enough data to make accurate predictions. Striking the balance between these extremes is the elusive bias-variance tradeoff.