Author: Adam Zewe | MIT News<\/p>\n
People use large language models for a huge array of tasks, from translating an article to identifying financial fraud. However, despite the incredible capabilities and versatility of these models, they sometimes generate\u00a0inaccurate responses.<\/p>\n
On top of that problem, the models can be overconfident about wrong answers or underconfident about correct ones, making it tough for a user to know when a model can be trusted.<\/p>\n
Researchers typically calibrate a machine-learning model to ensure its level of confidence lines up with its accuracy. A well-calibrated model should have less confidence about an incorrect prediction, and vice-versa. But because large language models (LLMs) can be applied to a seemingly endless collection of\u00a0diverse tasks, traditional calibration methods are ineffective.<\/p>\n
Now, researchers from MIT and the MIT-IBM Watson AI Lab have introduced a calibration method tailored to large language models. Their method, called Thermometer<\/a>, involves building a smaller, auxiliary model that runs on top of a large language model to calibrate it.<\/p>\n Thermometer is more efficient than other approaches \u2014 requiring less power-hungry computation \u2014 while preserving the accuracy of the model and enabling it to produce better-calibrated responses on tasks it has not seen before.<\/p>\n By enabling efficient calibration of an LLM for a variety of tasks, Thermometer could help users pinpoint situations where a model is overconfident about false predictions, ultimately preventing them from deploying that model in a situation where it may fail.<\/p>\n \u201cWith Thermometer, we want to provide the user with a clear signal to tell them whether a model\u2019s response is accurate or inaccurate, in a way that reflects the model\u2019s uncertainty, so they know if that model is reliable,\u201d says Maohao Shen, an electrical engineering and computer science (EECS) graduate student and lead author of a paper on Thermometer<\/a>.<\/p>\n Shen is joined on the paper by Gregory Wornell, the Sumitomo Professor of Engineering who leads the Signals, Information, and Algorithms Laboratory in the Research Laboratory for Electronics, and is a member of the MIT-IBM Watson AI Lab; senior author Soumya Ghosh, a research staff member in the MIT-IBM Watson AI Lab; as well as others at MIT and the MIT-IBM Watson AI Lab. The research was recently presented at the International Conference on Machine Learning.<\/p>\n Universal calibration<\/strong><\/p>\n Since traditional machine-learning models are typically designed to perform a\u00a0single task, calibrating them usually involves one task-specific method. On the other hand, since LLMs have the flexibility to perform many tasks, using a traditional method to calibrate that model for one task might hurt its performance on another task.<\/p>\n Calibrating an LLM often involves\u00a0sampling from the model multiple times\u00a0to obtain different predictions and then aggregating these predictions to obtain better-calibrated confidence. However, because these models have billions of parameters, the computational costs of\u00a0such\u00a0approaches rapidly add up.<\/p>\n \u201cIn a sense, large language models are universal because they can handle\u00a0various tasks. So, we need a universal calibration method that can also handle many different tasks,\u201d says Shen.<\/p>\n With Thermometer, the researchers developed a versatile technique that leverages a classical calibration method called temperature scaling to efficiently calibrate an LLM for a new task.<\/p>\n In this context, a \u201ctemperature\u201d is a\u00a0scaling parameter used to\u00a0adjust a\u00a0model\u2019s confidence\u00a0to be aligned with its prediction accuracy. Traditionally, one determines the right temperature using a labeled validation dataset of task-specific examples.<\/p>\n Since LLMs are often applied to new tasks, labeled datasets can be nearly impossible to\u00a0acquire. For instance, a user who wants to deploy an LLM to answer customer questions about a new product likely does not have a dataset containing such questions and answers.<\/p>\n Instead of using a labeled dataset, the researchers train an auxiliary model that runs on top of an LLM to automatically predict the temperature needed to calibrate it for this new task.<\/p>\n They use labeled datasets of a few representative tasks to train the Thermometer model, but then once it has been trained, it can generalize to new tasks in\u00a0a similar category without the need for\u00a0additional labeled data.<\/p>\n A Thermometer model trained on a\u00a0collection of multiple-choice question datasets, perhaps including one with algebra questions and one with\u00a0medical questions, could be used to calibrate an LLM that will answer questions about geometry or\u00a0biology, for instance.<\/p>\n \u201cThe aspirational goal is for it to work on any task, but we are not quite there yet,\u201d Ghosh says.\u00a0 \u00a0<\/p>\n The Thermometer model only needs to access a small part of the LLM\u2019s inner workings to predict the right temperature that will calibrate its prediction for data points of a specific task.\u00a0<\/p>\n An efficient approach<\/strong><\/p>\n Importantly, the technique does not require multiple training runs and only slightly slows the LLM. Plus, since temperature scaling does not alter a model\u2019s predictions, Thermometer preserves its accuracy.<\/p>\n When they compared Thermometer to several baselines on multiple tasks, it consistently produced better-calibrated uncertainty measures while requiring much less computation.<\/p>\n \u201cAs long as we train a Thermometer model on a sufficiently large number of tasks, it should be able to generalize well across any new task, just like a large language model, it is also a universal model,\u201d Shen adds.<\/p>\n The researchers also found that if they train a Thermometer model for a smaller LLM, it can be\u00a0directly\u00a0applied to\u00a0calibrate\u00a0a larger LLM within the same family.<\/p>\n In the future, they want to adapt Thermometer for more complex text-generation tasks and apply the technique to even larger LLMs. The researchers also hope to quantify the\u00a0diversity and\u00a0number of labeled datasets one would need to train a Thermometer model so it can generalize to a new task.<\/p>\n This research was funded, in part, by the MIT-IBM Watson AI Lab.<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"