Author: Alex Ouyang | Abdul Latif Jameel Clinic for Machine Learning in Health<\/p>\n
In today\u2019s hospitals and clinics, a dermatologist may use an artificial intelligence model for classifying skin lesions to assess if the lesion is at risk of developing into a cancer or if it is benign. But if the model is biased toward certain skin tones, it could fail to identify a high-risk patient.<\/p>\n
Perhaps one of the best known and most persistent challenges that AI research continues to reckon with is bias. Bias is often discussed in relation to training data, but model architecture can also contain and amplify bias, negatively influencing model performance in real-world settings. In high-stakes medical scenarios, the very real consequences of poor performance have made bias into a quintessential safety issue.<\/p>\n
A new paper<\/a> from researchers at MIT, Worcester Polytechnic Institute, and Google that was accepted to the 2026 International Conference for Learning Representations proposes a novel debiasing approach called \u201cWeighted Rotational DebiasING\u201d (i.e., WRING) that can be applied to vision language models (VLMs), like OpenAI\u2019s OpenCLIP.<\/p>\n VLMs are multi-modal models that can understand and interpret different data modalities like video, image, and text simultaneously. While debiasing approaches for VLMs do exist, the most commonly used approach is known as \u201cprojection debiasing,\u201d which leads to what has been termed the \u201cWhac-A-Mole dilemma\u201d<\/a>, an empirical observation that was formally introduced to AI research\u00a0in 2023.<\/p>\n Projection debiasing is a post-processing approach that removes the undesirable, biased information from model embeddings by \u201cprojecting\u201d the subspace out of a representation space of relationships, thereby cutting out the bias. But this approach has its drawbacks.<\/p>\n \u201cWhen you do that, you inadvertently squish everything around,\u201d says Walter Gerych, the paper\u2019s first author, who conducted this research last year as a postdoc at MIT. \u201cAll the other relationships that the model learns change when you do that.\u201d<\/p>\n Gerych, who is now an assistant professor of computer science at Worcester Polytechnic Institute, is joined on the paper by MIT graduate students Cassandra Parent and Quinn Perian; Google\u2019s Rafiya Javed; and MIT associate professors of electrical engineering Justin Solomon and Marzyeh Ghassemi<\/a>, who is an affiliate of the\u00a0Abdul Latif Jameel Clinic for Machine Learning and Health<\/a> and the Laboratory for Information and Decision Systems.\u00a0<\/p>\n While projection debiasing stops the model from acting upon the bias that\u2019s been projected out of the subspace, it can end up amplifying and creating other biases, hence the Whac-A-Mole dilemma. According to Ghassemi, the unintended amplification of model biases is \u201cboth a technical and practical challenge. For instance, when debiasing a VLM that retrieves images of clinical staff \u2014 if racial bias is removed \u2014 it could have the unintended consequence of amplifying gender bias.\u201d\u00a0<\/p>\n WRING works by moving certain coordinates within the high-dimensional space of a model \u2014 the ones that appear to be responsible for bias \u2014 to a different angle, so the model can no longer distinguish between different groups within a certain concept. This changes the representation within a specific space while leaving the model\u2019s other relationships intact. And like projection debiasing, WRING is a post-processing approach, which means it can be applied \u201con the fly\u201d to a pre-trained VLM.\u00a0<\/p>\n \u201cPeople already spent a lot of resources, a lot of money, training these huge models, and we don\u2019t really want to go in and modify something during training because then you have to start from scratch,\u201d Gerych explains. \u201c[WRING is] very efficient. It doesn\u2019t require more training of the model and it\u2019s minimally invasive.\u201d<\/p>\n In their results, the researchers found that WRING significantly reduced bias for a target concept without increasing bias in other areas. But for now, the approach is somewhat limited to Contrastive Language-Image Pre-training (CLIP) models, a type of VLM that connects images to language for search or classification.<\/p>\n \u201cExtending this for ChatGPT-style, generative language models, is the reasonable next step for us,\u201d says Gerych.<\/p>\n This work was supported, in part, by a National Science Foundation CAREER Award, AI2050 Award Early Career Fellowship, Sloan Research Fellow Award, the Gordon and Betty Moore Foundation Award, and MIT-Google Computing Innovation Award.<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"