Author: Rachel Gordon | MIT CSAIL<\/p>\n
Turns out, even language models \u201cthink\u201d they\u2019re biased. When prompted in ChatGPT, the response was as follows: \u201cYes, language models can have biases, because the training data reflects the biases present in society from which that data was collected. For example, gender and racial biases are prevalent in many real-world datasets, and if a language model is trained on that, it can perpetuate and amplify these biases in its predictions.\u201d A well-known but dangerous problem.\u00a0<\/p>\n
Humans (typically) can dabble with both logical and stereotypical reasoning when learning. Still, language models mainly mimic the latter, an unfortunate narrative we\u2019ve seen play out ad nauseam when the ability to employ reasoning and critical thinking is absent. So would injecting logic into the fray be enough to mitigate such behavior?\u00a0<\/p>\n
Scientists from MIT\u2019s Computer Science and Artificial Intelligence Laboratory (CSAIL) had an inkling that it might, so they set off to examine if logic-aware language models could significantly avoid more harmful stereotypes. They trained a language model to predict the relationship between two sentences, based on context and semantic meaning, using a dataset with labels for text snippets detailing if a second phrase \u201centails,\u201d \u201ccontradicts,\u201d or is neutral with respect to the first one. Using this dataset \u2014 natural language inference \u2014 they found that the newly trained models were significantly less biased than other baselines, without any extra data, data editing, or additional training algorithms.<\/p>\n
For example, with the premise \u201cthe person is a doctor\u201d and the hypothesis \u201cthe person is masculine,\u201d using these logic-trained models, the relationship would be classified as \u201cneutral,\u201d since there\u2019s no logic that says the person is a man. With more common language models, two sentences might seem to be correlated due to some bias in training data, like \u201cdoctor\u201d might be pinged with \u201cmasculine,\u201d even when there\u2019s no evidence that the statement is true.\u00a0<\/p>\n
At this point, the omnipresent nature of language models is well-known: Applications in natural language processing, speech recognition, conversational AI, and generative tasks abound. While not a nascent field of research, growing pains can take a front seat as they increase in complexity and capability.\u00a0<\/p>\n
\u201cCurrent language models suffer from issues with fairness, computational resources, and privacy,\u201d says MIT CSAIL postdoc Hongyin Luo, the lead author of a new paper about the work. \u201cMany estimates say that the CO2<\/sub> emission of training a language model can be higher than the lifelong emission of a car. Running these large language models is also very expensive because of the amount of parameters and the computational resources they need. With privacy, state-of-the-art language models developed by places like ChatGPT or GPT-3 have their APIs where you must upload your language, but there\u2019s no place for sensitive information regarding things like health care or finance. To solve these challenges, we proposed a logical language model that we qualitatively measured as fair, is 500 times smaller than the state-of-the-art models, can be deployed locally, and with no human-annotated training samples for downstream tasks. Our model uses 1\/400 the parameters compared with the largest language models, has better performance on some tasks, and significantly saves computation resources.\u201d\u00a0<\/p>\n This model, which has 350 million parameters, outperformed some<\/a> very large-scale language models with 100 billion parameters on logic-language understanding tasks. The team evaluated, for example, popular BERT pretrained language models with their \u201ctextual entailment\u201d ones on stereotype, profession, and emotion bias tests. The latter outperformed other models with significantly lower bias, while preserving the language modeling ability. The \u201cfairness\u201d was evaluated with something called ideal context association (iCAT) tests, where higher iCAT scores mean fewer stereotypes. The model had higher than 90 percent iCAT scores, while other strong language understanding models ranged between 40 to 80.\u00a0<\/p>\n Luo wrote the paper alongside MIT Senior Research Scientist James Glass. They will present the work at the Conference of the European Chapter of the Association for Computational Linguistics in Croatia.\u00a0<\/strong><\/p>\n Unsurprisingly, the original pretrained language models the team examined were teeming with bias, confirmed by a slew of reasoning tests demonstrating how professional and emotion terms are significantly biased to the feminine or masculine words in the gender vocabulary.\u00a0<\/p>\n With professions, a language model (which is biased) thinks that \u201cflight attendant,\u201d \u201csecretary,\u201d and \u201cphysician\u2019s assistant\u201d are feminine jobs, while \u201cfisherman,\u201d \u201clawyer,\u201d and \u201cjudge\u201d are masculine. Concerning emotions, a language model thinks that \u201canxious,\u201d \u201cdepressed,\u201d and \u201cdevastated\u201d are feminine.<\/p>\n While we may still be far away from a neutral language model utopia, this research is ongoing in that pursuit. Currently, the model is just for language understanding, so it\u2019s based on reasoning among existing sentences. Unfortunately, it can\u2019t generate sentences for now, so the next step for the researchers would be targeting the uber-popular generative models built with logical learning to ensure more fairness with computational efficiency.\u00a0<\/p>\n \u201cAlthough stereotypical reasoning is a natural part of human recognition, fairness-aware people conduct reasoning with logic rather than stereotypes when necessary,” says Luo. “We show that language models have similar properties. A language model without explicit logic learning makes plenty of biased reasoning, but adding logic learning can significantly mitigate such behavior. Furthermore, with demonstrated robust zero-shot adaptation ability, the model can be directly deployed to different tasks with more fairness, privacy, and better speed.\u201d<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Author: Rachel Gordon | MIT CSAIL Turns out, even language models \u201cthink\u201d they\u2019re biased. When prompted in ChatGPT, the response was as follows: \u201cYes, language […] Read More<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":462,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[24],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/6325"}],"collection":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=6325"}],"version-history":[{"count":0,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/6325\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media\/473"}],"wp:attachment":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=6325"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=6325"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=6325"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}