By Faruqui Ismail and NookaRaju Garimella<\/p>\n
Reporters with various forms of “fake news” from an 1894 illustration by Frederick Burr Opper<\/a><\/span><\/p>\n \u00a0<\/p>\n We\u2019ve always pictured the rise of artificial intelligence as being the end of civilization, at least from watching movies like \u2018The Terminator \u2013 Judgement Day<\/em>\u2019. We could not have imagined that something as insignificant as misinformation, would lead to the collapse of organisations; beginning wars and even mass suicides.<\/p>\n \u00a0<\/p>\n The definition of what we regard as \u201cFake\u201d news has a broad spectrum. Consider an article published in 2001, which was true at the time. That same article being published now, excluding the date\u2026 giving it an appearance of recently occurring events. Would be regarded as \u201cmisinformation\u201d or \u201cFake\u201d.<\/p>\n \u00a0<\/p>\n In summary, we identified a need to identify the truth from misinformation and created a product that would help us do that. We began by creating 2 robots using BeautifulSoup (bs4) and Selenium, these robots extracted data from various fake news sites according to Wikipedia. We then supplemented this data with GitHub data (refer to acknowledgements).<\/p>\n \u00a0<\/p>\n Post cleaning and reworking the data using some Natural Language Processing(NLP) techniques, we proceeded to create features. By asking the question, what makes a fake news article different from a non-fake news article? We agreed on the following:<\/p>\n<\/p>\n \u00a0<\/p>\n To increase the overall accuracy of the final prediction. These features were then checked to see if they were not too correlated, and that the sub contents of some of these features did not overlap e.g.:<\/p>\n<\/p>\n To avoid over-fitting of the model, feature transformation was done. This helped normalize the feature which helped prevent over-fitting. This visual is an example of the transformation done of % of upper case letters to the new article:<\/p>\n \u00a0<\/p>\n Est: 50<\/span><\/p>\n<\/td>\n \u00a0Depth: 10<\/span><\/p>\n<\/td>\n Precision: 0.6921<\/span><\/p>\n<\/td>\n Recall: 0.4833<\/span><\/p>\n<\/td>\n Accuracy: 0.4769<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 50<\/span><\/p>\n<\/td>\n \u00a0Depth: 30<\/span><\/p>\n<\/td>\n Precision: 0.8405<\/span><\/p>\n<\/td>\n Recall: 0.8166<\/span><\/p>\n<\/td>\n Accuracy: 0.7923<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 50<\/span><\/p>\n<\/td>\n \u00a0Depth: 90<\/span><\/p>\n<\/td>\n Precision: 0.8479<\/span><\/p>\n<\/td>\n Recall: 0.8416<\/span><\/p>\n<\/td>\n Accuracy: 0.8461<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 50<\/span><\/p>\n<\/td>\n \u00a0Depth: None<\/span><\/p>\n<\/td>\n Precision: 0.8143<\/span><\/p>\n<\/td>\n Recall: 0.7916<\/span><\/p>\n<\/td>\n Accuracy: 0.8076<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 100<\/span><\/p>\n<\/td>\n \u00a0Depth: 10<\/span><\/p>\n<\/td>\n Precision: 0.7159<\/span><\/p>\n<\/td>\n Recall: 0.6416<\/span><\/p>\n<\/td>\n Accuracy: 0.6153<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 100<\/span><\/p>\n<\/td>\n \u00a0Depth: 30<\/span><\/p>\n<\/td>\n Precision: 0.8352<\/span><\/p>\n<\/td>\n Recall: 0.8<\/span><\/p>\n<\/td>\n Accuracy: 0.7923<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 100<\/span><\/p>\n<\/td>\n \u00a0Depth: 90<\/span><\/p>\n<\/td>\n Precision: 0.8685<\/span><\/p>\n<\/td>\n Recall: 0.8583<\/span><\/p>\n<\/td>\n Accuracy: 0.8615<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 100<\/span><\/p>\n<\/td>\n \u00a0Depth: None<\/span><\/p>\n<\/td>\n Precision: 0.8936<\/span><\/p>\n<\/td>\n Recall: 0.9166<\/span><\/p>\n<\/td>\n Accuracy: 0.9076<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 150<\/span><\/p>\n<\/td>\n \u00a0Depth: 10<\/span><\/p>\n<\/td>\n Precision: 0.7066<\/span><\/p>\n<\/td>\n Recall: 0.6<\/span><\/p>\n<\/td>\n Accuracy: 0.5615<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 150<\/span><\/p>\n<\/td>\n \u00a0Depth: 30<\/span><\/p>\n<\/td>\n Precision: 0.8398<\/span><\/p>\n<\/td>\n Recall: 0.8333<\/span><\/p>\n<\/td>\n Accuracy: 0.8230<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 150<\/span><\/p>\n<\/td>\n \u00a0Depth: 90<\/span><\/p>\n<\/td>\n Precision: 0.8613<\/span><\/p>\n<\/td>\n Recall: 0.8583<\/span><\/p>\n<\/td>\n Accuracy: 0.8461<\/span><\/p>\n<\/td>\n<\/tr>\n Est: 150<\/span><\/p>\n<\/td>\n \u00a0Depth: None<\/span><\/p>\n<\/td>\n Precision: 0.8786<\/span><\/p>\n<\/td>\n Recall: 0.8833<\/span><\/p>\n<\/td>\n Accuracy: 0.8769<\/span><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n This entire project was then packaged into a web framework using Django. The view showed whether the data was e.g. unreliable, junk science, fake, true etc. It is in the process of being published on the web.\u00a0<\/p>\n \u00a0<\/p>\n Authors and Creators:<\/p>\n Nooka Raju Garimella<\/a><\/p>\n<\/p>\n Acknowledgements<\/strong>:<\/p>\n GitHub data: https:\/\/github.com\/several27\/FakeNewsCorpus<\/a><\/em><\/span><\/p>\n Wikipedia fake news list: https:\/\/en.wikipedia.org\/wiki\/List_of_fake_news_websites<\/a><\/em><\/span><\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<\/a><\/p>\n
\n
<\/a><\/p>\n
\n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n