Author: Kim Martineau | MIT Quest for Intelligence<\/p>\n
Neural networks have given researchers a powerful tool for looking into the future and making predictions. But one drawback is their insatiable need for data and computing power (“compute”) to process all that information. At MIT, demand for compute is estimated to be five times greater than what the Institute can offer. To help ease the crunch, industry has stepped in. An $11.6 million supercomputer recently\u00a0donated by IBM<\/a>\u00a0comes online this fall, and in the past year, both IBM and Google have provided cloud credits\u00a0to MIT Quest for Intelligence for distribution across campus. Four projects made possible by IBM and Google cloud donations are highlighted below.<\/p>\n Smaller, faster, smarter neural networks<\/strong><\/p>\n To recognize a cat in a picture, a deep learning model may need to see millions of photos before its artificial neurons \u201clearn\u201d to identify a cat. The process is computationally intensive and carries a steep\u00a0environmental cost<\/a>,\u00a0as new research attempting to measure artificial intelligence’s (AI\u2019s) carbon footprint has highlighted.\u00a0<\/p>\n But there may be a more efficient way. New MIT research shows that models only a fraction of the size are needed. \u201cWhen you train a big network there\u2019s a small one that could have done everything,\u201d says\u00a0Jonathan Frankle<\/a>, a graduate student in MIT\u2019s\u00a0Department of Electrical Engineering and Computer Science<\/a> (EECS).<\/p>\n With study co-author and EECS Professor\u00a0Michael Carbin<\/a>, Frankle\u00a0estimates<\/a>\u00a0that a neural network could get by with on-tenth the number of connections if the right subnetwork is found at the outset. Normally, neural networks are trimmed after the training process, with irrelevant connections removed then.\u00a0Why not train the small model to begin with, Frankle wondered?<\/p>\n Experimenting with a two-neuron network on his laptop, Frankle got encouraging results and moved to larger image-datasets like MNIST and CIFAR-10, borrowing GPUs where he could. Finally, through IBM Cloud, he secured enough compute power to train a real ResNet model. \u201cEverything I\u2019d done previously was toy experiments,\u201d he says. \u201cI was finally able to run dozens of different settings to make sure I could make the claims in our paper.\u201d<\/p>\n Frankle spoke from Facebook\u2019s offices, where he worked for the summer to explore ideas raised by his\u00a0Lottery Ticket Hypothesis<\/a>\u00a0paper, one of two picked for a best paper award at this year\u2019s International Conference on Learning Representations. Potential applications for the work go beyond image classification, Frankle says, and include reinforcement learning and natural language processing models. Already, researchers at\u00a0Facebook AI Research<\/a>,\u00a0Princeton University<\/a>, and\u00a0Uber<\/a>\u00a0have published follow-on studies.\u00a0<\/p>\n \u201cWhat I love about neural networks is we haven\u2019t even laid the foundation yet,\u201d says Frankle, who recently shifted from studying cryptography and tech policy to AI. \u201cWe really don\u2019t understand how it learns, where it\u2019s good and where it fails. This is physics 1,000 years before Newton.\u201d<\/p>\n Distinguishing fact from fake news<\/strong><\/p>\n Networking platforms like Facebook and Twitter have made it easier than ever to find quality news. But too often, real news is drowned out by misleading or outright false information posted online. Confusion over a recent video of U.S. House Speaker Nancy Pelosi doctored to make her sound drunk is just the latest example of the threat misinformation and fake news pose to democracy.\u00a0<\/p>\n \u201cYou can put just about anything up on the internet now, and some people will believe it,\u201d says\u00a0Moin Nadeem<\/a>, a senior and EECS major at MIT.<\/p>\n If technology helped create the problem, it can also help fix it. That was Nadeem\u2019s reason for picking a\u00a0superUROP<\/a>\u00a0project focused on building an automated system to fight fake and misleading news. Working in the lab of James Glass, a researcher at MIT\u2019s\u00a0Computer Science and Artificial Intelligence Laboratory<\/a>, and supervised by Mitra Mohtarami, Nadeem helped train a language model to fact-check claims by searching through Wikipedia and three types of news sources rated by journalists as high-quality, mixed-quality or low-quality.<\/p>\n To verify a claim, the model measures how closely the sources agree, with higher agreement scores indicating the claim is likely true. A high disagreement score for a claim like, \u201cISIS infiltrates the United States,\u201d is a strong indicator of fake news. One drawback of this method, he says, is that the model doesn\u2019t identify the independent truth so much as describe what most people think is true.<\/p>\n With the help of Google Cloud Platform, Nadeem ran experiments and built an interactive website that lets users instantly assess the accuracy of a claim. He and his co-authors presented their\u00a0results<\/a>\u00a0at the North American Association of Computational Linguistics (NAACL) conference in June and are continuing to expand on the work.<\/p>\n \u201cThe saying used to be that seeing is believing,\u201d says Nadeem, in\u00a0this video<\/a>\u00a0about his work. \u201cBut we\u2019re entering a world where that isn\u2019t true. If people can\u2019t trust their eyes and ears it becomes a question of what\u00a0can\u00a0<\/em>we trust?\u201d<\/p>\n Visualizing a warming climate<\/strong><\/p>\n From rising seas to increased droughts, the effects of climate change are already being felt. A few decades from now, the world will be a warmer, drier, and more unpredictable place.\u00a0Brandon Leshchinskiy<\/a>, a graduate student in MIT\u2019s\u00a0Department of Aeronautics and Astronautics<\/a>\u00a0(AeroAstro), is experimenting with generative adversarial networks, or GANs, to imagine what Earth will look like then.\u00a0<\/p>\n GANs produce hyper-realistic imagery by pitting one neural network against another. The first network learns the underlying structure of a set of images and tries to reproduce them, while the second decides which images look implausible and tells the first network to try again.<\/p>\n Inspired by researchers who used GANs to\u00a0visualize<\/a>\u00a0sea-level rise projections from street-view images, Leshchinskiy wanted to see if satellite imagery could similarly personalize climate projections. With his advisor, AeroAstro Professor\u00a0Dava Newman<\/a>, Leshchinskiy is currently using free IBM Cloud credits to train a pair of GANs on images of the eastern U.S. coastline with their corresponding elevation points. The goal is to visualize how sea-level rise projections for 2050 will redraw the coastline. If the project works, Leshinskiy hopes to use other NASA datasets to imagine future ocean acidification and changes in phytoplankton abundance.\u00a0<\/p>\n \u201cWe\u2019re past the point of mitigation,\u201d he says. \u201cVisualizing what the world will look like three decades from now can help us adapt to climate change.\u201d<\/p>\n Identifying athletes from a few gestures<\/strong><\/p>\n A few moves on the field or court are enough for a computer vision model to identify individual athletes. That\u2019s according to preliminary research by a team led by\u00a0Katherine Gallagher<\/a>, a researcher at MIT Quest for Intelligence.<\/p>\n The team trained computer vision models\u00a0on video recordings of tennis matches and soccer and basketball games\u00a0and found that the models could recognize individual players in just a few frames from key points on their body providing a rough outline of their skeleton.\u00a0<\/p>\n The team used a Google Cloud API to process the video data, and compared their models’ performance against\u00a0models trained on Google Cloud’s AI platform.\u00a0\u201cThis pose information is so distinctive that our models\u00a0can identify players with accuracy almost as good as models provided with\u00a0much\u00a0more information,\u00a0like hair color and clothing,\u201d she says.\u00a0<\/p>\n Their results are relevant for automated player identification in sports analytics systems, and they could provide a basis for further research on inferring player fatigue to anticipate when players should be swapped out. Automated pose detection could also help athletes refine their technique by allowing them to isolate the precise moves associated with a golfer\u2019s expert drive or a tennis player\u2019s winning swing.<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"