Author: Steve Nadis | MIT CSAIL<\/p>\n
Functional magnetic resonance imaging (fMRI), which measures changes in blood flow throughout the brain, has been used over the past couple of decades for a variety of applications, including \u201cfunctional anatomy\u201d \u2014 a way of determining which brain areas are switched on when a person carries out a particular task. fMRI has been used to look at people\u2019s brains while they\u2019re doing all sorts of things \u2014 working out math problems, learning foreign languages, playing chess, improvising on the piano, doing crossword puzzles, and even watching TV shows like \u201cCurb Your Enthusiasm.\u201d<\/p>\n
One pursuit that\u2019s received little attention is computer programming \u2014 both the chore of writing code and the equally confounding task of trying to understand a piece of already-written code. \u201cGiven the importance that computer programs have assumed in our everyday lives,\u201d says Shashank Srikant, a PhD student in MIT\u2019s Computer Science and Artificial Intelligence Laboratory (CSAIL), \u201cthat\u2019s surely worth looking into. So many people are dealing with code these days \u2014 reading, writing, designing, debugging \u2014 but no one really knows what\u2019s going on in their heads when that happens.\u201d Fortunately, he has made some \u201cheadway\u201d in that direction in a paper \u2014 written with MIT colleagues Benjamin Lipkin (the paper\u2019s other lead author, along with Srikant), Anna Ivanova, Evelina Fedorenko, and Una-May O\u2019Reilly \u2014 that was presented earlier this month<\/a> at the Neural Information Processing Systems Conference held in New Orleans.<\/p>\n The new paper built on a 2020 study<\/a>, written by many of the same authors, which used fMRI to monitor the brains of programmers as they \u201ccomprehended\u201d small pieces, or snippets, of code. (Comprehension, in this case, means looking at a snippet and correctly determining the result of the computation performed by the snippet.) The 2020 work showed that code comprehension did not consistently activate the language system, brain regions that handle language processing, explains Fedorenko, a brain and cognitive sciences (BCS) professor and a coauthor of the earlier study. \u201cInstead, the multiple demand network \u2014 a brain system that is linked to general reasoning and supports domains like mathematical and logical thinking \u2014 was strongly active.\u201d The current work, which also utilizes MRI scans of programmers, takes \u201ca deeper dive,\u201d she says, seeking to obtain more fine-grained information.<\/p>\n Whereas the previous study looked at 20 to 30 people to determine which brain systems, on average, are relied upon to comprehend code, the new research looks at the brain activity of individual programmers as they process specific elements of a computer program. Suppose, for instance, that there\u2019s a one-line piece of code that involves word manipulation and a separate piece of code that entails a mathematical operation. \u201cCan I go from the activity we see in the brains, the actual brain signals, to try to reverse-engineer and figure out what, specifically, the programmer was looking at?\u201d Srikant asks. \u201cThis would reveal what information pertaining to programs is uniquely encoded in our brains.\u201d To neuroscientists, he notes, a physical property is considered \u201cencoded\u201d if they can infer that property by looking at someone\u2019s brain signals.<\/p>\n Take, for instance, a loop \u2014 an instruction within a program to repeat a specific operation until the desired result is achieved \u2014 or a branch, a different type of programming instruction than can cause the computer to switch from one operation to another. Based on the patterns of brain activity that were observed, the group could tell whether someone was evaluating a piece of code involving a loop or a branch. The researchers could also tell whether the code related to words or mathematical symbols, and whether someone was reading actual code or merely a written description of that code.<\/p>\n That addressed a first question that an investigator might ask as to whether something is, in fact, encoded. If the answer is yes, the next question might be: where is it encoded? In the above-cited cases \u2014 loops or branches, words or math, code or a description thereof \u2014 brain activation levels were found to be comparable in both the language system and the multiple demand network.<\/p>\n A noticeable difference was observed, however, when it came to code properties related to what\u2019s called dynamic analysis.<\/p>\n Programs can have \u201cstatic\u201d properties \u2014 such as the number of numerals in a sequence \u2014 that do not change over time. \u201cBut programs can also have a dynamic aspect, such as the number of times a loop runs,\u201d Srikant says. \u201cI can\u2019t always read a piece of code and know, in advance, what the run time of that program will be.\u201d The MIT researchers found that for dynamic analysis, information is encoded much better in the multiple demand network than it is in the language processing center. That finding was one clue in their quest to see how code comprehension is distributed throughout the brain \u2014 which parts are involved and which ones assume a bigger role in certain aspects of that task.<\/p>\n The team carried out a second set of experiments, which incorporated machine learning models called neural networks that were specifically trained on computer programs. These models have been successful, in recent years, in helping programmers complete pieces of code. What the group wanted to find out was whether the brain signals seen in their study when participants were examining pieces of code resembled the patterns of activation observed when neural networks analyzed the same piece of code. And the answer they arrived at was a qualified yes.<\/p>\n \u201cIf you put a piece of code into the neural network, it produces a list of numbers that tells you, in some way, what the program is all about,\u201d Srikant says. Brain scans of people studying computer programs similarly produce a list of numbers. When a program is dominated by branching, for example, \u201cyou see a distinct pattern of brain activity,\u201d he adds, \u201cand you see a similar pattern when the machine learning model tries to understand that same snippet.\u201d<\/p>\n Mariya Toneva of the Max Planck Institute for Software Systems considers findings like this \u201cparticularly exciting. They raise the possibility of using computational models of code to better understand what happens in our brains as we read programs,\u201d she says.<\/p>\n The MIT scientists are definitely intrigued by the connections they\u2019ve uncovered, which shed light on how discrete pieces of computer programs are encoded in the brain. But they don\u2019t yet know what these recently-gleaned insights can tell us about how people carry out more elaborate plans in the real world. Completing tasks of this sort \u2014 such as going to the movies, which requires checking showtimes, arranging for transportation, purchasing tickets, and so forth \u2014 could not be handled by a single unit of code and just a single algorithm. Successful execution of such a plan would instead require \u201ccomposition\u201d \u2014 stringing together various snippets and algorithms into a sensible sequence that leads to something new, just like assembling individual bars of music in order to make a song or even a symphony. Creating models of code composition, says O\u2019Reilly, a principal research scientist at CSAIL, \u201cis beyond our grasp at the moment.\u201d<\/p>\n Lipkin, a BCS PhD student, considers this the next logical step \u2014 figuring out how to \u201ccombine simple operations to build complex programs and use those strategies to effectively address general reasoning tasks.\u201d He further believes that some of the progress toward that goal achieved by the team so far owes to its interdisciplinary makeup. \u201cWe were able to draw from individual experiences with program analysis and neural signal processing, as well as combined work on machine learning and natural language processing,\u201d Lipkin says. \u201cThese types of collaborations are becoming increasingly common as neuro- and computer scientists join forces on the quest towards understanding and building general intelligence.\u201d<\/p>\n This project was funded by grants from the MIT-IBM Watson AI lab, MIT Quest Initiative, National Science Foundation, National Institutes of Health, McGovern Institute of Brain Research, MIT Department of Brain and Cognitive Sciences, and the Simons Center for the Social Brain.<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"