Author: Kim Martineau | MIT Quest for Intelligence<\/p>\n
Learning to code involves recognizing how to structure a program, and how to fill in every last detail correctly. No wonder it can be so frustrating.<\/p>\n
A new program-writing AI, SketchAdapt<\/a>, offers a way out. Trained on tens of thousands of program examples, SketchAdapt learns how to compose short, high-level programs, while letting a second set of algorithms find the right sub-programs to fill in the details. Unlike similar approaches for automated program-writing, SketchAdapt knows when to switch from statistical pattern-matching to a less efficient, but more versatile, symbolic reasoning mode to fill in the gaps.<\/p>\n \u201cNeural nets are pretty good at getting the structure right, but not the details,\u201d says Armando Solar-Lezama<\/a>, a professor at MIT\u2019s Computer Science and Artificial Intelligence Laboratory<\/a> (CSAIL). \u201cBy dividing up the labor \u2014 letting the neural nets handle the high-level structure, and using a search strategy to fill in the blanks \u2014 we can write efficient programs that give the right answer.\u201d<\/p>\n SketchAdapt is a collaboration between Solar-Lezama and Josh Tenenbaum<\/a>, a professor at CSAIL and MIT\u2019s Center for Brains, Minds and Machines<\/a>. The work will be presented at the International Conference on Machine Learning<\/a> June 10-15.<\/p>\n Program synthesis, or teaching computers to code, has long been a goal of AI researchers. A computer that can program itself is more likely to learn language faster, converse fluently, and even model human cognition. All of this drew Solar-Lezama to the field as a graduate student, where he laid the foundation for SketchAdapt.<\/p>\n Solar-Lezama\u2019s early work, Sketch<\/a>, is based on the idea that a program\u2019s low-level details could be found mechanically if a high-level structure is provided. Among other applications, Sketch inspired spinoffs to automatically grade programming homework and convert hand-drawn diagrams into code. Later, as neural networks grew in popularity, students from Tenenbaum\u2019s computational cognitive science lab suggested a collaboration, out of which SketchAdapt formed.<\/p>\n Rather than rely on experts to define program structure, SketchAdapt figures it out using deep learning. The researchers also added a twist: When the neural networks are unsure of what code to place where, SketchAdapt is programmed to leave the spot blank for search algorithms to fill.<\/p>\n \u201cThe system decides for itself what it knows and doesn\u2019t know,\u201d says the study\u2019s lead author, Maxwell Nye<\/a>, a graduate student in MIT\u2019s Department of Brain and Cognitive Sciences<\/a>. \u00a0\u201cWhen it gets stuck, and has no familiar patterns to draw on, it leaves placeholders in the code. It then uses a guess-and-check strategy to fill the holes.\u201d<\/p>\n The researchers compared SketchAdapt\u2019s performance to programs modeled after Microsoft\u2019s proprietary RobustFill<\/a> and DeepCoder<\/a> software, successors to Excel\u2019s FlashFill feature, which analyzes adjacent cells to offer suggestions as you type \u2014 learning to transform a column of names into a column of corresponding email addresses, for example. RobustFill uses deep learning to write high-level programs from examples, while DeepCoder specializes in finding and filling in low-level details.<\/p>\n The researchers found that SketchAdapt outperformed their\u00a0reimplemented versions of RobustFill and DeepCoder at their respective specialized tasks.\u00a0SketchAdapt outperformed the\u00a0RobustFill-like program\u00a0at string transformations; for example, writing a program to abbreviate Social Security numbers as three digits, and first names by their first letter. SketchAdapt also did better than the\u00a0DeepCoder-like program\u00a0at writing programs to\u00a0transform\u00a0a list of numbers.\u00a0Trained\u00a0only\u00a0on examples of three-line list-processing programs, SketchAdapt was better able to transfer its knowledge to a new scenario and write\u00a0correct\u00a0four-line programs.<\/p>\n In yet another task, SketchAdapt outperformed both programs at converting math problems from English to code, and calculating the answer.<\/p>\n Key to its success is the ability to switch from neural pattern-matching to a rules-based symbolic search, says Rishabh Singh<\/a>, a former graduate student of Solar-Lezama\u2019s, now a researcher at Google Brain. \u201cSketchAdapt learns how much pattern recognition is needed to write familiar parts of the program, and how much symbolic reasoning is needed to fill in details which may involve new or complicated concepts.\u201d<\/p>\n SketchAdapt is limited to writing very short programs. Anything more requires too much computation. Nonetheless, it\u2019s intended more to complement programmers rather than replace them, the researchers say. \u201cOur focus is on giving programming tools to people who want them,\u201d says Nye. \u201cThey can tell the computer what they want to do, and the computer can write the program.\u201d<\/p>\n Programming, after all, has always evolved. When Fortran was introduced in the 1950s, it was meant to replace human programmers. \u201cIts full name was Fortran Automatic Coding System, and its goal was to write programs as well as humans, but without the errors,\u201d says Solar-Lezama. \u201cWhat it really did was automate much of what programmers did before Fortran. It changed the nature of programming.\u201d<\/p>\n The study\u2019s other co-author is Luke Hewitt. Funding was provided by the U.S. Air Force Office of Scientific Research, MIT-IBM Watson AI Lab and U.S. National Science Foundation.<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"