Author: Kim Martineau | MIT Quest for Intelligence<\/p>\n
A few years ago, the idea of tricking a computer vision system by subtly altering pixels in an image or hacking a street sign seemed like more of a hypothetical threat than anything to seriously worry about. After all, a self-driving car in the real world would perceive a manipulated object from multiple viewpoints, cancelling out any misleading information. At least, that\u2019s what one study\u00a0claimed<\/a>.<\/p>\n \u201cWe thought, there\u2019s no way that\u2019s true!\u201d says MIT PhD student Andrew Ilyas<\/a>, then a sophomore at MIT. He and his friends \u2014 Anish Athalye, Logan Engstrom, and Jessy Lin \u2014 holed up at the MIT Student Center and came up with an experiment to refute the study. They would print a set of three-dimensional turtles and show that a computer vision classifier could mistake them for rifles.<\/p>\n The results of their experiments,\u00a0published<\/a>\u00a0at last year\u2019s International Conference on Machine Learning (ICML), were widely covered in the\u00a0media<\/a>, and served as a reminder of just how vulnerable the artificial intelligence systems behind self-driving cars and face-recognition software could be. \u201cEven if you don\u2019t think a mean attacker is going to perturb your stop sign, it\u2019s troubling that it\u2019s a possibility,\u201d says Ilyas. \u201cAdversarial example research is about optimizing for the worst case instead of the average case.\u201d<\/p>\n With no faculty co-authors to vouch for them, Ilyas and his friends published their study under the pseudonym \u201cLab 6,\u201d a play on\u00a0Course 6<\/a>, their\u00a0Department of Electrical Engineering and Computer Science<\/a>\u00a0(EECS) major. Ilyas and Engstrom, now an MIT graduate student, would go on to publish five more papers together, with a half-dozen more in the pipeline.<\/p>\n At the time, the risk posed by adversarial examples was still poorly understood. Yann LeCun, the head of Facebook AI, famously downplayed the problem<\/a> on Twitter. \u201cHere\u2019s one of the pioneers of deep learning saying, this is how it is, and they say, nah!\u201d says EECS Professor\u00a0Aleksander Madry<\/a>. \u201cIt just didn\u2019t sound right to them and they were determined to prove why. Their audacity is very MIT.\u201d\u00a0<\/p>\n The extent of the problem has grown clearer. In 2017, IBM researcher\u00a0Pin-Yu Chen<\/a>\u00a0showed<\/a>\u00a0that a computer vision model could be compromised in a so-called black-box attack by simply feeding it progressively altered images until one caused the system to fail. Expanding on Chen\u2019s work at ICML last year, the Lab 6 team\u00a0highlighted<\/a>\u00a0multiple cases in which classifiers could be duped into confusing cats and skiers for guacamole and dogs, respectively.<\/p>\n This spring, Ilyas, Engstrom, and Madry presented a framework at ICML for making black-box attacks several times faster by exploiting information gained from each spoofing attempt. The ability to mount more efficient black-box attacks allows engineers to redesign their models to be that much more resilient.<\/p>\n \u201cWhen I met Andrew and Logan as undergraduates, they already seemed like experienced researchers,\u201d says Chen, who now works with them via the\u00a0MIT-IBM Watson AI Lab<\/a>. \u201cThey\u2019re also great collaborators. If one is talking, the other jumps in and finishes his thought.\u201d<\/p>\n That dynamic was on display recently as Ilyas and Engstrom sat down in Stata to discuss their work. Ilyas seemed introspective and cautious, Engstrom, outgoing, and at times, brash.<\/p>\n \u201cIn research, we argue a lot,\u201d says Ilyas. \u201cIf you\u2019re too similar you reinforce each other\u2019s bad ideas.\u201d Engstrom nodded. \u201cIt can get very tense.\u201d<\/p>\n When it comes time to write papers, they take turns at the keyboard. \u201cIf it\u2019s me, I add words,\u201d says Ilyas. \u201cIf it\u2019s me, I cut words,\u201d says Engstrom.<\/p>\n Engstrom joined Madry\u2019s lab for a\u00a0SuperUROP project as a junior; Ilyas joined last fall\u00a0as a first-year PhD student after finishing his undergraduate and MEng degrees early. Faced with offers from other top graduate schools, Ilyas opted to stay at MIT. A year later, Engstrom followed.<\/p>\n This spring the pair was back in the news again, with a\u00a0new way<\/a>\u00a0of looking at adversarial examples: not as bugs, but as features corresponding to patterns too subtle for humans to perceive that are still useful to learning algorithms. We know instinctively that people and machines see the world differently, but the paper showed that the difference could be isolated and measured.<\/p>\n They trained a model to identify cats based on \u201crobust\u201d features recognizable to humans, and \u201cnon-robust\u201d features that humans typically overlook, and found that visual classifiers could just as easily identify a cat from non-robust features as robust. If anything, the model seemed to rely more on the non-robust features, suggesting that as accuracy improves, the model may become more susceptible to adversarial examples.\u00a0<\/p>\n \u201cThe only thing that makes these features special is that we as humans are not sensitive to them,\u201d Ilyas\u00a0told Wired<\/a>.<\/p>\n Their eureka moment came late one night in Madry\u2019s lab, as they often do, following hours of talking. \u201cConversation is the most powerful tool for scientific discovery,\u201d Madry likes to say. The team quickly sketched out experiments to test their idea.<\/p>\n \u201cThere are many beautiful theories proposed in deep learning,\u201d says Madry. \u201cBut no hypothesis can be accepted until you come up with a way of verifying it.\u201d<\/p>\n \u201cThis is a new field,\u201d he adds. \u201cWe don\u2019t know the answers to the questions, and I would argue we don\u2019t even know the right questions. Andrew and Logan have the brilliance and drive to help lead the way.\u201d<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"