Author: Rob Matheson | MIT News Office<\/p>\n
MIT researchers have devised a method for assessing how robust machine-learning models known as neural networks are for various tasks, by detecting when the models make mistakes they shouldn\u2019t.<\/p>\n
Convolutional neural networks (CNNs) are designed to process and classify images for computer vision and many other tasks. But slight modifications that are imperceptible to the human eye \u2014 say, a few darker pixels within an image \u2014 may cause a CNN to produce a drastically different classification. Such modifications are known as \u201cadversarial examples.\u201d Studying the effects of adversarial examples on neural networks can help researchers determine how their models could be vulnerable to unexpected inputs in the real world.<\/p>\n
For example, driverless cars can use CNNs to process visual input and produce an appropriate response. If the car approaches a stop sign, it would recognize the sign and stop. But a 2018 paper found that placing a certain black-and-white sticker on the stop sign could, in fact, fool a driverless car\u2019s CNN to misclassify the sign, which could potentially cause it to not stop at all.<\/p>\n
However, there has been no way to fully evaluate a large neural network\u2019s resilience to adversarial examples for all test inputs. In a paper they are presenting this week at the International Conference on Learning Representations, the researchers describe a technique that, for any input, either finds an adversarial example or guarantees that all perturbed inputs \u2014\u00a0that still appear similar to the original \u2014\u00a0are correctly classified. In doing so, it gives a measurement of the network\u2019s robustness for a particular task.<\/p>\n
Similar evaluation techniques do exist but have not been able to scale up to more complex neural networks. Compared to those methods, the researchers\u2019 technique runs three orders of magnitude faster and can scale to more complex CNNs.<\/p>\n
The researchers evaluated the robustness of a CNN designed to classify images in the MNIST dataset of handwritten digits, which comprises 60,000 training images and 10,000 test images. The researchers found around 4 percent of test inputs can be perturbed slightly to generate adversarial examples that would lead the model to make an incorrect classification.<\/p>\n
\u201cAdversarial examples fool a neural network into making mistakes that a human wouldn\u2019t,\u201d says first author Vincent Tjeng, a graduate student in the Computer Science and Artificial Intelligence Laboratory (CSAIL). \u201cFor a given input, we want to determine whether it is possible to introduce small perturbations that would cause a neural network to produce a drastically different output than it usually would. In that way, we can evaluate how robust different neural networks are, finding at least one adversarial example similar to the input or guaranteeing that none exist for that input.\u201d<\/p>\n
Joining Tjeng on the paper are CSAIL graduate student Kai Xiao and Russ Tedrake, a CSAIL researcher and a professor in the Department of Electrical Engineering and Computer Science (EECS).<\/p>\n
CNNs process images through many computational layers containing units called neurons. For CNNs that classify images, the final layer consists of one neuron for each category. The CNN classifies an image based on the neuron with the highest output value. Consider a CNN designed to classify images into two categories: \u201ccat\u201d or \u201cdog.\u201d If it processes an image of a cat, the value for the \u201ccat\u201d classification neuron should be higher. An adversarial example occurs when a tiny modification to that image causes the \u201cdog\u201d classification neuron\u2019s value to be higher.<\/p>\n
The researchers\u2019 technique checks all possible modifications to each pixel of the image. Basically, if the CNN assigns the correct classification (\u201ccat\u201d) to each modified image, no adversarial examples exist for that image.<\/p>\n
Behind the technique is a modified version of \u201cmixed-integer programming,\u201d an optimization method where some of the variables are restricted to be integers. Essentially, mixed-integer programming is used to find a maximum of some objective function, given certain constraints on the variables, and can be designed to scale efficiently to evaluating the robustness of complex neural networks.<\/p>\n
The researchers set the limits allowing every pixel in each input image to be brightened or darkened by up to some set value. Given the limits, the modified image will still look remarkably similar to the original input image, meaning the CNN shouldn\u2019t be fooled. Mixed-integer programming is used to find the smallest possible modification to the pixels that could potentially cause a misclassification.<\/p>\n
The idea is that tweaking the pixels could cause the value of an incorrect classification to rise. If cat image was fed in to the pet-classifying CNN, for instance, the algorithm would keep perturbing the pixels to see if it can raise the value for the neuron corresponding to \u201cdog\u201d to be higher than that for \u201ccat.\u201d<\/p>\n
If the algorithm succeeds, it has found at least one adversarial example for the input image. The algorithm can continue tweaking pixels to find the minimum modification that was needed to cause that misclassification. The larger the minimum modification \u2014 called the \u201cminimum adversarial distortion\u201d \u2014 the more resistant the network is to adversarial examples. If, however, the correct classifying neuron fires for all different combinations of modified pixels, then the algorithm can guarantee that the image has no adversarial example.<\/p>\n
\u201cGiven one input image, we want to know if we can modify it in a way that it triggers an incorrect classification,\u201d Tjeng says. \u201cIf we can\u2019t, then we have a guarantee that we searched across the whole space of allowable modifications, and found that there is no perturbed version of the original image that is misclassified.\u201d<\/p>\n
In the end, this generates a percentage for how many input images have at least one adversarial example, and guarantees the remainder don\u2019t have any adversarial examples. In the real world, CNNs have many neurons and will train on massive datasets with dozens of different classifications, so the technique\u2019s scalability is critical, Tjeng says.<\/p>\n
\u201cAcross different networks designed for different tasks, it\u2019s important for CNNs to be robust against adversarial examples,\u201d he says. \u201cThe larger the fraction of test samples where we can prove that no adversarial example exists, the better the network should perform when exposed to perturbed inputs.\u201d<\/p>\n
\u201cProvable bounds on robustness are important as almost all [traditional] defense mechanisms could be broken again,\u201d says Matthias Hein, a professor of mathematics and computer science at Saarland University, who was not involved in the study but has tried the technique. \u201cWe used the exact verification framework to show that our networks are indeed robust \u2026 [and] made it also possible to verify them compared to normal training.\u201d<\/p>\n<\/div>\n
Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"