Author: Kim Martineau | MIT Quest for Intelligence<\/p>\n
From the “Mona Lisa” to the “Girl with a Pearl Earring,” some images linger in the mind long after others have faded. Ask an artist why, and you might hear some\u00a0generally-accepted principles for making memorable art. Now there\u2019s an easier way to learn: ask an artificial intelligence model to draw an example.\u00a0<\/p>\n
A new study using machine learning to generate images ranging from a memorable cheeseburger to a forgettable cup of coffee shows in close detail what makes a portrait or scene stand out. The images that human subjects in the study remembered best featured bright colors, simple backgrounds, and subjects that were centered prominently in the frame.\u00a0Results<\/a>\u00a0were presented this week at the\u00a0International Conference on Computer Vision<\/a>.\u00a0<\/p>\n \u201cA picture is worth a thousand words,\u201d says the study\u2019s co-senior author\u00a0Phillip Isola<\/a>, the Bonnie and Marty (1964) Tenenbaum CD Assistant Professor of Electrical Engineering and Computer Science at MIT.\u00a0\u201cA lot has been written\u00a0about memorability, but this method lets us actually visualize what memorability looks like. It gives us a visual definition for something that\u2019s hard to put into words.”<\/p>\n The work builds on an earlier model,\u00a0MemNet<\/a>, which rates the memorability of an image and highlights the features in the picture influencing its decision. MemNet\u2019s predictions are based on the results of an online study in which 60,000 images were shown to human subjects and ranked by how easily they were remembered.<\/strong><\/p>\n The model in the current study,\u00a0GANalyze<\/a>, uses a machine learning technique called generative adversarial networks, or GANs, to visualize a single image as it inches its way from “meh” to memorable. GANalyze lets viewers visualize the incremental transformation of, say, a blurry panda lost in the bamboo into a panda that dominates the frame, its black eyes, ears, and paws contrasting sharply and adorably with its white mug.<\/p>\n The image-riffing GAN has three modules. An assessor, based on MemNet, turns the memorability knob on a target image and calculates how to achieve the desired effect. A transformer executes its instructions, and a generator outputs the final image.\u00a0<\/p>\n The progression has the dramatic feel of a time-lapse image. A cheeseburger shifted to the far end of the memorability scale looks fatter, brighter, and, as the authors note, \u201ctastier,\u201d than its earlier incarnations. A ladybug looks shinier and more purposeful. In an unexpected twist, a pepper on the vine turns chameleon-like from green to red.\u00a0<\/p>\n The researchers also looked at which features influence memorability most. In online experiments, human subjects were shown images of varying memorability and asked to flag any repeats. The duplicates that were stickiest, it turns out, featured subjects closer up, making animals or objects in the frame appear larger. The next most important factors were brightness, having the subject centered in the frame, and in\u00a0a square or circular shape.<\/p>\n \u201cThe human brain evolved to focus most on these features, and that\u2019s what the GAN picks up on,\u201d says study co-author\u00a0Lore Goetschalckx<\/a>, a visiting graduate student from Katholieke Universiteit\u00a0Leuven in Belgium.<\/p>\n The researchers also reconfigured GANanalyze to generate images of varying aesthetic and emotional appeal. They found that images rated higher on aesthetic and emotional grounds were brighter, more colorful, and had a shallow depth of field that blurred the background, much like the most memorable pictures. However, the most aesthetic images were not always memorable.<\/p>\n GANalyze has a number of potential applications, the researchers say. It could be used to detect, and even treat, memory loss by enhancing objects in an augmented reality system.\u00a0<\/p>\n \u201cInstead of using a drug to enhance memory, you might enhance the world through an augmented-reality device to make easily misplaced items like keys stand out,\u201d says study co-senior author\u00a0Aude Oliva<\/a>, a principal research scientist at MIT\u2019s\u00a0Computer Science and Artificial Intelligence Laboratory<\/a> (CSAIL)\u00a0and executive director of the\u00a0MIT Quest for Intelligence.<\/a>\u00a0\u00a0<\/p>\n GANalyze could also be used to create unforgettable graphics to help readers retain information. \u201cIt could revolutionize education,\u201d says Oliva. Finally, GANs are already starting to be used\u00a0to generate synthetic, realistic images of the world to help train automated systems to recognize places and objects they are unlikely to encounter in real life.\u00a0<\/p>\n Generative models offer new, creative ways for humans and machines to collaborate. Study co-author\u00a0Alex Andonian<\/a>, a graduate student at MIT\u2019s\u00a0Department of Electrical Engineering and Computer Science<\/a>, says that’s why he\u00a0has chosen\u00a0to make them the focus of his PhD.<\/p>\n \u201cDesign software lets you adjust the brightness of an image, but not its overall memorability or aesthetic appeal \u2014 GANs let you do that,\u201d he says. \u201cWe\u2019re just starting to scratch the surface of what these models can do.\u201d\u00a0\u00a0\u00a0<\/p>\n The study was funded by the U.S. National Science Foundation.<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"