Facebook's New AI Can Paint, But Google's Knows How to Party

Neural networks are moving beyond recognition to creation. Which is going to make them better at what they do. The question is where that leads. JL

Cade Metz reports in Wired:

They can recognize the spoken word, translate from one language to another, target ads, or teach a robot to screw a cap onto a bottle. You can teach them not just to recognize images, but create images. These projects improv(e) neural networks, moving them closer to human-like intelligence. This helps visualize what networks are actually learning.
Facebook and Google are building enormous neural networks —artificial brains—that can instantly recognize faces, cars, buildings, and other objects in digital photos. But that’s not all these brains can do.
They can recognize the spoken word, translate from one language to another, target ads, or teach a robot to screw a cap onto a bottle. And if you turn these brains upside down, you can teach them not just to recognize images, but create images—in rather intriguing (and sometimes disturbing) ways.
Facebook is teaching its neural networks to automatically create small images of things like airplanes, automobiles, and animals, and about 40 percent of the time, these images can fool us humans into believing we’re looking at reality. “The model can tell the difference between an unnatural image—white noise you’d see on your TV or some sort of abstract art image—and an image that you would take on your camera,” says Facebook artificial intelligence researcher Rob Fergus. “It understands the structure of how images work” (see images above).
Meanwhile, the boffins at Google have taken things to the other extreme, using neural nets to turn real photos into something intriguingly unreal. They’re teaching machines to look for familiar patterns in a photo, enhance those patterns, and then repeat the process with the same image. “This creates a feedback loop: if a cloud looks a little bit like a bird, the network will make it look more like a bird,” Google says in a blog post explaining the project. “This in turn will make the network recognize the bird even more strongly on the next pass and so forth, until a highly detailed bird appears, seemingly out of nowhere.” The result is a kind of machine-generated abstract art (see below).

Click to Open Overlay Gallery

Google’s neural networks can see the rough outline of tower in a horizon and then enhance that outline until a tower appears in full. Google

On one level, these are party tricks—particularly Google’s feedback loop, which evokes hallucinatory flashbacks. And it should be noted that Facebook’s fake images are only 64-by-64 pixels. But on another level, these projects serve as ways of improving neural networks, moving them closer to human-like intelligence. This work, says David Luan, the CEO of a computer vision company called Dextro, “helps better visualize what our networks are actually learning.”
They’re also slightly disturbing—and not just because Google’s images feel like a drug trip gone wrong, crossing breeding birds with camels in some cases, or snails with pigs (see below). More than this, they hint at a world where we don’t realize when machines are controlling what we see and hear, where the real is indistinguishable from the unreal.

Click to Open Overlay Gallery

Google

Fooled Again

Working alongside a PhD student at New York University’s Courant Institute of Mathematical Sciences, Fergus and two other Facebook researchers revealed their “generative image model” work on Friday with a paper published to research repository arXiv.org. This system uses not one but two neural networks, pitting the pair against each other. One network is built to recognize natural images, and the other does its best to fool the first.
Yann LeCun, who heads Facebook’s 18-month-old AI lab, calls this adversarial training. “They play against each other,” he says of the two networks. “One is trying to fool the other. And the other is trying to detect when it is being fooled.” The result is a system that produces pretty realistic images.
According to LeCun and Fergus, this kind of thing could help restore real photos that have degraded in some way. “You can bring an image back to the space of natural images,” Fergus says. But the larger point, they add, is that the system takes another step towards what’s called “unsupervised machine learning.” In other words, it can help machines learn without human researchers providing explicit guidance along the way.
Eventually, LeCun says, you can use this model to train an image recognition system using a set of example images that are “unlabeled”—meaning no human has gone through and tagged them with text that identifies what’s in them. “Machines can learn the structure of an image without being told what’s in the image,” he says.
Luan points out that the current system still requires some supervision. But he calls Facebook’s paper “neat work,” and like the work being done at Google, he believes, it can help us understand how neural networks behave.

Layers Within Layers

Neural networks of the kind created by Facebook and Google span many “layers” of artificial neurons, each working in concert. Though these neurons perform certain tasks remarkably well, we don’t quite understand why. “One of the challenges of neural networks is understanding what exactly goes on at each layer,” Google says in its blog post (the company declined to discuss its image generation work further).

Click to Open Overlay Gallery

Google

By turning its neural networks upside-down and teaching them to generate images, Google explains, it can better understand how they operate. Google is asking its networks to amplify what it finds in an image. Sometimes, they just amplify the edges of a shape. Other times, they amplify more complex things, like the outline of a tower in a horizon, a building in a tree, or who’s knows what in a sea of random noise (see above). But in each case, researchers can better see what the network is seeing.
“This technique gives us a qualitative sense of the level of abstraction that a particular layer has achieved in its understanding of images,” Google says. It helps researchers “visualize how neural networks are able to carry out difficult classification tasks, improve network architecture, and check what the network has learned during training.”
Plus, like Facebook’s work, it’s kinda cool, a little strange, and a tad frightening. The better computers get at recognizing what’s real, it seems, the harder it gets for us.