How AI-Curated Photos Are Shaping Humans' Memory - Whether They Want That Or Not

Algorithms designed by corporations have the power to shape how photos are edited and presented.

And photos have the power to shape our view of the world in which we live as well as our memories of what has transpired.

Are we really ok with that? JL

Farhad Manjoo reports in the New York Times:

A.I.-curated memories are shaping our narratives about ourselves. History is about who tells your story. The machines, now, are increasingly making sense of our human world — shaping our reality in the deepest way possible, and like cameras themselves, they’re inescapable. Our memories are profoundly altered by pictures. One study has shown that taking photos reduces our ability to recall events in the world around us. Photos also shape our sense of selves, even to the point of creating new memories — a false photo can convince you that something happened to you even if it never did.

The first time Google Photos made me cry, it was with a sucker punch.

I had looked at my phone one morning in April, expecting more news of global woe. Instead there was an alert from Photos, letting me know that Google’s image-processing robots had created some kind of montage from my videos. I had seen such A.I.-produced clips before — Facebook’s tone-deaf year-in-review montages are a recurring blight — so I was not expecting much. Then I pressed Play and within 30 seconds, I was a crumpled, weepy wreck.

The montage was of my 5-year-old daughter, Samara, whose nearly every waking moment has been thoroughly and permanently memorialized by me, her camera-obsessed father. My obsession has created an archival nightmare; videos and pictures of Samara and her older brother, Khalil, both born in the time of smartphones, now span several terabytes — more images than any human might ever have time to meaningfully review. What, one might ask, was the point of capturing all these moments?

Now in this single two-minute montage, Google Photos had given me a glimpse of the answer.

Google’s computers can recognize faces, even as they age over time. Photos also seems to understand the tone and emotional valence of human interaction, things like smiles, giggles, frowns, tantrums, dances of joy and even snippets of dialogue like “happy birthday!” or “good job!” The resulting montage, synced to a swelling Hollywood score, mixed obvious highlights — birthdays, school plays — with dozens of ordinary moments of childhood bliss.

There was baby Samara getting a haircut, taking a few unsteady steps; toddler Samara playing with her brother, fighting with her brother, taking a brave dip underwater in swim class; preschool Samara eating pizza on a road trip, sassing the camera with her tongue out. I can’t post the video here; it feels like showing you her diary. But if Samara ever runs for kindergarten class president, the Google montage could be her “Man From Hope” and she would win in a landslide.

This is what I mean about a sucker punch: Who expects software to make them cry? Images on Instagram and Snapchat may move you regularly, but Google Photos is not social media; it is personal media, a service begun three years ago primarily as a database to house our growing collections of private snaps — and a service run mostly by machines, not by other humans posting and Liking stuff.

And yet Google Photos has become one of the most emotionally resonant pieces of technology I regularly use. It is remarkable not just for how useful it is — for how it has erased any headache in storing and searching through the tsunami of images we all produce. More than that, Photos is remarkable for what it portends about how we may one day understand ourselves through photography.

With its heavy focus on artificially intelligent curation, Google Photos suggests the dawning of a new age of personalized robot historian. The trillions of images we are all snapping will become the raw material for algorithms that will curate memories and construct narratives about our most intimate human experiences. In the future, the robots will know everything about us — and they will tell our stories.

But we’re getting ahead of ourselves. Before fretting about the sci-fi tomorrow, it’s worth marveling at the basic utility of Google Photos today. Tech companies have been trying to create ways to manage digital photos ever since we began ditching film. Most efforts have failed; the better our portable cameras get, the more photos we take, and the more photos we take, the less able we all are to make sense of the stash.

“With the invention of the smartphone, there was nothing that humans did, absolutely nothing, that they didn’t also make an image of,” said Martin Hand, a sociologist at Queen’s University in Kingston, Ontario, and the author of “Ubiquitous Photography,” an academic inquiry into the blessed problem of too many photos. “But that brought about its own problems — it started to get overwhelming.”

More than a decade ago, the tech world hit on one partial solution to picture overload: Make images social. Through services like Flickr, then Facebook and Instagram, we tried to curate our images by getting others to do it for us. The best photos of you were the ones that were ranked highly in your social feed; the worst were the ones you didn’t post.

But social media created another set of problems — there was a fear of missing out, a sense of performative anxiety, loneliness and an erosion of privacy. “There was a sense in which because everything was public, young people had to constantly curate the idea of themselves in public,” said Mr. Hand.

Google, too, tried to play the social photo game. The earliest incarnation of Google Photos was part of Google Plus, the search company’s ill-fated, just-shuttered social network. A few years ago, after realizing that social networking was not its forte, Google went back to the drawing board with Photos.

Its reimagined service would do three things: Offer nearly limitless storage for your photos essentially free (you can pay more to have your images stored in higher-resolution sizes). It put them in the cloud, so they could be accessed anywhere. And, crucially, Photos would lean on Google’s famed A.I. to address what it saw as the key problem of the smartphone era — the fact that we all take photos but rarely look at them.

“We noticed that you would never relive or reminisce about any of these moments,” said Anil Sabharwal, the Google vice president who led the team that built Photos, and still runs it. “You would go on this beautiful vacation, you’d take hundreds of beautiful photos, years would pass, and you would never look at any of them.”

When it started in 2015, Google Photos brought immediate relief. For instance, face recognition made sharing pictures automatic. Now, when I take a photo of my kids, Google recognizes them and shares those photos with my wife; her photos are shared with me. Incredibly, instantly, without thinking, we each have a complete collection of the children’s photos, and any anxiety about keeping them secure has vanished.

Then there are Google’s daily prompts to reminisce. It’s difficult to overstate how good Google’s machines are at mining your collection to find new stuff to awe you. In one series, called Then and Now, it will find pictures of the same person, or groups of people, in similar poses at two different time periods: Your children on the first day of school this year versus last year, or you in front of the Empire State Building 10 years ago and today.

Last month, Google released a new home device, the Home Hub, a voice-activated gadget whose screen shows a never-ending slide show of this sort of nostalgia bait. It’s magical. I’ve had the Home Hub in my office for more than a week and it has deeply altered how I experience my photos. They have come alive.

And yet as much as I can’t quit Photos, I’m also vaguely terrified by what it promises about the future. There’s a raft of social science research that shows our memories are profoundly altered by pictures. One study has shown that mindlessly taking photos reduces our ability to recall events in the world around us. Photos also shape our sense of selves, even to the point of creating new memories — a false photo can convince you that something happened to you even if it never did.

I worry, given all this, about how A.I.-curated memories are shaping our narratives about ourselves. I think of Samara and children like her: how she will one day watch videos like the one Google produced of her, and she will come to certain conclusions about her childhood only because a for-profit, ad-supported tech company’s machines made choices about what sort of scenes to show and what to hide.

At the moment, there is no calamity here: Google Photos’ videos are sunny and bright. But if history is about who tells your story, Photos pushes us into a new realm.

The machines, now, are increasingly making sense of our human world — shaping our reality in the deepest way possible, and like cameras themselves, they’re inescapable.