How Facebook's Master Algorithm Powers the Social Network

The very technological and algorithmic forces that make Facebook so attractive are also what make it so easily manipulated and potentially malign. JL


Christopher Mims reports in the Wall Street Journal:

Every time one of Facebook’s two billion monthly users opens the app, a personalization algorithm sorts all the posts that a person could theoretically see, and dishes up the fraction it thinks she or he would like to see first. The system autonomously scans links, suppressing ones that match what it learned from human-generated data. Facebook’s master algorithm now can extract meaning from our posts and photos; capturing 50 minutes of every American’s day, up from 40 minutes.
Instagram engineers faced a Herculean task in early 2016. Fearing that people would miss the most important posts, Instagram’s leadership asked the engineers to transform the chronological photo feed into a curated list of posts based on users’ individual preferences.

Development of a similar algorithm for Facebook ’s News Feed, which determines what 2 billion Facebook users see, required an enormous investment of time by some of the world’s most highly compensated engineers.
At Instagram, three or four engineers got the job done in less than five weeks, says Joaquin Candela, Facebook’s head of applied machine learning. The team was able to clone the existing News Feed algorithm, then tweak it to suit Instagram.
However much Instagram’s engineers tweaked it, the fact that most of what powers Instagram came straight from Facebook’s News Feed shows the dominance and success of this basic engine of social media. Think of it—and the endless, modular chunks of AI that go into it—as Facebook’s master algorithm (my words, not Facebook’s).
Mr. Candela recently explained how his team works with or creates tools for almost every other engineering team at the company—and how AI has become core to most things Facebook does.
If telling us what to look at next is Facebook’s raison d’être, then the AI that enables that endless spoon-feeding of content is the company’s most important, and sometimes most controversial, intellectual property. A sorted, curated feed tuned for engagement is the product of a device that may someday be viewed by historians as a milestone on par with the steam engine.
Only this engine, built to capture human attention, has shown itself to be exploitable by bad actors and possibly detrimental to our democracy, even when it is functioning as advertised. This has prompted congressional hearings for Facebook and other tech companies, scheduled for November. Facebook has been a vessel for Russian influence and the spread of fake news, and a potential cause for envy and unhappiness. The personalization of content that Facebook’s master algorithm allows, and the hyperpartisan news sites that have risen to feed it, have created, for many users, personalized “filter bubbles” of what is essentially nonoverlapping reality.
At the same time, the company’s announcement that it is hiring more humans to screen ads and filter content shows there is so much essential to Facebook’s functionality that AI alone can’t accomplish.
AI algorithms are inherently black boxes whose workings can be next to impossible to understand—even by many Facebook engineers. “If you look at all the engineers at Facebook, more than one in four are users of our AI platform,” says Mr. Candela. “But more than 70% [of those] aren’t experts.”
How so many Facebook engineers can use its AI algorithms without necessarily knowing how to build them, Mr. Candela says, is that the system is “a very modular layered cake where you can plug in at any level you want.” He adds, “The power of this is just hard to describe.” Pieces of that platform are performing all kinds of “domain-specific” tasks across Facebook’s properties, from translation to speech recognition.

Information Butler, or Time Vampire?

Every time one of Facebook’s two billion monthly users opens the Facebook app, a personalization algorithm sorts through all the posts that a person could theoretically see, and dishes up the fraction it thinks she or he would like to see first. The system weighs hundreds of frequently updated signals, says Mr. Candela. Without AI, many of these signals would be impossible to analyze.

An example of updated signals would be Facebook’s recent fight against clickbait—links to stories that are “misleading, sensational or spammy.” Training the algorithm takes human labor: A team analyzed hundreds of thousands of posts in 10 languages, flagging offending headlines that either withheld information (“Here’s the one thing…”) or exaggerated (“…will blow your mind”). The resulting system autonomously scans links, suppressing the ones that match what it learned from the human-generated data.
Facebook’s master algorithm now also can extract additional meaning from our posts and photos, Mr. Candela says.
The “recommendations” feature, for instance, allows a person to ask what to check out on a trip to Barcelona. Because Facebook’s AI actually “knows” what La Sagrada Familia is and where it is located, anyone who recommends it in a comment will see it pop up on a map above the post.
These capabilities are versatile enough that Facebook users have repurposed them in unexpected ways, Mr. Candela says. When Hurricane Irma hit Florida, people used Facebook to build a map of stores with bottled water for sale. The person who created the post activated the “recommendations” feature; others added to it simply by commenting on the post with retailers’ addresses.
Mr. Candela says teams add new features to Facebook’s master algorithm to “add value to social interactions.” Chief Executive Mark Zuckerberg recently said the company’s goal was to “bring the world closer together.”
However it is phrased, it is measured in the way people engage with Facebook’s apps and networks, whether that is increasing the number of posts they like or comment on, or how useful they find machine-translated posts, or how often they use M, Facebook’s Messenger-based smart assistant, Mr. Candela says.
Time spent on Facebook’s various properties correlates with the company’s revenue, and that number was going up at last report: In April 2016, Facebook said it was capturing on average 50 minutes of every American’s day, up from 40 minutes in July 2014.
The unstated assumption behind the work of Facebook’s more than 20,000 employees is that getting people to use Facebook more is a good thing. It is certainly hard to imagine a world without it, given how it has become central to the way we connect, find news and keep up with friends and family.
But given what we have learned over the past year, it is worth asking whether the intentions of the hugely powerful Oz that is Facebook’s master algorithm are ultimately benevolent or malign.