Twitter Has Suspended 70 Million Fake or Suspicious Accounts - in Two Months

The interesting question is whether this will enhance its credibility - or simply reduce advertiser and user interest. JL

Craig Timberg and Elizabeth Dwoskin report in The Washington Post:

Twitter has sharply escalated its battle against fake and suspicious accounts, suspending more than 1 million a day in recent months. Twitter suspended more than 70 million accounts in May and June, and the pace has continued in July.The aggressive removal of unwanted accounts may result in a rare decline in the number of monthly users in the second quarter.

Twitter has sharply escalated its battle against fake and suspicious accounts, suspending more than 1 million a day in recent months, a major shift to lessen the flow of disinformation on the platform, according to data obtained by The Washington Post.

The rate of account suspensions, which Twitter confirmed to The Post, has more than doubled since October, when the company revealed under congressional pressure how Russia used fake accounts to interfere in the U.S. presidential election. Twitter suspended more than 70 million accounts in May and June, and the pace has continued in July, according to the data.

The aggressive removal of unwanted accounts may result in a rare decline in the number of monthly users in the second quarter, which ended June 30, according to a person familiar with the situation who was not authorized to speak. Twitter declined to comment on a possible decline in its user base.

Twitter’s growing campaign against bots and trolls — coming despite the risk to the company’s user growth — is part of the continuing fallout from Russia’s disinformation offensive during the 2016 presidential campaign, when a St. Petersburg-based troll factory was able to use some of America’s most prominent technology platforms to deceive voters on a mass scale to exacerbate social and political tensions.

The extent of account suspensions, which has not previously been reported, is one of several recent moves by Twitter to limit the influence of people it says are abusing its platform. The changes, which were the subject of internal debate, reflect a philosophical shift for Twitter. Its executives long resisted policing misbehavior more aggressively, for a time even referring to themselves as “the free speech wing of the free speech party.”

Del Harvey, Twitter’s vice president for trust and safety, said in an interview that the company is changing the calculus between promoting public discourse and preserving safety. She added that Twitter only recently was able to dedicate the resources and develop the technical capabilities to target malicious behavior in this way.

“One of the biggest shifts is in how we think about balancing free expression versus the potential for free expression to chill someone else’s speech,” Harvey said. “Free expression doesn’t really mean much if people don’t feel safe.”

But Twitter’s increased suspensions also throw into question its estimate that fewer than 5 percent of its active users are fake or involved in spam, and that fewer than 8.5 percent use automation tools that characterize the accounts as bots. (A fake account can also be one that engages in malicious behavior and is operated by a real person. Many legitimate accounts are bots, such as to report weather or seismic activity.)

Harvey said the crackdown has not had “a ton of impact” on the numbers of active user — which stood at 336 million at the end of the first quarter — because many of the problematic accounts were not tweeting regularly. But moving more aggressively against suspicious accounts has helped the platform better protect users from manipulation and abuse, she said.

Legitimate human users — the only ones capable of responding to the advertising that is the main source of revenue for the company — are central to Twitter’s stock price and broader perceptions of a company that has struggled to generate profits.

Independent researchers and some investors long have criticized the company for not acting more aggressively to address what many considered a rampant problem with bots, trolls and other accounts used to amplify disinformation. Though some go dormant for years at a time, the most active of these accounts tweet hundreds of times a day with the help of automation software, a tactic that can drown out authentic voices and warp online political discourse, critics say.

“I wish Twitter had been more proactive sooner,” said Sen. Mark R. Warner (Va.), the top ranking Democrat on the Senate Intelligence Committee. “I’m glad that — after months of focus on this issue — Twitter appears to be cracking down on the use of bots and other fake accounts, though there is still much work to do.”

[Russian operatives used Twitter and Facebook to target veterans and military personnel, study says]

The decision to forcefully target suspicious accounts followed a pitched battle within Twitter last year over whether to implement new detection tools. One previously undisclosed effort called “Operation Megaphone” involved quietly buying fake accounts and seeking to detect connections among them, said two people familiar with internal deliberations. They spoke on the condition of anonymity to share details of private conversations.

The name of the operation referred to the virtual megaphones — such as fake accounts and automation — that abusers of Twitter’s platforms use to drown out other voices. The program, also known as a white hat operation, was part of a broader plan to get the company to treat disinformation campaigns by governments differently than it did more traditional problems such as spam, which is aimed at tricking individual users as opposed to shaping the political climate in an entire country, according to these people. Harvey said she had not heard of the operation.

Some executives initially were reluctant to act aggressively against suspected fake accounts and raised questions about the legality of doing so, said the people familiar with internal company debates. In November, one frustrated engineer sought to illustrate the severity of the problem by buying thousands of fake followers for a Twitter manager, said two people familiar with the episode. Bots can be readily purchased on a gray market of websites.

A person with access to one of Twitter’s “Firehose” products, which organizations buy to track tweets and social media metrics, provided the data to The Post. The Firehose reports what accounts have been suspended and unsuspended, along with data on individual tweets.

Bots, trolls and fake accounts are nearly as old as Twitter, which started operations in 2006. In 2015, Twitter’s then-chief executive Dick Costolo acknowledged the problem in a company memo: “We suck at dealing with abuse and trolls on the platform and we've sucked at it for years.”

Twitter was not alone among tech companies in failing to adequately anticipate and combat Russian disinformation, which intelligence agencies concluded was part of the Kremlin’s attempt to help elect Republican Donald Trump, damage Democrat Hillary Clinton and undermine the faith of Americans in their political system.

The aftermath of the election — and the dawning realization of the key role unwittingly played by U.S. tech companies — threw some of the industry’s biggest players into crises from which they have not entirely emerged, while subjecting them to unprecedented scrutiny. Political leaders have demanded that Silicon Valley do better in the 2018 midterm elections despite a lack of new laws or clear federal guidance on how to crack down on disinformation without impinging on constitutional guarantees of free speech.

Twitter had said in several public statements this year that it was targeting suspicious accounts, including in a recent blog post that nearly 10 million accounts a week were being “challenged” — a step that attempts to ascertain the authenticity of an account’s ownership and requires users to respond to a prompt such as verifying a phone or email address.

In March, Twitter CEO Jack Dorsey announced a companywide initiative to promote “healthy conversations” on the platform. In May, Twitter announced major changes to the algorithms it uses to police bad behavior. Twitter is expected to make another announcement related to this initiative next week.

But researchers have for years complained that the problem is far more serious and that Twitter’s definition of a fake account is too narrow, allowing them to keep counts low. Several independent projects also have followed particular bots and fake accounts over many years, and even after the recent crackdown, researchers point to accounts with obviously suspicious behaviors, such as gaining thousands of followers in just a few days or tweeting around the clock.

“When you have an account tweeting over a thousand times a day, there’s no question that it’s a bot,” said Samuel C. Woolley, research director of the Digital Intelligence Lab at the Institute for the Future, a Palo Alto, Calif.-based think tank. “Twitter has to be doing more to prevent the amplification and suppression of political ideas.”

Several people familiar with internal deliberations at Twitter say the recent changes were driven by political pressure from Congress in the wake of revelations about manipulation by a Russian troll factory, which Twitter said controlled more than 3,000 Twitter accounts around the time of the 2016 presidential election. Another 50,258 automated accounts were connected to the Russian government, the company found.

[Russian content on Facebook, Google and Twitter reached far more users than companies first disclosed, congressional testimony says]

News reports about the severity of the bot problem and a rethinking of Twitter’s role in promoting online conversation also factored into Twitter’s more aggressive stance, these people said.

During congressional hearings last fall, lawmaker questions forced Twitter to look harder at its bot and troll problem, according to several people at the company. It also revealed gaps in what the company had done so far — and limits on the tools at the company’s disposal in responding to official inquiries.

Twitter launched an internal task force to look into accounts run by the Russian troll factory, called the Internet Research Agency (IRA), and received data from Facebook and other sources, including a threat database known as QIntel, according to two people familiar with the company’s processes.

One major discovery was the relationship between the Russian accounts and Twitter’s long-standing spam problems, the people said. Many of the accounts used by Russian operatives, the company researchers found, were not actually created by the IRA. Instead, the IRA had purchased bots that already existed and were being sold on a black market. Older accounts are more pricey than newly created ones because they are more likely to get through Twitter’s spam filters, said Jonathon Morgan, chief executive of New Knowledge, a start-up focused on helping Internet companies fight disinformation.

The discovery of the connection between the Russian bots and the spam problem led company officials to argue for a bigger crackdown, according to the people familiar with the situation. An internal battle ensued over whether the company’s traditional approach to spam would work in combating disinformation campaigns organized and run by nation-states such as Russia.

[Twitter finds hundreds of accounts tied to Russian operatives]

Rather than merely assessing the content of individual tweets, the company began studying thousands of behavioral signals, such as whether users tweet at large numbers of accounts they don't follow, how often they are blocked by people they interact with, whether they have created many accounts from a single IP address, or whether they follow other accounts that are tagged as spam or bots.

Sometimes the company suspends the accounts. But Twitter also limits the reach of certain tweets by placing them lower in the stream of messages, sometimes referred to as “shadow banning,” because the user may not know they are being demoted.

Harvey said the effort built on the technical expertise of an artificial intelligence start-up called Magic Pony that the company acquired in 2016. The acquisition “laid the groundwork that allowed us to get more aggressive,” Harvey said. “Before that, we had this blunt hammer of your account is suspended, or it wasn’t.”

The data obtained by The Post shows a steady flow of suspensions and spikes on particular days, such as Dec. 7, when 1.2 million accounts were suspended, nearly 50 percent higher than the average for that month. There was also a pronounced increase in mid-May, when Twitter suspended more than 13 million in a single week — 60 percent more than the pace in the rest of that month.

Harvey said the company was planning to go further in the year ahead. “We have to keep observing what the newest vectors are, and changing our ways to counter those,” she said. “This doesn’t mean we’re going to sit on our laurels.”