How Your Online Activity Became A De Facto Social Credit Score

There are no rules as to how collected data is used, analyzed by artificial intelligence or other methodologies - or how those analyses are interpreted, labelled - and re-sold. JL

Violet Blue reports in Engadget:

Airbnb's new "trait analyzer" algorithms compile data dossiers on users, decides whether you've been bad or good, gives you a score, and then "flag and investigate suspicious activity before it happens." AI combines data points and historical events,  race and location, digital footprints and crime statistics, to predict victims and perpetrators. Instagram, Facebook, YouTube, and Airbnb are deciding what legal behaviors are acceptable for service, and now groups of people being denied involvement in mainstream economic, political, cultural and social activities — at scale.

Kaylen Ward's Twitter fundraiser for the Australian bushfire relief has ended. The Los Angeles-based model said she raised $1 million (by comparison Jeff Bezos donated $690,000). At the start of Ms. Ward's successful donation drive she had three Instagram accounts — none of which were part of the campaign.

Despite that, Instagram kicked her off all three accounts, saying her behavior on Twitter violated Instagram's sexually suggestive content guidelines. On Twitter, Ms. Ward -- as The Naked Philanthropist -- offered a privately-sent nude photo to those who provided verifiable proof of donation to organizations including Australian Red Cross and The Koala Hospital. Her fundraiser complied with Twitter's Terms of Service.

If the thought of companies stalking you online and denying you services because they think you're a sinner gives you the Orwell Anti-Sex League chills, you should know that Airbnb just asked Instagram to hold its beer.

The same day Ms. Ward launched her fundraising campaign, reports emerged detailing Airbnb's new "trait analyzer" algorithms that compile data dossiers on users, decides whether you've been bad or good, gives you a score, and then "flag and investigate suspicious activity before it happens."

The Evening Standard reported on Airbnb's patent for AI that crawls and scrapes everything it can find on you, "including social media for traits such as 'conscientiousness and openness' against the usual credit and identity checks and what it describes as 'secure third-party databases'."

They added, "Traits such as "neuroticism and involvement in crimes" and "narcissism, Machiavellianism, or psychopathy" are "perceived as untrustworthy." Further:

It uses artificial intelligence to mark down those found to be "associated" with fake social network profiles, or those who have given any false details. The patent also suggests users are scored poorly if keywords, images or video associated with them are involved with drugs or alcohol, hate websites or organisations, or sex work.

It adds that people "involved in pornography" or who have "authored online content with negative language" will be marked down.

When reached for comment, Airbnb provided Engadget two different responses from two different people. The first was boilerplate, describing its "trait analyzer" patent that Airbnb sent to press: the company claimed it did not "necessarily implement" all or part of its patent filings.

Engadget also asked Airbnb for comment regarding its profiling users based on their offsite behaviors and its denial of services to customers who work in legal adult entertainment.

Airbnb's second response (via email) was to ask to speak off the record (on the phone). Engadget declined off-the-record comment. This Airbnb spokesperson emailed again, stating:

Regarding your question on sex work, we do not allow sex work in Airbnb listings and have policies in place to enforce this rule... We take action to remove accounts that we believe to be associated with sex trafficking and child exploitation regardless of whether the activty [SIC] is occuring [SIC] in Airbnb listings, and we also work cooperatively with law enforcement authorities in such cases.

Yeah, that wasn't our question on sex work. But the statement is revealing in that it looks like Airbnb is bending over backwards to not say it's indiscriminately discriminating against sex workers -- while it's becoming widely documented that Airbnb does exactly that.

"Airbnb for everyone" (not)

Adult performer Cadence Lux's Airbnb account was suddenly terminated this month. Last week she went to create a new account under her legal, non-performer name, yet Airbnb knew it was her. The company denied her an account, saying Lux's " information is associated with activities that pose a risk to the Airbnb community."

Guilt by association, certainly. But it's also the direct opposite of what Airbnb told local press when bragging about its profiling and surveillance tools this week.

It's no wonder The Observer recently reported that Sex Workers Are at the Forefront of the Fight Against Mass Surveillance and Big Tech. "Algorithms are affecting people, and disproportionately they're going to affect people at the margins," Analyst and researcher Bardot Smith said. "So queer people, people of color, sex working people. Obviously, the intersections of all these identities, and basically it comes down to people that they have determined don't deserve access to money and resources."

Needless to say, Lux was not using Airbnb for anything other than having a safe place to sleep. It's almost like everyone forgets porn is a legal job in the United States. Okay Airbnb, we get it! You like to refuse service to adult women who engage in consensual activities of bodily autonomy that have nothing to do with your service because you don't approve, or you don't believe women, or whatever. But hey, I digress.

They don't need to. According to Bloomberg, Silicon Valley startup Trooly began working with Airbnb in 2015; in 2017 Airbnb purchased Trooly's intellectual property and engineering team. Airbnb's email to Engadget confirmed that its problematic patent making headlines this month was part of the Trooly acquisition.

At the time, Trooly had two serious competitors in the AI-powered, big data "background check" space: UK startup Onfido and US-based Checkr.

Trooly was late to the game and raised less funding, but the three men who created it offered something TechCrunch described as "drastically different." Namely, scraping everything about you online, letting AI decide if you're good or bad, and applying predictive policing.

"The way we do that is we use public and legally permissible digital footprints," Trooly co-founder and CEO Savi Baveja told press in 2016. "In about 30 seconds — using very little input information about the individual or business — we return a scorecard that does three things: it verifies whether the input information is authentic; it screens for any relevant and seriously antisocial or pro-social prior behavior; and then it runs a series of predictive models on that footprint to say what is the propensity of this individual or small business for future antisocial or pro-social behavior."

The most famous social credit system in operation is that used by China's government. It "monitors millions of individuals' behavior (including social media and online shopping), determines how moral or immoral it is, and raises or lowers their "citizen score" accordingly," reported Atlantic in 2018.

Black Mirror: Nosedive (Season 3 / Netflix)

"Those with a high score are rewarded, while those with a low score are punished." Now we know the same AI systems are used for predictive policing to round up Muslim Uighurs and other minorities into concentration camps under the guise of preventing extremism.

Trooly — nee Airbnb — is combining social credit scores with predictive policing. Tools like PredPol use AI that combines data points and historical events, factors like race and location, digital footprints and crime statistics, to predict likelihood of when and where crimes will occur (as well as victims and perpetrators). It's no secret that predictive policing replicates and perpetuates discrimination.

Combine this with companies like Instagram, Facebook, YouTube, and yes, Airbnb deciding what legal behaviors are acceptable for service, and now we're looking at groups of historically marginalized people being denied involvement in mainstream economic, political, cultural and social activities — at scale.

This week AP reported that Facebook and Instagram are doing exactly that. "Activists, sex therapists, abuse survivors, artists and sex educators," are being unfairly censored by both services. "And it's no small matter for them. Artists can be suddenly left without their audience, businesses without access to their customers and vulnerable people without a support network ... it means that a company in Silicon Valley, whose online platforms have become not only our town squares but diaries, magazines, art galleries and protest platforms, gets final say on matters of free speech and self-expression."

The point is, we can write this all off as Instagram and Airbnb being shitty and sexist (spoiler: they are), and being a big bunch of hypocrites about stated values (also true). But also not actually caring about whom they harm and how they're trading our future freedoms and rights and chances at equality for their selfish entitlement to live out their lives of repressed and eventually volatile desperation in luxury (yep).

Or we can grab every chance we can to hold them to account.

Last year I marched in the 2019 SF Pride Parade with the Transgender Law Center. In early morning downtown San Francisco, we assembled on the same block as Airbnb — their float was across the street.

The sun was beginning its morning arc, and we were cold in the shadows of condo and office towers belonging to Salesforce, others occupied by Facebook, Google, and more. I live in the Castro; I'd looked at Airbnbs in my neighborhood for friends visiting for the celebration. The listings I got were monthly. Rentals that, to me, belong on the open, competitive market offering tenant-protecting agreements.

But I was thinking about a queer friend of mine who is a sex worker, recently driven out of SF by the high rents and scarcity. One year after Airbnb started working with Trooly she was banned from Airbnb after three years of stellar reviews from hosts. She is a sex worker and queer activist, and had never done sex work in any Airbnb. Airbnb did not respond to requests for comment on her expulsion.

I was preparing for the Pride march, my group was making sure everyone had sunscreen, taking selfies, and not gonna lie, in today's political climate we were low-key stressed someone would try and pull a mass-shooter situation. We were there because it mattered more than all the things any shooter might believe in.

I looked at the Airbnb float and stuck a pin in it.

I realized that by Airbnb's presence in San Francisco Pride and doing old-school discrimination against queer sex workers, that Airbnb didn't know the ground they walked on in a terrible way. Airbnb's presence in SF Pride was either harmfully ignorant or disgustingly repudiating the fact that sex workers were a critical part of Stonewall and Compton. The two riots indisputably critical to the advancement of LGBTQ rights leading to the fact that there are any Pride parades in the US at all.

So this year I'm going to find Airbnb's float and ask: which is it?