Sometimes the New Media Are Biased - Sometimes It's the Reader

Chicken or egg? JL

Jonathan Rothwell reports in the New York Times:

The bias consumers bring with them distorts their rating of news content, new research shows, and those who are most distrustful of the news media tend to be the most biased readers.The evidence also suggests that people are at greater risk of bias if they habitually turn to more extreme sources — such as those least often preferred by political moderates. Certain outlets create bias, perhaps by offering lower-quality content, though it’s just as plausible that more biased consumers gravitate toward the same news outlets.

Gallup survey data indicates that Americans are increasingly distrustful about potentially biased news. But they should also worry about the partiality of their own judgment as well as how their news consumption habits may affect it.

The bias consumers bring with them distorts their rating of news content, new research shows, and those who are most distrustful of the news media tend to be the most biased readers.

The evidence also suggests that people are at greater risk of bias if they habitually turn to more extreme sources — such as those least often preferred by political moderates.

How to study bias

Social scientists have devised a number of ways to research bias, which is notoriously hard to measure. In one famous study, the economists Claudia Goldin and Cecilia Rouse found that the share of women admitted to city orchestras increased substantially when evaluators were prevented from seeing the auditioning musicians and could judge based only on the quality of the performance.

The lesson from this and other studies of discrimination is that withholding irrelevant information can enhance judgment. Thus, the overwhelming consensus among scientists is that peer-review publication should be based on blind review.

News without labels

In this context, Knight Foundation partnered with Gallup in 2017 to create an experimental news platform, as part of a larger research initiative. The platform pulled news articles and other content from diverse media outlets and invited a random sample of Americans who had taken Gallup surveys to participate in rating the trustworthiness of the content.

Half the participants were not allowed to see the source of the news — only to read its content. The other half were allowed to see the source as they would on a typical website. A total of 3,081 people provided ratings of 1,645 different articles originally published by one of seven well-known sources.

The results, which have been published by Gallup and Knight, show that the blinded group is significantly more trusting of the news content. People identifying with the Republican Party who read media perceived as left-leaning like The New York Times and Vox without knowing where it came from rated it as more trustworthy than the nonblinded group did. Similarly, those identifying with the Democratic Party who read media perceived as right-leaning like Fox News rated it higher when they did not know the source.

This database also provides two novel measures: trust ratings of those articles by blind reviewers, who are significantly more impartial, and a comparison with the ratings of people with a normal news consumption experience that allows them to see the brand.

Consider that a reader’s trustworthy rating of a news article is the sum of the article’s inherent qualities, the reader’s personal views and brand prejudice. The experimental and control groups are alike, except that brand prejudice is removed from the experimental group. The difference in an article’s trustworthiness score between the two groups is equal to the brand prejudice — or bias — of the control group (those who can see the news source).

For those in that group, an individual’s bias was calculated for each article by taking the absolute value difference in trustworthiness ratings between his or her rating and the mean score for that article as provided by blind reviewers; the rating uses a scale of one to five, with increments of 0.5.

Who is at greatest risk of bias?

Among all readers in the group who could see the news source, 35 percent exhibit large bias — meaning their trust rating of an article diverged from the blind-review group by 1.5 points or more on the 1-to-5-point scale.

Not surprisingly, those with more extreme political views tend to provide more biased ratings of news. Those who described their political views as very liberal or very conservative exhibited large bias across 43 percent of the articles they rated, whereas those who described their views as moderate exhibited bias just 31 percent of the time. Likewise, those who leaned toward one party but did not fully identify with it exhibited about the same bias as the moderates.

The data also suggests that those who approve of President Trump rate news articles with more bias than those who disapprove of the president (39.2 percent versus 32.8 percent). However, Trump supporters tend to be less biased than those identifying as “very liberal.”Strikingly, those with the strongest distrust of the news media provide the most biased ratings. Respondents were asked: “In general, how much trust do you have in the mass media — such as newspapers, TV and radio — when it comes to reporting the news fully, accurately and fairly?”

Those who say they don’t trust the news media at all provided biased ratings 47 percent of the time, whereas those who trust the news media “a fair amount” provided ratings with large bias just 30 percent of the time.

Aside from the content provider, articles were classified by whether they were about politics, economics or science. Articles about politics generated significantly more bias than those about science and economics. If the article mentioned President Trump or Hillary Clinton, the bias was even more pronounced.

In contrast with political views, demographic characteristics of the reader, including gender, age, education and race/ethnicity, provided little explanatory power in explaining bias.

Chosen news sources shape perceptions

Another reason some people may demonstrate high levels of bias in reading the news is that they habitually consume highly biased news, distorting their frame of reference. The Knight-Gallup data provides some evidence for this. Respondents were asked: “Is there a news source that you trust to report the news fully, accurately, and fairly?” Those who responded “yes” were then asked to list the source.

There were very large differences in measured bias across the various news sources people regularly consume. Those who turn to Rush Limbaugh, Breitbart and Fox News tend to generate the most biased ratings of news content. Rush Limbaugh listeners demonstrated large rating bias in 52 percent of news content rated, and it was 50 percent for Breitbart readers. Fox News watchers showed large bias 45 percent of the time.

The viewers of MSNBC showed large bias in 38 percent of articles. Readers of The New York Times and listeners to NPR were close to the national average, with 36 percent and 34 percent exhibiting large bias.

The two news sources associated with the least biased consumers were The Wall Street Journal (26 percent) and PBS (14 percent). Those who habitually watch PBS rate articles from a diversity of sources almost as if they were blind reviewers, whereas consumers of Fox News — and to a lesser extent MSNBC — seem to give relatively more weight to brand rather than content when judging news articles.

One interpretation of these findings is that certain outlets create bias, perhaps by offering lower-quality content, though it’s just as plausible that more biased consumers gravitate toward the same news outlets.

Providing better-quality news is the challenge of journalists and their organizations. In a low-cost era of publishing when just about anyone can disseminate views, the bigger challenge may be how to educate and prepare the citizenry to seek out and identify high-quality information.