How Data Can Predict Which Cops Might Commit Misconduct

There are data available and they can predict, inexorably, where the problems may lie. The challenge, as in most organizations, is whether management really has the will - and the courage - to follow where the data lead. JL

Rob Arthur reports in 538:

The data show that officers who rack up many complaints are more likely to end up having a complaint sustained, suggesting they really are bad cops. Officers with a history of one or two complaints in 2011-13 have a 30 percent to 37 percent chance of receiving a complaint in the following two years. Repeaters — those with 15 or 20 incidents — are certain to have a complaint against them in 2014-15. There is a clear signal in the data of the bad actors and, whether they are going to commit misconduct.

Protests against police violence raged on in Chicago, with some demonstrators demanding Mayor Rahm Emanuel’s resignation over the shooting of 17-year-old Laquan McDonald by a police officer who is now charged with his murder. The charges came more than a year after officer Jason Van Dyke shot McDonald 16 times, drawing scrutiny of the department’s handling of the case and of police misconduct in Chicago more broadly.
At the heart of the reaction to the McDonald case are the questions of whether officers like Van Dyke are the exception or the rule, and how much responsibility the police department has to identify and discipline violent officers, as well as those who commit other types of misconduct.

The “few bad apples” theory of police violence posits that a small portion of the police force is ill-intentioned or inclined to misconduct or violence, while the majority of officers are good cops. Until recently, this theory was difficult for civilians to investigate, but department data on complaints against officers obtained through a legal challenge shows that police misconduct in Chicago is overwhelmingly the product of a small fraction of officers and that it may be possible to identify those officers and reduce misconduct.
This far-reaching data set, a product of the nonprofit Invisible Institute’s Citizens Police Data Project, comprehensively covers nearly five years of complaints against Chicago police officers. Each of the 28,588 records in the database offers a detailed account of the incident, including information on the accused officer, the complainant, the type of alleged misconduct, and whether the complaint was found legitimate by an internal investigation. The data is broken into subcategories depending on the specific allegation, but here are the most common types of complaints. (Instances of excessive force, like that alleged in the McDonald case, make up about two-thirds of the category “Arrest & Lock-up.”)

This data wasn’t released without a fight. Jamie Kalven, writer and founder of the Invisible Institute, spent years investigating police misconduct in Chicago public housing projects but was frustrated by the department’s failure to release any information on the subject — not even its own records of complaints against officers. In collaboration with the University of Chicago’s Mandel Legal Aid Clinic, Kalven pursued lawsuits against the department, first forcing the city to release lists of the officers with several complaints against them. The Invisible Institute then filed a series of Freedom of Information Act requests to obtain all complaints against police officers from Jan. 1, 2011, through Dec. 7, 2015. (Efforts to compel the release of still more complaint data are ongoing but are at risk because of legal action by the Fraternal Order of Police.)
The extensive catalog of complaints against officers appears to bear out the theory of a few bad apples: Among the 7,758 police officers who received a complaint during that time period, more than half received less than one per year (officers with zero complaints do not appear in the database). Meanwhile, the bad apples seem to be the ones racking up the grievances.

To avoid the overworked “bad apple” metaphor, the Invisible Institute prefers to call officers with many complaints against them “repeaters.” Repeaters only make up a small fraction of the more than 12,000 officers on Chicago’s force — perhaps 1 percent to 10 percent of the officers in the database, depending on where you draw the line — but are responsible for a huge fraction of the complaints: 10 percent of the officers who had received complaints generated 30 percent of the total departmental complaints since 2011. The 10 individual repeaters with the most complaints in the past five years averaged 23.4 complaints against them in that span.
In addition to the outsize number of complaints, Kalven said, repeaters have broader, cultural impacts, both within and outside of the department. Within the department, repeaters might normalize misconduct toward residents, pushing other cops toward wrongdoing. Outside of the department, the behavior of these officers can turn the community against the police. “That’s what the ‘few bad apples’ theory doesn’t capture: the kind of compounding, metastasizing arms that flow from the impunity of bad cops,” Kalven said.
The Chicago Police Department declined to comment on this article, saying in an email that it could not address studies provided by outside sources. But the paradigm of repeaters is promising, in a way. It suggests that if we could identify the bad actors within a police department and remove them from the force, the overall performance of the department would improve. Not only would dismissing these officers trigger a significant reduction in complaints and misconduct, the thinking goes, it could spark a cultural shift toward more responsible policing.
A data-driven mechanism to reduce police misconduct would be extremely valuable to the Chicago Police Department and the city of Chicago. Even laying aside the moral imperative to prevent abuse, the financial cost of police misconduct to the cash-strapped city is immense. Direct costs, in terms of legal fees and the funds disbursed in settlements, exceeded $500 million over 10 years, according to a Better Government Association study. The McDonald case alone was settled for $5 million.
Van Dyke had at least 20 complaints against him in his career, many alleging excessive force, before McDonald’s shooting. Overall, the data shows that officers who rack up many complaints against them are more likely to end up having a complaint sustained, suggesting that they really are bad cops.
In my analysis of the Invisible Institute data, I found that the number of complaints an officer receives in a certain year predicts whether and how many complaints he or she will have in the following year.¹ Over multiple years, the signal becomes even stronger. Officers with a baseline history of one or two complaints in 2011-13 have a 30 percent to 37 percent chance of receiving a complaint in the following two years.² But repeaters — those with 15 or 20 incidents in the first part of the data set — are almost certain to have a complaint against them in 2014-15.

Complaints predict future complaints

COMPLAINTS IN 2011-13	PROBABILITY OF A COMPLAINT IN 2014-15
1	30%
2	37
3	44
4	52
5	60
6	67
7	71
8	78
9	81
10+	96

Spokesmen for the police have explained the high complaint totals of repeaters as a consequence of the “bad neighborhoods” to which some officers are assigned. The logic goes that under constant harassment and threat of violence, police may behave more aggressively (albeit still within the boundaries of the rules) and therefore be accused more frequently of misconduct. After considering the beat an officer was patrolling, I found that some neighborhoods did see an increased number of complaints. Even after controlling for neighborhood, however, individual officers with more complaints in 2011-13 remained more likely to have complaints filed against them in 2014-15.
The kinds of allegations made against a given officer often follow a particular pattern as well. For example, officers accused of illegal search and seizure in 2011-13 were much more likely to be accused of the same offense in the following years. An identical correspondence applied for verbal abuse, corruption and other complaint categories. About 30 percent of the time, an officer’s most common category of complaint in 2011-13 was the same category in 2014-15. That’s about 50 percent higher than you would expect by chance.
Whether those complaints have merit is another matter, one the department investigates and judges on a case-by-case basis, levying discipline accordingly. I found that complaints were not only predictive of the number and type of future complaints — they also forecast whether the department would determine misconduct. Officers with 10 or more complaints in early years of the data set were about six times more likely to have a complaint from the last two years sustained against them.³
The city had the tools to identify and curtail troublesome officers before Kalven pursued legal action. All of the complaints were stored within the department long before the Invisible Institute had access. As Kalven put it, “All the knowledge to transform the system existed within the system.”
It is clear that repeaters can not only be identified, but their misconduct can also be predicted. For all the complexity of policing, there is a clear signal in the data of who the bad actors are and, to a lesser extent, whether they are going to commit misconduct. Whether police departments will make use of that signal is another, trickier matter. For the departments themselves, enacting meaningful reform is the more challenging undertaking. But with the increased public scrutiny driven by McDonald’s death and an imminent Department of Justice investigation, the department may be forced to change.