There are many tools across the country used to assess pretrial risk for criminal defendants. In Colorado, multiple counties take part in what is called the Colorado Pretrial Assessment Tool, also known as CPAT. CPAT was developed for bond decisions and is used to measure a given defendant’s risk of failure to appear or for being arrested again while under pretrial release.
But CPAT has been criticized in the past, and questions remain even among those who use the tool regularly. In July, the University of Northern Colorado released a study about the effectiveness and validation of the CPAT.
“I think this is the end of the CPAT as we know it as a result of this report,” said Jeff Clayton, executive director of the American Bail Coalition. He added that there was a push in the past to make the CPAT a statewide program.
CPAT was created in 2012 and involved a sample of 2,000 defendants from 10 Colorado counties, according to the study. Through analysis of 177 variables, a 12-item CPAT was created with scores range from 0-82 in four categories, indicating risk of failure to appear or reoffend under pretrial release, which are ranked from lowest to highest, according to the study.
The risk factors and scoring include information gathered from the details of a person’s possessions, history of alcohol or drug usage, whether a person has a home phone or cell phone, mental health treatment, age, prior jail or prison time, active warrants, pending cases and history of revoked bond or supervision.
Clayton says that risk factors make CPAT biased toward the homeless population of the state. The U.S. Census website states that in July 2019, solely descended Black or African American population of Colorado was only 4.6%. In 2018, the Homelessness Research Institute estimated 40% of the national homeless population were Black and were overrepresented among people experiencing homelessness.
“Which means we’re hammering African American defendants at the lowest socio-economic level,” Clayton said.
By 2017, CPAT was used in 25 of Colorado’s 64 counties.
The goal of the UNC study was to validate and examine CPAT, then using stakeholder feedback, to pilot a revised version of the tool called CPAT-Revised, or CPAT-R. The study had three phases and was conducted between 2018 and June 2020, according to the study.
Phase One included a retroactive validation of CPAT covering over 3,500 cases in 2015 and 2016 with data from seven participating counties across the state, an online survey to criminal justice stakeholders, as well as court data from Denver Municipal Courts, the Colorado Judicial Branch and the Colorado Bureau of Investigation.
CPAT was validated for assigning the correct risk of new arrest and failure to appear, according to the study. However, the pretrial risk assessments received a validation score between .50 and .75, and CPAT’s validation score was at .58, at the lowest end of the score range. Allegedly, validation scores can indicate a degree of predictive performance, and a good ranking is between .55-.63 and excellent is .71 or above.
The study also found that further examination of risk category designation and tool modification could improve the predictive performance of CPAT. Analysis indicated that with modifications, the tool could improve with an estimated validation score above .60, and modifications could be made to categories such as risk items and specification of risk outcomes.
Phase two used focus groups and observations of the seven counties, according to the study. The participants for the groups were recruited from a survey. The groups consisted of pretrial officers, judges, prosecutors, public defenders and other criminal justice administrators.
The groups received questions pertaining to how the participants viewed CPAT and utility of the tool in their work. The study identifies that some items relating to CPAT were unclear to respondents, like what counts as mental health problems, and many defense attorneys felt the tool penalizes indigent defendants, such as renting instead of owning, not having a phone and that “those with lower or no income are scored higher than those who have more economic advantages.”
Part of CPAT involves an interview process with defendants, which all four groups expressed were important to gather non-system information but allows for inaccurate information to be provided.
One defense attorney expressed that in their experience, the interview “tends to be highly subjective and falls prey to the biases of the interviewer, who is often unsympathetic to indigent defendants.” Further, the interviewer in the area where the attorney operates is often the check-in person for pretrial tracking services and tends to “play favorites” with clients.
Of note, the study mentions that “many of the respondents did not prioritize CPAT in their decision-making process” and that the risk assessment score and category are perceived as being no more or less important to other factors. One prosecutor wrote that the score was a “piece of information” and not something that could be relied upon to make arguments to judges.
Further, the study found that overriding a CPAT score was “especially” relevant to judges dealing with public safety. Based on the feedback, the study states that the risk assessment score was thought to differ occasionally from risk decided on professional discretion alone.
Stakeholders also wanted to understand why the questions for CPAT were picked. One pretrial officer was quoted as saying, “because I feel like there’s just not a lot of faith in the tool as it is. Especially with the other stakeholders.”
The third phase of the study was the piloting of CPAT-R over three months from November 2018 to February 2019. CPAT-R showed an accurate assignment of risk of new arrest and FTA and was estimated to perform at an AUC score of .65.
The pilot CPAT-R was assessed for “accuracy equity and predictive parity” throughout race/ethnicity, gender and residential status groups, according to the study. This informed a modification to the pilot, removing two of the risk factors: prior violent arrest and self-reported time living at current residence. Assessing the features of the risk factor definitions also led to the removal of self-reported prior alcohol or drug problem as a scored response.
The recommended CPAT-R was re-validated on a sample of primary pretrial outcomes: new arrest, FTA or both. “The test sample’s assessment errors were comparable, indicating accuracy equity across race/ethnicity and gender, and minimal bias across sub-groups. Differences in predictive parity across residential status were reduced but were still greater than the other sub-group categories,” according to the study.
For Clayton, one table in the report exemplified his concerns on CPAT. When a person is predicted as risky and is not, “we get it wrong for Black defendants significantly more than white defendants.” Based on these numbers, 13 out of 100 Black people will be assessed at a higher risk, and the magnitude of that over time is immense.
Table 22 shows validation samples for race/ethnicity assessment errors. The overall prediction error for the pilot CPAT-R showed .41 for whites, .50 for Blacks and .43 for Hispanics. The failure prediction error was at .52 for whites, .66 for Blacks and .50 for Hispanics. The success predication error was .23 for whites, .27 for Blacks and .27 for Hispanics.
When looking at the total number of individuals evaluated by the risk assessment, which are all in large jurisdictions of Colorado, Clayton wondered how many minority groups were overrepresented and what had been the result?
— Avery Martinez