Assessing Risk Assessment in Action

posted by Judge_Burke @ 15:30 PM
December 27, 2017

In recent years there has been an increasing use of assessment tools in an effort to achieve “evidence base sentencing.” For some the tools have become not an aid to judgement, but a substitute for it. There are notable critics such as former Attorney General Holder. There are critics who warn that some of the actuarial tools just are not very good predictors or have their own set of biases. Professor Megan Stevenson has written this interesting new paper available via SSRN.

Here is the abstract:

Recent years have seen a rush towards evidence-based tools in criminal justice.  As part of this movement, many jurisdictions have adopted actuarial risk assessment to supplement or replace the ad-hoc decisions of judges.  Proponents of risk assessment tools claim that they can dramatically reduce incarceration without harming public safety. Critics claim that risk assessment will exacerbate racial disparities. Despite extensive and heated rhetoric, there is virtually no evidence on how use of this “evidence-based” tool affects key outcomes such as incarceration rates, crime, or racial disparities.  The research discussing what “should” happen as a result of risk assessment is hypothetical and largely ignores the complexities of implementation.

This Article is one of the first studies to document the impacts of risk assessment in practice.  It evaluates pretrial risk assessment in Kentucky, a state that was an early adopter of risk assessment and is often cited as an example of best-practices in the pretrial area.  Using rich data on more than one million criminal cases, the paper shows that a 2011 law making risk assessment a mandatory part of the bail decision led to a significant change in bail setting practice, but only a small increase in pretrial release. These changes eroded over time as judges returned to their previous habits.  Furthermore, the increase in releases was not cost-free: failures-to-appear and pretrial crime increased as well.  Risk assessment had no effect on racial disparities in pretrial detention once differing regional trends were accounted for.

Kentucky’s experience does not mean we should abandon risk assessment, but it should temper the hyperbolic hopes (and fears) about its effects.  Risk assessment in practice is different from risk assessment in the abstract, and its impacts depend on context and details of implementation.  If indeed risk assessment is capable of producing large benefits, it will take research and experimentation to learn how to achieve them.  Such a process would be evidence-based criminal justice at its best: not a flocking towards methods that bear the glossy veneer of science, but a careful and iterative evaluation of what works and what does not.


Leave a Reply