Evidence-based decision making is all the rage in American criminal justice.

The basic idea is that everything we do with offenders – diversion, pretrial release, sentencing, supervision, and so forth – should be research-based, with the overriding goal of reducing future offending. If a given intervention does not reduce risk, then it should probably be abandoned in favor of more effective alternatives.

So, how does incarceration – one of the oldest criminal justice interventions in continuous use in the United States – measure up?

The Two Hands of a Longstanding Debate

Whether incarceration decreases recidivism risk is a matter of longstanding theoretical debate.

On the one hand, the unpleasantness of incarceration might increase the motivation of the defendant to avoid post-release crime that could land the defendant back behind bars. This effect is sometimes referred to as “specific deterrence.”

On the other hand, specific deterrence might be offset by any of a variety of countervailing factors. For instance, critics have long observed that incarceration cuts the defendant off from family and other potentially positive influences, and instead places the defendant in a social environment full of negative influences.

The disruptions and stigma of incarceration, moreover, may create or exacerbate long-term difficulties for the defendant in obtaining employment, education, and stable housing. Difficulties in these areas are known risk factors for recidivism.

A Difficult Theory to Research

With plausible theories offered by both sides, good empirical research is necessary to resolve the debate. However, rigorous testing of the effects of incarceration has proven difficult.

It is obviously inappropriate simply to compare the recidivism rates of those sentenced to incarceration with those sentenced to probation. Since perceived recidivism risk is a major factor in determining which defendants are sentenced to probation, one would naturally expect less repeat-offending from the probation group.

From a purely scientific standpoint, the best way to test the effect of incarceration would be to randomly assign defendants to different sentences, much as pharmaceutical companies test drugs by randomly assigning some patients to receive a placebo and others the real medication. However, randomizing incarceration would be objectionable on ethical and constitutional grounds.

Instead, the most common methods for testing correctional interventions employ a “quasi-experimental” design. This involves using statistical controls to isolate as much as possible the effect of the intervention being studied.

It is known, for instance, that men, on average, have higher recidivism rates than women. Therefore, in order to make a valid comparison of the recidivism rates of defendants sentenced to incarceration with those sentenced to probation, it is necessary to correct for any differences in the gender balance of the incarceration and probation groups.

A careful quasi-experimental study will also want to control for many other potentially confounding variables, including, among others, offense severity and criminal history.

A Challenge: Controlling for All of the Relevant Variables

No matter how careful, though, a single quasi-experimental study cannot provide a definitive answer to the effectiveness of a given intervention. Studies are usually limited in time and place, meaning that there will be questions about the generalizability of the results to other times and places.

Moreover, it is often impossible or prohibitively costly to obtain good data regarding all potentially confounding variables.

Additionally, researchers must often make difficult methodological decisions lacking a clear right answer. Consider, for instance, the questions that must be answered when attempting to quantify criminal history: Should arrests be included or just convictions? Should juvenile adjudications be included? Should misdemeanors be included, and, if so, should they be given a different weight than felonies? How should multicount cases be handled? And so forth.

For this reason, interventions normally do not get the “evidence-based” label until they have been subjected to multiple tests and have demonstrated consistent success across varying assessments.

A New Meta-analysis: A ‘Null Effect’

Among social scientists, efforts to combine multiple studies in order to provide an overall evaluation of an intervention are called “meta-analyses.” Social scientists have developed rigorous meta-analytic methods that provide increasingly reliable ways of drawing lessons from diverse groups of studies.

An important new meta-analysis of the effect of incarceration on recidivism was recently published in the journal Crime & Justice. Damon Petrich and his co-authors examined 116 different studies that tracked about 4.5 million offenders in 15 different countries. This is the largest, most up-to-date meta-analysis of the incarceration question now available.

Petrich, et al., conclude:

T]he current analysis shows that custodial sanctions have no effect on reoffending or slightly increase it when compared with the effects of noncustodial sanctions such as probation. This finding is robust regardless of variations in methodological rigor, types of sanctions examined, and sociodemographic characteristics of samples. All sophisticated assessments of the research have independently reached the same conclusion.

Given the strength and consistency of the evidence, the authors urge that the failure of custodial sanctions, overall, to reduce reoffending should now be regarded as a “criminological fact.”

Incarceration Must Be Justified on Other Grounds

In commenting on the policy implications of their meta-analysis, the authors assert, “[T]he criminological fact of a null effect for custodial sanctions undermines any justification [for incarceration] based on specific deterrence.”

This leaves open other potential justifications for incarceration, including just desserts, incapacitation, and general deterrence. Where such grounds for incarceration are lacking, however, it would seem inadvisable for incarceration to be used in the hope that it will deter the defendant from committing more crimes in the future.

A taste of prison, the research suggests, may be more likely to increase than decrease the defendant’s risk.

This article was originally published on the State Bar of Wisconsin’s Criminal Law Section Blog. Visit the State Bar sections or the Criminal Law Section webpages to learn more about the benefits of section membership.