- Assist persons convicted of a crime or at-risk populations of individuals with potential to become involved in the justice system.
Rated programs and practices are included in an online clearinghouse.
The terms “program” and “practice” have particular meanings in CrimeSolutions. A “program” is a specific set of activities, carried out according to guidelines, to achieve a defined purpose. Program profiles on CrimeSolutions tell us whether a specific program was successful in achieving justice-related outcomes when it was carefully evaluated.
A “practice” is a general category of programs, strategies or procedures that share similar characteristics, in terms of the issues they address and how they address them. Practice profiles tell us about the aggregate results from multiple evaluations of similar programs, strategies or procedures.
CrimeSolutions employs a standardized process to evaluate programs and practices in order to determine both (1) the reliability of the science methods used to evaluate effectiveness, and (2) the level of effectiveness, if any, of the program or practice under evaluation.
This article discusses key aspects of the purpose, design, benefits and limitations of the CrimeSolutions resource, providing links to key program web resources.
Of perhaps greatest utility, it explains to the reader where to find and how to take advantage of CrimeSolutions when searching for or studying a program or practice, or deciding whether to implement it.
Overview of CrimeSolutions
The CrimeSolutions clearinghouse identifies justice system programs and practices that have been submitted to rigorous scientific evaluations. The CrimeSolutions evaluation process closely examines programs and practices for scientific evidence of effectiveness. If the evidence is strong and clear enough, a program will be rated “Effective,” “Promising,” or “No Effects.” If the evidence is insufficiently strong and clear, however, CrimeSolutions reviewers will withhold an effectiveness rating. Those unrated programs or practices are deemed to be emerging, inconclusive or unclear.
Table 1 shows, as of May 2021, totals of rated programs and practices and subtotals for those focused-on corrections.
Programs and practices selected for CrimeSolutions review are first identified, screened, reviewed and rated, through a standardized process. See How We Review and Rate Programs form Start to Finish and How We Review and Rate Practices from Start to Finish.
Two certified program reviewers assess each screened program and practice with objective scoring instruments. Reviewers examine scientific evaluations of programs. Broader practices are reviewed by reference to meta-analyses that synthesize existing rigorous evaluations of those practices.
The importance of evidence-based practices and rigorous science
At the core of CrimeSolutions’ effectiveness ratings is the reliance on available, rigorous evidence. Programs and practices are considered evidence-based when their effectiveness has been demonstrated by causal evidence demonstrated in high quality outcome evaluations. The use of scientific methods yields causal evidence by ruling out alternative explanations for observed change.
The more rigorous the scientific method used to evaluate a program or practice, the more certain an effectiveness rating will be. The science must be sufficiently strong and certain before CrimeSolutions will assign an effectiveness rating for a given practice or program.
The so-called gold standard of evaluation methods is the randomized controlled trial, or RCT. In RCTs, participants are randomly divided into treatment and control groups. Carefully controlled and measured division of participants ensures that, to the extent possible, the only difference between the groups is one receives the experimental treatment and the other does not. As an evaluable method, the randomized controlled trial is uniquely capable of isolating and measuring the effect of experimental treatment. Beyond randomization, RCTs must be well-coordinated, with provisions to ensure, for instance, there is no contamination between the treatment and control groups.
In some studies, a rigorous comparison is either not feasible, or not needed to establish the answer to the scientific inquiry. A randomized trial would not be feasible when, for example, denial of the treatment to a control group would be unethical, or the subject of a study is retrospective. And generally, RCTs rely on relatively large sample sizes to demonstrate measurable effect.
The distinct value of CrimeSolutions for different professional audiences
Having objective scientific evaluations of available corrections programs and practices can offer immediate benefits for a variety of stakeholders, including academic researchers engaged in program evaluations, practitioners, policymakers and trainers.
Some of those CrimeSolutions benefits, by audience group, are:
Corrections professionals may be able to improve program or practice effectiveness by one or more of the following actions:
- Familiarizing themselves with rated and evaluated programs in their field.
- Replicating a program or practice found in CrimeSolutions.
- Adapting a program or practice from CrimeSolutions.
Policymakers can shape funding decisions by:
- Creating incentives to use evaluated and rated programs and practices.
- Creating incentives for ongoing innovations and creation of evidence-based programs and practices.
Program trainers can improve training by:
- Developing training materials for evaluated and rated programs.
Researchers can improve justice programming and become more informed on criminal justice research by:
- Consulting CrimeSolutions evidence standards to strengthen evaluation designs.
- For programs rated “Promising” by CrimeSolutions, focusing on using rigorous evaluation designs to build the body of evidence and potentially increase confidence in program effectiveness.
- For programs or practices rated “Inconclusive” by CrimeSolutions, researchers may have an opportunity to build the program evidence base sufficiently for a subsequent CrimeSolutions evaluation to establish, with the requisite degree of scientific certainty, a program or practice is promising or effective or has no effects.
- Improve the available body of knowledge: Where an existing program or practice has been rated “Promising” by CrimeSolutions, there is incentive for independent researchers to conduct additional evaluation work, using a more rigorous scientific design where applicable, such as an RCT, to test or re-test program strength and effectiveness.
Limitations of CrimeSolutions
Anyone interested in tapping the benefits of CrimeSolutions’ ratings of program and practice effectiveness should bear in mind certain practical or possible limitations of the resource. They include:
- The fact a program or practice works in one setting, with a particular population, does not mean it will be equally effective in every setting, or for different populations. Adopters of programs or practices rated Effective or Promising on CrimeSolutions should be prepared to evaluate scientifically the program or practice once applied in their own environment.
- Programs often have mixed results for various outcomes – some positive, some negative. CrimeSolutions assigns ratings to programs based on the preponderance of evidence from up to three studies that have assessed outcomes related to crime, delinquency, or victimization prevention, intervention or response.
- Although CrimeSolutions is a rich resource, with more than 700 posted programs or practices, not all criminal justice, juvenile justice and victim services programs and practices have qualified for CrimeSolutions rating. Agencies should take care to select programming well matched to their needs.
CrimeSolutions’ eight-step process for reviewing and rating programs and practices
CrimeSolutions follows the eight-step evaluation process to determine whether a given program or practice will receive a rating:
- Preliminary Program Identification — programs and practices identified through literature searches and nominations from the field.
- Initial Program Screening — programs and practices reviewed to confirm they fall within the substantive scope of CrimeSolutions.
- Expanded Literature Search — expanded search to all evaluations and research and program materials that may be of interest to reviewers.
- Initial Evidence Screening — identified studies reviewed to determine whether the criteria for “evidence” of effectiveness has been met.
- Selection of Evidence Base — senior researcher selects evaluations or meta-analyses that follow rigorous study designs and methods.
- Expert Review — two certified reviewers perform an evidence review, using a scoring instrument, that assesses the quality and strength of the evidence, as well as the extent to which the evidence indicates the program or practice achieves its goals.
- Study Classification — each study is assigned one of five classifications on the quality of the study.
- Effectiveness Rating — CrimeSolutions assigns evidence ratings addressing program or practice effectiveness. For practices, ratings are assigned by outcome; one practice could receive multiple, differing ratings, based on how effective it is in addressing various outcomes.
Evidence ratings falling into three classes:
- Effective — Strong evidence to indicate the program achieves justice-related outcome(s) when implemented with fidelity to the design.
- Promising — Some evidence to indicate that the program achieves justice-related outcome(s) when implemented with fidelity to the design.
- No Effects — Strong evidence to indicate the program had no effects or had harmful effects.
Not all programs and practices that make it past the initial review receive a rating. Many evaluations are determined by reviewers to be inconclusive, and for those programs or practices no rating is assigned.
Examples of corrections practices rated by CrimeSolutions
The following are two examples of correctional practices rated by CrimeSolutions:
Rehabilitation programs for adult offenders
This practice includes programs that are designed to reduce recidivism among adults convicted of a crime by improving their behaviors, skills, mental health, social functioning and access to education and employment. Adults may become participants in rehabilitation programs during multiple points in their involvement with the criminal justice system.
Evidence Rating for Outcomes — A single outcome was rated “Promising” for this practice. Looking at 634 independent effect sizes, researchers found a statistically significant mean effect size of 0.203 for recidivism. This finding indicated those who participated in rehabilitation programs demonstrated reductions in criminal offending, compared with a control group who did not participate.
Practice Goals/Target Population — Rehabilitation programs are designed to reduce recidivism among adults convicted of a crime by improving their behaviors, skills, mental health, social functioning and access to education and employment. Adults convicted of a crime may become participants in rehabilitation programs at multiple points in their involvement with the criminal justice system, and programs are typically provided in conjunction with some form of sanction (e.g., incarceration or probation). Therefore, most programs are delivered to persons within correctional settings, or in community settings following their release (i.e., probation or parole-based programs). Community-based settings may be delivered in inpatient facilities, such as psychiatric hospitals and outpatient treatment centers, or in supportive residential housing such as halfway houses. In addition, some rehabilitation programs (such as drug courts) serve as alternatives to incarceration, diverting individuals into services in the community rather than into correctional facilities.
Practice Components — Rehabilitation programs do not generally follow a common, well-defined treatment protocol.[2] Instead, interventions and services may vary significantly by program. All programs address at least one of the risk factors commonly associated with offending (such as mental health status, substance use, education level or employment status). For example, a drug court program may provide for treatment only to address substance abuse issues. More commonly, however, rehabilitation programs combine multiple services: for example, a drug court program that provides not only substance abuse treatment, but also individual counseling and vocational training.
The general types of treatment services provided by rehabilitation programs include group work (structured via protocol or psychoeducational content); cognitive behavioral therapy (CBT) or CBT-like components (thinking skills, relapse prevention or anger management); counseling (group, individual, mentoring); academic work (GED or college classes); employment-related (work-release, job placement, vocational training); supportive residential (therapeutic community, halfway house); drug court or other specialized court; multimodal, mixed treatments (individual case management); intensive supervision (reduced probation or parole); or restorative interventions (mediation, reparations, community service).
Practice Theory — Effective rehabilitation programs typically use treatment methods that are based on behavioral and social learning theories of change. Behavioral theory suggests individuals are conditioned to behave in a certain way based on experiences with reinforcement and punishment (Skinner 1965). In contrast, social learning theory posits people learn behaviors from one another, through observation, imitation and modeling.[3] Therefore, rehabilitation programs are designed to reduce criminal behaviors through the positive reinforcement of conventional behaviors learned through observation or modeling.[4] For example, a program may help a participant learn how to manage his or her anger by modeling appropriate responses instead.
Treatment in secure corrections for serious juvenile offenders (2015)
This practice includes interventions targeting serious (violent and chronic) juveniles sentenced to serve time in secure corrections. The overall goal is to decrease recidivism rates when juveniles are released and return to the community. The practice is rated Effective for reducing general recidivism and serious recidivism of violent and chronic justice-involved youth.
Evidence Rating for Outcomes — An Effective rating was assigned to two outcomes for this practice:
- Crime & Delinquency — Multiple crime/offense types. To determine the impact of treatment on general recidivism rates, researchers examined 30 comparisons, between treatment groups and control groups, which used intent-to-treat data. The authors found a significant odds ratio of 1.307 in favor of the treatment group, meaning chronic and violent justice-involved youth who received treatment in secure corrections had lower recidivism rates than comparison group juveniles who did not receive treatment.
- Crime & Delinquency — Serious recidivism. Researchers examined 15 comparisons, looking at serious recidivism (which included reincarceration or reinstitutionalization) of justice-involved youth. The authors calculated a significant odds ratio of 1.354, meaning the treatment provided in secure confinement significantly reduced the serious recidivism of chronic and violent justice-involved youth.
Practice Goals/Target Population — Interventions targeting serious (violent and chronic) justice-involved youth sentenced to serve time in secure corrections aim to decrease recidivism rates when juveniles are released and return to the community. These interventions can include psychological approaches, social and educational methods and environmental conditions, all of which support the learning of prosocial attitudes and behaviors.
A juvenile is generally defined as a young person aged 12 to 21 years old. This practice targets juveniles who commit violent offenses or chronically offend. Violent justice-involved youth are juveniles who have committed offenses in which someone has been hurt or seriously injured and requires medical attention. Violent offenses include murder, voluntary manslaughter, kidnapping, assault, robbery, endangerment and arson. Youth chronically involved in the juvenile justice system are those who have three or more previous legal adjudications.
Practice Components — There are a number of different types of treatment that may be available to justice-involved youth in secure corrections. The treatment types include behavioral, cognitive–behavioral, cognitive, education and nonbehavioral.
Conclusion
We created CrimeSolutions to help criminal justice, juvenile justice and crime victim service professionals better understand crime and identify program and practice solutions that address the unique needs of their communities.
CrimeSolutions helps justice professionals who are not social scientists improve the effectiveness of programs. The systematic, independent review process and evidence ratings are intended to give practitioners access to social science evidence that is otherwise difficult to obtain, and serve as a basis for gauging the quality of evidence. In short, CrimeSolutions strives to help practitioners answer the questions: has it worked, and in what context?
Notes
[note 1] In 2016, nearly 2.3 million research articles were published, according to the National Science Foundation, Science & Engineering Indicators 2018, National Science Foundation. https://www.nsf.gov/statistics/2018/nsb20181 (December 2018).
[note 2] Mark Lipsey and Francis Cullen, “The Effectiveness of Correctional Rehabilitation: A Review of Systematic Reviews,” Annual Review of Law and Social Science 3 no. 1 (2007):297–320.
[note 3] Alfred Bandura, “Self-Efficacy: Toward a Unifying Theory of Behavioral Change,” Psychological Review 84 no. 2 (1977):191–215.
[note 4] Mark Lipsey and Francis Cullen, “The Effectiveness of Correctional Rehabilitation: A Review of Systematic Reviews,” Annual Review of Law and Social Science 3 no. 1 (2007):297–320.
