Failure to Make Data Driven Decisions

Written by Doug Thomas

My mother told me on more than one occasion that the road to Hell is paved with good intentions. And we have all heard about what happens when one assumes things. Unfortunately, many of the decisions made by juvenile drug court professionals are smothered with good intentions based on assumptions backed only by tradition, personal beliefs, what others have said, and other unreliable sources.

The decisions made by juvenile drug courts are too important and far reaching to be left to subjective experiences, misinformation, and half-baked truths—no matter how well intentioned. Think about the decisions juvenile drug court professionals make every day—who is accepted and who is rejected; what kind of treatment youth receive; who is deprived of liberty and detained and who remains free in the community; when are incentives offered and when are sanctions applied; and ultimately who succeeds and who fails.

It does not have to be this way. There are too many sources of reliable and valid data on which JDC professionals can base their decisions. The science of risk and needs assessment is strong, and JDC professionals can have confidence in the data generated by these instruments and make decisions accordingly. There is also a great deal of information on evidence-based practices (e.g., structured case planning, motivational interviewing, drug testing strategies) that helps JDC professionals to identify, prioritize, systematically address the specific risks and needs of JDC participants, and carefully monitor and manage progress. The rapid proliferation of evidence-based programs, practices, and interventions for delinquent youth and those using alcohol and drugs takes the guess work out of placing the right kid in the right program at the right time.

To avoid taking the wrong road and making poor—or even harmful—decisions for JDC-involved youth, JDC professionals must avail themselves of the full range of tools, strategies, and other resources available to gather the valid and reliable data necessary to make accurate, effective, and correct decisions about the risks, needs, and appropriate responses for youth, including official court records, social histories, risk and needs screening and assessment instruments, results of clinical assessments, and, yes, professional judgement based on experience, education, and training. To assure JDC-involved youth achieve the desired results and expected benefits of the JDC experience, JDC professionals must strive to match the youth with effective programs and interventions that best match the risks and needs of participants as well as their ability to respond to selected programming.

Finally, JDC professionals should not trust assessments of JDC performance and success to mere intentions and hopeful assumptions. Assessments of JDC performance and individual success must be based on empirically based measures of short- and long-term success, including successful completion of treatment programs, law-abiding behavior, family support, social connectedness, and freedom from alcohol and drug use. Simply to assume success is to commit one of the deadliest sins.

Short-Term Solution:

Identify data sources already used in your JDC and look at the data. Figure out what the data is telling you. This can often lead to key insights into JDC practices that teams would have never figured out if they had not decided to stop and look at their data. This can also help JDCs identify aspects of the program that might be good candidates for implementing evidence-based practices.

Long-Term Solution:

Develop and maintain regular assessments of the JDC and its practices as they relate to outcome measures for JDC youth. This might include hiring or contracting an evaluator, and this might also require team members to collect data in ways in which it was not previously collected. Then, and arguably the most critical aspect of the process, JDCs should look at their data frequently to track JDC practices and youth outcomes in order to identify and implement evidence-based changes when the data suggest so. This is the process of making data-driven decisions and can help steer a court into progress and beneficial outcomes for youth and out of stale status quo practices and potentially negative decisions for youth.

“Very well written! However, the challenge for many jurisdictions is lack of professional training to understand the importance of data collection practices, analysis of collected data, and how to appropriately and systemically plan and implement process/program improvements. Additionally, lack of funding to deploy EBP’s, assessment tools, etc…and the lack of qualified team members to monitor fidelity when they do implement researched supported tools.”

Rosie Medina
Director of Special Programs
El Paso County, TX

“There have been times in our court where we make changes that I think are going to affect the data a certain way and then when we actually look at it, what we expected isn't happening. Relying on data to make program changes also ensures that we are not spinning our wheels. If we make a change and it affects the data positively then we know we are on the right track. But if we don't see the improvement that we are looking for then we know we need to try again. Not only is looking at performance measure data important but client satisfaction and feedback data is also critical. We don't have to guess if the participants like what we put in place; we can simply ask them and find out. The participants are also a rich source of ideas that we may never have thought of. I think relying on data is one of the best ways to avoid falling into common pitfalls. Often times we can be led by our feelings or what we "think" is going on. Using data to drive decision making ensures we can stay objective and avoid letting unintentional bias cloud our judgment.”

Lindsey R. Lucero
Second Judicial District
Juvenile Drug Court Program Manager