|
Local Validation: Is Close Enough Good Enough?
SESSION INFO
Monday, March 2, 2026
1:00 PM - 2:30 PM
Session Type: Workshop
One of the national standards for community supervision that APPA released in 2024 includes using a locally validated Risk and Needs Assessment (RNA). But what does local or valid even mean, and why are these criteria important? An assessment validated in a near-by jurisdiction (“close enough”) is often assumed to be acceptable (“good enough”), but it that true? Predictive validity, accuracy, and fairness vary widely across jurisdictions, and tools that perform well in one place can over- or under-classify system-impacted individuals in another. This session unpacks what validation really means, why accuracy is not the same as validity, and how overreliance on validations in other places perpetuates inequity. Drawing on real-world case examples we will show what happens when agencies test tools against their own data. We will outline a practical roadmap for conducting local validation, even with limited resources, and discuss how transparency and equity can be built into the process. Participants will leave with concrete strategies for making the case to leadership, commissioning local validation, and interpreting results for practice.
|
|
|
SESSION PRESENTERS
Katie Meyer
CAIS/JAIS Program Manager, Evident Change
Katie Meyer is a program manager at Evident Change, where she leads product modernization, training design, and strategic initiatives that bridge research and practice in community corrections. With more than fifteen years of experience in the criminal-legal field, she has been at the forefront of advancing systemic change in both adult and youth justice settings, guided by an unwavering belief in the inherent dignity and capacity for change of every individual. In recent years, Katie has championed responsivity-based models within the justice system, recognizing their potential to complement—and, at times, challenge—the dominance of risk- and need-focused paradigms. She brings both hands-on experience and a researcher’s curiosity to helping agencies translate evidence into human-centered practice.
Shantae Motley
, Evident Change
Shantaé M. Motley-Hodges, PhD is a Researcher IV at Evident Change and has served in research and leadership roles across public agencies, academia, and nonprofit organizations. Formerly the Director of Research at the Texas Juvenile Justice Department and an Assistant Professor at Prairie View A&M University, her work focuses on advancing equity-centered reforms in youth justice and child welfare systems. Her research interests include dual-status youth, implementation science, racial and gender disparities in system involvement, trauma-informed practices, and cross-system collaboration. She holds a Ph.D. in Juvenile Justice from Prairie View A&M University.
Stewart Wilson
Data Solutions Analyst, Evident Change
Stewart Wilson is an Associate Data Solutions Analyst whose work bridges policy, practice, and data within juvenile justice systems. He has led validation studies of two juvenile risk assessment tools—a pre-dispositional assessment and a detention screening instrument—using administrative data and statistical modeling to test their accuracy, consistency, and fairness. Stewart’s analyses have informed redesigned tools that better predict outcomes while reducing racial disparities; in one jurisdiction, his work helped cut detention recommendations in half for the validation cohort. With a commitment to making data meaningful for real-world practice, he focuses on translating statistical insights into actionable guidance that helps agencies improve decision-making and promote more equitable outcomes for youth.
|
|
|
|
|
|