In this New Thinking podcast, Reuben J. Miller, assistant professor of social work at the University of Michigan, and his research collaborator Hazelette Crosby-Robinson discuss some of the criticisms that have been leveled against risk assessment tools. Those criticisms include placing too much emphasis on geography and criminal history, which can distort the actual risk for clients from neighborhoods that experience an above-average presence of policing and social services. “Geography is often a proxy for race,” Miller says. Miller and Crosby-Robinson spoke with the Center for Court Innovation’s Director of Communications Robert V. Wolf after they participated in a panel on the “The Risk-Needs-Responsivity Framework”  at Justice Innovation in Times of Change, a regional summit on Sept. 30, 2016 in North Haven, Conn.

521x291.jpg" alt="Reuben J. Miller, assistant professor of social work at the University of Michigan, and his research collaborator Hazelette Crosby-Robinson participate in a panel at " title="Reuben J. Miller, assistant professor of social work at the University of Michigan, and his research collaborator Hazelette Crosby-Robinson participate in a panel at " class="image image-img_assist_custom-521x291 " width="520" height="291">Reuben J. Miller, assistant professor of social work at the University of Michigan, and his research collaborator Hazelette Crosby-Robinson participate in a panel at “Justice Innovation in Times of Change,” a regional summit.

WOLF: Hi, I’m Rob Wolf, Director of Communications at the Center for Court Innovation and today with me at the Justice Innovation in Times of Change Conference here at the Quinnipiac School of Law in North Haven, Connecticut are two of the panelists who participated in a discussion about risk needs assessment tools. They are Professor Reuben Miller, who is an assistant professor of social work at the School of Social Work at the University of Michigan and his research assistant at the School of Social Work, Hazelette Crosby-Robinson.  Thank you so much for taking the time after your panel to sit down and talk with me.

MILLER: Thank you for having us.

WOLF: So, I wanted to just start off talking about the risk assessment tools and some of the criticisms that have been leveled against them because, as we heard on the panel from Sarah Fritsche, a colleague of mine at the Center for Court Innovation, their use has exploded and they’ve been embraced as a decision-making tool in the criminal justice setting.

MILLER: Sure.

WOLF: But you raised some potential concerns about them and some of their limitations and I wondered if you could share what some of those limitations are as you see them.

MILLER: Sure, I’m happy to. So, Hazelette is my research associate and collaborator. She’s super modest.

So, I’d like to first preface this by saying, some scholars have suggested that we’ve really entered an actuarial age. So it’s not just risk assessment in criminal justice, but a whole cost benefits calculus, a whole risk calculus that’s based on actuarial models that try to predict future harm. So they try to predict, much like an insurance company would try to predict the future risk of a car accident. In a criminal justice setting, these risk needs assessments are trying to, one, gauge the needs of incarcerated individuals or people who have been convicted of a crime to try to figure out where they could shore up deficits in their skill sets or in their general stability. So for example, they might examine things like housing stability, or whether or not one was employed, or what kinds of service needs they may have. So for example, if one has a history of substance use and abuse, that would indicate that they need treatment or some sort of intervention based around these things.

And at the same time, they’re trying to gauge the risk of re-offense, so the risk that they will commit a crime. So there are a number of criticisms. The literature that engages this is fairly long. I tend to think about some of the movers and shakers in this field, Kelly Hannah-Moffat, Bernard Harcourt, Sonia Star, Faye Taxman. Faye Taxman’s work is actually helping us to think about important ways that we can implement risk assessment that reduce some of the biases that are sort of baked into it, but just to talk about some of the critiques that have come from this literature and of course my own, on the one hand there are static factors like where one lives, so geography, their prior criminal history. These are things that they can’t avoid. And the privileging of recidivism as an indicator of success. These are all problematic for the following reasons.

So geography is often a proxy for race. We know that we live in a country that has a pattern of residential racial segregation. And we know that policing and criminal justice resources of all kinds are overwhelmingly distributed in areas where poor people of color tend to live. The problem is, people are now being arrested from, returned to, and even given programs designed to rehabilitate them all within low income communities. Very bounded geographic districts. And so what you get is, you get the overwhelming concentration of criminal justice resources, and you get a signaling of what that all means. So if the substance abuse treatment house is located in a neighborhood, then that tells me that there’s substance abusers there. Right? And so that signals narcotics forces to the community. It says something about the community. Halfway houses are also overwhelmingly there. And so one must think about what the concentration of these things do. So now okay, as it relates to risk. Being in a neighborhood like this triggers a higher risk score. It is indeed one of the measures of risk, and so in that way it’s a proxy for race. Sorry, I know I’m talking quite a bit, but –

WOLF: No, and just to kind of summarize though, or to recap what you’ve said so far, the way risk assessment tools work, they place a high value on the location someone’s from. They place a high value on their history with arrest.

MILLER: That’s absolutely right.

WOLF: And so, if there’s a preponderance of enforcement there, some people are more likely to have an arrest record or –

MILLER: The study from Stop and Frisk made this abundantly clear. That even when people aren’t doing anything wrong they’re being overwhelmingly stopped if they’re black or Latino. And so we know that criminal justice contact increases the likelihood that one will be arrested. And so anyway, this is a big problem of using prior arrest records for example and even prior conviction records, so now you’ve got a bunch of arrests. By the time you get to the prosecuting attorney, they’re going to say, Look, you’ve been arrested 14 times. “Well, I’ve been arrested 14 times but never charged.” No, but you have a history of arrest, and so I’m going to now charge you because I see a pattern. This is how statistical discrimination might work, or does in fact work in practice. So now the prosecuting attorney sees a pattern. Sends it before the judge, who looks at this pattern and interprets it to make a decision about the length of the sentence when the conviction is read, as is a jury if it ever goes to trial. 97% of cases never go to trial, but when it goes to trial, juries are presented with the same evidence of patterns which have more to do with where the police are concentrated than what people are actually doing.

WOLF: So what do you say to the notion that these instruments are validated? That they predict? This information, whether there’s a potential bias incorporated into them, they still can predict six months to a year out whether someone is going to recommit a crime.

MILLER: Yes, with great reliability. But it’s a population being normed against itself. And so, overwhelmingly concentrate criminal justice resources in a particular neighborhood, which leads to more arrests, which leads to more convictions, which leads to more imprisonment. Then I look at those who were imprisoned, and I use that to validate my measures. So the problem, is this sort of self-fulfilling prophecy, this feedback loop, this is one problem.

Another problem is that, and Kelly Hannah-Moffat points this out brilliantly, correlation and causation are very different things. It’s like the standard social science response that any bench chair social scientist gives when they look at two relationships and people use that as some sort of cause, but likelihood that particular groups of people are more likely to commit a crime, doesn’t mean that having committed a crime in the past means you actually will commit a crime. And so what we’re doing is, we’re treating relationship as if it’s a cause, as if it’s a fact. And so I will sentence you now based on my assumption of your future danger to commit a crime based on a set of assumptions that I use to justify the overwhelming concentration of police to begin with. Police aren’t the culprits here. It’s a rationality, it’s a way to approach problems, that I think must be critically investigated.

WOLF: And you also pointed out in your presentation that perhaps the cultural context, the environment and the changing policy culture where for instance, marijuana arrest which were so vigorously pursued several years ago are now considered a low priority, or they’re not even being done anymore. And yet, people have a record of those arrests and if history of arrest is a factor, someone in the audience also questioned this, should we drop those particular kind of arrests as a factor because we don’t care about them anymore? Do they indicate further likelihood of going against the law or are they just something someone did because they like marijuana and that’s it?

MILLER: That’s right. And this is a part of the rigidity of risk assessment. This is rigidity of risk categories. So to place one in a category, you are an offender. And in Michigan, where I’ve done a lot of research and where I’ve worked, habitual offenses … and it’s not like this in Michigan, but it’s like this in many, many states, most states I would argue … being a habitual offender means more time, greater risk, more punishment.

CROSBY-ROBINSON: Up to life.

MILLER: Absolutely. So what does it mean to habituate? What am I looking at? Well, if I’m not being careful about the criminal codes, if I’m not carefully examining what I considered a crime at a given moment in time, and adjusting my instrument for that. Which must happen, probably, annually. If I’m not adjusting my instrument for that, if not quarterly. If I’m not adjusting that for different understandings of what is right and wrong, then what I’ll end up doing is habituating someone. Giving them longer sentences, giving them harsher treatment, deeper levels of punishment, or indicating they need deeper levels of intervention.

WOLF: So tell me what recommendations you’d make. Because you also made a point in the panel that there are some good things about risk assessment. They do take away discretion form judges or people whose own bias might lead them to make the wrong decisions?

MILLER: Absolutely. The benefit of risk assessment is to use it to avoid the criminal record to begin with. This one bit of it. So if you have low risk, low leverages, as my colleague pointed out earlier today, then you are not indicated for intervention of any kind. And it’s better to just release these folks without intervention of any kind.

WOLF: Right, and that’s what the research supports.

MILLER: The research supports it, absolutely. So what risk assessment allows … the careful prosecutor, judge, public defender, et cetera to do is to remove some of the discretion, because much of the decisions that are being made are based on a gut feeling. So I am reading something in the defendant. They don’t have remorse, or they haven’t shown accountability for their actions, or they have, as one of the panelists raised, belligerent interactions, let’s say with their parent or the prosecuting attorney or the defendant. And my assessment is happening divorced from what it means to actually be in court in that moment in time. How might a child, 17 years old, respond to facing 20 years in prison? How should they respond? Should they be depressed, sad, angry, avoidant? What are our expectations in this moment. And so risk assessment, what it allows us to do is say, Okay, let me take a step back, let me look at what actually happened. Let me get away from my intuition, let me think about a more objective way to assess how this defendant should be treated.

An interesting note … So here it is. We can use smart risk assessments to think carefully and critically about how we treat offenders, what level of intervention that we lay out whether that intervention be prison, or jail time, or a diversion program, or a treatment group. There’s no perfect way to do this which is why constant reevaluation is necessary. You can’t settle, this is the instrument for me. You can’t settle. It’s not the instrument for you –

CROSBY-ROBINSON: Continuous improvement.

MILLER: Continuous improvement.

WOLF: And maybe testing … if I understood what Sarah Fritchey said, my colleague the researcher for the Center of Court Innovation, that you also can test these instruments within certain populations and see, are they producing more negative outcomes for an African-American population? And ask these questions that you’re asking to weed out the bias that might be built into that.

MILLER: Absolutely. The questions that we’re raising are in some ways a set of philosophical questions but they’re questions about the application, the use of, the embrace of, instruments to determine whether or not someone is a future danger. Perhaps this is just the wrong approach altogether. Not the risk assessment … not that one doesn’t need to think about ways that they can help predict behaviors of individuals. I think that’s useful in some ways, but it certainly needs to be challenged, it needs to be questioned. What am I predicting? Who am I predicting this for? What are the possibilities for this person once these predictions are made? These are questions that need to be addressed.

WOLF: So, Miss Crosby-Robinson, let me ask you, as we talk about these kinds of assessments, you bring to bear your own set of experiences with the correctional system as a researcher and you’re own past history which you refer to on the panel as someone who had been formerly incarcerated. And I wonder what insights that had allowed you to bring to bear to this notion? Presumably a long time ago they didn’t have these risk assessments, I don’t know … when you were initially had your first contact with the correctional system, the justice system. And now they do and you’ve had a lot of contact and opportunity to interview and spend time with people are incarcerated and I wonder where you come down on this issue?

CROSBY-ROBINSON: Well, first of all, I think it’s a good idea to have a risk assessment, as Ruben has said earlier, because it removes some of this pressure from judges and prosecutors to make these decisions based on their own personal bias or how they’re feeling at the time. But what it does not account for are all of the little various innuendos that a person is going through when they come out. Family reunification can create a stressor. If somebody’s coming out and they have to be paroled to a family address, a suitable relative for placement. So they’re coming to this family address, but the family address that the parole officers decided that the person can parole to is not really the best environment, and sometimes the issues that they had that led to their incarceration stem from the family issues that they were having at the time. Or it’s not in the right environment or they don’t have really enough support from their family. And things happen because lives are fluid and things change.

For instance, we interviewed a person who was 17 years old and she was pregnant. She had a mental illness, she’d been in the mental health system since she was 8 years old. She lived with her grandmother. We interviewed people three times, as soon as they were discharged and then 30 days after they’d been out. Then 60 days and 90 days. And so, following her, by the time we got to her third interview, her grandmother dies. She’s living in her grandmother’s house. This is the only stable person she’s known in her life. Her grandmother has raised her since 9 years old. She just had a baby, the baby isn’t even a year old yet. Now she’s 18 years old, she has a mental illness, and she’s relying on that system to become her support now where her grandmother was everything. Well, these are things that a risk assessment would just not pick up. Because you never know what’s going to happen. So now what happens to this individual? We’re at the end of the time that we follow this person for our study. But you know, the question is in our mind, what happens?

And another thing that I find frustrating, is no matter what your risk assessment is and if you get it right or not, then when a person who gets out into the community, whatever the risk assessment decided that they need as a support or an intervention, there’s no community resource for that.

WOLF: The theme I’m hearing from both of you is that these risk needs assessment tools cannot be judged or effectively used apart from the environment, whether it’s the environment that created the measurements of risk, or the needs. Because if you can identify the needs and say great, but if you don’t have the resources in the community it’s meaningless information.

MILLER: So, Faye Taxman has a great paper. She finds that, on the need side of things, substance abuse treatment is indicated in about 90% of the folks who were justice involved, but the capacity to provide the treatment, either in prison or out. Something like 25% of folks in prison were able to actively engage in regular substance abuse treatment that needed it. And so what this does is it creates another deficiency that one might judge or regard as a part of the risk of this individual recidivate. Did you complete programming? What was programming available, either in prison or out?

WOLF: Well, this had been a very vigorous and interesting conversation and I really appreciate you’re both taking the time to speak to me about your work.

MILLER: Yeah, thanks for having us.

WOLF: So, I’ve been speaking with Professor Reuben Miller, assistant professor of social work at the School of Social Work at the University of Michigan and his research assistant and collaborator, Hazelette Crosby-Robinson and we’re all here today in New Haven at Quinnipiac University school of law for the Conference of Justice Innovation in Times of Change, which is sponsored by the Center for Court Innovation and the Department of Justice’s Bureau of Justice Assistance and hosted by Quinnipiac University. You can find out more about risk needs assessment and criminal justice reform in general at our website, www.courtinnovation.org. I’m Rob Wolf, thanks for listening.