A ship’s captain who successfully sails through a strait at night learns nothing, and adds nothing, to their knowledge of the world.
(Please hang with me. I promise this post will not be a long, metaphysical rant).
Returning to the example. As paradoxical as it may strike one at first blush, a captain whose ship founders on the rocks while sailing through the strait both learns and adds to their knowledge. As philosopher Ernst von Glasersfeld once opined, “The only aspect of that ‘real’ world that actually enters into the realm of experience is its constraints.”
The principle identified by von Glasersfeld applies not only to life lessons, but also to scientific advancement and, of course, feedback informed treatment (FIT). Indeed, identifying and learning from “constraints” — that is, when things go wrong — is the very purpose of FIT.
It’s why, for example, when a client refuses to complete the outcome and alliance measures, my first impulse is to “lean in” and explore their reasons, rather than instantly set the scales aside. It’s also why I’m most intrigued by studies which find that FIT fails to improve outcome (1, 2). In both instances, my curiosity is piqued. “Finally,” I think, “a chance to learn something …”. Doing so, cognitive science has long shown, is not as easy or straightforward as simply adjusting our beliefs in light of new facts. Quite to the contrary.
We are prone to see what we expect, fit the “different” into our current way of viewing the world or ignore it altogether. One brief example before turning attention to FIT (aka Routine Outcome Monitoring [ROM]). For most of the history of the field, the failure to engage in and respond to psychological intervention has been attributed to a host of client variables (e.g., degree or type of dysfunction, poor attachment history, IQ, etc.). Therapists, for their part, have been held accountable for making the correct diagnosis and administering the right treatment.
And yet, despite continuous growth in the size of the DSM, and number of treatment approaches, no improvement in the outcome of psychotherapy has occurred in the last 50 years — a fact I first talked publicly about in 2014 and which über-researchers James Prochaska and John Norcross finally acknowledged in the most recent issue of the American Psychologist. While some have argued that the field’s flat outcomes indicate the effectiveness therapy has reached a natural limit, an alternate point of view is that we should consider looking beyond the current ways of thinking about what matters most in successful treatment.
On this score, one possibility has been staring the field 
in the face for decades: the impact of the individual therapist on outcome. Research has long shown, for example, that who does the treatment contributes 5 to 9 times more to outcome than the type of therapy, psychiatric diagnosis, or client history. The same body of evidence documents some practitioners are consistently more effective than others. When researcher Scott Baldwin and colleagues looked into why, they found 97% of the difference was attributable to therapist variability in the alliance. Said another way, more effective therapists are able to establish a strong working relationship with a broader and more diverse group of clients. I hope you’re seeing new possibilities for improving effectiveness. If you’re a regular reader of my blog, you already know my colleagues and I published the only study to date documenting that a focus on therapist development via routine outcome measurement, feedback, and deliberate practice improves both agency and individual practitioner outcomes.
Turning to FIT, in my first post of the New Year, I talked about the strong sense of “anticipointment” I felt when thinking about the future of our field. A colleague from the Netherlands, Kai Hjulsted, responded, saying he’d been having the same feeling about FIT! The source, he said, was a study by a Dutch researcher conducted in a crisis intervention setting which, “contrary to expectations,” found, “Patients with psychiatric problems and severe distress seeking emergency psychiatric help did not benefit from direct feedback.”
I was well aware of this study, having served on the researcher’s dissertation committee. And over the last decade, multiple studies have been published showing little or no benefit from feedback (e.g., 1, 2, 3).
How to make sense of such findings? Having spoken with numerous practitioners (and even some researchers), I can tell you the tendency is to fit the results into our current way of viewing the world. So, seen through a traditional medicopsychiatric lens, the inevitable conclusion is FIT does not work with people with certain, specific diagnosis (e.g., severe distress, in crisis, or those with eating disorders). Such a conclusion makes no sense, however, if the totality of evidence is considered. Why? Because the results are decidedly mixed. Thus, in one study, FIT makes a difference with people in crisis, in another it does not. With one group of “severely distressed” clients, feedback appears to make matters worse, with another, chances of improvement increase 2.5 times.
What then can we conclude?
An answer begins to emerge as soon as we’re able to get beyond thinking of FIT as just one more in a long list of treatment methods rather than a fundamental, organizing principle of agency and practice culture. As is hopefully obvious, learning to administer measurement scales takes little time.
Cultural change, by contrast, is a much longer process. How long? Norwegian researcher Heidi Brattland and colleagues found it took three years ongoing training and support to successfully implement FIT. Had they stopped to evaluate, like all other studies to date, after an average of 4 hours of instruction, no impact on outcomes would have been recorded.
While its now clear that time and support are critical to 
successful implementation, these two variables alone are not sufficient to make sense of emerging, apparently unsupportive studies of FIT. Addressing such findings requires we look at the type of design used in most research: the randomized controlled trial. That I’ll do in my next post, in particular addressing two, top notch, well-executed studies many have assumed show FIT is not effective in psychological care for people with eating disorders and severe distress.
And so, as I asked at the outset, please “hang with me.”
Until next time,
Scott
Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence
