It’s just two weeks ago. I was on a call with movers and shakers from a western state. They were looking to implement Feedback Informed Treatment (FIT)–that is, using measures of progress and the therapeutic relationship to monitor and improve the quality and outcome of mental health services.
I was in the middle of reviewing the empirical evidence in support of FIT when one of the people on the call broke in. “I’m a little confused,” they said hesitantly, “I thought you were the anti-evidence-based practice guy.”
It’s not the first time I’ve been asked this question. In truth, it’s easy to understand why some might believe this about me. For more than two decades, I have been a vocal critic of the idea that certain treatments are more effective for some problems than others. Why? Because of the evidence! Indeed, one of the most robust findings over the last 40 years is that all approaches work equally well.
Many clinicians, and a host of developers of therapeutic approaches, mistakenly equate the use of a given model with evidence-based practice. Nothing could be further from the truth. Evidence-based practice is a verb not a noun.
According to American Psychological Association and the Institute of Medicine, there are three components: (1) using the best evidence; in combination with (2) individual clinical expertise; while ensuring the work is consistent with (3) patient values and expectations. “FIT,” I responded, “not only is consistent with, but operationalizes the definition of evidence-based practice, providing clinicians with reliable and valid tools for identifying when services need to be adjusted in order to improve the chances of achieving a successful outcome.”
Here’s another recent question: “I’ve read somewhere that FIT doesn’t work.” When I inquired further, the asker indicated they’d been to a conference and heard about a study showing FIT doesn’t improve effectiveness (1). With the rising popularity of FIT around the world, I understand how someone might be rattled by such a claim. And yet, from the outset, I’ve always recommended caution.
In 2012, I wrote about findings reported in the first studies of the ORS and SRS, indicating they were simply, “too good to be true.” Around that same time, I also expressed my belief that therapists were not likely to learn from, nor become more effective as a result of measuring their results on an ongoing basis. Although later proven prophetic (1, 2), mine wasn’t a particularly brilliant observation. After all, who would expect using a stopwatch would make you a faster runner? Or a stethoscope would result in more effective heart surgeries? Silly, really.
What does the evidence indicate?
- The latest, most comprehensive meta-analysis of studies published in the prestigious, peer-reviewed journal, Psychotherapy Research, found that routine use of the ORS and SRS resulted in a small, yet significant impact on outcomes.
- Improving the outcome of care requires more than measurement. If FIT is to have any effect on engagement and progress in care, clinicians must be free of programmatic and structural barriers that restrict their ability to respond in real time to the feedback they receive. As obvious as it may seem, studies in which clinicians measure, but cannot change what they are doing in response show little or no effect (1).
- With one exception, results reported in studies of FIT are confounded by the amount of training therapists receive, and the stage of implementation they (or the agency in which they work) are in, at the time the research is conducted. In many of the investigations published to date, participating therapists received 1 hour of training or less prior to beginning, and no supervision during, the study (1). Consistent with findings from the field of implementation science documenting that productive use of new clinical practices takes from three to five years, a new study conducted in Scandinavia found the impact of FIT grew over time, with few results seen in the first and second year of use. By year four, however, patients were 2.5 times more likely to improve when their therapists used FIT. In short, it takes time to learn how to do FIT, and for organizations to make the structural changes required for the development and maintenance of a feedback culture.
- Improving individual therapist effectiveness requires deliberate practice. It turns out,the best therapists devote twice as much time to the process. More, when employed purposefully and mindfully, the outcomes of average practitioners steadily rise at a rate consistent with performance improvements obtained by elite athletes (Click here if you want to watch an entertaining and informative video on the subject from the recent Achieving Clinical Excellence conference).
Before ending, let me mention one other question that comes up fairly often. “Why don’t you wear shoes when you present?” The picture to the left was taken at a workshop in Sweden last week and posted on Facebook! Over the years, I’ve heard many explanations: (1) it’s a Zen thing; (2) because I’m from California; (3) to make the audience feel comfortable; (3) to show off my colorful socks; and so on.
The truth, it turns out, is like the findings about FIT reported above, much more mundane. Care to guess?
(You can find my answer below)
Until next time,
Scott
Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence
P.S.: Registration for the FIT Intensives is open. As has happened in previous years, we are on track to sell out early. Email me at info@scottdmiller.com with any questions or click on the icons for more information or sign up.
P.P.S: Men’s shoes hurt my feet and back ache. I get neither when walking about in my stocking feet while standing and presenting all day.
Love it Scott. Over the years I did wonder about the reason for no shoes. Turns out the answer is rather mundane.
Maybe similar to many questions in research?
“After all, who would expect using a stopwatch would make you a faster runner? Or a stethoscope would result in more effective heart surgeries? Silly, really.”
Hits the nail on the head. I always look forward to your posts. This one, in particular, answered a question I had – just as other studies or methods have to be closely scrutinized in they don’t always effect the outcome in proportion to it’s marketing claim…how do I know that to be true of FIT.
What I am taking away is that FIT is a tool….the value is in the work the therapist does with the tool.
I agree. As I am engaging with my clients. I inform them that I have acquired tools to assist them, but if I do not provide the right tool and instruction, I can not expect the client to achieve quality results.
I think sometimes thinkers… overthink things.
Just like the socks…
I read somewhere that humans are narrative in nature and we create a story to reduce ambiguity and therefore anxiety.
Scott, try Sanuk Men’s Hemp shoes.
Scott – great article and insights, thanks!
Not interested in talking you out if your sock feet, but as someone who’s mask struggled with men’s shoes for years, I have found Keen Uneek 02s to be the one shoe I can wear all day without getting uncomfortable. They get a little more attention than I’d like, but worth it for the comfort they provide.
Keep it simple: no shoes = comfortable
Hi Scott.
Funny to think that an approach based on actually listening to what decades of evidence tells us, and seeking evidence of outcomes and progress each and every session, has been interpreted as being an ‘anti-evidence’ perspective.
Perhaps it should be evidence-based practiSe instead of evidence-based practiCe, to make sure it is read as a verb, not noun
Crystal
Scott,
Great read! Check out Xeroshoes.com. They follow the evidence about what is good for your feet.
Interesting article and a good follow-up from the meta-analysis article. I suspect that we are seeing less powerful findings because we now have many more people doing FIT (or what they think is FIT) poorly. Early adopters were likely executing it with more specific intent as they weren’t told to do it, they wanted to do it. Many doing it now, as the article said have little training. I have certainly seen anecdotal evidence of that in a number of organisations.
I think that’s part of it Vanessa. The problem is that use of the measures does not equal implementation and integration of the tools into care.
Who is the client? To start with I saw FIT primarily as a good help to therapist and client to talk about the ongoing proces in the therapy room and in clients life. But during the years FIT has – as far as I can see – more and more turned into an instrument that is used by parties (clients) outside the therapy room to control what is going on in therapy and get arguments for saving money, themselves having no clue about therapy as such. Luckily some clients figure out not to score too optimistic if they feel a need for more hours (claim a right to get what they have been promised by the insurance company). As a whole I think this strong controling element from outside in the use of FIT alienates the relation between therapist and the client sitting next to you.
How do you look at this?
With kind regards
Henning
Copenhagen
I have been fascinated by the the ORS and SRS since I first encountered your work in the Heroic Client and The Heart and Soul of Change 14 yrs ago.I have used them in my practice but I have struggled to be consistent and have been frustrated with myself for this. But I think you touch on something here. There was an expectation in early my training that I should focus on allowing the client to unravel their personal narrative, cathart, and receive feedback. All this was theirs and I needed to remain as neutral and aloof from the process as possible. And I think my clients have expected this to a degree as well. It is what they have seen in every movie and TV show depicting therapy. So asking them to reflect on the specific interactions with the therapist felt intensely confrontational/“off script”.
Yet the increased effectiveness of the model in the extended Scandinavian study would make sense in that it allowed both the Therapists and the clients an opportunity to grow into a new way of thinking about therapy. As the participants became more acculturated to this model they could engage it more effectively. If the client expects that giving feedback on the therapy is a “normal” part of the process then integrating the FIT model would make sense.
Of course it raises a question about “teaching” clients how to use therapy effectively and how to incorporate this in a framework such as I work in- private practice- where you are often working under the time pressure of third party payment/managed care.
Appreciate your work! And apologies for a rather lengthy comment
Hey Scott. I’m new to the field and really interested in deliberate practice. I had the opportunity to use FIT during my practicum training – such a great tool.
If you are not an anti-shoe guy, a pair on the market that actually fit like socks are called ‘Chenna Baree.’ I spent years trying to find a shoe that had this level of flexibility and allowed for direct responsiveness to the ground. I use them for the deliberate practice of Kung Fu. They also perhaps serve as a symbol of good therapy