“Everyone understands how a toilet works, don’t you?” ask cognitive scientists Sloman and Fernbach.
The answer, according to their research, is likely no. Turns out, peoples’ confidence in their knowledge far outstrips their ability to explain how any number of simple, every day items work — a coffeemaker, zipper, bicycle and yes, a toilet. More troubling, as complexity increases, the problem only worsens. Thus, if you struggle to explain how glue holds two pieces of paper together — and most, despite being certain they can, cannot — good luck accounting for how an activity as complicated as psychotherapy works.
So pronounced is our inability to recognize the depth of our ignorance, the two researchers have given the phenomenon a name: the “Illusion of Explanatory Depth.” To be sure, in most instances, not being able to adequately and accurately explain isn’t a problem. Put simply, knowing how to make something work is more important in everyday life than knowing how it actually works:
- Push the handle on the toilet and the water goes down the drain, replaced by fresh water from the tank;
- Depress the lever on the toaster, threads of bare wire heat up, and the bread begins to roast;
- Replace negative cognitions with positive ones and depression lifts.
Simple, right?
Our limited understanding serves us well until we need to build or improve upon any one of the foregoing. In those instances, lacking true understanding, 
we could literally believe anything — in the case of the toilet, a little man in the rowboat inside the tank makes it flush — and be just as successful. While such apparent human frailty might, at first pass, arouse feelings of shame or stupidity, truth is operating on a “need to know” basis makes good sense. It’s both pragmatic and economical. In life, you cannot possibly, and don’t really need to know everything.
And yet, therein lies the paradox: we passionately believe we do. That is, until we are asked to provide a detailed, step-by-step, scientifically sound accounting — only then, does humility and the potential for learning enter the picture.
When research on routine outcome monitoring (ROM) first began to appear, the reported impact on outcomes was astonishing. Some claimed it was the most important development in the field since the invention of psychotherapy! They were also quite certain how it worked: like a blood test, outcome and alliance measures enabled clinicians to check progress and make adjustments when needed. Voila!
Eight years ago, I drew attention to the assertions being made about ROM, warning “caution was warranted. ” It was not a bold statement, rather a reasoned one. After all, throughout the waning decades of the last millennium and into the present, proponents of cognitive (CT) and cognitive behavioral therapy (CBT) had similarly overreached, claiming not only that their methods were superior in effect to all others, but that the mechanisms responsible were well understood. Both proved false. As I’ve written extensively on my blog, CT and CBT are no more effective in head to head comparisons with other approaches. More, studies dating back to 1996 have not found any of the ingredients, touted by experts as critical, necessary to success (1, 2, 3).
That’s why I was excited when researchers Mikeal, Gillaspy, Scoles, and Murphy (2016) published the first dismantling study of the Outcome and Session Rating Scales, showing that using the measures in combination, or just one or the other, resulted in similar outcomes. Some were dismayed by these findings. They wrote to me questioning the value of the tools. For me, however, it proved what I’d said back in 2012, “focusing on the measures misses the point.” Figure out why their use improves outcomes and we stop conflating features with causes, and are poised to build on what most matters.
On this score, what do the data say? When it comes to feedback informed treatment, two key factors count:
- The therapist administering the measures; and
- The quality of the therapeutic relationship.
As is true of psychotherapy-in-general, the evidence indicates that who uses the scales is more important that what measures are used (1, 2). Here’s what we know:
- Therapists with an open attitude towards getting feedback reach faster progress with their patients;
- Clinicians able to create an environment in which clients provide critical (e.g., negative) feedback in the form of lower alliance scores early on in care have better outcomes (1, 2); and
- The more time a therapists spend consulting the data generated by routinely administering outcome and alliance measures, the greater their growth in effectiveness over time.
In terms of how FIT helps, two lines of research are noteworthy:
- In a “first of its kind” study, psychologist Heidi Brattland found that the strength of the therapeutic relationship improved
more over the course of care when clinicians used the Outcome and Session Rating Scales (ORS & SRS) compared to when they did not. Critically, such improvements resulted in better outcomes for clients, ultimately accounting for nearly a quarter of the effect of FIT. - Brattland also found therapists, “significantly differed in the influence of … [FIT] on the alliance, in the influence of the alliance on outcomes, and the residual direct effect of [FIT] … posttreatment” (p. 10). Consistent with other studies, such findings indicate routine measurement can be used to identify a clinician’s “growth edge” — what, where, and with whom — they might improve their ability to relate to and help the diverse clients met in daily work. Indeed, the combination of FIT, use of aggregate data to identify personal learning objectives, and subsequent engagement in deliberate practice has, in the only study in history of psychotherapy to date, been shown to improve effectiveness at the individual practitioner level.
“Inch by inch, centimeter by centimeter,” I wrote back in 2012, “the results of [new] studies will advance our understanding and effectiveness.” I’m hopeful that the discussion in this and my two prior posts (1, 2) will help those interested in improving their results avoid the vicious cycle of hope and despair that frequently accompanies new ideas in our field, embracing the findings and what they can teach us rather than looking for the next best thing.
Until next time,
Scott
Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence
P.S: The March Intensives are sold out. Register now for the summer trainings to ensure your spot:
As a therapist who has read superficially about FIT and is interested in deliberate practice, it has been my assumption that if I dug in further I would find that many of my worries about the application of the ORS and SRS were addressed in specific training with them. Perhaps some of the push-back I’ve seen you write about is due to a similarly shallow understanding?
I really enjoyed the analogy of a toilet. This also really resonates with me “subsequent engagement in deliberate practice has, in the only study in history of psychotherapy to date, been shown to improve effectiveness at the individual practitioner level.” Kind of scary when you think about it.
I’m looking forward to your book! I’ve found out about FIT/ROM through your blog and I’m very eager to learn more, thank you for great posts!
Great stuff. Looking forward to listening to your new podcast.
I just listened to the podcast. It was a great reminder that deliberate practice is not a noun. I think it has been easy for me to treat it as a noun rather than verb. Also, I was intrigued by your initial statement of concern with deliberate practice focused on specific models. I would love to understand a bit more about this.
Very excited to read the new book as soon as I can get my hands on it!
I love reading and listening to your presentations. Your content is always thought provoking and interesting. Thanks for what you do.
I find myself discussing with my colleagues over and over again about how psychotherapy works and what frustrates me the most is the lack of interest in this topic. I found that a majority of psychotherapists around me don’t even know or care about the common factors or the client’s preferences, not to mention that they are quite rigid in their views about what deliberate practice is. So, for me, your work is a mine gold and I’m thankful for your constant effort. I look forward to learn more about FIT.
“focusing on the measures misses the point.” Figure out why their use improves outcomes and we stop conflating features with causes, and are poised to build on what most matters.
– I find the above quotes from the post to account for all things therapy. When the relationship is established, tools take a backseat. The combination of the relationship and more of what works for that individual is how we aide/bear witness to/guide progress in the lives of those we serve.
I appreciate these blog posts and your willingness to tackle some of the challenges that come up with FIT. Starting to use FIT more consistently in my practice has challenged me in ways I did not expect. It’s one thing to believe you’re really open to feedback and another thing to be faced with negative feedback and figure out how to learn from it gracefully! An uncomfortable, but necessary spot to be in for growth.
Just pre-ordered your new book “Deliberate Practice”. Can’t wait til it arrives.
Working at my growing edge is scary but the other side of this is excitement. I appreciate what you are saying and look forward to reading “Better Results” when it comes out.
I listened to you and Dr. Daryl Chow presentation about the “Psychotherapy’s Fatal Flaw and How to Fix It”. I gained a lot off insights on how we can become more effective by identifying our deficit and seeking coaching/supervision to overcome this deficit. Most of counselors may think only of their effectiveness ignoring their weakness.
Thank, you for sharing.
I was part of training of the trainers years ago, and I am constantly awed by the consistency of the messaging around outcomes-right up to your newest interview with Daryl Chow; it’s not about the measures. But it’s ALL about the measures
Just listened to your podcast “Psychotherapy’s Fatal Flaw and How to Fix It”. 12 years later, after attending the 2008 workshop in Chicago, and after 16 years of utilizing the ORS/SRS and consistently incorporating feedback into my work, the essence of identifying growing edges out of each of the potential factors contributing to positive change and outcomes was never more clear. I’ve gotten lost on chasing down one or the other of the factors, but this synthesis was quite elegant in its practical simplicity, yet daunting in its own way. Thank you!!
I have followed your work for years and look forward to the book. I was trained as a pastoral psychotherapist and clinical educator with the association for clinical pastoral education which included a baseline of 4 years full time residency and with about 9 hours a week of intensive group and individual supervision for my clinical practice as an educator for spiritual care and my role as a psychotherapist. I continued in many hours of individual supervision to achieve a certification that no longer exists; AAPC Diplomate. Our training included verbatim, co-visitation and tapes of our work. While I have learned many modalities and theories, the most important was the individual accurate and often very painful feedback on my individual ways to avoid, resist and obscure the relationships. I agree with you on so much of what you are saying which is subversive in psychotherapy. I hope your upcoming book and work will include a small section to promote local organizing of people who are interested in improving therapy, not just participating in the programs, trainings and consultation that you offer. Thank you for your work!
I just finished listening to the podcast “Psychotherapy’s Fatal Flaw and once again am in awe and so intrigued by your work. Years ago, I recall being taught about the importance of the relationship and now, as the podcast host noted, I get constant emails about if I just learn X technique, then somehow that will be the missing link that will make all the difference in my effectiveness with clients.
I appreciated the analogy of the technique being the vehicle and the vehicle not going anywhere without first deciding what are we doing and where are we going. I am also trained as a coach and as I was reflecting on your discussions of the 4 pillars, (Individualized learning objectives, feedback, successive refinement, role of coach to guide) goal consensus, relationship factors, therapist factors and hope and expectancy, I realized that the coach training I took was quite helpful in exploring these areas and how rarely they have come up in any therapy training I have been to. Thanks to both of you for highlighting and researching their enormous value in the therapeutic relationship and outcomes!
I do enjoy your blog. I did enjoy Psychotherapy’s Fatal Flaw And How to Fix It.
thank you for your work.
I just posted the comment I’m including below on Clearly Clinical blog, but wanted to share the impact of this podcast on some professional development decisions with you, because I love seeing you at Evolution of Psychotherapy, but I’m deciding my time and money will be better utilized in pursuing learning FIT, ROM, and Deliberate Practice this year and I appreciate the accessibility of the upcoming Deliberate Practice Book, the eLearning course, and I finally purchased the FIT manuals as a starting point. I truly believe 1) I have an open attitude about receiving feedback, 2) I can improve creating an environment in which clients provide critical feedback, and 3) I am willing to spend time consulting the data generated by routinely administering outcome and alliance measures (once I learn how to do that). I feel excited (and a little scared) and hopeful!
comment about podcast- This podcast was so thought (and practice) provoking for me. Elizabeth’s comments in the interview about clinicians’ limited time really nailed it for me. I work for a nonprofit agency with too much paperwork and too little time (and money), and I often set goals that are within that realm of “clinical knowledge” Daryl described, such as “read this treatment manual about Adults with ADHD and start an ADHD group,” or even “go to Evolution of Psychotherapy conference to learn from experts in the field and choose area for professional development to work on until the next Ev of Psych.” I went to my first Evolution of Psychotherapy as a grad student in 2014 and then again in 2018 and have been trying to figure out the time and finances to go in 2020… I love that experience, but while listening to this I’m realizing I spend close to $2,000 (of my own money) attending that conference, all the while telling myself I can’t afford to attend a FIT or Deliberate Practice training in Chicago, and likely not even pursuing such training, worried about the time, money, and vulnerability of embarking on this journey. After listening to this podcast, I find myself rethinking my strategies. I just purchased the FIT manuals. I will preorder the Deliberate Practice book. I may have missed the contest for the e-learning course (?), but I’m so grateful this has been put on my radar as a way to pursue learning to use FIT. $179 will be affordable to me, and I feel more prepared to dedicate my time, energy, and finances in learning to use FIT, ROM, and Deliberate Practice. Thank you.
Mary, email me at: scottdmiller@talkingcure.com and I’ll send you a discount code to attend this summer’s trainings in Chicago!
S
I’ve just started learning about FIT and I have to say that at first I was intimidated by the idea due to the possibility of receiving negative feedback. But I put ego to the side and dug deep into what the fear was and discovered that it’s more imposter syndrome than anything else. Now I’m of the mindset that FIT is a growth tool and can take someone from good to great.