Is it just me or has public discourse gone mad?
A brief perusal of social media largely finds accusation, name calling, and outrage instead of exploration, dialogue and debate. Not that any of the latter options were ever simple, straightforward, or successful, but somehow, somewhere, taking a stand has replaced extending a hand.
Thus, slightly more than a year ago, I was compared to an ignorant, cult leader by a person — a researcher and proponent of CBT — who’d joined an open discussion about a post of mine on Facebook. From there, the tone of the exchange only worsened. Ironically, after lecturing participants about their “ethical duties” and suggesting we needed to educate ourselves, he labelled the group “hostile” and left, saying he was going to “unfriend and block” me.
As I wrote about in my last blogpost, I recently received an email from someone accusing me of “hiding” research studies that failed to support feedback informed treatment (FIT). Calling it “scandalous,” and saying I “should be ashamed,” they demanded I remove them from my mailing list. I did, of course, but without responding to the email.
And, therein lies the problem: no dialogue.
For me, no dialogue means no possibility of growth or change — on my part or other’s. To be sure, when you are public person, you have to choose 
to what and whom you respond. Otherwise, you could spend every waking moment either feeling bad or defending yourself. Still, I always feel a loss when this happens. I like talking, am curious about and energized by different points of view.
That’s why when my Dutch colleague, Kai Hjulsted, posted a query about the same study I’d been accused of hiding, I decided to devote several blogposts to the subject of “negative research results.” Last time, I pointed out that some studies were confounded by the stage of implementation clinicians were in at the time the research was conducted. Brattland et al.’s results indicate, consistent with findings from the larger implementation literature, it takes between two and four years to begin seeing results. Why? Because becoming feedback-informed is not about administering the ORS and SRS — that can be taught in a manner of minutes — rather, FIT is about changing practice and agency culture.
(By the way, today I heard through the grapevine that a published study of a group using FIT that found no effect has, in its fourth and fifth years of implementation, started to experience fairly dramatic improvements in outcome and retention)
As critical as time and ongoing support are to successful use of FIT, these two variables alone are insufficient for making sense of emerging, apparently unsupportive studies. Near the end of my original post, I noted needing to look at the the type of design used in most research; namely, the randomized controlled trial or RCT.
In the evaluation of health care outcomes , the RCT is widely considered the “gold standard” — the best way for discovering the truth. Thus, when researcher Annika Davidsen published her carefully designed and executed study showing that adding FIT to the standard treatment of people with eating disorders made no difference in terms of retention or outcome, it was entirely understandable some concluded the approach did not work with this particular population. After all, that’s exactly what the last line of the abstract said, “Feedback neither increased attendance nor improved outcomes for outpatients in group psychotherapy for eating disorders.”
But what exactly was “tested” in the study?
Read a bit further, and you learn participating “therapists … did not use feedback as intended, that is, to individualize the treatment by adjusting or altering treatment length or actions according to client feedback” (p. 491). Indeed, when critical feedback was provided by the clients via the measures, the standardization of services took precedence, resulting in therapists routinely responding, “the type of treatment, it’s length and activities, is a non-negotiable.” From this, can we really conclude FIT was ineffective?
More, unlike studies in medicine, which test pills containing a single active ingredient against others that are similar in every way except they are missing that key ingredient, RCTs of psychotherapy test whole treatment packages (e.g., CBT, IPT, EMDR, etc.). Understanding this difference is critical when trying to make sense of psychotherapy research.
When what is widely recognized as the first RCT in medicine was published in 1948, practitioners could be certain streptomycin caused the 
improvement in pulmonary tuberculosis assessed in the study. By contrast, an investigation showing one psychotherapeutic approach works better than a no treatment control does nothing to establish which, if any of, the ingredients in the method are responsible for change. Consider cognitive therapy (CT). Many, many RCTs show the approach works. On this score, there is no doubt. People who receive it are much better off than those placed on a waiting list or in control groups. That said, how cognitive therapy works is another question entirely. Proponents argue its efficacy results from targeting the patterns of “distorted thinking” causally responsible for maladapative emotions and behaviors. Unfortunately, RCTs were never designed and are not equipped to test such assumptions. Other research methods must be used — and when they have been, the results have been surprising to say the least.
In my next post, I will address those findings, both as they apply to popular treatment models such as CT and CBT but also, and more importantly, to FIT.
Stay tuned. In the meantime, I’m interested in your thoughts thus far.
Until then,
Scott
Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence
Scott, you never disappoint. I keep reading because you keep pushing, challenging, thinking, growing. Thanks. It means a great deal.
Thank yo Dustin. That means a lot!
Dear Scott, bullies are everywhere!
I am over 75 and winding down but enjoy your posts . But I always try to obtain feedback, although in less formal ways than you propose. I believe that so often we miss what our clients need, as well as assessing our therapeutic relationship.
Moreover, as the lead author of a chapter on “ Diagnosis and treatment “ for workplace bullying targets, ( Springer 2019 ) I believe that my clients require psycho eduction, goal based therapies eg CBT plus process therapy ie the relationship between therapist and client.
May be it is just my long term clients that require this, but I think that there is a lesson for us all. We need to tailor our work to fit our client, not the other way around, ie we should not be fitting the client to our preferred therapy model.I have seen over 50 years of therapy, too many therapy fads come and go, some more useful or dangerous than others. We need to focus on treatment not fad.
Evelyn M Field
I appreciate you looking at how the studies are performed! So many get stuck on the results. A good practitioner should pay attention to the methodologies!
Thanks Scott. Jeff Zeig told me years ago tht if he doesn’t get rocks thrown at him he knows it’s time to lift his game. So the rocks thrown at you could indicate that you’re on to something important. I’m a research ignoramus, and try to trsnquilise myself by thinking that 93% of statistics are made up on the spot, and not all things that are important can be measured, and not all things that can be measured are important … so it looks like those rocks are going to keep on happening as you persist in yo/ur quest. Thanks for your determined spirit. Your friend from down under, Rob.
As always you present a balanced view of what the research actually says. Thank you.
Scott,
Cult, not really. I have done a little bit of research on cults, and I don’t agree with the writer. You challenge and encourage clinicians, Scott. I was recently informed by a lawyer that doing counseling with evidence based therapies is fine, using therapies not evidence based is ok, just don’t call it counseling, and inform the client when it isn’t counseling, and get their consent as such. Just thinking back to your Tarot presentation at the Evolution. Hope this helps.
Hi supreme eternal leader….errr Scott,
Such important points! There is such a big difference between using the Ors/srs or any outcome measures and being feedback informed! I am seeing more and more individuals and organisations monitoring their outcomes and so few being feedback informed just like the study you mentioned! When I do training these days on FIT I have really moved away from the science of the measures (most people know that already) to how to use them in a feedback informed way right back at the introducing the measure stage!
Keep up the good work!
Vanessa
Your insights make a real difference for me on an everyday basis – when I get presented with these similar criticisms and concerns from colleagues and managers. Thank you for making sense of complex matter. I’m continuously puzzled by why it brings enormous value to some of us, and enormous aversion to others (and I wonder if there’s any research on that topic, beyond theories of difference in mindsets?). Your work helps me “lean in”.
Anders
Hi Scott, thanks for always stimulating conversation about research and practice. It has brought my interest back into the research world again.
For me, the tension brings up polarised views between needing a clear framework vs being led by client feedback. Ultimately, there needs to be both at play to do therapy. But therapists sit at different points on the continuum of rigidity and chaos when it comes to their therapuetic style. For example, some therapists prefer to have one way of working, and refer clients on when they dont fit their model. While others (like me) prefer to learn many approaches and adapt to the clients needs, while not becoming chaotic in my approach.
The extreme viewpoints invite siding with feedback only OR model adherence, whereas a Both/And is needed in my opinion.
Thanks Scott for priviledging client needs and preferences and inviting therapists to grow in their work as a result. This fits with me, while I play with different models to find the right one for each of my clients.
High regards, Mark
“You remind me of a religious cult, completing ignoring any evidence that seem to question your own beliefs. You seem to be more concerned about protecting your own belief system than to find ways to enhance the field.”
What a great example of projection! Loved it. Thanks Scott! I needed a laugh after Brexit!
I have a simple question about Feedback Informed Therapy – Is FIT a specific program using the ORS and SRS? or can an organization consider itself to provide FIT if it uses another measurement system?
We are implementing a program called Reaching Recovery that has 4 measurement tools. A Level of care tool, a self report measure of the clients experience of their own recovery, and a clinical measure completed by the clinician. The fourth tool is a measure of organizational recovery supports. This system was developed by the Mental Health Center of Denver. Check it out.
Regards,
Susie
Thanks for your note and question Susie. FIT is not specific to the ORS and SRS. The key determinant is whether the measures solicit client feedback and that such information is used to inform services.
Hi Scott, thanks for an interesting blog! I try to read most of your posts. This time I`m trying to give you a little feedback too. though I need google translate:)) I would like to give you support regarding the use of FIT and the importance of a feedback culture.
I have been working with implementing FIT on my own workplace (In Norway/Trondheim) since 2012. During these years I have never doubted that FIT is a useful tool. And now I see that it is up to myself to make it even better.. One of the first things I tell my colleagues is that FIT is not a Method, but a tool that gives us therapists more insight and teaches us to listen better. FIT engages us to communicate better with clients. It`s a fact I`ve learned from you first, and as I later experienced myself. It has also been said that FIT is not a perfect tool, but it is better than what we have had in the past. Sometimes I have to put the tool away, because it prevents a good collaboration in one way or another. But this no longer worries me either. The most important thing is to be in a dialogue with my clients, and agree on the best possible way to communicate feedback. Implementing a feedback culture is most important. That said, FIT is suitable for most people, and only exceptionally do we, the client and me, choose to put the tool aside. I am not a researcher, nor a professional who spends a lot of time reading different research results, but I have great faith in what I experience in practice. I dare say that FIT has helped me to be a better helper, and I am constantly developing;)
Best regards
Ragnhild Smistad
Thanks Ragnhild for taking the time to share your experience. As you say, “The most important thing is to be in a dialogue with my clients, and agree on the best possible way to communicate feedback. Implementing a feedback culture is most important.” Exactly. I wish you all the best in your work.
Scott
Whenever someone says there is a controversy in psychotherapy (or psychology for that matter) I often think, “In order for the there to be a controversy, don’t people have to talk to each other first?”
In an ironic twist of fate, psychology is still a field with major communication issues. I think one’s willingness to engage in a protracted conversation about these things is a test of individual character more than anything else. If you’re willing to stick with the debate and to continually struggle with challenging ideas, then it is likely that you actually care about what is true. If not, you’re likely motivated by something else.
With fire comes heat and light! I don’t think you can have it any other way.