SCOTT D Miller - For the latest and greatest information on Feedback Informed Treatment

  • About
    • About Scott
    • Publications
  • Training and Consultation
  • Workshop Calendar
  • FIT Measures Licensing
  • FIT Software Tools
  • Online Store
  • Top Performance Blog
  • Contact Scott
info@scottdmiller.com 773.404.5130

More Making Sense of Negative Research Results about Feedback Informed Treatment

January 30, 2020 By scottdm 17 Comments

outrageIs it just me or has public discourse gone mad?

A brief perusal of social media largely finds accusation, name calling, and outrage instead of exploration, dialogue and debate.  Not that any of the latter options were ever simple, straightforward, or successful, but somehow, somewhere, taking a stand has replaced extending a hand.

Thus, slightly more than a year ago, I was compared to an ignorant, cult leader by a person — a researcher and proponent of CBT — who’d joined an open discussion about a post of mine on Facebook.  From there, the tone of the exchange only worsened.  Ironically, after lecturing participants about their “ethical duties” and suggesting we needed to educate ourselves, he labelled the group “hostile” and left, saying he was going to “unfriend and block” me.

As I wrote about in my last blogpost, I recently received an email from someone accusing me of “hiding” research studies that failed to support feedback informed treatment (FIT).  Calling it “scandalous,” and saying I “should be ashamed,” they demanded I remove them from my mailing list.  I did, of course, but without responding to the email.

And, therein lies the problem: no dialogue. 

For me, no dialogue means no possibility of growth or change — on my part or other’s.  To be sure, when you are public person, you have to choose cult memberto what and whom you respond.  Otherwise, you could spend every waking moment either feeling bad or defending yourself.  Still, I always feel a loss when this happens.  I like talking, am curious about and energized by different points of view.

That’s why when my Dutch colleague, Kai Hjulsted, posted a query about the same study I’d been accused of hiding, I decided to devote several blogposts to the subject of “negative research results.”  Last time, I pointed out that some studies were confounded by the stage of implementation clinicians were in at the time the research was conducted.  Brattland et al.’s results indicate, consistent with findings from the larger implementation literature, it takes between two and four years to begin seeing results.  Why?  Because becoming feedback-informed is not about administering the ORS and SRS — that can be taught in a manner of minutes — rather, FIT is about changing practice and agency culture.

(By the way, today I heard through the grapevine that a published study of a group using FIT that found no effect has, in its fourth and fifth years of implementation, started to experience fairly dramatic improvements in outcome and retention)

As critical as time and ongoing support are to successful use of FIT, these two variables alone are insufficient for making sense of emerging, apparently unsupportive studies.  Near the end of my original post, I noted needing to look at the the type of design used in most research; namely, the randomized controlled trial or RCT.

RCTIn the evaluation of health care outcomes , the RCT is widely considered the “gold standard” — the best way for discovering the truth.   Thus, when researcher Annika Davidsen published her carefully designed and executed study showing that adding FIT to the standard treatment of people with eating disorders made no difference in terms of retention or outcome, it was entirely understandable some concluded the approach did not work with this particular population.  After all, that’s exactly what the last line of the abstract said, “Feedback neither increased attendance nor improved outcomes for outpatients in group psychotherapy for eating disorders.”

But what exactly was “tested” in the study?

Read a bit further, and you learn participating “therapists … did not use feedback as intended, that is, to individualize the treatment by adjusting or altering treatment length or actions according to client feedback” (p. 491).  Indeed, when critical feedback was provided by the clients via the measures, the standardization of services took precedence, resulting in therapists routinely responding, “the type of treatment, it’s length and activities, is a non-negotiable.”  From this, can we really conclude FIT was ineffective?

More, unlike studies in medicine, which test pills containing a single active ingredient against others that are similar in every way except they are missing that key ingredient, RCTs of psychotherapy test whole treatment packages (e.g., CBT, IPT, EMDR, etc.).  Understanding this difference is critical when trying to make sense of psychotherapy research.

When what is widely recognized as the first RCT in medicine was published in 1948, practitioners could be certain streptomycin caused the loveimprovement in pulmonary tuberculosis assessed in the study.  By contrast, an investigation showing one psychotherapeutic approach works better than a no treatment control does nothing to establish which, if any of, the ingredients in the method are responsible for change.  Consider cognitive therapy (CT).  Many, many RCTs show the approach works.  On this score, there is no doubt.  People who receive it are much better off than those placed on a waiting list or in control groups.  That said, how cognitive therapy works is another question entirely.  Proponents argue its efficacy results from targeting the patterns of “distorted thinking” causally responsible for maladapative emotions and behaviors.  Unfortunately, RCTs were never designed and are not equipped to test such assumptions.  Other research methods must be used — and when they have been, the results have been surprising to say the least.

In my next post, I will address those findings, both as they apply to popular treatment models such as CT and CBT but also, and more importantly, to FIT.

Stay tuned.  In the meantime, I’m interested in your thoughts thus far.

Until then,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

ICCE FIT Implementation Intensive 2020ICCE FIT Masters 2020ICCE FIT Deliberate Practice Intensive 2020

 

 

 

 

 

Filed Under: Feedback Informed Treatment - FIT

Comments

  1. Dustin Tibbitts says

    January 31, 2020 at 8:05 pm

    Scott, you never disappoint. I keep reading because you keep pushing, challenging, thinking, growing. Thanks. It means a great deal.

    Reply
    • scottdm says

      January 31, 2020 at 8:19 pm

      Thank yo Dustin. That means a lot!

      Reply
  2. Evelyn M Field says

    January 31, 2020 at 8:45 pm

    Dear Scott, bullies are everywhere!
    I am over 75 and winding down but enjoy your posts . But I always try to obtain feedback, although in less formal ways than you propose. I believe that so often we miss what our clients need, as well as assessing our therapeutic relationship.
    Moreover, as the lead author of a chapter on “ Diagnosis and treatment “ for workplace bullying targets, ( Springer 2019 ) I believe that my clients require psycho eduction, goal based therapies eg CBT plus process therapy ie the relationship between therapist and client.
    May be it is just my long term clients that require this, but I think that there is a lesson for us all. We need to tailor our work to fit our client, not the other way around, ie we should not be fitting the client to our preferred therapy model.I have seen over 50 years of therapy, too many therapy fads come and go, some more useful or dangerous than others. We need to focus on treatment not fad.

    Evelyn M Field

    Reply
  3. Meghan E Lawless says

    January 31, 2020 at 8:45 pm

    I appreciate you looking at how the studies are performed! So many get stuck on the results. A good practitioner should pay attention to the methodologies!

    Reply
  4. Rob McNeilly says

    January 31, 2020 at 9:02 pm

    Thanks Scott. Jeff Zeig told me years ago tht if he doesn’t get rocks thrown at him he knows it’s time to lift his game. So the rocks thrown at you could indicate that you’re on to something important. I’m a research ignoramus, and try to trsnquilise myself by thinking that 93% of statistics are made up on the spot, and not all things that are important can be measured, and not all things that can be measured are important … so it looks like those rocks are going to keep on happening as you persist in yo/ur quest. Thanks for your determined spirit. Your friend from down under, Rob.

    Reply
  5. Roy Thompson says

    January 31, 2020 at 9:07 pm

    As always you present a balanced view of what the research actually says. Thank you.

    Reply
  6. Andrew Henry says

    January 31, 2020 at 9:55 pm

    Scott,

    Cult, not really. I have done a little bit of research on cults, and I don’t agree with the writer. You challenge and encourage clinicians, Scott. I was recently informed by a lawyer that doing counseling with evidence based therapies is fine, using therapies not evidence based is ok, just don’t call it counseling, and inform the client when it isn’t counseling, and get their consent as such. Just thinking back to your Tarot presentation at the Evolution. Hope this helps.

    Reply
  7. Vanessa Spiller says

    February 1, 2020 at 2:39 am

    Hi supreme eternal leader….errr Scott,

    Such important points! There is such a big difference between using the Ors/srs or any outcome measures and being feedback informed! I am seeing more and more individuals and organisations monitoring their outcomes and so few being feedback informed just like the study you mentioned! When I do training these days on FIT I have really moved away from the science of the measures (most people know that already) to how to use them in a feedback informed way right back at the introducing the measure stage!

    Keep up the good work!

    Vanessa

    Reply
  8. Anders Tvernø Lassen says

    February 1, 2020 at 8:40 am

    Your insights make a real difference for me on an everyday basis – when I get presented with these similar criticisms and concerns from colleagues and managers. Thank you for making sense of complex matter. I’m continuously puzzled by why it brings enormous value to some of us, and enormous aversion to others (and I wonder if there’s any research on that topic, beyond theories of difference in mindsets?). Your work helps me “lean in”.

    Anders

    Reply
  9. Matk says

    February 2, 2020 at 1:48 am

    Hi Scott, thanks for always stimulating conversation about research and practice. It has brought my interest back into the research world again.

    For me, the tension brings up polarised views between needing a clear framework vs being led by client feedback. Ultimately, there needs to be both at play to do therapy. But therapists sit at different points on the continuum of rigidity and chaos when it comes to their therapuetic style. For example, some therapists prefer to have one way of working, and refer clients on when they dont fit their model. While others (like me) prefer to learn many approaches and adapt to the clients needs, while not becoming chaotic in my approach.

    The extreme viewpoints invite siding with feedback only OR model adherence, whereas a Both/And is needed in my opinion.

    Thanks Scott for priviledging client needs and preferences and inviting therapists to grow in their work as a result. This fits with me, while I play with different models to find the right one for each of my clients.

    High regards, Mark

    Reply
  10. Barry McInnes says

    February 2, 2020 at 6:26 pm

    “You remind me of a religious cult, completing ignoring any evidence that seem to question your own beliefs. You seem to be more concerned about protecting your own belief system than to find ways to enhance the field.”

    What a great example of projection! Loved it. Thanks Scott! I needed a laugh after Brexit! :)

    Reply
  11. Susie Winston says

    February 2, 2020 at 10:19 pm

    I have a simple question about Feedback Informed Therapy – Is FIT a specific program using the ORS and SRS? or can an organization consider itself to provide FIT if it uses another measurement system?
    We are implementing a program called Reaching Recovery that has 4 measurement tools. A Level of care tool, a self report measure of the clients experience of their own recovery, and a clinical measure completed by the clinician. The fourth tool is a measure of organizational recovery supports. This system was developed by the Mental Health Center of Denver. Check it out.
    Regards,
    Susie

    Reply
    • scottdm says

      February 3, 2020 at 10:29 am

      Thanks for your note and question Susie. FIT is not specific to the ORS and SRS. The key determinant is whether the measures solicit client feedback and that such information is used to inform services.

      Reply
  12. Ragnhild Smistad says

    February 3, 2020 at 6:08 pm

    Hi Scott, thanks for an interesting blog! I try to read most of your posts. This time I`m trying to give you a little feedback too. though I need google translate:)) I would like to give you support regarding the use of FIT and the importance of a feedback culture.
    I have been working with implementing FIT on my own workplace (In Norway/Trondheim) since 2012. During these years I have never doubted that FIT is a useful tool. And now I see that it is up to myself to make it even better.. One of the first things I tell my colleagues is that FIT is not a Method, but a tool that gives us therapists more insight and teaches us to listen better. FIT engages us to communicate better with clients. It`s a fact I`ve learned from you first, and as I later experienced myself. It has also been said that FIT is not a perfect tool, but it is better than what we have had in the past. Sometimes I have to put the tool away, because it prevents a good collaboration in one way or another. But this no longer worries me either. The most important thing is to be in a dialogue with my clients, and agree on the best possible way to communicate feedback. Implementing a feedback culture is most important. That said, FIT is suitable for most people, and only exceptionally do we, the client and me, choose to put the tool aside. I am not a researcher, nor a professional who spends a lot of time reading different research results, but I have great faith in what I experience in practice. I dare say that FIT has helped me to be a better helper, and I am constantly developing;)

    Best regards
    Ragnhild Smistad

    Reply
    • scottdm says

      February 4, 2020 at 7:45 am

      Thanks Ragnhild for taking the time to share your experience. As you say, “The most important thing is to be in a dialogue with my clients, and agree on the best possible way to communicate feedback. Implementing a feedback culture is most important.” Exactly. I wish you all the best in your work.

      Scott

      Reply
  13. Jon Hook says

    February 15, 2020 at 10:42 pm

    Whenever someone says there is a controversy in psychotherapy (or psychology for that matter) I often think, “In order for the there to be a controversy, don’t people have to talk to each other first?”

    In an ironic twist of fate, psychology is still a field with major communication issues. I think one’s willingness to engage in a protracted conversation about these things is a test of individual character more than anything else. If you’re willing to stick with the debate and to continually struggle with challenging ideas, then it is likely that you actually care about what is true. If not, you’re likely motivated by something else.

    With fire comes heat and light! I don’t think you can have it any other way.

    Reply
  14. Barry Winstead says

    February 17, 2020 at 8:25 pm

    I wish I felt more hopeful, but I don’t see the impasse in the relationship between the “specific ingredients” devotees and the “common factor” oriented folks softening anytime soon. Hmmm … what country’s political/cultural climate does that remind you of?

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

 

SEARCH

Subscribe for updates from my blog.

  

Upcoming Training

Mar
17

Feedback Informed Treatment (FIT) Intensive ONLINE


Mar
22

FIT Supervision Intensive 2021 ONLINE


Mar
30

FIT SPRING CAFÉ


Aug
02

FIT Implementation Intensive 2021


Aug
04

Training of Trainers 2021

FIT Software tools

FIT Software tools

NREPP Certified

HTML tutorial

LinkedIn

Topics of Interest:

  • Behavioral Health (110)
  • behavioral health (4)
  • Brain-based Research (2)
  • CDOI (14)
  • Conferences and Training (67)
  • deliberate practice (27)
  • Dodo Verdict (9)
  • Drug and Alcohol (3)
  • evidence-based practice (65)
  • excellence (61)
  • Feedback (38)
  • Feedback Informed Treatment – FIT (204)
  • FIT (25)
  • FIT Software Tools (12)
  • ICCE (26)
  • Implementation (7)
  • medication adherence (3)
  • obesity (1)
  • PCOMS (11)
  • Practice Based Evidence (38)
  • PTSD (4)
  • Suicide (1)
  • supervision (1)
  • Termination (1)
  • Therapeutic Relationship (8)
  • Top Performance (39)

Recent Posts

  • Developing a Sustainable Deliberate Practice Plan
  • Making Sense of Client Feedback
  • Umpires and Psychotherapists
  • Augmenting the Two-Dimensional Sensory Input of Online Psychotherapy
  • Death of a Friend

Recent Comments

  • Asta on The Expert on Expertise: An Interview with K. Anders Ericsson
  • Michael McCarthy on Culture and Psychotherapy: What Does the Research Say?
  • Jim Reynolds on Culture and Psychotherapy: What Does the Research Say?
  • gloria sayler on Culture and Psychotherapy: What Does the Research Say?
  • Joseph Maizlish on Culture and Psychotherapy: What Does the Research Say?

Tags

addiction Alliance behavioral health brief therapy Carl Rogers CBT cdoi common factors conferences continuing education denmark evidence based medicine evidence based practice Evolution of Psychotherapy excellence feedback feedback informed treatment healthcare holland icce international center for cliniclal excellence medicine mental health meta-analysis Norway NREPP ors outcome measurement outcome rating scale post traumatic stress practice-based evidence psychology psychometrics psychotherapy psychotherapy networker public behavioral health randomized clinical trial SAMHSA session rating scale srs supershrinks sweden Therapist Effects therapy Training