Nearly two decades have passed since I met Saul Rosenzweig at his home in St. Louis, Missouri.  He was well into his nineties and still working every day.  Truth is, I was surprised to learn he was still alive!

In 1936, he’d penned an article –three and a half pages in total– that became one of the most often cited in psychotherapy research.  He was only 29 years of age at the time.

Then, as now, Rosenzweig’s central premise is controversial: all psychotherapy approaches, regardless of their specific components, produce equivalent outcomes.

Although he didn’t coin the phrase, his observation has since been referred to as, “The Dodo Verdict.”  That’s because he began the article quoting a line uttered by the Dodo from Alice’s Adventures in Wonderland, “Everyone has won, and all must have prizes.”

Over the last eight decades, many have claimed to “beat the dodo verdict” — to have developed an approach more effective than others.   You know them by acronym: CBT, EMDR, ACT, CRT, MI, TFT, SFBT, CDOI, and so on.   Initial research is always promising.  Nevertheless, as I’ve detailed in many blogs over the years, later studies invariably find the “new and improved” is no more effective than the “tried and true” — whatever the accepted standard might be at that moment (1, 2, 3, 4).

The field’s focus on methods is understandable.  The assumption is psychotherapy works like medicine.  To be effective, an approach must contain ingredients specifically remedial to the disorder being treated.  No one questions whether antibiotics are better than aspirin for strep throat.  Why?  Because the former contains an ingredient that kills the bacteria responsible for the infection.  In a similar way, CBT  is widely believed to work because its methods target the root cause of depression, dysfunctional thoughts.

So critical are the techniques of the various psychotherapy approaches believed to be, developers create protocols and manuals for insuring they are delivered correctly.  Professional, regulatory, and funding bodies (e.g., American Psychological Association, National Institute for Health and Care Excellence [NICE], National Institute of Mental Health) have embraced and, in some instances, mandated their use.

But what do the data say?

In 2005, my colleagues and I reviewed the available evidence and concluded, “Although training in manualized psychotherapies does enhance therapist learning of and technical competence in a given approach, there is no relationship between such manuals and outcome.”

And now, a new, updated study.  Briefly, researchers Truijens, Zühlke‐van Hulzen, and Vanheule, conducted a systematic review of the literature — six studies directly comparing manualized and nonmanualized psychotherapy, and nine meta-analyses.  Their conclusion?  “Manualized treatment is not empirically supported … [and] should not be promoted as being superior to nonmanualized psychotherapy.”  It’s Dodo come back life.

What can a mental health professional do to improve their effectiveness?

Here again, the data point the way to finally “beating the Dodo.”  It involves a change of focus.  Instead of learning the latest treatment approach, work on becoming a more effective version of you.  The process is known as deliberate practice.  It begins by creating a detailed map of your clinical performance; specifically, measuring your results, and then using the information to identify opportunities for professional growth.Research to date documents gradual growth in effectiveness consistent with performance improvements obtained by elite athletes.

Want to learn more?  Click here for a free article–actually, the chapter on the subject from our latest book, The Cycle of Excellence.   Still interested?  Watch the recent interview I did on the subject with YouTube blogger, Chris Dorsano.

That’s it for now.  Best for the Holidays,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

In my travels each year, I meet 1,000’s of clinicians–professionals who truly want to help others, and are willing to try almost anything to do so.

That’s why I always “lean in” whenever one expresses concern about the rising popularity of using formal measures of progress and the therapeutic relationship to inform and improve the quality and effectiveness of behavioral health services.

The clinicians I meet are usually aware of the research behind the call to incorporate client feedback into care: (1) many people who begin in treatment (~50 to 65%), leave without experiencing a measured improvement in their functioning or well being; and (2) randomized controlled trials show that doing so improves outcomes and reduces costs (1, 2).

Still, they wonder, “What do clients think?” “Do they consider filling out the forms helpful, or a burden?”

Those asking the question have a point. After all, the measures must be used every session with every client, despite studies to date showing the effects of asking for feedback are largely confined to those not making progress. Said another way, many asked to complete measures (between 35-50%), will experience no benefit from participating in the process.

When answering this question, I’ve always relied on what I’ve been able to glean from available studies (1, 2) and my personal experience. “Rarely do my client’s object,” I say, adding, “Neither have other clinicians reported that their clients consider it a burden.” And the research to date, although sparse, largely confirms that experience.

Overall, clients report that using measures is both positive and beneficial, enabling them to: (1) play an active in the care they receive; (2) maintain a focus on what matters most to them; and (3) collaborate more effectively with their treatment provider.

That said, problems have been noted. Chief among these are therapists failing to explain the nature and purpose of the scales (e.g., validity, confidentiality of the data, rationale) as well as not using the resulting feedback to guide service delivery (e.g., increase responsiveness to individual client needs, goals, and preferences; promote greater reflection and collaboration between therapist and client; address problems in the therapeutic relationship; and change the nature, frequency, type or provider of services).

Neither of these findings are particularly surprising. If there’s anything more frustrating than being forced to complete seemingly irrelevant paperwork, it is being asked for feedback and then having it ignored. That said, if the results are so damned obvious, “Why do the problems keep happening?”

The answer, in a word, is: integration.

Learning how to administer progress and alliances measures is simple. Instructions for giving, scoring, and plotting the Outcome and Session Rating Scales, literally takes less than 90 seconds. Making the tools an integral part of one’s clinical work–instead of bookends marking the beginning and end of each visit–is an entirely different matter. Back in May, I blogged about a new study showing that such integration takes time, between two to four years. Once done, however, the results are impressive, with clients whose therapists actively use measures two and a half times more likely to experience improvement.

Integrating feedback tools into your clinical style is the subject of the upcoming, three-day “Advanced Intensive” training on Feedback-Informed Treatment in Chicago.

We go far beyond the basics, giving you concrete strategies and skills for engaging clients in the feedback process, and using the resulting information to improve the quality and outcome of the services you offer.

In 2013, the methods you will learn were vetted by the National Registry and deemed evidence-based. As in prior years, the training is on track to sell out early. Click here, or on either of the banners below, to register or learn more.

Until next time,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

It’s just two weeks ago.  I was on a call with movers and shakers from a western state.  They were looking to implement Feedback Informed Treatment (FIT)–that is, using measures of progress and the therapeutic relationship to monitor and improve the quality and outcome of mental health services.

I was in the middle of reviewing the empirical evidence in support of FIT when one of the people on the call broke in.  “I’m a little confused,” they said hesitantly, “I thought you were the anti-evidence-based practice guy.”

It’s not the first time I’ve been asked this question.  In truth, it’s easy to understand why some might believe this about me.  For more than two decades, I have been a vocal critic of the idea that certain treatments are more effective for some problems than others.  Why?  Because of the evidence!  Indeed, one of the most robust findings over the last 40 years is that all approaches work equally well.

Many clinicians, and a host of developers of therapeutic approaches, mistakenly equate the use of a given model with evidence-based practice.  Nothing could be further from the truth.  Evidence-based practice is a verb not a noun.

According to American Psychological Association and the Institute of Medicine, there are three components: (1) using the best evidence; in combination with (2) individual clinical expertise; while ensuring the work is consistent with (3) patient values and expectations.  “FIT,” I responded, “not only is consistent with, but operationalizes the definition of evidence-based practice, providing clinicians with reliable and valid tools for identifying when services need to be adjusted in order to improve the chances of achieving a successful outcome.”

Here’s another recent question: “I’ve read somewhere that FIT doesn’t work.”  When I inquired further, the asker indicated they’d been to a conference and heard about a study showing FIT doesn’t improve effectiveness (1).  With the rising popularity of FIT around the world, I understand how someone might be rattled by such a claim.  And yet, from the outset, I’ve always recommended caution.

In 2012, I wrote about findings reported in the first studies of the ORS and SRS, indicating they were simply, “too good to be true.”  Around that same time, I also expressed my belief that therapists were not likely to learn from, nor become more effective as a result of measuring their results on an ongoing basis.   Although later proven prophetic (1, 2), mine wasn’t a particularly brilliant observation.  After all, who would expect using a stopwatch would make you a faster runner?  Or a stethoscope would result in more effective heart surgeries?  Silly, really.

What does the evidence indicate?

• The latest, most comprehensive meta-analysis of studies published in the prestigious, peer-reviewed journal, Psychotherapy Research, found that routine use of the ORS and SRS resulted in a small, yet significant impact on outcomes.

• Improving the outcome of care requires more than measurement.  If FIT is to have any effect on engagement and progress in care, clinicians must be free of programmatic and structural barriers that restrict their ability to respond in real time to the feedback they receive.  As obvious as it may seem, studies in which clinicians measure, but cannot change what they are doing in response show little or no effect (1).

• With one exception, results reported in studies of FIT are confounded by the amount of training therapists receive, and the stage of implementation they (or the agency in which they work) are in, at the time the research is conducted.  In many of the investigations published to date, participating therapists received 1 hour of training or less prior to beginning, and no supervision during, the study (1).  Consistent with findings from the field of implementation science documenting that productive use of new clinical practices takes from three to five years, a new study conducted in Scandinavia found the impact of FIT grew over time, with few results seen in the first and second year of use.  By year four, however, patients were 2.5 times more likely to improve when their therapists used FIT.  In short, it takes time to learn how to do FIT, and for organizations to make the structural changes required for the development and maintenance of a feedback culture.

• Improving individual therapist effectiveness requires deliberate practice.   It turns out, the best therapists devote twice as much time to the process.  More, when employed purposefully and mindfully, the outcomes of average practitioners steadily rise at a rate consistent with performance improvements obtained by elite athletes (Click here if you want to watch an entertaining and informative video on the subject from the recent Achieving Clinical Excellence conference).

Before ending, let me mention one other question that comes up fairly often. “Why don’t you wear shoes when you present?”  The picture to the left was taken at a workshop in Sweden last week and posted on Facebook!  Over the years, I’ve heard many explanations: (1) it’s a Zen thing; (2) because I’m from California; (3) to make the audience feel comfortable; (3) to show off my colorful socks; and so on.

The truth, it turns out, is like the findings about FIT reported above, much more mundane.  Care to guess?

(You can find my answer below)

Until next time,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

P.S.: Registration for the FIT Intensives is open.  As has happened in previous years, we are on track to sell out early.  Email me at info@scottdmiller.com with any questions or click on the icons for more information or sign up.

P.P.S: Men’s shoes hurt my feet and back ache.  I get neither when walking about in my stocking feet while standing and presenting all day.