[Read more…]

Chances are your are carrying a smartphone–maybe you’re even reading this post on your Android or Iphone!  One thing I’m almost certain of is that the device you own–can’t live without–is not a Nokia.

The nearly complete absence of the brand is strange.  Not long ago, the company dominated the mobile phone market.  At one time, seventy percent of phones in consumers’ hands were made by Nokia.

“And then,” to quote Agathe Christie, “there were none!”

Today, Nokia’s global market share is an anemic 3 %.

What happened?  Here, the answer is no mystery.  It was not a lack of position, talent, innovative ideas, or know-how.  Rather, the company failed at implementation.  Instead of rapidly adapting to changing conditions, it banked on its brand name and past success to carry it through.  Vague considerations trumped concrete goals.  Spreadsheets and speeches replaced communication, consensus-building, and commitment.   The moral of the story?  No matter how successful the brand or popular the product, implementation is hard.

Nowhere is this truer than in healthcare.  Change is not only constant but accelerating. Each week, hundreds of research findings are published.  Just as frequently, new technologies come online. All have the potential to do good, to improve the quality and outcome of treatment.

Research to date, for example, documents that seeking ongoing, formal feedback from those receiving behavioral health services as much as triples the effectiveness of the care offered, while simultaneously cutting the rate of drop out by 50%, and decreasing the risk of deterioration by 33%.  Enough evidence has amassed to warrant the approach–known as, “Feedback Informed Treatment”– being listed on the National Registry of Evidence-Based Programs and Practices.

Any yet, despite the massive amount of time and resources, agencies and practitioners devote to staying “up-to-date,” most implementation efforts struggle, and far too many fail–according to the available evidence, about 70-95% (a figure equivalent to the number of start-up businesses in the United States that belly up annually).

In their chapter in the new book, Feedback Informed Treatment in Clinical Practice, Randy Moss and Vanessa Mousavizadeh, provide step-by-step instructions, based on the latest research and real-world experience, for creating an organizational culture that supports implementation success.  Recently, I had a chance to talk with Randy about the chpater.  Whether you’ve got the book or not, I think you’ll find the knowledge and experience in the video below, helpful:

Online version of the Fidelity Readiness Index and Fidelity Measure

In the meantime, while on the subject of implementation, here’s a cool, electronic version of a tool you can use to track the progress of your efforts.  It helps identify where you and your organization are in the process as well as identify and set goals in order to remain on track (it’s one of the reasons reviewers at SAMHSA gave our application perfect marks for implementation support).  Thanks to my Danish colleague, Rasmus Kern for developing and making it available.  By the way, the program contains both English and Danish versions (we’ll be releasing more languages soon).

Take the FRIFM Assessment

Until next time,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

How difficult could it be? One scale to assess progress, a second to solicit the client’s perception of the therapeutic relationship. Each containing four questions, administration typically takes between 30 to 60 seconds.

Since first being developed 23 years ago, scores of randomized-controlled and naturalistic studies have found the use of simple tools in care significantly improves outcome (1, 2).

And yet, while both simple and effective, research shows integrating the measures into care is far from easy. First came studies documenting that implementing feedback-informed care took time — up to three years for agencies to see results (3). It’s why we developed and have been offering a two-day itensive training on the subject for more than a decade — the only one of its kind. The next one is scheduled for January (click here for more information or to register). We outline the evidence-based steps and help managers, supervisors, team leads and staff develop a plan.

Other studies show while the vast majority of clients have highly favorable reactions to the use of the scales in care, they do have questions. In a recently survey of 13 clients in a private practice setting, Glenn Stone and colleagues (4) reported 70% were surprised at being asked for feedback! More than half found it helpful to see their progress represented graphically from week to week, roughly the same number who felt the scales helped them identify and maintain the right focus in sessions. At the same time, a handful reported feeling confused about the some of the questions; specifically, how to answer an item (e.g., social well being) when it contained multiple descriptors (e.g., school, work and friendship) — each of which could be answered differently.

Such a fantastic question! One we have addressed, among many others, at every three-day, Feedback Informed Treatment Intensive since 2003 (the next one scheduled back-to-back with the Implementation training in January). Seeing the question in this research report made me think I needed to do a “FIT tip” video for those who’ve started using the measures but have yet to attend.

So what is the “best practice” when clients ask how to complete a question which contains multiple descriptors?

The evidence shows that using standardized measures to solicit feedback from clients regarding progress and their experience of the working relationship improves retention and outcome.

How much? By 25% (1)

And now, major news out of California.

Psychologists — who are required to earn 36 hours of continuing education credits every other year — can now earn credit towards renewal of their license by implementing feedback-informed treatment (FIT) in their daily clinical work (2)!

Here’s the challenge. The same body of research documenting that FIT works, shows clinicians struggle when it comes to putting it into practice. Private practitioners express concerns about the time it takes. Those working in agencies talk about the challenges of finding alternatives when FIT data indicate the present course of treatment is not working. Finally, many who start, stop after a short while, noting that FIT didn’t add much beyond their “clinical knowledge and experience.”

All such concerns are real. Indeed, as reported a few years back on this blog (3), implemention takes time and skill — in agency settings, up to three years of effort, support and training before the benefits of being feedback-informed begin to materialize. When they do, however, clients are 2.5 times more likely to benefit from care.

So, don’t give-up. Instead, upskill!

In September, the International Center for Clinical Excellence is sponsoring the FIT “Training of Trainers.” Held only once every-other-year, the TOT focuses exclusively on the process of training and supporting others in their use of feedback informed treatment. As with all ICCE events, space is limited to 40 participants. Click on the link above or icon below for more information or to register!

Until next time,

Scott

Scott D. Miller, Ph.D.
Director, ICCE

It’s a question that comes up at some point in most trainings on feedback-informed treatment (FIT):

“Can I use FIT with all my clients?”

Having encountered it many times, I now have a pretty good sense of the asker’s concerns. Given our training as mental health professionals, we think in terms of diagnosis, treatment approach and service delivery setting.

Thus, while the overall question is the same, the particulars vary:

“Does FIT work (with people in recovery for drug and alcohol problems, those with bipolar illness or schizophrenia, in crisis settings, group or family therapy, long term, inpatient or residential care)?

Occasionally, someone will report having read a study indicating FIT either did not work or made matters worse when used with certain types of clients (1) or in particular settings (2).

While I have posted detailed responses to specific studies published over the last couple of years (3), the near limitless number of ways questions of efficacy can be parsed ultimately renders such an approach clinically and pragmatically useless. Think of the panoply of potential parameters one would need to test. The latest edition of the Diagnostic and Statistical Manual of Mental Disorders alone contains 297 diagnoses. Meanwhile, more than a decade ago, Scientific American identified 500 different types of psychotherapy. If history is a reliable guide, the number of approaches has most certainly increased by now. Add to that an ever evolving number of settings in which treatment takes place — most recently online — and the timeline for answering even the most basic questions quickly stretches to infinity.

So, returning to the question, “how best to decide whether to use FIT with ‘this or that’ client, treatment approach, or service setting?”

That’s the subject of my latest FIT TIP: Does FIT work with all Clients?

As promised, every 10 days or so, I’ve released a brief video providing practical guidance for optimzing your use of feedback in treatment. To date, these have addressed:

How to solicit clinically useful feedback

How to use client feedback to improve effectiveness

Identifying the best electronic FIT outcome system for you

What to do when your client is not making progress

Should you start FIT with established clients

OK, that’s it for now! Until next time,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

P.S.: Registration for the next FIT Implementation Online intensive is open. If you or your agency are thinking about or beginning to implement FIT, this is the event to attend. In this highly interactive training, you will learn evidence-based steps for ensuring success.

I know, I know. You’re thinking, “A post about the next pandemic?!”

Some will insist, “We’re not done with the current one!” Others will, with the wave of a hand counter, “I’m so tired of this conversation, let’s move on. How about sushi for lunch?”

Now, however, is the perfect time to assess what happened and how matters should be handled in the next global emergency.

Scientists are already on the case. A recent study conducted by the University of Manchester and Imperial College London analyzed the relative effects of different non-pharmacological interventions aimed at controlling the spread of COVID-19. The investigation is notable for both its scope and rigor, analyzing data from 130 countries and accounting for date and strictness of implementation! The results are sure to surprise you. For example, consistent with my own, simple analysis of US state-by-state data published in the summer of 2020, researchers found, “no association between mandatory stay-at-home interventions on cross-country Covid-19 mortality after adjusting for other non-pharmaceutical interventions concurrently introduced.” Read for yourself what approaches did make a difference.

One non-pharmacological intervention that was not included in the analysis was the involvement of “human factors” experts — sociologists, anthropologists, psychologist, implementation scientists — in the development and implementation of COVID mitigation efforts. Indeed, those leading the process treated the last two years like a virus problem rather than a human management problem. The results speak for themselves. Beyond the division and death, the US is experiencing a dramatic mental health crisis, especially among our nation’s youth.

Which brings me to the latest edition of The Book Case — the podcast I do together with my friend and colleague Dr. Dan Lewis. In it, we consider two books with varying perspectives on the outbreak of COVID-19. As acknowledged at the outset of this post, I understand you may already have made up your mind. Whatever you’ve decided, I believe these two books will give you pause to reconsider and refine your thinking.

That’s all for now. Until next time,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Effectiveness

“Why don’t more therapists do FIT?” a grad student asked me during a recent consultation.  Seated nearby in the room were department managers, supervisors, and many experienced practitioners.

“Well,” I said, queuing up my usual, diplomatic answer, “Feedback informed treatment is a relatively new idea, and the number of therapists doing it is growing.”

Unpersuaded, the student persisted, “Yeah, but with research showing such positive results, seems like ethically everyone should be doing FIT.  What’s all the hesitance about?”

What’s that old expression?  Out of the mouths of babes . . .

Truth is, a large, just released study showed FIT — specifically, the routine monitoring of outcome and relationship with the Outcome and Session Rating Scales — improved effectiveness by 25% over and above usual treatment services (1).

TWENTY-FIVE PERCENT!

In a second, pilot study conducted in a forensic psychiatric setting, use of the ORS and SRS dramatically reduced dropout rates (2).

What other clinical practice/technique can claim similar impacts on outcome and retention in mental health services?

Needless to say, perhaps, the student’s comments were more pointed.  Use of FIT at the agency was decidedly uneven.  Despite being a “clinical standard” for more than two years, many on staff — practitioners and supervisors alike — were not using the tools, or had started and then, just as quickly, stopped.

Here’s where the recent study might offer some help.  The impact of FIT notwithstanding, researchers Bram Bovendeerd and colleagues found its use in routine practice was easily derailed.   In their own words, they observe “implementation is challenging … and requires a careful plan of action.”

Even then, fate can intervene.

In their next paper, they describe how, even when organizational culture is receptive to FIT, contextual variables can get in the way.   At one clinic, for example, it was the unexpected illness of a key staff member leaving everyone else to take up the slack.  Curiously, when asked to explain the decline in use of the measures that followed, the therapists did not cite the increase in workload.  Rather, in what appears to be a classic example of attempting to reduce cognitive dissonance — we know using the measures work, but we’re not doing it anyway — they developed and expressed doubts about the validity of the measures!  Anyway, loads more interesting insights in the interview (below) I did with the lead researcher not long ago.

We’ll be addressing these and other implementation challenges at the next FIT Implementation coming up in August.  Registration is open.  Generally, the training sells out a month or more in advance.  Click here for more information or to register.

Until next time, please share your thoughts in a comment.

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

Deliberate Practice is hot.  Judging from the rising number of research studies, workshops, and social media posts, it hard to believe the term did not appear in the psychotherapy literature until 2007.

The interest is understandable.  Among the various approaches to professional development — supervision, continuing education, personal therapy — the evidence shows deliberate practice is the only one to result in improved effectiveness at the individual therapist level.

Devoting time to rehearsing what one wants to improve is hardly a novel idea.  Any parent knows it to be true and has said as much to their kids.  Truth is, references to enhancing one’s skills and abilities through focused effort date back more than two millennia.   And here is where confusion and misunderstanding begin.

• Clinical practice is not deliberate practice.  If doing therapy with clients on a daily basis were the same as engaging in deliberate practice, therapists would improve in effectiveness over the course of their careers.  Research shows they do not.  Instead, confidence improves.  Let that sink in.  Outcomes remain flat but confidence in our abilities continuously increases.  It’s a phenomenon researchers term “automaticity” — the feeling most of us associate with having “learned” to do something –where actions are carried out without much conscious effort.  One could go so far as to say clinical practice is incompatible with deliberate practice, as the latter, to be effective, must force us to question what we do without thinking.
• Deliberate practice is not a special set of techniques.  The field of psychotherapy has a long history of selling formulaic approaches. Gift-wrapped in books, manuals, workshops, and webinars, the promise is do this — whatever the “this” is — and you will be more effective.  Decades of research has shown these claims to be empty.  By contrast, deliberate practice is not a formula to be followed, but a form.  As such, the particulars will vary from person to person depending on what each needs to learn.  Bottom line: beware pre-packaged content.
• Applying deliberate practice to mastering specific treatment models or techniques.  Consider a recent study out of the United Kingdom (1).  There, like elsewhere, massive amounts of money have been spent training clinicians to use cognitive behavioral therapy (CBT).  The expenditure is part of a well-intentioned government program aimed at improving access to effective mental health services (2).  Anyway, in the study, clinicians participated in a high intensity course that included more than 300 hours of training, supervision, and practice.  Competence in delivering CBT was assessed at regular intervals and shown to improve significantly throughout the training.  That said, despite the time, money, and resources devoted to mastering the approach, clinician effectiveness did not improve.  Why?  Contrary to common belief, competence in delivering specific treatment protocols contributes a negligible amount to the outcome of psychotherapy.  As common sensical as it likely sounds, to have an impact, whatever we practice must target factors have leverage on outcome.

My colleague, Daryl Chow and I, have developed a tool for helping practitioners develop an effective deliberate practice plan.  Known as the “Taxonomy of Deliberate Practice Activities” (TDPA), it helps you identify aspects of your clinical performance likely to have the most impact on improving your effectiveness.   Step-by-step instructions walk you through the process of assessing your work, setting small individualized learning objectives, developing practice activities, and monitoring your progress.   As coaching is central to effective deliberate practice, a version of the tool is available for your supervisor or coach to complete.  Did I mention, its free?  Click here to download the TDPA contained in the same packet as the Outcome and Session Rating Scales.  While your are at it, join our private, online discussion group where hundreds of clinicians around the world meet, support one another, and share experiences and ideas.

OK, that’s it for now.  All the best,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

 
P.S.: Improve your ability to deliver effective presentations online or in-person at the upcoming “Training of Trainers” workshop.  This event is held only once, every two years — this time, online!

“We don’t do ‘treatment,’ can we use FIT?”

It’s a question that comes up with increasing frequency as use of the Outcome and Session Rating Scales in the helping professions spreads around the globe and across diverse service settings.

When I answer an unequivocal, “yes,” the asker often responds as though I’d not heard what they said.

Speaking slowly and enunciating, “But Scott, we don’t do “t r e a t m e n t.‘”  Invariably, they then clarify, “We do child protection,” or “We’re not therapists, we are case managers,” or providers in any of a large number of supportive, criminal justice, or other statutory social services.

How “treatment” became synonymous with psychotherapy (and other medical procedures) is a mystery to me.   The word, as Merriam-Webster defines it, is merely the way we conduct ourselves — our specific manner, actions and behaviors — towards others.

With this definition in mind, working “feedback-informed” simply means interacting with people as though their experience of the service is both primary and consequential.  The challenge, I suppose, is how to do this when lives may be at risk (e.g., child protection, probation and parole), or when rules and regulations prescribe (or proscribe) provider and agency actions irrespective of how service users feel or what they prefer.

Over the last decade, many governmental and non-governmental organizations have succeeded in making statutory services feedback-informed — and the results are impressive.  For recipients, more engagement and better outcomes.  For providers, less burnout, job turnover, and fewer sick days.

I had the opportunity to speak with the members and managers of one social service agency — Gladsaxe Kommune in Denmark — this last week.   They described the ups, downs, and challenges they faced — including retraining staff, seeking variances to existing laws from authorities, — while working to transform agency practice and culture.  If you work in this sector, I know you’ll find their experience both inspiring and practical.  You can find the video below.  Another governmental agency has created a step-by-step guide (in English) for implementing feedback informed treatment (FIT) in statutory service settings.  It’s amazingly detailed and comprehensive.  It’s also free.  To access, click here.

Cliff note version of the results of implementing FIT in statutory services?

• 50% fewer kids placed outside the home
• 100% decrease in complaints filed by families against social service agencies and staff
• 100% decrease in staff turnover and sick days

OK, that’s it for now.  Please leave a comment.  If you, or your agency, is considering implementing FIT, please join us for the two-day intensive training in August.  This time around, you can participate without leaving home as the entire workshop will be held online.  For more information, click on the icon below.

All the best,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

Criticizing umpires is as much a part of watching baseball as eating hotdogs and wearing team jerseys on game day.  The insults are legion, whole websites are dedicated to cataloging them:

“Open your eyes!”

“Wake up, you are missing a great game!”

“Your glasses fogged up?”

“Have you tried eating more carrots?”

“I’ve seen potatoes with better eyes!”

“Hey Ump, how many fingers am I holding up?

Are you “seeing” a common theme here?

And interestingly, the evidence indicates fans have reason to question the judgement and visual acuity of most umpires.  A truly massive study of nearly 4 million pitches examined the accuracy of their calls over 11 regular seasons.  I didn’t know this, but it turns out, all major league stadiums are equipped with fancy cameras which track every ball thrown from mound to home plate.  Using this data, researchers found “botched calls and high error rates are rampant.”   How many you ask?  A staggering 34,246 incorrect calls in the 2018 season alone!  It gets worse.  When the pressure was on — a player at bat, for example, with two strikes — umpire errors skyrocket, occurring nearly one-third of the time.  Surely, the “umps” improve with time an experience?  Nope.  In terms of accuracy, youth and inexperienced win out every time!

Now, let me ask, are your “ears burning” yet?

Turns out, umpires and psychotherapists share some common traits.  So, for example, despite widespread belief to the contrary, clinicians are not  particularly good at detecting deterioration in clients.  How bad are we?  In one study, therapists correctly identified clients who worsen in their care a mere two-and-a-half percent of the time (1)!  Like umpires, “we call ’em as we see ’em.”  We just don’t see them.  And if you believe we improve with experience, think again.  The largest study in the history of research on the subject — 170 practitioners treating 6500 clients over a 5 year period — reveals that what is true of umpires applies equally to clinicians.  Simply put, on average, our outcomes decline the longer we are in the field.

If you are beginning to feel discouraged, hold on a minute.  While the data clearly show umpires make mistakes, the same evidence documents most of their calls are correct.  Similarly, therapists working in real world settings help the majority of their clients achieve meaningful change — between 64 and 74% in our database of thousands of clinicians and several million completed treatment episodes.

Still, you wouldn’t be too far “off base” were you to conclude, “room for improvement exists.”

Truth is, umpires and therapists are calling “balls and strikes” much the same way they did when Babe Ruth and Alfred Alder were key players.  Solutions do exist.  As you might guess, they are organized around using feedback to augment and improve individual judgement ability.  So far, major league baseball (and its umpires) has resisted.  In psychotherapy, evidence shows clients of therapists who formally and routinely solicit feedback regarding the quality of the therapeutic relationship and progress over time are twice as likely to experience improvement in treatment.

The measures are free for practitioners to use and available in 25+ languages.  If you don’t have them, click here to register.   You’ll likely need some support in understanding how to use them effectively.  Please join the conversation with thousands of colleagues from around the world in the ICCE Discussion Forum.  If you find yourself wanting to learn more, click on the icon below my name for information about our next upcoming intensive — online, by the way!

What more is there to say, except: BATTER UP!

Until next time, wishes for a safe and healthy Holiday season,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

Bad therapy.

Are you guilty of it?  A quick internet search turned up only 15 books on the subject.  It’s strange, especially when you consider that between 5 and 10% of clients are actually worse off following treatment and an additional 35-40% experience no benefit whatsoever!  (Yep, that’s nearly 50%)

And what about those numerous “micro-failures.”  You know the ones I’m talking about?  Those miniature ruptures, empathic missteps, and outright gaffs committed during the therapy hour.  For example, seated opposite your client, empathic look glued to your face and suddenly you cannot remember your client’s name?  Or worse, you call them by someone else’s.  The point is, there’s a lot of bad therapy.

Why don’t we therapists talk about these experiences more often?  Could it possibly be that we don’t know?  Four years on, I can still remember the surprise I felt when Norwegian researcher, Jorgen Flor, found most therapists had a hard time recalling any clients they hadn’t helped.

One group does know — and recently, they’ve been talking their experiences!  The Very Bad Therapy podcast is one of my favorites.  After listening to sixty-some-odd episodes of clients exposing our shortcomings, I reached out to the podcast’s two fearless interviewers, clinicians Ben Fineman and Carrie Wiita, to learn what had motivated them to start the series in the first place and what, if anything, they’d learned along the way.  Here’s what I promise: they have no shame (and its a good thing for us they don’t)!

OK, that’s it for now.  Until next time, all the best,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

P.S.: For the first time ever, we’re offering the FIT Implementation Intensive Online.  It’s one of the four courses required for certification as an ICCE FIT Trainer.  As with our “in-person” events, we have an international faculty and strictly limit the number of participants to 40 to ensure the highest quality experience.  Click here for more information or to register.

“Everyone understands how a toilet works, don’t you?” ask cognitive scientists Sloman and Fernbach.

The answer, according to their research, is likely no.  Turns out, peoples’ confidence in their knowledge far outstrips their ability to explain how any number of simple, every day items work — a coffeemaker, zipper, bicycle and yes, a toilet.   More troubling, as complexity increases, the problem only worsens.  Thus, if you struggle to explain how glue holds two pieces of paper together — and most, despite being certain they can, cannot — good luck accounting for how an activity as complicated as psychotherapy works.

So pronounced is our inability to recognize the depth of our ignorance, the two researchers have given the phenomenon a name: the “Illusion of Explanatory Depth.”  To be sure, in most instances, not being able to adequately and accurately explain isn’t a problem.  Put simply, knowing how to make something work is more important in everyday life than knowing how it actually works:

• Push the handle on the toilet and the water goes down the drain, replaced by fresh water from the tank;
• Depress the lever on the toaster, threads of bare wire heat up, and the bread begins to roast;
• Replace negative cognitions with positive ones and depression lifts.

Simple, right?

Our limited understanding serves us well until we need to build or improve upon any one of the foregoing.  In those instances, lacking true understanding, we could literally believe anything — in the case of the toilet, a little man in the rowboat inside the tank makes it flush — and be just as successful.   While such apparent human frailty might, at first pass, arouse feelings of shame or stupidity, truth is operating on a “need to know” basis makes good sense.  It’s both pragmatic and economical.  In life, you cannot possibly, and don’t really need to know everything.

And yet, therein lies the paradox: we passionately believe we do.  That is, until we are asked to provide a detailed, step-by-step, scientifically sound accounting — only then, does humility and the potential for learning enter the picture.

When research on routine outcome monitoring (ROM) first began to appear, the reported impact on outcomes was astonishing.  Some claimed it was the most important development in the field since the invention of psychotherapy!  They were also quite certain how it worked: like a blood test, outcome and alliance measures enabled clinicians to check progress and make adjustments when needed.  Voila!

Eight years ago, I drew attention to the assertions being made about ROM, warning “caution was warranted. ” It was not a bold statement, rather a reasoned one.   After all, throughout the waning decades of the last millennium and into the present, proponents of cognitive (CT) and cognitive behavioral therapy (CBT) had similarly overreached, claiming not only that their methods were superior in effect to all others, but that the mechanisms responsible were well understood.  Both proved false.  As I’ve written extensively on my blog, CT and CBT are no more effective in head to head comparisons with other approaches.  More, studies dating back to 1996 have not found any of the ingredients, touted by experts as critical, necessary to success (1, 2, 3).

That’s why I was excited when researchers Mikeal, Gillaspy, Scoles, and Murphy (2016) published the first dismantling study of the Outcome and Session Rating Scales, showing that using the measures in combination, or just one or the other, resulted in similar outcomes.  Some were dismayed by these findings.  They wrote to me questioning the value of the tools.  For me, however, it proved what I’d said back in 2012, “focusing on the measures misses the point.”  Figure out why their use improves outcomes and we stop conflating features with causes, and are poised to build on what most matters.

On this score, what do the data say?  When it comes to feedback informed treatment, two key factors count:

1. The therapist administering the measures; and
2. The quality of the therapeutic relationship.

As is true of psychotherapy-in-general, the evidence indicates that who uses the scales is more important that what measures are used (1, 2).  Here’s what we know:

• Therapists with an open attitude towards getting feedback reach faster progress with their patients;
• Clinicians able to create an environment in which clients provide critical (e.g., negative) feedback in the form of lower alliance scores early on in care have better outcomes (1, 2); and
• The more time a therapists spend consulting the data generated by routinely administering outcome and alliance measures, the greater their growth in effectiveness over time.

In terms of how FIT helps, two lines of research are noteworthy:

• In a “first of its kind” study, psychologist Heidi Brattland found that the strength of the therapeutic relationship improved more over the course of care when clinicians used the Outcome and Session Rating Scales (ORS & SRS) compared to when they did not.  Critically, such improvements resulted in better outcomes for clients, ultimately accounting for nearly a quarter of the effect of FIT.
• Brattland also found therapists, “significantly differed in the influence of … [FIT] on the alliance, in the influence of the alliance on outcomes, and the residual direct effect of [FIT] … posttreatment” (p. 10).  Consistent with other studies, such findings indicate routine measurement can be used to identify a clinician’s “growth edge” — what, where, and with whom — they might improve their ability to relate to and help the diverse clients met in daily work.  Indeed, the combination of FIT, use of aggregate data to identify personal learning objectives, and subsequent engagement in deliberate practice has, in the only study in history of psychotherapy to date, been shown to improve effectiveness at the individual practitioner level.

“Inch by inch, centimeter by centimeter,” I wrote back in 2012, “the results of [new] studies will advance our understanding and effectiveness.”  I’m hopeful that the discussion in this and my two prior posts (1, 2) will help those interested in improving their results avoid the vicious cycle of hope and despair that frequently accompanies new ideas in our field, embracing the findings and what they can teach us rather than looking for the next best thing.

Until next time,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

P.S: The March Intensives are sold out.  Register now for the summer trainings to ensure your spot:

Is it just me or has public discourse gone mad?

A brief perusal of social media largely finds accusation, name calling, and outrage instead of exploration, dialogue and debate.  Not that any of the latter options were ever simple, straightforward, or successful, but somehow, somewhere, taking a stand has replaced extending a hand.

Thus, slightly more than a year ago, I was compared to an ignorant, cult leader by a person — a researcher and proponent of CBT — who’d joined an open discussion about a post of mine on Facebook.  From there, the tone of the exchange only worsened.  Ironically, after lecturing participants about their “ethical duties” and suggesting we needed to educate ourselves, he labelled the group “hostile” and left, saying he was going to “unfriend and block” me.

As I wrote about in my last blogpost, I recently received an email from someone accusing me of “hiding” research studies that failed to support feedback informed treatment (FIT).  Calling it “scandalous,” and saying I “should be ashamed,” they demanded I remove them from my mailing list.  I did, of course, but without responding to the email.

And, therein lies the problem: no dialogue. 

For me, no dialogue means no possibility of growth or change — on my part or other’s.  To be sure, when you are public person, you have to choose to what and whom you respond.  Otherwise, you could spend every waking moment either feeling bad or defending yourself.  Still, I always feel a loss when this happens.  I like talking, am curious about and energized by different points of view.

That’s why when my Dutch colleague, Kai Hjulsted, posted a query about the same study I’d been accused of hiding, I decided to devote several blogposts to the subject of “negative research results.”  Last time, I pointed out that some studies were confounded by the stage of implementation clinicians were in at the time the research was conducted.  Brattland et al.’s results indicate, consistent with findings from the larger implementation literature, it takes between two and four years to begin seeing results.  Why?  Because becoming feedback-informed is not about administering the ORS and SRS — that can be taught in a manner of minutes — rather, FIT is about changing practice and agency culture.

(By the way, today I heard through the grapevine that a published study of a group using FIT that found no effect has, in its fourth and fifth years of implementation, started to experience fairly dramatic improvements in outcome and retention)

As critical as time and ongoing support are to successful use of FIT, these two variables alone are insufficient for making sense of emerging, apparently unsupportive studies.  Near the end of my original post, I noted needing to look at the the type of design used in most research; namely, the randomized controlled trial or RCT.

In the evaluation of health care outcomes , the RCT is widely considered the “gold standard” — the best way for discovering the truth.   Thus, when researcher Annika Davidsen published her carefully designed and executed study showing that adding FIT to the standard treatment of people with eating disorders made no difference in terms of retention or outcome, it was entirely understandable some concluded the approach did not work with this particular population.  After all, that’s exactly what the last line of the abstract said, “Feedback neither increased attendance nor improved outcomes for outpatients in group psychotherapy for eating disorders.”

But what exactly was “tested” in the study?

Read a bit further, and you learn participating “therapists … did not use feedback as intended, that is, to individualize the treatment by adjusting or altering treatment length or actions according to client feedback” (p. 491).  Indeed, when critical feedback was provided by the clients via the measures, the standardization of services took precedence, resulting in therapists routinely responding, “the type of treatment, it’s length and activities, is a non-negotiable.”  From this, can we really conclude FIT was ineffective?

More, unlike studies in medicine, which test pills containing a single active ingredient against others that are similar in every way except they are missing that key ingredient, RCTs of psychotherapy test whole treatment packages (e.g., CBT, IPT, EMDR, etc.).  Understanding this difference is critical when trying to make sense of psychotherapy research.

When what is widely recognized as the first RCT in medicine was published in 1948, practitioners could be certain streptomycin caused the improvement in pulmonary tuberculosis assessed in the study.  By contrast, an investigation showing one psychotherapeutic approach works better than a no treatment control does nothing to establish which, if any of, the ingredients in the method are responsible for change.  Consider cognitive therapy (CT).  Many, many RCTs show the approach works.  On this score, there is no doubt.  People who receive it are much better off than those placed on a waiting list or in control groups.  That said, how cognitive therapy works is another question entirely.  Proponents argue its efficacy results from targeting the patterns of “distorted thinking” causally responsible for maladapative emotions and behaviors.  Unfortunately, RCTs were never designed and are not equipped to test such assumptions.  Other research methods must be used — and when they have been, the results have been surprising to say the least.

In my next post, I will address those findings, both as they apply to popular treatment models such as CT and CBT but also, and more importantly, to FIT.

Stay tuned.  In the meantime, I’m interested in your thoughts thus far.

Until then,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

A ship’s captain who successfully sails through a strait at night learns nothing, and adds nothing, to their knowledge of the world.

(Please hang with me.  I promise this post will not be a long, metaphysical rant).

Returning to the example.  As paradoxical as it may strike one at first blush, a captain whose ship founders on the rocks while sailing through the strait both learns and adds to their knowledge.  As philosopher Ernst von Glasersfeld once opined, “The only aspect of that ‘real’ world that actually enters into the realm of experience is its constraints.”

The principle identified by von Glasersfeld applies not only to life lessons, but also to scientific advancement and, of course, feedback informed treatment (FIT).  Indeed, identifying and learning from “constraints” — that is, when things go wrong — is the very purpose of FIT.

It’s why, for example, when a client refuses to complete the outcome and alliance measures, my first impulse is to “lean in” and explore their reasons, rather than instantly set the scales aside.   It’s also why I’m most intrigued by studies which find that FIT fails to improve outcome (1, 2).  In both instances, my curiosity is piqued.  “Finally,” I think, “a chance to learn something …”.   Doing so, cognitive science has long shown, is not as easy or straightforward as simply adjusting our beliefs in light of new facts.  Quite to the contrary.

We are prone to see what we expect, fit the “different” into our current way of viewing the world or ignore it altogether.  One brief example before turning attention to FIT (aka Routine Outcome Monitoring [ROM]).  For most of the history of the field, the failure to engage in and respond to psychological intervention has been attributed to a host of client variables (e.g., degree or type of dysfunction, poor attachment history, IQ, etc.).  Therapists, for their part, have been held accountable for making the correct diagnosis and administering the right treatment.

And yet, despite continuous growth in the size of the DSM, and number of treatment approaches, no improvement in the outcome of psychotherapy has occurred in the last 50 years — a fact I first talked publicly about in 2014 and which über-researchers James Prochaska and John Norcross finally acknowledged in the most recent issue of the American Psychologist.  While some have argued that the field’s flat outcomes indicate the effectiveness therapy has reached a natural limit, an alternate point of view is that we should consider looking beyond the current ways of thinking about what matters most in successful treatment.

On this score, one possibility has been staring the field in the face for decades: the impact of the individual therapist on outcome.  Research has long shown, for example, that who does the treatment contributes 5 to 9 times more to outcome than the type of therapy, psychiatric diagnosis, or client history.  The same body of evidence documents some practitioners are consistently more effective than others.  When researcher Scott Baldwin and colleagues looked into why, they found 97% of the difference was attributable to therapist variability in the alliance.  Said another way, more effective therapists are able to establish a strong working relationship with a broader and more diverse group of clients.  I hope you’re seeing new possibilities for improving effectiveness.  If you’re a regular reader of my blog, you already know my colleagues and I published the only study to date documenting that a focus on therapist development via routine outcome measurement, feedback, and deliberate practice improves both agency and individual practitioner outcomes.

Turning to FIT, in my first post of the New Year, I talked about the strong sense of “anticipointment” I felt when thinking about the future of our field.  A colleague from the Netherlands, Kai Hjulsted, responded, saying he’d been having the same feeling about FIT!  The source, he said, was a study by a Dutch researcher conducted in a crisis intervention setting which, “contrary to expectations,” found, “Patients with psychiatric problems and severe distress seeking emergency psychiatric help did not benefit from direct feedback.”

I was well aware of this study, having served on the researcher’s dissertation committee.  And over the last decade, multiple studies have been published showing little or no benefit from feedback (e.g., 1, 2, 3).

How to make sense of such findings?  Having spoken with numerous practitioners (and even some researchers), I can tell you the tendency is to fit the results into our current way of viewing the world.  So, seen through a traditional medicopsychiatric lens, the inevitable conclusion is FIT does not work with people with certain, specific diagnosis (e.g., severe distress, in crisis, or those with eating disorders).  Such a conclusion makes no sense, however, if the totality of evidence is considered.  Why?  Because the results are decidedly mixed.  Thus, in one study, FIT makes a difference with people in crisis, in another it does not.  With one group of “severely distressed” clients, feedback appears to make matters worse, with another, chances of improvement increase 2.5 times.

What then can we conclude?

An answer begins to emerge as soon as we’re able to get beyond thinking of FIT as just one more in a long list of treatment methods rather than a fundamental, organizing principle of agency and practice culture.  As is hopefully obvious, learning to administer measurement scales takes little time.

Cultural change, by contrast, is a much longer process.  How long?  Norwegian researcher Heidi Brattland and colleagues found it took three years ongoing training and support to successfully implement FIT.  Had they stopped to evaluate, like all other studies to date, after an average of 4 hours of instruction, no impact on outcomes would have been recorded.

While its now clear that time and support are critical to successful implementation, these two variables alone are not sufficient to make sense of emerging, apparently unsupportive studies of FIT.  Addressing such findings requires we look at the type of design used in most research: the randomized controlled trial.   That I’ll do in my next post, in particular addressing two, top notch, well-executed studies many have assumed show FIT is not effective in psychological care for people with eating disorders and severe distress.

And so, as I asked at the outset, please “hang with me.”

Until next time,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

“I heard Scott Miller say it,” the man sitting next to me said.

“Really?” I responded, somewhat incredulous.  After all, I didn’t recall ever saying such a thing.  More to the point, it’s just not something I would say.  Its wrong.  Then again, it was clear he didn’t know that I was Scott Miller.  Not wanting to disrupt the presenter before me at the conference, I’d quietly snuck into the room, dressed in my “civies,” shortly after the meeting had started.

“Yep,” he replied, his voice full of certainty, “as I understand it, he says it all the time.”  Making “air” quote marks with his fingers, “If the client is not changing within three visits, you should terminate the treatment.”

You should have seen the look on his face when later our eyes met as I took to the stage to do my presentation.

On many occasions over the years, I’ve heard people quote me saying things I’ve never said.  Here’s a common one:

“Scott Miller says all you need for successful treatment is a good therapeutic relationship.”

Just so you know, I’ve never said this.  Ever. And yet, once, after I corrected a nationally known practitioner in front of a large audience, he nonetheless repeated it in his newsletter.  Here’s what I have said and continue to say:

• All treatment models work equally well.  All.
• Of all the factors affecting outcome, the specific approach used has the smallest impact.  By comparison, the relationship between client and therapist contributes eight to nine times more. 

Another:

“Scott Miller says therapists help 80% of their clients.”

Once again, I’ve never said this.  In this instance, the misquote is more understandable.  Here’s what I do say at almost every workshop:

• Decades of research and hundreds of study document psychotherapy works.
• The effectiveness of psychotherapy has remained fairly stable over the years.
• In most studies, the average treated client is better off than 80% of the untreated comparison sample.

I’m certain it’s the last of these statements that causes problems.  Presented, as it, is in “researchspeak,” it’s easy to misunderstand.  Read it again and you’ll see it does not mean we help 80% of our clients.  Rather, its about the advantage therapy offers relative to receiving no treatment at all.  By contrast, the percentage we help — as I’ve blogged about on numerous occasions — is actually around 50%.

Now, in the interest of fairness, let me mention something I often say but have never heard misquoted.  At nearly of all my workshops, I joke, “The ORS and SRS may not be the best scales ever developed … but they are free!”  The latter part of that statement is absolutely true.  Clinicians wishing to solicit feedback from clients about their progress and the quality of the therapeutic relationship simply need to register for a free paper and pencil license.  That said, recent research out of the University of Nottingham is giving me pause about the other half.

Just last week, I interviewed Professor Sam Malins who, together with a team of others, has been studying the use of the scales in real world clinical settings for a number of years.  In addition to replicating a number of important findings (e.g., the longer you use the ORS and SRS, the more impact they have on retention and effectiveness; spending time gathering information for diagnosis and treatment planning results in poorer outcomes, the scales can be used to identify skill development opportunities), he also found the Outcome Rating Scale predicted …  oh, just watch the video yourself.  That way, I won’t be misquoted!

Until next time,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence

Remember these?

Did you ever own or try one?

Remember Beanie Babies?  According to one news story, interest was such, “People neglected other areas of their lives to spend all day trading, and some even invested their children’s college funds in toys that they believed would bring an astronomical return on investment.”

And how about advertising executive Gary Dahl’s product?  You remember him, right?  In the 1970’s, he became an overnight millionaire selling rocks marketed as pets.  Rocks!  Waterbeds, Crocs, cause-branded plastic wristbands, oxygen bars, Pogs, Silly Bandz, and the Macarena — the list is as endless as our attention span is short.

If you’ve been in practice for a while, you know the field of mental health is not immune to fad and fashion.  Like past diet crazes, the drugs, diagnoses, and treatment methods that capture professional interest and then quickly disappear can be hard to remember.  Thus, in the 1980’s it was Xanax, multiple personality disorder, and satanic cult abuse.  The 1990’s brought us the “decade of the brain,” borderline personality disorder, and a flood of Prozac and cognitive-behavior therapy knockoffs.  More recently, mindfulness, energy meridians, and trauma have become the objets de grand intérêt.

One more trend is Feedback-Informed Treatment (FIT).  Known also as Routine Outcome Monitoring, Patient Focused Research, and Measurement-based care, it involves using scales to solicit feedback from clients regarding their experience in treatment and using the resulting information in real time to improve quality and outcome.  The weight of the scientific evidence is such that professional and regulatory bodies in the U.S. and abroad have already deemed ROM a “standard” of care in the delivery of psychological services and clinical supervision (American Psychological Association Presidential Task Force on Evidence-based Practice, 2006; Association of State and Provincial Psychology Boards, 2019; Joint Commission, 2018; Tasca et al., 2019).

But is it just another fad?

Data from two recent surveys suggest FIT is moving beyond the “innovation” to the “mass adoption” phase among mental health providers and treatment agencies.  The study conducted by Myoutcomes® targeted members of Facebook-related psychotherapy interest groups and other sources, finding fairly dramatic growth in: (1) awareness of the empirical support for using measures to assess progress and the quality of the therapeutic relationship; and (2) experience with standardized measurement tools in psychotherapy.

Whether such results indicate FIT will stick around long enough to be the “game changer” remains to be seen.  What is known for sure is that, while important, awareness of, experience with, and research support for the process are insufficient to sustain the interest.  Research shows, for instance, successful implementation of FIT takes significant time, planning, and support, without which between 70 and 95% of efforts fail.  Why?  Because working feedback-informed is about changing culture, not using measurement scales in treatment.  Success requires that all members of management — from administration to clinical supervisors — understand and are actively involved in implementation.  Indeed, when practitioners rather than a team led by a manager are held accountable, the likelihood of FIT being a game changer plummets (80% versus 14 %).

Bottom line?  PLEASE don’t download the ORS and SRS and begin experimenting — testing it out to see “if it works.”  As I blogged about last week, the likelihood of failure is simply too high.   Instead, bring your team to our upcoming Spring intensives in Chicago.  You’ll not only “rub shoulders” with colleagues from around the world and our international faculty, but also leave with a thorough grounding in FIT, as well as skills for transforming the culture in which you work.

Until next time,

Scott

Director, International Center for Clinical Excellence

A couple of weeks ago I received an email from the leader of a group asking me to send them copies of the ORS and SRS. “We are to start using these straight away,” the person wrote.

I replied, of course, providing a link to my website where the scales could be downloaded along with a brief note, highlighting the Feedback Informed Treatment (FIT) Manuals and the upcoming March Intensives in Chicago.  “Both are great resources,” I said, “for learning how to use the measures to improve the quality and outcome of behavioral health services.”

I received a quick and polite response, thanking me for the links but going on to say, “I’ve seen the scales. They are really very simple and self-explanatory so I don’t think we really need much in the way of training or support materials.”

I’d like to say I was surprised. After all, what medical professional would say something similar?  Say, about a stethoscope?  To wit, “No thanks, seems pretty simple, stick these thingies in my ears, and the other end on the patient’s chest and listen…”.

But I was not — surprised, that is. Why?  Of the several hundred downloads of the measures from my website per week, and the more than 200,000 over the last decade, very few practitioners have sought or received any training.  Indeed, most have never even read the FIT manuals!

The impact on those who are initially enthusiastic about seeking formal feedback from their clients is as predictable as it is sad: they quickly give up.  How do I know, you ask?  Every week, as I’m out and about, training and consulting, I run into practitioners who say:

“Yeah, I heard of FIT, I even tried the scales…but they didn’t work…”

“I tried the ORS and SRS scales for a while, but I didn’t get any clinically useful information from them…”

“My clients weren’t honest … so I stopped using them”

And so I ask, what does it take to help people get the information and training they need to succeed?  The question is far from trivial or self-serving.  The measures are free to download and the latest research shows using them more than doubles the chances of helping clients experience meaningful change.  The only caveat is that, despite their simplicity, learning to employ the tools effectively takes time and support.  How do I know that?  Research, of course!

If you have thoughts about what I can do to address this problem, please let me know. In the meantime, in an effort to help, here are several offers:

1. For the time being, get the FIT Treatment and Training Manuals for 50% off;

2. Register for the combined FIT Advanced and Supervision Intensive now using the code FIT-Promo at checkout and get an additional discount off the early bird rate;

3. Sign up for the cutting-edge FIT e-learning program — where you can learn at your own pace from the comfort of your home — and receive the new, deliberate practice module, for free (if interested, email me for details about this offer).

Yes, please feel free to share these links and codes with your colleagues.  And, once more, if you have additional suggestions, I’m interested in hearing them.  Please post a comment below.

That’s it for now.

All the best for the Holidays,

Scott

Scott D. Miller, Ph.D.
Director, International Center for Clinical Excellence