SCOTT D Miller - For the latest and greatest information on Feedback Informed Treatment

info@scottdmiller.com 773.404.5130

What can therapists learn from the CIA? Experts versus the "Wisdom of the Crowd"

By Leave a Comment

Central psychotherapy agency

What can we therapists learn from the CIA?  In a phrase, “When it comes to making predictions about important future events, don’t rely on experts!”

After a spate of embarrassing, high-profile intelligence failures, a recent story showed how a relatively small group of average people made better predictions about critical world events than highly-trained analysts with access to classified information.  The four-year study, known as the Good Judgment Project, adds to mounting evidence regarding the power of aggregating independent guesses of regular folks–or what is known as, “the wisdom of the crowd.”

When it comes to therapy, multiple scientific studies show that inviting the “wisdom of the crowd” into treatment as much as doubles effectiveness, while simultaneously cutting drop out and deterioration rates.

Whatever your profession, work setting, or preferred therapeutic approach, the process involves formally soliciting feedback from clients and then comparing the results to empirically established benchmarks.   Getting started is easy:

You can learn more at: www.whatispcoms.com.  Plus, the ICCE–the world’s largest online community of professionals using feedback to enhance clinical judgment–is available at no cost to support you in your efforts.

While you’re at it, be sure and join fellow practitioners from the US, Canada, Europe, and Australia for the “Training of Trainers” or two-day FIT Implementation Intensive coming up this August in Chicago.  You’ll not only learn how to use the measures, but also tap into the collective wisdom of clients and practitioners around the globe.   Space is limited, and we are filling up quickly, so don’t wait to register.

Good News and Bad News about Psychotherapy

By 3 Comments

good news bad news

Have you seen this month’s issue of, “The National Psychologist?”  If you do counseling or psychotherapy, you should read it.  The headline screams, “Therapy: No Improvement for 40 Years.”  And while I did not know the article would be published, I was not surprised by the title nor it’s contents.  The author and associate editor, John Thomas, was summarizing the invited address I gave at the recent Evolution of Psychotherapy conference.

Fortunately, it’s not all bad news.  True, the outcomes of psychotherapy have not been improving.  Neither is there much evidence that clinicians become more effective with age and experience.  That said, we can get better.  Results from studies of top performing clinicians point the way.  I also reviewed this exciting research in my presentation.
Even if you didn’t attend the conference, you can see it here thanks to the generosity of the Milton H. Erickson Foundation.  Take a look at the article and video, then drop me a line and let me know what you think.  To learn more, you can access a variety of articles for free in the scholarly publications section of the website.

Click here to access the article from the National Psychologist about Scott Miller’s speech at the Evolution of Psychotherapy Conference in Anaheim, California (US) 

Excellence in Amsterdam: The 2013 ACE Conference

By Leave a Comment

My how time flies!  Nearly three weeks have passed since hundreds of clinicians, researchers, and educators met in Amsterdam, Holland for the 2013 “Achieving Clinical Excellence.”  Participants came from around the globe–Holland, the US, Germany, Denmark, Italy, Russia, Norway, Sweden, Denmark, New Zealand, Romania, Australia, France–for three days of presentations on improving the quality and outcome of behavioral healthcare.  Suffice it to say, we had a blast!

The conference organizers, Dr. Liz Pluut and Danish psychologist Susanne Bargmann did a fantastic job planning the event, organizing a beautiful venue (the same building where the plans for New York City were drafted back in the 17th century), coordinating speakers (36 from around the globe), arranging meals, hotel rooms, and handouts.

Dr. Pluut opened the conference and introduced the opening plenary speaker, Dr. K. Anders Ericsson, the world’s leading researcher and “expert on expertise.”  Virtually all of the work being done by me and my colleagues at the ICCE on the study of excellence and expertise among therapists is based on the three decades of pioneering work done by Dr. Ericsson.  You can read about our work, of course, in several recent articles: Supershrinks, The Road to Mastery, or the latest The Outcome of Psychotherapy: Past, Present and Future (which appeared in the 50th anniversary edition of the journal, Psychotherapy).

Over the next several weeks, I’ll be posting summaries and videos of many of the presentations, including Dr. Ericsson.  One key aspect of his work is the idea of “Deliberate Practice.”  Each of the afternoon sessions on the first day focused on this important topic, describing how clinicians, agency managers, and systems of care can apply it to improve their skills and outcome.

The first of these presentations was by psychologist Birgit Valla–the leader of Family Help, a mental health agency in Stange, Norway–entitled, “Unreflectingly Bad or Deliberately Good: Deciding the Future of Mental Health Services.”  Grab a cup of coffee and listen in…

Oh, yeah…while on the subject of excellence, here’s an interview that just appeared in the latest issue of the UK’s Therapy Today magazine:

It starts on page 32.

What to Pay Attention to in Therapy?

By Leave a Comment

A week or so ago, I received an email from my friend, colleague, and mentor Joe Yeager.  He runs a small listserve that sends out interesting and often provocative information.  The email contained pictures from a new and, dare I say, ingenious advertising campaign for Colgate brand dental floss.  Before I give you any of further details, however, take a look at the images yourself:

All right.  So what caught your attention?  If you’re like most people–including me–you probably found yourself staring at the food stuck in the teeth of the men in all three images.  If so, the ad achieved its purpose.  Take a look at the pictures one more time.  In the first, the woman has one too many fingers on her left hand.  The second image has a “phamtom arm” around the man’s shoulder.  Can you see the issue in the third?

The anomalies in the photos are far from minor!  And yet, most of us, captured by the what initially catches our eye, miss them.

Looking beyond the obvious is what Feedback Informed Treatment (FIT) is all about.  Truth is, much of the time therapy works.  What we do pay attention to gets results–except when it doesn’t!  At those times, two things must happen: (1) we have to know when what we usually do isn’t working with a given person; and (2) look beyond the obvious and see a bigger picture.  Doing this takes effort and support.    What can you do?

1. Download two free, brief, simple to use tools for tracking outcome and engagement in care (the ORS and SRS) and begin using them in your work;

2. Join the International Center for Clinical Excellence, the world’s largest, free, online, non-denominational organization of behavioral health professionals;

3. Read the six cutting-edge treatment and training manuals on feedback-informed treatment–a series which helped earn FIT the highest ratings from the Substance Abuse and Mental Health Services Administration (SAMHSA);

4. Attend a training in Chicago or abroad.

 

Dealing with Scientific Objections to the Outcome and Session Rating Scales: Real and Bogus

By Leave a Comment

The available evidence is clear: seeking formal feedback from consumers of behavioral health services decreases drop out and deterioration while simultanesouly improving effectiveness.  When teaching practitioners how to use the ORS and SRS to elicit feedback regarding progress and the therapeutic relationship,  three common and important concerns are raised:

  1. How can such simple and brief scales provide meaningful information?
  2. Are consumers going to be honest?
  3. Aren’t these measures merely assessing satisfaction rather than anything meaninful?

Recently, I was discussing these concerns with ICCE Associate and Certified Trainer, Dan Buccino.

Briefly, Dan is a clinical supervisor and student coordinator in the Adult Outpatience Community Psychiatry program at Johns Hopkins.  He’d not only encountered the concerns noted above but several additional objections.  As he said in his email, “they were at once baffling and yet exciting, because they were so unusal and rigorous.”

“It’s a sign of the times,” I replied, “As FIT (feedback informed treatment) becomes more widespread, the supporting evidence will be scrutinized more carefully.  It’s a good sign.”

Together with Psychologist and ICCE Senior Associate and Trainer, Jason Seidel, Dan crafted detailed response.  When I told them that I believed the ICCE community would value having access to the document they created, both agreed to let me publish it on the Top Performance blog.  So…here it is.  Please read and feel free to pass it along to others.

 

 

 

The Importance of "Whoops" in Improving Treatment Outcome

By Leave a Comment

“Ring the bells that still can ring,
Forget your perfect offering
There is a crack in everything,
That’s how the light gets in.”

Leonard Cohen, Anthem

Making mistakes.  We all do it, in both our personal and professional lives.  “To err is human…,” the old saying goes.  And most of us say, if asked, that we agree whole heartedly with the adage–especially when it refers to someone else!  When the principle becomes personal, however, its is much more difficult to be so broad-minded.

Think about it for a minute: can you name five things you are wrong about?  Three?  How about the last mistake you made in your clinical work?  What was it?  Did you share it with the person you were working with?  With your colleagues?

Research shows there are surprising benefits to being wrong, especially when the maker views such errors differently.  As author Alina Tugend points out in her fabulous book, Better by Mistake, custom wrongly defines a mistake as ” the failure of a planned sequence of mental or physical activities to achieve its intended outcome.”  When you forget a client’s name during a session or push a door instead of pull, that counts as  slip or lapse.  A mistake, by contrast, is when “the plan itself is inadequate to achieve it’s objectives” (p. 11).  Knowing the difference, she continues, “can be very helpful in avoiding mistakes in the future” because it leads exploration away from assigning blame to the exploring systems, processes, and conditions that either cause mistakes or thwart their detection.

Last week, I was working with a talented and energetic group of helping professionals in New Bedford, Massachusetts.  The topic was, “Achieving Excellence: Pushing One’s Clinical Performance to the Next Level of Effectiveness.”  As part of my presentation, I talked about becoming more, “error-centric” in our work; specifically, using ongoing measurement of the alliance to identify opportunities for improving our connection with consumers of behavioral health services.  As an example of the benefits of making mistakes the focus of professional development efforts, I showed a brief video of Rachel Hsu and Roger Chen, two talented musicians who performed at the last Achieving Clinical Excellence (ACE) conference.  Rachel plays a piece by Liszt, Roger one by Mozart.  Both compositions are extremely challenging to play.  You tell me how they did (by the way, Rachel is 8 years old, Roger. 9):

Following her performance, I asked Rachel if she’d made any mistakes during her performance.  She laughed, and then said, “Yes, a lot!”  When I asked her what she did about that, she replied, “Well, its impossible to learn from my mistakes while I’m playing.  So I note them and then later practice those small bits, over and over, slow at first, then speeding up, until I get them right.”

After showing the video in New Bedford, a member of the audience raised his hand, “I get it but that whole idea makes me a bit nervous.”  I knew exactly what he was thinking.  Highlighting one’s mistakes in public is risky business.  Studies documenting that the most effective clinicians experience more self-doubt and are more willing to admit making mistakes is simply not convincing when one’s professional self-esteem or job may be on the line.  Neither is research showing that health care professionals who admit making mistakes and apologize to consumers are significantly less likely to be sued.  Becoming error centric, requires a change in culture, one that not only invites discloure but connects it with the kind of support and structure that leads to superior results.

Creating a “whoops-friendly” culture will be a focus of the next Achieving Clinical Excellence conference, scheduled for May 16-18th, 2013 in Amsterdam, Holland.  Researchers and clinicians from around the world will gather to share their data and experience at this unique event.  I promise you don’t want to miss it.  Here’s a short clip of highlights from the last one:

My colleague, Susanne Bargmann and I will also be teaching the latest research and evidence based methods for transforming mistakes into improved clinical performance at the upcoming FIT Advanced Intensive training in Chicago, Illinois.   I look forward to meeting you at one of these upcoming events.  In the meantime, here’s a fun, brief but informative video from the TED talks series on mistakes:

By the way, the house pictured above is real.  My family and I visited it while vacationing in Niagara Falls, Canada in October.  It’s a tourist attraction actually.  Mistakes, it seems, can be profitable.

What is the Real Source of Effectiveness in Smoking Cessation Treatment? New Research on Feedback Informed Treatment

By Leave a Comment

When it rains, it pours!  So much news to relay regarding recent research on Feedback Informed Treatment (FIT).  Just received news this week from ICCE Associate Stephen Michaels that research using the ORS and SRS in smoking cessation treatment is in print!   A few days prior to that, Kelley Quirk sent a copy of our long-awaited article on the validity and reliability of the Group Session Rating Scale.  On that very same day, the editors from the journal Psychotherapy sent proofs of an article written by me, Mark Hubble, Daryl Chow, and Jason Seidel for the 50th anniversary issue of the publication.

Let’s start with the validity and reliability study.  Many clinicians have already downloaded and been using Group Session Rating Scale.  The measure is part of the packet of FIT tools available in 20+ languages on both my personal and the International Center for Clinical Excellence websites.   The article presents the first research on the validity and reliability of the measure.  The data for the study was gathered at two sites I’ve worked with for many years.   Thanks to Kelley Quirk and Jesse Owen for crunching the numbers and writing up the results!   Since the alliance is one of the most robust predictors of outcome, the GSRS provides yet another method for helping therapists obtain feedback from consumers of behavior health services.

Moving on, if there were a Nobel Prize for patience and persistence, it would have to go to Stephen Michaels, the lead author of the study, Assessing Counsellor Effects on Quit Rates and Life Satisfactions Scores at a Tobacco Quitline” (Michael, Seltzer, Miller, and Wampold, 2012).  Over the last four years, Stephen has trained Quitline staff in FIT, implemented the ORS and SRS in Quitline tobacco cessation services, gathered outcome and alliance data on nearly 3,000 Quitline users, completed an in-depth review of the available smoking cessation literature, and finally, organized, analyzed, and written up the results.

What did he find?  Statistically significant differences in quit rates attributable to counselor effects.  In other words, as I’ve been saying for some time, some helpers are more helpful than others–even when the treatment provided is highly manualized and structured.  In short, it’s not the method that matters (including the use of the ORS and SRS), it’s the therapist.

What is responsible for the difference in effectiveness among therapists?  The answer to that question is the subject of the article, “The Outcome of Psychotherapy: Yesterday, Today, and Tomorrow” slated to appear in the 50th anniversary issue of Psychotherapy.  In it, we review controversies surround the question, “What makes therapy work?” and tip findings from another, soon-to-be-published empirical analysis of top performing clinicians.  Stay tuned.

Clinical Support Tools for the ORS and SRS

By 1 Comment

I have so much to be grateful for at this time.  Most of all, I’m happy to be home with my family.  As we have in the past, this year we’ll be spending the holiday at the home of our long time friends John and Renee Dalton.  The two always put out a fantastic spread and our son, Michael, is fast friends with their two kids.

I’m also grateful for the International Center for Clinical Excellence (ICCE) community.  Currently, ICCE has over 4200 members located around the world, making the organization the largest, web-based community of professionals, educators, managers, and clinicians dedicated to using feedback to pursue excellence in the delivery of behavioral health services.  Recently, the site was highlighted as one of the best resources for practitioners available on the web.  Articles, how-to videos, and discussion forums are available everyday, all day–and for free!  No come-ons for books or webinars and no “cult of personality”–just sharing among peers.  If you are not a member, you can join at: www.centerforclinicalexcellence.com

A special thanks goes to several ICCE senior advisors and associates, including Susanne Bargmann, Jason Seidel, Cynthia Maeschalck, Bob Bertolino, Bill Plum, Julie Tilsen, and Robbie Babbins-Wagner.  These folks are the backbone of the organization.  Together, they make it work.  Most recently, we all joined together to create the ICCE Feedback Informed Treatment and Training Manuals, a cutting edge series covering every aspect of FIT–from the empirical foundations to implementation–in support of our application to SAMSHA for recognition as an “evidence-based practice.”

As a way of supporting everyone using the ORS and SRS, I wanted to make a couple of clinical support tools available.  If you are using the measures, the first item will need no introduction.  It’s a 10 cm ruler!  Save the file and print it off and you also have a ready reminder of the upcoming Achieving Clinical Excellence conference, coming up in May 2013.  Like last time, this will feature the latest inforamtion about feedback informed practice!  The second item is a reliable change graph.  If you are using the paper and pencil measures, rather than one of the existing web based systems (www.fit-outcomes.com, www.myoutcomes.com), you can use this tool to determine whether a change in scores from session to session is reliable (that is, greater than chance, the passage of time, and measurement error [and therefore, due to the care being provided]) or even clinically significant (that is, both reliable and indicating recovered).  The last item is an impressive summary of various systems for monitoring progress in treatment.

In addition ACE Health have developed openFIT, a plug-in which seamlessly integrates the ORS, SRS and associated algorithms into any existing Electronic Health Record, Case Management System of eMental Health application.

I wish everyone a peaceful and rewarding Thanksgiving holiday.

 

An Easy Way to Improve Our Schools (and Psychotherapy)

By Leave a Comment

If you didn’t see the October Atlantic Monthly, you really missed a great issue.  In it, Amanda Ripley wrote a delightful and informative article about a simple and straightforward method for improving the performance of the public schools: have kids grade teachers.   What kind of grades you ask?   Not those on standardized achievement tests, and certainly not measures of a teacher’s popularity.  Although both of those methods are widely advocated and used, neither has proven particularly predictive of student performance.  Rather, grades should be based on how well teachers engage students; particularly whether the kids believe the teacher makes them want to work hard, pay attention, understand the course material, and identify and correct their mistakes.  Indeed, in thousands of surveys, kids as young as kindergartners “can identify with uncanny accuracy, their most–and least effective teachers.”

The findings stood out for me not only because I am a parent but also because they mirror results from psychotherapy research.  First, data gathered over the last three decades documents that client engagement is the number one process-related predictor of treatment outcome.  Second, a growing number of studies indicates that clients can identify “with uncanny accuracy” the most and least effective treatment services and providers.  Importantly, this same body of evidence shows that client assessments of their sessions and progress can be used to enhance treatment results in general as well as the effectiveness and skill level of individual clinicians.

Viewpoint clearly matters–and in the case of schools and psychotherapy, it is the recipient of the service whose opinion we should be seeking.  In her article, Ripley identifies the types of questions that can be used in schools.  If you are a therapist, two brief, simple-to-use scales are available for free.  Research has shown that regularly using the measures to solicit client feedback improves both retention in and outcome of psychotherapy.  The largest, international professional community dedicated to enhancing the quality and outcome of behavioral healthcare is available to support you in your use of the tools.  There, you will find a wealth of information, discussion forums, and how-to videos available at no charge 24 hours a day, 7 days a week, 365 days a year.

In March, many members from around the world will be joining colleagues from around the world for four days of intensive training.  Why not join us?  We work and play hard.  Rest assured that by the end of the four days, you’ll be playing an “A” game.  Click here to register today.  In the meantime, here’s what participants from last year said about the event.

Research on the Outcome Rating Scale, Session Rating Scale & Feedback

By Leave a Comment

PCOMS - Partners for change outcome management system Scott D Miller - SAMHSA - NREPP“How valid and reliable are the ORS and SRS?”  “What do the data say about the impact of routine measurement and feedback on outcome and retention in behavioral health?”  “Are the ORS and SRS ‘evidence-based?'”

These and other questions regarding the evidence supporting the ORS, SRS, and feedback are becoming increasingly common in the workshops I’m teaching in the U.S. and abroad.

As indicated in my December 24th blogpost, routine outcome monitoring (PROMS) has even been endorsed by “specific treatments for specific disorders” proponent David Barlow, Ph.D., who stated unequivocally that “all therapists would soon be required to measure and monitor the outcome of their clinical work.”  Clearly, the time has come for all behavioral health practitioners to be aware of the research regarding measurement and feedback.

Over the holidays, I updated a summary of the data to date that has long been available to trainers and associates of the International Center for Clinical Excellence.  The PDF reviews all of the research on the psychometric properties of the outcome and session ratings scales as well as the studies using these and other formal measures of progress and the therapeutic relationship to improve outcome and retention in behavioral health services.  The topics is so important, that I’ve decide to make the document available to everyone.  Feel free to distribute the file to any and all colleagues interested in staying up to date on this emerging mega-trend in clinical practice.

The Study of Excellence: A Radically New Approach to Understanding "What Works" in Behavioral Health

By 2 Comments

“What works” in therapy?  Believe it or not, that question–as simple as it is–has and continues to spark considerable debate.  For decades, the field has been divided.  On one side are those who argue that the efficacy of psychological treatments is due to specific factors (e.g., changing negative thinking patterns) inherent in the model of treatment (e.g., cognitive behavioral therapy) remedial to the problem being treated (i.e., depression); on the other, is a smaller but no less committed group of researchers and writers who posit that the general efficacy of behavioral treatments is due to a group of factors common to all approaches (e.g., relationship, hope, expectancy, client factors).

While the overall effectiveness of psychological treatment is now well established–studies show that people who receive care are better off than 80% of those who do not regardless of the approach or the problem treated–one fact can not be avoided: outcomes have not improved appreciably over the last 30 years!  Said another way, the common versus specific factor battle, while generating a great deal of heat, has not shed much light on how to improve the outcome of behavioral health services.  Despite the incessant talk about and promotion of “evidence-based” practice, there is no evidence that adopting “specific methods for specific disorders” improves outcome.  At the same time, as I’ve pointed out in prior blogposts, the common factors, while accounting for why psychological therapies work, do not and can not tell us how to work.  After all, if the effectiveness of the various and competing treatment approaches is due to a shared set of common factors, and yet all models work equally well, why learn about the common factors?  More to the point, there simply is no evidence that adopting a “common factors” approach leads to better performance.

The problem with the specific and common factor positions is that both–and hang onto your seat here–have the same objective at heart; namely, contextlessness.  Each hopes to identify a set of principles and/or practices that are applicable across people, places, and situations.  Thus, specific factor proponents argue that particular “evidence-based” (EBP) approaches are applicable for a given problem regardless of the people or places involved (It’s amazing, really, when you consider that various approaches are being marketed to different countries and cultures as “evidence-based” when there is in no evidence that these methods work beyond their very limited and unrepresentative samples).  On the other hand, the common factors camp, in place of techniques, proffer an invariant set of, well, generic factors.  Little wonder that outcomes have stagnated.  Its a bit like trying to learn a language either by memorizing a phrase book–in the case of EBP–or studying the parts of speech–in the case of the common factors.

What to do?  For me, clues for resolving the impasse began to appear when, in 1994, I followed the advice of my friend and long time mentor, Lynn Johnson, and began formally and routinely monitoring the outcome and alliance of the clinical work I was doing.  Crucially, feedback provided a way to contextualize therapeutic services–to fit the work to the people and places involved–that neither a specific or common factors informed approach could.

Numerous studies (21 RCT’s; including 4 studies using the ORS and SRS) now document the impact of using outcome and alliance feedback to inform service delivery.  One study, for example, showed a 65% improvement over baseline performance rates with the addition of routine alliance and outcome feedback.  Another, more recent study of couples therapy, found that divorce/separation rates were half (50%) less for the feedback versus no feedback conditions!

Such results have, not surprisingly, led the practice of “routine outcome monitoring” (PROMS) to be deemed “evidence-based.” At the recent, Evolution of Psychotherapy conference I was on a panel with David Barlow, Ph.D.–a long time proponent of the “specific treatments for specific disorders” (EBP)–who, in response to my brief remarks about the benefits of feedback, stated unequivocally that all therapists would soon be required to measure and monitor the outcome of their clinical work.  Given that my work has focused almost exclusively on seeking and using feedback for the last 15 years, you would think I’d be happy.  And while gratifying on some level, I must admit to being both surprised and frightened by his pronouncement.

My fear?  Focusing on measurement and feedback misses the point.  Simply put: it’s not seeking feedback that is important.  Rather, it’s what feedback potentially engenders in the user that is critical.  Consider the following, while the results of trials to date clearly document the benefit of PROMS to those seeking therapy, there is currently no evidence of that the practice has a lasting impact on those providing the service.  “The question is,” as researcher Michael Lambert notes, “have therapists learned anything from having gotten feedback? Or, do the gains disappear when feedback disappears? About the same question. We found that there is little improvement from year to year…” (quoted in Miller et al. [2004]).

Research on expertise in a wide range of domains (including chess, medicine, physics, computer programming, and psychotherapy) indicates that in order to have a lasting effect feedback must increase a performer’s “domain specific knowledge.”   Feedback must result in the performer knowing more about his or her area and how and when to apply than knowledge to specific situations than others.  Master level chess players, for example, have been shown to possess 10 to 100 times more chess knowledge than “club-level” players.  Not surprisingly, master players’ vast information about the game is consilidated and organized differently than their less successful peers; namely, in a way that allows them to access, sort, and apply potential moves to the specific situation on the board.  In other words, their immense knowledge is context specific.

A mere handful studies document similar findings among superior performing therapists: not only do they know more, they know how, when, and with whom o apply that knowledge.  I noted these and highlighted a few others in the research pipeline during my workshop on “Achieving Clinical Excellence” at the Evolution of Psychotherapy conference.  I also reviewed what 30 years of research on expertise and expert performance has taught us about how feedback must be used in order to insure that learning actually takes place.  Many of those in attendance stopped by the ICCE booth following the presentation to talk with our CEO, Brendan Madden, or one of our Associates and Trainers (see the video below).

Such research, I believe, holds the key to moving beyond the common versus specific factor stalemate that has long held the field in check–providing therapists with the means for developing, organizing, and contextualizing clinical knowledge in a manner that leads to real and lasting improvements in performance.