SCOTT D Miller - For the latest and greatest information on Feedback Informed Treatment

scottdmiller@ talkingcure.com +1.773.454.8511

Do Psychotherapists Improve with Time and Experience?

By 14 Comments

researchThe practice known as “routine outcome measurement,” or ROM, is resulting in the publication of some of the biggest and most clinically relevant psychotherapy studies in history.  Freed from the limits of the randomized clinical trial, and accompanying obsession with manuals and methods, researchers are finally able to examine what happens in real world clinical practice.

A few weeks ago, I blogged about the largest study of psychotherapy ever published.  More than 1,400 therapists participated.  The progress of over 26,000 people (aged 16-95) treated over a 12 year period in primary care settings in the UK was tracked on an ongoing basis via ROM.  The results?  In an average of 8 visits, 60% of those treated by this diverse group of practitioners achieved both reliable and clinically significant change—results on par with tightly controlled RCT’s.  The study is a stunning confirmation of the effectiveness of psychotherapy.

This week, another mega-study was accepted for publication in the Journal of Counselexperienceing Psychology.   Once more,
ROM was involved.  In this one, researchers Goldberg, Rousemanier, Miller, Whipple, Nielsen, Hoyt, and Wampold examined a large, naturalistic data set that included outcomes of 6500 clients treated by 170 practitioners whose results had been tracked an average of 5 years.

Their question?

Do therapists become more effective with time and experience?

Their answer?  No.

readerFor readers of this blog, such findings will not be particularly newsworthy.  As I’ve frequently pointed out, experience has never proven to be a significant predictor of effectiveness.

What might be a bit surprising is that the study found clinicians’ outcomes actually worsened with time and experience.  That’s right.  On average, the longer a therapist practiced, the less effective they became!  Importantly, this finding remained even when controlling for several patient-level, caseload-level, and therapist-level characteristics, as well as when excluding several types of outliers.

Such findings are noteworthy for a number of reasons but chiefly because they contrast sharply with results from other, equally-large studies documenting that therapists see themselves as continuously developing in both knowledge and ability over the course of their careers.   To be sure, the drop in performance reported by Goldberg and colleagues wasn’t steep.  Rather, the pattern was a slow, inexorable decline from year to year.

Where, one can wonder, does the disconnect come from?  How can therapists’ assessments of themselves and their work be so at odds with the facts?  Especially considering, in the study by Goldberg and colleagues, participating clinicians had ongoing access to data regarding their effectiveness (or lack thereof) on real-time basis!  Even the study I blogged about previously—the largest in history where outcomes of psychotherapy were shown to be quite positive—a staggering 40% of people treated experienced little or no change whatsoever.  How can such findings be reconciled with others indicating that clinicians routinely overestimate their effectiveness by 65%?

Turns out, thboundariese boundary between “belief in the process” and “denial of reality” is remarkably fuzzy.  Hope is a  significant contributor to outcome—accounting for as much as 30% of the variance in results.  At the same time, it becomes toxic when actual outcomes are distorted in a manner that causes practitioners to miss important opportunities to grow and develop—not to mention help more clients.  Recall studies documenting that top performing therapists evince more of what researchers term, “professional self-doubt.”  Said another way, they are less likely to see progress where none exists and more likely to values outcomes over therapeutic process.

What’s more, unlike their more average counterparts, highly effective practitioners actually become more effective with time and experience.  In the article below, my colleagues and I at the International Center for Clinical Excellence identify several evidence-based steps any practitioner follow to match such results.

Let me know your thoughts.

Until next time,

Scott

Scott D. Miller, Ph.D.
headerMain8.pngRegistration is now open for our March Intensives in Chicago.  Join colleagues from around the world for the FIT Advanced and the FIT Supervision workshops.

 

Did you know your clients can tell if you are happy?

By 3 Comments

Are_You_Happy

It’s true.  Adding to a growing literature showing that the person of the therapist is more important than theoretical orientation, years of experience, or discipline, a new study documents that clients are sensitive to the quality of their therapist’s life outside of treament.  In short, they can tell when you are happy or not.  Despite our best efforts to conceal it, they see it in how we interact with them in therapy.  By contrast, therapists’ judgements regarding the quality of the therapy are biased by their own sense of personal well-being. The solution?  Short of being happy, it means we need to check in with our clients on a regular basis regarding the quality of the therapeutic relationship.  Multiple randomized clinical trials show that formally soliciting feedback regarding progress and the alliance improves outcome and continued engagement in treatment.  One approach, “Feedback-Informed Treatment” is now listed on SAMHSA’s National Registry of Evidence-Based Programs and Practices.  Step-by-step instructions and videos for getting started are available on a new website: www.pcomsinternational.com. Seeking feedback from clients not only helps to identify and correct potential problems in therapy, but is also the first step in pushing one’s effectiveness to the next level.  In case you didn’t see it, I review the research and steps for improving performance as a therapist in an article/interview on the Psychotherapy.net website.  It’s sure to make you happy!

Excellence in Amsterdam: The 2013 ACE Conference

By Leave a Comment

My how time flies!  Nearly three weeks have passed since hundreds of clinicians, researchers, and educators met in Amsterdam, Holland for the 2013 “Achieving Clinical Excellence.”  Participants came from around the globe–Holland, the US, Germany, Denmark, Italy, Russia, Norway, Sweden, Denmark, New Zealand, Romania, Australia, France–for three days of presentations on improving the quality and outcome of behavioral healthcare.  Suffice it to say, we had a blast!

The conference organizers, Dr. Liz Pluut and Danish psychologist Susanne Bargmann did a fantastic job planning the event, organizing a beautiful venue (the same building where the plans for New York City were drafted back in the 17th century), coordinating speakers (36 from around the globe), arranging meals, hotel rooms, and handouts.

Dr. Pluut opened the conference and introduced the opening plenary speaker, Dr. K. Anders Ericsson, the world’s leading researcher and “expert on expertise.”  Virtually all of the work being done by me and my colleagues at the ICCE on the study of excellence and expertise among therapists is based on the three decades of pioneering work done by Dr. Ericsson.  You can read about our work, of course, in several recent articles: Supershrinks, The Road to Mastery, or the latest The Outcome of Psychotherapy: Past, Present and Future (which appeared in the 50th anniversary edition of the journal, Psychotherapy).

Over the next several weeks, I’ll be posting summaries and videos of many of the presentations, including Dr. Ericsson.  One key aspect of his work is the idea of “Deliberate Practice.”  Each of the afternoon sessions on the first day focused on this important topic, describing how clinicians, agency managers, and systems of care can apply it to improve their skills and outcome.

The first of these presentations was by psychologist Birgit Valla–the leader of Family Help, a mental health agency in Stange, Norway–entitled, “Unreflectingly Bad or Deliberately Good: Deciding the Future of Mental Health Services.”  Grab a cup of coffee and listen in…

Oh, yeah…while on the subject of excellence, here’s an interview that just appeared in the latest issue of the UK’s Therapy Today magazine:

It starts on page 32.

How Cool is Kuhl? A Man with Vision on a Mission

By Leave a Comment

This week, my colleague and friend, Dr. David Mee-Lee, sent me a link to a blogpost written by Don Kuhl.  Actually, I was already a subscriber to Don’s Minful MIDweek blog (you should be too), but my travel this week had prevented me from reading his latest installment.  His posts always leave me inspired and give me something to think about.  This week was no different.  More on that in a moment.

In the meantime, let me tell you about Don.  He is the founder and CEO of The Change Companies, a company whose mission is to create tailored materials and programs to support behavioral change for special populatons.  And create they do.  Hundreds of bright, attractive, highly readable publications and guided workbooks for use by professionals and the people they serve.  Their material is exhaustive and comprehensive, including adult behavioral health, criminal justice, education and prevention, clinical assessment, and faith-based programs.  A side note, it was Don and his skillful team at The Change Companies that produced the ICCE Feedback Informed Treatment and Training Manuals.  If you’ve not seen them, you should.  They are the cutting edge of information about FIT.

What is most striking about Don, however, is his passion.  I met him at a conference in San Francisco nearly a decade ago.  On several occasions, he flew to Chicago from his home base in Carson City, Nevada just to meet, talk, and share ideas.  The photo above is from one of the meetings he arranged.  Don is devoted to improving the quality and experience of behavioral health services for professionals and clients alike.  Simply said, Don Kuhl is cool.

In his blogpost this week, Don wrote about that meeting with Jim Prochaska, David Mee-Lee, me, and Bill Miller.  He referred to it as a “highlight” of his recent professional life, a lucky event resulting from his mindful pursuit of relationships with “people who have smiles on their faces and goodness in their hearts.”

My thought?  I was and am the lucky one.  Thanks Don.  Thanks Change Companies.  Keep up the good work.

What is the Real Source of Effectiveness in Smoking Cessation Treatment? New Research on Feedback Informed Treatment

By Leave a Comment

When it rains, it pours!  So much news to relay regarding recent research on Feedback Informed Treatment (FIT).  Just received news this week from ICCE Associate Stephen Michaels that research using the ORS and SRS in smoking cessation treatment is in print!   A few days prior to that, Kelley Quirk sent a copy of our long-awaited article on the validity and reliability of the Group Session Rating Scale.  On that very same day, the editors from the journal Psychotherapy sent proofs of an article written by me, Mark Hubble, Daryl Chow, and Jason Seidel for the 50th anniversary issue of the publication.

Let’s start with the validity and reliability study.  Many clinicians have already downloaded and been using Group Session Rating Scale.  The measure is part of the packet of FIT tools available in 20+ languages on both my personal and the International Center for Clinical Excellence websites.   The article presents the first research on the validity and reliability of the measure.  The data for the study was gathered at two sites I’ve worked with for many years.   Thanks to Kelley Quirk and Jesse Owen for crunching the numbers and writing up the results!   Since the alliance is one of the most robust predictors of outcome, the GSRS provides yet another method for helping therapists obtain feedback from consumers of behavior health services.

Moving on, if there were a Nobel Prize for patience and persistence, it would have to go to Stephen Michaels, the lead author of the study, Assessing Counsellor Effects on Quit Rates and Life Satisfactions Scores at a Tobacco Quitline” (Michael, Seltzer, Miller, and Wampold, 2012).  Over the last four years, Stephen has trained Quitline staff in FIT, implemented the ORS and SRS in Quitline tobacco cessation services, gathered outcome and alliance data on nearly 3,000 Quitline users, completed an in-depth review of the available smoking cessation literature, and finally, organized, analyzed, and written up the results.

What did he find?  Statistically significant differences in quit rates attributable to counselor effects.  In other words, as I’ve been saying for some time, some helpers are more helpful than others–even when the treatment provided is highly manualized and structured.  In short, it’s not the method that matters (including the use of the ORS and SRS), it’s the therapist.

What is responsible for the difference in effectiveness among therapists?  The answer to that question is the subject of the article, “The Outcome of Psychotherapy: Yesterday, Today, and Tomorrow” slated to appear in the 50th anniversary issue of Psychotherapy.  In it, we review controversies surround the question, “What makes therapy work?” and tip findings from another, soon-to-be-published empirical analysis of top performing clinicians.  Stay tuned.

Clinical Support Tools for the ORS and SRS

By 1 Comment

I have so much to be grateful for at this time.  Most of all, I’m happy to be home with my family.  As we have in the past, this year we’ll be spending the holiday at the home of our long time friends John and Renee Dalton.  The two always put out a fantastic spread and our son, Michael, is fast friends with their two kids.

I’m also grateful for the International Center for Clinical Excellence (ICCE) community.  Currently, ICCE has over 4200 members located around the world, making the organization the largest, web-based community of professionals, educators, managers, and clinicians dedicated to using feedback to pursue excellence in the delivery of behavioral health services.  Recently, the site was highlighted as one of the best resources for practitioners available on the web.  Articles, how-to videos, and discussion forums are available everyday, all day–and for free!  No come-ons for books or webinars and no “cult of personality”–just sharing among peers.  If you are not a member, you can join at: www.centerforclinicalexcellence.com

A special thanks goes to several ICCE senior advisors and associates, including Susanne Bargmann, Jason Seidel, Cynthia Maeschalck, Bob Bertolino, Bill Plum, Julie Tilsen, and Robbie Babbins-Wagner.  These folks are the backbone of the organization.  Together, they make it work.  Most recently, we all joined together to create the ICCE Feedback Informed Treatment and Training Manuals, a cutting edge series covering every aspect of FIT–from the empirical foundations to implementation–in support of our application to SAMSHA for recognition as an “evidence-based practice.”

As a way of supporting everyone using the ORS and SRS, I wanted to make a couple of clinical support tools available.  If you are using the measures, the first item will need no introduction.  It’s a 10 cm ruler!  Save the file and print it off and you also have a ready reminder of the upcoming Achieving Clinical Excellence conference, coming up in May 2013.  Like last time, this will feature the latest inforamtion about feedback informed practice!  The second item is a reliable change graph.  If you are using the paper and pencil measures, rather than one of the existing web based systems (www.fit-outcomes.com, www.myoutcomes.com), you can use this tool to determine whether a change in scores from session to session is reliable (that is, greater than chance, the passage of time, and measurement error [and therefore, due to the care being provided]) or even clinically significant (that is, both reliable and indicating recovered).  The last item is an impressive summary of various systems for monitoring progress in treatment.

In addition ACE Health have developed openFIT, a plug-in which seamlessly integrates the ORS, SRS and associated algorithms into any existing Electronic Health Record, Case Management System of eMental Health application.

I wish everyone a peaceful and rewarding Thanksgiving holiday.

 

An Easy Way to Improve Our Schools (and Psychotherapy)

By Leave a Comment

If you didn’t see the October Atlantic Monthly, you really missed a great issue.  In it, Amanda Ripley wrote a delightful and informative article about a simple and straightforward method for improving the performance of the public schools: have kids grade teachers.   What kind of grades you ask?   Not those on standardized achievement tests, and certainly not measures of a teacher’s popularity.  Although both of those methods are widely advocated and used, neither has proven particularly predictive of student performance.  Rather, grades should be based on how well teachers engage students; particularly whether the kids believe the teacher makes them want to work hard, pay attention, understand the course material, and identify and correct their mistakes.  Indeed, in thousands of surveys, kids as young as kindergartners “can identify with uncanny accuracy, their most–and least effective teachers.”

The findings stood out for me not only because I am a parent but also because they mirror results from psychotherapy research.  First, data gathered over the last three decades documents that client engagement is the number one process-related predictor of treatment outcome.  Second, a growing number of studies indicates that clients can identify “with uncanny accuracy” the most and least effective treatment services and providers.  Importantly, this same body of evidence shows that client assessments of their sessions and progress can be used to enhance treatment results in general as well as the effectiveness and skill level of individual clinicians.

Viewpoint clearly matters–and in the case of schools and psychotherapy, it is the recipient of the service whose opinion we should be seeking.  In her article, Ripley identifies the types of questions that can be used in schools.  If you are a therapist, two brief, simple-to-use scales are available for free.  Research has shown that regularly using the measures to solicit client feedback improves both retention in and outcome of psychotherapy.  The largest, international professional community dedicated to enhancing the quality and outcome of behavioral healthcare is available to support you in your use of the tools.  There, you will find a wealth of information, discussion forums, and how-to videos available at no charge 24 hours a day, 7 days a week, 365 days a year.

In March, many members from around the world will be joining colleagues from around the world for four days of intensive training.  Why not join us?  We work and play hard.  Rest assured that by the end of the four days, you’ll be playing an “A” game.  Click here to register today.  In the meantime, here’s what participants from last year said about the event.

The Road to Clinical Excellence is Paved with Practice, Mistakes, & Hard Work

By Leave a Comment

Last week, I received an email from David Claud.  I’ve known Dave for the better part of a decade, having met–I believe–at a Ericksonian Conference in Florida where he lives and works.  He and the crew at the Center for Family Service in Palm Beach County figure prominently in the history of routine outcome measure and feedback.  After hearing me speak, Dave took the measures back to the center and, together with the staff, became one of the first agencies in the country to formally adopt and use the ORS and SRS.  Additionally, data gathered at CFS was used in some of the initial validation studies of the measures.  Finally, their own research, cited in the second edition of The Heart and Soul of Change document dramatic improvements in outcome as well as decreased lengths of stay, cancellation and no show rates (40, 40, and 25% respectively).

Anyway, in his email, Dave included a link to a recent article by Ann Hulbert in Slate magazine.  I’m lucky to have friends like Dave and others who keep me informed and up-to-date.  The title of the piece certainly got my attention: “The Dark Side of the New Theories of Success: What the New Success Books Don’t tell you about Superachievement.”

As readers of my blog know, I’ve been pouring through the literature on excellence over this last year in an attempt to understand why some clinicians achieve reliably better outcomes than others.  I first wrote about our findings in an article titled, “Supershrinks: Learning from the Field’s Most Effective Practitioners” that appeared in the Psychotherapy Networker.  Since then, I’ve continued to work and research, together with senior associates at the International Center for Clinical Excellence, to deepen and refine the “steps to clinical excellence” that any therapist could follow to improve performance.

Alas, I’m not alone in my interest in the literature on expertise.  A number of books, starting with Gladwell’s delightfully engaging Outliers, have appeared in the last year or so on the subject, including: The Talent Code, Bounce: The Science of Success, The Genius in All of Us: Why Everything You’ve Been Told About Genetics, Talent, and IQ Is Wrong and my personal favorite Talent Is Overrated: What Really Separates World-Class Performers from Everybody Else.  The appearance of so many books is interesting.  With few exceptions (i.e., sports psychology), K. Anders Erickson and colleagues labored in viritual academic obscurity for decades formulating hypotheses, conducting research and assembling evidence.  And then suddenly: boom!  EVERYBODY is talking about their work.

Always wanting to “hear” both sides of the story, I immediately clicked on the link in Dave’s email and read the article.  I was dumbfounded.  Hulbert’s gripe about the recent spate of books is in fact the central point of each: achieving superior performance in any field is bloody hard work.  “They don’t always do realistic justice to the grunt work they champion,” whines Hulbert, tending instead to, “gloss over the sweaty specifics….distracting us from how arduous, tedious, and dependent on adult pushiness it can be…[and] glamorizing its intensity.”

My response: “Oh, contraire mon fraire!”

All of the books and research studies point to the years of dedicated and painstaking work involved in achieving world class levels of performance across a variety of domains (sports, music, medicine, computer programming, and psychology).  K. Ander’s Erickson–who will, by the way, be one of the keynote presenters at the upcoming “Achieving Clinical Excellence” conference–is fond of saying, “Unlike play, deliberate practice is not inherently motivating; and unlike work, it does not lead to immediate social and monetary rewards…and actually generates costs…”.  Little wonder few of us–myself included–engage in it on any regular basis.

The question that begs an answer is, “why would anyone do it?”  Consider the brief video clip below:

Impressive, huh?  I can’t imagine the amount of time it must have taken to master such a performance.  No camera tricks. Just plain old fashioned trial-and-error, practice, and hard work.

We are finding the same pattern among top performing therapists.  In short, they have an “error-centric” approach to practice–constantly looking for what they do that doesn’t work and taking time to plan, identify and try alternatives, and then reflect and refine their process-improvement efforts.  Such activity is cognitively taxing and, in most instances, not immediately rewarding (financially or otherwise).  But there is more to the story.  It turns out that superior performance is not a matter of working harder.  Most of us work hard at our jobs.  Rather, becoming a better clinician is about working smarter.   Here, the literature on expertise provides clear, empirically-supported guidelines.

If you’re feeling inspired, why not pick up one of the books?  Also, be sure and join us at the upcoming “Achieving Clinical Excellence” conference where the ideas and steps will be discussed in detail.

International "Achieving Clinical Excellence" Conference

By 3 Comments

Mark your calendars!  The International Center for Clinical Excellence is pleased to announce the “Achieving Clinical Excellence” (ACE) conference to be held at the Westin Hotel in Kansas City, Missouri on October 20-22nd, 2010.

K. Anders Erickson, Ph.D., the editor of The Cambridge Handbook of Expertise and Expert Performance and recognized “expert on experts,” will keynote the event. Through a combination of plenary presentations and intensive workshops, an internationally renowned faculty of researchers and educators, including Scott D. Miller, Ph.D. and John Norcross, Ph.D., will help participants discover the means to achieve excellence in clinical practice, leadership, ethics, and personal care.

Attendees will also meet and learn directly from internationally ranked performers from a variety of professions, including medicine, science, music, entertainment, and sports.  As just one example, the Head Coach of the Olympic, Gold-Medal-winning Women’s volleyball team, Hugh McCutcheon, will present at the conference.  In addition to a pre-conference day on ethics and law, internationally renowned concert pianist David Helfgott, whose heart-warming story was featured in the award winning film Shine, will perform on Thursday evening, October 21st. Join us in Kansas City for three days of science, skill building, and inspiration.

Superior Performance as a Psychotherapist: First Steps

By Leave a Comment

So what is the first step to improving your performance?  Simply put, knowing your baseline.  Whatever the endeavor, you have to keep score.  All great performers do.  As a result, the performance in most fields has been improving steadily over the last 100 years.

Consider, for instance, the Olympics. Over the last century, the best performance for every event has improved–in some cases by 50%!  The Gold Medal winning time for the marathon in the 1896 Olympics was just one minute faster than the entry time currently required just to participate in the Chicago and Boston marathons.

By contrast, the effectiveness of psychological therapies has not improved a single percentage point over the last 30 years.  How, you may wonder, could that be?  During the same time period: (1) more than 10,000 how-to books on psychotherapy have been published; (2) the number of treatment approaches has mushroomed from 60 to 400; and (3) there are presently 145 officially approved, evidenced-based, manualized treatments for 51 of the 397 possible DSM IV diagnostic groups.  Certainly, given such “growth,” we therapists must be more effective with more people than ever before.  Unfortunately, however, instead of advancing, we’ve stagnated, mistaking our feverish peddling for real progress in the Tour de Therapy.

Truth is, no one has been keeping score, least of all we individual practitioners. True, volumes of research now prove beyond any doubt that psychotherapy works.  Relying on such evidence to substantiate the effectiveness of one’s own work, however, is a bit like Tiger Woods telling you the par for a particular hole rather than how many strokes it took him to sink the ball.  The result on outcome, research indicates, is that effectiveness rates plateau very early in most therapists careers while confidence level continue to grow.

In one study, for example, when clinicians were asked to rate their job performance from A+ to F, fully two-thirds considered themselves A or better. No one, not a single person in the lot, rated him or herself as below average. As researchers Sapyta, Riemer, and Bickman (2005) conclude, “most clinicians believe that they produce patient outcomes that are well above average” (p. 146). In another study, Deirdre Hiatt and George Hargrave used peer and provider ratings, as well as a standardized outcome measure, to assess the success rates of therapists in a sample of mental health professionals. As one would expect, providers were found to vary significantly in their effectiveness. What was disturbing is that the least effective therapists in the sample rated themselves on par with the most effective!

The reason for stagnant success rates in psychotherapy should be clear to all: why try to improve when you already think your the best or, barring that, at least above average?

Here again, expanding our search for excellence beyond the narrow field of psychotherapy to the subject of expertise and expert performance in general can provide some helpful insights. In virtually every profession, from carpentry to policework, medicine to mathematics, average performers overestimate their abilities, confidently assigning themselves to the top tier. Therapists are simply doing what everyone else does. Alas, they are average among the average.

Our own work and research proves that clinicians can break away from the crowd of average achievers by using a couple of simple, valid, and reliable tools for assessing outcome. As hard as it may be to believe, the empirical evidence indicates that performance increases between 65-300% (click here to read the studies). Next time, I’ll review these simple tools as well as a few basic methods for determining exactly how effective you are. Subscribe now so you’ll be the first to know.

One more note, after posting last time, I heard from several readers who had difficulty subscribing. After doing some research, we learned that you must use IE 7 or Firefox 3.0.7 or later for the subscribe function to work properly.  Look forward to hearing from you!

In the meantime, the transcript below is of a recent interview I did for Shrinkrap radio.  It’s focused on our current work: