Why ongoing, formal feedback is critical for improving outcomes in healthcare

January 8, 2010 By scottdm 3 Comments

Not long ago, I had a rather lengthy email exchange with a well-known, high profile psychotherapist in the United States. Feedback was the topic. We both agreed that feedback was central to successful psychotherapy. We differed, however, in terms of method. I argued for the use of simple, standardized measures of progress and alliance (e.g., ORS and SRS). In support of my opinion, I pointed to several randomized clinical trials documenting the impact of routine outcome monitoring on retention and progress. I also cited studies showing traditionally low correlations between consumers and clinician’s rating of outcome and alliance and clinicians frighteningly frequent inability to predict deterioration and drop out in treatment. He responded that such measures were an “unnecessary intrusion,” indicating that he’d always sought feedback from his clients albeit on an “informal basis.” television-reception

When I mentioned our own research which had found that clinicians believed they asked consumers for feedback more often than they actually did, he finally seemed to agree with me. “Of course,” he said immediately–but then he added, “I don’t need to ask in order to get feedback.” In response to my query about how he managed to get feedback without asking, he responded (without a hint of irony), “I have unconditional empathic reception.” Needless to say, the conversation ended there.

It’s a simple idea, feedback. Yet, as I jet around the globe teaching about feedback-informed clinical practice, I’m struck by how hard it seems for many in healthcare to adopt. Whatever the reason for the resistance–fear, hubris, or inertia–the failure to seek out valid and reliable feedback is a conceit that the field can no longer afford. Simply stated, no one has “unconditional empathic reception.” As the video below makes clear, we all need help seeing what is right before our eyes.

Magical Moments in Kansas

January 2, 2010 By scottdm Leave a Comment

Registrations are already coming in for the first International Conference on “Achieving Clinical Excellence.”

Not too long ago, I announced that internationally known researcher K. Anders Erickson, Ph.D.–the “expert on experts”–had agreed to present at the event. At that time, I also indicated that a number of internationally accomplished performers from a variety of professions (including psychology, business, medicine, science, music, entertainment, and sports) would be with us in Kansas City to teach and inspire. Attendees will learn the science and skills for achieving their personal best as a behavioral health practitioner by literally learning from the best.

Practice, as anyone who has been reading my recent blogosts and publications knows, is key for achieving excellence–and not just any old kind will do. To be effective, it must be deliberate, reflective, and ongoing. What’s more, it must be accompanied with high levels of support and detailed instruction from exceptional teachers.

No performer embodies these two principles–dedicated practice and exceptional teaching–better than FISM gold-medial winning magician, Michael Ammar. Magic magazine named him one of the 100 most influential magicians of the century.

Michael will open the second day of the the conference. First, he’ll perform. What can I say? You’ll be astonished and amazed. After that, Michael will talk about the role of practice in achieving excellence. He’s a master teacher who has spent years studying the elements of successful practice. He’ll not only inspire you, he will provide you with the means to excel.

Join us for three action packed days of fun, science and skill building. In the meantime, if you have a spare moment, watch Michael Ammar perform one of the classics of magic: the cups and balls.

The Study of Excellence: A Radically New Approach to Understanding "What Works" in Behavioral Health

December 24, 2009 By scottdm 2 Comments

“What works” in therapy? Believe it or not, that question–as simple as it is–has and continues to spark considerable debate. For decades, the field has been divided. On one side are those who argue that the efficacy of psychological treatments is due to specific factors (e.g., changing negative thinking patterns) inherent in the model of treatment (e.g., cognitive behavioral therapy) remedial to the problem being treated (i.e., depression); on the other, is a smaller but no less committed group of researchers and writers who posit that the general efficacy of behavioral treatments is due to a group of factors common to all approaches (e.g., relationship, hope, expectancy, client factors).

While the overall effectiveness of psychological treatment is now well established–studies show that people who receive care are better off than 80% of those who do not regardless of the approach or the problem treated–one fact can not be avoided: outcomes have not improved appreciably over the last 30 years! Said another way, the common versus specific factor battle, while generating a great deal of heat, has not shed much light on how to improve the outcome of behavioral health services. Despite the incessant talk about and promotion of “evidence-based” practice, there is no evidence that adopting “specific methods for specific disorders” improves outcome. At the same time, as I’ve pointed out in prior blogposts, the common factors, while accounting for why psychological therapies work, do not and can not tell us how to work. After all, if the effectiveness of the various and competing treatment approaches is due to a shared set of common factors, and yet all models work equally well, why learn about the common factors? More to the point, there simply is no evidence that adopting a “common factors” approach leads to better performance.

The problem with the specific and common factor positions is that both–and hang onto your seat here–have the same objective at heart; namely, contextlessness. Each hopes to identify a set of principles and/or practices that are applicable across people, places, and situations. Thus, specific factor proponents argue that particular “evidence-based” (EBP) approaches are applicable for a given problem regardless of the people or places involved (It’s amazing, really, when you consider that various approaches are being marketed to different countries and cultures as “evidence-based” when there is in no evidence that these methods work beyond their very limited and unrepresentative samples). On the other hand, the common factors camp, in place of techniques, proffer an invariant set of, well, generic factors. Little wonder that outcomes have stagnated. Its a bit like trying to learn a language either by memorizing a phrase book–in the case of EBP–or studying the parts of speech–in the case of the common factors.

What to do? For me, clues for resolving the impasse began to appear when, in 1994, I followed the advice of my friend and long time mentor, Lynn Johnson, and began formally and routinely monitoring the outcome and alliance of the clinical work I was doing. Crucially, feedback provided a way to contextualize therapeutic services–to fit the work to the people and places involved–that neither a specific or common factors informed approach could.

Numerous studies (21 RCT’s; including 4 studies using the ORS and SRS) now document the impact of using outcome and alliance feedback to inform service delivery. One study, for example, showed a 65% improvement over baseline performance rates with the addition of routine alliance and outcome feedback. Another, more recent study of couples therapy, found that divorce/separation rates were half (50%) less for the feedback versus no feedback conditions!

Such results have, not surprisingly, led the practice of “routine outcome monitoring” (PROMS) to be deemed “evidence-based.” At the recent, Evolution of Psychotherapy conference I was on a panel with David Barlow, Ph.D.–a long time proponent of the “specific treatments for specific disorders” (EBP)–who, in response to my brief remarks about the benefits of feedback, stated unequivocally that all therapists would soon be required to measure and monitor the outcome of their clinical work. Given that my work has focused almost exclusively on seeking and using feedback for the last 15 years, you would think I’d be happy. And while gratifying on some level, I must admit to being both surprised and frightened by his pronouncement.

My fear? Focusing on measurement and feedback misses the point. Simply put: it’s not seeking feedback that is important. Rather, it’s what feedback potentially engenders in the user that is critical. Consider the following, while the results of trials to date clearly document the benefit of PROMS to those seeking therapy, there is currently no evidence of that the practice has a lasting impact on those providing the service. “The question is,” as researcher Michael Lambert notes, “have therapists learned anything from having gotten feedback? Or, do the gains disappear when feedback disappears? About the same question. We found that there is little improvement from year to year…” (quoted in Miller et al. [2004]).

Research on expertise in a wide range of domains (including chess, medicine, physics, computer programming, and psychotherapy) indicates that in order to have a lasting effect feedback must increase a performer’s “domain specific knowledge.” Feedback must result in the performer knowing more about his or her area and how and when to apply than knowledge to specific situations than others. Master level chess players, for example, have been shown to possess 10 to 100 times more chess knowledge than “club-level” players. Not surprisingly, master players’ vast information about the game is consilidated and organized differently than their less successful peers; namely, in a way that allows them to access, sort, and apply potential moves to the specific situation on the board. In other words, their immense knowledge is context specific.

A mere handful studies document similar findings among superior performing therapists: not only do they know more, they know how, when, and with whom o apply that knowledge. I noted these and highlighted a few others in the research pipeline during my workshop on “Achieving Clinical Excellence” at the Evolution of Psychotherapy conference. I also reviewed what 30 years of research on expertise and expert performance has taught us about how feedback must be used in order to insure that learning actually takes place. Many of those in attendance stopped by the ICCE booth following the presentation to talk with our CEO, Brendan Madden, or one of our Associates and Trainers (see the video below).

Such research, I believe, holds the key to moving beyond the common versus specific factor stalemate that has long held the field in check–providing therapists with the means for developing, organizing, and contextualizing clinical knowledge in a manner that leads to real and lasting improvements in performance.

The Evolution of Psychotherapy: Meeting Michael Hoyt

December 16, 2009 By scottdm 2 Comments

I’m still reeling from the experience in Anaheim this last week. I met so many leaders in the field, heard so many presentations on cutting edge clinical practice–as well as was reminded of some “classic” principles of effective psychotherapy.

One of the people I met was colleague and friend, Michael F. Hoyt, Ph.D. Michael and I go back 15+ years, having met–I believe–the first time at a workshop I was giving in Northern California (somewhere in the Bay Area where Michael works and resides). Since that time, we chatted regularly, and written editorials and book chapters together. His books (The First Session in Brief Therapy, Brief Therapy & Managed Care, The Handbook of Constructive Therapies, Some Stories are Better than Others) always balance theory and practice and are among my favorites.

My two favorite books are also his most recent: The Present is a Gift and Brief Psychotherapies: Principles & Practice (Hint: his chapters on couples therapy are among the best I’ve ever read). Anyway, the two of us caught up at the ICCE booth this last week at the Evolution conference.

The Crown Jewel of Research on CDOI: Professor Jan Blomqvist receives 2.9 million crown grant for RCT on feedback in Sweden

October 20, 2009 By scottdm 2 Comments

If you’ve been following me on Twitter, then you know that last week I was touring and teaching in different spots around Europe. First, I presented two days in Copenhagen. Then I keynoted the British Association of Counseling and Psychotherapy Conference in Newcastle, England. Early Saturday morning, I flew from London to Stockholm. My long time friend and associate, Gunnar Lindfelt picked me up at Arlanda airport and drove me back to his lovely home in the city. There, we gorged on smoked salmon, “svensk godies” (small candies, my favorite of which is “skum bananer”–dark chocolate covered marshmellow in the shape of a banana) and Cider–a non-alcoholic fizzy apple drink that is an old time Swedish favorite.

It was Gunnar Lindfeldt, a gifted clinician and expert in the treatment of drug and alcohol problems, who first introduced me to the work of Swedish psychologist Jan Blomqvist. In 1998, Blomqvist published a book entitled, “Beyond Treatment? Widening the Approach to Alcohol Problems and Solutions“ in which he made the provocative argument that common rather than specific factors held the key to effective care. Since writing the book, Jan Blomqvist has continued his research and is currently a full professor at SORAD, the Centre for Social Research on Alcohol and Drugs at Stockholm University.

Anyway, I had the pleasure of meeting with Professor Blomqvist at his home in Uppsala, Sweden this last week. Over homemade spinach soup, freshly-baked bread and cheese, we chatted about the state of the field. The pièce de résistance, however, was hearing about the 2.9 million Swedish crown grant he had just been awarded for a 4 year long study of outcome-informed treatment of alcohol problems, called “Putting the Client in the Driver’s Seat.”

The study to be conducted by Professor Blomqvist will be the largest, most comprehensive, randomized clinical trial on client-directed outcome informed clinical work. A centerpience of the study will be the routine use of the ORS and SRS and provision of feedback in the delivery of treatment services. Importantly, unlike all other studies to date, this project completely avoids claims of “allegiance effects” as no developers of measures or supporters of CDOI are participating. Stay tuned to the “Top Performance” blog for additional updates! While you are waiting, take a moment and read Professor Blomqvist’s provocative take on “addiction” in slide viewer below.

J Blomqvist 3 from Scott Miller

The Field, the Future, and Feedback

October 2, 2009 By scottdm 1 Comment

There is an old (but in many ways sad) joke about two clinicians–actually, the way I first heard the story, it was two psychiatrists. The point of the story is the same regardless of the discipline of the provider. Anyway, two therapists meet in the hallway after a long day spent meeting clients. One, the younger of the two, is tired and bedraggled. The other, older and experienced, looks the same as s/he did at the start of the day: eyes bright and attentive, hair perfectly groomed, clothes and appearance immaculate. Taken aback by the composure of the more experienced colleague, the younger therapist asks, “How do you do it? How do you listen to the trials and tribulations, the problem and complaints, the dire lives and circumstances of your clients, minute and minute, hour upon hour…and yet emerge at the end of the day in such good shape?” Slowly shaking his head from left to right, the older and more experienced clinician immediately reached out, tapping the less experienced colleague gently on the shoulder, and then after removing the thick plugs stuffed into both of his years, said, “Excuse me, what did you say?”

Let’s face it: healthcare is in trouble. Behavioral healthcare in particular is in even worse shape. And while solutions from politicians, pundits, industry insiders and professionals are circulating in Washington with all the sound and fury of a hurricane, the voice of consumers is largely absent. Why? Of course, many of the barriers between providers and consumers are systemic in nature and as such, out of the control of average clinicians and consumers. Others, however, are local and could be addressed in an instance with a modicum of interest and attention on the part of professionals.

Chief among the steps practitioners could take to bridge to chasm between them and consumers is the adoption of routine, ongoing feedback. Seeking and utlizing real-time feedback from consumers has the added advantage of significantly boosting outcomes and increasing retention in services (several studies documenting the impact of feedback are available in the “Scholarly publications and Handouts” section of my website). Healthcare providers can download two well validated and easy-to-use scales right now for free by clicking on the Performance Metrics tab to the left.

So far, however, few in healthcare seem interested and others are downright hostile to the idea of asking consumers for input. Consider the following story by reporter Lindsey Tanner entitled, “Take two, call me in the morning…and keep it quiet.” Tanner discovered that some in healthcare are demanding that people (patients. clients, consumers) sign “gag orders” prior to being treated–agreeing in effect not to post comments about the provider (negative and otherwise) to online sites such as Zagats.com, Angieslist.com, and RateMds.com. According to the article, a Greensboro, N.C. company, ironically called “Medical Justice” is, for a fee, now providing physicians with standardized waiver agreements and advising all doctors to have patients sign on the dotted line. And if the patient refuses? Simple: find another doctor.

Can you imagine a hotel chain or restaurant asking you to sign a legally-binding agreement not to disclose your experience prior to booking your room or handing you the menu? Anyone who has travelled lately knows the value of the information contained on consumer-driven websites such as TripAdvisor.com. It’s outlandish really–except in healthcare.

To be sure, there is at least one important difference between healthcare and other service industries. Specifically, healthcare providers, unlike business owners and service managers, are prevented from responding to online complaints by existing privacy laws. However, even if this problem were insurmountable–which it is not–how then can one explain the continuing reluctance on the part of professionals to give people access to their own healthcare records? And this despite federal regulations under the Health Insurance Portability and Accountability Act (HIPAA) permitting complete and unfettered access (click here to read the recent NPR story on this subject). Clearly, the problem is not legal but rather cultural in nature. Remember when Elaine from Seinfeld asked to see her chart?

Earlier this summer, my family and I were vacationing in Southwest Michigan. One day, after visiting the beach and poking around the shops in the lakeside town of South Haven, we happened on a small Italian bistro named,Tello. Being from a big city famous for its good eats, I’ll admit I wasn’t expecting much. The food was delicious. More surprising, was the service. Not only were the staff welcoming and attentive, but at the end of the meal, when I thought the time had come to pay the bill, the folder I was given contained a small PDA rather than the check. I was being asked for my feedback.Answering the questions took less than a minute and the manager, Mike Sheedy, appeared at our table within moments of my hitting the “send” button. He seemed genuinely surprised when I asked if he felt uncomfortable seeking feedback so directly. “Have you learned anything useful?” I then inquired. “Of course,” he answered immediately, “just last week a customer told us that it would be nice to have a children’s menu posted in the window alongside the standard one.” I was dumbstruck as one of the main reasons we had decided to go into the restaurant rather than others was because the children’s menu was prominently displayed in the front window!

How NOT to Achieve Clinical Excellence: The Sorry State of Continuing Professional Education

September 30, 2009 By scottdm 5 Comments

Greg Neimeyer, Ph.D., is causing quite a stir in continuing education circles. What has he done? In several scholarly publications, he’s reviewed the existing empirical literature and found that continuing professional education in heavioral health is not particularly, well, …educational. Indeed, in a soon-to-be published piece in the APA journal, Professional Psychology, he notes, “While the majority of studies report high levels of participants’ satisfaction with their CE experiences, little attention has been paid to assessing actual levels of learning, the translation of learning into practice, or the impact of CE on actual professional service delivery outcomes.” Neimeyer then goes on to cite a scholarly review published in 2002 by Daniels and Walter which pointed out that “a search [of the research literature] revealed no controlled studies of the impact of continuing education in the…behavioral health disciplines” (p. 368). Said another way, the near ubiguitous mandate that clinicians attend so many hours per year of approved “CE” events in order to further their knowledge and skill base has no empirical support.

Personally, my guess is that any study that might be done on CE in Behavioral Health would show little or no impact on performance anyway. Why? Studies in other fields (i.e., medicine, flight training) have long documented that traditional CE activities (i.e., attending conferences, lectures, reading articles) have no demonstrable effect. So, what does work? The same research that calls the efficacy of current CE activities into questions provide clear guidance: namely, brief, circumscribed, skill-based training, followed by observed practice, real-time feedback, and performance measurement. Such characteristics are, in fact, part and parcel of expert performance in any field. And yet, it is virutally non-existent in behavioral health.

Let me give you an example of a CE offering that arrived in my box just this week. The oversized, multi-color, tri-fold brochure boldly asserts a workshop on CBT featuring the “top evidence-based techniques.” Momentarily setting aside the absolute lack of evidence in support of such trainings, consider the promised content–and I’m not kidding: clinical applications of cognitive behavior therapy, motivational interviewing, cognitive therapy, mindfulness and acceptance based therapies, and behavior therapy. As if that were not enough, the outline for the training indicates that participants will learn 52 other bulleted points, including but not limited to: why CBT, integration of skills intro practice, identifying brain-based CBT strategies, the latest research on CBT, the stages of change, open-ended and reflective listening, behavioral activiation, acceptance and commitment, emotional regulation and distrss tolerance skills, the ABC technique to promote rational beliefs, homework assignments that test core beliefs, rescripting techniques for disturbing memories and images…and so on…AND ALL IN A SINGLE 6 HOUR DAY! You say you have no money? Your agency has suffered budget cuts? No worries, the ad states in giant print, as the same content is available via CD, web and podcast.

Such an agenda defies not only the evidence but strains credulity to the breaking point. Could anyone accomplish so much in so little time? Clinicians deserve and should demand more from the CE events they register for and, in many instances, are mandated to attend in order to maintain licensure and certification. The International Center for Clinical Excellence web platform will soon be launched. The mission of the site, as indicated in my blog post of August 25th, is to “support clinical excellence through creating virtual clinical networks, groups and clinical communities where clinicians can be supported in the key behavior changes required for developing clinical excellence.” Members of the site will use a variety of social networking and collaborative tools to learn skills, obtain real-time feedback, and measure their performance. Anyway, kudos to Dr. Greg Neimeyer for confronting the ugly truth about CE in behavioral health and saying it out loud!