Clinician Beware: Ignoring Research Can be Hazardous to Your Professional (and Economic) Health

September 25, 2010 By scottdm Leave a Comment

“Studies show…”

“Available data indicate…”

“This method is evidence-based…”

My how things have changed. Twenty years ago when I entered the field, professional training, continuing education events, and books rarely referred to research or evidence. Now, everyone refers to the “data.” The equation is simple: no research = no money. Having “an evidence-base” increasingly determines book sales, attendance at continuing education events, and myriad other funding and reimbursement decisions.

So what do the data actually say? S adly, the answer is often, “it depends on who you ask.” If you read the latest summary and treatment recommendations for post-traumatic stress disorder (PTSD) posted by the Cochrane Collaboration, you are told that TFCBT and EMDR are the most effective, “state of the art” treatments on offer. Other summaries, as I recently blogged about, arrive at very—even opposite—conclusions; namely, all psychotherapies (trauma-focused and otherwise) work equally well in the treatment of PTSD. For the practicing clinician (as well as other consumers of research), the end result is confusion and, dare I say, despair.

Unable to resolve the discrepant findings, the research is either rejected out of hand (“it’s all crap anyway”) or cherry-picked (“your research is crap, mine is good”). In a world where experts disagree–and vehemently–what is the average Joe or Jane therapist to do?

Fortunately, there is another way, beyond agnosticism and instead of fundamentalism. In a word, it is engagement. This last week, I spent 5 days teaching an intensive workshop with ICCE Senior Associate Susanne Bargmann to a group of Danish psychologists on “Statistics and Research Design.” That’s right. Five days, 6 hours a day spent away from work and clients learning how to understand, read, and conduct research.

The goal of the training was simple and straight-forward: help practitioners learn to evaluate the methods and meanings, strengths and weaknesses, and political and paradigmatic influences associated with research and evidentiary claims. At the conclusion of the five days, none of those assembled had difficulty engaging with and understanding the reasons for the seemingly discrepant findings noted above. As a result, they could state with confidence “what works” with PTSD, helping clarify this not only to colleagues, payers, and policy members but also to consumers of behavioral health services.

The “Statistics and Research Design” course will be held again in Denmark in 2011. If the experience of this year’s participants proves anything, it is that, “The only thing therapists have to fear about statistics and research design, is fear itself.” Please contact Vinther and Mosgaard directly for more information.

Finally, as part of the International Center for Clinical Excellence (ICCE) efforts to improve the quality and outcome of behavioral health services worldwide, two additional intensive trainings will be offered in Chicago, Illinois (USA). First, the “Advanced Training in Feedback-Informed Treatment (FIT).” And second, the annual “Training of Trainers.” In the Advanced Training, participants learn:

· The empirical foundations of feedback-informed clinical work (i.e., empirically supported factors underlying successful clinical work, the impact of feedback on performance)

· Clinical skills for enhancing client engagement that cut across different therapeutic orientations and diverse treatment populations

· How to integrate outcome management tools (including one or more of the following: ORS, SRS, CORE, and OQ 45) into clinical practice

· How to use the outcome management tools to inform and improve service delivery

· How to significantly improve your clinical skills and outcomes via feedback and deliberate practice

· How to use data generated from outcome measures to inform management, supervision, and training decisions

· Strategies for successful implementation of CDOI and FIT in your organization or practice

Need more information about the course? Email us or click on the video below to hear more about the course. In the meantime, space is limited so register early at: http://www.eventbrite.ie/o/the-international-centre-for-clinical-excellence-298540255.

What Works in the Treatment of Post Traumatic Stress Disorder? The Definitive Study

September 15, 2010 By scottdm 1 Comment

What works in the treatment of people with post-traumatic stress? The influential Cochrane Collaboration–an “independent network of people” whose self-professed mission is to help “healthcare providers, policy makers, patients, their advocates and carers, make well-informed decisions, concludes that, “non trauma-focused psychological treatments [do] not reduce PTSD symptoms as significantly…as individual trauma focused cognitive-behavioral therapy (TFCBT), eye movement desensitization and reprocessing, stress mamangement and group TFCBT.” The same conclusion was reached by the National Institute for Health and Clinical Excellence (or NICE) in the United Kingdom which has developed and disseminated practice guidelines that unequivocally state that , “all people with PTSD should be offered a course of trauma focused psychological treatment (TFCBT) or eye movement desensitization and reprocessing (EMDR).” And they mean all: adults and kids, young and old. Little room for left for interpretation here. No thinking is required. Like the old Nike ad, you should: “Just do it.”

Wait a minute though…what do the data say? Apparently, the NICE and Cochrane recommendations are not based on, well…the evidence–at least, that is, the latest meta-analytic research! Meta-analysis, you will recall, is a procedure for aggregating results from similar studies in order to test a hypothesis, such as, “are certain approaches for the treatment of post traumatic stress more effective than others?” A year ago, I blogged about the publication of a meta-analysis by Benish, Imel, & Wampold which clearly showed that there was no difference in outcome between treatments for PTSD and that the designation of some therapies as “trauma-focused” was devoid of empirical support, a fiction.

So, how to account for the differences? In a word, allegiance. Although written by scientists, so-called “scholarly” reviews of the literature and “consensus panel” opinions inevitably reflect the values, beliefs, and theoretical predilections of the authors. NICE guidelines, for example, read like a well planned advertising campaign for single psychotherapeutic modality: CBT. Indeed, the organization is quite explicit in it’s objective: “provide support for the local implementation of…appropriate levels of cognitive beheavioral therapy.” Astonishingly, no other approach is accorded the same level of support or endorsement despite robust evidence of the equivalence of outcomes among treatment approaches. Meanwhile, the review of the PTSD literature and treatment recommendations published by the Cochrane Collaboration has not been updated since 2007–a full two years following the publication of the Benish et al. (2008) meta-analysis–and that was penned by a prominent advocate of…CBT…Trauma-focused CBT.

As I blogged about back in January, researchers and prominent CBT proponents, published a critique of the Benish et al. (2008) meta-analysis in the March 2010 issue of Clinical Psychology Review (Vol. 30, No. 2, pages 269-76). Curiously, the authors chose not to replicate the Benish et al. study, but rather claim that bias, arbitrariness, lack of transparency, and poor judgement accounted for the findings. As I promised at the time, I’m making the response we wrote–which appeared in the most recent issue of Clinical Psychology Review—available here.

Of course, the most important finding of the Benish et al. (2008) and our later response (Wampold et al. 2010) is that mental health treatments work for people with post traumatic stress. Such a conclusion is unequivocal. At the same time, as we state in our response to the critique of Benish et al. (2008), “there is little evidence to support the conclusion…that one particular treatment for PTSD is superior to others or that some well defined ingredient is crucial to successful treatments of PTSD.” Saying otherwise, belies the evidence and diverts attention and scarce resources away from efforts likely to improve the quality and outcome of behavioral health services.

View more documents from Scott Miller.

DODO BIRD HYPOTHESIS PROVEN FALSE! Study of PTSD finally proves Wampold, Miller, and other "common factor" proponents wrong

January 8, 2010 By scottdm 3 Comments

Researchers Anke Ehlers, Jonathon Bisson, David Clark, Mark Creamer, Steven Pilling, David Richards, Paula Schnurr, Stuart Turner, and William Yule have finally done it! They slayed the “dodo.” Not the real bird of course–that beast has been extinct since the mid to late 17th century but rather the “dodo bird” conjecture first articulated by Saul Rozenzweig, Ph.D. in 1936. The idea that all treatment approaches work about equally well has dogged the field–and driven proponents of “specific treatments for specific disorders” positively mad. In a soon to be published article in Clinical Psychology Review, the authors claim that bias, overgeneralization, lack of transparency, and poor judgement account for the finding that “all therapeutic approaches work equally well for people with a diagnosis of PTSD” reported in a meta-analysis by Benish, Imel, & Wampold (2008).

I guess this means that a public admission by me, Wampold, and other common factors researchers is in order…or maybe not! Right now, we are writing a response to the article. All I can say at this point is, “unbelievable!” As soon as it becomes available, you’ll find it right here on this blog. I’ll be drawing inspiration from Saul Rosenzweig who passed away in 2004. It was such an honor to meet him. Still working at 96 years of age.

Whoa Nellie! A 25 Million Dollar Study of Treatments for PTSD

October 27, 2009 By scottdm 1 Comment

I have in my hand a frayed and yellowed copy of observations once made by a well known trainer of horses. The trainer’s simple message for leading a productive and successful professional life was, “If the horse you’re riding dies, get off.”

You would think the advice straightforward enough for all to understand and benefit. And yet, the trainer pointed out, “many professionals don’t always follow it.” Instead, they choose from an array of alternatives, including:

Buying a strong whip
Switching riders
Moving the dead horse to a new location
Riding the dead horse for longer periods of time
Saying things like, “This is the way we’ve always ridden the horse.”
Appointing a committee to study the horse
Arranging to visit other sites where they ride dead horses more efficiently
Increasing the standards for riding dead horses
Creating a test for measuring our riding ability
Complaining about how the state of the horse the days
Coming up with new styles of riding
Blaming the horse’s parents as the problem is often in the breeding.

When it comes to the treatment of post traumatic stress disorder, it appears the Department of Defense is applying all of the above. Recently, the DoD awarded the largest grant ever awarded to “discover the best treatments for combat-related post-traumatic stress disorder” (APA Monitor). Beneficiaries of the award were naturally ecstatic, stating “The DoD has never put this amount of money to this before.”

Missing from the announcements was any mention of research which clearly shows no difference in outcome between approaches intended to be therapeutic—including, the two approaches chosen for comparison in the DoD study! In June 2008, researchers Benish, Imel, and Wampold, conducted a meta-analysis of all studies in which two or more treatment approaches were directly compared. The authors conclude, “Given the lack of differential efficacy between treatments, it seems scientifically questionable to recommend one particular treatment over others that appear to be of comparable effectiveness. . . .keeping patients in treatment would appear to be more important in achieving desired outcomes than would prescribing a particular type of psychotherapy” (p. 755).

Ah yes, the horse is dead, but proponents of “specific treatments for specific disorders” ride on. You can hear their rallying cry, “we will find a more efficient and effective way to ride this dead horse!” My advice? Simple: let’s get off this dead horse. There are any number of effective treatments for PTSD. The challenge is decidedly not figuring out which one is best for all but rather “what works” for the individual. In these recessionary times, I can think of far better ways to spend 25 million than on another “horse race” between competing therapeutic approaches. Evidence based methods exist for assessing and adjusting both the “fit and effect” of clinical services—the methods described, for instance, in the scholarly publications sections of my website. Such methods have been found to improve both outcome and retention by as much as 65%. What will happen? Though I’m hopeful, I must say that the temptation to stay on the horse you chose at the outset of the race is a strong one.