The practice known as “routine outcome measurement,” or ROM, is resulting in the publication of some of the biggest and most clinically relevant psychotherapy studies in history. Freed from the limits of the randomized clinical trial, and accompanying obsession with manuals and methods, researchers are finally able to examine what happens in real world clinical practice.
I’ve previously blogged about one of the largest studies of psychotherapy ever published. More than 1,400 therapists participated. The progress of over 26,000 people (aged 16-95) treated over a 12 year period in primary care settings in the UK was tracked on an ongoing basis via ROM. The results? In an average of 8 visits, 60% of those treated by this diverse group of practitioners achieved both reliable and clinically significant change—results on par with tightly controlled RCT’s. The study is a stunning confirmation of the effectiveness of psychotherapy.
This week, another mega-study was accepted for publication in the Journal of Counseling Psychology. Once more, ROM was involved. In this one, researchers Goldberg, Rousemanier, Miller, Whipple, Nielsen, Hoyt, and Wampold examined a large, naturalistic data set that included outcomes of 6500 clients treated by 170 practitioners whose results had been tracked an average of 5 years.
Their question?
Do therapists become more effective with time and experience?
Their answer? No.
For readers of this blog, such findings will not be particularly newsworthy. As I’ve frequently pointed out, experience has never proven to be a significant predictor of effectiveness.
What might be a bit surprising is that the study found clinicians’ outcomes actually worsened with time and experience. That’s right. On average, the longer a therapist practiced, the less effective they became! Importantly, this finding remained even when controlling for several patient-level, caseload-level, and therapist-level characteristics, as well as when excluding several types of outliers.
Such findings are noteworthy for a number of reasons but chiefly because they contrast sharply with results from other, equally-large studies documenting that therapists see themselves as continuously developing in both knowledge and ability over the course of their careers. To be sure, the drop in performance reported by Goldberg and colleagues wasn’t steep. Rather, the pattern was a slow, inexorable decline from year to year.
Where, one can wonder, does the disconnect come from? How can therapists’ assessments of themselves and their work be so at odds with the facts? Especially considering, in the study by Goldberg and colleagues, participating clinicians had ongoing access to data regarding their effectiveness (or lack thereof) on real-time basis! Even the study I blogged about—the largest in history where outcomes of psychotherapy were shown to be quite positive—a staggering 40% of people treated experienced little or no change whatsoever. How can such findings be reconciled with others indicating that clinicians routinely overestimate their effectiveness by 65%?
Turns out, the boundary between “belief in the process” and “denial of reality” is remarkably fuzzy. Hope is a significant contributor to outcome—accounting for as much as 30% of the variance in results. At the same time, it becomes toxic when actual outcomes are distorted in a manner that causes practitioners to miss important opportunities to grow and develop—not to mention help more clients. Recall studies documenting that top performing therapists evince more of what researchers term, “professional self-doubt.” Said another way, they are less likely to see progress where none exists and more likely to values outcomes over therapeutic process.
What’s more, unlike their more average counterparts, highly effective practitioners actually become more effective with time and experience. In the article below, my colleagues and I at the International Center for Clinical Excellence identify several evidence-based steps any practitioner follow to match such results.
