The practice known as “routine outcome measurement,” or ROM, is resulting in the publication of some of the biggest and most clinically relevant psychotherapy studies in history. Freed from the limits of the randomized clinical trial, and accompanying obsession with manuals and methods, researchers are finally able to examine what happens in real world clinical practice.
A few weeks ago, I blogged about the largest study of psychotherapy ever published. More than 1,400 therapists participated. The progress of over 26,000 people (aged 16-95) treated over a 12 year period in primary care settings in the UK was tracked on an ongoing basis via ROM. The results? In an average of 8 visits, 60% of those treated by this diverse group of practitioners achieved both reliable and clinically significant change—results on par with tightly controlled RCT’s. The study is a stunning confirmation of the effectiveness of psychotherapy.
This week, another mega-study was accepted for publication in the Journal of Counsel
ing Psychology. Once more,
ROM was involved. In this one, researchers Goldberg, Rousemanier, Miller, Whipple, Nielsen, Hoyt, and Wampold examined a large, naturalistic data set that included outcomes of 6500 clients treated by 170 practitioners whose results had been tracked an average of 5 years.
Their question?
Do therapists become more effective with time and experience?
Their answer? No.
For readers of this blog, such findings will not be particularly newsworthy. As I’ve frequently pointed out, experience has never proven to be a significant predictor of effectiveness.
What might be a bit surprising is that the study found clinicians’ outcomes actually worsened with time and experience. That’s right. On average, the longer a therapist practiced, the less effective they became! Importantly, this finding remained even when controlling for several patient-level, caseload-level, and therapist-level characteristics, as well as when excluding several types of outliers.
Such findings are noteworthy for a number of reasons but chiefly because they contrast sharply with results from other, equally-large studies documenting that therapists see themselves as continuously developing in both knowledge and ability over the course of their careers. To be sure, the drop in performance reported by Goldberg and colleagues wasn’t steep. Rather, the pattern was a slow, inexorable decline from year to year.
Where, one can wonder, does the disconnect come from? How can therapists’ assessments of themselves and their work be so at odds with the facts? Especially considering, in the study by Goldberg and colleagues, participating clinicians had ongoing access to data regarding their effectiveness (or lack thereof) on real-time basis! Even the study I blogged about previously—the largest in history where outcomes of psychotherapy were shown to be quite positive—a staggering 40% of people treated experienced little or no change whatsoever. How can such findings be reconciled with others indicating that clinicians routinely overestimate their effectiveness by 65%?
Turns out, th
e boundary between “belief in the process” and “denial of reality” is remarkably fuzzy. Hope is a significant contributor to outcome—accounting for as much as 30% of the variance in results. At the same time, it becomes toxic when actual outcomes are distorted in a manner that causes practitioners to miss important opportunities to grow and develop—not to mention help more clients. Recall studies documenting that top performing therapists evince more of what researchers term, “professional self-doubt.” Said another way, they are less likely to see progress where none exists and more likely to values outcomes over therapeutic process.
What’s more, unlike their more average counterparts, highly effective practitioners actually become more effective with time and experience. In the article below, my colleagues and I at the International Center for Clinical Excellence identify several evidence-based steps any practitioner follow to match such results.
Let me know your thoughts.
Until next time,
Scott
Scott D. Miller, Ph.D.
Registration is now open for our March Intensives in Chicago. Join colleagues from around the world for the FIT Advanced and the FIT Supervision workshops.
Hi Scott “et. al.”?
From this distance (Denmark) I love and enjoy your cont’d energetic insistence on transmitting good sane safe knowledge on what’s really happening!! As this piece.
I still have your excited “enjoy” in my ears attending my first FIT-seminar with Susanne Bargmann in Hørsholm. Enjoying applies to me still as our knowledge about our own both way possibilities accumulates.
So thanks and go on!
Yours sincerely
Piet
Scott, when you write that 60% ‘treated’ in the UK study showed ‘improvement’, how does that compare with the 2002 Hansen Lambert & Foreman study which gave us figures of 14% ‘recovered’, 21% ‘improved’, 57% ‘no change’, and 8% ‘deteriorated’? That UK study you site from the Brit J Psychiatry didn’t include the drop-outs did it? I accept the main findings of both of these recent studies – that therapy takes as long as therapy takes in the UK study, and that experience doesn’t equal more skilled therapists; but I’m dubious on the percentages improving in general. That new study was from a university counselling clinic – and the authors note that they seldom see SMIs and excluded the ‘early terminators’ from the ‘treated’ stats. I see you use the word ‘treated’ – perhaps wisely – as presumably most of the drop-outs didn’t feel ‘treated’ (although there might have been a % of one-case wonders).
I would like to know the numbers arriving at the front door of mental health services and the numbers leaving the back door ‘recovered’ – as all indications from the numbers now in prison with MH difficulties is suggesting that either a lot are not turning up or have become so disillusioned with MH services they end up elsewhere.
cheers
Nick
Scott, you’re starting to depress me with these posts! I’d like to think I’m still improving and although I have to admit I did not ready the article (yet) that you’re referencing, I wonder if complacency plays a part in the decline. I remember when I first started doing therapy in private practice, I was so nervous that I was also more alert to what was happening in sessions. I spent a lot of time before and after sessions thinking about what I wanted to do and how it went. Now that I’m more comfortable, I certainly don’t commit the same amount of time and mental energy to one session.
Sorry for that Gary! No, I don;’t think it’s complacency–at least not in the traditional sense. The bottom line is that you have to constantly and consistently work at “your edge” in order to grow and develop. It’s not an easy message, but applying the principles outlined will certainly be rewarding! Don’t despair.
Hi…
thanks for sharing this informative blog with us. great stuff. I like your content of list you included. please
keep sharing such innovative ideas with us.
Psychotherapy Shepherds Bush
Good post I think I’m going to link to this post for some of my customers as many seem to have no clue what so ever about spam
Psychotherapy Shepherds Bush
Amazing article, You have explained every point in a good manner and in a efficient way. The ideas are amazing and content is good.
Keep it up.
wonderful way of writing a blog. the best way to explain how to improve and do psychotherapists with time and experience.
Thank you !! Your blog has fun curated ideas about Counselling. Each of us has a different way of dealing with it, we may want to share it or deal with it alone. We may have feelings we feel we shouldn’t. It can be helpful talking through things at our own pace. You can refer our site to have Counselling and Psychotherapy at Notting Hill.
Very helpful and it is must to change therapy techniques. Being a Counsellor and Psychotherapist i firmly believe in the same to.
https://www.nottinghill-counselling.co.uk/
I like how you explained that a psychologist can help you talk about your problems and behavioral changes. I’d be interested to learn more about how psychologists help and provide solutions to problems in your head. Thanks for helping me understand that a psychologist is a listener of your issues.
Nice Blog! Thank you so much for sharing this kind of wonderful things.