Are you any good as a therapist? The Legacy of Paul W. Clement

March 26, 2014 By scottdm 4 Comments

Twenty years ago, I came across an article published in the journal, Professional Psychology. It was written by a psychologist in private practice, Paul Clement. The piece caught my eye for a number of reasons. First, although we’d never met, Paul lived and worked in a town near my childhood home: Pasadena, California. Second, the question he opened his article with was provocative, to say the least, “Are you any good?” In other words, how effective are YOU as a psychotherapist? Third, and most important, he had compiled and was reporting a quantitative analysis of his results over the last 26 years as a practicing clinician. It was both riveting and stunning. No one I knew had ever done had published something similar before.

In graduate school, I’d learned to administer a variety of tests (achievement, vocational, personality, projective, IQ, etc.). Not once, however, did I attend a course or sit in a lecture about how to measure my results. I was forced to wonder, “How could that be?” Six years in graduate school and not a word about evaluating one’s outcomes. After all, if we don’t know how effective we are, how are any of us supposed to improve?

What was the reason for the absence of measurement, evaluation, and analysis? It certainly wasn’t because psychotherapy wasn’t effective. A massive amount of research existed documenting the effectiveness of treatment. Paul’s research confirmed these results. Of those he’d worked with, 75% were improved at termination. Moreover, such results were obtained in a relatively brief period of time, the median number of sessions used being 12.

Other results he reported were not so easy to accept. In short, Paul’s analysis showed that his outcomes had not improved over the course of his career. At the conclusion of the piece, he observed, “I had expected to find that I had gotten better and better over the years, but my data failed to suggest any systematic change in my therapeutic effectiveness across the 26 years in question…it was a bad surprise for me.” (p. 175).

For years, I carried the article with me in my briefcase, hoping that one day, I might better understand his findings. Maybe, I thought, Clement was simply an outlier? Surely, we get better with experience. It was hard for me to believe I hadn’t improved since my first, ham-handed sessions with clients. Then again, I didn’t really know. I wasn’t measuring my results in any meaningful way.

The rest is history. Within a few short years, I was routinely monitoring the outcome and alliance at every session I did with clients. Thanks to my undergraduate professor, Michael Lambert, Ph.D., I began using the OQ 45 to assess outcomes. Another mentor, Dr. Lynn Johnson had developed a 10-item scale for evaluating the quality of the therapeutic relationship, know as the Session Rating Scale. Both tools became an integral part of the way I worked. Eventually, a suggestion by Haim Omer, Ph.D., led me to consider creating shorter, less time consuming visual analogue versions of both measures. In time, together with colleagues, the ORS and SRS were developed and tested. Throughout this process, Paul Clement, and his original study remained an important, motivating force.

Just over a year ago, Paul sent me an article evaluating 40 years of his work as a psychotherapist. Once again, I was inspired by his bold, brave, and utterly transparent example. Not only had his outcomes not improved, he reported, they’d actually deteriorated! Leave it to him to point the way! As readers of this blog know, our group is busy at work researching what it takes to forestall such deterioration and improve effectiveness. Last year, we summarized our findings in the 50th Anniversary issue of Psychotherapy. As I write, we are preparing a more detailed report for publication in the same journal.

Yesterday, I was drafting an email, responding to one I’d recently received from him, when I learned Paul had died. I will miss him. In this, I know I’m not alone.

Looking for Results in All the Wrong Places: What Makes Feedback Work?

September 16, 2012 By scottdm Leave a Comment

As anyone knows who reads this blog or has been to one of my workshops, I am a fan of feedback. Back in the mid-1990’s, I began using Lynn Johnson’s 10-item Session Rating Scale in my clinical work. His book, Psychotherapy in the Age of Accountability, and our long relationship, convinced me that I needed to check in regularly with my clients. At the same time, I started using the Outcome Questionnaire (OQ-45). The developer, Michael Lambert, a professor and mentor, was finding that routinely measuring outcome helped clinicians catch and prevent deterioration in treatment. In time, I worked with colleagues to develop a set of tools, the brevity of which made the process of asking for and receiving feedback about the relationship and outcome of care, feasible.

Initial research on the measures and feedback process was promising. Formally and routinely asking for feedback was associated with improved outcomes, decreased drop-out rates, and cost savings in service delivery! As I warned in my blogpost last February, however, such results, while important, were merely “first steps” in a scientific journey. Most importantly, the research to date said nothing about why the use of the measures improved outcomes. Given the history of our field, it would be easy to begin thinking of the measures as an “intervention” that, if faithfully adopted and used, would result in better outcomes. Not surprisingly, this is exactly what has happened, with some claiming that the measures improve outcomes more than anything since the beginning of psychotherapy. Sadly, such claims rarely live up to their initial promise. For decades the quest for the holy grail has locked the field into a vicious cycle of hope and despair, one that ultimately eclipses the opportunity to conduct the very research needed to facilitate understanding of the complex processes at work in any intervention.

In February, I wrote about several indirect, but empirically robust, avenues of evidence indicating that another variable might be responsible for the effect found in the initial feedback research. Now, before I go on, let me remind you that I’m a fan of feedback, a big fan. At the same time, its important to understand why it works and, specifically, what factors are responsible for the effect. Doing otherwise risks mistaking method with cause, what we believe with reality. Yes, it could be the measures. But, the type research conducted at the time did not make it possible to reach that conclusion. Plus, it seemed to me, other data pointed elsewhere; namely to the therapist. Consider, for example, the following findings: (1) therapists did not appear to learn from the feedback provided by measures of the alliance and outcome; (2) therapists did not become more effective over time as a result of being exposed to feedback. In other words, as with every other “intervention” in the history of psychotherapy, the effect of routinely monitoring the alliance and outcome seems to vary by therapist.

Such results, if true, would have significant implications for the feedback movement (and the field of behavioral health in general). Instead of focusing on methods and interventions, efforts to improve the outcome of behavioral health practice should focus on those providing the service. And guess what? This is precisely what the latest research on routine outcome measurement (ROM) has now found. Hot off the press, in the latest issue of the journal, Psychotherapy Research, Dutch investigators de Jong, van Sluis, Nugter, Heiser, and Spinhoven (2012) found that feedback was not effective under all circumstances. What variable was responsible for the difference? You guessed it: the therapist–in particular, their interest in receiving feedback, sense of self-efficacy, commitment to use the tools to receive feedback, and…their gender (with women being more willing to use the measures). Consistent with ICCE’s emphasis on supporting organizations with implementation, other research points to the significant role setting and structure plays in success. Simon, Simon, Harris and Lambert (2011), Reimer and Bickman (2012), and de Jong (2012) have all found that organizational and administrative issues loom large in mediating the use and impact of feedback in care.

Together with colleagues, we are currently investigating both the individual therapist and contextual variables that enable clinicians to benefit from feedback. The results are enticing. The first will be presented at the upcoming Achieving Clinical Excellence conference in Holland, May 16-18th. Other results will be reported in the 50th anniversayry issue of the journal, Psychotherapy, to which we’ve been asked to contribute. Stay tuned.

The Impact of Mentors

July 20, 2010 By scottdm Leave a Comment

Brendan Madden Scott D. Miller Jeffrey K. Zeig

A little over month ago, I blogged about how the outcome and session rating scales were originally conceived of and developed. A few days prior to that, I wrote about where the whole idea of using measures to solicit feedback had started. In both instances, my teachers and supervisors played a significant role. Immediately following a two day workshop I’d given in Israel, psychologist Haim Omer suggested developing a visual analog scale that could be used in lieu of the longer Outcome Questionnaire 45.2–an idea that literally changed the entire arc of my professional career. Drs. Lynn Johnson and Michael Lambert–a supervisor and professor I met and worked with as a graduate student–were the first to pioneer feedback-informed treatment (FIT). Some twenty plus years into my career, I remain in contact with both, calling, seeking input, discussing ideas, and soliciting feedback.

“Professional coaches,” says the noted “expert on experts” K. Anders Erickson, “…play an essential role in guiding…future experts in a safe and effective manner” (p. 698). Needless to say, I’ve been very fortunate to have such visionary mentors. One more story.

In 1984, I wrote a letter to Dr. Jeffrey K. Zeig, the director of the Milton H. Erickson Institute. I was in my second year of a Ph.D. program in psychology and, like many graduate students, dead broke. While taking a course on hypnosis as part of my studies, I’d become interested in the work of Milton Erickson.

“I’d like to learn more,” I wrote at the time, “Would it be possible for me to visit the Institute, watch some videos and have a chance to talk with you?” I wasn’t too far away. I could drive to Phoenix where the Institute was located. I could even arrange to stay with friends to save money. “Dr. Zeig,” I continued, “I’m a graduate student and don’t have much money, but I’d be willing to do some work in kind.” I’d pasted mailing labels on thousands of brochures for the local hypnosis and therapy organizations, for example, in exchange for being able to attend professional continuing education events. “I’ll vacuum and clean the office, wash vehicles, do filing. Whatever might be helpful to you or the Institute.”

Within a couple of weeks, an envelop from the Milton H. Erickson Institute arrived. In it was a letter that was brief and to the point. “Please call me,” it said, and was signed Jeffrey K. Zeig, Ph.D. Needless to say, I called straight away. We chatted for a few minutes. He told me that I was welcome to visit the Institute, watch videos, talk with some of the staff and even spend some time with him. And then he asked, “Do you think you could afford five dollars?” I was floored.

Ever since meeting him on that hot summer day in Phoenix, he’s been an important teacher and mentor. It’s particularly noteworthy that whenever we talk–by phone, email, or in chance meetings on airplanes while criss-crossing the globe–he invariably asks, “What are you learning?” And then he listens, intently.

Last week, we were catching up on the phone and Jeff told me that his long-held desire to open an international psychotherapy training and research facility had finally been fulfilled. Briefly, The Institute for Applied Therapeutic Change is a real clinic where professionals and students can learn the latest in behavioral healthcare from leading experts in the field and while working with real clients (click on the text above for the complete press release).

“I can hardly wait to attend some of the events,” I said. “And when are you available to teach?” he responded. Stunned again. I’m so fortunate and can hardly wait to participate in the Institute activities as both a presenter and student. Stay tuned to the Foundation website for more details!