What’s in an Acronym? CDOI, FIT, PCOMS, ORS, SRS … all BS?

June 7, 2014 By scottdm Leave a Comment

“What’s in a name?”

–William Shakespeare

A little over a week ago, I received an email from Anna Graham Anderson, a graduate student in psychology at Aarhus University in Denmark. “I’m writing,” she said, “in hopes of receiving some clarifications.”

Anna Graham Anderson

Without reading any further, I knew exactly where Anna was going. I’d fielded the same question before. As interest in measurement and feedback has expanded, it comes up more and more frequently.

Anna continued, “I cannot find any literature on the difference between CDOI, FIT, PCOMS, ORS, and SRS. No matter where I search, I cannot find any satisfying clues. Is it safe to say they are the same?” Or, as another asked more pointedly, “Are all these acronyms just a bunch of branding B.S.?”

I answered, “B.S.? No. Confusing? Absolutely. So, what is the difference?”

As spelled out in each of the six treatment and training manuals, FIT, or feedback-informed treatment, is, “a panetheoretical approach for evaluating and improving the quality and effectiveness of behavioral health services. It involves routinely and formally soliciting feedback from consumers regarding the therapeutic relationship and outcome of care and using the resulting information to inform and tailor service deliver.”

Importantly, FIT is agnostic regarding both the method of treatment and the particular measures a practitioner may employ. Some practitioners use the ORS and SRS, two brief, simple-to-use, and free measures of progress and the therapeutic relationship–but any other valid and reliable scales could be used.

Of all the acronyms associated with my work, CDOI is the one I no longer use. For me, it had always problematic as it came precariously close to being a treatment model, a way of doing therapy. I wasn’t interested in creating a new therapeutic approach. My work and writing on the common factors had long ago convinced me the field needed no more therapeutic schools. The phrase, “client-directed, outcome-informed” described the team’s position at the time, with one foot in the past (how to do therapy), the other in the future (feedback).

And PCOMS? A long time ago, my colleagues and I had a dream of launching a web-based “system for both monitoring and improving the effectiveness of treatment” (Miller et. al, 2005). We did some testing at an employee assistance program in located in Texas, formed a corporation called PCOMS (Partners for Change Outcome Management System), and even hired a developer to build the site. In the end, nothing happened. Overtime, the acronym, PCOMS, began to be used as an overall term referring to the ORS, SRS, and norms for interpreting the scores. In February 2013, the Substance Abuse and Mental Health Service Adminstration (SAMHSA) formally recognized PCOMS as an evidence-based practice. You can read more about PCOMS at: www.whatispcoms.com.

I expect there will be new names and acronyms as the work evolves. While some remain, others, like fossils, are left behind; evidence of what has come before, their sum total a record of development over time.

The Importance of "Whoops" in Improving Treatment Outcome

December 2, 2012 By scottdm Leave a Comment

“Ring the bells that still can ring,
Forget your perfect offering
There is a crack in everything,
That’s how the light gets in.”

Leonard Cohen, Anthem

Making mistakes. We all do it, in both our personal and professional lives. “To err is human…,” the old saying goes. And most of us say, if asked, that we agree whole heartedly with the adage–especially when it refers to someone else! When the principle becomes personal, however, its is much more difficult to be so broad-minded.

Think about it for a minute: can you name five things you are wrong about? Three? How about the last mistake you made in your clinical work? What was it? Did you share it with the person you were working with? With your colleagues?

Research shows there are surprising benefits to being wrong, especially when the maker views such errors differently. As author Alina Tugend points out in her fabulous book, Better by Mistake, custom wrongly defines a mistake as ” the failure of a planned sequence of mental or physical activities to achieve its intended outcome.” When you forget a client’s name during a session or push a door instead of pull, that counts as slip or lapse. A mistake, by contrast, is when “the plan itself is inadequate to achieve it’s objectives” (p. 11). Knowing the difference, she continues, “can be very helpful in avoiding mistakes in the future” because it leads exploration away from assigning blame to the exploring systems, processes, and conditions that either cause mistakes or thwart their detection.

Last week, I was working with a talented and energetic group of helping professionals in New Bedford, Massachusetts. The topic was, “Achieving Excellence: Pushing One’s Clinical Performance to the Next Level of Effectiveness.” As part of my presentation, I talked about becoming more, “error-centric” in our work; specifically, using ongoing measurement of the alliance to identify opportunities for improving our connection with consumers of behavioral health services. As an example of the benefits of making mistakes the focus of professional development efforts, I showed a brief video of Rachel Hsu and Roger Chen, two talented musicians who performed at the last Achieving Clinical Excellence (ACE) conference. Rachel plays a piece by Liszt, Roger one by Mozart. Both compositions are extremely challenging to play. You tell me how they did (by the way, Rachel is 8 years old, Roger. 9):

Following her performance, I asked Rachel if she’d made any mistakes during her performance. She laughed, and then said, “Yes, a lot!” When I asked her what she did about that, she replied, “Well, its impossible to learn from my mistakes while I’m playing. So I note them and then later practice those small bits, over and over, slow at first, then speeding up, until I get them right.”

After showing the video in New Bedford, a member of the audience raised his hand, “I get it but that whole idea makes me a bit nervous.” I knew exactly what he was thinking. Highlighting one’s mistakes in public is risky business. Studies documenting that the most effective clinicians experience more self-doubt and are more willing to admit making mistakes is simply not convincing when one’s professional self-esteem or job may be on the line. Neither is research showing that health care professionals who admit making mistakes and apologize to consumers are significantly less likely to be sued. Becoming error centric, requires a change in culture, one that not only invites discloure but connects it with the kind of support and structure that leads to superior results.

Creating a “whoops-friendly” culture will be a focus of the next Achieving Clinical Excellence conference, scheduled for May 16-18th, 2013 in Amsterdam, Holland. Researchers and clinicians from around the world will gather to share their data and experience at this unique event. I promise you don’t want to miss it. Here’s a short clip of highlights from the last one:

My colleague, Susanne Bargmann and I will also be teaching the latest research and evidence based methods for transforming mistakes into improved clinical performance at the upcoming FIT Advanced Intensive training in Chicago, Illinois. I look forward to meeting you at one of these upcoming events. In the meantime, here’s a fun, brief but informative video from the TED talks series on mistakes:

By the way, the house pictured above is real. My family and I visited it while vacationing in Niagara Falls, Canada in October. It’s a tourist attraction actually. Mistakes, it seems, can be profitable.

What is the Real Source of Effectiveness in Smoking Cessation Treatment? New Research on Feedback Informed Treatment

November 24, 2012 By scottdm Leave a Comment

When it rains, it pours! So much news to relay regarding recent research on Feedback Informed Treatment (FIT). Just received news this week from ICCE Associate Stephen Michaels that research using the ORS and SRS in smoking cessation treatment is in print! A few days prior to that, Kelley Quirk sent a copy of our long-awaited article on the validity and reliability of the Group Session Rating Scale. On that very same day, the editors from the journal Psychotherapy sent proofs of an article written by me, Mark Hubble, Daryl Chow, and Jason Seidel for the 50th anniversary issue of the publication.

Let’s start with the validity and reliability study. Many clinicians have already downloaded and been using Group Session Rating Scale. The measure is part of the packet of FIT tools available in 20+ languages on both my personal and the International Center for Clinical Excellence websites. The article presents the first research on the validity and reliability of the measure. The data for the study was gathered at two sites I’ve worked with for many years. Thanks to Kelley Quirk and Jesse Owen for crunching the numbers and writing up the results! Since the alliance is one of the most robust predictors of outcome, the GSRS provides yet another method for helping therapists obtain feedback from consumers of behavior health services.

Moving on, if there were a Nobel Prize for patience and persistence, it would have to go to Stephen Michaels, the lead author of the study, Assessing Counsellor Effects on Quit Rates and Life Satisfactions Scores at a Tobacco Quitline” (Michael, Seltzer, Miller, and Wampold, 2012). Over the last four years, Stephen has trained Quitline staff in FIT, implemented the ORS and SRS in Quitline tobacco cessation services, gathered outcome and alliance data on nearly 3,000 Quitline users, completed an in-depth review of the available smoking cessation literature, and finally, organized, analyzed, and written up the results.

What did he find? Statistically significant differences in quit rates attributable to counselor effects. In other words, as I’ve been saying for some time, some helpers are more helpful than others–even when the treatment provided is highly manualized and structured. In short, it’s not the method that matters (including the use of the ORS and SRS), it’s the therapist.

What is responsible for the difference in effectiveness among therapists? The answer to that question is the subject of the article, “The Outcome of Psychotherapy: Yesterday, Today, and Tomorrow” slated to appear in the 50th anniversary issue of Psychotherapy. In it, we review controversies surround the question, “What makes therapy work?” and tip findings from another, soon-to-be-published empirical analysis of top performing clinicians. Stay tuned.

Clinical Support Tools for the ORS and SRS

November 20, 2012 By scottdm 1 Comment

I have so much to be grateful for at this time. Most of all, I’m happy to be home with my family. As we have in the past, this year we’ll be spending the holiday at the home of our long time friends John and Renee Dalton. The two always put out a fantastic spread and our son, Michael, is fast friends with their two kids.

I’m also grateful for the International Center for Clinical Excellence (ICCE) community. Currently, ICCE has over 4200 members located around the world, making the organization the largest, web-based community of professionals, educators, managers, and clinicians dedicated to using feedback to pursue excellence in the delivery of behavioral health services. Recently, the site was highlighted as one of the best resources for practitioners available on the web. Articles, how-to videos, and discussion forums are available everyday, all day–and for free! No come-ons for books or webinars and no “cult of personality”–just sharing among peers. If you are not a member, you can join at: www.centerforclinicalexcellence.com

A special thanks goes to several ICCE senior advisors and associates, including Susanne Bargmann, Jason Seidel, Cynthia Maeschalck, Bob Bertolino, Bill Plum, Julie Tilsen, and Robbie Babbins-Wagner. These folks are the backbone of the organization. Together, they make it work. Most recently, we all joined together to create the ICCE Feedback Informed Treatment and Training Manuals, a cutting edge series covering every aspect of FIT–from the empirical foundations to implementation–in support of our application to SAMSHA for recognition as an “evidence-based practice.”

As a way of supporting everyone using the ORS and SRS, I wanted to make a couple of clinical support tools available. If you are using the measures, the first item will need no introduction. It’s a 10 cm ruler! Save the file and print it off and you also have a ready reminder of the upcoming Achieving Clinical Excellence conference, coming up in May 2013. Like last time, this will feature the latest inforamtion about feedback informed practice! The second item is a reliable change graph. If you are using the paper and pencil measures, rather than one of the existing web based systems (www.fit-outcomes.com, www.myoutcomes.com), you can use this tool to determine whether a change in scores from session to session is reliable (that is, greater than chance, the passage of time, and measurement error [and therefore, due to the care being provided]) or even clinically significant (that is, both reliable and indicating recovered). The last item is an impressive summary of various systems for monitoring progress in treatment.

In addition ACE Health have developed openFIT, a plug-in which seamlessly integrates the ORS, SRS and associated algorithms into any existing Electronic Health Record, Case Management System of eMental Health application.

I wish everyone a peaceful and rewarding Thanksgiving holiday.

An Easy Way to Improve Our Schools (and Psychotherapy)

November 13, 2012 By scottdm Leave a Comment

If you didn’t see the October Atlantic Monthly, you really missed a great issue. In it, Amanda Ripley wrote a delightful and informative article about a simple and straightforward method for improving the performance of the public schools: have kids grade teachers. What kind of grades you ask? Not those on standardized achievement tests, and certainly not measures of a teacher’s popularity. Although both of those methods are widely advocated and used, neither has proven particularly predictive of student performance. Rather, grades should be based on how well teachers engage students; particularly whether the kids believe the teacher makes them want to work hard, pay attention, understand the course material, and identify and correct their mistakes. Indeed, in thousands of surveys, kids as young as kindergartners “can identify with uncanny accuracy, their most–and least effective teachers.”

The findings stood out for me not only because I am a parent but also because they mirror results from psychotherapy research. First, data gathered over the last three decades documents that client engagement is the number one process-related predictor of treatment outcome. Second, a growing number of studies indicates that clients can identify “with uncanny accuracy” the most and least effective treatment services and providers. Importantly, this same body of evidence shows that client assessments of their sessions and progress can be used to enhance treatment results in general as well as the effectiveness and skill level of individual clinicians.

Viewpoint clearly matters–and in the case of schools and psychotherapy, it is the recipient of the service whose opinion we should be seeking. In her article, Ripley identifies the types of questions that can be used in schools. If you are a therapist, two brief, simple-to-use scales are available for free. Research has shown that regularly using the measures to solicit client feedback improves both retention in and outcome of psychotherapy. The largest, international professional community dedicated to enhancing the quality and outcome of behavioral healthcare is available to support you in your use of the tools. There, you will find a wealth of information, discussion forums, and how-to videos available at no charge 24 hours a day, 7 days a week, 365 days a year.

In March, many members from around the world will be joining colleagues from around the world for four days of intensive training. Why not join us? We work and play hard. Rest assured that by the end of the four days, you’ll be playing an “A” game. Click here to register today. In the meantime, here’s what participants from last year said about the event.

National Psychotherapy Day: A Recognition, Celebration, and Call for Action

September 24, 2012 By scottdm Leave a Comment

With all the challenges facing the profession, it is important to highlight people and organizations that are working hard to make a difference. On that note, tomorrow, Tuesday the 25th of September 2012 is the very first National Psychotherapy Day. Having a day of unified, active promotion of psychotherapy is the brain child of psychologist Ryan Howes. At his side is the Psychotherapy Foundation (PF), a nonprofit foundation, dedicated to promoting the therapeutic relationship as an “effective, long-lasting, collaborative approach” to resolving emotional, behavioral, and relational problems. What’s not to like? Dr. Howes and the PF are encouraging people who have seen a therapist to talk or blog about their experience. They are calling on therapists to commit to sharing research documenting the effectiveness of psychotherapy with the public (write a letter to the editor of your local paper, offer to do an interview, give a brief presentation at the Chamber of Commerce).

Surveys show that the two primary barriers to seeking the help of a therapist are: (1) cost of the service (cited by 81%); and (2) lack of confidence in the outcome of therapy (78%). Of these two barriers, the first is entirely understandable. Times are tough and treatment costs money. It is for these this reason that Dr. Howe and the PF are asking all who participate in the day to support their local, low-fee counseling centers in whatever way possible.

The second barrier is more troubling and, frankly, difficult to understand and address. Research overwhelmingly supports the efficacy of psychological treatment. Indeed, the American Psychological Association issued a rare, formal resolution this last month recognizing the effectiveness of psychotherapy! Listen to the language:

Whereas the effects of psychotherapy …are widely accepted to be significant and large;
Whereas the results of psychotherapy tend to last long and be less liely to equire additional treatment courses than psychopharmacological treatments;
Whereas comparisons of different forms of psychotherapy most often result in relatively nonsignificant difference, and contextual and relationship factors (not captured by a patient’s diagnosis or by the therapists use of a specific psychotherapy) mediate or moderate outcomes;
Whereas the best research evidence conclusively shows that individual, group, and couple/family psychotherapy are effective for a broad range of…problems with children, adolescents, adults, and older adults;
THEREFORE be it resolved that, as a healing practice and professional service, psychotherapy is effective and highly cost effective…and should be included in the health care system as an established evidence-based practice.

Strong words, right? Even so, it’s very clear that the public’s lingering doubts about effectiveness will require than a proclamation. It is for this reason that Dr. Howes and PF are asking all those currently in care to provide constructive feedback to their therapist. Therapists, in turn, are encouraged to seek and respond to feedback from their clients. As reviewed here on this blog, numerous studies document the positive impact that routine feedback from clients has on retention and outcome of service. Free evidence-based tools are available for download from this website for soliciting formal feedback from consumers. Plus, the International Center for Clinical Excellence web-based community–the largest group of clinicians and researchers dedicated to improving the quality and outcome of psychotherapy via the use of ongoing feedback–stands ready and willing to be of support.

So, why the turquoise? Well, its’ the official color of National Psychotherapy Day. To show your support, Dr. Howes and PF are asking all to wear something with that color tomorrow.

Looking for Results in All the Wrong Places: What Makes Feedback Work?

September 16, 2012 By scottdm Leave a Comment

As anyone knows who reads this blog or has been to one of my workshops, I am a fan of feedback. Back in the mid-1990’s, I began using Lynn Johnson’s 10-item Session Rating Scale in my clinical work. His book, Psychotherapy in the Age of Accountability, and our long relationship, convinced me that I needed to check in regularly with my clients. At the same time, I started using the Outcome Questionnaire (OQ-45). The developer, Michael Lambert, a professor and mentor, was finding that routinely measuring outcome helped clinicians catch and prevent deterioration in treatment. In time, I worked with colleagues to develop a set of tools, the brevity of which made the process of asking for and receiving feedback about the relationship and outcome of care, feasible.

Initial research on the measures and feedback process was promising. Formally and routinely asking for feedback was associated with improved outcomes, decreased drop-out rates, and cost savings in service delivery! As I warned in my blogpost last February, however, such results, while important, were merely “first steps” in a scientific journey. Most importantly, the research to date said nothing about why the use of the measures improved outcomes. Given the history of our field, it would be easy to begin thinking of the measures as an “intervention” that, if faithfully adopted and used, would result in better outcomes. Not surprisingly, this is exactly what has happened, with some claiming that the measures improve outcomes more than anything since the beginning of psychotherapy. Sadly, such claims rarely live up to their initial promise. For decades the quest for the holy grail has locked the field into a vicious cycle of hope and despair, one that ultimately eclipses the opportunity to conduct the very research needed to facilitate understanding of the complex processes at work in any intervention.

In February, I wrote about several indirect, but empirically robust, avenues of evidence indicating that another variable might be responsible for the effect found in the initial feedback research. Now, before I go on, let me remind you that I’m a fan of feedback, a big fan. At the same time, its important to understand why it works and, specifically, what factors are responsible for the effect. Doing otherwise risks mistaking method with cause, what we believe with reality. Yes, it could be the measures. But, the type research conducted at the time did not make it possible to reach that conclusion. Plus, it seemed to me, other data pointed elsewhere; namely to the therapist. Consider, for example, the following findings: (1) therapists did not appear to learn from the feedback provided by measures of the alliance and outcome; (2) therapists did not become more effective over time as a result of being exposed to feedback. In other words, as with every other “intervention” in the history of psychotherapy, the effect of routinely monitoring the alliance and outcome seems to vary by therapist.

Such results, if true, would have significant implications for the feedback movement (and the field of behavioral health in general). Instead of focusing on methods and interventions, efforts to improve the outcome of behavioral health practice should focus on those providing the service. And guess what? This is precisely what the latest research on routine outcome measurement (ROM) has now found. Hot off the press, in the latest issue of the journal, Psychotherapy Research, Dutch investigators de Jong, van Sluis, Nugter, Heiser, and Spinhoven (2012) found that feedback was not effective under all circumstances. What variable was responsible for the difference? You guessed it: the therapist–in particular, their interest in receiving feedback, sense of self-efficacy, commitment to use the tools to receive feedback, and…their gender (with women being more willing to use the measures). Consistent with ICCE’s emphasis on supporting organizations with implementation, other research points to the significant role setting and structure plays in success. Simon, Simon, Harris and Lambert (2011), Reimer and Bickman (2012), and de Jong (2012) have all found that organizational and administrative issues loom large in mediating the use and impact of feedback in care.

Together with colleagues, we are currently investigating both the individual therapist and contextual variables that enable clinicians to benefit from feedback. The results are enticing. The first will be presented at the upcoming Achieving Clinical Excellence conference in Holland, May 16-18th. Other results will be reported in the 50th anniversayry issue of the journal, Psychotherapy, to which we’ve been asked to contribute. Stay tuned.

Feedback Informed Treatment: Update

August 16, 2012 By scottdm Leave a Comment

Chicago, IL (USA)

The last two weeks have been a whirlwind of activity here in Chicago. First, the “Advanced Intensive.” Next came the annual “Training of Trainers.” Each week, the room was filled to capacity with practitioners, researchers, supervisors, and agency directors from around the globe receiving in-depth training in feedback-informed practice. It was a phenomenal experience. As the video below shows, we worked and played hard!

Already, people are signing up for the next “Advanced Intensive” scheduled for the third week of March 2013 and the new three-day intensive training on FIT supervision scheduled for the 6-9th of August 2013. Both events follow and are designed to complement the newly released ICCE FIT Treatment and Training Manuals. In fact, all participants receive copies of the 6 manuals, covering every detail of FIT practice, from the empirical evidence to implementation. The manuals were developed and submitted to support ICCE’s submission of FIT to the National Registry of Evidence Based Practices (NREPP). As I blogged about last March, ICCE trainings fill up early. Register today and get the early bird discount.

A Lotta Help from One’s Friends: The Role of Community in the Pursuit of Excellence

August 3, 2012 By scottdm Leave a Comment

Dateline: Chicago, IL USA

Hard not to be impressed with the USA Women’s Gymnastic team. What skill, percision, expertise, and excellence.

By now, I’m sure you’ve seen the interviews. In all instances, each and every one has focused on the team. Despite some in the media attempting to make stars out of the individual members, the atheletes have continually highlighted, “The Team.” When asked to account for their success or the source of their ambition, the reason cited has been: THE TEAM.

Sixteen year old McKayla Maroney said, “I think we’re as close as we can be. We’ve all been working and training together for a long time…I’ve known (fellow team member) Kyla since I was 6 years old. We are all best of friends. They did so great today and I just love this team so much.”

As highlighted in our recent article, “The Road to Mastery,” excellence does not occur in a vacuum. Surrounding every great performer is a community (teachers, coaches, mentors, and peers). In the busy world that is modern clinical practice, where can practitioners finda trsutworthy and supportive community of peers? A group of colleagues that will challenge them to keep growing as professionals and people?

In a word, the ICCE. In December 2009, the International Center for Clinical Excellence was launched and since them become the largest, global, web-based community of clinicians, researchers, administrators, and policy makers dedicated to excellence in behavioral health. The ICCE has it’s own gold-medal winning team! Practitioners working together in locations around the globe.

Practitioners like Jason Seidel, Psy.D., who represented ICCE at last week’s meeting of the American Psychological Association. Jason presented on Feedback Informed Treatment (FIT) and then participated in a panel discussion on Practice Based Evidence together with Paul Clement, Michael Lambert, Bill Stiles, Carol Goodheart, and David Barlow. Jason rocked the packed house with his tight summary of the empirical support for FIT and argument in favor of practice-based evidence!

Then there’s Daryl Chow, a psychologist from Singapore, who is currently finishing up a quantitative study of “Supershrinks.” His research is the first to employ a sophisticated statistical analysis of therapists practices related to superior outcomes. Suffice it to say, his results are mind blowing. Daryl’s work won him a scholarship to this year’s “Training of Trainers” course. If you’re not signed up for that event, you can meet him today by joining the ICCE and looking him up!

There are many, many other dedicated and supportive members. Join and share your expertise with the community today!

A Handy "Little Helper" for the Outcome Rating Scale: A Freebie from the ACE Conference Committee

April 24, 2012 By scottdm Leave a Comment

This last week the planning committee for the upcoming Achieving Clinical Excellence (ACE) conference meet once again in Horsholm, Denmark. In the picture from left to right: Liz Plutt, Bill Andrews, myself, Rick Plutt (Conference Chair), and Bogdan Ion. Taking the photo was Susanne Bargmann.

The agenda for the three day event is now set: (1) one day pre-conference on feedback informed treatment (FIT); (2) two days of plenaries and presentations by an international group of clinicians, researchers, and educators.

On day one, the conference kicks off with a keynote address by the world’s “expert on expertise,” Dr. K. Anders Ericsson. Throughout the day, other speakers will translate Dr. Ericsson’s research into practical steps for enhancing the performance of mental health professionals, agencies, and systems of care.

Day two kicks off with a keynote address by Dr. Robbie Wagner addressing the question, “what barriers stand in the way of improving our effectiveness?” Once again, the rest of the day will be spent identifying solutions for the problems standing in the way of expertise and expert performance.

We still have several openings for presentations at the conference. If you have experiences or data related to: (1) measuring outcomes; (2) implementing feedback informed treatment; (3) the qualities of super effective clinicians or treatment approaches, then PLEASE click go to the ICCE website and submit a description for consideration.

It’ll be a fun, inspiring, and rewarding three days in Amsterdam. Don’t miss it! Register today and get the early bird special, saving you 100’s of dollars!

In the meantime, click on the link below to download a handy little tool for scoring the Outcome and Session Rating Scales. It’s a combination bookmark and 10 centimeter ruler.

Ace Ruler (PDF Format)

Mental Health Practice in a Global Economy

April 17, 2012 By scottdm 2 Comments

Did you feel it? The seismic shift that occurred in field of mental health just a little over a month ago? No? Nothing? Well, in truth, it wasn’t so much a rip in the space-time continuum as a run. That “run,” however, promises to forever alter the fabric of clinical practice–in particular how clinicians earn and maintain a certain standard of living.

For decades, licensing statutes have protected behavioral health professionals from competing with providers living outside of their state and local jurisdiction. In order to bill or receive reimbursement, mental health professionals needed to be licensed in the state in which treatment services were offered. Over the years, the various professional organizations have worked to make it easier for professionals to become licensed when they move from one state to the another. Still, it ain’t easy and, some practitioners and professional groups would argue, for good reason. Such laws, to some extent, insure that fees charged for services are commensurate with the cost of living in the place where therapists live and work. The cost of therapy in Manhattan varies considerably, for example, depending on whether one is talking about the city located in state of New York or Kansas.

As far as outcomes are concerned, however, there is no evidence that people who pay more necessarily get better results. Indeed, as reviewed here on this blog, available evidence indicates little or no difference in outcome between highly trained (and expensive) clinicians and minimally trained (and less expensive) para-professionals and students. If the traditional geographic (licensing) barriers were reduced or eliminated, consumers would with few exceptions gravitate to the best value for their money. In the 1980’s and 90’s, for example, comsumers deserted small, Main Street retailers when big box stores opened on the outskirts of town offering the same merchandise at a lower price. Now, big box retailers are closing en masse as consumers shift their purchases to less expensive, web based outlets.

And that’s precisely the shift that began a little over a month ago in the field of mental health. The U.S. Military eliminated the requirement that civilian providers be licensed in the same jurisdiction or state in which treatment is offered. The new law allows care to be provided wherever the receipient of services lives and regardless of where the provider is licensed. Public announcements argued that the change was needed to make services available to service members and veterans living in isolated or rural areas where few providers may be available. Whatever the reason, the implications are profound: in the future, clinicians, like Main Street retailers, will be competing with geographically distant providers.

Just one week prior to the announcement by the U.S. Military, I posted a blogpost highlighting a recent New York Times column by author and trend watcher, Thomas Friedman. In it, I argued that “Globalization and advances in information technology were…challenging the status quo…access. At one time, being average enabled one to live an average life, live in an average neighborhood and, most importantly, earn an average living. Not so anymore. Average is now plentiful, easily accessible, and cheap. What technology can’t do in either an average or better way, a younger, less-trained but equally effective provider can do for less. A variety of computer programs and web-based systems provide both psychological advice and treatment.”

Truth is, the change is likely to be a boon to consumers of mental health services: easier access to services at a better price. What can clinicians do? First, begin measuring outcome. Without evidence of their effectiveness, individual providers will lose out to the least expensive provider. No matter how much people complain about “big box and internet retailers,” most use them. The savings are too great to ignore.

What else can clinicians do? The advice of Friedman, which I quoted in my recent blogpost, applies, “everyone needs to find their extra–their unique value contribution that makes them stand out in whatever is their field.” Measuring outcome and finding that “something special” is what the International Center for Clinical Excellence is all about. If you are not a member, please join the thousands of other professionals online today. After that, why not spend time with peers and cutting edge instructors at the upcoming “advanced intensive” or “training of trainers” workshops this summer.

The Outcome and Session Rating Scales: Support Tools

March 30, 2012 By scottdm 6 Comments

Japan, Sweden, Norway, Denmark, Germany, France, Israel, Poland, Chile, Guam, Finland, Hungary, Mexico, Australia, China, the United States…and many, many more. What do all these countries have in common? In each, clinicians and agencies are using the ORS and SRS scales to inform and improve behavioral health services. Some are using web-based systems for administration, scoring, interpretation and data aggregation (e.g., myoutcomes.com and fit-outcomes), many are accessing paper and pencil versions of the measures for free and then administering and scoring by hand.

Even if one is not using a web-based system to compare individual client progress to cutting edge norms, practitioners can still determine simply and easily whether reliable change is being made by using the “Reliable Change Chart” below. Recall, a change on the ORS is considered reliable when the difference in scores exceeds the contribution attributable to chance, maturation, and measurement error. Feel free to print out the graph and use it in your practice.

To learn how to get the most out of the measures, be sure and download the six FIT Treatment and Training Manuals. The six manuals cover every aspect of feedback-informed practice including: empirical foundations, basic and advanced applications (including FIT in groups, couples, and with special populations), supervision, data analysis, and agency implementation. Each manual is written in clear, step-by-step, non-technical language, and is specifically designed to help practitioners and agencies integrate FIT into routine clinical practice. Indeed, the manuals were submitted as part of ICCE’s application for consideration of FIT as an “evidence-based practice” to the National Registry of Evidence-Based Programs and Practices

ORS Reliable Change Chart

A Progress Report on the Science (and Art ) of Psychotherapy: The Psychotherapy Networker 30th Anniversary Edition

March 18, 2012 By scottdm Leave a Comment

The 30th Anniversary Edition of the Psychotherapy Networker has hit newsstands. In it, is an article by Diane Cole taking the measure of psychotherapy. Her question? Has the field gotten any better over the last three decades? The entire issue is a “must read,” starting with editor Rich Simon’s lengthy and thought provoking editorial, “Still Crazy After All These Years.”

Even if you are not a subscriber, much of the current edition is available FOR FREE online at the Networker website. It is an honor that the work that I have been doing on excellence and expert performance, together with many Senior Associates at ICCE (Susanne Bargman, Cynthia Maeschalck, Julie Tilsen, Rob Axsen, Jason Seidel, and Bob Bertolino) is featured prominently in this special issue magazine.

Don’t miss it! And don’t miss the Networker conference scheduled this week in Washington, D.C. I’ll be there on Friday delivering the luncheon keynote address and a workshop on pushing your clinical performance to the next level of effectiveness!

The Achieving Clinical Excellence Conference CALL FOR PAPERS

March 13, 2012 By scottdm Leave a Comment

In October 2010, the first annual “Achieving Clinical Excellence” was held in Kansas City, Missouri. A capacity crowd joined leading experts on the subject of top performance for three days worth of training and inspiration. K. Anders Ericsson reviewed his groundbreaking research, popularized by Malcolm Gladwell and others. ICCE Director, Scott D. Miller translated the research into speciific steps for improving clinical performance. Finally, classical piansts David Helfgott, Rachel Hsu, and Roger Chen, demonstrated what can be accomplished when such evidence-based strategies are applied to the process of learning specific skills.

The ICCE is proud to announce the 2nd “ACE” conference to be held May 16th-18th, 2013 in Amsterdam, Holland. Join us for three educational, inspiring, and fun-filled days. Register today and receive a significant “Early Bird” discount. The ACE conference committee is also issuing an international “Call for Papers.” If you, your agency, or practice are committed to excellence, using outcomes to inform practice, or have published research on the subject, please visit the conference website to submit a proposal.

Here’s what attendees said about the last event:

Implementation Science, FIT, and the Training of Trainers

March 8, 2012 By scottdm Leave a Comment

The International Center for Clinical Excellence (ICCE) is pleased to announce the 6th annual Training of Trainers event to be held in Chicago, Illinois August 6th-10th, 2012. As always, the ICCE TOT prepares participants provide training, consultation, and supervision to therapists, agencies, and healthcare systems in Feedback-Informed Treatment (FIT). Attendees leave the intensive, hands-on training with detailed knowledge and skills for:

Training clinicians in the Core Competencies of Feedback Informed Treatment (FIT/CDOI);
Using FIT in supervision;
Methods and practices for implementing FIT in agencies, group practices, and healthcare settings;.
Conducting top training sessions, learning and mastery exercises, and transformational presentations.

Multiple randomized clinical trials document that implementing FIT leads to improved outcomes and retention rates while simultanesouly decreasing the cost of services.

This year’s “state of the art” faculty include: ICCE Director, Scott D. Miller, Ph.D., ICCE Training Director, Julie Tilsen, Ph.D., and special guest lecturer and ICCE Coordinator of Professional Development, Cynthia Maeschalck, M.A.

Join colleagues from around the world who are working to improve the quality and outcome of behavioral healthcare via the use of ongoing feedback. Space is limited. Click here to register online today. Last year, one participants said the training was, “truly masterful. Seeing the connection between everything that has been orchestrated leaves me amazed at the thought, preparation, and talent that has cone into this training.” Here’s what others had to say:

Getting FIT: Another Opportunity

February 4, 2012 By scottdm Leave a Comment

The March Advanced Intensive in Feedback Informed Treatment is full! Not a single space left. For several weeks, we put folks on a waiting list. When that reached nearly 20, we told most they’d probably have to wait until next year to attend.

Wait no more!

The ICCE is pleased to announce a second, “Advanced Intensive” Training schedule for July 30th through August 2nd, 2012 in Chicago, IL, USA. If you’ve read the books and attended a one or two day introductory workshop and want to delve deeper in your understanding and use of the principles and practices of FIT, this is the training for you! Multiple randomized clinical trials document that FIT improves outcomes and retention rates while decreasing costs of behavioral health.

Four intensive days focused on skill development. Participants will receive a thorough grounding in:

The empirical foundations of FIT (i.e., research supporting the common factors, outcome and alliance measures, and feedback)
Alliance building skills that cut across different therapeutic orientations and diverse client populations
How to use outcome management tools (including one or more of the following: ORS, SRS, CORE, and OQ 45 to inform and improve treatment)
How to determine your overall clinical success rates
How to significantly improve your outcome and retention rate via feedback and deliberate practice
How to use technology for support and improvement of the services you offer clients and payers
How to implement FIT in your setting or agency

The training venue is situated along the beautiful “Magnificent Mile,” near Northwestern hospital, atop a beautiful tall building steps from the best retail therapy and jazz clubs in Chicago. As always, the conference features continental breakfast every morning, a night of Blues at one of Scott’s favorite haunts and dinner at arguably the best Italian restaurant in Chicago.

Unlike any other training, the ICCE “Advanced Intensive” offers both pre and post attendance support to enhance learning and retention. All participants are provided with memberships to the ICCE Trainers Forum where they can interact with the course instructors and participants, download coarse readings, view “how-to” videos, and reach out to and learn from the thousands of other member-clinicians around the world.

Don’t wait. Register today here.

If you are interested in hanging out in Chicago a few extra days, why not register for both the “Advanced Intensive” and the 2012 “Training of Trainers” workshop? Thanks to the demand, for the first time ever, the two events are being held back to back. Sign up for both events by May 31st and receive 25% off for the trainings! To obtain your discount code for both events, email: events@centerforclinicalexcellence.com today.

Looking Back, Looking Forward

January 6, 2012 By scottdm Leave a Comment

Bidding goodbye to last year and welcoming the new always puts me in a reflective frame of mind. How did my life, work, and relationships go? What are my hopes for the future?

Just two short years ago, together with colleagues from around the world, the International Center for Clinical Excellence (ICCE) was launched. Today, the ICCE is the largest, global, web-based community of providers, educators, researchers, and policy makers dedicated to improving the quality and outcome of behavioral health services. Clinicians can choose to participate in any of the 100-plus forums, create their own discussion group, immerse themselves in a library of documents and how-to videos, and consult directly with peers. Membership costs nothing and the site is free of the advertising. With just a few clicks, practitioners are able to plug into a group of like-minded clinicians whose sole reason for being on the site is to raise everyone’s performance level. I have many people to thank for the success of ICCE: senior associates and trainers, our community manager Susanne Bargmann, director of training Julie Tilsen, and our tech wizard Enda Madden.

As membership in ICCE has grown from a few hundred to well over 3000, many in the community have worked together to translate research on excellence into standards for improving clinical practice. Routine outcome monitoring (ROM) has grown in popularity around the world. As a result, new measures and trainings have proliferated. In order to insure quality and consistency, a task force was convened within ICCE in 2010 to develop a list of “Core Competencies”—a document establishing the empirical and practice foundations for outcome-informed clinical work. In 2011, the ICCE Core Competencies were used to develop and standardize the curricula for the “Advanced Intensive” and “Training-of-Trainers” workshops as well as the exam all attendees must pass to achieve certification as an ICCE Trainer. As if these accomplishments were not enough, a small cadre of ICCE associates banded together to compose the Feedback Informed Treatment and Training Manuals—six practical, “how-to”volumes covering everything from empirical foundations to implementation. None of this would have been possible without the tireless contributions of Bob Bertolino, Jason Seidel, Cynthia Maeschalck, Rob Axsen, Susanne Bargmann, Bill Robinson, Robbie Wagner, and Julie Tilsen.

Looking back, I feel tremendous gratitude–both for the members, associates, and trainers of ICCE as well as the many people who have supported my professional journey. This year, two of those mentors passed away: Dick Fisch and James Hillman. During my graduate school years, I read James Hillman’s book, Suicide and the Soul. Many years later, I had the opportunity to present alongside him at the “Evolution of Psychotherapy” conference. Dick, together with his colleagues from MRI, had a great influence on my work, especially during the early years when I was in Milwaukee with Insoo Berg and Steve de Shazer in Milwaukee doing research and writing about brief therapy. Thinking about Dick reminded me of two other teachers and mentors from that period in my life; namely, John Weakland and Jay Haley.

Looking forward, I am filled with hope and high expectations. The “Advanced Intensive” training scheduled for March 19-22^nd is booked to capacity—not a single spot left. Registrations for this summer’s “Training of Trainers” course are coming in at a record pace (don’t wait if you are thinking about joining me, Cynthia and Rob). Currently, I am awaiting word from the National Registry of Evidence Based Programs and Practices (NREPP) formally recognizing “Feedback Informed Treatment” (FIT) as an evidence-based approach. The application process has been both rigorous and time-consuming. It’s worth it though. Approval by this department within the federal government would instantly raise awareness about as well as increased access to funding for implementing FIT. Keep your fingers crossed!

There’s so much more:

Professor Jan Blomqvist, a researcher at the Center for Alcohol and Drug Research at Stockholm University (SoRAD) launched what will be the largest, independent evaluation of feedback informed treatment to date, involving 80+ clinicians and 100’s of clients located throughout Sweden. I provided the initial training to clinicians in October of last year. ICCE Certified Trainers Gunnar Lindfeldt and Magnus Johansson are providing ongoing logistic and supervisory support.
The most sophisticated and empirically robust interpretive algorithms for the Outcome Rating Scale (based on a sample of 427,744 administrations of the ORS, in 95,478 unique episodes of care, provided by 2,354 different clinicians) have been developed and are now available for integration into software and web based applications. Unlike the prior formulas–which plotted the average progress of all consumers successful and not–the new equations provide benchmarks for comparing individual consumer progress to both successful and unsuccessful treatment episodes.
The keynote speakers and venue for the Second Achieving Clinical Excellence Conference have been secured. We’ll be meeting at one of the nicest hotels in Amsterdam, Holland, May 16-18=9^th, 2013. Thanks go to the planning committee: Bill Andrews, Susanne Bargmann, Liz Plutt, Rick Plutt, Tony Jordan, and Bogdan Ion. Please visit the conference website and submit a proposal for a workshop or presentation.
Finally, I’ve been asked to deliver the lunchtime keynote at the upcoming Psychotherapy Networker Conference scheduled on March 23, 2012. The topic? Achieving excellence as a behavioral health practitioner. Last year, my colleague Mark Hubble and I published the lead article in the May-June issue of the magazine, describing the latest research on top performing clinicians. I’m deeply honored by the opportunity to speak at this prestigious event.

More coming in the weeks ahead. Until then, look forward to connecting on ICCE.

Cutting Edge Feedback

November 22, 2011 By scottdm Leave a Comment

Earth | Time Lapse View from Space, Fly Over | NASA, ISS

Using feedback to guide and improve the quality and outcome of behavioral health services is growing in popularity. The number of systems available for measuring, aggregating, and interpreting the feedback provided by consumers is increasing. The question, of course, is, “which is best?” And the answer is, “it depends on the algorithms being used.”

Over a decade ago, my colleagues and I developed a set of mathematic equations that enabled us to plot the “expected treatment response” or ETR of a client based on their first session Outcome Rating Scale (ORS) score. Although the math was complicated, the idea was not: therapists and clients could compare outcomes from session to session to the benchmark provided by the ETR. If too much or too little progress were being made, client and therapist could discuss what changes might be made to the services being offered in order to insure more effective or durable progress. It was a bold idea and definately “cutting edge” at the time–after all, 10 years ago, few people were even measuring outcomes let alone trying to provide benchmarks for guiding clinical practice. The formulas developed at that time for plotting change in treatmentare still being used by many around the world with great effect. At the same time, it was merely a first attempt.

I am proud and excited to be able to announce the development and launch of a new set of algorithms–the largest and most sophisticated to date–based on a sample of 427,744 administrations of the ORS, in 95,478 unique episodes of care, provided by 2,354 different clinicians. Unlike the prior formulas–which plotted the average progress of all consumers successful and not–the new equations provide benchmarks for comparing individual consumer progress to both successful and unsuccessful treatment episodes. Consider an analogy to the field of medicine. No one would be interested in a test for the effectiveness of a particular cancer treatment that compared an individual’s progress to to the average of all patients whether they lived or died. People want to know, “will I live?” And in order to answer that question, the ETR of both successful and ultimately unsuccessful treatments must be determined and the individual clients progress compared to both benchmarks. Adjustments can be made to the services offered when the client’s session by session outcomes fit the ETR of treatments that ended unsuccessfully.

An example of the type of feedback provided by the new algorithms is found below. The graph displays three zones of potential progress (or ETR’s) for a client scoring 15 on the ORS at intake. Scores falling in the “green” area from session to session are similar to treatments that ended successfully. As might be expected, those in the “red” zone, ended unsuccessfully. Finally, scores in the “yellow” zone had mixed results. In each instance, both the client and therapist are provided with instant feedback: green = on track, red = off track, yellow = concern.

The new algorithms will be a major focus of the upcoming “Advanced Intensive in Feedback-Informed Treatment (FIT)” scheduled for March 19th-22nd, 2012. All those subscribing to the event also receive the newly released series of FIT treatment manuals. Space is limited, as always, to 35 people and we are filling fast so please don’t wait. So many exciting developments!

Now, if you haven’t already done so, click on the video at the start of this post. I was floored by these satellite images. In some way, I hope that the new algorithms, FIT training manuals, and the ICCE community can inspire a similar sense of perspective!

Feedback-Informed Treatment as Evidence-based Practice: APA, SAMSHA, and NREPP

November 1, 2011 By scottdm 1 Comment

What is evidence-based practice? Visit the UK-based NICE website, or talk to proponents of particular theoretical schools or therapeutic models, and they will tell you that being “evidence-based” means using the approach research has deemed effective for a particular diagnosis (e.g., CBT for depression, EMDR for trauma). Over the last two decades, numerous organizations and interest groups have promoted lists of “approved” treatment approaches–guidelines that clinicians and funding bodies should follow when making practice decisions. Throughout the 1990’s, for example, division 12 within the American Psychological Association (APA) promoted the idea of “empirically supported treatments.”

However, when one considers the official definition of evidence-based practice offered by the Institute of Medicine and the APA, it is hard to fathom how anyone could come to such a conclusion. According to the APA, evidence-based practice is, “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.” (see American Psychologist, May 2006). Nothing here about “empirically supported treatments” or the mindless application of specific treatment protocols. Rather, according to the APA and IOM, clinicians must FIT the treatment to the client, their preferences, culture, and circumstances. And how can one do that? Well, conspicuously absent from the definition is, “consult a set of treatment guidelines.” Rather, when evidence-based, clinicians must monitor “patient progress (and of changes in the patient’s circumstances—e.g.,job loss, major illness) that may suggest the need to adjust the treatment. If progress is not proceeding adequately, the psychologist alters or addresses problematic aspects of the treatment (e.g., problems in the therapeutic relationship or in the implementation of the goals of the treatment) as appropriate.”

The principles and practices of feedback-informed treatment (FIT) are not only consistent with but operationalize the American Psychological Association’s (APA) definition of evidence-based practice. To wit, routinely and formally soliciting feedback from consumers regarding the therapeutic alliance and outcome of care and using the resulting information to inform and tailor service delivery. And indeed, over the last 9 months, together with Senior Associates, I completed and submitted an application for FIT to be reviewed by NREPP–SAMSHA’s National Registry of Evidence-based Practices and Approaches! As part of that application and ICCE’s commitment to improving the quality and outcome of behavioral health, we developed a list of “core competencies” for FIT practice, a series of six detailed treatment and implementation manuals, a gap assessment tool that organizations can use to quickly and expertly assess implementation and fidelity problems, and supportive documentation and paperwork. Finally, we developed and rigorously tested training curricula and administered the first standardized exam for certifying FIT practitioners and trainers. We are in the final stages of that review process soon and I’m sure I’ll be making a major announcement right here on this blog shortly. So, stay tuned.

In the meantime, this last Saturday, clinicians located the globe–Canada, New Zealand, Australia, the US,a nd Romania–sat for the first administration of ICCE “Core Competency” Exam. Taking the test is the last step in becoming an ICCE “Certified Trainer.” The other requirements include: (1) attending the “Advanced Intensive” and “Training of Trainers” workshops; and (2) submitting a training video on FIT for review. The exam was administered online using the latest technology.

The members, directors, and senior associates of ICCE want to congratulate (from top left):

Eeuwe Schuckard, Psychologist, Wellington, New Zealand;
Aaron Frost, Psychologist, Brisbane, Australia;
Cindy Hansen, BA-Psych, HHP, Manager Myoutcomes;
David Prescott, Director of Professional Development, Becket Family of Services, Portland, Maine;
Arnold Woodruff, LMFT, Clinical Director, Home for Good, Richmond, Virginia;
Bogdan, Ion, Ph.D., Bucharest University, Bucharest, Romania;
Daniel Buccino, Clinical Supervisor, Community Psychiatry Program. Johns Hopkins;
Dwayne Cameron, Outreach Counselor, Prince Albert, Saskatoon, Canada;
Mark Goheen, the Clinical Practice Lead at Fraser Health, British Columbia.

If you are not yet a member of the ICCE community, please join the largest, fastest growing, and friendly group of behavioral health professionals today at: www.centerforclinicalexcellence.com.

Deliberate Practice: What’s all the fuss about?

August 24, 2011 By scottdm Leave a Comment

Whatever they might be engaged in—dancing, singling, teaching, or doing therapy, top performers make “it” look so easy. Witnessing such a performance inspires awe and wonder, leading many unitiated to whisper about some being lucky, “born with talent.” How else can one explain the superior abilities of those we admire? “Sure, most will readily acknowledge, “the best work hard.” “BUT,” they quickly add, “THAT person has a gift.” Even the dictionary contributes to this continuing belief, defining gift as “innate capacity, talent, or endowment.” What is widely believed however is not always true: flat earth, phrenology, cold fusion, and…innate talent.

Turns out that the factor that separates the best from the rest is the amount of time spent practicing. And not just any kind of practice. Deliberate practice. A planned, conscious effort aimed at making small, continuous, and measurable improvement in one’s level of performance.

“It don’t come easy,” to borrow a line from an old rock and roll tune. If you are not exhausted at the end of the process you are probably not doing it right. As detailed previously on this blog, it requires time, patience, reflection, and—as we recently wrote in the Psychotherapy Networker, support. Interestingly, despite the challenges, the idea is catching on in the therapy world. I spoke with Alex Millham who lives and works in the U.K. not long ago about the subject. Click to read the interview.

Becoming FIT: The 2011 Training of Trainers

August 10, 2011 By scottdm 1 Comment

August 10th, 2011

Chicago, IL

The first week of August was one of the hottest weeks on record in Chicago. It was also the location of the hottest training on “feedback-informed treatment” (FIT)–the 5th Annual “Training of Trainers” weeklong intensive training. We worked intensively over 5 days preparing an international group of administrators, supervisors, researchers, and clinicians to train others in the principles and practices of FIT. We also played hard: dinners, music, magic, and more. Here’s what attendee’s said about this years event:

Be sure and join us for the 4-day “Advanced Intensive” scheduled in March. More information can be found at: scottdmiller.com.

Yes, More Evidence: Spanish version of the ORS Validated by Chilean Researchers

June 16, 2011 By scottdm Leave a Comment

Last week, Chile. This week, Perth, Australia. Yesterday, I landed in Sydney following a 30 hour flight from the United States. I managed to catch the last flight out to Perth before all air travel was grounded due to another ash clound–this time coming from Chile! I say “another” as just over a year ago, I was trapped behind the cloud of ash from the Icelandic eruption! So far so good. Today, I’ll spend the day talking about “excellence” in behavioral healthcare.

Before heading out to teach for the day, I wanted to upload a report from a recent research project conducted in Chile investigating the statistical properties of the ORS. I’ve attached the report here so you can read for yourself. That said, let me present the highlights:

The spanish version of the ORS is reliable (alpha coefficients .90-.95).
The spanish version of the ORS shows good construct and convergent validity (correlations with the OQ45 .5, .58).
The spanish version of the ORS is sensitive to change in a treated population.

The authors of the report that was presented at the Society for Psychotherapy Research meeting conclude, “The ORS is a valid instrument to be used with the Chilean population.”

As asked in my blogpost last week, “how much more evidence is needed?” Now, more than ever, clinicians needs simple, valid, reliable, and feasible tools for evaluating the process and outcome of behavioral healthcare. The ORS and SRS FITS the bill!

How Much More Evidence Is Needed? A New Meta-Analysis on Feedback-Informed Treatment

June 9, 2011 By scottdm 1 Comment

Received an email from friend and colleague John Norcross, Ph.D. Attached were the results of a meta-analysis completed by Michael Lambert and Kenichi Shimokawa on Feedback-Informed Treatment (FIT) which will appear in the second edition of his book, Psychotherapy Relationships that Work (Oxford University Press). For those who cannot wait, you can access the same results in the lastest issue of the APA journal Psychotherapy (Volume 48, Number 1, March 2011, pages 72-79).

Briefly, the chapter begins with a review of the literature on feedback–a body of evidence that, by the way, dates back to 1930’s and has always shown small to moderate effects on the outcome of treatment. In reviewing studies specific to the ORS and SRS, the authors conclude, “”>the results indicated that those in the feedback group ha[ve] 3.5 times higher odds of experiencing reliable change while having less than half the odds of experiencing deterioration.” Additionally, Lambert and Shimokawa report few if any meaningful differences between therapies informed by the ORS and SRS and those using the well-established and widely used Outcome Questionnaire (OQ). Finally, and importantly, the authors note that in “busy practices…the brevity of the [ORS and SRS]…expedite and ease practical difficulties” thereby decreasing barriers to implementation.

How much more evidence will it take before feedback informed treatment becomes standard practice? All of the available data is summarized in the materials below.

Measures and Feedback January 2011

View more documents from Scott Miller

Be sure and join other clinicians and researchers who are discussing FIT at the International Center for Clinical Excellence–the largest, free, web-based community dedicated to improving the quality and outcome of behavioral health.

Finally, if you are in thinking about or in the process of becoming FIT in your agency or practice, please join us at the upcoming “Training of Trainers” workshop held the first week of August. Registration is limited to 35 participants and we have only a few spots left! Here’s what attendees from last year had to say about the event…

The Mystery of Mastery: Excellence Takes Center Stage in the Psychotherapy Networker

May 16, 2011 By scottdm Leave a Comment

The Psychotherapy Networker has long been the most popular periodical among practicing clinicians. Rumor has it that the magazine has 80,000+ subscribers and sells over 120,000 copies of each issue. If you want to know what therapists are thinking and talking about, the Networker magazine is the place to look. And in the May/June issue, the topic of excellence is front and center.

The reason is obvious: the field of psychotherapy is in trouble. Think about it for a moment. What real accomplishments can psychotherapy boast of in recent years? What was the last truly revolutionary discovery in the field of psychotherapy? What “treatment” (analogous to penicillin in medicine) has ever successfully eradicated a mental or emotional disorder? In fact, while we’ve been at our posts, provisioning and parading an army of techniques and methods, rates of depression and anxiety have soared. Even if one disagrees with this grim assessment of the field’s contributions and influence, it’s hard to be sanguine about our status. Over the last decade, median incomes for psychologists, both applied and academic, have dropped by thousands of dollars. In the same period, workloads have increased, professional autonomy has been subverted, and funding for public behavioral healthcare has all but disappeared. Meanwhile, the very relevance of psychotherapy is an open question in the minds of many current and prospective consumers. Despite overwhelming evidence that therapy works, and that more than 90 percent of people say they’d prefer to talk about their problems than take psychopharmacological drugs, most people doubt the efficacy of treatment. Perhaps this accounts for the fact that the use of medications has steadily increased, while visits to a psychotherapist have been decreasing.

What can be done?

In 2007, we wrote an article that appeared in the pages of the Networker on the subject of “top performing” clinicians–those that consistently achieve superior results with their clients. Over the last four years, we’ve continued to research and write on the subject and in the latest issue of the Networker we review the latest findings. ICCE Associate, Dr. Bob Bertolino, also has an article in the issue detailed the steps required to reac excellence in agencies and healthcare systems.

Scott Miller Mark Hubble Bob Bertolino

Never has a moment in the history of the field existed when the need for a “culture of excellence” has been more pressing or when the qualities of that culture are more unambiguous. Seeing as we spend so much of our lives at work anyway—often more in total than with our families, friends, and in leisure—the question is, “why not?” If not for ourselves, then for our clients, the very people the research shows benefit the most from top performance and on whom our livelihoods depend. Don’t wait. Click on the links above to read both articles.

Changing Home-Based Mental Health Care for Good: Using Feedback Informed Treatment

February 8, 2011 By scottdm Leave a Comment

Some teach. Some write. Some publish research. Arnold Woodruff and Kathy Levenston work for a living! Kathy Levenston specializes in working with foster and adopted children.

Arnold Woodruff developed the first intensive in-home program run by a community services board in Virginia. He has over 30 years of experience, and has served as the President of the Virginia Association for Marriage and Family Therapy. And now, these two dedicated professionals, certified trainers and associates of the International Center for Clinical Excellence, have just purchased Home for Good, the first home-based mental health program in the Richmond, VA area to use Feedback-Informed Treatment (FIT).

The program is now a 100% employee-owned company and part of a larger vision the two have for establishing customer-friendly mental health care to people in the Richmond area. Home for Good has been providing Intensive In-home Services (counseling, case management, and crisis support) to children, adolescents, and their families for the past two years. Home for Good has achieved superior results compared to other mental health programs, based on an analysis of data genderated from routine administration of the Outcome Rating Scale in clinical practice. Home for Good’s results are continuing to improve with the use of Feedback-Informed Treatment. Home for Good will soon be offering additional services, including outpatient individual, family, and group therapy.

The Growing Evidence Base for Feedback-Informed Treatment (FIT)

January 25, 2011 By scottdm Leave a Comment

Dateline: February 2, 2011
Location: Anchorage, AK
Greetings from Anchorage, Alaska where I’ve been traveling and teaching about feedback-informed treatment (FIT). On Monday, I worked with dedicated behavioral health professionals living and working in Barrow–the northern most point in the United States. FIT has literally reached the “top of the world.” How incredible is that?

Here I am pictured in front of a sign which locals told me would prove I’d made the long journey to the village of 5,000. I look forward to returning soon to help the group with the “nuts and bolts” of implementing FIT across various behavioral health services–practitoners were keen to get started.

As I’ve crisscrossed the state, I’ve been proud to share the growing evidence-base for feedback informed work. Below, the data is summarized in a free, downloadable PDF file, “Measures and Feedback,” which has been updated to include the latest research using the ORS and SRS to improve the quality and outcome of treatment. If you accessed this file back in 2010, be sure to get this updated version.

Measures and feedback 2016 from Scott Miller

Getting FIT in the New Year: The Latest Evidence

January 18, 2011 By scottdm Leave a Comment

John Norcross, Ph.D. is without a doubt the researcher that has done the most to highlight the evidence-base supporting the importance of the relationship between clinician and consumer in successful behavioral healthcare. The second edition of his book, Psychotherapy Relationships that Work, is about to be released. Like the last edition, this volume is a virtual treasure trove of research findings and empirically supported practices.

Among the many gems in the book is a chapter by Michael J. Lambert, Ph.D–pioneering researcher on “feedback-informed treatment” (FIT). As usual, he does a masterful job summarizing the existing research on the subject. The data are overwhelmingly positive: seeking and using standardized feedback regarding the progress and outcome of treatment cuts drop out and deterioration rates and significantly improves outcome.

Lambert also reports the results of two meta-analyses. One performed on studies using his own OQ System family of measures, the other based on research using the ORS and SRS. Not only did he find ample empirical support for the two systems, but in the case of the ORS and SRS those therapies informed by feedback, “had 3.5 times higher odds of experiencing reliable change.” Additionally, and importantly, the brief, 4-item ORS and SRS scales performed the same as the longer and more detailed OQ 45.2.

What can you do? First, order John’s book. Second, if you are not FIT, now is the time to register to use the measures. And if you need support, why not join the International Center for Clinical Excellence? Like the measures, there is no cost. Right now, professionals from different disciplines, working in diverse settings are connecting with and learning from each other. Here’s a nudge: you’ll be able to reach John Norcross there—he’s one of ICCE’s newest members.

Pushing the Research Envelope: Getting Researchers to Conduct Clinically Meaningful Research

November 5, 2010 By scottdm Leave a Comment

At the recent ACE conference, I had the pleasure of learning from the world’s leading experts on expertise and top performance. Equally stimulating were conversations in the hallways between presentations with clinicians, policy makers, and researchers attending the event. One of those was Bill Andrews, the director of the HGI Practice Research Network in the UK who work over the last 3+ years has focused on clinicians whose outcomes consistently fall in the top quartile of effectiveness.

In this brief interview, Bill talks about the “new direction” his research on top performing clinicians is taking. He is truly “pushing the research envelope, challenging the field to move beyond the simplistic randomized clinical trials comparing different treatment packages. Take a look:

Am-ACE-ing Events in Kansas City: The First International Achieving Clinical Excellence Conference

October 27, 2010 By scottdm Leave a Comment

Here’s a riddle for you:

What do therapists, researchers, case managers, magicians, surgeons, award winning musicians, counselors, jugglers, behavioral health agency directors, and balloon twisting artists have in common?

Answer:

They all participated in the first “Achieving Clinical Excellence” held last week in Kansas City, Missouri.

It’s true. The “motley” crew of presenters, entertainers, and attendees came to Kansas City learn the latest, evidence-based strategies for helping clinicians achieve their “personal best” and, in the process, improve the quality and outcome of behavioral health services. Not only did participants and presenters come from all over the globe–Australia, New Zealand, Norway, Sweden, Denmark, Austria, the UK, Ireland, Scotland, Germany, Canada, Holland, and elsewhere–but ICCE web 2.0 technology was used to stream many of the presentations live to a worldwide audience (click on the link to watch the recordings).

“The atmosphere was positively electric,” one participant remarked to me on break, “and so friendly. First, I was inspired. Each presentation contained something new, a take-away. Then I wanted to sit with other attendees and discuss the content.”

And thanks to “Gillis for Children and Families,” who not only sponsored and ran the event, but provided a full breakfast and lunch each day of the conference, participants had ample opportunity to meet, process, and network with each other.

Rich Simon Anders Ericsson Michael Ammar

Rich Simon, Ph.D., the editor of the Psychotherapy Networker, kicked off the event using his time at the podium to place the conference’s emphasis on excellence within the broader history of the field of psychotherapy. He was followed by K. Anders Ericsson, the editor of the influential Cambridge Handbook of Expertise and Expert Performance, reviewed research on expert performance gathered over the last 3 decades. Scott D. Miller, Ph.D., translated existing research on expert performance into steps for improving outcomes in behavioral health. On day 2, professional magician Michael Ammar delivered a stunning performance of close up magic while teaching a specific method of deliberate practice that clinicians can use to improve their skills. Meanwhile, break out sessions led by psychologists, physicians, counselors, pharmacists, and agency directors addressed “nuts and bolts” applications.

Rachel Hsu Roger Shen

In between each plenary and breakout session, top performers from a variety of fields entertained and inspired. Moving performances on the violin and piano by nine year old Rachel Hsu and eleven year old Roger Shen amazed and challenged everyone in attendance. “It is not talent,” Rachel told me, “It’s a lot of hard work–4 to 5 hours a day, everyday of the week, including weekends.” The take home lesson from these exception kids was clear: there are no short cuts when it comes to top performance. If you want to achieve your personal best you must work hard. Promises otherwise are so much more snake oil.

On Thursday evening, the Australian classical pianist, David Helfott, whose lifestory was the subject of the award winning film, “Shine” entertained conference attendees. His partner, Gillian, introduced and provided the audience with a brief history of David’s life, unfortunate treatment in the mental health system, and their long marriage. The audience rose to their feet in a standing ovation at the conclusion of the performance. There were few dry eyes in the house. Afterwards, the two spent nearly an hour meeting and greeting attendees personally. Once again, portions of the performance were broadcast live via ICCE web 2.0 technology to a world wide audience.

The inspiration that conference attendees felt continues on the International Center for Clinical Excellence web-based community. Join us as we work to help each other achieve our personal best. Still looking for inspiration? Take a look at the following two videos; first, a montage of events at ACE; and second, Mr. Ah’ Lee Robinson, the director of the Kansas City Boys Choir, whose story and performance brought the conference to a moving conclusion.

What is "Best Practice?"

October 20, 2010 By scottdm Leave a Comment

You have to admit the phrase “best practice” is the buzzword of late. Graduate school training programs, professional continuing education events, policy and practice guidelines, and funding decisions are tied in some form or another to the concept. So, what exactly is it? At the State and Federal level, lists of so-called “evidence-based” interventions have been assembled and are being disseminated. In lockstep, as I reviewed recently, are groups like NICE. Their message is simple and straightforward: best practice is about applying specific treatments to specific disorders.

Admittedly, the message has a certain “common sense” appeal. The problem, of course, is that behavioral health interventions are not the psychological equivalent of penicillin. In addition to the numerous studies highlighted on this blog documenting the failure of the “specific treatments for specific disorders” perspective, consider research published in the Spring 2010 edition of the Journal of Counseling and Development by Scott Nyman, Mark Nafziger, and Timothy Smith. Briefly, the authors examined outcome data to “evaluate treatment effectiveness across counselor training level [and found] no significant outcome differences between professional staff and …. interns, and practicum students” (p. 204). Although the researchers are careful to make all the customary prevarications, the conclusion—especially when combined with years of similar findings reported in the literature– is difficult to escape: counseling and psychotherapy are highly regulated activities requiring years of expensive professional training that ultimately fails to make the practitioner any better than they were at the outset.

What gives? Truth is, the popular conceptualization of “best practice” as a “specific treatment for a specific disorder” is hopelessly outdated. In a report few have read, the American Psychological Association (following the lead of the Institute of Medicine) redefined evidence-based, or best practice, as, “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.” Regarding the phrase “clinical expertise” in this definition, the Task Force stated, “Clinical expertise…entails the monitoring of patient progress (and of changes in the patient’s circumstances—e.g., job loss, major illness) that may suggest the need to adjust the treatment (Lambert, Bergin, & Garfield, 2004a). If progress is not proceeding adequately, the psychologist alters or addresses problematic aspects of the treatment (e.g., problems in the therapeutic relationship or in the implementation of the goals of the treatment) as appropriate” (p. 273; emphasis included in the original text).

Said another way, instead of choosing the “specific treatment for the specific disorder” from a list of approved treatments, best practice is:

· Integrating the best evidence into ongoing clinical practice;

· Tailoring services to the consumer’s characteristics, culture, and preferences;

· Formal, ongoing, real-time monitoring of progress and the therapeutic relationship.

In sum, best practice is Feedback Informed Treatment (FIT)—the vision of the International Center for Clinical Excellence. And right now, clinicians, researchers and policy makers are learning, sharing, and discussion implementing FIT in treatment settings around the globe on the ICCE web-based community.

Word is getting out. As just one example, consider Accreditation Canada, which recently identified FIT as a “leading practice” for use in behavioral health services. According to the website, leading practices are defined as “creative, evidence-based innovations [that] are commendable examples of high quality leadership and service delivery.” The accreditation body identified FIT as a “simple, measurable, effective, and feasible outcome-based accountability process,” stating that the approach is a model for the rest of the country! You can read the entire report here.

How exactly did this happen? Put bluntly, people and hard work. ICCE senior associates and certified trainers, Rob Axsen and Cynthia Maeschalck, with the support and backing of Vancouver Coast Health, worked tirelessly over the last 5 years both implementing and working to gain recognition for FIT. Similar recognition is taking place in the United States, Denmark, Sweden, England, and Norway.

You can help. Next time someone—be it colleague, trainer, or researcher—equates “best practice” with using a particular model or list of “approved treatment approaches” share the real, official, “approved” definition noted above. Second, join Rob, Cynthia, and the hundreds of other practitioners, researchers, and policy makers on the ICCE helping to reshape the behavioral health practice worldwide.