SCOTT D Miller - For the latest and greatest information on Feedback Informed Treatment

  • About
    • About Scott
    • Publications
  • Training and Consultation
  • Workshop Calendar
  • FIT Measures Licensing
  • FIT Software Tools
  • Online Store
  • Top Performance Blog
  • Contact Scott
scottdmiller@ talkingcure.com +1.773.454.8511

Are you any good as a therapist? The Legacy of Paul W. Clement

March 26, 2014 By scottdm 4 Comments

Paul Clement

Twenty years ago, I came across an article published in the journal, Professional Psychology.  It was written by a psychologist in private practice, Paul Clement.  The piece caught my eye for a number of reasons.  First, although we’d never met, Paul lived and worked in a town near my childhood home: Pasadena, California.  Second, the question he opened his article with was provocative, to say the least, “Are you any good?”  In other words, how effective are YOU as a psychotherapist?  Third, and most important, he had compiled and was reporting a quantitative analysis of his results over the last 26 years as a practicing clinician.  It was both riveting and stunning.  No one I knew had ever done had published something similar before.

In graduate school, I’d learned to administer a variety of tests (achievement, vocational, personality, projective, IQ, etc.).  Not once, however, did I attend a course or sit in a lecture about how to measure my results.   I was forced to wonder, “How could that be?”  Six years in graduate school and not a word about evaluating one’s outcomes.  After all, if we don’t know how effective we are, how are any of us supposed to improve?

What was the reason for the absence of measurement, evaluation, and analysis?   It certainly wasn’t because psychotherapy wasn’t effective.  A massive amount of research existed documenting the effectiveness of treatment.  Paul’s research confirmed these results.  Of those he’d worked with, 75% were improved at termination.  Moreover, such results were obtained in a relatively brief period of time, the median number of sessions used being 12.

Other results he reported were not so easy to accept.  In short, Paul’s analysis showed that his outcomes had not improved over the course of his career.   At the conclusion of the piece, he observed, “I had expected to find that I had gotten better and better over the years, but my data failed to suggest any systematic change in my therapeutic effectiveness across the 26 years in question…it was a bad surprise for me.” (p. 175).

For years, I carried the article with me in my briefcase, hoping that one day, I might better understand his findings.   Maybe, I thought, Clement was simply an outlier?  Surely, we get better with experience.  It was hard for me to believe I hadn’t improved since my first, ham-handed sessions with clients.  Then again, I didn’t really know.  I wasn’t measuring my results in any meaningful way.

The rest is history.  Within a few short years, I was routinely monitoring the outcome and alliance at every session I did with clients.  Thanks to my undergraduate professor, Michael Lambert, Ph.D., I began using the OQ 45 to assess outcomes.  Another mentor, Dr. Lynn Johnson had developed a 10-item scale for evaluating the quality of the therapeutic relationship, know as the Session Rating Scale.  Both tools became an integral part of the way I worked.  Eventually, a suggestion by Haim Omer, Ph.D., led me to consider creating shorter, less time consuming visual analogue versions of both measures.  In time, together with colleagues, the ORS and SRS were developed and tested.  Throughout this process, Paul Clement, and his original study remained an important, motivating force.

Just over a year ago, Paul sent me an article evaluating 40 years of his work as a psychotherapist.   Once again, I was inspired by his bold, brave, and utterly transparent example.  Not only had his outcomes not improved, he reported, they’d actually deteriorated!  Leave it to him to point the way!   As readers of this blog know, our group is busy at work researching what it takes to forestall such deterioration and improve effectiveness.  Last year, we summarized our findings in the 50th Anniversary issue of Psychotherapy.  As I write, we are preparing a more detailed report for publication in the same journal.

Yesterday, I was drafting an email, responding to one I’d recently received from him, when I learned Paul had died.  I will miss him.  In this, I know I’m not alone.

Filed Under: Top Performance Tagged With: clinician, Haim Omer, Lynn Johnson, Michael Lambert, OQ45, ors, outcome rating scale, Paul Clement, popular psychology, practice-based evidence, psychotherapy, session rating scale, srs, top performance

Dealing with Scientific Objections to the Outcome and Session Rating Scales: Real and Bogus

December 15, 2012 By scottdm Leave a Comment

The available evidence is clear: seeking formal feedback from consumers of behavioral health services decreases drop out and deterioration while simultanesouly improving effectiveness.  When teaching practitioners how to use the ORS and SRS to elicit feedback regarding progress and the therapeutic relationship,  three common and important concerns are raised:

  1. How can such simple and brief scales provide meaningful information?
  2. Are consumers going to be honest?
  3. Aren’t these measures merely assessing satisfaction rather than anything meaninful?

Recently, I was discussing these concerns with ICCE Associate and Certified Trainer, Dan Buccino.

Briefly, Dan is a clinical supervisor and student coordinator in the Adult Outpatience Community Psychiatry program at Johns Hopkins.  He’d not only encountered the concerns noted above but several additional objections.  As he said in his email, “they were at once baffling and yet exciting, because they were so unusal and rigorous.”

“It’s a sign of the times,” I replied, “As FIT (feedback informed treatment) becomes more widespread, the supporting evidence will be scrutinized more carefully.  It’s a good sign.”

Together with Psychologist and ICCE Senior Associate and Trainer, Jason Seidel, Dan crafted detailed response.  When I told them that I believed the ICCE community would value having access to the document they created, both agreed to let me publish it on the Top Performance blog.  So…here it is.  Please read and feel free to pass it along to others.

 

 

 

Filed Under: Feedback Informed Treatment - FIT Tagged With: accountability, behavioral health, Certified Trainers, evidence based practice, feedback, interviews, mental health, ors, practice-based evidence, psychometrics, research, srs

Clinical Support Tools for the ORS and SRS

November 20, 2012 By scottdm 1 Comment

I have so much to be grateful for at this time.  Most of all, I’m happy to be home with my family.  As we have in the past, this year we’ll be spending the holiday at the home of our long time friends John and Renee Dalton.  The two always put out a fantastic spread and our son, Michael, is fast friends with their two kids.

I’m also grateful for the International Center for Clinical Excellence (ICCE) community.  Currently, ICCE has over 4200 members located around the world, making the organization the largest, web-based community of professionals, educators, managers, and clinicians dedicated to using feedback to pursue excellence in the delivery of behavioral health services.  Recently, the site was highlighted as one of the best resources for practitioners available on the web.  Articles, how-to videos, and discussion forums are available everyday, all day–and for free!  No come-ons for books or webinars and no “cult of personality”–just sharing among peers.  If you are not a member, you can join at: www.centerforclinicalexcellence.com

A special thanks goes to several ICCE senior advisors and associates, including Susanne Bargmann, Jason Seidel, Cynthia Maeschalck, Bob Bertolino, Bill Plum, Julie Tilsen, and Robbie Babbins-Wagner.  These folks are the backbone of the organization.  Together, they make it work.  Most recently, we all joined together to create the ICCE Feedback Informed Treatment and Training Manuals, a cutting edge series covering every aspect of FIT–from the empirical foundations to implementation–in support of our application to SAMSHA for recognition as an “evidence-based practice.”

As a way of supporting everyone using the ORS and SRS, I wanted to make a couple of clinical support tools available.  If you are using the measures, the first item will need no introduction.  It’s a 10 cm ruler!  Save the file and print it off and you also have a ready reminder of the upcoming Achieving Clinical Excellence conference, coming up in May 2013.  Like last time, this will feature the latest inforamtion about feedback informed practice!  The second item is a reliable change graph.  If you are using the paper and pencil measures, rather than one of the existing web based systems (www.fit-outcomes.com, www.myoutcomes.com), you can use this tool to determine whether a change in scores from session to session is reliable (that is, greater than chance, the passage of time, and measurement error [and therefore, due to the care being provided]) or even clinically significant (that is, both reliable and indicating recovered).  The last item is an impressive summary of various systems for monitoring progress in treatment.

In addition ACE Health have developed openFIT, a plug-in which seamlessly integrates the ORS, SRS and associated algorithms into any existing Electronic Health Record, Case Management System of eMental Health application.

I wish everyone a peaceful and rewarding Thanksgiving holiday.

 

Filed Under: FIT Software Tools Tagged With: behavioral health, cdoi, excellence, feedback, healthcare, icce, mental health, ors, Outcome, practice-based evidence, srs

Feedback Informed Treatment as Evidence-Based Practice

May 23, 2012 By scottdm Leave a Comment

Back in November, I blogged about the ICCE application to SAMSHA’s National Registry for consideration of FIT as an official evidence-based approach (EBP).  Given the definition of EBP by the Institute of Medicine and the American Psychological Association, Feedback Informed Treatment seems a perfect, well, FIT.  According to the IOM and APA, evidence-based practice means using the best evidence and tailoring services to the client, their preferences, culture, and circumstances.  Additionally, when evidence-based, clinicians must monitor “patient progress (and of changes in the patient’s circumstances—e.g.,job loss, major illness) that may suggest the need to adjust the treatment. If progress is not proceeding adequately, the psychologist alters or addresses problematic aspects of the treatment (e.g., problems in the therapeutic relationship or in the implementation of the goals of the treatment) as appropriate.”

In late Summer 2011, ICCE submitted 1000’s of pages of supporting documents, research studies, as well as video in support of the application.  This week, we heard that FIT passed the “Quality of Research” phase of the review.  Now, the committee is looking at the “Readiness for Dissemination” materials, including the six detailed treatment and implementation manuals on feedback informed treatment.  Keep your fingers crossed.  We’ve been told that the entire process should be completed sometime in late fall.

In the meantime, we are preparing for this summer’s Advanced Intensive and Training of Trainer workshops.  Once again, clinicians, educators, and researchers from around the world will be coming together for cutting edge training.  Only a few spots remain, so register now.

Filed Under: Feedback Informed Treatment - FIT Tagged With: American Psychological Association, evidence based medicine, evidence based practice, feedback informed treatment, FIT, icce, Institute of Medicine, NREPP, practice-based evidence, SAMHSA, Training

Improving Outcomes in the Treatment of Obesity via Practice-Based Evidence: Weight Loss, Nutrition, and Work Productivity

April 9, 2010 By scottdm 4 Comments

Obesity is a large and growing problem in the United States and elsewhere.  Data gathered by the National Center for Health Statistics indicate that 33% Americans are obese.  When overweight people are added to the mix, the figure climbs to a staggering 66%!   The problem is not likely to go away soon or on its own as the same figures apply to children.

Researchers estimate that weight problems are responsible for over 300,000 deaths annually and account for 12% of healthcare costs or 100 billion–that’s right, $100,000,000,000–in the United States alone.   The overweight and obese have higher incidences of arthritis, breast cancer, heart disease, colorectal cancer, diabetes, endometrial cancer, gallbladder disease, hypertension, liver disease, back pain, sleeping problems, and stroke–not to mention the tremendous emotional, relational, and social costs.  The data are clear: the overweight are the target of discrimination in education, healthcare, and employment.  A study by Brownell and Puhl (2003), for example, found that: (1) a significant percentage of healthcare professionals admit to feeling  “repulsed” by obese person, even among those who specialize in bariatric treatment; (2) parents provide less college support to their overweight compared to “thin” children; and (3) 87% of obese individuals reported that weight prevented them from being hired for a job.

Sadly, available evidence indicates that while weight problems are “among the easiest conditions to recognize,” they remain one of the “most difficult to treat.”  Weight loss programs abound.  When was the last time you watched television and didn’t see an ad for a diet pill, program, or exercise machine?  Many work.  Few, however, lead to lasting change.

What might help?

More than a decade ago, I met Dr. Paul Faulkner, the founder and then Chief Executive Officer of Resources for Living (RFL), an innovative employee assistance program located in Austin, Texas.  I was teaching a week-long course on outcome-informed work at the Cape Cod Institute in Eastham, Massachusetts.  Paul had long searched for a way of improving outcomes and service delivery that could simultaneously be used to provide evidence of the value of treatment to purchasers–in the case of RFL, the large, multinational companies that were paying him to manage their employee assistance programs.  Thus began a long relationship between me and the management and clinical staff of RFL.  I was in Austin, Texas dozens of times providing training and consultation as well as setting up the original ORS/SRS feedback system known as ALERT, which is still in use at the organization today.  All of the original reliability, validity, norming, and response trajectories were done together with the crew at RFL.

Along the way, RFL expanded services to disease management, including depression, chronic obstructive pulmonary disease, diabetes, and obesity.  The “weight management” program delivered coaching and nutritional consultation via the telephone informed by ongoing measurement of outcomes and the therapeutic alliance using the SRS and ORS.  The results are impressive.  The study by Ryan Sorrell, a clinician and researcher at RFL, not only found that the program and feedback led to weight loss, but also significant improvements in distress, health eating behaviors (70%), exercise (65%), and presenteeism on the job (64%)–the latter being critical to the employers paying for the service.

Such research adds to the growing body of literature documenting the importance of “practice-based” evidence, making clear that finding the “right” or “evidence-based” approach for obesity (or any problem for that matter) is less important than finding out “what works” for each person in need of help.  With challenging, “life-style” problems, this means using ongoing feedback to inform whatever services may be deemed appropriate or necessary.  Doing so not only leads to better outcomes, but also provides real-time, real-world evidence of return on investment for those footing the bill.

Filed Under: Behavioral Health, Feedback Informed Treatment - FIT, Practice Based Evidence Tagged With: behavioral health, cdoi, cognitive-behavioral therapy, conferences, continuing education, diabetes, disease management, Dr. Paul Faulkner, evidence based medicine, evidence based practice, Hypertension, medicine, obesity, ors, outcome rating scale, practice-based evidence, public behavioral health, randomized clinical trial, session rating scale, srs, Training

Research on the Outcome Rating Scale, Session Rating Scale & Feedback

January 7, 2010 By scottdm Leave a Comment

PCOMS - Partners for change outcome management system Scott D Miller - SAMHSA - NREPP“How valid and reliable are the ORS and SRS?”  “What do the data say about the impact of routine measurement and feedback on outcome and retention in behavioral health?”  “Are the ORS and SRS ‘evidence-based?'”

These and other questions regarding the evidence supporting the ORS, SRS, and feedback are becoming increasingly common in the workshops I’m teaching in the U.S. and abroad.

As indicated in my December 24th blogpost, routine outcome monitoring (PROMS) has even been endorsed by “specific treatments for specific disorders” proponent David Barlow, Ph.D., who stated unequivocally that “all therapists would soon be required to measure and monitor the outcome of their clinical work.”  Clearly, the time has come for all behavioral health practitioners to be aware of the research regarding measurement and feedback.

Over the holidays, I updated a summary of the data to date that has long been available to trainers and associates of the International Center for Clinical Excellence.  The PDF reviews all of the research on the psychometric properties of the outcome and session ratings scales as well as the studies using these and other formal measures of progress and the therapeutic relationship to improve outcome and retention in behavioral health services.  The topics is so important, that I’ve decide to make the document available to everyone.  Feel free to distribute the file to any and all colleagues interested in staying up to date on this emerging mega-trend in clinical practice.

Measures And Feedback from Scott Miller

Filed Under: evidence-based practice, Feedback Informed Treatment - FIT, Practice Based Evidence Tagged With: behavioral health, continuing education, david barlow, evidence based medicine, evidence based practice, feedback, Hypertension, icce, medicine, ors, outcome measurement, outcome rating scale, post traumatic stress, practice-based evidence, proms, randomized clinical trial, session rating scale, srs, Training

Climate Change in Denmark

December 5, 2009 By scottdm Leave a Comment

hans_christian_andersen_gbHans Christian Andersen, the author of such classic stories as The Ugly Duckling and the Emperor’s New Clothes, once wrote, “Life itself is the most wonderful fairy tale of all.”  That sentiment is certainly true of my own life.  For the last 16 years, I’ve been privileged to travel around the world conducting training and providing consultation.  Each year, I meet literally thousands of therapists and I’m consistently impressed and inspired by their dedication and persistence.  Truth be told, that “spirit”–for lack of a better word–is actually what keeps me in the field.

This last year, I’ve spent a considerable amount of time working with practitioners in Denmark.  Interest in Feedback-Informed Treatment has taken off–and I have the frequent flyer miles to prove it! While I’ve been traveling to the homeland of Hans Christian Andersen for many years (actually my maternal grandfather and his family immigrated to the United States from a small town just outside Copenhagen), momentum really began building following several years of workshops arranged by Henrik and Mette Petersen who run Solution–a top notch organization providing both workshops and year-long certification courses in short-term, solution-focused, and systemic therapies.

In October, I worked with 100+ staff who work at Psykoterapeutisk Center Stolpegård–a large outpatient center just outside of Copenhagen.  For two days, we talked about research and practice in psychotheapy, focusing specifically on using outcome to inform and improve clinical services.  Peter Koefoed, chief psychologist and head of Training organized the event.   I was back in Denmark not quite one month later for two days with Henrik and Mette Petersen and a then third day for a small, intensive training with Toftemosegaard–a center for growth and change–smack dab in the middle of Copenhagen.

At each event, I was honored to be accompanied by Danish psychologist Susanne Bargmann, who is an Associate and Certified Trainer for the Center for Clinical Excellence (ICCE).  I first met Susanne at a two-day workshop sponsored by Solutions a number of years ago.  Her attitude and drive is infectious.  She attended the Training of Trainer’s event in Chicago and now runs a listserve for Danish practitioners interested in feedback-informed treatment (FIT) (by the way, if you are interested in joining the group simply click on her name above to send an email).

Recently, she published an important article in Psycholog Nyt–the official magazine for the Danish Psychological Association. The article is really the first written in Danish by a Danish practitioner to suggest “practice-based evidence” as a scientifically credible alternative to the narrow “specific treatments for specific problems” paradigm that has come to dominate professional discourse and practice the world over.

Anyway, I’ll be back in Denmark several times in 2010.  In May, I’ll be teaching “Supershrinks: Learning from the Field’s Most Effective Practitioners.”  The course, as I understand it, is already sold out.  No worries though as the workshop is being offered again in November–so sign up early (click here to access my workshop calendar).  Also, in September, Susanne and I will jointly teach a course for psychologists on research entitled, “Forskning og Formidling”–a required training for those seeking specialist approval by the Danish Psychological Association. Finally, as I’ve done for the last several years, I’m scheduled to do two days for Solution as well.  If you live and work in Denmark, I truly hope to see you at one of these events.

Bargman Nye Veje For Evidensbegrebet from Scott Miller

 

Filed Under: Behavioral Health, excellence, Feedback Informed Treatment - FIT Tagged With: cdoi, Danish Psychological Association, denmark, icce, international center for cliniclal excellence, ors, outcome rating scale, practice-based evidence, session rating scale, srs, supershrinks

The Debate of the Century

August 27, 2009 By scottdm

doubt_diceWhat causes change in psychotherapy?  Specific treatments applied to specific disorders?  Those in the “evidence-based” say so and have had a huge influence on behavioral healthcare policy and reimbursement.  Over the last 10 years, my colleagues and I have written extensively and traveled the world offering a different perspective: by and large, the effectiveness of care is due to a shared group of factors common to all treatment approaches.

In place of “evidence-based” practice, we’ve argued for “practice-based”evidence.  Said another way, what really matters in the debate is whether clients benefit–not the particular treatment approach.  Here on my website, clinicians can download absolutely free measures that can be used to monitor and improve outcome and retention (click Performance Metrics).

bruce-wampold-364px

Anyway, the message is finally getting through.  Recently, uber-statistician and all around good guy Bruce Wampold, Ph.D. debated prominent EBP proponent Steve Hollon.  Following the exchange, a vote was taken.  Bruce won handily: more than 15:1.

Scroll down to “Closing Debate” (Thursday)

Filed Under: Behavioral Health, Practice Based Evidence Tagged With: bruce wampold, cdoi, evidence based medicine, evidence based practice, ors, outcome rating scale, PCOMS, performance metrics, practice-based evidence, psychotherapy, session rating scale, srs, steve hollon

SEARCH

Subscribe for updates from my blog.

loader

Email Address*

Name

Upcoming Training

Oct
01

Training of Trainers 2025


Nov
20

FIT Implementation Intensive 2025

FIT Software tools

FIT Software tools

LinkedIn

Topics of Interest:

  • Behavioral Health (112)
  • behavioral health (5)
  • Brain-based Research (2)
  • CDOI (14)
  • Conferences and Training (67)
  • deliberate practice (31)
  • Dodo Verdict (9)
  • Drug and Alcohol (3)
  • evidence-based practice (67)
  • excellence (63)
  • Feedback (40)
  • Feedback Informed Treatment – FIT (246)
  • FIT (29)
  • FIT Software Tools (12)
  • ICCE (26)
  • Implementation (7)
  • medication adherence (3)
  • obesity (1)
  • PCOMS (11)
  • Practice Based Evidence (39)
  • PTSD (4)
  • Suicide (1)
  • supervision (1)
  • Termination (1)
  • Therapeutic Relationship (9)
  • Top Performance (40)

Recent Posts

  • Agape
  • Snippets
  • Results from the first bona fide study of deliberate practice
  • Fasten your seatbelt
  • A not so helpful, helping hand

Recent Comments

  • Dr Martin Russell on Agape
  • hima on Simple, not Easy: Using the ORS and SRS Effectively
  • hima on The Cryptonite of Behavioral Health: Making Mistakes
  • himalaya on Alas, it seems everyone comes from Lake Wobegon
  • himalayan on Do certain people respond better to specific forms of psychotherapy?

Tags

addiction Alliance behavioral health brief therapy Carl Rogers CBT cdoi common factors conferences continuing education denmark evidence based medicine evidence based practice Evolution of Psychotherapy excellence feedback feedback informed treatment healthcare holland icce international center for cliniclal excellence medicine mental health meta-analysis Norway NREPP ors outcome measurement outcome rating scale post traumatic stress practice-based evidence psychology psychometrics psychotherapy psychotherapy networker public behavioral health randomized clinical trial SAMHSA session rating scale srs supershrinks sweden Therapist Effects therapy Training