The Outcome and Session Rating Scales: Support Tools

March 30, 2012 By scottdm 6 Comments

Japan, Sweden, Norway, Denmark, Germany, France, Israel, Poland, Chile, Guam, Finland, Hungary, Mexico, Australia, China, the United States…and many, many more. What do all these countries have in common? In each, clinicians and agencies are using the ORS and SRS scales to inform and improve behavioral health services. Some are using web-based systems for administration, scoring, interpretation and data aggregation (e.g., myoutcomes.com and fit-outcomes), many are accessing paper and pencil versions of the measures for free and then administering and scoring by hand.

Even if one is not using a web-based system to compare individual client progress to cutting edge norms, practitioners can still determine simply and easily whether reliable change is being made by using the “Reliable Change Chart” below. Recall, a change on the ORS is considered reliable when the difference in scores exceeds the contribution attributable to chance, maturation, and measurement error. Feel free to print out the graph and use it in your practice.

To learn how to get the most out of the measures, be sure and download the six FIT Treatment and Training Manuals. The six manuals cover every aspect of feedback-informed practice including: empirical foundations, basic and advanced applications (including FIT in groups, couples, and with special populations), supervision, data analysis, and agency implementation. Each manual is written in clear, step-by-step, non-technical language, and is specifically designed to help practitioners and agencies integrate FIT into routine clinical practice. Indeed, the manuals were submitted as part of ICCE’s application for consideration of FIT as an “evidence-based practice” to the National Registry of Evidence-Based Programs and Practices

ORS Reliable Change Chart

The Effects of Feedback on Medication Compliance and Outcome: Follow Up on The University of Pittsburgh Study

September 9, 2010 By scottdm Leave a Comment

Late last year, I blogged about a study being conducted at the University of Pittsburgh by Dr. Jan Pringle, the director of the Program Evaluation Research Unit in the School of Pharmacology and her colleague, Dr. Michael Melczak. You’ll recall, there were two conditions in the study. In the first, pharmacists–the practitioner most likely to interact with patients about prescriptions–engaged in “practice as usual.” In the second condition, pharmacists used the ORS and the SRS to chart, discuss, and guide patient progress and the pharmacist-patient alliance. At the time, I reported that initial findings showed that patients of pharmacists who used the measures to solicit feedback “were significantly more likely to take their medications at the levels that would be likely to result in clinical impact than the patients who saw a pharmacists who did not use the scales…for hypertensive and hyperlipidemia drugs especially.” Well, the official results are finally available.

After controlling for age, gender, and other individual and control conditions (including measures of interactions with pharmacies), patients in the feedback condition increased their rate of “percent of days covered”–that is, taking the medication as prescribed–significantly (average 11%, a result considered “impressive” when compared to other, traditional efforts aimed at improving compliance). Interestingly, additional analyses showed that the impact of the SRS–a measure of the therapeutic alliance–was greatest for the hyplipidemia and hypertensive medications (as opposed to the anti-diabetic medications).

Drs. Pringle and Melczak are currently in the process of planning a series of additional studies involving a larger number of patients and pharmacists. Both will be presenting at the upcoming Achieving Clinical Excellence conference.

Finally, take a look at the video that was developed to begin training pharmacists to use the measures with customers filling prescriptions at local pharmacies. According to Dr. Pringle, “we expect to training about 240 pharmacists across 118 pharmacies in the western and central portions of Pennsylvania how to use the ORS and SRS…the program represents a collaboration between the University of Pittsburgh, CECity (a technology company), RiteAid, and Highmark ( a Blues insurer).” Exciting stuff, eh?

Where Necessity is the Mother of Invention: Forming Alliances with Consumers on the Margins

April 11, 2010 By scottdm 3 Comments

Spring of last year, I traveled to Gothenburg, Sweden to provide training GCK–an top notch organization led by Ulla Hansson and Ulla Westling-Missios providing cutting-edge training on “what works” in psychotherapy. I’ll be back this week again doing an open workshop and an advanced training for the group.

While I’m always excited to be out and about traveling and training, being in Sweden is special for me. It’s like my second home. My family roots are Swedish and Danish and, it just so happens, I speak the language. Indeed, I lived and worked in the country for two years back in the late seventies. If you’ve never been, be sure and put it on your short list of places to visit…

AND IMPORTANTLY, go in the Summer! (Actually, the photos above are from the famous “Ice Hotel”–that’s right, a hotel completely made of icc. The lobby, bar, chairs, beds. Everything! If you find yourself in Sweden during the winter months, it’s a must see. I promise you’ll never forget the experience).

Anyway, the last time I was in Gothenburg, I met a clinician whose efforts to deliver consumer-driven and outcome-informed services to people on the margins of society were truly inspiring. During one of the breaks at the training, therapist Jan Larsson introduced himself, told me he had been reading my books and articles, and then showed me how he managed to seek and obtain feedback from the people he worked with on the streets. “My work does not look like ‘traditional’ therapeutic work since I do not meet clients at an office. Rather, I meet them where they live: at home, on a bench in the park, or sitting in the library or local activity center.”

Most of Jan’s clients have been involved with the “psychiatric system” for years and yet, he says, continue to struggle and suffer with many of the same problems they entered the system with years earlier. “Oftentimes,” he observed, “a ‘treatment plan’ has been developed for the person that has little to do with what they think or want.”

So Jan began asking. And each time they met, they also completed the ORS and SRS–“just to be sure,” he said. No computer. No I-phone app. No sophisticated web-based adminsitration system. With a pair of scissors, he simply trimmed copies of the measures to fit in his pocket-sized appointment book.

His experience thusfar? In Swedish Jan says, “Det finns en livserfarenhet hos klienterna som bara väntar på att bli upptäckt och bli lyssnad till. Klienterna är så mycket mer än en diagnos. Frågan är om vi är nyfikna på den eftersom diagnosen har stulit deras livberättelse.” Translated: “There is life experience with clients that is just waiting to be noticed and listened to. Clients are so much more than their diagnosis. The question is whether we are curious about them because the diagnosis has stolen their life story.”

I look forward to catching up Jan and the crew at GKC this coming week. I also be posting interviews with Ulla and Ulla as well as ICCE certified trainers Gun-Eva Langdahl (who I’ll be working with in Skelleftea) and Gunnar Lindfeldt (who I’ll be meeting in Stockholm). In the meantime, let me post several articles he sent by Swedish research Alain Topor on developing helpful relationships with people on the margins. Dr. Topor was talking about the “recovery model” among people considered “severely and persistently mentally ill long before it became popular here in the States. Together with others, such as psychologist Jan Blomqvist (who I blogged about late last year), Alain’s work is putting the consumer at the center of service delivery.

Improving Outcomes in the Treatment of Obesity via Practice-Based Evidence: Weight Loss, Nutrition, and Work Productivity

April 9, 2010 By scottdm 4 Comments

Obesity is a large and growing problem in the United States and elsewhere. Data gathered by the National Center for Health Statistics indicate that 33% Americans are obese. When overweight people are added to the mix, the figure climbs to a staggering 66%! The problem is not likely to go away soon or on its own as the same figures apply to children.

Researchers estimate that weight problems are responsible for over 300,000 deaths annually and account for 12% of healthcare costs or 100 billion–that’s right, $100,000,000,000–in the United States alone. The overweight and obese have higher incidences of arthritis, breast cancer, heart disease, colorectal cancer, diabetes, endometrial cancer, gallbladder disease, hypertension, liver disease, back pain, sleeping problems, and stroke–not to mention the tremendous emotional, relational, and social costs. The data are clear: the overweight are the target of discrimination in education, healthcare, and employment. A study by Brownell and Puhl (2003), for example, found that: (1) a significant percentage of healthcare professionals admit to feeling “repulsed” by obese person, even among those who specialize in bariatric treatment; (2) parents provide less college support to their overweight compared to “thin” children; and (3) 87% of obese individuals reported that weight prevented them from being hired for a job.

Sadly, available evidence indicates that while weight problems are “among the easiest conditions to recognize,” they remain one of the “most difficult to treat.” Weight loss programs abound. When was the last time you watched television and didn’t see an ad for a diet pill, program, or exercise machine? Many work. Few, however, lead to lasting change.

What might help?

More than a decade ago, I met Dr. Paul Faulkner, the founder and then Chief Executive Officer of Resources for Living (RFL), an innovative employee assistance program located in Austin, Texas. I was teaching a week-long course on outcome-informed work at the Cape Cod Institute in Eastham, Massachusetts. Paul had long searched for a way of improving outcomes and service delivery that could simultaneously be used to provide evidence of the value of treatment to purchasers–in the case of RFL, the large, multinational companies that were paying him to manage their employee assistance programs. Thus began a long relationship between me and the management and clinical staff of RFL. I was in Austin, Texas dozens of times providing training and consultation as well as setting up the original ORS/SRS feedback system known as ALERT, which is still in use at the organization today. All of the original reliability, validity, norming, and response trajectories were done together with the crew at RFL.

Along the way, RFL expanded services to disease management, including depression, chronic obstructive pulmonary disease, diabetes, and obesity. The “weight management” program delivered coaching and nutritional consultation via the telephone informed by ongoing measurement of outcomes and the therapeutic alliance using the SRS and ORS. The results are impressive. The study by Ryan Sorrell, a clinician and researcher at RFL, not only found that the program and feedback led to weight loss, but also significant improvements in distress, health eating behaviors (70%), exercise (65%), and presenteeism on the job (64%)–the latter being critical to the employers paying for the service.

Such research adds to the growing body of literature documenting the importance of “practice-based” evidence, making clear that finding the “right” or “evidence-based” approach for obesity (or any problem for that matter) is less important than finding out “what works” for each person in need of help. With challenging, “life-style” problems, this means using ongoing feedback to inform whatever services may be deemed appropriate or necessary. Doing so not only leads to better outcomes, but also provides real-time, real-world evidence of return on investment for those footing the bill.

Research on the Outcome Rating Scale, Session Rating Scale & Feedback

January 7, 2010 By scottdm Leave a Comment

“How valid and reliable are the ORS and SRS?” “What do the data say about the impact of routine measurement and feedback on outcome and retention in behavioral health?” “Are the ORS and SRS ‘evidence-based?'”

These and other questions regarding the evidence supporting the ORS, SRS, and feedback are becoming increasingly common in the workshops I’m teaching in the U.S. and abroad.

As indicated in my December 24th blogpost, routine outcome monitoring (PROMS) has even been endorsed by “specific treatments for specific disorders” proponent David Barlow, Ph.D., who stated unequivocally that “all therapists would soon be required to measure and monitor the outcome of their clinical work.” Clearly, the time has come for all behavioral health practitioners to be aware of the research regarding measurement and feedback.

Over the holidays, I updated a summary of the data to date that has long been available to trainers and associates of the International Center for Clinical Excellence. The PDF reviews all of the research on the psychometric properties of the outcome and session ratings scales as well as the studies using these and other formal measures of progress and the therapeutic relationship to improve outcome and retention in behavioral health services. The topics is so important, that I’ve decide to make the document available to everyone. Feel free to distribute the file to any and all colleagues interested in staying up to date on this emerging mega-trend in clinical practice.

Measures And Feedback from Scott Miller