NIMH Dumps the DSM-5: The No News Big News

May 10, 2013 By scottdm 1 Comment

Almost a year ago, I blogged about results from field trials of the soon-to-be-released, fifth edition of the Diagnostic and Statistical Manual of Mental Disorders. Turns out, many of the diagnoses in the “new and improved” version were simply unreliable. In fact, the likelihood of two clinicians, applying the same criteria to assess the same person for the two most common mental health conditions—anxiety and depression—and agreeing, was worse than it was with DSM IV, the ICD-10, or the DSM-III!

The question of validity, that is how well the diagnoses relate to real world phenomena, has never been addressed empirically in any edition. Essentially, DSM is a collection of symptom clusters, not too dissimilar from categorizing people according to the four humours—and, it turns out, about as helpful in determining the appropriate or likely outcome of any treatment provided.

Despite these serious shortcomings, the volume exerted tremendous power and influence over research and practice for the last three decades. Nearly all graduate programs teach it, research is organized around its content, and insurance companies and payers (including the Federal government) demand it for reimbursement. In short, everyone acted “as if” it were true—that is, until last week when NIMH Director, Thomas Insel, announced the organization was abandoning the DSM. As if having woken up from a thirty-year- nap the reason given was the volume’s lack of validity! Really?

The day the announcement was made, I received a bunch of emails. Most of the writers were elated. They knew I’d been critical of the volume for many years. “Finally,” one said, “a return to sanity.” My response? Not so fast.

To begin, DSM is not going away any time soon. Sorry, but if you want to be paid, keep your trusty copy nearby.

More troubling— if you read the fine print—NIMH is promising a better system, based on “a new idea everyone should welcome.” Just what is that idea? Mental health problems are biological in origin. To achieve better outcomes, NIMH funded researchers need to map the “cognitive, circuit, and genetic aspects of mental disorders” so as to identify “new and better targets for treatment.” Insel calls it, “precision medicine.”

Now, I don’t know about you, but the new idea sounds a heck of a lot like the old one to me! Psychiatry’s biological bandwagon blew into town last century and has been playing the same tune ever since. Remember the “dexamethasone suppression test” for differentiating endogenous from non-endogenous depression? How about the claims made about Xanax in the treatment of panic or the “new” anti-psychotics? There’s always prefrontal lobotomy which like the DSM, proponents continued to use and promote long after its lack of efficacy and brain disabling side effects were known. Heck, the originator won a Nobel Prize!

As far the promise of something better is concerned, history should chasten any hope one might feel. Honestly, when was the last time the field failed to claim significant progress was being made? Each new treatment approach is pitched as a vast improvement over “old ideas.” CBT is better than psychodynamic, specific is better than eclectic, evidence-based treatments are better than routine clinical practice, and so on—except none of these widely promulgated notions holds empirical water.

If “news” = new + different, then the NIMH announcement, like so much of what you find on TV and other social media, is definitely not news. It’s more of the same. Precision medicine in mental health is: 90% promise + 10% hyperbole, or marketing.

Here are a couple of newsworthy facts with immediate implications for mental health policy, practice, and research:

Treatment works. Evidence gathered over the last four decades documents that people who receive therapy are better off than 80% of those (with the same problem or concern) as those without the benefit of treatment.
A majority of potential consumers (78%) cite “lack of confidence” in the outcome of treatment as a barrier to seeking help from a mental health professional.
Tracking a consumer’s engagement and progress during treatment enables clinicians to tailor services to the individual, resulting in lower costs, fewer drop outs, and as much as three times the effects!

Just a thought—if we really want to step into the future, rather than geneticists, neurologists, and radiologists perhaps the field could start by listening to consumers. That’s exactly the point Ernesto Sirolli made at a recent TED talk. If you haven’t seen it, here it is:

The DSM 5: Mental Health’s "Disappointingly Sorry Manual" (Fifth Edition)

June 11, 2012 By scottdm 2 Comments

Have you seen the results from the field trials for the fifth edition of the Diagnostic and Statistical Manual? The purpose of the research was to test the reliability of the diagnoses contained in the new edition. Reliable (ri-lahy–uh-buhl), meaning “trustworthy, dependable, consistent.”

Before looking at the data, consider the following question: what are the two most common mental health problems in the United States (and, for that matter, most of the Western world)? If you answered depression and anxiety, you are right. The problem is that the degree of agreement between experts trained to used the criteria is unacceptably low.

Briefly, reliability is estimated using what statisticians call the Kappa (k) coefficient, a measure of inter-rater agreement. Kappa is thought to be a more robust measure than simple percent agreement as it takes into account the likelihood of raters agreeing by chance.

The results? The likelihood of two clinicians, applying the same criteria to assess the same person, was poor for both depression and anxiety. Although there is no set standard, experts generally agree that kappa coefficients that fall lower that .40 can be considered poor; .41-.60, fair; .61-.75, good; and .76 and above, excellent. Look at the numbers below and judge for yourself:

Diagnosis	DSM-5	DSM4	ICD-10	DSM-3
Major Depressive Disorder	.32	.59	.53	.80
Generalized Anxiety Disorder	.20	.65	.30	.72

Now, is it me or do you notice a trend? The reliability for the two most commonly diagnosed and treated “mental health disorders” has actually worsened over time! The same was found for a number of the disorders, including schizophrenia (.46, .76, .81), alcohol use disorder (.40, .71, .80), and oppositional defiant disorder (.46, .51., .66). Antisocial and Obsessive Personality Disorders were so variable as to be deemed unreliable.

Creating a manual of “all known mental health problems” is a momumental (and difficult) task to be sure. Plus, not all the news was bad. A number of diagnoses demonstrated good reliability (autism spectrum disorder, posttraumatic stress disorder (PTSD), and attention-deficit/hyperactivity disorder (ADHD) in children (.69, .67, .61, respectively). Still, the overall picture is more than a bit disconcerting–especially when one considers that the question of the manual’s validity has never been addressed. Validity (vuh–lid-i-tee), meaning literally, “having some foundation; based on truth.” Given the lack of any understanding of or agreement on the pathogenesis or etiology of the 350+ diagnoses contained in the manual, the volume ends up being, at best, a list of symptom clusters–not unlike categorizing people according to the four humours (e.g., phlegmatic, choleric, melancholy, sanquine).

Personally, I’ve always been puzzled by the emphasis placed on psychiatric diagnoses, given the lack of evidence of diagnostic specific treatment effects in psychotherapy outcome research. Additionally, a increasing number of randomized clinical trials has provided solid evidence that simply monitoring alliance and progress during care significantly improves both quality and outcome of the services delivered. Here’s the latest summary of feedback-related research.

My New Year’s Resolution: The Study of Expertise

January 2, 2009 By scottdm Leave a Comment

Most of my career has been spent providing and studying psychotherapy. Together with my colleagues at the Institute for the Study of Therapeutic Change, I’ve now published 8 books and many, many articles and scholarly papers. If you are interested you can read more about and even download many of my publications here.

Like most clinicians, I spent the early part of my career focused on how to do therapy. To me, the process was confusing and the prospect of sitting opposite a real, suffering, client, daunting. I was determined to understand and be helpful so I went graduate school, read books, and attended literally hundreds of seminars.

Unfortunately, as detailed in my article, Losing Faith, written with Mark Hubble, the “secret” to effective clinical practice always seemed to elude me. Oh, I had ideas and many of the people I worked with claimed our work together helped. At the same time, doing the work never seemed as simple or effortless as professional books and training it appear.

Each book and paper I’ve authored and co-authored over the last 20 years has been an attempt to mine the “mystery” of how psychotherapy actually works. Along the way, my colleagues and I have paradoxically uncovered a great deal about what contributes little or nothing to treatment outcome! Topping the list, of course, are treatment models. In spite of the current emphasis on “evidence-based” practice, there is no evidence that using particular treatment models for specific diagnostic groups improves outcome. It’s also hugely expensive! Other factors that occupy a great deal of professional attention but ultimately make little or no difference include: client age, gender, DSM diagnosis, prior treatment history; additionally, therapist age, gender, years of experience, professional discipline, degree, training, amount of supervision, personal therapy, licensure, or certification.

In short, we spend a great deal of time, effort, and money on matters that matter very little.

For the last 10 years, my work has focused on factors common to all therapeutic approaches. The logic guiding these efforts was simple and straightforward. The proven effectiveness of psychotherapy, combined with the failure to find differences between competing approaches, meant that elements shared by all approaches accounted for the success of therapy. And make no mistake, treatment works. The average person in treatment is better off than 80% of those with similar problems that do not get professional help.

In the Heart and Soul of Change, my colleagues and I, joined by some of the field’s leading researchers, summarized what was known about the effective ingredients shared by all therapeutic approaches. The factors included the therapeutic alliance, placebo/hope/expectancy, structure and techniques in combination with a huge, hairy amount of unexplained “stuff” known as “extratherapeutic factors.”

Our argument, at the time, was that effectiveness could be enhanced by practitioners purposefully working to enhance the contribution of these pantheoretical ingredients. At a minimum, we believed that working in this manner would help move professional practice beyond the schoolism that had long dominated the field.

Ultimately though, we were coming dangerously close to simply proposing a new model of therapy–this one based on the common factors. In any event, practitioners following the work treated our suggestions as such. Instead of say, “confronting dysfunctional thinking,” they understood us to be advocating for a “client-directed” or strength-based approach. Discussion of particular “strategies” and “skills” for accomplishing these objectives did not lag far behind. Additionally, while the common factors enjoyed overwhelming empirical support (especially as compared to so-called specific factors), their adoption as a guiding framework was de facto illogical. Think about it. If the effectiveness of the various and competing treatment approaches is due to a shared set of common factors, and yet all models work equally well, why would anyone need to learn about the common factors?

Since the publication of the first edition of the Heart and Soul of Change in 1999 I’ve struggled to move beyond this point. I’m excited to report that in the last year our understanding of effective clinical practice has taken a dramatic leap forward. All hype aside, we discovered the reason why our previous efforts had long failed: our research had been too narrow. Simply put, we’d been focusing on therapy rather than on expertise and expert performance. The path to excellence, we have learned, will never be found by limiting explorations to the world of psychotherapy, with its attendant theories, tools, and techniques. Instead, attention needs to be directed to superior performance, regardless of calling or career.

A significant body of research shows that the strategies used by top performers to achieve superior success are the same across a wide array of fields including chess, medicine, sales, sports, computer programming, teaching, music, and therapy! Not long ago, we published our initial findings from a study of 1000’s of top performing clinicians in an article titled, “Supershrinks.” I must say, however, that we have just “scratched the surface.” Using outcome measures to identify and track top performing clinicians over time is enabling us, for the first time in the history of the profession, to “reverse engineer” expertise. Instead of assuming that popular trainers (and the methods they promote) are effective, we are studying clinicians that have a proven track record. The results are provocative and revolutionary, and will be reported first here on the Top Performance Blog! So, stay tuned. Indeed, why not subscribe? That way, you’ll be among the first to know.