.png)
.jpg)
Have you seen the results from the field trials for the fifth edition of the Diagnostic and Statistical Manual? The purpose of the research was to test the reliability of the diagnoses contained in the new edition. Reliable (ri-lahy–uh-buhl), meaning “trustworthy, dependable, consistent.”
Before looking at the data, consider the following question: what are the two most common mental health problems in the United States (and, for that matter, most of the Western world)? If you answered depression and anxiety, you are right. The problem is that the degree of agreement between experts trained to used the criteria is unacceptably low.
Briefly, reliability is estimated using what statisticians call the Kappa (k) coefficient, a measure of inter-rater agreement. Kappa is thought to be a more robust measure than simple percent agreement as it takes into account the likelihood of raters agreeing by chance.
The results? The likelihood of two clinicians, applying the same criteria to assess the same person, was poor for both depression and anxiety. Although there is no set standard, experts generally agree that kappa coefficients that fall lower that .40 can be considered poor; .41-.60, fair; .61-.75, good; and .76 and above, excellent. Look at the numbers below and judge for yourself:
| Diagnosis | DSM-5 | DSM4 | ICD-10 | DSM-3 |
| Major Depressive Disorder | .32 | .59 | .53 | .80 |
| Generalized Anxiety Disorder | .20 | .65 | .30 | .72 |
Now, is it me or do you notice a trend? The reliability for the two most commonly diagnosed and treated “mental health disorders” has actually worsened over time! The same was found for a number of the disorders, including schizophrenia (.46, .76, .81), alcohol use disorder (.40, .71, .80), and oppositional defiant disorder (.46, .51., .66). Antisocial and Obsessive Personality Disorders were so variable as to be deemed unreliable.
Creating a manual of “all known mental health problems” is a momumental (and difficult) task to be sure. Plus, not all the news was bad. A number of diagnoses demonstrated good reliability (autism spectrum disorder, posttraumatic stress disorder (PTSD), and attention-deficit/hyperactivity disorder (ADHD) in children (.69, .67, .61, respectively). Still, the overall picture is more than a bit disconcerting–especially when one considers that the question of the manual’s validity has never been addressed. Validity (vuh–lid-i-tee), meaning literally, “having some foundation; based on truth.” Given the lack of any understanding of or agreement on the pathogenesis or etiology of the 350+ diagnoses contained in the manual, the volume ends up being, at best, a list of symptom clusters–not unlike categorizing people according to the four humours (e.g., phlegmatic, choleric, melancholy, sanquine).
Personally, I’ve always been puzzled by the emphasis placed on psychiatric diagnoses, given the lack of evidence of diagnostic specific treatment effects in psychotherapy outcome research. Additionally, a increasing number of randomized clinical trials has provided solid evidence that simply monitoring alliance and progress during care significantly improves both quality and outcome of the services delivered. Here’s the latest summary of feedback-related research.
Hi Scott
We are struggling here in Australia with the DSM V with its obvious conflict of interest issues in relation to the authors and seeming bent towards psychopharmiceuticals as a first option treatment. As you have clearly set out in your blog there are further issues in relation to diagnosis issues. I came across the following article I thought you may be interested in.
Here are two statements on DSM 5. The first is Allen Frances’ critical commentary on DSM 5. It is well done, although I still think we need to wait for the research-founded NIMH brain-based system, due around 2015-2020.
Following Frances’ statement is the David Kupfer rebuttal, which, as always, fails to deal with the issues raised.
We also should note that DSM-5 is maintaining the wording “attenuated psychosis syndrome” as an item needing further study. The term APS is highly inappropriate. Ultra high risk (UHR) is the term used in Australia–and there it is used for prevention, rather than fixating on a diagnosis. All of us need to be aware that the science behind UHR is impressive and building. The potential to prevent teenage disturbance has been demonstrated. We need to think prevention, whereas an APS diagnosis will lead to increased use of antipsychotics and loss of gray matter. In short, the battle must continue!
As Kirkegaard said many years ago:
“Once you label me, you negate me.”
Naming “disorders” brings with it major problems.
Despite the valiant efforts of Allen Frances, Paula Caplan, David Elkins, and many others, I suspect DSM 5 will arrive as scheduled. The American Psychological Association and my friends who are in private practice just smile when I criticize. They are fully aware that lowering diagnostic thresholds will be good for their “business.”
(On the other hand, I find the new personality style (aka “disorder”) concepts rather helpful. With a little study,they are easy to teach and well received by those who attend Mary and my workshops.)
The Biggest Problems With DSM-5?
What’s Wrong With DSM-5?
Stephen M. Strakowski, MD; Allen J. Frances, MD
Authors and Disclosures
Posted: 06/01/2012
• Print This
• Email this
• Discuss This
• Share
• Introduction
• The Biggest Problems With DSM-5?
• What Would Dr. Frances Do?
• A Safe, Credible DSM-5 by 2013?
Information from Industry
Did you know that clinical and EEG features of Lennox-Gastaut syndrome may change over time?
Learn more about patient assessment
Editor’s Note:
The fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) is scheduled for release in May 2013. As preliminary draft revisions to the current DSM diagnostic criteria have been made available for review, multiple prominent psychiatrists have spoken out against finalizing a revised edition at this time. The proposed updates were addressed at this year’s American Psychiatric Association (APA) Annual Meeting, and recently it was revealed on DSM5.org that 2 controversial condition proposals — attenuated psychosis syndrome and mixed anxiety depressive disorder — will now be included in section III of the revised manual, meaning that more research is required before consideration as formal disorders. Dr. Allen J. Frances, MD, a professor emeritus at Duke University and chair of the DSM-IV Task Force, has been an outspoken dissenter against numerous proposed DSM-5 revisions. Medscape recently invited Dr. Stephen M. Strakowski, Chairman, Department of Psychiatry and Behavioral Neuroscience, University of Cincinnati College of Medicine, to interview Dr. Frances about his views on revising psychiatry’s principal diagnostic resource.
Introduction
Dr. Strakowski: Dr. Frances, you have been fairly critical of the creation of the newest psychiatry diagnostic manual, the DSM-V. What is your background that allows you to comment? And what are your major concerns, particularly in light of the announcement this week that 2 of the more controversial proposals would be dropped?
Dr. Frances: My most relevant background was working on DSM-III, DSM-IIIR, and chairing the DSM-IV Task Force; conducting research on psychiatric diagnosis; and, perhaps equally important, having evaluated and treated many thousands of patients.
My great fear is that DSM-5 will inappropriately expand the boundaries of psychiatry and increase our already existing problem of diagnostic inflation. This will result in inappropriate medication use, unnecessary stigma, and the misallocation of scarce mental health resources. The DSM-5 proposal that got me started on this reluctant crusade was “psychosis risk” (since renamed “attenuated psychotic symptoms”). “Psychosis risk” is a well-intentioned research diagnosis aimed at preventing or reducing the burden of schizophrenia. However, it would have terrible, unintended consequences were it to become an official DSM-5 disorder and be widely applied in general clinical practice. Most of the leading researchers in the field are against its premature inclusion in DSM-5, and thankfully the DSM-5 Task Force just announced that they will drop the proposed change.
That the DSM-5 team finally backed down on its worst suggestion is a big plus — thank goodness that psychosis risk is no longer in play. Also good is their rejection of mixed anxiety depression and the partial correction of what had been a ridiculously loose definition of attention-deficit/hyperactivity disorder (ADHD), allowing for onset as late as age 12, which would confound ADHD with other causes of hyperactivity, impulsivity, and distractibility. That said, there are many other radical and potentially dangerous ideas in DSM-5.
Dr. Strakowski: The pushback to not releasing a new edition might be that DSM-IV is old enough now and there have been sufficient advances that a new DSM is needed. Do you think that assumption is correct?
Dr. Frances: The exponential basic science advances in understanding how the normal brain works have not yet translated into any advances in how clinical psychiatry is practiced. Regrettably and surprisingly, there are still no biological tests for any of the mental disorders. We are years away from having an Alzheimer’s battery and there is nothing very encouraging in the diagnostic pipeline for any of the other problems we treat. The brain has turned out to be elusively complex; the more we learn about it, the more we understand how little we know and how tough it will be to pin down pathogeneses. Each disorder in the DSM is probably just a final common presentation caused by hundreds of different and heterogeneous biological pathways. Clearly, it will take decades to make meaningful steps in understanding psychopathology — a real slog with very small, very hard-won steps.
In the meantime, we are left with a descriptive psychiatry that has great limitations but enormous practical uses and influence. It makes no sense to change the system based on the arbitrary decisions of those in charge of the rewrite. Indeed, a strong case could be made that the world would be a better place were we all still using DSM-III. Each change in a DSM is an opportunity for clinical and forensic misuse, with unpredictable and harmful unintended consequences. Changes are also costly to psychiatric research in direct ways, such as the cost of changing instruments midstream, and even more importantly in indirect ways, such as the noncomparability of findings across time.
It might be worth the risks and costs of DSM-5 changes if there were compelling science supporting them, but there isn’t. That is why I have been calling for an independent scientific review of the DSM-5 proposals and also why APA has refused. I would bet my house that none of the DSM-5 changes would ever make it through an impartial and rigorous scientific review.
Dr. Strakowski: But isn’t the goal of the DSM to provide guidance to psychiatrists in order to better define patients (“cases”) so as to better direct treatment? Wouldn’t it be helpful, then, to expand and identify as many cases as possible, which might very well require new diagnoses and descriptions? What are the real risks to our patients or to psychiatry?
Dr. Frances: We want to identify the patients we can help and we want to avoid identifying the ones who don’t need treatment or who may be harmed by it. Psychiatry does best with, and is most needed by, patients who are moderately to severely ill. Studies consistently show that they benefit with strong treatment effects. In contrast, studies of the mildly ill show equally consistently that they experience no significant benefit compared with placebo. We already have a perverse misallocation of resources, with two thirds of severely depressed people receiving no mental health care, while many on antidepressants are placebo responders who don’t really need them.
DSM-5 will make this much worse by dramatically reducing thresholds for existing disorders and by introducing new and untested disorders with high prevalence rates. Literally tens of millions of new “patients” will be created overnight with no evidence that they can be accurately identified, no proven effective treatment, and every indication that the medication treatments offered can have very harmful side effects. This will lead to worse directed treatments, more harm than good for patients, and even greater distortions in the allocation of scarce resources.
Dr. Strakowski: Now that attenuated psychosis syndrome and mixed anxiety depressive disorder are no longer in play, what are your biggest concerns with the remaining DSM-5 proposals?
Dr. Frances: Again, the DSM-5 Task Force responding to criticism is good news. But there is also bad news. First off, DSM-5 reported embarrassingly poor field trial results at this year’s meeting, yet it still plans to go ahead with its May 2013 publication. This violates the original DSM-5 plan (and promise) to rewrite the poorly drafted criteria sets and then to retest them to ensure that they were acceptable. As it stands, DSM-5 not only lacks credibility itself but also will tarnish the credibility of our entire field. The only possible motivation for this shortsighted decision is that APA is choosing publishing profits over public trust.
Also discouraging are all the remaining reckless suggestions that now appear fast-tracked to make it into DSM-5. Here is a short list:
1. Disruptive mood dysregulation disorder
2. Minor neurocognitive disorder
3. Removing the bereavement exclusion for major depressive disorder
4. Lowered ADHD threshold (by raising the allowed age of onset to 12)
5. Lowered threshold and poor reliability for generalized anxiety disorder
6. Combining substance abuse with substance dependence under the rubric of addictive disorders yielding low reliability and unnecessary stigma
7. A category for “behavioral addictions” that will promote “Internet addiction” as a NOS (not otherwise specified) diagnosis; can “addictions” to sex, shopping, and work, etc, be far behind?
8. Wording of the pedophilia criteria set that tries to sneak in hebephilia and invites forensic abuse
9. Making binge-eating a mental disorder
10. An unusable personality section that the APA Assembly voted unanimously to oppose
What Would Dr. Frances Do?
Dr. Strakowski: What would be your recommendation vis-à-vis making diagnostic changes, improving criteria sets, or altering DSM-IV at this particular time?
Dr. Frances: There were 5 problems that emerged since the publication of DSM-4 that urgently needed addressing:
1. The sky-rocketing rates of autism — an increase of twentyfold
2. The sky-rocketing rates of childhood bipolar disorder — an increase of twentyfold
3. The tripling of rates of ADHD
4. The doubling of rates of adult bipolar disorder
5. The misuse of the label “paraphilia NOS” to sanction the questionably constitutional involuntary commitment of rapists as a veiled form of preventive detention
Dr. Strakowski: What approach would you recommend, then, in dealing with these issues? Are criteria change and a new DSM-5 needed, perhaps with fewer new diagnoses as we discussed earlier, or a modification of existing criteria? Or perhaps a more rigorous use of the criteria as they are currently written?
Dr. Frances: For childhood bipolar and autism, I would recommend black box warnings in DSM-5 cautioning against overdiagnosis as well as conferences sponsored by APA and other pertinent associations to reeducate clinicians on proper use of the criteria sets. For ADD: tightening the criteria set. The current DSM-5 proposal would dramatically loosen it. For bipolar II, I would recommend requiring more and longer hypomanic episodes. And last, I would propose rewriting the entire paraphilia section. The DSM-IV wording is imprecise and has been badly twisted in hearings determining involuntary psychiatric commitment under sexually violent predator statutes.
What Would Dr. Frances Do?
Dr. Strakowski: What would be your recommendation vis-à-vis making diagnostic changes, improving criteria sets, or altering DSM-IV at this particular time?
Dr. Frances: There were 5 problems that emerged since the publication of DSM-4 that urgently needed addressing:
1. The sky-rocketing rates of autism — an increase of twentyfold
2. The sky-rocketing rates of childhood bipolar disorder — an increase of twentyfold
3. The tripling of rates of ADHD
4. The doubling of rates of adult bipolar disorder
5. The misuse of the label “paraphilia NOS” to sanction the questionably constitutional involuntary commitment of rapists as a veiled form of preventive detention
Dr. Strakowski: What approach would you recommend, then, in dealing with these issues? Are criteria change and a new DSM-5 needed, perhaps with fewer new diagnoses as we discussed earlier, or a modification of existing criteria? Or perhaps a more rigorous use of the criteria as they are currently written?
Dr. Frances: For childhood bipolar and autism, I would recommend black box warnings in DSM-5 cautioning against overdiagnosis as well as conferences sponsored by APA and other pertinent associations to reeducate clinicians on proper use of the criteria sets. For ADD: tightening the criteria set. The current DSM-5 proposal would dramatically loosen it. For bipolar II, I would recommend requiring more and longer hypomanic episodes. And last, I would propose rewriting the entire paraphilia section. The DSM-IV wording is imprecise and has been badly twisted in hearings determining involuntary psychiatric commitment under sexually violent predator statutes.
Scott, this is great information and I will add it to my practice course!!
One of my favorite things to show students is research on the DSM IV-TR from chief DSM architects Williams, Gibbon, First, Spitzer, Davies, Borus, Howes, Kane, Pope, Rounsaville, & Wittchen, 1992.
As I am sure you know, overall agreement of trained psychiatrists in those trials ranged from .68- .72 for Axis I and .56-.64 for Axis II
And this was for the same CLASS of disorders not the specific TYPE ……..WOW!!!
Individual reliability ratings for SPECIFIC disorders were as low as .26
THIS IS THE PART I LOVE!!!
I use this data to get under the skin of my psychopathology professor colleague across the isle by explaining to students that if they are ever given a test in a DSM related course where they are expected to read case scenarios and apply a diagnosis, that research from the chief architects of the DSM would indicate that between 40-50% (or higher) of them would fail the exam depending on the “expected” diagnoses!!! Based on the new research I will adjust the expected fail rate upward!!
I promise each year that if any of them fail a psychopathology exam that I will assist in crafting an appeal based on the research ☺
Thanks for the great blog posts and info…
Chris