Home > Advocacy, Commentary > IOM Panelists: Balance = Experts + Information – Bias

IOM Panelists: Balance = Experts + Information – Bias

December 16th, 2013

The issue on everyone’s mind is whether the Institute of Medicine panel is the right group to come up with a new ME/CFS clinical case definition. Our team has been debating this for nearly two weeks now, and we have not reached a unanimous conclusion. But our research and discussions illuminate some key issues, particularly on the inclusion of psychologists and the possibility of bias in three panelists.
 

Enough Experts?

 
The overall panel balance is 53% known and 47% unknown to the ME/CFS community. None of us think this is sufficient, but we do not agree on what the right split would be. Some of us think the panel should be 100% ME/CFS experts; others think the addition of at least one or two more experts would be appropriate.

Mary Dimmock has raised a very important point: in order to judge whether this panel is adequate, we have to think about the task. If this panel is going to create a case definition for the broader set of conditions that meet the various CFS criteria, then some might make the argument that this panel is appropriate. But if the panel is focused on the disease we described to the FDA in April, then Mary concludes that the panel needs more, and a broader cross-section, of ME/CFS experts and fewer psychologists. As Mary has reported, HHS and IOM have been vague in detailing which disease group is the focus of the panel.

We’re faced with a chicken-egg dilemma: identifying the disease focus should happen prior to the panel selection, but instead we have a panel with a seemingly vague task focus. This panel composition seems to hold a middle ground: there are not an overwhelming number of ME/CFS experts, but neither are there an overwhelming number of experts in unexplained chronic fatigue, psychogenic models, or “other multi-symptom, complex disorders” as described in the Statement of Work. To our team, the key issue is bias.
 

Psyched Out

 
There are two psychologists on the panel: Dr. Margarita Alegria and Dr. Charles Cleeland. None of us are particularly troubled by Dr. Cleeland, as the focus of his career has been chronic pain (a significant problem for many ME/CFS patients) and outcomes measures. Dr. Alegria is another story, as discussed in the next section.

In a comment on an earlier blog post, AJ asked if panels on other diseases have included psychologists or psychiatrists. We did some research into the question, and found mixed results:

  • Psychologists/psychiatrists were included in panels on treatment for Gulf War Illness (5 of 15), epilepsy (3 of 17), living with chronic illness (3 of 17), chronic pain (5 of 19), and breast cancer and the environment (1 of 15).
  • No psychologists/psychiatrists were included in panels on Gulf War and infectious disease, veterans and Agent Orange, cardiovascular and chronic lung disease, hypertension, hepatitis and liver cancer, and ALS in veterans.
  • The only other IOM panel creating a case definition, the case definition for chronic multisymptom illness (aka GWI), includes 3 psychologists/psychiatrists out of 16 total members.

The ME/CFS panel includes 2 psychologists out of 15 total members.  Creating a case definition for ME/CFS will involve confronting the psychosocial literature and assessing its validity. For this reason, we do not categorically reject the presence of psychologists on the panel. However, we have serious concerns about psychologists who are ignorant of the methodological flaws in much of that literature. We are also concerned about the bias Dr. Alegria may bring to the task, and explore that in more detail below.
 

Bad Bias vs. Good Bias

 
The IOM Conflict of Interest Policy does not bar individuals from serving just because they have adopted positions on the issues facing the panel. In fact, the policy admits that members with potential biases might be necessary to ensure a competent and balanced panel. Bias will preclude committee service when “one is totally committed to a particular point of view and unwilling, or reasonably perceived to be unwilling, to consider other perspectives or relevant evidence to the contrary.” (p. 4)

There are multiple panelists with potential bias. For example, Drs. Bateman, Chu, Keller, Klimas, Lerner and Natelson have all taken public positions on ME/CFS case definitions. At least one team member personally knows each of these individuals, so we believe we can say that none of them are unwilling to consider other perspectives or relevant evidence.

Of greater concern to our team are three unknown panelists who may have biases that could be very harmful to the interests of the ME/CFS community. It can be challenging to assess bias at a distance based on publications, but we have legitimate concerns about three panelists: Dr. Alegria, Dr. Theodore Ganiats, and Dr. Cynthia Mulrow.

Dr. Alegria

As we mentioned in our profile of Dr. Alegria, she has co-authored three publications potentially relevant to her views on ME/CFS. The first two (here and here) address the relationship between somatic symptoms and psychiatric disorders like depression. Only 14 symptoms were assessed in these papers, and fatigue was not one of them. The papers found that having 3 or more of the 14 symptoms was associated with depression/anxiety and mental health service use. However, one of the papers correctly noted that the data are ill suited to infer causality between the physical symptoms and mental health issues.

After examining the third paper in its entirety, we came to mixed conclusions about whether it represents unreasonable bias on Dr. Alegria’s part. The paper builds on previous research showing that neurasthenia has the highest rate of comorbidity among ICD-10 disorders, and examines whether prevalence and comorbidity rates hold up across racial/ethnic groups. Comorbidity was examined only in depressive, anxiety and substance abuse disorders. Space does not permit a full parsing of the data here, including significant methodological limitations. Although the paper itself does not directly examine the relationship between neurasthenia and CFS, statements about CFS included in the paper have caused serious concerns among advocates. We carefully considered the statements and whether these are indications of unreasonable bias.

Likewise, as Lee [51] argued, the ‘‘disappearance’’ of culture-bound syndromes is related to changing sociocultural conditions, including economic and political factors and changes in managed care and pharmaceutical forces, to name a few. Indeed, chronic fatigue syndrome (CFS), also a controversial illness which has been argued to be a variant of neurasthenia [4, 21] has become increasingly diagnosed in the US [52, 53], whereas neurasthenia is virtually no longer diagnosed in the US context [51].

Many advocates are correctly concerned about the statement above that CFS is “a controversial illness which has been argued to be a variant of neurasthenia.” The first reference is to a thoroughly offensive article from 1991 that argued that CFS is a culturally sanctioned form of illness behavior.  The second reference is from 2007, but examined the concordance of CFS and three neurasthenia criteria in four medical practices in Pune, India. Neither reference can legitimately be used to support the statement that CFS should be seen as a variant of neurasthenia today.

After noting again that neurasthenia has almost disappeared from US clinical practice, the paper states:

This has implications for diagnosis and treatment of individuals who may present with symptoms of neurasthenia, but may otherwise be misdiagnosed as having depression, anxiety, or CFS.

This sentence can be read two ways. In one interpretation, neurasthenia patients are misdiagnosed with depression, anxiety or CFS because “neurasthenia” is not used in the US, and this does not imply that CFS as an entity is actually neurasthenia. In the second interpretation, people are diagnosed with CFS incorrectly because CFS is actually neurasthenia. Depending on which interpretation you choose, Dr. Alegria’s presence on the panel can be quite alarming, as she may have a simplistic, outdated view of ME/CFS or worse. But other team members see these few sentences in the paper within the context of Dr. Alegria’s research focus, which does not appear to reflect a tendency to classify patients with diagnosable physical conditions as having mental disorders.

Regardless of our personal interpretations of the meaning of these three papers, we all share a strong concern about her presence on the panel. She seems to have little experience relevant to an ME/CFS case definition at best, and potentially disastrous views if she does in fact believe that the physical symptoms of ME/CFS are equivalent to neurasthenia. Given the damaging legacy of psychogenic approaches to ME/CFS, we believe that her potential bias must be thoroughly investigated before the panel is finalized.

Dr. Ganiats

In our profile of Dr. Ganiats, we noted that Dr. Ganiats himself has not published anything directly indicating a bias on ME/CFS. However, he has been very closely associated with the American Academy of Family Practitioners. The AAFP has published problematic material about ME/CFS as recently as 2012. The AAFP’s consumer health information site presents a simplistic view of CFS, including the statement that “Most symptoms improve with time,” which is patently untrue, as many of us can personally attest.

We also noted Dr. Ganiats’ close association with colleague Dr. William Sieber. Dr. Sieber is a psychologist who has given presentations claiming that CFS symptoms are due to psychological problems. Dr. Ganiats has also published a number of papers on “quality of well-being” with Dr. Robert Kaplan, the Director of NIH’s Office of Behavioral and Social Sciences Research. We are wary of assuming bias by association, but Dr. Ganiats’s connections raise the possibility that he may have an outdated view of ME/CFS as merely fatigue of potentially psychogenic origin.

One team member suggested that Dr. Ganiats adds little to the panel as a general practitioner, especially since Dr. Bateman was a general practitioner for many years prior to opening her Fatigue Consultation Clinic. On the other hand, Dr. Ganiats brings his close relationship with AAFP and his experience in doctor-patient decision making to the table. But if Dr. Ganiats shares the views of AAFP or Dr. Sieber on ME/CFS, then he may have a strong bias towards the psychogenic view, or may at least be predisposed to broader views of the illness. This bias is unacceptable, and should be thoroughly considered by IOM before the panel is finalized.

Dr. Mulrow

Some readers took issue with our statement that Dr. Mulrow’s experience on IOM and in systematic reviews made her “very qualified to tackle the massive systematic evidence review that will be part of the IOM ME/CFS study.” And if her experience with IOM and reviews were the only question, that statement would be accurate on its own. However, as noted in our original profile, she has publications on ME/CFS that give us great pause.

Specifically, Dr. Mulrow led an evidence review on CFS in 2001 that concluded, in part, “The superiority of one case definition over another is not well established. The validity of any definition is difficult to establish because there are no clear biologic markers for CFS, and no effective treatments specific only to CFS have been identified.” It’s arguable whether this was true in 2001, but we certainly hope her view is different now.

Dr. Mulrow also co-authored a paper on treatment interventions in CFS in 2001 and based in part on that AHRQ review. Based on 2001 evidence, CBT and GET were considered promising treatments, despite some high drop out rates and lack of concrete outcome measures.

While the paper does say that effectiveness must be considered in light of those methodological issues, the analysis of these studies seems rather forgiving. The presence of multiple methodological problems did not preclude the studies being treated as highly valid. This points to potential risks in evidence-based medicine and meta-analysis approaches. The protocol for the review is established in advance, but if it is overly simplistic or has low inclusion criteria, the results of the review will reflect that lack of rigor.

Several team members feel very strongly that Dr. Mulrow is a potential problem on the panel precisely because of her systematic review expertise. They are willing to accept that she may bring an open perspective to ME/CFS itself, but fear that her process approach is ill-suited for an area that has been consistently underfunded for decades leading to an overemphasis on cheap (possibly less rigorous) science. Other team members feel that systematic review experience is necessary to help un-bias the view of the literature by setting evaluation criteria in advance. But again, if she is unwilling to approach the field as it is today then this is a problem.
 

Hiding in Plain Sight

 
There is one panelist who continues to stump us: Dr. Michael Shelanski. As we described in the original profile, he is a a pathologist focused on cell biology and animal models in Alzheimer’s disease. We don’t doubt his credentials in that field, but we fail to see how relevant that experience will be to creating a case definition for ME/CFS

Several big names in ME/CFS are missing from the IOM panel. The biggest head scratcher is Dr. Leonard Jason, who received at least seven nominations from the ME/CFS community. Dr. Jason is the authority on ME/CFS definition assessment and comparison. If he cannot serve on the panel for any reason, then his expertise must be harnessed by the panel another way.

The absence of infectious disease experts besides Dr. Lerner is another disappointment. Dr. Ian Lipkin and Dr. Jose Montoya each received at least five nominations. Dr. Lipkin is currently researching evidence for pathogens in ME/CFS, while Dr. Montoya is a clinician-scientist at Stanford University who spoke on a panel at the April FDA meeting. Either would be a strong addition to the panel to build on Dr. Lerner’s experience.

Dr. Christopher Snell received at least seven nominations, and is the leading expert on exercising testing in ME/CFS. But Dr. Keller is an adequate replacement, given her familiarity with Dr. Snell’s techniques and her own growing expertise in working with ME/CFS patients.

The IOM did not appoint a cardiologist or endocrinologist to the panel, and this is a disappointment. Cardiac and endocrine abnormalities have been part of the ME/CFS picture for decades, and both were listed in the Statement of Work (although not as requirements). While Dr. Natelson is a neurologist, inclusion of a researcher specializing in cognitive impairments in ME/CFS would also be advisable.
 

A Final Step

 
Feedback on the provisional panel is due December 23rd. Our next post will include the feedback we intend to submit, and we hope that other advocates will also participate in whatever way they believe is appropriate.

 

Acknowledgements: This post was a group effort, and would not have been possible without the assistance and participation of Lori Chapo-Kroger, Claudia Goodell, Chris Heppner, Denise Lopez-Majano, Mike Munoz, Darlene Prestwich, Tamara Staples, WillowJ, and one advocate who wished to remain anonymous.

 

  1. floydguy
    December 17th, 2013 at 08:52 | #1

    Re: task at hand

    I don’t have a statement in hand but in my mind (major hazard…brain fog, blah, blah) I thought Nancy Lee made it clear that this committee was vacuuming up anything associated with fatigue or un-wellness. So that would include anyone who has 6 months of idiopathic “fatigue”/unwellness to those bed-bound with clear immune, neurological, endocrine, cardiac, PEM and all the rest recognized abnormalities.

    I would find it a major surprise if the CDC/HHS looked at this differently. Their bias always seems to be to suck as many random people into this illness as they possibly can.

  2. Ess
    December 17th, 2013 at 12:41 | #2

    We KNOW what ‘their’ BIAS is; ‘they’ have demonstrated this OVER and OVER and OVER again–and it is all leading to the same SINK HOLE with the IoM contract which = a DISASTER for the ME/CFS patient community.

    A HUGE is ‘impasse’ is what OLD, OUTDATED, STACK of MISinformation will be given to the panel as REFERENCE—and what UP-TO-DATE CURRENT findings will be DISALLOWED within the parameters ‘laid out’.

    There is NO LOGICAL SENSIBLE INTELLIGENT reason for such a contract WITH the IoM, NO LESS, to be taking place ‘at this moment’ in time . . .

  3. Ess
    December 17th, 2013 at 13:08 | #3

    P.S. — This ‘exercise’ (of the IoM contract) BEGS the QUESTION–Just WHAT exactly is going on here??

    THE ME/CFS EXPERTS AND RESEARCHERS WORLD-WIDE are being GROSSLY and DIRECTLY DISregarded by Kathleen Sebelius of HHS. WHAT ????!!!!! THAT is KEY as to the AGENDA of IoM, CFSAC, CDS, HHS.

    This picture couldn’t be ‘more OUT OF WACK’—YET, we can get ‘stuck’ in it because the ONgoing MIStreatment/ABUSE becomes the ‘NORM’.

    THIS–THIS — MIStreatment and INjustice towards ME/CFS and we persons with this DEBILITATING, LIFE-ALTERING SERIOUS DISEASE is far from ‘NORMAL’ and DEFIES all LOGIC and INTELLIGENCE.

    We must look at the bigger picture. WHAT is going on here ??!!!! The ME/CFS EXPERTS AND RESEARCHERS have banned together to HELP us–after all these years/decades of this disease largely being IGnored–the ME/CFS EXPERTS AND RESEARCHERS are asking for cancellation of the IOM contract and adoption of the CCC. Let’s give them our support!

  4. Emma
    December 17th, 2013 at 15:33 | #4

    Endless thanks for all the work you and the group are putting in. You’re doing such important work.

  5. Alex Young aka alex3619
    December 17th, 2013 at 17:15 | #5

    The process itself is biased, and the team cannot change that.

    The IOM has to include 25% non-experts, but it doesn’t have to have 47%. Of those non-experts at least one should be a patient representative or representative of a patient organization, according to IOM standards. At least one is already a patient or former patient, so that part of the standards is covered.

    The UK Department of Work and Pensions has just released an “evidence-based” report endorsing the Work Capability Assessments as accurate. Clearly the report is flawed, and I do not think this is from personal bias, I think its from methodological bias.

    Such methodological bias ties the hands of researchers. The are not free to explore the topic, but are required to explore only certain things. This limits the available outcomes. We really need to see their contract, not the statement of work.

  6. Holly
    December 17th, 2013 at 23:28 | #6

    Thank you for doing this research and passing the information along. I am truly grateful. Please continue to keep us informed and let us know of any action we should be taking.

  7. Justin Reilly
    December 18th, 2013 at 05:46 | #7

    I think it should be noted that in the following paper (2012) you cite by Dr. Alegria, she is concerned about what she asserts is the proven under-detection of mental disorders among people with physical symptoms. This paper is recent enough that I think we can assume that she will still be concerned with and looking for under-diagnosed mental disorders in people with physical symptoms. It’s not, by itself, a smoking gun, but I think it buttresses the conclusion that she is totally inappropriate and unacceptable for this panel.

    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3487465/
    “Previous research has demonstrated that psychiatric conditions are less likely to be identified among patients with comorbid psychological and physical symptoms, that physicians are less likely to discuss mental health concerns with patients who have greater medical complexity, and that physicians, not patients, initiate biomedical evaluations and interventions for physical symptoms (Ring et al., 2005; Rost et al., 2000). Under-recognition of mental disorders is particularly problematic among racial/ethnic minorities (Borowsky et al., 2000), with evidence that providers talk less about these conditions with minority patients (Tai-Seale et al., 2007). Future research on the under-detection of mental disorders among individuals with physical symptoms should address contextual factors by considering potential contributions from clinical providers and healthcare systems.”

  8. Justin Reilly
    December 18th, 2013 at 05:54 | #8

    In the below paper, which you haven’t mentioned, Alegria et al. discuss the National Latino and Asian-American Study (NLAAS), of which it seems they were authors. They felt it was important to incorporate “cultural idioms of distress (e.g., “ataque de nervios” and neurasthenia) as part of the prevalence estimates of psychiatric disorders.” This again is not a smoking gun, but it is more evidence that she thinks neurasthenia, a “culturally-bound” psychiatric disorder, is important and merits more study.

    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2774128/#R28

  9. Justin Reilly
    December 18th, 2013 at 06:02 | #9

    On her institution’s website, two students (post-grads it seems) mention that Dr. Alegria supervised their study of neurasthenia.
    http://www.multiculturalmentalhealth.org/centerprofilesarchive.asp

  10. Justin Reilly
    December 18th, 2013 at 06:36 | #10

    I also found two more papers where she mentioned the need to count neurasthenia as a mental disorder among Asian-Americans. It also seemed to be a major part (one of three topics mentioned) of a Continuing Education seminar on women’s health she gave.

    • Jennie Spotila
      December 18th, 2013 at 07:47 | #11

      Great work finding those papers, Justin. I’m not sure how the team missed them because we specifically looked for neurasthenia papers from her. I’m not worried if she thinks neurasthenia is worth studying. But I AM worried if she thinks ME/CFS is in any way equivalent to neurasthenia. I’ll be posting my feedback to IOM in another day or so, and we express these concerns about her.

  11. Justin Reilly
    December 18th, 2013 at 08:37 | #12

    Allegria says: CFS is a “Neurasthenia Spectrum Disorder”!!

    In the Neurasthenia paper of Alegria’s you cite, below, you seem to have missed this important quote: “Likewise, further research is needed to examine the concordance of neurasthenia spectrum disorders (e.g., ICD-10 neurasthenia, CFS, CCMD-2 and CCMD-3 [these last two are the two Chinese Criteria] for neurasthenia) across non-clinical, population-based samples across cultural contexts [cf. 21, 23].”

    “Prevalence of Neurasthenia, Comorbidity, and Association with Impairment Among a Nationally Representative Sample of US Adults” (2012)

    http://forums.phoenixrising.me/index.php?threads/prevalence-of-neurasthenia-comorbidity-and-association-with-impairment-among-a-nationally-represen.27105/

    For this proposition, the authors cite the Pune, India study of neurasthenia spectrum disorders. I don’t have access to the full article, but the abstract is damning enough:

    “Clinically significant fatigue or weakness is a common but understudied clinical problem in India…Alternative criteria sets used in different clinical contexts suggest a range of conditions constituting neurasthenia spectrum disorders (NSDs)…CFS, ICD-10 neurasthenia, DSM-IV draft criteria for neurasthenia, and CCMD-2 [ie Chinese Criteria] neurasthenia.”

    This study was an attempt to see if CFS and the other three “neurasthenia categories” mentioned were able to define idiopathic chronic fatigue well!

    Paralikar V, Sarmukaddam S, Agashe M, Weiss MG (2007) Diagnostic concordance of neurasthenia spectrum disorders in Pune, India. Soc Psychiatry Psychiatr Epidemiol 42:561–572

    http://link.springer.com/article/10.1007/s00127-007-0196-x#page-1

    I do not have access to the other study cited:
    Starcevic V, Kelin K, Munjiza M (1996) Characteristics of neurasthenia: examination and cross-cultural applicability of ICD-10 diagnostic criteria for research. Eur J Psychiatry 11:289– 297

    • Jennie Spotila
      December 18th, 2013 at 08:58 | #13

      We didn’t miss those quotes. We discuss them in this post. And the group was mixed in their interpretation of whether Alegria equates CFS and neurasthenia, as we explain in the post.

      Our feedback to IOM in in final revision, but I’ll post it this week. And others who have concerns about Alegria (or anyone else) should speak up!! You have found some good material on Alegria, Justin, so I hope you will write to IOM and express your concerns. But I also understand that not every advocate feels it is appropriate to do so.

  12. Anushka
    December 18th, 2013 at 10:22 | #14

    All too long psycho-somatic “research” and psychiatrist have highjacked ME with severe consequences for millions of patients around the world.

    Nothing else has hurt ME patients more than the psych lobby.

    The most vulnerable, our children with ME, have been maltreated by psychiatric community world wide. Children and mothers with ME receive – in the best case scenario – no help, in the worst case scenario their children are taken away from them and young adults are incarceration against their will into mental institution. Mothers with ME with children with ME are falsely accused of Muenchhausen Proxy. Such practices can only be compared with a 18. century witch hunt or living under a regime which abuses its own citizen.

    You don’t ask a tyrant to negotiation table.

    You don’t ask your torturer, who mistreated you and your children, to assess the pain he has inflicted on you.

    There is only one answer for ME

    ZERO TOLERANCE FOR PSYCHOLOGIST, PSYCHIATRIST AND PSYCHO-SOMATIC RESEARCH IN THE FIELD ME.

    Further more, stop accepting less than we deserve.

    We need immunologist, infectious disease and cancer specialist who have REAL expertise assessing the current advance made in immunological research in to ME .

  13. Justin Reilly
    December 20th, 2013 at 02:22 | #15

    Jennie,
    Pls look carefully at your section on Dr. Alegria; you do NOT discuss the quote I said you missed: “Likewise, further research is needed to examine the concordance of neurasthenia spectrum disorders (e.g., ICD-10 neurasthenia, CFS, CCMD-2 and CCMD-3 [these last two are the two Chinese Criteria] for neurasthenia) across non-clinical, population-based samples across cultural contexts [cf. 21, 23].”

    (1) It was not quoted,
    (2) It was not discussed or considered

    If it had been discussed your group would not be “mixed in their interpretation of whether Alegria equates CFS and neurasthenia, as we explain in the post.” If the group considered her quote that CFS is a “Neurasthenia Spectrum Disorder” along with the cite to support, which was a study to determine if any of four “Neurasthenia Categories” including CFS could define Idiopathic Chronic Fatigue in India, there would be no doubt in your minds that Alegria does equate CFS and Neurasthenia.

Comments are closed.