The Statement of Work
I have obtained a copy of the Statement of Work (SOW) for the Institute of Medicine study on clinical diagnostic criteria for ME/CFS. I am making that document available to you in its entirety through this link. I am also offering my interpretation of some of the most significant provisions of that SOW in this post.
But first, I want to state up front how I got this document and what I’ve done with it. There have been a multitude of accusations and allegations swirling in the ME/CFS advocacy community over the last week, and I figure that someone somewhere will question my involvement, so I’ll just get this out of the way right now. I requested the SOW from Dr. Nancy Lee by email on September 24th. Dr. Lee’s office responded on September 25th with a brief variation on the original announcement. On September 27th, I responded to Dr. Lee’s office pointing out that I had requested a copy of the actual SOW document, and I attached a copy of the SOW from the IOM’s study on treatments for Gulf War Illness as an example. I also filed an expedited FOIA request for the document. Finally, at 6:20 pm on September 30th, Dr. Lee’s office sent me the Statement of Work document I link to above and that I discuss in this post. Simultaneously with publishing this blog post, I am also sending an email to a number of advocates giving them the links to the document and this post. Let me be clear: this post contains my opinions and mine alone. I did not share or discuss the SOW with anyone (except my husband) before publishing this post.
We have a number of lawyers, doctors and other professionals in the advocacy community. I hope they will examine the SOW closely and offer their own analysis. I also hope that our expert clinicians and researchers (including the 35 experts who signed the Open Letter) will also scrutinize the document. We need open dialogue among all the factions in the advocacy and scientific community. If there is any hope of salvaging the IOM process and shaping it to meet the needs of ME/CFS patients and their families, then everyone needs to share information and communicate with one another. We won’t all agree, but we can stop slitting each others’ throats.
On to my analysis – again, the full document can be viewed here. I want to focus your attention on five issues: what disease is being studied; the requirements for involving experts; the requirements for stakeholder input; the project timeline; and whether I think this whole thing is good or bad.
This SOW is seeking a single clinical definition for what could be multiple disorders.
The first page of the SOW states, “HHS will request that the IOM develop consensus clinical diagnostic criteria for this disorder.” (emphasis added) The term “ME/CFS” is used throughout the document and is defined on page 2 as follows:
For the purposes of this document ME/CFS shall be used to refer to Myalgic Encephalomyelitis (ME), Chronic Fatigue Syndrome (CFS), Chronic Fatigue and Immune Dysfunction Syndrome (CFIDS), Neuroendocrine Immune Disorder, and other terminologies in use for this illness. (emphasis added)
I submit to you that this statement tells us a great deal. ME and CFS are not two separate diseases in the view of HHS. This is a value judgment that tells IOM that all the entities referred to by those names are really the same thing. Furthermore, all the entities described in the case definitions listed on page two, and the entity(ies) being studied in the CDC multi-site study, are all the same thing. This does not preclude the IOM committee from reaching a different conclusion, but this assumption is a pretty powerful frame for the scope of work.
My experts may not be your experts.
The biggest bone of contention regarding use of the IOM is the open question of how many ME/CFS experts will be involved in the committee. Most advocates, including me, tried to stop this contract because we feared that experts would not comprise the majority of the committee. The CFIDS Association position announced on September 23rd did not oppose the IOM contract as long as the new clinical definition was created by ME/CFS experts. Now that we have the SOW, we can make a better guess about how this will pan out. On page two, the SOW states that the committee shall have expertise in:
- the pathophysiology, spectrum of disease, and clinical care of ME/CFS;
- infectious disease;
- primary care, nursing, and other healthcare fields;
- health education;
- and the patient/family perspective.
I listed the categories on separate lines based on the placement of semi-colons, which is how the sentence would be understood grammatically. I would also like to point out that this list is different from the description of necessary expertise in the HHS announcement about the contract from September 23rd. Missing is the earlier reference to an expert in behavioral health, but the inclusion of “other healthcare fields” could easily be used as justification for including a psychologist or other behavioral expert. Also missing is expertise in epidemiology and developing clinical case definitions. Finally, I have to point out one additional qualification in the SOW: “Individuals with expertise in other multi-symptom, complex disorders or diseases may also be included on the committee.”
The SOW does not define “expertise,” as in how much experience is needed to qualify as an expert in ME/CFS. In their position issued last week, PANDORA said, “we define an ME/CFS clinical expert as a doctor who has primarily provided clinical care to ME/CFS patients for at least 10 years. We define an ME/CFS research expert as someone who primarily researches ME/CFS, particularly ME/CFS researchers who have published on biomarkers and definitions.” (emphasis in original; page 3 of Position Statement on ME/CFS Criteria). This seems like a logical definition to me, but no such definition is included in the SOW.
Another gap is the distribution of expertise among committee members. The SOW specifies that the expert committee should include 12 to 16 people (the HHS announcement said approximately 15). There is no minimum number of experts for any of the categories, including pathophysiology and clinical care of ME/CFS or the patient/family perspective. A literal reading of the SOW does not even preclude one person from claiming both of those areas of expertise. Not only does the SOW not require that a majority of the committee be subject matter experts, it doesn’t even require more than one of them. And I am certain that I am not the only advocate who finds the reference to “expertise in other multi-symptom, complex disorders” to be ominous.
Stakeholder input is mentioned but not defined.
The SOW includes a requirement for stakeholder input, but a careful reading shows that the requirement is not very specific and could be meaningless. The first topic area of the task described on page two states, “Conduct a study to identify the evidence for various diagnostic clinical criteria of ME/CFS using a process with stakeholder input, including practicing clinicians and patients” (emphasis added). On page three, the SOW describes that process with a bit more detail:
During one or more public meetings, the Committee shall consider the testimony of ME/CFS patients, caregivers, and advocacy groups in addition to relevant subject-matter experts such as clinicians, researchers, service providers, and public health officials.
Sound good? Let’s parse this sentence. First, it specifies “one or more public meetings.” That means only one public meeting is required. There are no requirements as to the duration of that meeting or meetings. By comparison, the public meeting for the IOM’s Gulf War Illness definition study lasted five hours. One hour was devoted to discussion from five patients. Six physicians/researchers had a combined 90 minutes to speak. Open public comment was limited to 30 minutes. I have found no evidence that IOM has employed a comment docket for that panel, like FDA did for the April ME/CFS meeting. Under the terms of this ME/CFS SOW, a five hour public meeting with a single hour dedicated to patients would be sufficient. I don’t know whether the IOM will do a better job in our case, but the SOW does not require it.
There are many different groups included in the SOW “stakeholder input.” Testimony will be considered from patients, caregivers, advocacy groups, clinicians, researchers, service providers AND public health officials. Seven categories, each with very different skill sets and goals. The FDA April 25th meeting spent most of the four hours hearing from patients and caregivers, with a few groups and clinicians sprinkled in. Furthermore, the April 25th meeting posed a series of very specific questions to focus discussion on what was most relevant to FDA, and also used a facilitation format to identify high priority common themes.
Finally, this part of the SOW says the committee shall “consider” the stakeholder testimony. It does not tell the committee HOW to consider the input or how much significance and weight it deserves. There is no stated goal for considering that input or answering stakeholder concerns. The SOW is intentionally broad, so the committee will have complete discretion to disregard all the input it receives at one extreme or rely on that input to shape the entire outcome at the other extreme.
There is a timeline but no start date.
Page three of the SOW lists a draft timeline of the project stages, but provides no start date for the clock to begin ticking. If we assume that last week’s HHS announcement is accurate, then the start date was some time in September. In that case, the 18 month timeline for this project will conclude by March 31, 2015 (that is also the deadline noted on page four of the SOW). You can see the full timeline in the SOW document, but I want to point out a couple important dates that I calculated by assuming work began at the end of September 2013.
- By December 2013: “Seek committee nominations, assemble committee, begin background research; contact stakeholders” That means nominations to the committee will be due very soon, and we better get ready for that. There is no explanation of how nominations will be collected or what is meant by “contact stakeholders,” including which groups are included or the purpose(s) of the contact.
- January -March 2014: “Hold first committee meeting; plan workshops – identify topics and potential speakers; identify information needs; develop plans for receiving stakeholder input” The SOW does not explain what the “workshops” will be, and obviously there is no stakeholder input by this point because plans for collecting it are still being formulated.
- March-July 2014: “Hold second and third committee meetings with workshops; develop report outline and workplan; hold working group conference calls; seek and receive input from stakeholders” Are the workshops for the committee or for the public? Notice that the report is being outlined at the same time as stakeholder input is being received.
- August-December 2014: “Hold fourth and fifth committee meetings; draft and revise the report; draft recommendations” Presumably stakeholder input has been concluded. Hopefully it will be considered as the recommendations are drafted. Stakeholder input is not mentioned in the timeline after July 2014.
Good Or Bad?
It is important to understand the process that the IOM typically uses in its studies in order to assess whether this SOW is good or bad for us. Detailed information describing that process is available on the IOM website. I highly recommend reviewing that information. A few things stand out to me:
- The sponsor of a study has no control over the study once the “statement of task” and budget are finalized. I have not been able to determine if the SOW I analyze here is the same as the finalized statement of task referred to by IOM.
- Panelists serve without pay.
- Input is gathered in public meetings, but committee deliberations are held in private.
- Select criteria for committees: “The committee must include experts with the specific expertise and experience needed to address the study’s statement of task. . . . Having the right expertise is not sufficient for success. It is also essential to evaluate the overall composition of the committee in terms of different experiences and perspectives.”
At the risk of repeating myself, I will say again that the caliber and prominence of the Institute of Medicine is well-deserved. The conclusions published by the IOM have an extraordinary influence over whatever topic is at issue. My quarrel is with the study defined in the SOW, not with the IOM itself.
While some people may trust that the IOM will select the “right” panel for the ME/CFS clinical case definition study, it is my opinion that this is based to some degree on hope or optimism. There is nothing in the SOW that requires the committee to have a majority comprised of ME/CFS experts. It is entirely possible that IOM will appoint more experts than non-experts, but there is no proof that this will be the case. It could just as easily swing the other way. For people who insist this committee will have the right make up, please show me the data that supports this certainty.
I am very afraid that the IOM will appoint more non-experts than experts to the committee. I can’t prove that I’m right, but the SOW and the IOM process leaves holes big enough to drive a truck through. I am also deeply concerned that the extent of stakeholder input is left to the discretion of the committee, and it’s possible that there will be little opportunity to for us participate. Again, I hope I’m wrong but the SOW does not mandate much stakeholder participation. I also do not have much hope that we can force the cancellation of this study. The government is more than preoccupied with the shutdown and budget woes, and there is scant political will for Congress to act on our behalf.
So for my part, I am focused on two things regarding the IOM: participation in suggesting nominees for the committee, and closely tracking this study to maximize transparency and accountability. I want to hear more from the 35 experts who signed the open letter to Secretary Sebelius. I want to hear more from the experts who did not sign the letter, for any reason. And I want to hear more from my fellow advocates. Pick apart this SOW. Let’s crowdsource analysis and action.