The Holy Grail
Many CFS patients are holding their breath awaiting the results of the multiple lab XMRV study, nicknamed the Lipkin study after its coordinator Dr. Ian Lipkin of the Center for Infection and Immunity. I am not one of them. I don’t usually judge a study before the results are in, but this research has been a roller coaster ride since it was first announced. I’ve collected all the public statements I could find about the study, mostly because I couldn’t find such a chronology anywhere else. I think it’s instructive to view these comments along with the evolving XMRV science in order to understand where the study is now, and what reasonable expectations we might have for its results.
In October 2009, researchers from the Whittemore Peterson Institute and collaborators published a paper in Science reporting their detection of a retrovirus, XMRV, in 67% of CFS patient samples and 3.7% of healthy control samples. To say this paper (Lombardi et al.) unleashed a firestorm would be an understatement. I won’t review all the circumstances here, but this article is a place to start for background.
Right Thing, Right Time
Almost one year later, in September 2010, the National Institutes of Health hosted its First International Workshop on XMRV. No research group had replicated the XMRV findings during that year, including the Centers for Disease Control. However, an FDA study found MLV (mouse leukemia virus) sequences in a small CFS cohort (Lo et al.). The conflicting results between CDC and FDA had caused its own drama created earlier in the summer.
Dr. Francis Collins (Director of NIH) announced at the XMRV workshop that he had asked Dr. Anthony Fauci (Director of the National Institute of Allergy and Infectious Diseases) to coordinate a multi-lab study to sort out the mess. Dr. Fauci, in turn, appointed Dr. Ian Lipkin of Columbia University to run the study. Initial reports stated that the study would involve fresh blood samples from 100 CFS patients and 100 healthy controls drawn from multiple locations in the United States. Dr. Lipkin’s lab would receive the samples, blind and split them, and then labs at the CDC, FDA and the Whittemore Peterson Institute would test the samples for the presence of XMRV using their own assays.
I believe that Dr. Collins did the right thing in September 2010. In the CFS world, XMRV was a highly charged topic. The online equivalent of bar fights seemed to break out almost daily. The scientists on both sides of the issue were not behaving much better. Early news of the Lo et al. paper had created an expectation that it would replicate Lombardi et al., and so the controversy was only fueled by its publication in August reporting not XMRV, but MLV instead. For NIH to step in with some money for a blinded study with well-characterized samples was exactly what was needed. The study was widely viewed as most likely to provide the definitive answer on whether XMRV was or was not associated with CFS. Personally, I felt everyone could take a deep breath and wait for the results, which were initially expected in mid-2011.
It All Falls Apart
But it wasn’t long before the XMRV theory took on the appearance of a teetering house of cards. At every step, Dr. Lipkin responded to questions about whether the study should continue and why it was necessary.
In December 2010, a paper by Hue et al. (pdf) used phylogenetic analysis to show that XMRV had acquired sequence diversity in a human cell line but not in human samples, a finding strongly suggestive of contamination in Lombardi et al. The study authors pronounced their findings as the end of XMRV as a human pathogen, but others were not so convinced. Dr. Vincent Racaniello of Columbia University even went so far as to retract his initial statements about the studies and stated that research should continue. Dr. Lipkin told Nature (as reprinted by the CFIDS Association):
These papers emphasize the pitfalls of molecular assays and raise concerns. Nonetheless, it is premature to rule out XMRV or related viruses as factors in prostate cancer or CFS. Links have also been made based on serology and the presence of viral proteins as well as of viral sequences. Thus, we still need appropriately powered, rigorous blinded studies of well characterized patients and controls. One such study is underway under the auspices of the National Institutes of Health.
In March 2011, data was presented at the Conference on Retroviruses and Opportunistic Infections that XMRV was actually the result of a recombination event that happened during the serial passage of a human tumor cell line through mice. Dr. Robert Silverman, a co-author on the October 2009 Science paper, told the Chicago Tribune that he was concerned about contamination in his lab. Also in March, more details emerged about the Lipkin study, now reported to involve 150 patients and 150 controls at a total cost of $1.3 million.
The NIH held a State of the Knowledge Workshop on CFS in April 2011, and included a session on XMRV chaired by Dr. Harvey Alter (a co-author on Lo et al.). Dr. Alter stated his opinion that the Lipkin study would be definitive on the issue. Dr. Judy Mikovits, in addressing the urgent needs of patients for treatment, said that patients couldn’t wait until “when the Lipkin study’s done two years from now.” She made reference to the study design requiring her to test “1300” samples and that each one takes “2 months.”
In May 2011, XMRV researcher Dr. Ila Singh reported no XMRV in CFS patients. The paper included 14 CFS patients identified by the Whittemore Peterson Institute as repeatedly testing positive for XMRV, but Singh found none. Dr. Racaniello published the following statement from Dr. Lipkin:
We have a plethora of explanations for how CFS/XMRV/MLV studies could go awry. However, we don’t have evidence that they have. Absent an appropriately powered study representing blinded analyses by Mikovitz and Lo/Alter of samples from well characterized subjects using their reagents, protocols and people, all we have is more confusion.
I remain agnostic. We won’t have answers until the end of 2011.
So between September 2010 and May 2011, the study had expanded from 200 total samples to 300, with a reported cost of $1.3 million. Results had been promised in mid-2011, but now were pushed back to the end of the year. During these eight months, data suggesting XMRV was a contaminant continued to pile up and still no group had confirmed the Lombardi or the Lo papers.
Later in May 2011, Science published the recombination data presented at CROI in March along with a study by Dr. Jay Levy reporting that 61 CFS patients tested negative for XMRV, including 43 who had previously tested positive by WPI. Science published an Editorial Expression of Concern (pdf) along with these papers, saying that the validity of the original study “is now seriously in question.” Apparently, Science had asked the study authors to voluntarily retract the paper, but they refused. Despite all this, NIH continued with its plans for the Lipkin study. Lipkin told the Washington Post that the study would continue. He was also quoted by the Wall Street Journal as saying: “We need to see whether [the Whittemore Peterson institute] can replicate their own work with blinded samples. I think they should have the opportunity to do that.”
Organizations like the CFIDS Association and CFS patients alike stated their intentions to await the Lipkin results. In early September, the CFIDS Association reported that the results of the Lipkin study were now delayed to the beginning of 2012. The cause of all the delay was not discussed publicly to my knowledge, but the study was now running more than 6 months behind schedule. This is not unusual in science, but this study was designed to put an end to the controversy. The results were desperately needed, at least by the patient community. Every time someone expressed doubts about the XMRV theory, a chorus of voices was raised in support of waiting for the Lipkin results. The study would be the gold standard, and it was unfair to draw conclusions before it was finished.
Science published two more papers in September 2011 that really seemed to be the death knell of XMRV in CFS. The Blood Working Group, formed to investigate whether XMRV posed a threat to the blood supply, published results of testing by nine separate labs of 15 patient samples reported to be XMRV or MLV positive in previous studies. Bottom line – only WPI and Dr. Frank Ruscetti’s lab at NCI detected any positives, and their results did not agree. In addition, Dr. Silverman officially retracted his data from the Lombardi paper. Behind the scenes shenanigans around the retraction were summarized by Retraction Watch.
Much drama ensued in the following months, including allegations that Dr. Judy Mikovits (no longer with WPI) wanted to stop the Lipkin study. With the end of the Blood Working Group, some patients began to move on from the XMRV hypothesis. Others focused on the Lipkin study as the last possible hope to confirm the theory. And others seemed to become entrenched in the belief that the Lipkin study would prove nothing, and therefore should be stopped because its results would do more harm than good.
Just before Christmas 2011, Science retracted the Lombardi paper in full followed by the full retraction of the Lo study by its authors on December 27th. Despite the retractions, Dr. Lipkin told the New York Times that the retraction was premature “while we sort out what’s real and not real.” Dr. Mikovits was quoted in the same article as saying that the Lipkin study would be the definitive answer. In contrast, Bruce Alberts (the editor of Science) told the Washington Post that there was no reason for the Lipkin study to continue. (the Post reports the cost of the study as $2.3 million but I have not seen that figure elsewhere) Dr. Racaniello described the study as “less compelling” and unlikely to clarify things. Finally, on December 28th, Dr. Lipkin posted a statement on his website saying that the study would continue. He further stated that results were expected in early 2012, with 123/150 CFS samples and 88/150 control samples already distributed for analysis.
Where Are They Now?
It is now March 2012, eighteen months after this study was announced by Dr. Collins. Why are we still waiting for results? The study criteria were settled in November 2010. But more than a year later, not all the samples had been collected. There have undoubtedly been complications, including Dr. Mikovits’ departure from WPI, and we know that patient recruitment can take time. But does it take more than a year to collect 300 samples? I asked Dr. Lipkin by email on February 28th when study results could be expected and he said “This depends on when the labs complete their work.” This seems to indicate that the labs are not finished. Even when the three labs have completed their testing, the results have to be unblinded by Dr. Lipkin and then analyzed. My conjecture is that we will be waiting at least several more months for the results of this study.
There has also been some confusion over whether the samples collected for this study will be used for any other research. In his December 28, 2011 statement, Dr. Lipkin stated:
please be assured that more than 85% of the funding associated with this initiative is invested in patient recruitment and characterization and sample collection, archiving, and distribution. Thus, irrespective of study outcome there will be unprecedented opportunity to explore hypotheses other than that disease is due to XMRV or MLV infection.
The CFIDS Association reported in January 2012 that the samples from the Lipkin study would be evaluated for other pathogens by Lipkin’s team and that this work would be funded by CFI. However, in a comment on that post, Dr. Nancy Klimas stated “In the end there are no samples left at Columbia for future studies.” According to Dr. Klimas, the CFI-funded study collected separate samples from the same patients, to be used in the separate CFI study. I asked Dr. Lipkin for clarification about the use of the XMRV/MLV study samples and he said, “The samples will be banked for future work. I don’t have funding for this work. Any help toward securing it would be deeply appreciated.” This seems to conflict with what Dr. Klimas posted on the CFIDS Association blog.
Why is this important? The XMRV/MLV study is quite expensive by CFS research standards: $1 million in NIH funding is supporting the research. Is that money supporting the collection of well-characterized samples that can be used for other purposes? That would be wonderful. Is CFI going to tack its pathogen discovery effort on to this federally funded work? The IRB approvals and patient consent forms for collecting the samples would have to be clear on what was being done with them. I’m not sure why there are contradicting versions of what the samples will be used for, since that should have been settled months ago.
There is also the issue of where Dr. Mikovits will do the work planned in this study. Dr. Lipkin’s December 2011 statement said that he believed Dr. Mikovits, not WPI, should participate in the study, and that a way had been found to fully engage her. According to Caroline T. Anderson’s blog, Dr. Lipkin worked with Drs. Mikovits and Ruscetti to find a lab where they could do the work. He told Nature that:
“[The WPI is] no longer involved because the whole point was to have Mikovits try to reproduce her work, and having someone else at the institute do so wouldn’t address the questions,” Lipkin says. “It’s critical that she do the work. She doesn’t have a lab at present, so it’s going to be done at NCI.”
When I asked him to confirm this arrangement, Dr. Lipkin referred me to Drs. Mikovits and Ruscetti. I contacted both by email, and Dr. Mikovits responded that she is unable to answer any questions at the present time due to her legal situation. This is quite understandable. I have not received a response from Dr. Ruscetti.
Don’t Hold Your Breath
Even if all these questions about timing, samples and location are sorted out, I don’t think the patient community should be holding its breath in anticipation of the results. I have no problem with the study design itself. A blinded, multi-site study looking for XMRV/MLV in well-characterized CFS patients was exactly the study we needed in 2010. And if the study could have been completed on schedule by early 2011, it would likely have been a contribution to the discourse about XMRV and CFS. But now? Now the study as designed is completely unnecessary.
First there is the cognitive dissonance caused by NIH’s conflicting positions on the study. On the one hand, NIH is investing approximately $1 million in this study. For CFS, that is a significant investment of resources. But on the other hand, NIH says on its own website that:
The body of evidence now indicates that XMRV does not play a role in ME/CFS. . . . . Although it appears that XMRV is a contaminant and not associated with human disease, NIH continues to fund studies to determine whether results of this study can be replicated and the lack of association between XMRV and ME/CFS confirmed.
Why continue to spend money on looking for evidence of something that the funding institution has stated is a contaminant?
Second, there are the contradictory positions taken by some participants in the study. In retracting their paper, Lo et al. said “it is our current view that the association of murine gamma retroviruses with CFS has not withstood the test of time or of independent verification and that this association is now tenuous.” Lo et al. were unable to replicate their own findings on their own samples. They have retracted their data. How does it make sense to participate in the Lipkin study to test more samples looking for retroviruses that they do not believe are associated with CFS?
Third, there are the retractions. This is the deal breaker for me. Both Lombardi and Lo have been completely retracted. In other words, that data is gone. Retraction is not like divorce, where the parties agree the marriage is over. Retraction is annulment, where the marriage is considered to have never happened at all. It’s like rewinding the clock to September 2009 before Lombardi was published. There is no published evidence linking XMRV or MLVs to CFS. None. The Lipkin study was designed to clear up the conflicting data between Lombardi and Lo on one side and everyone else on the other. But now, there is only one side. There are dozens of papers that show no association between XMRV and CFS. Three papers published in Science retested some Lombardi and/or Lo samples and found them negative. And the original papers are gone – whoosh – evaporated. There is no data to support the hypothesis. There is no conflict in the published data now. There is nothing to verify.
If NIH wants to fund some sort of million dollar pathogen study in well-characterized CFS samples, I am fully in support. But shouldn’t such a study go through the usual peer review process? The Lipkin funding did not. It was a special allocation because of the controversy in the field. The NIH typically spends $6 million per year on CFS research. This study represents an allocation of almost 17% of the annual budget to a study that did not go through peer review, and that is trying to verify a hypothesis that has no supporting data. I don’t think this is the right precedent to set in CFS research.
Dr. Lipkin said in December that “85% of the funding associated with this initiative is invested in patient recruitment and characterization and sample collection, archiving, and distribution.” Great, but don’t waste that money! $850,000 worth of patient recruitment and sample collection is a treasure trove in CFS research. Correctly identifying and recruiting CFS cohorts is very expensive, and a barrier to many researchers. Why not save all the samples for a study to test a valid hypothesis? Why not create an NIH repository of CFS samples? Using up the samples on XMRV tests is almost like pouring the plasma down the drain. It won’t tell us anything. The Blood Working Group study already established that no lab (including NCI and WPI) had assays that could reliably detect XMRV/MLV in blood.
In his email to me, Dr. Lipkin said “we are doing our best under difficult circumstances to bring the best science to CFS.” Of that I have no doubt. Throughout his entire involvement in the study, Dr. Lipkin has been consistent in his statements. He expanded the size of the study, recruited the best CFS clinicians to identify patients, and secured consensus on how CFS would be defined in the study. He has a tremendous track record in controversial pathogen investigations. I am not questioning the caliber of his science.
I question whether the study is the best use of one million extremely scarce and desperately needed CFS research dollars. I question whether the study results will be accepted by all the parties. Mixed results will leave a tiny toehold for the XMRV believers to insist on their theory, and even fully negative results will still not satisfy those who have adopted the HGRV theory. Positive results would cause quite a quandary, given the conviction among most researchers that XMRV is not a human pathogen. I question whether we can learn anything of value from the study at this point in the scientific process. This study is not the holy grail that some have made it out to be, if for no other reason than that the results are unlikely to change anyone’s mind or provide any new answers.