Discussion of the Potti and Nevins papersThe treatments tested by Potti used (only?!) the following series of anti-cancer drugs. I put stars beside the names of the three of these drugs that I myself was treated with a year ago. Every one of these drugs acts (or is believed to act) either by damaging microtubules (and thus interfering with mitosis) or by damaging DNA or interfering with how DNA copies itself.
* means I was treated with them; # means microtubule-binding, plant derived; ^ DNA binding, heart damaging, bacteria derived
Adriamycin^: These 3 are similar antibiotics, inhibit RNA synthesis, damage heart.
Cytoxan* (cyclophosphamide): The one most-used "nitrogen mustard" (I had 9 months of it). Alkylates guanines.
Topotecan: An inhibitor of topoisomerase enzymes.
Etoposide#: Derived from podophyllotoxin, a microtubule poison.
Pemetrexed: A folic acid analog therefore inhibits purine & pyrimidine synthesis
Vinorelbine*#: A "vinca alkaloid", only slightly different from vinblastine + vincristine (which they injected into me)
Paclitaxel#: = "taxol" (the anti-microtubule drug from yew tree bark.
Temozolomide: Forms covalent bonds to guanine; another example of a "nitrogen mustard"
Cisplatin: Cross-links DNA (but by different chemical reactions than mustard chemicals)
The search for new cancer treatments is 99.9% concentrated on chemicals that interfere with either microtubules or with DNA.
Nevins and Potti did not claim to be looking for new treatments; But did 60 Minutes seem to assume that? The following URL will take you to "retraction watch", and Dr. Keith Baggerly's evidence that the data was rigged in Potti's and Nevin's papers: retraction watch
The seven responses are somewhat interesting.
The goal was systematic genomic criteria to choose which of these known drugs would be most effective on a given patient, or which combinations of drugs. E.g. a person who has such and such a genetic variation will survive longest if treated with cyclophosphamide, for example.
Let me interject my opinion that it is pseudo-logical (for everyone!?) to assume that inhibiting an abnormality will selectively kill those cells that have the abnormality. We don't expect diet and exercise to kill over-weight people. Why should we expect cancer cells to be killed by inhibitors of microtubules or DNA synthesis? It's as if we tried to prevent people from speeding and running stop signs and red lights by putting some substance in gasoline that damages automobile engines. Why are people so single-minded and confident in this approach? Why do they do not even seek treatments that kill cancer cells based on any other of their many abnormalities, besides DNA synthesis and mitosis?
But all these drugs used by Potti are based on this pseudo-logic, which have dominated cancer research for 60 years. This dominance is supported by some actual facts. All the anti-growth chemotherapy druges have been available for many years. Several of these drugs have actually cured thousands of people, and delayed mortality in hundreds of thousands of others. Cyclophosphamide and cisplatin are examples. But none are new. The development of every one of these anti-cancer drugs was based on a single premise, which is sixty years old, and has a strangle-hold on cancer funding. I refer to the following pair of semi-true, semi-logical assumptions:
A) That cancer is caused by cells dividing or copying DNA without control ("too fast" many people mistakenly believe), and
B) That cancer cells are killed by preventing their abnormal mitosis or their abnormal DNA synthesis, or by blocking whatever causes their uncontrolled DNA synthesis and/or mitosis (such as excess phosphorylation of proteins, or over-stimulation of receptor molecules).
Cancer cells also have many other abnormalities:
Professor Nevin's plan was to look for correlations between particular DNA sequences (in patients and in tumors) and cure rates produced by each of these long-ago-discovered drugs that interfere with microtubules and DNA synthesis. He bet a lot of money and hundreds of people's lives; I mean lives of researchers (careers and personal dedication) and also the lives of patients who were tested, treated with the drugs, and whose hopes were trifled with.
I suggest that the "War on Cancer" be renamed the "Search for Santa", and that Dr. Potti's crime was that he claimed to have found the Santa Claus for whom so many thousands of researchers were searching.
Naturally, everyone is very disappointed and very angry. It is a major setback in the Search for Santa. It means we will have to go on looking, for as long as it takes, and using ever-improved methods for Santa detection.
But what if the problem is not that Santa is hiding, but that he doesn't exist. That wouldn't make it fraud to have looked for him.
Who did more harm, however, and continues to do more harm? This one MD who falsely claimed to have found proof that Santa exists; or the thousands of researchers and administrators who focused efforts narrowly toward finding what this one MD reported that he really had found?
Notice that his actions were especially harmful to the dozens of co-authors on his research papers, who really did share the work, and justifiably hoped to share the credit. Average people will give them some of the blame. Maybe the logic is that since the co-authors would have shared the credit, if the discoveries had been true, then somehow or other they should share the blame.
Whenever a senior scientist leaves key details to lower ranking, vulnerable-to-pressure young scientists, this policy tends to stimulate the younger scientists to produce whatever results that he knows will make the senior scientist happiest, sometimes to the extent of falsifying data.
The senior scientist shares equally in whatever credit results from the data, but in case of fraud, or suspicion of fraud, almost all the blame goes to the younger scientist.
Imagine this pressure: "Get the results we need to make us famous, or go back to South Dakota".
Also imagine someone noticing something fraudulent in a job application, and deliberately choosing that candidate because he is more likely to be willing to fake data.
Nevertheless, please believe me that 99.99% of research scientists are completely honest. We know that there are hundreds of true discoveries waiting for someone to find, and also that important claims will quickly be tested.
Some questions for this class to think about and be ready to discuss:i) If there had been any possible way to fiddle the data so as to produce any meaningful result, then Potti would have found the way to do it (before publication), or Baggerly, or Nevins would have found some way to get some meaning out of the data.
ii) Notice that they did not claim to be testing any specific hypothesis (such as that people having such an such a genetic peculiarity will survive longer when treated with cyclophosphamide).
iii) Their hypothesis was that SOME such pattern of correlations exists. Finding ANY pattern of correlation would count as a positive result. Absolutely any pattern whatsoever would be equally a success.
iv) Whether or not Potti, Nevins and their collaborators looked closely enough at their data before the accusations of fraud, don't you suppose that they looked really, really, REALLY hard for patterns in the data after the accusations?
v) If they couldn't find any pattern, even under this most extreme imaginable pressure, doesn't that failure amount to very conclusive proof that the whole genomics approach to cancer research can never succeed?
(That is not at all how anybody is interpreting the retractions. They aren't concluding that the holy grail has been proven not to exist. They are only concluding that these particular researchers pretended to find it, but that it is still out there, waiting for somebody to find it.) vi) Do you agree that it would be a huge, important, quasi-Nobel-level discovery if someone proved that this holy grail doesn't exist, cannot possibly be found, and that genomics is a big mistake applied to cancer?
vii) If Potti conclusively proved the impossibility of helping cancer treatment by genomics, why isn't that an even bigger and more important discovery than he and Nevins had hoped for?
viii) Should we nominate the team of Baggerly and Potti to share a Nobel Prize, for their joint discovery that only by fiddling data can correlations be found between genome sequences and effectiveness of particular chemotherapy drugs. Can you deny that the combination of their efforts unintentionally proved that?
ix) Nevins and Potti set up corportations that took out patents on their "discovery". Why don't critics conclude that Nevins and Potti genuinely believed in the truth of what they had published?
x) Many breakthroughs in higher mathematics have been disproofs of the possibility of particular operations (trisecting angles; proving Euclid's fifth postulate based on the other four; solving the general quintic equation; calculating exact values of pi or e using algebraic equations with finite numbers of terms; and Godel's proof of the impossibility of proving internal consistency of Whitehead and Russell's system of symbolic logic. Wow: prove that you can't prove that you can prove. My brain is over-heating!)
xi) Biologists are not used to this sort of thing. An exception is the "diffraction limit" of maximum resolution with microscopes, depending on the wave-length of the light and the angle of illumination.
xii) What other experimental impossibilities can we imagine might become breakthrough concepts in the future? Perhaps differentiation of N differentiated cell types is only possible by using at least N squared different genes for transcription factors, and at least N cubed different promoter sequences. (Why should mathematicians have all the fun deducing impossibilities?) If 15 co-authors accused of fraud can't find any consistent pattern to their data, doesn't that prove something?
xiii) Are Genomics scientists going to seek evidence proving which kinds of phenomena their methods can't possibly explain?
xiv) Why was a Nobel Prize given for phase contrast microscopy, but not for Ernst Abbe's diffraction limit? (Maybe because Abbe earned more money that Nobel!)
xv) The television exposé describes patients being told that this research program could save them. How likely is that, if the key data were survival times? "Once we find out when you die, that will give us information that will help save other people's lives, or help them survive longer and with less degradation." THAT might actually be true. xvi) Read some of the retraction statements that are appended to the on-line versions of the retracted papers. Do these sound as if written by a committee of administrators and lawyers, rather than repentant scientists?
Links:a few papers from Nevins' group pre-dating Potti:
West et al. (2001) Predicting the clinical status of human breast cancer by using gene expression profiles. PNAS 98(20), 11462-7. [TEN Authors; this is before Dr. Potti arrived. And already Nevins had other younger scientists doing the same kind of research; seeking the same "holy grail"]
Huang et al. (2003) Gene expression phenotypes of oncogenic signaling pathways. Cell Cycle. 2(5):415-7.
Pittman et al. (2004) Integrated modeling of clinical and gene expression information for personalized prediction of disease outcomes. PNAS 101(22):8431-6.
West et al. (2006) Embracing the complexity of genomic data for personalized medicine. Genome Res. 16(5):559-66.
two key Potti et al. papers from 2006:
Potti et al. (2006) Genomic signatures to guide the use of chemotherapeutics. Nat. Med. 12(11):1294-300.
criticism of Potti et al. by Coombes and Baggerly:
Coombes et al. (2007) Microarrays: retracing steps. Nat Med. 13(11):1276-7. [This is followed by a brief reply by Potti et al.]
Baggerly and Coombes (2009) Deriving chemosensitivity from cell lines: forensic bioinformatics and reproducible research in high-throughput biology. Ann. Appl. Statistics 3(4):1309–1334. [This is the detailed analysis of the Potti et al. (2006) paper.]
back to index page