Medical Statistics www.improvingmedicalstatistics.com
and Interpretation of Clinical Trials
Columbia University 'Miracle' Story
(Following the details of this saga is commentary by Dr. Bruce Flamm)
A Story of Possible Fraud in the Medical Literature and a Medical Journal and Medical School Responding Poorly
A study was published in the Journal of Reproductive Medicine in 2001, which if true, was a landmark study. The study reported that women undergoing in vitro fertilization in Korea who were prayed for by individuals in Australia, Canada, and the United States conceived at twice the rate as those patients that did not receive prayer. The prayer groups were reportedly organized predominantly by one of the authors, presumably by the currently imprisoned lawyer, Daniel Wirth. None of the patients who were prayed for were told they were in a study or that they were being prayed for by other people. No informed consent was obtained.
The prayer group was said to have a remarkable increase in the rate of achieving a successful pregnancy- 50% in the prayer group (44/88) vs. 26% in the no prayer group (21/81, P=.0013). If legitimate, this would have been an historic publication. In reality, history will record this publication as a classic example of flawed and possibly fraudulent medical research.
The lead author, whose name was eventually withdrawn from the study, was Dr. Rogerio Lobo. Dr. Lobo was the chairman of the Obstetrics and Gynecology Department at Columbia University in New York at the time. He was also on the editorial board of the Journal of Reproductive Medicine, the same medical journal that published this study.
The other two authors were Daniel Wirth, a lawyer, and Dr. Kwang Cha. Dr. Cha was director of the Cha Columbia Infertility Center at the time of the 'miracle' study but appears to have ended his relationship with Columbia University shortly after the study was published.
Daniel Wirth is a lawyer without a medical degree but with a degree in parapsychology (a field dealing with ESP and other unproven phenomena). Mr. Wirth had previously written a number of articles published in parapsychology journals claiming to document paranormal phenomena and faith healing. (Only later would Daniel Wirth be imprisoned after being convicted of unrelated fraud charges which included the use of the names of dead people for illegal financial gain.)
Dr. Bruce Flamm, who is a clinical professor of OB/Gyn at University of California at Irvine, wrote several letters to the Journal of Reproductive Medicine. He questioned the complex design of the trial. He also asked for the name to be released of the supposed independent statistician who was said to have been an outside source helping to guarantee the integrity of the data. Dr. Flamm, in addition, suggested the possibility of fabrication. Dr. Flamm’s subsequent phone calls to the journal were not answered nor were his letters published in the Journal of Reproductive Medicine. Other individuals also wrote letters to the journal questioning the study's methodology and results. None of these letters were published, as would have been appropriate near the time of publication of this presumably landmark trial.
Dr. Flamm went on to write a critique of the study in the Scientific Review of Alternative Medicine in 2002. Other authors subsequently contacted Dr. Flamm about multiple independent inquiries into other questionable work by Daniel Wirth, the lawyer without a medical degree.
In the meantime, the lack of informed consent resulted in an investigation of Columbia University (Dr. Lobo’s institution) by the United States Department of Health and Human Services (DHSS). Dr. Lobo was listed as the lead author on the paper, as well as being identified by a Columbia University press release as having led the study. However, Thomas Morris, Columbia University Vice President, responded to the DHSS investigation that Dr. Lobo had first learned of the study from Dr. Cha six to twelve months after the study had been completed and had primarily provided editorial review and assistance with publication. Dr. Lobo would later withdraw his name from the study ostensibly because of this limited relationship to the study.
When the scandal surrounding the 'miracle' study became public in mid 2004, Columbia University, which is a highly respected institution, began an investigation. However, the investigation was apparently halted when Dr. Lobo formally removed his name from the study. Details of this investigation were not made public. The lack of public reporting of the details of their investigation of this study could be viewed as shirking of responsibility by Columbia University. This trial would not have received the same credence without the association of the study with Dr. Lobo and Columbia University. Columbia University had even initially issued a press release claiming that the study had several safeguards in place to eliminate bias and that the study was carefully designed to eliminate bias.
Both physicians listed as authors at the time of publication of this study were Columbia University physicians. Without these two physicians associated with Columbia University, the only author remaining was a lawyer without a medical degree with a history of prior publications on mysterious phenomena and faith healing. Any journal even considering publication would have demanded more documentation of the integrity and factual nature of the data presented. Columbia University’s response after problems with this study have become apparent has been clearly insufficient.
In addition, the actions of the Journal of Reproductive Medicine (JRM) have been indefensible for a medical journal. Prior to publication, their peer-review system failed to detect a flawed study. After publication, the journal had to be aware of the considerable controversy that this article provoked. The journal did not publish any letter to the editor or viewpoint criticizing this study even when they temporarily withdrew the paper from their website. (The journal withdrew the paper following the May 2004 publication in the London Observer of “Exposed: Conman’s Role in Prayer-power IVF “Miracle”.) In November 2004, they put the study back on their web site, reportedly telling the press that they stood by the report. The Journal of Reproductive Medicine at the same time published a defense of the article by the sole remaining unincarcerated author of the study, Dr. Cha.
Dr. Cha noted that his defense of the article was written as a “reply… to correspondence we received.” Almost unbelievably for a medical journal, the rebuttal by Cha was printed by the Journal of Reproductive Medicine without that journal ever previously or concurrently publishing any letters or commentary they received regarding criticism and limitations of this study.
After continued criticism in the national news media, the Journal of Reproductive Medicine belatedly published a letter from Dr. Flamm in their January 2005 issue. This was more than 3 years after the initial publication of the 'miracle' study.
What should now be done?
Columbia University should reopen an inquiry into the publication of this article and make public the information gathered from their previous inquiry. Without the Columbia University connection, this paper would not have been published in a similar fashion or received the undeserved credence that connection conferred. The paper is now back in the public domain on the Journal of Reproductive Medicine’s internet site and would never have been there in the first place without the Columbia University connection. Columbia University shares responsibility for this flawed and possibly fraudulent study being in the public domain.
The Journal of Reproductive Medicine also has a major responsibility in rectifying this situation. An appropriate response for the Journal of Reproductive Medicine would be to now commission an outside investigation of the article and their own journal’s handling of this matter by impartial and respected medical experts. In the meantime, this study, whose data analysis and data collection involved an imprisoned felon, should be withdrawn from the medical literature. A good rule of thumb for a medical journal is that anyone who uses the names of dead children in order to fraudulently obtain bank loans, jobs, and passports is not a reliable source of data.
E Roehm Jan 2005
What follows the above synopsis of the Columbia 'Miracle Study are comments by Dr. Bruce Flamm on the multiple valuable lessons to be learned from this unusual sequence of events.
The Columbia "Miracle" Study
plethora of lessons about medical research, evidence-based medicine, and the
peer-review system are examined.
Bruce L. Flamm, MD, Clinical Professor
Obstetrics & Gynecology
University of California, Irvine
A study led by Columbia University Medical School faculty appeared to demonstrate miraculous results. In reality, this seemingly impeccable study demonstrates a wide variety of research flaws and peer-review errors. One of the three authors is now serving five years in Federal prison for criminal fraud, another has formally removed his name from the study, and the final author refuses to respond to questions about the scandal.
The Cha/Wirth/Lobo study claimed to demonstrate, using meticulous scientific methodology, that supernatural or paranormal phenomena actually exist. (1) These mysterious phenomena apparently caused a 100% increase in the success rate of complex infertility treatments. How did a bizarre study claiming extraordinarily unlikely and apparently supernatural results end up in a peer-reviewed medical journal?
The following are some of the many important lessons that can be learned from this incredible research saga:
1. Always be suspicious of outrageous claims. The Columbia 'miracle' study is an excellent example of what Dr. C.N. Reckens, chairman of the Dutch Union Against Quackery calls a, “seemingly impeccable paper proving absurd claims.” (2) A related phrase cherished by most scientists is, “extraordinary claims demand extraordinary evidence.” Thus, the first lesson this paper teaches us is that we should be highly suspicious of studies that appear to show astonishing results. Of course, every once in a while someone will discover something that is truly astonishing! Nevertheless, such results wave a red flag indicating extreme caution should be taken as the study is reviewed.
2. One red flag should prompt a search for others. The extraordinary claims of the Cha/Wirth/Lobo study should have prompted reviewers to look for other red flags. In this case the peer review system completely failed. For example, an obvious red flag was that one of the authors, Daniel Wirth, had no medical degree but rather a degree in para-psychology, a dubious field that deals with ghosts and supposed psychic phenomena. A five-minute Google search of this author yields dozens of red flags that researchers at Columbia University as well as the peer-reviewers and editors at the Journal of Reproductive Medicine obviously missed. (3) If anyone had taken a moment to look, they would have found that Mr. Wirth had already published many papers claiming bizarre healing phenomena in paranormal magazines and alternative medicine journals.
3. An overly complex study design is a red flag. The Cha/Wirth/Lobo study involved a complex and convoluted study design involving various groups, levels, and tiers of overlapping and intertwining intervention groups. Multiple glaring red flags in the Cha/Wirth/Lobo manuscript should have caused reviewers to at least glance at prior publications by Daniel Wirth, the author with no medical credentials. This would have revealed a long pattern of similarly unusual study designs and bizarre healing methods. For example, one prior Wirth study involved only 15 patients yet evaluated some eight different interventions in various groups and combinations. (4) The strange study included LeShan “elevated state of consciousness” healers, Reiki “life force transferring” healers, Intercessory Prayer healers, Non-contact therapeutic touch “energy field altering” healers, Guided imagery, biofeedback, visualization and relaxation techniques. There were almost as many study interventions as study participants! For comparison, imagine a study with only 15 study patients that claimed to evaluate the safety and efficacy of eight different investigational drugs!
4. Lack of informed consent is a red flag. The Cha/Wirth/Lobo study clearly states that patients undergoing treatment were not aware of the fact that they were being used as study subjects. This was a shocking revelation, particularly in light of current governmental laws governing research and strict HIPAA regulations. One would think that this would give reviewers a good reason to look very carefully at the study design and methodology. However, reviewers at Columbia University along with peer-reviewers and editors of the Journal of Reproductive Medicine were apparently not alarmed by this information.
5. Investigation of a study by the Federal Government is a red flag. The Columbia University press release announcing the publication of this study mentioned the fact that study patients did not know they were taking part in a study. This fact was subsequently mentioned in the New York Times in an article that caught the attention of the U.S. DHHS Office of Human Subjects Protection. This prompted a Federal investigation of Columbia University. Limited information about this investigation has been made public and can be viewed at the DHHS Internet site given below. During the investigation Columbia department chairman Dr. Rogerio Lobo, who had been listed in the Columbia University press release and the New York Times as the study's lead author, claimed to have not been involved with the study until 6 to 12 months after its completion. Three years later Dr. Lobo formally removed his name from the study. Concerns about a study that are sufficient to spark a Federal investigation should clearly have been sufficient to concern peer reviewers and editors. However this red flag that was ignored.
6. If the claimed results are not only astonishing but also defy the know laws of physics then we are faced with the reddest of all red flags. Before such a study is accepted for publication it would be wise to await replication by an independent research group. A fundamental rule of scientific research is that a valid study can always be replicated by other researchers. Conversely, if other research groups are not able to replicate a study then it is almost certain that the original results were not valid.
7. Letters to the Editor serve an important peer-review role. In some cases even the most cautious peer-reviewers and editors may miss important errors in a manuscript. Letters to the editor often correct such problems. In this case peer-reviewers and editors at the Journal of Reproductive Medicine missed an entire array of red flags. Worse yet, they undermined evidence-based medicine's system of checks and balances by refusing to publish even a single letter critical of the study. For three years letters from concerned physicians and scientists were completely ignored by the Journal of Reproductive Medicine. The JRM editors' stonewalling serves as an excellent example of exactly how editors of medical journals should never behave. At the other end of the spectrum, the British Medical Journal publishes rapid responses from readers on their Internet site within 24 hours of their submission. Selected letters are then published in the paper version of the British Medical Journal. This open and public peer review system serves as an excellent example of how editors of medical journals should behave.
8. Studies claiming supernatural results may defeat peer-review systems. The Cha/Wirth/Lobo study involved spectacular outcomes apparently related to distant Christian prayers. Religion is a very sensitive subject and may create a serious 'blind spot' in evidence-based peer-review systems. Some journal peer-reviewers may be unable to function objectively when faced with religious claims. If psychic healers or fortune tellers had claimed to have doubled the success rate of infertility treatments by utilizing Taro cards or Ouija boards their manuscript would have been immediately rejected as utter nonsense by any legitimate medical journal. Yet, the apparently supernatural results of the Cha/Wirth/Lobo study were accepted and published by a supposedly evidence-based peer-reviewed medical journal. Why?
9. Once a flawed medical study is published in a peer-reviewed medical journal, the damage has already been done. In the winter of 2001, newspapers and magazines around the world announced the astounding results of the Columbia 'miracle' study. By June of 2004, a Google search for the terms, “Wirth, Columbia, prayer” revealed more than 600 links, many to sites touting the supposedly miraculous results. For almost three years the public had no knowledge of the study's serious flaws. During those years the study was cited many times, even in peer-reviewed medical journals, as strong scientific evidence for the power of faith healing.
10. The peer-review system can continue to fail even when serious flaws and problems are made public. As of January, 2005, in spite of all the flaws and problems delineated above, Columbia University apparently stands by the 'miracle' study and has concluded its investigation of the matter. The publication was briefly removed from the Journal of Reproductive Medicine Internet site but has now been reinstated and can be viewed at the JRM Internet site at the link given below. Amazingly, after ignoring readers concerns about the study for three years, the JRM published a letter in the October 2004 issue in which co-author Kwang Cha was allowed to defend the bizarre study. However, as of January 2005, the editors of the JRM have refused to publish even a single letter critical of the Cha/Wirth/Lobo study.
Internet Links for more information about the Columbia 'Miracle” study
Scientific Review of Alternative Medicine on-line articles:
The Complete Cha/Wirth/Lobo 'miracle' study:
The Columbia University Miracle Study:Flawed and Fraud:
The Federal Indictment of Daniel Wirth:
Time Magazine July 2004:
Time Magazine December 2004:
Chronicle of Higher Education June 2004:
DHHS Investigation of Columbia University and Cha/Wirth/Lobo Study
1. Cha KY, Wirth DP, Lobo RA. Does Prayer Influence the Success of in Vitro Fertilization-Embryo Transfer: Report of a Masked, Randomized Trial. Journal of Reproductive Medicine. 2001;46:781–787
2. Renckens CN. Alternative treatments in reproductive medicine: Much ado about nothing. Human reproduction 2002;17:528-533
3. Flamm BL. Faith healing by prayer: Review of Cha, KY, Wirth, DP, Lobo, RA. Does prayer influence the success of in vitro fertilization-embryo transfer? The Scientific Review of Alternative Medicine. 2002; 6(1):47-50 Available on-line at the site listed above.
4. Complementary Healing Therapies. Wirth DP, Barrett, MA. International Journal of Psychosomatics. 1994:41;1-67.
5. Flamm BL. Faith Healing Confronts Modern Medicine. The Scientific Review of Alternative Medicine. 2004; 8(1):9-14. Available on-line at the site listed above.