(www.improvingmedicalstatistics.com)                                          
      home   author    contact   web site dedication
  

......................m.........................

Specific guide to this web site for:

 1.  Medical School
      Educators 
      in Statistics

 2.  Medical Students

 3.  Science media writers

 4.  High School & College
     Statistic Teachers

   Misadventures:

1. Harvard led MI study

2. JACC study 
   (J. of Amer. Coll.
   Cardio.)

3. NEJM cath study

4. Amer. J. of Cardio.
    review of literature

5. ALLHAT
    controversy 

6. Oat bran study

7. Pregnancy & Alcohol

8. Are Geminis really
    different?
      
9. Columbia 'Miracle' Study  
                                                 

Additional Topics:

Celebrex

Limitations of Meta-Analyses

Large Randomized Clinical Trials

Tale of Two Large
Trials

Advocate meta-analyses

Network meta-analyses

Web Site Entry For Medical School Educators In Medical Statistic

   The task of educating medical students in the critical appraisal skills required for properly evaluating clinical studies is a large one.

   This web site has resource material that may be useful in helping medical students recognize poorly formulated statistics or inappropriately drawn conclusions in medical studies. 

     A number of medical studies with significant problems in either clinical trial design, medical statistics, or properly formulated conclusions are examined in detail. There will be differing viewpoints from those on this website regarding these case studies, but it is hoped those differences will provide useful starting points for discussion.

  This web site besides being a collection of suboptimally performed or interpreted medical studies, also reviews some of the strengths and weaknesses of meta-analyses and large randomized clinical trials. 

  The information on this web site which has been created by the author can be used for the instruction of medical students and residents without restriction as long as the source is documented.  It is hoped that this site will prove a useful resource.

 The following is a summary of the content of this web site in regards to usefulness for  educators of medical students:

Example 1:    This is probably not particularly useful for medical students. This has to do with a randomized clinical trial with an inherently biased protocol.  It does provide an example where a suboptimal treatment protocol used in a clinical trial is a crucial factor in regards to the implications of that trial for clinical practice.
A trial led by Harvard Medical School faculty with a protocol so biased (unintentionally) that the trial was inherently was an invalid comparison.

 Example 2: This would be a beneficial case study for medical students. An incredibly poorly conceived statistical analysis in a major medical journal.   A statistical analysis in a major medical journal JACC (Journal of American College of Cardiology) which was so bad, that it may have  led to a subsequent improvement in the review process of JACC.

Example 3: This study is an example of a trial's conclusions being the opposite of what the trial data suggests.  This study would be useful for discussion.  A potential for a type 2 statistical error is also illustrated as well.  An example of a study in the New England Journal of Medicine where the conclusions run exactly the opposite of what the data suggests.  

 Example 4:     It is probably not worth the time of the students to fully understand the details. Though there is some potential interest in understanding how a poorly done review of literature can make the conclusions of a trial completely incorrect. Understanding this example may be fairly tedious.  How an incorrectly done review of the literature in a single article in the medical literature could potentially adversely affect patient care.

  Example 5:     This ALLHAT trial critiqued would be of  interest only to those with special interest in clinical trials that affect current hypertension guidelines.  It is unclear whether this example would be conducive to medical student education. The ALLHAT trial- how conclusions that extend beyond data can provoke controversy
   (ALLHAT Trial Critique download- pdf format)

 Example 6:     This is a classic type II statistical error with conclusions that do not recognize the limitations of the trial protocol nor the weakness resulting from the small numbers of subjects involved.  Death of the oat bran fad.  Death of the oat bran fad.  (Murdered by a poorly conceived study.)

 Example 7:     The topic of alcohol and pregnancy may be of potential interest.  The study provides a starting point for discussion of 1) the strengths and weaknesses of descriptive rather than randomized studies,  2) limitations of epidemiologic derived information that is not yet confirmed by randomized trials, 3) and consideration of whether the results of a multiple regression analysis can always be considered the last word on a subject. Pregnancy and alcohol, what do some of the studies really suggest?

    Example 8:    This consists of comments by Dr. Peter Sleight in an insightful article regarding the hazards of inappropriate subgroup analysis. The Hazards of Subgroup Analysis  (Are Gemini's really different?)

    Example 9:  The unfortunate saga of possible fraud in a medical study with a medical journal and a medical school failing to respond appropriately.  The Columbia University 'Miracle' Study-

    Example 10:  These topics may be useful as starting points for discussion with medical students. 

The Very Large Randomized Clinical Trials (Strengths and  Limitations)
    Some strengths and limitations of large randomized clinical trials are considered.
A Tale of Two Large Trials  
    A very well done large randomized trial is compared to a poorly interpreted large clinical trial. Both these trials have and will affect clinical practice guidelines.
 Limitations of Meta-Analyses
    Often a meta-analysis is popularly thought to be the final word concerning a medical topic. Some of the potential limitations of meta-analyses are discussed
Network meta-analyses, additional potential for error
      A type of meta-analysis which is more subject to error    

    Example 11    How medical science advances when there are problems in the scientific literature.  (How these particular issues might look "10 years later")  This discussion of the self correcting aspects of the medical literature may be of interest.

     Celebrex    This may be useful as a starting point for discussion with medical students regarding a multitude of topics. One issue, is that of the reliability of cross trial comparison (particularly given that drug representatives often will try to make invalid cross-trial comparisons). This is an example of comparing the placebo group from one trial to a treatment group in another trial, which is always problematic. The study was written by superb investigators, but this one aspect of the statistical analysis is quite questionable.  (On the other hand, this 2001 article in JAMA brought beneficial widespread attention to the potential of a COX-2 inhibitor being associated with increased cardiovascular risk.)  
____________________________________________

 The task of educators of medical students in the field of interpreting clinical trials is a crucial one.  Following graduation, the students as physicians will be confronted with a barrage of clinical trials.  Skills helping to discern optimally performed and interpreted clinical trials from those that are inherently weak will be useful to the current student throughout his or her career in medicine.

A few topics have download files in addition to the other information available:

 1. The hazards of subgroup analysis. Are Geminis really different? 
      download (pdf)

 2. Example of bias apparent in a meta-analysis in 30 seconds:    
      download (pdf)

 3. A Type II Statistical Error   Incorrect conclusions resulting from insufficient numbers. Death of the oat bran fad      
      download (pdf)

 4. Miscounted and Misinterpreted.  How not to conduct a study.
The NEJM cath study   
     download (pdf)

 5. The ALLHAT hypertension trial-- a detailed critique  
     download (pdf)

 6 Celebrex-- an unreliable cross trial comparison 
     download (pdf)

 7. A Tale of Two Trials-  A superbly conducted trial contrasted to a suboptimally interpreted trial:
     download (pdf)

 8 Limitations of Meta-analysis: overview:
     download (pdf)

 9. Lessons from an unusual saga of possible fraud in the literature  
The Columbia 'Miracle' Study:   
     download (pdf) 
                    .

home