(www.improvingmedicalstatistics.com)                                          
      home   author    contact   web site dedication
  

......................m.........................

Specific guide to this web site for:

 1.  Medical School
      Educators 
      in Statistics

 2.  Medical Students

 3.  Science media writers

 4.  High School & College
     Statistic Teachers

   Misadventures:

1. Harvard led MI study

2. JACC study 
   (J. of Amer. Coll.
   Cardio.)

3. NEJM cath study

4. Amer. J. of Cardio.
    review of literature

5. ALLHAT
    controversy 

6. Oat bran study

7. Pregnancy & Alcohol

8. Are Geminis really
    different?
      
9. Columbia 'Miracle' Study  
                                                 

Additional Topics:

Celebrex

Limitations of Meta-Analyses

Large Randomized Clinical Trials

Tale of Two Large
Trials

Advocate meta-analyses

Network meta-analyses

back

Analysis of the Flawed Statistical Methods and Conclusions Found in the Article by Geltman et al

In the September 1990 Journal of American College of Cardiology there is a study performed by Geltman¹ el al which evaluated myocardial perfusion in patients with angina who had angiographically normal coronary arteries. The fundamental framework from which they performed the statistical analysis of the data had no validity.

  The study consisted of a control group and a chest pain group. The authors subdivided the chest pain group into those patients having a low myocardial perfusion reserve and those patients without a low myocardial perfusion reserve. This resulted in "three groups" 1) chest pain patients with a low myocardial perfusion reserve, 2) chest pain patients with a normal or high myocardial perfusion reserve, and 3) the undivided control group.
  

The authors then inappropriately statistically compared the "three groups" in regards to myocardial perfusion reserve, which is the same value that was used as the basis for selectively subdividing the chest pain group. They also compared the "three groups" for the values used to derive myocardial perfusion reserve. (Myocardial perfusion reserve = maximal myocardial blood flow / resting myocardial perfusion.) Unfortunately, there is no valid statistical justification for comparing two groups by subdividing one of those groups on the basis of a particular criteria and then statistically analyzing the resulting subgroups for differences in that same particular criteria (or in the component values from which that criteria is derived). This is a completely biased and invalid way to analyze data.  

The data in the Geltman study does not support the contention that angina in patients with normal coronary arteries is attributable to abnormalities of perfusion at rest, maximal myocardial perfusion, or myocardial perfusion reserve. The study does indicate that there is a statistically significant difference between the frequency of patients (8 of 17) with low myocardial perfusion reserve, and the frequency of controls (2 of 16) with low myocardial perfusion reserve. However, if one uses the individual data provided by Figure 1 in the Geltman study and plots myocardial perfusion reserve, resting myocardial blood flow, and peak myocardial blood flow for the two groups taken as a whole, the overlap between the control group and the patient group is more readily apparent. (Figure A) The invalid statistical analysis performed in the Geltman article made this less obvious. (Since only a portion of the normal controls were shown in Figure 1 in the original article, a complete set of individual data points for the normal controls is not fully obtainable for representation in Figure A.)

Hence, the paper does not in a statistically valid way achieve the objective stated in the initial introduction of "determining whether angina in such patients (normal coronaries in angina) is attributable to abnormalities of perfusion at rest, or maximal perfusion at rest or vasodilator reserve". There is too much overlap between the groups taken as a whole to reasonably attribute on the basis of the data presented that the angina in these patients is on the basis of an abnormality of perfusion at rest, maximal perfusion, or myocardial perfusion reserve.  

  The invalid statistical analysis that was performed tended to obscure the true implications of the data. It is essential that a journal such as JACC serve as a gatekeeper in order to prevent inappropriate statistical analyses from being published, because this will inevitably lead to skewed conclusions that then permeate the literature.  

To make this more apparent consider the following analogy. Initially assume there are two groups of persons (Group A and Group B) who are being analyzed in regard to individual height. Also assume that the height of five feet is considered the lower end of normal human height. Then arbitrarily divide Group B into two separate subgroups. Those individuals less than five feet tall are designated as "Group B small" and those greater than five feet tall are designated as "Group B large." Now, statistically compare the height of the "Group B small" versus "Group B large" and Group A. The "Group B small" has a statistically significant reduction in height compared to "Group B large" and Group A. ("Group B small" is smaller because it is preselected as such. The "Group B small" presumably would also have a statistically significant reduction in head to waist measurements and waist to foot measurements compared to the other groups. This is simply a result of the initial predefined selection bias.) All these 'statistically significant" findings do not lead to a valid conclusion that the initial Group A is different from Group B. Also, these "statistically significant" findings exist whether or not there is truly a subgroup present in the initial Group B that is somehow inherently and statistically different from a similarly defined subgroup present in Group A. Group A was never analyzed and further subgrouped in the same fashion as Group B. This is the same framework of analysis used by Geltman in their article and it is obviously fundamentally flawed.


1. Geltman EM, et al. Increased myocardial perfusion at rest and diminished perfusion reserve in patients with angina and angiographically normal coronary arteries. J AM Coll Cardio 1990; 16:586-95

back 

i