(www.improvingmedicalstatistics.com)                                          
      home   author    contact   web site dedication
  

         

......................m.........................

Specific guide to this web site for:


 1.  Medical School
      Educators 
      in Statistics

 2.  Medical Students
 3.  Science media writers

 4.  High School & College
     Statistic Teachers


   Misadventures:


1. Harvard led MI study

2. JACC study 
   (J. of Amer. Coll.
   Cardio.)

3. NEJM cath study
4. Amer. J. of Cardio.
    review of literature
5. ALLHAT
     controversy 

6. Oat bran study
7. Pregnancy & Alcohol
8. Are Geminis really
    different?
      
9. Columbia 'Miracle' Study  
                                                 

Additional Topics:

Celebrex

Limitations of Meta-Analyses

Large Randomized Clinical Trials

Tale of Two Large
Trials

Advocate meta-analyses

Network meta-analyses


 

 

 


 Beware of Meta-analyses Bearing False Gifts.

Meta-analyses performed by strong advocates of a particular position in an ongoing controversy are at higher risk for bias.

A meta-analysis is subject to a set of potential problems and pitfalls similar to a routine clinical trial. It has been documented that the conclusions of a meta-analysis (a summation of multiple smaller trials) can be shown to differ from a subsequent, large, more definitive, randomized clinical trial1.
 
The initial hurdle for doing a clinically meaningful meta-analysis is the criteria for how similar the studies must be in order to be included in the meta-analysis. The more similar the studies, the more valid the meta-analysis. If the meta-analysis combines multiple trials for similar patient populations where a placebo is compared to the same drug; or compares identical treatment regimens, then the meta-analysis results and conclusions are more likely to be in concordance with reality. A less restrictive requirement of similarities between the studies allows more trials to be entered into the meta-analysis which makes it easier to reach statistical significance. However, this can decrease the reliability of the conclusions of the meta-analysis.

The interpretation of a meta-analysis is potentially subject to an author’s bias by what inclusion and exclusion criteria is selected, the type of statistical evaluation performed, decisions made on how to deal with disparities between the trials, and how the subsequent results are presented. 

Whether the conclusions of a meta-analysis are broad reaching or limited can be affected by the inherent bias that the author of the meta-analysis brings to the study.

Human nature dictates that each of us tends to find it more satisfying to confirm a previously held opinion, particularly a published opinion, rather than create an analysis that refutes our own prior conclusions. Hence, interpretive bias is even more likely to occur when a meta-analysis is conducted by an author with a strong particular viewpoint in an area of controversy. When the meta-analysis is conducted by a strong advocate of a particular position, it is more likely to be biased in concordance with the author's previously advocated opinion.

  A meta-analysis2 was subsequently published  after the ALLHAT trial publication3 by the some of the same authors who were involved in formulating ALLHAT’s inappropriate conclusions. The authors of this meta-analysis tried to bolster their contention that the ALLHAT trial demonstrated that a diuretic drug should be the initial drug used for the treatment of hypertension.  The overly broad conclusions of this meta-analysis do not appropriately reflect the differences in blood pressure between the diuretic led therapy vs. the other therapies studied.
(see critique of ALLHAT meta-analysis for details).

Two separate meta-analyses analyzed the effects of oat bran and other soluble fibers on cholesterol levels.4,5   A prior flawed individual study incorrectly stated that oat bran does not significantly lower cholesterol.6  A subsequent meta-analysis written by the senior author of that study was interpreted in a manner to minimize any incongruity with the prior initial incorrect study. (see critique of oat bran meta-analysis)  A second meta-analysis concluded that oat bran modestly reduced cholesterol. details

A more widely recognized source of potential bias which can affect every type of medical study, including a meta-analysis, is a financial one. As an example, an employee of a pharmaceutical company authoring any type of study would routinely tend to have a bias favorable to the company's product.

A meta-analysis is not some infallible, final, arbitrator of a clinical question. (See Limitations of Meta-Analyses.)  This is particularly the case when there are significant differences between the trials being combined and when different patient populations are being studied.  A more reliable source is a single, large, well done study comparing treatment options consistent with the modern management of patients and with the results interpreted in a conservative fashion.   (See Tale of Two Large Trials)

1. Discrepancies Between Meta-Analyses and Subsequent Large Randomized, Controlled Trials. LeLorier J, Gregoire G, et al.  NEJM, 1997; 337:536-42

2.  Psaty B, Lumley T, Furberg C, et al.  Health outcomes associated with various antihypertensive therapies used as first-line agents, a network meta-analysis. JAMA 2003; 289: 2534-2544

3. ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group. Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs. diuretic: the antihypertensive and lipid-lowering treatment to prevent heart attack trial (ALLHAT). JAMA 2002; 288: 2981-2997

4.  Ripsin CM, Keenan J, Van Horn L, et al. JAMA 1992; 267:3317-25.  Oat Products and Lipid Lowering. A Meta-analysis. 

5.  Brown L, Rosner B, Lillett W, Sacks F. Cholesterol-lowering effects of dietary fiber: a meta-analysis. Am J Clin Nutr 111; 69:30-42

6.  Swain JF, Rouse,IL, Curley CB, Sacks F.  Comparison of the effects of oat bran and low-fiber wheat on serum lipoprotein levels and blood pressure. N Engl J Med 1990; 322:147-52.