Specific guide to this web site for:
2. Medical Students
3. Science media writers
4. High School & College
1. Harvard led MI study
(J. of Amer. Coll.
4. Amer. J. of Cardio.
review of literature
6. Oat bran study
7. Pregnancy & Alcohol
8. Are Geminis
9. Columbia 'Miracle' Study
Limitations of Meta-Analyses
Large Randomized Clinical
Tale of Two Large
Limitations of Meta-Analyses
limitations and problems with meta-analyses.
The advocate meta-analysis
an advocate for a particular viewpoint in a controversial area creates a
meta-analysis, biases are likely to be present in the creation of the
meta-analysis as well as its interpretation. This represents another potential
false gift of a meta-analysis.
most "opaque" of meta-analyses.
Poorly done studies included in analysis, inappropriate combinations of very
different study populations, as well as other problems become even more
difficult to see in a network meta-analysis
meta-analysis creates information that does not exist in study:
Monounsaturated fatty acids replacing saturated fatty acids
meta-analysis by authors from Harvard Medical School and other prestigious
institutions creates data that does not actually reliably exist in their study because of the misuse of
a statistical approach. The inaccurate information in this study has subsequently being widely quoted and dispersed in the literature.
a meta-analysis places researchers one step further from the data being
interpreted which can make the flawed use of statistical techniques more difficult to
see at first blush.
is a critique of the study:
Major types of dietary fat and risk of coronary heart disease: a,
analysis of 11 cohort studies. by Jakobsen MU, O'Reilly EJ, Heitmann BL, Spiegelman D, Willett WC, Ascherio A et al. Am J Clin Nutr 2009;89:1425-32.1
This meta-analysis said
information on the effects of replacing saturated fats with either
monounsaturated fats or polyunsaturated fats in regards to the
development of coronary heart disease
The study, which pools the results of 11 descriptive nutritional studies, provides
no reliable information on the effects of replacing saturated
fatty acids with monounsaturated fatty acids.
"Short Take" section to understand the mistakes these researchers
take: Monounsaturated fatty acids are often associated with foods such as olive
oil since monounsaturated fatty acids are the dominant constituent of
olive oil. However, in many Western diets, in contrast to the Mediterranean diet,
the major source of monounsaturated fatty acids is animal fat. The authors
in this study specifically state the primary source of monounsaturated fatty
acids for study participants was animal fat. (Butter, cream, and meats
all contain both
monounsaturated fatty acids and saturated fatty acids.)
As an example, 200 calories of ground beef contains
an approximate average of 5 grams of saturated fat
and 6 grams of monounsaturated fat.2
Both types of fats are in the same food. Seemingly
sophisticated statistical manipulation can not reliably transform food intake where the
monounsaturated fats are eaten in combination with saturated fats to information
on the effects of monounsaturated fats replacing saturated fats.
A letter to
the editor by was written and included the above critique. (Roehm E. Am J Clin Nutr 2009;
In the author
response to the letter, they reiterated their conclusions without any
specific defense of their statistical approach.
the article has subsequently been cited over 200 times in the medical literature,
including frequent quotations of the invalid conclusions concerning the
"information in this study" about the effects of saturated fatty acids
replaced by monounsaturated fatty acids.
provides no reliable information about saturated fats being replaced by
Unfortunately, a meta-analyses puts researchers one step further from the data
that is being interpreted. When a flawed statistical analysis creates
information not even present in the original data, it may not be obvious that the conclusions are erroneous. This meta-analysis was performed by
some of the most talented and established researchers in the field of nutrition
who have performed a multitude of well conducted and analyzed studies. Why was
this particular study so off-base?
It was most likely the application of a new statistical approach to a
meta-analysis. Meta-analyses tend to make the data somewhat "opaque".
A simple reading of a meta-analysis may not show that the studies combined are
of poor quality, that distinctly different populations of subjects are being inappropriately
combined, or in this particular case, information and conclusions are created
that do not reliably exist in the information that makes up the studies.
is the fundamental problem with this study? The
authors suggest that the replacement of saturated fatty acids (SFAs) with
polyunsaturated fatty acids (PUFAs) rather than with monounsaturated fatty acids
(MUFAs) prevents coronary artery disease. The only problem is that article does
not contain reliable information on saturated fats being replaced by
monounsaturated fatty acids. The 11 studies that make up this
meta-analysis are dominated by studies where the source of monounsaturated fatty
acids (MUFAs)and the saturated fatty acids (SFAs) come from the very same food
products: meat and dairy products. The
authors state that "the main source of MUFAs was animal fat".
Though monounsaturated fats are often associated with olive oil, which is the
dominant source of this fat in a Mediterranean diet, many Western diets contain only comparatively small amounts of monounsaturated fats from plant
Because meat and dairy products contain both MUFAs and SFAs, the ingestion of
these types of foods does not provide reliable information on the effects of
MUFAs being replaced by SFAs.
For example, 200
calories of ground beef contains an average of 5 grams saturated fatty acids
(SFAs) and 6 grams of monounsaturated fatty acids (MUFAs). Statistical
manipulation can not validly convert this to the equivalent of the replacement
of SFAs intake by MUFAs intake. Both types of fats are contained in the same food
item. Similarly butter, cream, and dairy products have significant amounts of
both SFAs and MUFAs in them. Since the dominant source of monounsaturated fats in this
study is meat and dairy fat, it is a fallacy that a statistical technique can
provide valid and reliable results from this information regarding the effects
of MUFAs replacing by SFAs on any disease process, including coronary heart
Despite this, in their article, the researchers suggest in the conclusions that
the replacement of saturated fatty acids with polyunsaturated fatty acids,
rather than MUFAs, prevents coronary heart disease. Also, Table 2 in the article
gives information on "MUFAs for SFAs" and in the discussion section,
the article refers to "a lower intake of SFAs and a concomitant higher
energy intake from MUFAs".
However, the concept that this study
provides information on monounsaturated fatty acids (MUFAs) replacing saturated
fatty acids (SFAs) rather than MUFA intake in conjunction with SFA intake is a
misinterpretation of the data. The hamburger, given as an example earlier,
contains both monounsaturated fat and saturated fat and no statistical technique
will reliably convert the ingestion of that hamburger into information on
saturated fatty acids being replaced by monounsaturated fatty acids.
what happened? What was the statistical technique that was employed?
The statistical model expressed energy intake of MUFAs, PUFAs, and SFAs as
percentages of total energy intake as continuous variables. Then a hazard
ratio (HR) was constructed to estimate the change of risk of heart disease (CHD)
with a lower intake of saturated fatty acids and a higher intake of MUFAs and
PUFAs. According to the authors: "The estimated HRs for unsaturated fatty
acids and carbohydrates can be
interpreted as the estimated differences in risk of a 5% lower energy
intake from SFAs and a concomitant higher energy intake from unsaturated fatty
acids and carbohydrates, respectively."
The problem with respect to MUFAs replacing SFAs is that the majority of MUFAs
were the result of the intake of meat and dairy products which also have
saturated fatty acids in them. Hence, no statistical technique, including this
one, can give reliable information from this data regarding the effect on
disease processes of MUFAs replacing SFAs.
The nature of the data in a meta-analysis puts the researcher one step away
further from the actual data making it less obvious to those involved that this
statistical modeling technique is not creating reliable information.
of further events
A letter to
the editor was written and published which included the above
Roehm E. Am J Clin Nutr 2009;
The authors' response to the letter reiterated their conclusions
without any specific defense of their statistical approach.
The article has been referenced over 200 times since publication. Many times the
completely invalid information regarding MUFAs replacing SFAs is cited and displayed
prominently in a chart comparing the effects of MUFAs, PUFAs, and SFAs. The
author of this website was personally attending a major academic nutrition
conference during which one of the speakers showed slides of the
"information from this study on MUFAs replaced by SFAs". Hopefully, in
the future, the lack of reliable information in this study concerning MUFAS
replaced by SFAs will become known and this study will stop being cited in
regards to this point of information.
It is the opinion of the author of
this website that it would be extremely difficult to find even a single
biostatistician in a full time academic position, at a United States medical
school not previously associated with this study, who has the opinion after
reading this critique in full as well as the study itself, that this study
provides reliable information on the effects of replacing saturated fatty acids
with monounsaturated fatty acids.
Jakobsen MU, O'Reilly EJ, Heitmann BL, Pereira MA, Bälter K, Fraser G,
Goldbourt U, Hallmans G, Knekt P, Liu S, Pietinen P, Spiegelman D, Stevens J,
Virtamo J, Willett WC, and Ascherio A. Major types of dietary fat and risk
of coronary heart disease: a pooled analysis of 11 cohort studies. Am J Clin
(Researchers are from multiple institutions including the Harvard School
of Public Health, Copenhagen Institute of Preventative Medicine, University of
Minnesota School of Public Health, Loma Linda University Department of
Epidemiology and BioStatistics, Harvard Medical School Brigham and Women's
Hospital Division of Preventitive Medicine.)
Letter to the Editor: Monounsaturated fats. Roehm E. Am J Clin Nutr 2009;
to E Roehm: Jakobsen MU.Am
Clin Nutr 2009; 90:698
E Roehm, MD