Critique of Am J Clin Nutr 2009; 89: 1425-32 Jakobsen MU

Other related topics

Common limitations and problems with meta-analyses.

The advocate meta-analysis

When an advocate for a particular viewpoint in a controversial area creates a meta-analysis, biases are likely to be present in the creation of the meta-analysis as well as its interpretation. This represents another potential false gift of a meta-analysis.

Network meta-analyses

The most "opaque" of meta-analyses.
Poorly done studies included in analysis, inappropriate combinations of very different study populations, as well as other problems become even more difficult to see in a network meta-analysis

A meta-analysis creates information that does not exist in study:

Monounsaturated fatty acids replacing saturated fatty acids

A meta-analysis by authors from Harvard Medical School and other prestigious institutions creates data that does not actually reliably exist in their study because of the misuse of a statistical approach. The inaccurate information in this study has subsequently being widely quoted and dispersed in the literature.

Unfortunately, a meta-analysis places researchers one step further from the data being interpreted which can make the flawed use of statistical techniques more difficult to see at first blush.

This meta-analysis said it provided information on the effects of replacing saturated fats with either monounsaturated fats or polyunsaturated fats in regards to the development of coronary heart disease

Problem: The study, which pools the results of 11 descriptive nutritional studies, provides no reliable information on the effects of replacing saturated fatty acids with monounsaturated fatty acids.

Read the "Short Take" section to understand the mistakes these researchers made.

Short take: Monounsaturated fatty acids are often associated with foods such as olive oil since monounsaturated fatty acids are the dominant constituent of olive oil. However, in many Western diets, in contrast to the Mediterranean diet, the major source of monounsaturated fatty acids is animal fat. The authors in this study specifically state the primary source of monounsaturated fatty acids for study participants was animal fat. (Butter, cream, and meats all contain both monounsaturated fatty acids and saturated fatty acids.)

As an example, 200 calories of ground beef contains an approximate average of 5 grams of saturated fat and 6 grams of monounsaturated fat.² Both types of fats are in the same food. Seemingly sophisticated statistical manipulation can not reliably transform food intake where the monounsaturated fats are eaten in combination with saturated fats to information on the effects of monounsaturated fats replacing saturated fats.

A letter to the editor by was written and included the above critique. (Roehm E. Am J Clin Nutr 2009; 90:697-8)

In the author response to the letter, they reiterated their conclusions without any specific defense of their statistical approach.

Unfortunately, the article has subsequently been cited over 200 times in the medical literature, including frequent quotations of the invalid conclusions concerning the "information in this study" about the effects of saturated fatty acids being replaced by monounsaturated fatty acids.

This study provides no reliable information about saturated fats being replaced by monounsaturated fats.

Long take:

Unfortunately, a meta-analyses puts researchers one step further from the data that is being interpreted. When a flawed statistical analysis creates information not even present in the original data, it may not be obvious that the conclusions are erroneous. This meta-analysis was performed by some of the most talented and established researchers in the field of nutrition who have performed a multitude of well conducted and analyzed studies. Why was this particular study so off-base?
It was most likely the application of a new statistical approach to a meta-analysis. Meta-analyses tend to make the data somewhat "opaque". A simple reading of a meta-analysis may not show that the studies combined are of poor quality, that distinctly different populations of subjects are being inappropriately combined, or in this particular case, information and conclusions are created that do not reliably exist in the information that makes up the studies.

What is the fundamental problem with this study? The authors suggest that the replacement of saturated fatty acids (SFAs) with polyunsaturated fatty acids (PUFAs) rather than with monounsaturated fatty acids (MUFAs) prevents coronary artery disease. The only problem is that article does not contain reliable information on saturated fats being replaced by monounsaturated fatty acids. The 11 studies that make up this meta-analysis are dominated by studies where the source of monounsaturated fatty acids (MUFAs)and the saturated fatty acids (SFAs) come from the very same food products: meat and dairy products. The authors state that "the main source of MUFAs was animal fat".

Though monounsaturated fats are often associated with olive oil, which is the dominant source of this fat in a Mediterranean diet, many Western diets contain only comparatively small amounts of monounsaturated fats from plant sources.
Because meat and dairy products contain both MUFAs and SFAs, the ingestion of these types of foods does not provide reliable information on the effects of MUFAs being replaced by SFAs.
For example, 200 calories of ground beef contains an average of 5 grams saturated fatty acids (SFAs) and 6 grams of monounsaturated fatty acids (MUFAs). Statistical manipulation can not validly convert this to the equivalent of the replacement of SFAs intake by MUFAs intake. Both types of fats are contained in the same food item. Similarly butter, cream, and dairy products have significant amounts of both SFAs and MUFAs in them. Since the dominant source of monounsaturated fats in this study is meat and dairy fat, it is a fallacy that a statistical technique can provide valid and reliable results from this information regarding the effects of MUFAs replacing by SFAs on any disease process, including coronary heart disease.

Despite this, in their article, the researchers suggest in the conclusions that the replacement of saturated fatty acids with polyunsaturated fatty acids, rather than MUFAs, prevents coronary heart disease. Also, Table 2 in the article gives information on "MUFAs for SFAs" and in the discussion section, the article refers to "a lower intake of SFAs and a concomitant higher energy intake from MUFAs".

However, the concept that this study provides information on monounsaturated fatty acids (MUFAs) replacing saturated fatty acids (SFAs) rather than MUFA intake in conjunction with SFA intake is a misinterpretation of the data. The hamburger, given as an example earlier, contains both monounsaturated fat and saturated fat and no statistical technique will reliably convert the ingestion of that hamburger into information on saturated fatty acids being replaced by monounsaturated fatty acids.

So what happened? What was the statistical technique that was employed?
The statistical model expressed energy intake of MUFAs, PUFAs, and SFAs as percentages of total energy intake as continuous variables. Then a hazard ratio (HR) was constructed to estimate the change of risk of heart disease (CHD) with a lower intake of saturated fatty acids and a higher intake of MUFAs and PUFAs. According to the authors: "The estimated HRs for unsaturated fatty acids and carbohydrates can be interpreted as the estimated differences in risk of a 5% lower energy intake from SFAs and a concomitant higher energy intake from unsaturated fatty acids and carbohydrates, respectively."
The problem with respect to MUFAs replacing SFAs is that the majority of MUFAs were the result of the intake of meat and dairy products which also have saturated fatty acids in them. Hence, no statistical technique, including this one, can give reliable information from this data regarding the effect on disease processes of MUFAs replacing SFAs.

The nature of the data in a meta-analysis puts the researcher one step away further from the actual data making it less obvious to those involved that this statistical modeling technique is not creating reliable information.

Sequence of further events
A letter to the editor was written and published which included the above critique. Roehm E. Am J Clin Nutr 2009; 90:697-8

The authors' response to the letter reiterated their conclusions without any specific defense of their statistical approach.

The article has been referenced over 200 times since publication. Many times the completely invalid information regarding MUFAs replacing SFAs is cited and displayed prominently in a chart comparing the effects of MUFAs, PUFAs, and SFAs. The author of this website was personally attending a major academic nutrition conference during which one of the speakers showed slides of the "information from this study on MUFAs replaced by SFAs". Hopefully, in the future, the lack of reliable information in this study concerning MUFAS replaced by SFAs will become known and this study will stop being cited in regards to this point of information.

It is the opinion of the author of this website that it would be extremely difficult to find even a single biostatistician in a full time academic position, at a United States medical school not previously associated with this study, who has the opinion after reading this critique in full as well as the study itself, that this study provides reliable information on the effects of replacing saturated fatty acids with monounsaturated fatty acids.

1. Jakobsen MU, O'Reilly EJ, Heitmann BL, Pereira MA, Bälter K, Fraser G, Goldbourt U, Hallmans G, Knekt P, Liu S, Pietinen P, Spiegelman D, Stevens J, Virtamo J, Willett WC, and Ascherio A. Major types of dietary fat and risk of coronary heart disease: a pooled analysis of 11 cohort studies. Am J Clin Nutr 2009;89:1425-32.

(Researchers are from multiple institutions including the Harvard School of Public Health, Copenhagen Institute of Preventative Medicine, University of Minnesota School of Public Health, Loma Linda University Department of Epidemiology and BioStatistics, Harvard Medical School Brigham and Women's Hospital Division of Preventitive Medicine.)

2. Letter to the Editor: Monounsaturated fats. Roehm E. Am J Clin Nutr 2009; 90:697-8