A
blog of Bridge
Environment, updated most Thursdays
Which technological creations do pigs fear more, GMO feed or mutant super birds? |
Last week, I defined, put into historical context, and considered the human health effects
of genetically modified (GM) food items, also known as GMOs.
While promising to explore environmental and political/economic considerations
in future blogs, I concluded that human health concerns were more modest than
critics would have you believe, but that there is potential value in conducting
long-term testing of food varieties, GM or otherwise, that differ in
substantial ways from their predecessors. As I will describe in the
political/economic post, there may also be reasons to worry about financial and
political influences on the approval process.
About the same time as that blog posting, a scientific study
was published on the health of pigs which ate a GM diet (Carman et al. 2013). The authors used what one might
call a shotgun approach. Instead of a targeted study with a specific concern in
mind, they examined many anatomical and biochemical characteristics of 84
slaughtered pigs fed a GMO diet and 84 fed an equivalent non-GMO diet. Out of
the many characteristics they tested, they found differences they described as
statistically significant for two characteristics: uterine size and the rate of
severe stomach inflammation. Though there have been a few balanced blog posts
and media reports about this study, the majority used headlines such as “GMO
feed turns pig stomachs to mush!,” perhaps not surprising for a source called
Natural News. But MSN Now ran “GMO feed wreaked havoc on pigs’ stomachs.” If
general media reports are to be believed, this study confirmed our fears: GMO
foods do horrible things to our health.
Is it true? Was my advice last week off-base? Let me start by
reassuring you that the study does not change my conclusions at all. Here is
why.
First, let’s consider uteruses. According to the authors,
they were 25% larger in pigs fed GMO corn and soy (median 0.105% of total body
weight versus 0.086% body weight) and this difference was stastically different
at the 2.5% level (known as a p-value, where p stands for probability). However,
their claim is complicated and, in some cases misleading, due to several factors:
- the math: as reported in their tables, uteruses of GMO-fed pigs were 22% larger than those of non-GMO-fed pigs, but this may be a typo since the results from the table do not match what is reported in the text;
- attrition: several pigs died in the experiment (11 non-GM-fed pigs and 12 GM-fed pigs) and one non-GM-fed female pig failed to develop a uterus at all, so there may be a bias based on which pigs developed and survived until the end of the experiment; and
- the health significance: we have no understanding of whether larger uteruses for pigs at this stage in development is a good or bad thing; in fact GMO-fed pigs were slightly larger at slaughter so the difference may simply indicate faster sexual maturity.
The authors’ claim of statistical significance raises even
more concerns. Scientists typically only make strong claims about results if
observed differences have a 5% or smaller chance of occurring due to random
variation. Scientists picked the 5% p-value threshold because of a desire to
maintain high standards prior to claiming that an observed difference is real. Even
then, one in twenty times a scientist will report a meaningful finding that was
simply due to random differences among similar individuals.
This grey area of scientific proof becomes far murkier when
multiple comparisons are made. Because of the shotgun appraoch of this study,
where one treatment was conducted and many comparisons were made, we would
expect a far greater chance of an observed difference being due to chance than
if only one observation had been made. My first real statistics professor
referred to such shotgun approaches as p-ing all over the page, and this paper
is guilty. For example, they measured eight separate organs from the same set
of pigs. In each of those eight comparisons, there would be a 5% chance of
mistakenly thinking there was an effect of GMO feed when in fact the difference
was random chance. Collectively over the eight comparisons, there would be a 1
in 3 chance1 of thinking at least one organ size difference was
attributable to diet when in fact the pigs were essentially the same. To
correct for this multiple comparison bias, scientists are supposed to adjust
the threshold for considering a result significant. In the case of eight
comparisons, the new standard of significance would be 0.64% for each organ,
and the authors’ p-value of 2.5% would not be adequate to claim a true
difference between the uteruses of GMO- and non-GMO-fed pigs.
Inflamed stomachs were even more problematic than large
uteruses. The authors claim that severe inflammation occurred over 2.5 times
more often in GM-fed pigs and that the difference at a p-value of 0.4%.
However, the authors made 16 separate comparisons of pathological conditions.
To correct for the multiple comparisons, they should have adjusted their
significance level per condition down to 0.32%. Once again, valid use of
statistics would keep them from claiming a true difference. It is even more
interesting when we examine the other observed differences between GM-fed and
non-GM-fed pigs, many of which were related to the stomach. Whereas GM-fed pigs
more often had severe stomach inflammation, they also more commonly had no
inflammation, and less often had mild or moderate inflammation. There are
statistical tests to compare multiple category data like these, but it is not
surprising that the authors failed to use them considering their failure to
address multiple comparisons. Had they performed it, such an analysis would
have provided ambivalent results because of the fact that GM-fed pigs had
higher incidence of stomach health but also of severe inflammation. GM-fed pigs
also had lower incidence of stomach erosion, pin-point ulcers, and bleeding
ulcers, but higher incidence of frank ulcers (not sure what they are…aren’t all
ulcers honest?). GM-fed pigs also had lower incidence of heart, liver, and
spleen abnormalitlies. Mind you, none of these differences were statistically
significant, either, so all of this analysis should be taken with a very large
grain of salt.
What really stands out for me in this study is not the effect
of a GM diet, but the condition of all pigs raised commercially for meat
production. Over the course of these pigs’ short lifetime (less than six
months), more than one in eight died prior to slaughter, even with veterinary
treatment. The article reassures us that these death rates are “within expected
rates for US commercial piggeries.” Of the survivors, more than 1 in 10 had
heart abnormalities, 1 in 5 had abnormal lymph nodes, over half had moderate to
severe stomach inflammation, nearly 3 in 5 had pneumonia, and 4 in 5 had
stomach erosions. The condition of these animals definitely makes me ponder eating more seafood.
Back to GMO health effects…applying scientific standards for
statistical interpretation, this study becomes inconclusive. We could choose to
be like the authors and interpret trends in the data that may simply be a
result of random chance. This exercise yields a complex picture without any
obvious indication that GM-fed pigs were healthier or less healthy than their
non-GMO-fed counterparts. Should we dismiss the findings entirely? I don’t
think so. Some of those trends may be a result of real effects. However, follow
up study would be necessary and should be focused on particular concerns and
analyzed correctly. At this point, though, there still is no credible evidence
of health effects associated with common GMO food supplies. I maintain my
conclusions from last week, and promise to flesh out larger concerns
surrounding environmental impacts and political/economic influence over the
coming weeks.
News outlets that presented this research otherwise have
shown you their lack of respect for understanding science and, purposely or
inadvertently, played on our human tendency to panic over uncertainties. You
might want to consider better news sources in the future.
Best,
Josh
For
more information, read our other blog posts and visit us at Bridge Environment.
1 If
each comparison is treated as significant when the statistics report a 5%
chance of mistaking random variation for a true result, then each has a 95%
chance of correctly identifying random differences as being just that. To get
it correct for eight different comparisons, we have to multiply 0.95 by itself
eight times, 0.958 = 0.66. In other words, the likelihood that we
correctly eight observed differences as due to random chance is only 66%,
leaving a 34% chance…one in three, of seeing at least one false positive.