Marks Daily Apple
Serving up health and fitness insights (daily, of course) with a side of irreverence.
14 Sep

Weighing the Evidence: Science and Anecdote in Nutrition Studies

I was a science major (biology) in college, yet I have always been a little suspicious of the use of the “scientific method” when it comes to biological systems – especially humans. I guess it started when we were all taught in labs as far back as high school to strictly adhere to the scientific method, which generally goes as follows:

  1. Ask a question
  2. Formulate a hypothesis
  3. Perform an experiment
  4. Collect and organize data
  5. Draw a conclusion

But from those earliest lab experiences, I found it was pretty sketchy to draw conclusions based on what often appeared to be nothing more than some random set of data points. Weigh the excised thymus of irradiated rats and plot a line that shows the rate of atrophy, etc…I wanted black and white answers, solid trend lines and reliable conclusions but usually all I got was an ill-defined line that was different from what my lab peers got, sometimes by a little, sometimes by a lot. Yes or no, right or wrong was what I sought – but that’s not how science works. It works more like this: the prevailing science is deemed good or acceptable until something or someone proves that it’s not good any more. Hey, that sounds like Conventional Wisdom. (Remember how fats were good for a million years, then they were bad for a few decades, and now they are good again…all based on the latest science?)

My skepticism has only grown over the years as my own experience in sports drug-testing (I was the Anti-doping Commissioner for the sport of triathlon worldwide for over a decade) showed me first-hand how unreliable certain complex testing systems could often be, and how a few wavy lines on a graph indicating a difference of a few parts per billion in a test could sometimes needlessly end a promising career or allow someone else to skate by unethically. More recently, I have been disappointed at how bad many current medical diagnoses are, even using the latest high-tech, most expensive equipment. (I will relate in detail in a coming post how the “gold-standard” body fat testing equipment erroneously pegged my own body fat at 16.9% – twice what it likely is.) Finally, I am noticing an increase in the number of “retractions” in scientific journals, where peer-reviewed studies are found to have been falsified or erroneous (last month a major 2003 genetic study that had been cited as an authority in 140 subsequent papers was retracted). I suspect that with pressure to publish and the complexity of designing studies these days, a not-insignificant amount of work gets published that probably shouldn’t – like many drug trials, for instance.

Through all the hard science, you may have also noticed that for every study that says that X is likely true, there’s usually another one that says, no, X is likely not true. Even hard science has its biases. And don’t forget correlation is not causation.

Of course, this skepticism of science can become an issue when one puts forth the hypothesis – as I have here – that “Primal Blueprint eating and lifestyle adjustments will result in superior health outcomes for humans”. Obviously, many people want to see the hard science. They want to see data points, charts and graphs and conclusions before they embark on such an unproven adventure. They want double blind crossover studies done in bariatric (my spell-check just wanted me to correct that to “barbaric”) wards or room-sized calorimeters. But all of that is not likely to be done anytime soon. That is, 100% conclusive evidence isn’t immediately forthcoming. And meanwhile the world is going to hell in an obese hand basket. So I now propose to you that sometimes anecdotal “evidence” can be nearly as valid as the “hard science.”

Now don’t get me wrong. We have to start with some basis in arguable facts. And I believe the facts clearly point to a Primal lifestyle for health and longevity. There is strong, solid science in favor of the Primal Blueprint. I’ve presented it here (on MDA) and in my book and many others have done likewise with their books and websites. But what happens when new science is difficult to decipher? When the media sensationalizes and distorts the conclusions? When there is no clear message but only an endless series of murky, conflicting results? I often get these questions from readers trying to make sense of it all, and though I still point to the science I also understand that there are other ways to weigh the evidence.

You see, biological systems are – to say the least – non-linear, and humans are certainly among the most complex of any organisms subject to review. In human studies, confounding variables often make it extremely difficult (almost impossible) to truly isolate or identify the one variable that might provide a benefit (or at least a measurable effect). Of course, that makes it doubly ironic that much of today’s Conventional Wisdom is based on snapshot observations from short-term and only partially controlled studies. For example, as we have discussed here ad nauseum, most “meat is unhealthy for you” studies don’t account for amount and types of carbs, as well as antioxidant intake and exercise, as well as the composition of the meat, as well as (yes, I said it) the family genetic history, and so on ad infinitum. To do so – to isolate all these variables and account for their very real interactions – would be wickedly expensive, if not impossible.

So we are left in some cases with anecdotal reports. But if you have enough of them, I say you have what you need to make a legitimate informed decision regarding whether or not this lifestyle is for you. My friends Drs. Michael and Mary Dan Eades can point to thousands of patients over decades that they have put on low carb, primarily meat-based diets with great success. You could even call it one big experiment, albeit without a control group. (I guess the controls would be all the family and friends that didn’t embark on the diet). They didn’t necessarily set this up as a study, but they most certainly collected detailed data in monitoring their patients over the course of weeks, months or years. And they found that an overwhelming majority of these people lost weight in the form of body fat and experienced improvements in blood lipids. Thousands of them.

I was fascinated by some of the detailed reports from participants in our own latest 30-day Primal Challenge. We eventually heard from over 400 people, almost all of whom related an experience of increased well-being in one form or another: lost body fat, increased muscle strength, more energy, fewer colds, improvement in blood glucose or blood lipids, etc. Now most true scientists I know would take these results and throw them all out, saying that there was no real “control group” nor was this a “double blind” study (where neither the subjects nor the researchers would have known which program they were on? – yeah, right) and that this “proves” nothing. Who knows, maybe only the people reporting were those with positive results and all the rest were unsuccessful and didn’t feel like writing to us. But I would like to think that in this case, we have enough anecdotal evidence to corroborate the intuitive recognition (and supports the existing scientific literature) that when we eat and move and live as our genes evolved and expect us to, it will almost always result in an improvement in our condition.

For extra credit, read “Why I am Not a Scientist” by Jonathan Marks.

I’d love to hear your thoughts. How do you weigh the evidence (both scientific and anecdotal) when making personal health decisions? Hit me up with a comment. Thanks, everyone!

You want comments? We got comments:

Imagine you’re George Clooney. Take a moment to admire your grooming and wit. Okay, now imagine someone walks up to you and asks, “What’s your name?” You say, “I’m George Clooney.” Or maybe you say, “I’m the Clooninator!” You don’t say “I’m George of George Clooney Sells Movies Blog” and you certainly don’t say, “I’m Clooney Weight Loss Plan”. So while spam is technically meat, it ain’t anywhere near Primal. Please nickname yourself something your friends would call you.

  1. I think the bigger problem is economics, and that powerful interests want to maintain status quo.

    The same problem is in the Physics community. They have very firm indication that Relativity (as defined currently) is not working beyond our solar system. It does not work well when accelaration due to gravity drops below a limit. We are getting a lot of artifacts like Dark Matter and Dark Energy. But the problem is economic, rather than scientific. There is no theory to take the place of relativity. Even though we know that it is wrong no one will admit to it. Because if they do what are they going to do. There is no money in trying to work finding a new theory, at least not for the majority. They need the grants which only come when you can show them what you can do in the given time frame.

    Economics trumps the Scientific method everytime.

    Anand Srivastava wrote on September 17th, 2009
  2. Interesting thoughts. I’m a scientist and care deeply about food and health, and I agree that many of your points are valid. At the same time, however, well done science can tell us a lot.

    Anecdotal evidence certainly has its value as well, but the problem with it is all it represents is an observation, not a real test. It is the first part of the scientific method only–telling us what, but not why.

    Personally I think we have to be skeptical of all data, anecdotal or peer-reviewed, because as you say, biology can be very non-linear. I let the science inform my decisions, but always try to remember that even if the data is 100% unambiguous (sometimes it is) there can still be more than meets the eye.

    For my own health I make my best guess on the science (which is arguably better than most people’s) then do what feels the best for me. I think we all need to do this since we each start at a different baseline.

    Thanks for making me think harder today :)

    Darya wrote on September 17th, 2009
  3. i’m a chemist. i like to think of the scientific method this way: you’re stuck in a dark room and you’re trying to find out what the other objects are in the room by throwing little pebbles at everything. it’s long, it’s laborious, but really really exciting when one of your little pebbles hits a light switch and part of the room is illuminated. it’s fun and effective. also, i think being a scientist is part of human nature!

    science itself isn’t at fault for all of the issues related to poor scientific research. there are a lot of politics and special interests that go into what gets funded. everything is motivated by money/power. this is more of a cultural issue than a problem with science.

    jellysoda wrote on September 19th, 2009
  4. Late to the party but found the conversation here interesting. I think we can look to history and see where both conventional wisdom and science have been extremely foolish. My motto is no matter who is selling, caveat emptor.

    noelJensen wrote on September 26th, 2009
  5. I don’t need scientific proof to know that hydrogenated soybean oil, HFCS and bran is BAD.

    It’s obvious there is something seriously wrong with our food system if great masses end up with the same diseases…all in the same place.

    Donnersberg wrote on April 19th, 2011

Leave a Reply

If you'd like to add an avatar to all of your comments click here!

© 2016 Mark's Daily Apple

Subscribe to the Newsletter and Get a Free Copy
of Mark Sisson's Fitness eBook and more!