Weighing the Evidence: Science and Anecdote in Nutrition Studies

I was a science major (biology) in college, yet I have always been a little suspicious of the use of the “scientific method” when it comes to biological systems – especially humans. I guess it started when we were all taught in labs as far back as high school to strictly adhere to the scientific method, which generally goes as follows:

  1. Ask a question
  2. Formulate a hypothesis
  3. Perform an experiment
  4. Collect and organize data
  5. Draw a conclusion

But from those earliest lab experiences, I found it was pretty sketchy to draw conclusions based on what often appeared to be nothing more than some random set of data points. Weigh the excised thymus of irradiated rats and plot a line that shows the rate of atrophy, etc…I wanted black and white answers, solid trend lines and reliable conclusions but usually all I got was an ill-defined line that was different from what my lab peers got, sometimes by a little, sometimes by a lot. Yes or no, right or wrong was what I sought – but that’s not how science works. It works more like this: the prevailing science is deemed good or acceptable until something or someone proves that it’s not good any more. Hey, that sounds like Conventional Wisdom. (Remember how fats were good for a million years, then they were bad for a few decades, and now they are good again…all based on the latest science?)

My skepticism has only grown over the years as my own experience in sports drug-testing (I was the Anti-doping Commissioner for the sport of triathlon worldwide for over a decade) showed me first-hand how unreliable certain complex testing systems could often be, and how a few wavy lines on a graph indicating a difference of a few parts per billion in a test could sometimes needlessly end a promising career or allow someone else to skate by unethically. More recently, I have been disappointed at how bad many current medical diagnoses are, even using the latest high-tech, most expensive equipment. (I will relate in detail in a coming post how the “gold-standard” body fat testing equipment erroneously pegged my own body fat at 16.9% – twice what it likely is.) Finally, I am noticing an increase in the number of “retractions” in scientific journals, where peer-reviewed studies are found to have been falsified or erroneous (last month a major 2003 genetic study that had been cited as an authority in 140 subsequent papers was retracted). I suspect that with pressure to publish and the complexity of designing studies these days, a not-insignificant amount of work gets published that probably shouldn’t – like many drug trials, for instance.

Through all the hard science, you may have also noticed that for every study that says that X is likely true, there’s usually another one that says, no, X is likely not true. Even hard science has its biases. And don’t forget correlation is not causation.

Of course, this skepticism of science can become an issue when one puts forth the hypothesis – as I have here – that “Primal Blueprint eating and lifestyle adjustments will result in superior health outcomes for humans”. Obviously, many people want to see the hard science. They want to see data points, charts and graphs and conclusions before they embark on such an unproven adventure. They want double blind crossover studies done in bariatric (my spell-check just wanted me to correct that to “barbaric”) wards or room-sized calorimeters. But all of that is not likely to be done anytime soon. That is, 100% conclusive evidence isn’t immediately forthcoming. And meanwhile the world is going to hell in an obese hand basket. So I now propose to you that sometimes anecdotal “evidence” can be nearly as valid as the “hard science.”

Now don’t get me wrong. We have to start with some basis in arguable facts. And I believe the facts clearly point to a Primal lifestyle for health and longevity. There is strong, solid science in favor of the Primal Blueprint. I’ve presented it here (on MDA) and in my book and many others have done likewise with their books and websites. But what happens when new science is difficult to decipher? When the media sensationalizes and distorts the conclusions? When there is no clear message but only an endless series of murky, conflicting results? I often get these questions from readers trying to make sense of it all, and though I still point to the science I also understand that there are other ways to weigh the evidence.

You see, biological systems are – to say the least – non-linear, and humans are certainly among the most complex of any organisms subject to review. In human studies, confounding variables often make it extremely difficult (almost impossible) to truly isolate or identify the one variable that might provide a benefit (or at least a measurable effect). Of course, that makes it doubly ironic that much of today’s Conventional Wisdom is based on snapshot observations from short-term and only partially controlled studies. For example, as we have discussed here ad nauseum, most “meat is unhealthy for you” studies don’t account for amount and types of carbs, as well as antioxidant intake and exercise, as well as the composition of the meat, as well as (yes, I said it) the family genetic history, and so on ad infinitum. To do so – to isolate all these variables and account for their very real interactions – would be wickedly expensive, if not impossible.

So we are left in some cases with anecdotal reports. But if you have enough of them, I say you have what you need to make a legitimate informed decision regarding whether or not this lifestyle is for you. My friends Drs. Michael and Mary Dan Eades can point to thousands of patients over decades that they have put on low carb, primarily meat-based diets with great success. You could even call it one big experiment, albeit without a control group. (I guess the controls would be all the family and friends that didn’t embark on the diet). They didn’t necessarily set this up as a study, but they most certainly collected detailed data in monitoring their patients over the course of weeks, months or years. And they found that an overwhelming majority of these people lost weight in the form of body fat and experienced improvements in blood lipids. Thousands of them.

I was fascinated by some of the detailed reports from participants in our own latest 30-day Primal Challenge. We eventually heard from over 400 people, almost all of whom related an experience of increased well-being in one form or another: lost body fat, increased muscle strength, more energy, fewer colds, improvement in blood glucose or blood lipids, etc. Now most true scientists I know would take these results and throw them all out, saying that there was no real “control group” nor was this a “double blind” study (where neither the subjects nor the researchers would have known which program they were on? – yeah, right) and that this “proves” nothing. Who knows, maybe only the people reporting were those with positive results and all the rest were unsuccessful and didn’t feel like writing to us. But I would like to think that in this case, we have enough anecdotal evidence to corroborate the intuitive recognition (and supports the existing scientific literature) that when we eat and move and live as our genes evolved and expect us to, it will almost always result in an improvement in our condition.

For extra credit, read “Why I am Not a Scientist” by Jonathan Marks.

I’d love to hear your thoughts. How do you weigh the evidence (both scientific and anecdotal) when making personal health decisions? Hit me up with a comment. Thanks, everyone!

About the Author

Mark Sisson is the founder of Mark’s Daily Apple, godfather to the Primal food and lifestyle movement, and the New York Times bestselling author of The Keto Reset Diet. His latest book is Keto for Life, where he discusses how he combines the keto diet with a Primal lifestyle for optimal health and longevity. Mark is the author of numerous other books as well, including The Primal Blueprint, which was credited with turbocharging the growth of the primal/paleo movement back in 2009. After spending three decades researching and educating folks on why food is the key component to achieving and maintaining optimal wellness, Mark launched Primal Kitchen, a real-food company that creates Primal/paleo, keto, and Whole30-friendly kitchen staples.

If you'd like to add an avatar to all of your comments click here!