Last week I shared news from the Pharma front, specifically the latest on questionable business practices and undisclosed alliances that continue to dog the industry’s public image. Pharma, however, isn’t the only part of the health field cast in doubt these days. What would you say to a person who claimed that some 90% of medical studies were false – flawed in their parameters, riddled with basic error, skewed by presumption and bias? (No, it’s not me sayin’….) I’m talking about an insider: a highly esteemed, widely acclaimed, much admired, often cited medical researcher who has to turn down speaking engagements left and right. He’s a professional with a flair for statistics and penchant for credibility. His initial groundbreaking research (meta-research, actually) hit the medical scene years ago, but he’s still going strong and in more public headlines these days as the focus of an in-depth Atlantic Magazine feature.
His name is John Ioannidis, a name perhaps familiar to some who live, breathe or earn a living in the medical field. He’s made his mark as a keen analyst and critic of medical research, but he didn’t set out to rock the establishment boat. Early in his career at Harvard, he studied the intricate and thorny labyrinth of diagnosing and treating rare conditions. With little research to guide them, they navigate with “intuition and rules of thumb.”
As The Atlantic author explains, he discovered that practitioners were led by the same process when diagnosing and treating common medical conditions. As he analyzed the relative lack of hard science and credible studies, he took up the cause of examining research integrity, the foundation of the medical field and clinical treatment. He embarked on a massive analysis of then-current research and discovered a barrage of contradictory findings even in the “best,” peer-reviewed studies.
In 2005, Ioannidis published the results from his initial landmark analysis in PLoS Medicine, an open access online journal.
He presented a mathematical proof illustrating the breakdown of accuracy in various models of research: 80% of non-randomized trials, 20% of randomized trials, and 10% of large randomized trials can be expected to produce false results. In addition to these calculations, Ioannidis suggested that “researchers were frequently manipulating data analyses, chasing career-advancing findings rather than good science, and even using the peer-review process…to suppress opposing views.” (Ouch.)
He quickly followed up with a meta-analysis that left little room for argument or dismissal. To bring his point home his point about faulty research credibility, Ioannidis took apart 49 of the best known, most “highly cited original clinical research studies” in the field. Of the 49, 45 studies professed the targeted “intervention was effective.” Of this group, 14 (more than one-third) had been partially or wholly contradicted by later research. Ioannidis later found that subsequent studies often cited old findings without considering contrary results shown in later research – a pattern that caused refuted or questionable medical connections and recommendations to persist in both the research world and clinical practice.
As The Atlantic article brought Ioannidis’ research to public light, some have responded with frustration, wondering “What now?” It’s a fair reaction to some degree. Ioannidis’ conclusions are more than disillusioning; they’re jolting in their foundational, systemic emphasis. First, there’s the question of where his findings leave the medical field. Five years after his PLoS report, not much has changed on the research or clinical front. His message, which could and should have lit a fire under medical researchers, funding agencies, government oversight groups, and clinical practitioners, has been lauded but largely unheeded. Too many apparently feel too tethered to (or buried within) the existing system. The same brand of studies get funded and published and the media continues to spotlight the same spectacle. Nonetheless, the critical assessment is there, and perhaps some clinicians are the more wisely skeptical for it.
Ioannidis doesn’t suggest that the medical field is imploding before our eyes. He says researchers have been sloppy. He takes them to task for initiating their study with leading questions, with maneuvering data, and contriving results. The field as a whole is fraught with accidental and purposeful error, but it’s not going to hell in a hand basket. As Ioannidis suggests, there are still research practices that offer the best promise of accuracy, namely the large, randomized trials. These take money and time, however, which means there needs to be support, clout behind their undertaking. Few topics and treatments will be able to garner these resources. It’s a bit of a scientific cul-de-sac, if you will.
Ioannidis’ assessments don’t reflect the undoing of the research field, but they do remind us to take things with a grain of salt. His critiques represent a necessary check, a much needed perspective on the nature of science itself – science as it’s observed, examined and interpreted in the real world. As direct and concrete as scientific events and medical occurrences might be, we’re ultimately fumbling observers, experimental seekers even in the best of circumstances, even with the best of intentions. Nonetheless, we’re obliged to bring our highest standards and most open minds to the enterprise. Like the practitioner treating a patient with a rare disease, it pays to listen to the patient, to truly see rather than dismiss his/her experience.
Beyond the medical establishment’s inner workings, there’s the inevitable question the rest of us are asking ourselves: what do we, medical care and information consumers, do with these insights? Along that vein, Ioannidis’ analysis implies a broader scope, more holistic approach to health. In pursuit of a healthy and prevention-minded lifestyle, Ioannidis suggests we can give up the magic bullet mentality that has us chasing the latest “miracle” drug or nutrient to begin with. A single health measure in isolation, as Ioannidis suggests, isn’t a panacea. Popping a single nutrient, as those old, now refuted vitamin E studies proposed, isn’t going to make any kind of significant difference. (A rational person somehow always knew this on some level.) We need to understand, to acknowledge the nature of the beast – our own physiological (and dare I say, Primal) functioning. There’s truth in that underlying logic after all.
Once we give up the quest for the cure-all, we’re ready to live in the here and now, which is where real change happens after all. We contribute to our own health care and the relationships we have with our doctors. We can approach and question the research put before us when it directs our course of treatment or informs the choices we make. We can accept the complex responsibility that contributes to our overall wellness. Exchange the magic bullet metaphor for the mosaic. A healthy life is a compilation, an intermingling design formed by a thousand small acts, decisions, and questions. Each healthy thing we do, each life-giving choice we make adds animation and dimension to our overall wellness picture. There’s a curative integrity that we can cultivate for ourselves.
I hope today’s post has offered some food for thought. Let me know what you think, and have a great weekend.
If you'd like to add an avatar to all of your comments click here!
Ⓒ 2023 Mark's Daily Apple
