What makes a scientist?

Tom Naughton in his brilliant presentation called, “Diet, Health and the Wisdom of Crowds” recounts when a researcher responded to one of his critiques of their study on dietary cholesterol. Tom summed up the researcher’s attitude as essentially, “I’m a scientist and you’re not, so shut up.” I interpreted the response as, “you must be a shill for the egg industry, while I’m an expert of impeachable character and zero biases.” Still not a shining example of open-mindedness, but I digress.

If the foundation of science is truth, then the definitions of science, scientist, and the scientific method should be well understood and pervasive across the scientific community. (Notice that I said should.) What better source in the context of nutrition than Lehninger Principles of Biochemistry? It’s a biochemistry textbook used in undergraduate courses at many universities. Here’s how the authors define what “scientists” are:

Scientists are individuals who rigorously apply the scientific method to understand the natural world. Merely having an advanced degree in a scientific discipline doesn’t make one a scientist, nor does the lack of such a degree prevent one from making important scientific contributions. A scientist must be willing to challenge any idea when new findings demand it. The ideas that a scientist accepts must be based on measurable, reproducible observations, and the scientist must report these observations with complete honesty.¹

Many people, including myself, are self taught in many different areas. As long as the evidence is sound and reasonably interpreted, the source shouldn’t matter. Medicine in particular seems very compartmentalized and it’s rare for a doctor to stray outside of their specialty. Most endocrinologists I’ve spoken to usually refer their type 2 diabetic patients to nutritionists to discuss dietary interventions. The nutritionists, in my experience, don’t know the first thing about hormones or basic metabolism, but they’re considered the experts. Sure, every once and a while a medical professional tries to bridge the gap between disciplines, but they’re usually not rewarded for their “extracurricular” efforts.

I don’t blame people for being confused when it comes to controversial topics. People are generally not taught how to extract truth from data. The way most arguments are debated is to bifurcate an issue, pit the two extremes against each other and let them battle it out. Whichever side raises the most points that meet the observers’ preexisting ideas of truth wins. People without strong feelings either way usually throw their hands up and declare that the truth must lie somewhere in between. Others recognize the possibility of a false dilemma and rightfully proclaim that a balanced approach is not always warranted. Nevertheless, even conscientious people are subject to biases. If only there was a method to evaluate how nature operates and determine its behaviors while minimizing opinion masqueraded as fact. Oh, wait...there is! It’s called the scientific method.

The scientific method is actually a collection of paths, all of which may lead to scientific discovery. In the hypothesis and experiment path, a scientist poses a hypothesis, then subjects it to experimental test.²

Reasonable, intelligent people can have different points of view, but ultimately their beliefs must be tested and proven through more than one method. The Bohr-Einstein debates over quantum mechanics is often used as an example of an old guard suppressing the ideas of the upstarts. I disagree. While Einstein’s distrust of quantum theory isolated him from the mainstream developments in physics, his attempts to disprove his opponents’ ideas were grounded in experimentation, not empty rhetoric. The new ideas eventually won out not because they were more attractive or politically correct. Predictions of quantum mechanics have been verified experimentally to an extremely high degree of accuracy. Conversely, neither Ancel Keys nor John Yudkin had enough evidence to turn their hypotheses into proper theories.

Like two cars playing chicken, Keys’s fat-centric theory and Yudkin’s sugar-centric one raced boldly toward a one-lane bridge of scientific consensus—both determined to reach that narrow road before his rival. Although neither theory was as strong as either man wished, Yudkin’s, figuratively speaking, was the first to swerve, driven off path by inconsistent data and a mob of contemporaries already sold on the danger of saturated fat.³

Perhaps the most important point to be made is the definition of science itself. To understand science is to understand...well, understanding.

Science is both a way of thinking about the natural world and the sum of the information and theory that result from such thinking. The power and success of science flow directly from its reliance on ideas that can be tested: information on natural phenomena that can be observed, measured, and reproduced and theories that have predictive value. The progress of science rests on a foundational assumption that is often unstated but crucial to the enterprise: that the laws governing forces and phenomena existing in the universe are not subject to change. The Nobel laureate Jacques Monod referred to this underlying assumption as the “postulate of objectivity.” The natural world can therefore be understood by applying a process of inquiry—the scientific method. Science could not succeed in a universe that played tricks on us. Other than the postulate of objectivity, science makes no inviolate assumptions about the natural world. A useful scientific idea is one that (1) has been or can be reproducibly substantiated and (2) can be used to accurately predict new phenomena.

Scientific ideas take many forms. The terms that scientists use to describe these forms have meanings quite different from those applied by nonscientists. A hypotheses is an idea or assumption that provides a reasonable and testable explanation for one or more observations, but it may lack extensive experimental substantiation. A scientific theory is much more than a hunch. It is an idea that has been substantiated to some extent and provides an explanation for a body of experimental observations. A theory can be tested and built upon and is thus a basis for further advance and innovation. When a scientific theory has been repeatedly tested and validated on many fronts, it can be accepted as a fact.

In one important sense, what constitutes science or a scientific idea is defined by whether or not it is published in the scientific literature after peer review by other working scientists. About 16,000 peer-reviewed scientific journals worldwide publish some 1.4 million articles each year, a continuing rich harvest of information that is the birthright of every human being.⁴

While I agree with the sentiment of peer-review, the current process is in dire need of an overhaul. The article “Lies, Damned Lies, and Medical Science” sums up the issues with nutritional research succinctly⁵:

...the odds are that in any large database of many nutritional and health factors, there will be a few apparent connections that are in fact merely flukes, not real health effects—it’s a bit like combing through long, random strings of letters and claiming there’s an important message in any words that happen to turn up. But even if a study managed to highlight a genuine health connection to some nutrient, you’re unlikely to benefit much from taking more of it, because we consume thousands of nutrients that act together as a sort of network, and changing intake of just one of them is bound to cause ripples throughout the network that are far too complex for these studies to detect, and that may be as likely to harm you as help you. Even if changing that one factor does bring on the claimed improvement, there’s still a good chance that it won’t do you much good in the long run, because these studies rarely go on long enough to track the decades-long course of disease and ultimately death. Instead, they track easily measurable health “markers” such as cholesterol levels, blood pressure, and blood-sugar levels, and meta-experts have shown that changes in these markers often don’t correlate as well with long-term health as we have been led to believe.

On the relatively rare occasions when a study does go on long enough to track mortality, the findings frequently upend those of the shorter studies. (For example, though the vast majority of studies of overweight individuals link excess weight to ill health, the longest of them haven’t convincingly shown that overweight people are likely to die sooner, and a few of them have seemingly demonstrated that moderately overweight people are likely to live longer.) And these problems are aside from ubiquitous measurement errors (for example, people habitually misreport their diets in studies), routine misanalysis (researchers rely on complex software capable of juggling results in ways they don’t always understand), and the less common, but serious, problem of outright fraud (which has been revealed, in confidential surveys, to be much more widespread than scientists like to acknowledge).

If a study somehow avoids every one of these problems and finds a real connection to long-term changes in health, you’re still not guaranteed to benefit, because studies report average results that typically represent a vast range of individual outcomes. Should you be among the lucky minority that stands to benefit, don’t expect a noticeable improvement in your health, because studies usually detect only modest effects that merely tend to whittle your chances of succumbing to a particular disease from small to somewhat smaller. “The odds that anything useful will survive from any of these studies are poor,” says Ioannidis—dismissing in a breath a good chunk of the research into which we sink about $100 billion a year in the United States alone.

TL;DR

It’s no surprise (to me) that a fellow autodidact, Tom Naughton, would be able to distill all this down in a short speech he recorded three years ago. I wish this kind of thing was taught in schools, but sadly it is not.

Reference

1. Nelson, David L., Albert L. Lehninger, and Michael M. Cox. “A Note on the Nature of Science.” Lehninger Principles of Biochemistry. 5th ed. New York: W.H. Freeman, 2008. vii. Print.
2. Ibid.
3. Minger, Denise. “Ancel Keys and the Diet-Heart Hypothesis.” Death by Food Pyramid. Malibu: Primal Blueprint Publishing, 2013. 127. Print.
4. Nelson, David L., Albert L. Lehninger, and Michael M. Cox.
5. Freedman, David. “Lies, Damned Lies, and Medical Science.” The Atlantic. Atlantic Media Company, 4 Oct. 2010. Web. 4 Mar. 2014. <http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/2/>.