How to Understand Nutritional Studies

Everyone, it seems, has an opinion on what is healthy and what isn’t when it comes to eating. Should you eat fewer carbs or less fat? When should you eat more meals or fewer? Should you fast? What about superfoods? Is that plant-based burger healthy?

Nearly every day a new study comes out telling us what we should and shouldn’t eat. Peruse some recent headlines, for example:

• Soybean oil is linked to neurological conditions
• Refined carbs linked to insomnia 
• Fish oil has no effect on depression
• Walnuts slow cognitive decline

If you read the underlying studies, you’ll instantly discover some differences. But we’ll save you the trouble: Each one of these is a different type of study. Each type has advantages and disadvantages when it comes to nutrition research. Even though there are other types of studies (case studies and case-control studies are also used in nutrition), these four are the predominant types. 

Causation vs. correlation
Before jumping into specific studies, let’s define what nutrition studies do and don’t do. Most will find, at best, a relationship between two things – a diet, food or ingredient and a specific outcome. In the above headline, soybean oil is “linked” to metabolic disorders. 

When one action is related to another in research,  it’s called correlation. That doesn’t mean that soybean oil causes metabolic disorders. Most nutrition studies show a link or a correlation. Very few show causation, that is, a cause and effect. 

Unfortunately, researchers often use loose language to promote their studies, which are then picked up and amplified by media, who have very little training in discerning which research is of importance and which isn’t. They also may not read the underlying study or talked to the researchers, which can compound the problem. For example, one headline writer for the soybean oil study wrote, “America's most widely consumed oil causes genetic changes in the brain.”

Lab and animal studies
So, is soybean oil, which typically masquerades under the label “vegetable oil,” safe? Well, not if you’re a mouse in the hands of the study’s researchers. That’s right, the study was conduction on mice — not humans. 

When a study is done in the lab in petri dishes or in animals, it’s often the most preliminary type of research. It’s these studies that give researchers thoughts into how to conduct a study later in humans, but they don’t tell us much beyond, “Hey, this might be something we want to study.” Animal studies play a role in research — they give direction and are often cheaper and faster to perform — but often they do not pan out in humans. They make a lot of interesting headlines, however.

Cohort (or observational) studies
Should you skip that bowl of ice cream before you got to bed? Probably, but not only because you’re worried about the study that linked refined carbs to insomnia. It’s what scientists call a cohort study. In a cohort study, researchers look at one group or multiple groups, or cohorts, of people. There are many famous cohorts that are used to produce all sorts of nutrition research. You may have even heard of the Nurses’ Health Study or the Framingham Heart Study. Often, cohorts will have hundreds or thousands (and sometimes 10s of thousands) of participants. 

In a cohort study, participants are interviewed initially to determine their baseline before a disease develops. When you read about coffee leading to longer life or red wine reducing heart disease risk, it’s usually from these types of studies. Researchers look at outcomes over time and parse data to see what dietary decisions may have influenced those outcomes. In the Nurses’ Health Study, for example, women who enjoyed one or more drinks per day were at greater risk for breast cancer.

Like lab studies, researchers can only make casual links to outcomes — not the cause and effect relationships that give real clarity to the eating decisions, like having ice cream before bedtime, that we make.

Meta-analysis
“Fish oil is a panacea, helping lower risks for heart disease, obesity, mental health and inflammation.” Okay, maybe. This type of information is often derived from observational studies and meta-analyses, where researchers tease data from previous, multiple studies, that support fish oil for lots of different conditions. Meanwhile, other studies find no correlation. Back to fish oil in a minute. 

Meta-analysis studies combine data from many sources. When done right, they can provide estimates to the effect of interventions from across a large pool of data. But they can also include studies with dubious data and may draw conclusions that are not supported. You may recall a meta-analysis on saturated fat published a few years ago. The study’s conclusion: Evidence does not support higher consumption of polyunsaturated fats and lower consumption of saturated fats. Media reports suggested that researchers found no link between saturated fat and heart disease. 

While the study was never retracted, many researchers questioned its conclusion, charging it contained multiple errors and omissions. One response to the paper published by the Annals of Internal Medicine called the findings “seriously misleading and should be disregarded.”

When meta-analyses are called into question, it’s usually over the methodology. Even seasoned researchers have trouble sorting through all the math. What are consumers to do? Just understand that conclusions from meta-analysis aren’t the final word.

In the study linked above, researchers were trying to determine if fish oil could help with anxiety or depression. But the data from 31 different studies showed that long-chain omega-3 supplements (fish oil) probably have “little or no effect in preventing depression or anxiety symptoms.”

Randomized Controlled Trials 
The gold standard of studies is the randomized controlled trial. In this kind of study, participants are randomly assigned to one of two groups. One group receives an intervention – say a supplement or intensive nutritional counselling — and the other group receives a placebo or some other typical intervention. The groups do not know what they’re receiving and are thus “blind” to the experiment. In “double blind” studies, the researchers also do not know which group is receiving the intervention. 

These studies are often designed specifically to elicit a cause and effect because the intervention is the only factor that’s different between the two groups. Often, they produce disappointing results. For example, researchers, using walnuts as an intervention for two years in the study above, hoped to show slowed cognitive decline in their elderly participants. There was promising preliminary studies (lab and observational) which suggested that walnut consumption counteracted oxidative stress and inflammation, two drivers of cognitive decline, according to the study authors. 

Unfortunately, all those extra nuts didn’t help. The authors optimistically concluded that results were encouraging, just not enough upon which to base nutritional decisions.

Being a Smarter Consumer
Does that mean all the nutritional advice you hear is bad? Not at all. Each of these types of studies add to our overall understanding and knowledge of nutrition. In context and in volume, they help guide the advice physicians, nutritionists and the U.S. government provide. 

A single study that suggests eating more or less of an ingredient can make great headlines – and it may even provide important information for other researchers, but it’s probably not advice you want to follow without looking at the type of study researchers conducted. Understanding how they reached their conclusions help you put it in context.