A Study in Confusion

For years, sophisticated consumers have relied on the latest clinical trials to guide their health choices, but exactly how reliable are these studies? If you’re depending solely on scientific evidence for your health solutions, your assumptions about what’s right might well be wrong.

Every job has its occupational hazards. For me, it’s dinner parties. When chitchat turns to work and people discover I write about health, their eyes plunge to my plate and the questions fly: “What? You eat cheese? Isn’t cheese the devil? Hey, I read Goji berries cure cancer. What do you think?” At some point I utter my favorite dietary dictum, “Everything in moderation,” and skitter away, knowing full well that nothing will ever satisfy people’s yen for answers.

The answer to the question “what to eat?” should be stress-free, especially for well-educated, logical, health-conscious consumers who follow all the latest nutrition studies. But nutrition studies are rife with problems of their own. Remember when fat-free diets were supposed to be healthy — until they weren’t? Or when margarine was a healthy substitute for butter — until it wasn’t?

As much as people want nutrition research to be a simple, straightforward plotting from point A to point B, “the rhythm is more of a cha-cha — two steps forward and one step back,” explains Walter Willett, MD, professor of epidemiology and nutrition at the Harvard School of Public Health, in his book, Eat, Drink and Be Healthy (Free Press, 2005). Watching the dance is confusing, but steering your palate by it is crazy-making and might even be detrimental to your health.

Fending Off Food Angst

No one thought much about how food affected people’s health until the early 20th century, when scientists first broke food down into its basic building blocks: vitamins, amino acids, minerals and fats. Early nutrition advice focused on treating diseases of malnutrition, but by mid-century, rickets and scurvy were being replaced by diseases of dietary excess.

Americans today are nothing if not well fed. You know the score: More than 66 percent of us are overweight or obese. But do you know that four of the top 10 killers — coronary heart disease, cancer, stroke and diabetes — are directly related to eating too much of the wrong foods?

How can something that should be intuitive go so wrong? Mark Hyman, MD, editor-in-chief of Alternative Therapies in Health and Medicine and author of UltraMetabolism: The Simple Plan for Automatic Weight Loss (Scribner, 2006), thinks the misalignment stems from people’s bewilderment about what to eat. “There is considerable consensus on what constitutes good nutrition among scientists,” he says, “but somehow it gets confused, complicated and overinterpreted.”

Did someone say overinterpreted? Enter the media. Doctors were once the de facto source of health information, but now it’s journalists — and they have a voracious appetite for diet studies.

But not all nutrition studies are meant for prime time. Animal studies, for instance, are designed to generate conversation among researchers, not lead the evening news. And even the best human diet studies can be convoluted when squeezed into a few column inches or a 30-second news bite.

Recently, researchers reviewed more than 200 medical-news stories on the benefits and risks of three medications used to prevent major diseases and concluded that 40 percent were potentially misleading. So what’s a media-savvy, nutrition-conscious person to do? The first step is to understand the drawbacks of diet studies.

Rule #1: Diet studies are almost always wrong.

People’s diets are mind-numbingly complex. Unlike lab animals’ meals, ours don’t come in a pellet or a pill. People swallow thousands of different substances every day. Even if you managed to count every vitamin, protein and fat molecule in every bite of food and sip of liquid, you still couldn’t account for the thousands of potentially health-altering additives, pesticides and contaminants in most meals. And even the most meticulously designed studies are clouded by nondiet confounders, such as stress, sleep, exercise and environmental toxins. As Marion Nestle, PhD, MPH, professor of nutrition, food studies and public health at New York University and author of What to Eat (North Point Press, 2006) puts it, “Humans make terrible experimental animals.”

But that hasn’t kept nutrition researchers from banging their collective noggins against that wall for decades. To their credit, they’ve excavated a few potential truths from the mountains of uncertainty. Here’s a field guide to the three most common types of human studies, plus a word about animal studies:

Least accurate: Correlation Studies
Hair-raising headline: “American Men Are Five Times More Likely to Die From Prostate Cancer Than Japanese Men”
Angst Rating: Negligible

A correlation study compares the disease rates in two different populations — in this case, Japan and the United States. So should you stock up on tofu? Not so fast, says Lisa Schwartz, MD, MS, codirector of the White River Junction Outcomes Group at the Veterans Affairs Medical Center in Vermont. People in different cultures not only have different diets, they also have different genetics, stress levels, sleep patterns and exposure to toxins. In this example, researchers presumed the prostate protection came from soy foods, but they can’t prove it, she says. “You can correlate all sorts of things and hope you’re right, but at the end of the day, these studies are nothing but a leap of faith.”

More accurate: Cohort Studies
Hair-raising headline: “High-Fiber Diet Doesn’t Cut Risk of Colon Cancer”
Angst Rating: Don’t give up on whole grains

Nutritional cohort studies follow the diets of a group of people for several years to see who gets a disease and who doesn’t. Usually, the group will have something in common, like living in the same city or having the same job. The legendary Nurses’ Health Study, for example, monitored the diets of 122,000 female nurses for more than 20 years. Harvard researchers gleaned most of their diet info from questionnaires completed every four years. And therein lies the rub. Most people struggle to remember what they ate for lunch yesterday, much less four years ago.

Such gray areas can contribute to frustrating flip-flops, such as when a 1998 trial based on data from the Nurses’ Health Study showed women who gobbled folate and vitamin B6 had less heart disease. “The study said B vitamins are going to help everybody lower their risk of heart disease,” says Schwartz. The sheer size of the study, plus the fact that it appeared in the venerable Journal of the American Medical Association, heightened the buzz. But in 2006, a more accurate study disproved the B vitamin–heart disease theory. It just goes to show that diet studies like these should be viewed with healthy skepticism, since you often don’t know whether the differences are about the diet itself or the kind of people who eat that type of diet, says Schwartz. “The media is quick to give the stamp of approval to studies appearing in top medical journals, but people need to realize the science may still be wrong.”

Best bet: Randomized-Controlled Studies
Hair-raising headline: “Dark Chocolate Lowers Blood Pressure”
Angst rating: Whoopee!

The study that felled the B vitamin–heart disease hypothesis was a meta-analysis of several randomized-controlled trials, the third and most accurate diet-study design. To use this “gold standard” study, researchers randomly split participants into two groups. They assign one group a specific diet or nutrient and give the other one standard fare or a placebo. Ideally, the study is “blinded,” so neither researchers nor volunteers know who’s getting the real deal and who’s getting a sham. At the end of the study, scientists look to see if the intervention group (getting the specific diet) fared any better than the control group (getting standard food).

Although this type of study carries more street cred than its siblings, it’s not infallible. Some critics take umbrage with isolating a nutrient. “Taking one nature-made molecule such as a nutrient and using it to treat one disease doesn’t reflect biological reality or how these molecules work without our entire physiology,” says Hyman. “The one-pill, one-ill study doesn’t consider the impact of multiple components of our diet on multiple bodily systems.”

Other experts think such studies can wind up over-dramatizing food stories. “Eating a serving of trans fat won’t kill you, just as eating a particular fruit or vegetable won’t provide lifetime protection against heart disease or cancer,” says Nestle. “It’s what you habitually eat — and how much — that matters.”

On that point, nearly everyone agrees. But conducting a randomized-controlled trial on what people habitually eat (a.k.a. whole dietary patterns) is expensive, time consuming and challenging — especially getting folks to cooperate. Take the Women’s Health Initiative. One goal of the $1 billion study was to find out if a low-fat, high fruit, veggie and grain diet could lower a postmenopausal woman’s risk of developing certain cancers or heart disease. After enrolling nearly 50,000 women, researchers put half the volunteers on the diet (intervention) and told the other half to chow down as usual (control). They watched the women’s diets and health for an average of eight years. In the end, the study concluded that low-fat diets didn’t significantly lower the risk of breast cancer, colon cancer, heart disease or stroke.

But the research was clouded by the fact that the women in the intervention group never met the study goal, which was to limit their fat intake to a mere 20 percent of calories consumed. Even so, the low-fat group did see a 9 percent reduction in breast cancer occurrence. But even that result was inconclusive because the women on the low-fat diet were also eating more fruits, vegetables and grains.

So what caused the decline in breast cancer? Was it less fat or more rabbit food? That’s the billion-dollar question. Still, Schwartz says, randomized-controlled trials offer the best evidence.

A word about: Animal Studies
Hair-raising headline: “Aspartame Causes Cancer in Lab Rats”
Angst rating: Don’t panic — yet

When headlines warn about common food substances causing cancer in lab animals, should you sweat it? Schwartz says no. “The size of the dose and how it is given to animals doesn’t relate well to humans,” she explains.

That said, some topics have legs — such as animal studies linking aspartame to cancer. “We shouldn’t dismiss the fact that animal experimentation has raised questions about cancer and aspartame more than once,” says David Schardt, MS, a senior nutritionist for the Center for Science in the Public Interest. “People shouldn’t necessarily change their diets in response to animal research, but [a result like this] clearly means we should do more testing.” (For more on the potential downside of sugar substitutes like aspartame, see “Poor Substitutes.”)

Rule #2: The Ivory Tower is for sale.

Once you’ve determined how much weight to assign a diet study, the next thing to consider is the source. After all, with $800 billion on the line — the amount Americans spend annually on food and beverages — it’s no surprise that food-industry giants manipulate diet studies to fuel sales. “In the same way people are concerned about the pharmaceutical industry, they need to be concerned about diet advice,” says Schwartz. “There are tremendous self-interests at play.”

Indeed, the food industry backs most diet studies. Why does this matter? For starters, it skews the type of research being done. Giant food companies can afford to finance studies that cast their products in a favorable light. “Nobody’s funding clinical trials on fruits and vegetables,” says Nestle. “That’s why you don’t see diet studies on the health effects of artichokes.”

Another given: When researchers are beholden to food companies, science can suffer. Recently, experts tried to measure the extent of the suffering. From the 10,000 nutrition-related studies published in 2003, they picked 206 clinical trials studying the health effects of milk, soft drinks and fruit juices. The results, published early this year in the online medical journal PLoS Medicine, were telling.

The odds that a study reported a favorable benefit were four to eight times higher in trials backed entirely by industry than in those funded by independent sources. That’s because when the food giants fund research, they can influence how the researcher sets up and conducts the study, says Schardt. “You can set up an experiment to prove just about anything if you know what you’re doing.”

In the late 1990s, for example, the National Dairy Council (NDC) began funding research by Michael Zemel, PhD, director of the Nutrition Institute at the University of Tennessee in Knoxville, on the weight-loss potential of dairy foods. For his research on dairy products and calcium-fortified cereals, Zemel’s Nutrition Institute has received an estimated $2.5 million in research grants from the NDC and $550,000 from General Mills. According to The New York Times, Zemel himself nets about $50,000 a year in license fees from General Mills and a dairy farmer trade group called Dairy Management, Inc. (which manages the NDC), fees that allow them to use his patented research findings in their marketing campaigns.

Zemel has acknowledged these business relationships but firmly denies that they affect the results of his research. “I will take money for my research from anyone who wants to give it to me, with only one caveat: They can’t control the outcome, what I say or the publication of the research,” he told The New York Times in June 2005.

In return for its largess, the NDC has a fistful of data from Zemel’s lab showing dairy products help people lose weight. The studies became the foundation of a $200 million advertising campaign that promoted dairy as a diet food. But, try as they might, independent researchers couldn’t replicate Zemel’s findings.

Eventually, the government forced the NDC to dial down the misleading message, but it’s safe to say the dairy industry got its money’s worth. “Consumers are at a disadvantage,” says Schardt. “Between the slow regulatory process and the slap-on-the-wrist punishments, the companies have the upper hand.”

What to Eat?

No single diet or study is a good-health guarantee, so try to keep a watchful eye on the dietary evidence as it accumulates. In the wake of 50 years of nutrition research and hundreds of thousands of diet studies, a few clear notes repeatedly rise above the cacophony. Nestle sums them up in 10 words: “Eat less, move more, eat lots of fruits and vegetables.” For those who want more detail, she adds, “Go easy on junk foods.”

Now, if I can just remember to recite that at dinner parties, maybe we can all just relax and enjoy our food.

Like What You are Reading? Subscribe.

Catherine Guthrie is a Bloomington, Ind.–based freelance writer.

Share your thoughts: Join the conversation (1 comment)