Which Health Studies Are Reliable? ACSH Explains Epidemiology

By Alex Berezow, PhD — Apr 04, 2017
Public health is a field that's widely misunderstood, even by science journalists. That's because epidemiology is an inexact science, complicated by a large variability in the quality of the data it produces. Also, by its reliance on advanced statistical methods. 
Credit: Shutterstock

Every student in America should be required to take a class called, "What Do We Know, and How Do We Know It?" Perhaps if we learned from an early age how we know the things we claim to know, fewer Americans would fall for ridiculous conspiracy theories.

Public health is a field that is widely misunderstood, even by science journalists. That is because epidemiology is an inexact science that is complicated by a large variability in the quality of the data it produces, as well as by its reliance on advanced statistical methods. Let's leave the latter aside and focus on the former. Which epidemiological studies are most reliable and why?

From weakest to strongest, here are the most common epidemiological study designs:

Case report. A case report is the medical equivalent of "show-and-tell." Doctors find a patient with an unusual disease, and then they write a report about it. According to The Lancet, a case report "should not be a rarity but one that a general physician might encounter, in which there was some difficulty in reaching a diagnosis, and that provides a teaching point." 

Cross-sectional study. A cross-sectional study is a snapshot in time. It answers the question, "How many people have a particular characteristic (such as Disease X) at this very moment?" This study design does not provide any information on risk or cause.

Ecological study. An ecological study examines disease at a population level. A good example is a study that uses a scatter plot to correlate average alcohol consumption and the rate of coronary heart disease deaths in various countries. While such a study design provides clues for further research, it must be remembered that there is no data on individuals. All of the data is collected at the level of the population, which means that a person's individual exposure and disease outcome are not measured. Obviously, such a study design could be easily confounded by outside variables.

Case-control study. A case-control study compares people who have a particular disease ("cases") with a similar group of healthy people ("controls"). This kind of study is most useful for rare diseases. However, case-control studies are notorious for producing false positives. The reason is because people who have a rare disease are much likelier to remember their lives differently than healthy people. (This is a well-known phenomenon called "recall bias.") Consider people with brain cancer. Many of them will report using a cell phone on the side of their head where the cancer developed, regardless if it's true or not. 

Cohort study. A cohort study is the best possible observational study. In this study design, people are compared based on their level of exposure to a particular risk factor. For example, comparing smokers with non-smokers to determine which group is more likely to develop cancer 10 or 20 years later would be a prime example of a cohort study. Even though confounding variables can make a definite determination of cause difficult, the cohort study is the only kind of observational study that gives a reliable measure of risk. 

Randomized controlled trial. The randomized controlled trial (RCT) is the gold standard in epidemiology. It can determine both cause and risk. However, it is generally only ethical to perform this kind of study for medical interventions to determine if they are safe and effective. It would be unethical to force-feed a large group of people Twinkies every day to see if they develop diabetes. 

Systematic review/Meta-analysis. A systematic review attempts to examine all the relevant evidence published on a given topic, and a meta-analysis combines the data sets of smaller studies into a single, large data set. Because smaller studies often produce contradictory results, the point of reviews and meta-analyses is to arrive at a "consensus." Comprehensive reviews and meta-analyses may include data from dozens or even hundreds of papers.

Source: "Health Literacy: Evidence Pyramid." University of New Hampshire. Accessed: 4-Apr-2017.

Alex Berezow, PhD

Former Vice President of Scientific Communications

Dr. Alex Berezow is a PhD microbiologist, science writer, and public speaker who specializes in the debunking of junk science for the American Council on Science and Health. He is also a member of the USA Today Board of Contributors and a featured speaker for The Insight Bureau. Formerly, he was the founding editor of RealClearScience.

Recent articles by this author:
ACSH relies on donors like you. If you enjoy our work, please contribute.

Make your tax-deductible gift today!

 

 

Popular articles