The COVID-19 pandemic flooded the public with both reporting and original scientific research. It was inescapable. Many people waded into these waters, if not for the first time, certainly more than ever before. If you’ve ever tried to read a published peer-reviewed scientific paper, you know how daunting it can be. Jargon, dry language, and difficult-to-understand tables and figures make it cumbersome to wade through. Additionally, most people are not trained in statistics, research methodology, and critical thinking. It’s enough to make people give up and make science seemingly inaccessible.
Science holds a special place in our society. The title “scientist” is often equated with being smart, educated, and thoughtful. People also want to be associated with science, and thanks to the Internet, scientific findings, papers, and publications are just a few clicks away. Never has science been so accessible to so many people. The public knows about and even occasionally uses PubMed and Google Scholar. Just about every major outlet has at least one science reporter who covers newly published studies, patents, and general developments across all fields of scientific inquiry. And yet, with science so readily available at our fingertips, it remains inaccessible.
With all this in mind, I’ve constructed this how-to-guide for consuming scientific publications. It’s meant as an introduction/cheat sheet for consuming and thinking about the headlines, abstracts, and claims. It starts with key questions and ends with good practices.
Key Questions
When reading a paper or news article reporting on a new finding, ask yourself the following questions:
- What is the core issue?
- What is/was the goal? If you’re reading a news report, don’t forget that the goal is for your attention measured in clicks and eyeballs. Sensationalizing (think headlines), taking information out of context, and burying the lede can be misleading but are ways to keep your attention. Keep them in mind.
- What are the claims being made? Every study has them, and they can be found at just about any point: in the title, conclusions, and even in the study’s limitations.
What assumptions are being made?
"An assumption is an unexamined belief: what we think without realizing we think it. Our inferences (also called conclusions) are often based on assumptions that we haven’t thought about critically.”
Assumptions are sneaky. They can be both implicit and explicit and are woven throughout an article. It’s helpful to go back to the basics. Identify the authors' assumptions and how you would determine if those assumptions were accurate. Determine what evidence would prove an assumption to be true and what would prove an assumption to be false.
- Is evidence offered or provided? Yes or no.
- If so, what is the evidence, and what is the quality of it?
“The Sagan Standard is the adage that “extraordinary claims require extraordinary evidence.” This signifies that the more unlikely a certain claim is, given existing evidence on the subject, the greater the standard of proof that is expected of it.” - Effectiviology
Good practices
- Go back to the original source. This may not always be feasible because sometimes research is behind a paywall, but whenever possible, read the original article in its entirety.
- Seek out expertise. Seeking out the opinions of those with expertise in the field will help provide important context that may otherwise be left out of the reporting or your reading of the article.
- Be self-aware. Emotions and biases run deep, especially regarding politically charged topics. It’s equally important to examine the assumptions of you and the authors.
- Tolerate ambiguity and avoid oversimplification. The world is messy, and science reflects that. Numbers imply but do not guarantee certainty; neither do p-values. The most often used term in scientific articles is “more research is needed” for a reason.
The opinions expressed are those of the author and not her employer.
