How to Read Research Presented in Popular Media

A Lesson using the Article "Holding back students helps boost scores but invites discrimination"
By Ron Matus
The St. Petersburg Times
January 12, 2009 		                 

When reading about research in popular media sources, such as newspapers or the internet, it is important to carefully consider the claims being made.

To do this, you need to ask yourself two main questions:
1) What claim, if any, is being made?
2) How can we determine if the claim is justified?

In order to illustrate how to accurately read research reported in popular media, we'll use an example. This sample article reports on studies concerning the retention of third graders in Florida who do not pass the Florida Comprehensive Assessment Test (FCAT) in reading.While reading this sample article, consider the two questions from above.
Click here to go to article.


After reading an article, ask yourself the first question: What claim, if any, is being made?
It is important to identify the claims the article makes -- by doing so, you begin to analyze the article, the research, and its conclusions.
When searching for an article's claims, be sure to look through the entire article and the headline as well. Often the headline is the most dramatic claim; it is the media's selling point. Once you have found the article's claims, click here to check your answer.  


Now that we have found the article's claims, how can we determine if these claims are justified?
First, what kinds of claims are being made? In other words, are they causal or noncausal?
Causal claims use language that implies one variable acts upon, or causes another.
Examples of causal words: affect, promote, change, make, create
Noncausal claims use language that describes the association between variables.
Examples of noncausal phrases: tend to be, associated with, more/less likely to, predicts, related to
An article may include both kinds of language and claims, but it is important to consider which kind of claim is made overall. The overall claim is the primary claim the article makes -- this is what the reader comes away with. In short, the reader won't remember the author, ttile, or publisher; instead he or she will keep the basic idea of the article. 
Once you've decided what the primary claim of the article is, click here to check your answer.

Second, we need to look at the research itself.
The research is the bare bone facts on which the article is based. Without the research, we cannot determine if the primary claim is justified. More importantly, if the research is not properly conducted, any claim made based on the research is jeopardized. In addition, if the researchers or the participants are slanted in any way, the results of the research and the claims made must be questioned. So in order to decide if the primary claim of our sample article is justified, we must look to see if the research is credible. 
What does the article tell us about the research?
        It may be helpful to ask yourself the following questions:
  • How many studies concerning Florida's retention program were presented in the article?
    • Is it possible there are more studies that were not presented?
  • What information/data did the researchers collect?
  • Who were the participants?
  • Who were the researchers?
  • What were the results of the research?
Once you have asked yourself these questions and looked back through the article to find your answers, click here to check your answers.

Third, we have to determine which kind of study is being reported. There are three different types of research studies: descriptive, correlational, and experimental.Click here to find out how the three are different!
Each kind of study results in a different level of causality. By determining the type of research study reported, we can determine how causal the claims of the article should be. If we don't know which kind of research study is being reported, we cannot decide if the article's overall claim is justified.
So is the study descriptive, correlational, or experimental? (Click one)

Finally, we should have enough information to decide if the article’s overall claim is justified.
The article’s overall claim is causal, and only an experiment can determine causality. Therefore, the claim is not justified because the research conducted was correlational research. Correlational studies can predict relationships and make associations, but they cannot determine that one variable causes another.
In addition, the claim was based on a single study, not several studies. Monty Neil’s comments caution the reader. He says there is more research and information about Florida’s retention program. We do not know much about how the researchers conducted the experiment; it could have been poorly done, and as a result, the findings could be completely wrong. Furthermore, the sample was taken only from Florida; we do not know if the study would have the same results in another part of the country.

By asking yourself what claim an article is making and if the claim is justified, you will be able to accurately read and understand research reported in any form of popular media.

If you would like to learn more about Greene and Winters’ research concerning Florida’s retention program, their study can be found in the Economics of Education Review.