Today’s post is a guest piece from media researcher Ron Sellers, from Grey Matter Research. In his presentation “Insert Brian Williams Joke Here” he brings up a very important point for leaders to try to make an impact using “the latest research.” Before you do something embarrassing, this is worth a read:
We’ve all done it. You’re looking for a statistic for a presentation, a strategic plan, a speech, a marketing model, or something else. You do a quick search and find just what you need in an article or on a website. It seems completely reasonable. It fits what you need. So you plug it in and move on to other things.
The problem is, very often the survey findings, statistics, projections, and other numbers we see in the media have been misreported or incompletely/inaccurately described. I’m not just bashing on the media here – I’ve had people quote me statistics from non-media sources that I can quickly disprove with a little digging. I’ve even had statistics from my own studies quoted back to me…quite incorrectly.
Grey Matter Research has had our study findings quoted by hundreds of media outlets from USA Today to Christianity Today. It amazes me just how often the media tell only part of the story (often in a way that is misleading), or flat out mis-report the findings. I’m not accusing the media of intentional bias (although I’m sure that does happen), but of having limited time/space, having limited (or no) understanding of research, and often having a specific angle from which they want to approach a story and then needing to find statistics or research findings that fit that angle.
The media is also out to attract readers/listeners/viewers. Consider a hypothetical study that finds 15% of Americans support the Obama Administration’s treaty with Iran, 30% oppose it, and 55% have no opinion. Consider three different headlines about the research:
• Americans Oppose New Treaty 2-to-1
• Only a Minority of Americans Oppose New Treaty
• Most Americans Undecided on New Treaty
All three of those headlines accurately reflect the study findings, even if each one tells just a portion of the total story.
Sometimes it’s not just the media’s fault. I recall one news story about a study claiming that a huge proportion of “southern evangelicals” favored the use of torture in the interests of national security. The article never defined how “evangelical” was defined, nor what “southern” or “torture” meant. So I found the original study to do a little digging of my own, but it too never defined those terms, rendering the study largely meaningless.
Those definitions are tricky little beggars. Depending on who’s doing the research, evangelicals represent 35%, 22%, or 7% of the U.S. population. It all hinges on how the researchers define “evangelical” (which usually isn’t stated in those news reports).
According to the Guttmacher Institute, 79% of unmarried evangelical Millennials have been sexually active (about the same as Millennials in general). According to a Grey Matter study done for the National Association of Evangelicals, the number is 44%. The studies used very different definitions of “evangelical.” The difference between those two numbers would have a major impact on a sermon, ministry plan, or strategy document about young adult sexuality.
Too often, the media do very little to vet research that has been conducted before reporting on it. I’ve seen one particular “study” about Millennials covered extensively in the media, and some of the findings struck me as a bit odd. When I dug into it, it turned out the “study” was done mostly through the alumni associations of about ten different Midwestern colleges (which led to their finding that 95% of Millennials held a college degree).
Although I would really like to see things improve, I sincerely doubt that will happen (particularly as traditional media outlets continue to cut their budgets, combined with the fact that anyone with a keyboard and an ISP can be “a media outlet”). So the onus is on you, the user.
Bottom line: Before you plug that key finding into your next marketing plan, speech, or background document, do a little research of your own. Go to the original source of the research and find out how the research was really done and what the study really found. Avoid mis-informed decisions and embarrassment by doing some fact checking.
Believe me, it’s worth the time investment.