Health Versus Hype Project:
January’s Most Over- and Under-Hyped Medical Research Studies
Americans rely on the news media to accurately report on new health-related research. But the media often gives credence to dubious or inconclusive studies while ignoring credible research. Each month, CivicMeter identifies the most over- and under-hyped research studies in the realm of nutrition and health.
Most Over-Hyped Study: Tea Drinking Boosts Heart Health
A recent study examined whether regular tea drinking reduced the likelihood of heart disease. The study, published in January 2020 in the European Journal of Preventive Cardiology, touted the health benefits of drinking tea. It quickly garnered coverage in over 300 media outlets. But as the most over-hyped study in January, there was a disconnect between the media coverage this study received and the significance of the research.
First Disconnect: Strength of Conclusions
The study authors drafted measured and restrained conclusions. However, headlines and leads in news stories celebrated more confident, overstated versions of the paper’s conclusions.
 |
Second Disconnect: Weaker Study Fundamentals
When evaluated based on fundamentals of medical studies with strongly supported evidence and conclusions, the tea study lagged its peers.
The methods and results of the study diverged from what would be expected for a first-tier medical study in terms of strength of conclusions and thoroughness of methods.
 |
Third Disconnect: Good PR, Average Scientific Rigor
The third measure of hype to scientific credibility is the strength of the study and its authors compared to peer publications and researchers.
In this case, the popularity of the article in non-professional media overtook the credibility of the authors and the study among scientific peer publications. CivicMeter used quantitative measures of the article and authors based on the strength of the publication within its academic community, as well as the collective strength and expertise of its authors.
The study scored well in general popularity, but lower in research respectability.
Of the studies published over the past three months that focused on the health effects of a particular food, this study was especially popular, and especially weak when compared with other studies. The study may merit attention. However, its relative credibility to peer research simply did not warrant its over-exposure compared with other more scientifically rigorous or significant research.
Most Under-Hyped Study: Vegetable Fats Reduce Liver Cancer Risk
The most under-hyped study of the season contained findings that vegetable fats, compared with animal fats, reduced incidence of a common form of liver cancer.
The study, published in January 2020 in Clinical Gastroenterology and Hepatology, followed 138,483 women and men for an average of 26 years. The study followed food choices as well as disease incidence.
CivicMeter used the same quantitative measures as with the other review of study of tea consumption. This includes measures of influence and prestige of authors and their institutions, along with objective measures of the impact of the article in its scientific field.
This vegetable consumption study scored well scientifically, but lagged in the popularity contest with other medical articles of the last several months.
[CHART HERE]
Despite being scientifically credible, this study was barely covered by media outlets.
_________
Background of the Health Versus Hype Project
Medical studies that may have been ignored in past decades are now the darling of online and traditional media. The latest medical study on a particular diet, food or lifestyle choice now receives national and international attention. The main authors who have toiled for years in a barren laboratory become instantly well-known in the media.
This new attention begs the question: Is the scientific value living up to the media hype?
Over the past few years, researchers and journalists have identified a possible disconnect between the media hype and the medical reality of these studies. Certain journalists and bloggers have looked more critically at the source, conclusions and methods of the studies to assess whether these studies are deserving of the adulation they receive by the mass-media.
CivicMeter has taken this critical review one step further. As a policy center for the use and promotion of data and analytics in policy and public administration, the Health Versus Hype Project uses quantitative methods and rigorous review of analytic methods to rate and rank these studies. We also use measures of media attention to rank the popularity of studies. Sometimes the hype matches the health ratings of the articles. Often, however, there are wild disconnections between the valued perceived by the mass media versus the value perceived by the peer scientific community. Furthermore, the media often ignores red flags associated with weaker studies.
Beginning in 2019, CivicMeter began compiling a database of these studies and their receptions in online and traditional media.
We welcome your comments and feedback as we continue our review of research in 2020. Our database will only grow, and our analysis enriched as we continue our quest to apply objective analytics important matters of public management and policy.
Please contact CivicMeter Editor-in-Chief, Ted Ballantine, with any thoughts or suggestions.
Email: EBallantine@CivicMeter.com
