Institute:Office of National Coordinator (ONC) Workforce Training Curriculum
Component:The Culture of Health Care
Unit:Evidence-Based Practice
Lecture:Summarizing evidence
Putting evidence into practice
Slide content:Limitations of Systematic Reviews Not everyone accepts use of meta-analysis; Feinstein (1995) calls it statistical alchemy Meta-analyses on same topic sometimes reach different conclusions due to methodologic differences ( Hopayian , 2001) Truth determined by meta-analysis has the shortest half-life of all knowledge ( Poynard et al., 2002) Effect of publication bias may be exacerbated in systematic reviews ( Dickersin , 1997; Dwan , 2013) 13
Slide notes:13 Of course, there are some limitations to systematic reviews and summaries of evidence. Some people believe that the use of meta-analysis is misguided. Alvin Feinstein is a well-known epidemiologist who has written in many places about his concern with meta-analysishe has called it statistical alchemy. We do sometimes see meta-analyses performed on the same topic, using many of the same studies, but reaching different conclusions for a variety of methodologic [meth-uh-dl- oj - ik ] reasons. One study looked at the so-called half-life of knowledge, or how quickly knowledge became overturned. The domain of liver disease and meta-analysis actually had the shortest half-life, so when something was found to be the truth by meta-analysis, that truth lasted a shorter period of time than something discovered in a randomized controlled trial. Of course, publication bias may be exacerbated in systematic reviews because systematic reviews, in essence, are a sampling of studies, and they represent the spectrum of research done on a given topic. If theres publication bias, then the systematic reviews are going to be more compromised because they rely on information being appropriately published. When theres publication bias, it may lead us to draw incorrect conclusions from systematic reviews.