Thereâ€™s quite a bit of talk floating around the internet about a recent article in JAMIA that looks at reduction of medication errors in hospitals secondary to CPOE adoption (J Am Med Inform AssocÂ doi:10.1136/amiajnl-2012-001241). The article is available for free so I read through it last weekend. By the end I was looking at something that wasn’t all that impressive. The authors use a lot of sleight of hand, i.e. statistical models to tell a story about how CPOE â€œdecreases the likelihood of error on that order by 48%â€, which ultimately could potentially lead to a reduction in medication errors by approximately 12.5%â€. That would be great, except that the entire thing is based on statistical models, assumptions, survey data and a great big meta-analysis.
Anyone thatâ€™s been in healthcare for more than 10 minutes and taken a basic statistics course knows about the use of a meta-analysis to tell a story. Basically you read the paper, cut their findings in half, reduce that outcome by 25% and then take the remainder with a grain of salt. I remember being at UCSF in pharmacy school and being scolded for using any paper whose results depended on a meta-analysis. My statistics professor at UCSF used to say â€œgarbage in equals garbage outâ€ when talking about their use.
Overall itâ€™s a well written paper, but I never thought the medical community would latch onto it with such fervor. It’s become a CPOE supporters wet dream. I have to wonder how many of my colleagues took the time to read through it before Tweeting or re-Tweeting a link to it. Thatâ€™s one of the problems with social media, people see something role across their feed and they pass it on without taking a deeper look at it first. Or that’s what I assume. I suppose there are those that simply donâ€™t care that the article is suspect, as long as it supports the idea that CPOE is good. I believe CPOE is good. I believe it should be part of every healthcare system. But I also believe you have to be careful with the information you use to support the cause.
Ask yourself these questions before you read, or re-read the article:
- Where did the information come from?
- How old is the information?
- Is it a prospective randomized study with cross-over?
- Was direct observation used?
- Does it seem possible that CPOE alone can decrease the likelihood of a mistake by 48%, and decrease medication errors by 12.5%?
- What systems did they review?
- We all know how CPOE works, what mechanism would create such a drastic fall in errors? CPOE is akin to a pharmacy information system (PhIS). Do PhISs decrease errors in the pharmacy by 50%? Why not? What’s different?
- Are the results consistent with other CPOE studies? If not, whatâ€™s different?
Think about it people. Use common sense. Use your God given scientific knowledge. CPOE is good for healthcare, but be careful about using any olâ€™ thing to promote it. Make sure the sword you’re swinging isn’t made out of plastic.