The rise in EHR adoption has brought with it a 21st-century headache–alert fatigue. Every day clinicians deal with numerous medication-related alerts, such as allergies, drug interactions, and duplicate medications. Making matters worse is the fact that many alerts are clinically insignificant, causing cognitive overload and workflow disruptions, which could result in lower quality care. Faced with an overwhelming number of alerts, clinicians tend to override many of them, which raises patient safety concerns. Addressing alert fatigue, while being mindful of patient safety, requires more data concerning both why clinicians override alerts, and whether overridden alerts are potential safety breaches. Nanji et al. have provided an excellent analysis of both areas (1).
Overrides of Medication-related Clinical Decision Support Alerts in Outpatients provides a detailed analysis of over 150,000 alerts generated from more than 2 million orders. Alerts were categorized and analyzed for appropriateness. Results are reported as follows.
We reviewed 157 483 CDS alerts (7.9% alert rate) on 2 004 069 medication orders during the study period. 82 889 (52.6%) of alerts were overridden. The most common alerts were duplicate drug (33.1%), patient allergy (16.8%), and drug–drug interactions (15.8%). The most likely alerts to be overridden were formulary substitutions (85.0%), age-based recommendations (79.0%), renal recommendations (78.0%), and patient allergies (77.4%). An average of 53% of overrides were classified as appropriate, and rates of appropriateness varied by alert type (p<0.0001) from 12% for renal recommendations to 92% for patient allergies.
The authors report the average rate of appropriate overrides (53%) across nine categories; however, that statistic hides more interesting findings. The rate of inappropriate overrides varied significantly across all categories. I have sorted the findings by inappropriate override percentage (red) highest to lowest.
Drug–drug interaction (12/88)
Renal suggestion (12/85)
Age-based suggestion (39/60)
Formulary substitution (57/43)
Class–class interaction (69/31)
Duplicate drug (82/18)
Drug–class interaction (88/12)
Patient allergy (92/8)
Here is the authors’ take on their findings.
Half of the overrides were appropriate, and the proportion that was appropriate varied even more by alert type. Few overrides of renal dosage recommendations, drug–drug interactions, or age-related dosage recommendations were appropriate, while the vast majority of drug allergy, drug–class, duplicate drug, and drug formulary alerts were, suggesting that this last group might be particularly good targets for refinement to reduce alert fatigue by reducing the number of alerts. Notably, the appropriateness of overrides of drug–class and class–class alerts was much higher than that of individual drug–drug alerts. This could indicate that while providers generally agree with alerts based on larger categories of drugs and their potential interactions, they may feel that exceptions do exist for individual drugs. However, the high rates of inappropriate overrides for these individual drug–drug interactions indicate the need for further education and intervention.
Assuming these findings have wide applicability, it seems that serious patient safety issues are lurking. Drowning in a sea of alerts, clinicians are overriding potentially significant notifications.
Unlike Nanji et al., Genco and colleagues found that opioid-related alerts in emergency departments were largely inconsequential to the point where they were mostly noise (2). Here are their findings.
Results: Opioid drug alerts were more likely to be overridden than nonopioid alerts (relative risk 1.35; 95% confidence interval [CI] 1.21 to 1.50). Opioid drug-allergy alerts were twice as likely to be overridden (relative risk 2.24; 95% CI 1.74 to 2.89). Opioid duplicate therapy alerts were 1.57 times as likely to be overridden (95% CI 1.30 to 1.89). Fourteen of 4,581 patients experienced an adverse drug event (0.31%; 95% CI 0.15% to 0.47%), and 8 were due to opioids (57.1%). None of the adverse drug events were preventable by clinical decision support. However, 46 alerts were accepted for 38 patients that averted a potential adverse drug event. Overall, 98.9% of opioid alerts did not result in an actual or averted adverse drug event, and 96.3% of opioid alerts were overridden.
Conclusion: Overridden opioid alerts did not result in adverse drug events. Clinical decision support successfully prevented adverse drug events at the expense of generating a large volume of inconsequential alerts. To prevent 1 adverse drug event, providers dealt with more than 123 unnecessary alerts. It is essential to refine clinical decision support alerting systems to eliminate inconsequential alerts to prevent alert fatigue and maintain patient safety.
Somewhere between these two extremes lies the sweet spot for guarding patient safety without driving clinicians up a wall.
There are two basic problems: 1) preventing alerts that have no clinical value, and 2) preventing alert overrides that may result in patient harm. Clearly, the clinical value of alerts depends on the type. ED opioid alerts could be reduced significantly without creating major safety issues. Renal alerts, conversely, seem to be just the opposite. So, what to do???
Simpao et al. illustrate one possible approach—local adjustments to drug alerting rules (3). The researchers describe a series of changes to drug-drug interaction rules that resulted in fewer clinically-insignificant alerts. Here is a description of the initial conditions.
The hospital implemented a new inpatient EHR (Epic Systems, Verona, Wisconsin, USA) in January 2011. Based on recommended interaction severity and documentation level filtration settings, approximately 3549 of the available 4033 DDI alert rules from a third-party vendor (Medi-Span, Indianapolis, Indiana, USA) were implemented for pharmacists along with the new EHR, while a substantially fewer 1351 rules were implemented for inpatient providers. Within each DDI described by the vendor is a set of rules that specify the medications that are included in that particular class as well as the formulations and routes. Many of these rules are dynamic, subject to periodic updates by the database vendor. After the implementation, care providers and especially pharmacists experienced a sharp increase in the number of medication alerts, some categories of which were overridden more than 90% of the time.
Alerting behaviors for clinicians and pharmacists are outlined as follows.
In our system override reasons are required for maximum dose and drug allergy alerts, but not for DDI and duplicate medication alerts.
Prescribers have all alert actions available to them. Within the alert window, a prescribing clinician has the option to ‘remove’ or discontinue medications contributing to the DDI, maximum dose, or duplicate alert. Alerts are also specific to the ordering session, so should the same alert be triggered for the same patient or prescriber at a later date in a separate session, it would fire regardless of what actions were taken previously.
In the pharmacists’ workflow, local scope of practice does not allow pharmacists to enter or discontinue orders, so their actions are limited to override, view, and cancel. In addition, if interacting medications are being verified in the same session, a pharmacist will only see a DDI alert once for the pair. Pharmacists are also able to see the actions taken by prescribers when the alert appears in the verification process.
Finally, the researchers describe the process used for altering alert rules.
Using the dashboard to identify the most frequently triggered DDIs, a group of 10 pediatric clinical pharmacists then conducted a comprehensive, literature-based review of select additional DDIs to determine if removal was warranted. The clinical pharmacists were divided into pairs; each pair was assigned a different set of DDIs to review. Each team member independently reviewed the Medi-Span DDI monograph; this included a list of offending agents, the mechanism of interaction, management options to mitigate the interaction, and supporting literature. The clinical pharmacists then determined the clinical relevance of each DDI. Due to contradictions noted between available tertiary references and third-party DDI vendors, each Medi-Span monograph’s severity level and proposed management strategy was then compared to Lexi-Comp (Lexi-Comp, Hudson, Ohio) and Micromedex (Thomson Micromedex, Ann Arbor, Michigan, USA).9 When necessary, primary literature was reviewed to aid in assessing the clinical relevance of specific DDIs. If a DDI alert was deemed to be clinically insignificant, it was recommended to deactivate the alert rule so that the alert no longer fired. Override rates did not factor into removal decisions due to the presumed state of alert fatigue.
The intervention did lower the alert rate for clinicians; however, is this approach ideal for addressing alert fatigue at every hospital or practice? The process outlined is labor-intensive and potentially subject to changes in third-party databases (structure, rules, or content). Here is the question that intrigues me: Is this a software design problem or a database/rules problem? One reason for my questioning the source of the problem is this statement: “Alerts are also specific to the ordering session, so should the same alert be triggered for the same patient or prescriber at a later date in a separate session, it would fire regardless of what actions were taken previously.”
The software design seems to be one that grabs drug information from a database, makes use of it, then “forgets” the result. That is, the EHR software does not have a computational representation of the patient’s state or the prescriber’s behavior. The process history is not retained in a usable form. Seemingly, only the final results of the prescriber’s actions are captured in the form of the patient’s medication list. Given the use of third-party databases, I assume this is the design approach taken. I have wrestled with this issue up close…
The UAB 1917 Clinic EHR system provided drug alerts via access to a licensed drug database. All drug information was stored in relational tables. Drug-drug interactions were stored using many-to-many relations. When a drug was selected for a patient, a query was done in the EHR to grab the patient’s current medications, and that list was checked against the tables (drug-drug, drug-class, drug-food, etc.). Positive hits returned alerts that were shown to clinicians.
From a design perspective, one way to lower alert frequency would be to create a patient state object that dynamically managed medications. Such an object could store drug-alert history (or any other type of process information) that could be used for decision support. Stateless software designs—those that keep no in-memory image of the patient (or prescriber)—rely on database queries to provide information to EHR users. Patients and prescribers exist solely as collections of data elements. The EHR has no understanding or knowledge of what a patient is or what a prescriber is as it populates the user interface. Based on my experience and what I am reading in the literature, stateful designs would be a much better way to reduce alert fatigue AND track overrides by type, patient, and prescriber. Further, stateful designs might also make the efforts of those who are attempting to address this issue less costly and more productive.
To some extent, drug alert fatigue is a usability issue (4). After all, any software design choice has the potential to become a usability issue for some subset of users. But usability solutions that deal with surface/presentation changes alone will not solve the underlying problem. EHR systems that offer anything beyond basic decision support would be better served with stateful designs. If EHRs are going to offer advice, they will need computational images of system actors and not just access to queriable data stores.
- Nanji KC, Slight SP, Seger DL, Cho I, Fiskio JM, Redden LM, Volk LA, Bates DW. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc. 2014 May-Jun;21(3):487-91.
- Genco EK, Forster JE, Flaten H, Goss F, Heard KJ, Hoppe J, Monte AA. Clinically Inconsequential Alerts: The Characteristics of Opioid Drug Alerts and Their Utility in Preventing Adverse Drug Events in the Emergency Department. Ann Emerg Med. 2016 Feb;67(2):240-248
- Simpao AF, Ahumada LM, Desai BR, Bonafide CP, Gálvez JA, Rehman MA, Jawad AF, Palma KL, Shelov ED. Optimization of drug-drug interaction alert rules in a pediatric hospital’s electronic health record system using a visual analytics dashboard. J Am Med Inform Assoc. 2015 Mar;22(2):361-9.
- Payne TH, Hines LE, Chan RC, Hartman S, Kapusnik-Uner J, Russ AL, et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc. 2015 Nov;22(6):1243-50.