Decreased productivity is one of the most often complained-about side effects of EHR adoption. Documentation time usually leads the pack in terms of complaints. One way to avoid “buyer’s remorse” when choosing an EHR is to conduct extensive product testing before signing a contract. Of course, for busy clinicians, this is easier said than done. Testing presents two major hurdles for most practices–the time required and the need to develop a reliable test script.
Performing an adequate test of a user interface in a structured manner can take two hours or more. This seems like a lot unless one has had the misfortune of implementing an EHR that he or she later came to hate. There is no shortcut possible with the time issue. However, the recently released usability framework document, Technical Evaluation, Testing, and Validation of the Usability of Electronic Health Records, offered by NIST, can help a great deal with creating a test script.
This document contains two groups of checklists for EHR usability evaluation. The first focuses on EHR user interface design features that may engender or allow errors during care delivery. The second group addresses more general interface design issues. When combined, the two groups provide 195 items that can be very useful during EHR selection.
Data accuracy is one of the eight error-group items. This type of error is described as:
Displayed data are not accurate. For example, a physician ordered the wrong dose of a medication because the amount of the medication dose was truncated in the pick list menu display.
Some items from the data accuracy checklist are displayed below.
1C.1 Is information not truncated on the display (e.g., medication names and doses in pick list menu displays are accurate and complete and distinguishable from other items in the pick list)?
1C.2 Does accurate information automatically display (e.g., without requiring an active refresh command by the user)?
1C.3 Can inaccurate information be easily changed (e.g., allergies)?
1C.4 When a medication is renewed and then the dose is changed before signing, is the correct information displayed?
1C.5 Do changes in status (e.g., STAT to NOW) display accurately?
1C.6 If a medication schedule is changed, does the quantity correctly update?
1C.7 If a medication order is discontinued, is the information updated on all displays about the change?
These items provide information that can be rendered into test script elements. For example, 1C.1 and 1C.2 can be converted into specific evaluation elements as illustrated below.
- Lookup longer-than-usual drug names in the prescription writer. Do they appear on screen in their entirety? (1C.1)
- Enter allergy to penicillin. Will the system allow one to indicate this is an error and remove it from the patient’s record? (1C.2)
The entire set of 195 checklists items may be used in this manner.
The report also includes a few test scenarios that are intended as templates for end-user EHR testing. Each scenario contains a set of tasks designed to test various elements of the EHR user interface. By adding more detail to the NIST skeleton, clinicians who wish to do more extensive testing of their EHRs can create robust test scripts that will identify many common interface design flaws. If you wish to try your hand at creating test scripts and understanding how they help in discovering potential workflow issues, please read the series of posts “Preventing Your EHR from Working Against You.”
The best way to avoid buyer’s remorse is to give your EHR a meaningful test drive before signing on the dotted line. The NIST usability evaluation tools are great resources for designing quality test scripts. Yes, testing can take time, but the hours spent testing may save you years of misery.