Eleven years ago, when I began working on the EHR project at the University of Alabama-Birmingham, I sought to do everything possible to assure that the project was a success. This led to an intensive search of the literature for case studies and reviews of IT projects. I discovered a wonderful paper: “Why Healthcare Information Systems Succeed or Fail” by Heeks, Mundy, and Salazar that I think remains one of the best reviews of this topic. The authors offer four broad categories of failure: total, partial, sustainability, and replication.
“In all, we can identify four main forms of HCIS failure:
The total failure of a system never implemented or in which a new system isimplemented but immediately abandoned. A much-reported example is that of the London Ambulance Service’s new computerised despatching system. This suffered a catastrophic failure within hours of implementation, leaving paramedics unable to attend health care emergency victims in a timely manner (Health Committee, 1995).
The partial failure of an initiative in which major goals are unattained or in which there are significant undesirable outcomes. Anderson (1997:87), for instance, cites the case of “An information system installed at the University of Virginia MedicalCenter [which] was implemented three years behind schedule at a cost that was three times the original estimate.”
The sustainability failure of an initiative that succeeds initially but then fails after a year or so. Some of the case mix systems installed under the UK National HealthService’s Resource Management Initiative fall into this category. They were made fully operational and achieved some partial use but with limited enthusiasm from staff for using them. Ultimately, they were just switched off (HSMU, 1996).
The replication failure of an initiative that succeeds in its pilot location but cannot be repeated elsewhere. Although presenters may not realise it at the time, every health informatics conference is jam-packed with replication failures about to happen; with wonderful innovations that are tested once and then disappear without trace. As an audience, we hear all about the pilot, but we tend not to hear about the replication failure.”
Over the years, I have used these classifications when studying reports of failed projects. In my experience, the partial failure label (i.e. the main goals set for an implementation are never attained) accurately reflects the outcome of EHR implementations in many smaller practices. Typical examples of unmet goals are: not all providers use the EHR; providers who use the EHR do not use key features (especially those related to quality measures); the lab interface never works; the practice management system-EHR interface is temperamental; or the paper-electronic hybrid system stays in place for more than a few months. Any of these situations may prevent a practice from realizing the clinical or economic benefits of having an EHR. Unfortunately, I have found that these issues occur far too frequently.
Surprisingly, these problems may persist for a year or more after the practice considers its implementation complete. This has made me quite skeptical about claims of EHR adoption in small practices. In fact, when someone tells me about a successful EHR implementation, my first question is: “Do all providers use the system for all patient care-related tasks supported by the EHR?” Rarely is the answer “yes”. When it is, often productivity has suffered. Repeatedly encountering these situations forced me to rethink my idea of what constitutes a “successful” implementation.
I now find that it is helpful to think of implementations in small practices as occurring in three stages.
Stage I begins with project initiation until go-live and includes budgeting, product selection, training and implementation.
The second stage covers the period from go-live until providers return to their pre-implementation workload. This assumes that productivity (without regard to quality issues) is a reasonable and desirable measure of EHR implementation success. However, I know of practices that have not returned to their pre-EHR productivity levels, even after more than two years. A research report from UC Davis published last year implies that there may be a specialty-specific productivity hit.
The most recent MGMA report also confirms the reality of practice productivity issues as well as operating cost concerns. The numbers for all respondents revealed that 38.4 % had increased operating costs and 30.6% noted decreased productivity. Of those practices that felt they had “optimized their EHRs,” 26.8% noted an increase in operating costs and 16.5% had a decrease in productivity. Considering the potential economic hit of increased costs and decreased productivity, is there any mystery why some practices have cold feet?
The last stage begins when physicians start to use the clinical decision support and population management functions of their EHRs as part of on-going quality initiatives. In my experience, the smaller the practice the less likely it is to reach Stage III. (In fact, small practices lag by nearly every measure of EHR adoption.) Those practices that do make it to Stage III often fail to sustain these activities without significant outside assistance. It will be interesting to see what impact meaningful use has in this area because attaining meaningful use, in essence, requires becoming skilled at quality improvement. Of course, this is not impossible, but it does require resources that many practices do not routinely have available.
The MGMA report demonstrates that the degree of implementation success varies greatly from practice to practice. The interesting question is why. Do we need better EHRs? Do practices require more “systems assistance” in order to maximize the benefits of EHR technology? Are practices that show increased cost and/or decreased productivity partial failures or simply real-world examples of the best that can be done with today’s technology and current implementation methods? Whatever the case, the push for greater adoption must be tempered by an acknowledgement of the potentially negative economic impact of an EHR. We need better predictive instruments to identify the factors that hinder productivity or significantly raise operating costs.
Clinical practices are businesses. EHR adoption must be shown to be a predictably good business investment; otherwise, many practices will remain reluctant to take the risk. Clearly, as the national discussion moves forward, we need an objective definition of what constitutes a successful implementation.