Applying an evaluation framework for health information system design, development, and implementation

BACKGROUND: Dramatic advances in health information technologies necessitate a comprehensive evaluation framework covering all phases of development, from conception to routine operational use. OBJECTIVES: The purpose of this article was to illustrate the application of an informatics evaluation framework that provides a heuristic for matching the stage of system design and the level of evaluation. METHODS: An evaluation framework is illustrated in the context of five studies in different stages of system design. RESULTS: The studies discussed in this article represent the various stages of the framework. In addition, they are examples of how evaluation research studies not only contribute to the assessment of system design but also constitute distinct contributions to knowledge. DISCUSSION: The ability to engineer advanced information systems has exceeded the understanding of how to deploy them effectively in complex settings and to adapt them to a range of user populations. A systematic, continuous evaluation increases the likelihood that the conditions for success will be understood, including how to tailor a system, how to maintain the target population's use of the system, and how to innovate continually to enhance the functionality of the system and the users' experience. Without sound evaluation methodologies throughout the stages of system development, information systems have limited potential to influence healthcare processes positively.