TY - JOUR
T1 - Assessing performance of an Electronic Health Record (EHR) using Cognitive Task Analysis
AU - Saitwal, Himali
AU - Feng, Xuan
AU - Walji, Muhammad
AU - Patel, Vimla
AU - Zhang, Jiajie
N1 - Funding Information:
This study was supported in part by US Army Telemedicine Advanced Technology Research Center ( W81XWH-08-2-0025 and W81XWH-07-2-0108 ). We would also like to thank Ron Gimbel for providing valuable comments; Brian Bowers for providing the list of tasks for task analysis; and the program officer, Nancy Connolly, for her support.
PY - 2010/7
Y1 - 2010/7
N2 - Background: Many Electronic Health Record (EHR) systems fail to provide user-friendly interfaces due to the lack of systematic consideration of human-centered computing issues. Such interfaces can be improved to provide easy to use, easy to learn, and error-resistant EHR systems to the users. Objective: To evaluate the usability of an EHR system and suggest areas of improvement in the user interface. Methods: The user interface of the AHLTA (Armed Forces Health Longitudinal Technology Application) was analyzed using the Cognitive Task Analysis (CTA) method called GOMS (Goals, Operators, Methods, and Selection rules) and an associated technique called KLM (Keystroke Level Model). The GOMS method was used to evaluate the AHLTA user interface by classifying each step of a given task into Mental (Internal) or Physical (External) operators. This analysis was performed by two analysts independently and the inter-rater reliability was computed to verify the reliability of the GOMS method. Further evaluation was performed using KLM to estimate the execution time required to perform the given task through application of its standard set of operators. Results: The results are based on the analysis of 14 prototypical tasks performed by AHLTA users. The results show that on average a user needs to go through 106 steps to complete a task. To perform all 14 tasks, they would spend about 22 min (independent of system response time) for data entry, of which 11 min are spent on more effortful mental operators. The inter-rater reliability analysis performed for all 14 tasks was 0.8 (kappa), indicating good reliability of the method. Conclusion: This paper empirically reveals and identifies the following finding related to the performance of AHLTA: (1) large number of average total steps to complete common tasks, (2) high average execution time and (3) large percentage of mental operators. The user interface can be improved by reducing (a) the total number of steps and (b) the percentage of mental effort, required for the tasks.
AB - Background: Many Electronic Health Record (EHR) systems fail to provide user-friendly interfaces due to the lack of systematic consideration of human-centered computing issues. Such interfaces can be improved to provide easy to use, easy to learn, and error-resistant EHR systems to the users. Objective: To evaluate the usability of an EHR system and suggest areas of improvement in the user interface. Methods: The user interface of the AHLTA (Armed Forces Health Longitudinal Technology Application) was analyzed using the Cognitive Task Analysis (CTA) method called GOMS (Goals, Operators, Methods, and Selection rules) and an associated technique called KLM (Keystroke Level Model). The GOMS method was used to evaluate the AHLTA user interface by classifying each step of a given task into Mental (Internal) or Physical (External) operators. This analysis was performed by two analysts independently and the inter-rater reliability was computed to verify the reliability of the GOMS method. Further evaluation was performed using KLM to estimate the execution time required to perform the given task through application of its standard set of operators. Results: The results are based on the analysis of 14 prototypical tasks performed by AHLTA users. The results show that on average a user needs to go through 106 steps to complete a task. To perform all 14 tasks, they would spend about 22 min (independent of system response time) for data entry, of which 11 min are spent on more effortful mental operators. The inter-rater reliability analysis performed for all 14 tasks was 0.8 (kappa), indicating good reliability of the method. Conclusion: This paper empirically reveals and identifies the following finding related to the performance of AHLTA: (1) large number of average total steps to complete common tasks, (2) high average execution time and (3) large percentage of mental operators. The user interface can be improved by reducing (a) the total number of steps and (b) the percentage of mental effort, required for the tasks.
KW - Cognitive Task Analysis
KW - Distributed cognition
KW - Electronic Health Records
KW - GOMS
KW - KLM
KW - UFuRT
UR - http://www.scopus.com/inward/record.url?scp=77952890815&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77952890815&partnerID=8YFLogxK
U2 - 10.1016/j.ijmedinf.2010.04.001
DO - 10.1016/j.ijmedinf.2010.04.001
M3 - Article
C2 - 20452274
AN - SCOPUS:77952890815
SN - 1386-5056
VL - 79
SP - 501
EP - 506
JO - International Journal of Medical Informatics
JF - International Journal of Medical Informatics
IS - 7
ER -