Development of the Liang Handover Assessment Tool for Simulation (L-HATS)
INTRODUCTION: Clinical handovers are critical to patient safety and outcomes. Handover simulation prepares healthcare students for handoffs in the clinical setting upon graduation. UT Southwestern has developed a longitudinal handover educational curriculum in which student handovers will be assessed. Although valid and reliable tools exist for assessing clinical handovers, assessment tools adapted for the undergraduate simulation environment currently do not exist. Our objective was to develop a reliable and valid assessment tool that could be used by scholarly healthcare students to assess undergraduate simulated handovers throughout the longitudinal handover education curriculum. METHODS: A literature review was conducted to identify critical elements of high-quality, effective handovers. Following the tool's creation, we underwent several PDSA cycles to optimize the tool for medical student evaluation and ease of grading. Grader inclusion criteria were students who had completed the transition to clerkship (T2C) handover activity. A training curriculum was developed to train graders on proper use of the tool and to promote reliable grading with the tool. 62 pre-clinical student handovers were conducted in the simulation setting and recorded. The handovers were stratified into three levels (low, intermediate, and high quality), and 10 handovers were selected from each of the three levels for grading (30 handovers total). Each handover was scored by four clerkship medical students "graders". Two-way random effects intra-class correlation coefficients (ICC) were used to establish inter-rater reliability and inter-rater agreement among graders using the tool. Three external handover experts were used to establish the tool's validity using face validity. RESULTS: The product of this project is Liang Handover Assessment Tool for Simulation (L-HATS) which evaluated three domains: handover content, handover process, and language with a maximum score of 28. Two-way random effects ICC for agreement was 0.804, 95% CI [0.601, 0.906]. Two-way random effects ICC for reliability was 0.866, 95% CI [0.765, 0.930]. Three external handover experts have sufficiently validated the tool. CONCLUSIONS: The L-HATS had good to excellent inter-rater reliability and agreement. The L-HATS is the first reliable and valid handover assessment tool used for undergraduate simulation education. By using a two-way random effects model, the results suggest that the tool can be used in settings outside of the T2C handover simulation activity. Having good to excellent absolute agreement suggests that the tool is suitable for assigning grades. Future studies include comparing faculty vs student grading of handovers as well as evaluating the tool in the clinical setting.