Simulation performance evaluation: Inter-rater reliability of the DARE²-patient safety rubric

Background: The DARE²-patient safety rubric was developed for the performance evaluation of final year nursing students. The rubric contains four domains of competency: systematic patient assessment, clinical response, clinical-psychomotor skills, and communication proficiency. The aim of this research was to investigate the inter-rater reliability of data from the DARE². Method: A nonexperimental quantitative exploratory design was employed. Archived recorded performances of students (n=34) were independently evaluated by nurse lecturers who teach and examine in the simulation centre (n=4). Results: Intraclass correlation coefficients (ICC) were greater than 0.70 for three of the four domains of practice and 0.58 for the fourth (clinical-psychomotor skills). An ICC of 0.75 for the overall rubric score is indicative of excellent reliability. Percentage agreement for the overall rubric was 59%. Conclusion: These results support the inter-rater reliability of data from the DARE - patient safety rubric and highlights the difference between consensus and consistency estimates of inter-rater reliability.