281 THE INTER-RATER RELIABILITY OF THE RISK INSTRUMENT FOR SCREENING IN THE COMMUNITY (RISC): PRE AND POST TRAINING

Background Predicting risk of adverse healthcare outcomes is important to enable targeted delivery of early interventions. The Risk Instrument for Screening in the Community (RISC) measures the one-year risk of hospitalisation, institutionalisation and death in community-dwelling older adults and can be used to triage patients according to a five-point global risk score: Low (score 1, 2), medium (3) to high (4, 5). While validated, the effect of a standardised “RISC” training programme on its reliability is yet to be assessed. This study aimed to examine the inter-rater reliability (IRR) of the RISC before and after the RISC training programme for healthcare professionals.

Methods A descriptive correlational design was used. A sample (n = 32) of student Public Health Nurses (PHNs) scored six clinical cases (two low, medium and high-risk clinical scenarios), before and after RISC training. Participant scores were compared with expert scores for the same cases. IRR was measured with Gamma and Spearman correlations and Kappa statistics. Internal consistency was assessed using Cronbach's alpha.

Results The correlation between PHNs and experts RISC scores increased for each adverse outcome, pre and post training, and was statistically significant for institutionalisation (r = 0.72 to 0.80, p = 0.04) and hospitalisation, (r = 0.51 to 0.71, p < 0.01) but not death. The proportion of matches between PHNs and expert raters as a marker of agreement increased for all three outcomes. Examining risk levels separately showed that training improved agreement for low-risk but not all high-risk cases. When a paired-samples design was used, agreement reduced significantly post-training for cases deemed high-risk of institutionalisation (92% versus 75%, p = 0.01) and hospitalisation (88% versus 71%, p = 0.01).

Conclusion Overall, the RISC showed good IRR, which increased after RISC training. That reliability reduced for some high-risk cases suggests that the training programme requires adjustment to further improve IRR.