How does AI feedback impact student and qualified radiographers’ decision making? European Congress of Radiology 2023

Purpose: As artificial intelligence (AI) becomes more integrated into healthcare it is imperative to deploy this technology responsibly. This research attempts to add insight into the effect of forms of AI feedback on the interpretation accuracy of student and qualified radiographers, highlighting optimal aspects of AI feedback. Methods or Backround: A study was designed in Qualtrics®, deployed at ECR 2021 and promoted via social media (Twitter®/LinkedIn®), resulting in 94 respondents. Each participant was presented with a random selection of three musculoskeletal examinations from a dataset of 21. Diagnostic accuracy under four conditions (AI correct, AI incorrect, pathological and non-pathological examinations) was determined by comparison with a known ground truth diagnosis from three to five reporting radiographers and/or radiologists. ANOVA, t-tests and descriptive statistics were used to describe the data. Results or Findings: There is a statistically significant difference in diagnostic accuracy of all participants before any AI feedback and after provision of binary diagnosis, when the AI is correct (p=0.007). In pathological cases there was a statistically significant decrease in accuracy following presentation of the heatmap (p=0.015) and a subsequent increase in accuracy when the binary diagnosis was provided (p=0.013). Conclusion: Heatmap feedback was detrimental to participant accuracy in pathological cases and did not have a positive impact on accuracy in any condition. Binary diagnosis was beneficial to both groups across all of the four conditions with a significant impact on accuracy when the AI was correct and in pathological cases. Methods of explainability should be developed with cognisance of the impact on the end-user. Limitations: Participants were analysed as two broad groups. Impact of years of professional experience on interaction with the AI will be investigated. Ethics committee approval: This study was approved by the UU Nursing and Health research filter committee, approval number FCNUR-20-035. Funding for this study: Funding was received from the College of Radiographers Industry Partnership award Scheme (CoRIPS) and Ulster University