Computing eye gaze metrics for the automatic assessment of radiographer performance during X-ray image interpretation

Aim To investigate image interpretation performance by diagnostic radiography students, diagnostic radiographers and reporting radiographers by computing eye gaze metrics using eye tracking technology. Methods Three groups of participants were studied during their interpretation of 8 digital radiographic images including the axial and appendicular skeleton, and chest (prevalence of normal images was 12.5%). A total of 464 image interpretations were collected. Participants consisted of 21 radiography students, 19 qualified radiographers and 18 qualified reporting radiographers who were further qualified to report on the musculoskeletal (MSK) system. Outcome measures Eye tracking data was collected using the Tobii X60 eye tracker and subsequently eye gaze metrics were computed. Voice recordings, confidence levels and diagnoses provided a clear demonstration of the image interpretation and the cognitive processes undertaken by each participant. A questionnaire afforded the participants an opportunity to offer information on their experience in image interpretation and their opinion on the eye tracking technology. Results Reporting radiographers demonstrated a 15% greater accuracy rate (p ≤ 0.001), were more confident (p ≤ 0.001) and took a mean of 2.4s longer to clinically decide on all features compared to students. Reporting radiographers also had a 15% greater accuracy rate (p ≤ 0.001), were more confident (p ≤ 0.001) and took longer to clinically decide on an image diagnosis (p = 0.02) than radiographers. Reporting radiographers had a greater mean fixation duration (p = 0.01), mean fixation count (p = 0.04) and mean visit count (p = 0.04) within the areas of pathology compared to students. Eye tracking patterns, presented within heat maps, were a good reflection of group expertise and search strategies. Eye gaze metrics such as time to first fixate, fixation count, fixation duration and visit count within the areas of pathology were indicative of the radiographer's competency. Conclusion The accuracy and confidence of each group could be reflected in the variability of their eye tracking heat maps. Participants’ thoughts and decisions were quantified using the eye tracking data. Eye tracking metrics also reflected the different search strategies that each group of participants adopted during their image interpretations. This is the first study to use eye tracking technology to assess image interpretation skills between various groups of different levels of experience in radiography, especially on a combination of the MSK system, chest cavity and a variety of pathologies.