A Retrospective Analysis of Ophthalmology Triage Accuracy: Comparing the Performance of Ophthalmologists, Specialist Nurses, Staff Nurses, and an AI Triage System

Abstract

Purpose: To evaluate and compare the efficiency and accuracy of telephone triage of unscheduled eye care in relation to their respective roles (ophthalmologists, specialist nurses, staff nurses, and an AI system). Methods: In this retrospective study, we evaluated 2581 unscheduled eye care telephone triage forms from Singleton Hospital between 2022 and 2024. The four main data cohorts were ophthalmologists (929 forms), specialist nurses (900 forms), staff nurses (500 forms), and an AI system (ChatGPT-4O), which evaluated 252 clinical details. The primary outcome measures were adherence to “traffic-light” guidelines, rates of case escalation and de-escalation, and final triage outcome (appointment/advice, etc.). Results: There was no statistical difference in the percentage of adherence to the “traffic-light” guidelines between the three human groups. Specialist and staff nurses were significantly more likely to escalate a case (7% and 8%, respectively, compared to 4% for ophthalmologists); however, this resulted in no additional out-of-hours work. Ophthalmologists had the highest rate of accurate diagnostics from provisional to actual diagnosis being 46%, and managed the largest number of triage calls. Both specialist and staff nurses stated they were stressed by the triage process and also that casualty doctors frequently interrupted them while triaging, much less so for ophthalmologists. The AI system showed some promise; however, it needed further refinement. Conclusion: Although the different staff roles employed for telephone triage produce comparable patient outcomes, there are significant differences in terms of case escalation, diagnostic accuracy, and the impact on the clinic’s workflow. The amount of stress both specialist and staff nurses report experiencing with respect to triage, and the frequency of interruption by casualty doctors, demonstrate a need to optimize the system. The use of AI as a tool for future risk stratification appears promising; however, it will require additional development.