Inter-Rater Reliability of EyeSpy Mobile for Pediatric Visual Acuity Assessments by Parent Volunteers

Abstract

Purpose

To assess the inter-rater test reliability of the EyeSpy Mobile visual acuity smartphone algorithm when administered to children by eye professionals and parent volunteers.

Patients and Methods

Visual acuity test-retest results were analyzed for 106 children assigned to one of three different screenings: (1) An eye technician and pediatric ophthalmologist using their typical visual acuity testing method on a M&S computer; (2) An eye technician and pediatric ophthalmologist using EyeSpy Mobile; (3) An eye technician and parent volunteer using EyeSpy Mobile.

Results

All three phases demonstrated a strong agreement between the two testers, with mean test-retest equivalency results within 0.05 logMAR (2.5 letters, 90% CI). Whether testing using their typical technique on an M&S computer or using EyeSpy Mobile, eye professionals obtained statistically closer mean test-retest results than parent volunteers by 1 letter, with equivalency results within 0.03 logMAR (1.5 letters, 90% CI). Conversely, the number of retests within 2 vision lines was statistically greater when EyeSpy mobile was used by parents as compared to eye professional’s customary technique on the M&S computer.

Conclusion

EyeSpy Mobile provides clinically useful visual acuity test-retest results even when used by first-time parent volunteers. Adaptive visual acuity algorithms have the potential to improve reliability, lessen training requirements, and expand the number of vision screening volunteers in community settings.

Keywords:

• EyeSpy Mobile
• M&S computer
• visual acuity
• vision screening
• adaptive algorithm

Acknowledgments

We would like to acknowledge the support of Malin Joseph with the Phoenix Children’s Biostatistics Core in performing all statistical analyses and creating all tables and figures. Additionally, we would like to acknowledge Victoria Bernaud with the Phoenix Children’s Scientific Writing Core for assistance with manuscript writing, editing, and formatting.

Disclosure

JWO developed and has commercial interest in EyeSpy Mobile. He did not participate in any visual acuity testing using the application (Phases 2 and 3). JWO reports 50% owner of Cloudscape LLC, outside the submitted work; In addition, Dr James O’Neil has a patent US 17/240,382 pending to James O’Neil, R. Tirendi. The authors report no other conflicts of interest in this work.