Researchers have developed an innovative metric for objectively classifying pigmentation in a retinal image, rather than using subjective social constructs like ethnicity or other demographic variables, as is currently the case when building health datasets.
Called the Retinal Pigment Score (RPS), the open-source measurement system will enable more equitable artificial intelligence (AI) algorithms for detecting and managing eye conditions, such as diabetic retinopathy.
As described in Nature Communications, the RPS was created by a team of international researchers using deep learning to assess more than 70,000 colour fundus photographs of the retina from UK Biobank.
The international study, led by Moorfields Eye Hospital NHS Foundation Trust, UCL Institute of Ophthalmology and University of Washington, successfully validated the metric on cohorts in Tanzania, China, and Australia. Researchers found that retinal pigmentation varies significantly within ethnic groups with substantial overlap between ethnic groups, making traditional ethnicity-based classification unreliable, in the same way that hair, eye and skin colour overlap between ethnic groups.
If widely adopted, the RPS would overcome the problem of AI models being trained on patient data and labelled for ethnicity, which may be incomplete or missing and not reflect the biological state of the eye. The RPS could enable ophthalmology AI developers and medical product regulators to objectively evaluate whether training data includes adequate diversity and ensure efficacy after clinical deployment on a wide range of patients.
A montage of macula-centred true colour images shows a spectrum of fundus pigmentation commonly observed in clinical practice (arranged from left to right, progressing from lighter to darker pigmentation).
The findings also have potential implications for studying systemic diseases, such as Alzheimer’s and cardiovascular disease, with retinal images serving as a biomarker — a rapidly growing scientific field known as Oculomics.
Co-lead author Abraham Olvera-Barrios, Moorfields clinical research fellow, said: “RPS is a game-changer for both clinical research and AI development. It provides a precise, unbiased measure of retinal pigmentation, ensuring more inclusive and accurate healthcare solutions. If widely adopted, the RPS could overcome the limitations of subjective categorisation, or missing, ethnicity data.”
Fellow lead author Anand E Rajesh said: “We are excited by the potential of this new metric to transform the way researchers and regulators think about eye data classification. By incorporating RPS, researchers can evaluate and improve AI model performance across varied biological backgrounds, fostering algorithmic fairness in medical AI.”
Senior author Cathy Egan said: “This novel approach challenges the reliance on ethnicity as a surrogate marker for biological variability and addresses the critical challenge of phenotypic diversity within retinal imaging datasets, a key concern in developing equitable and effective AI algorithms. With the global epidemic of diabetes, diabetic retinopathy is overwhelming the medical workforce and AI presents a potential solution. The RPS will be an important metric to demonstrate that these models work safely and fairly for all people with diabetes.”
The RPS algorithm has been made publicly available in hopes of encouraging other researchers to use it for the development of AI systems that are inclusive and unbiased.
To access the RPS algorithm, visit the project repository at GitHub.
Access the Nature Communications publication here.
For more information, please contact alex.black3@nhs.net
