Foundation models showed limited generalizability when applied to unseen imaging contexts and populations, with disparities in model performance observed across Danish and Greenlandic populations, and local fine-tuning improved discrimination but did not resolve calibration issues.
DR prevalence differed substantially between the Danish and Greenlandic datasets used in this study.
When fine-tuned and evaluated within the same population, discrimination was similar in Denmark and Greenland, with RETFound DINOv2 achieving the highest performance in both.
External validation of Danish-fine-tuned models on the Greenlandic dataset showed substantially worse discrimination performance across all models.
Sequential fine-tuning from the Danish to the Greenlandic dataset improved discrimination compared to direct external validation.
Calibration remained poor across all experimental settings and all models, regardless of fine-tuning strategy.
Foundation models demonstrated limited generalizability when applied to ultra-wide field retinal imaging in unseen populations.
This research suggests that AI models designed to detect diabetic retinopathy (a complication of diabetes that can cause vision loss) may not work equally well across different patient populations or when used with a different type of eye camera than they were originally trained on. The researchers tested three state-of-the-art AI 'foundation models' — RETFound DINOv2, VisionFM, and EyeCLIP — using ultra-wide field retinal cameras (which capture a broader view of the retina) in screening programs in Denmark and Greenland. They found that while the models performed adequately when trained and tested within the same population (reaching accuracy scores of around 0.76 on a 0–1 scale), performance dropped significantly when a model trained on Danish patients was directly applied to Greenlandic patients (accuracy scores of 0.59–0.62). Retraining the models using some local data from Greenland helped improve accuracy but did not fully close the gap. A particularly important finding concerns model 'calibration' — that is, whether the model's predicted probabilities of having diabetic retinopathy actually match reality. Even when models had reasonable accuracy at distinguishing between patients with and without retinopathy, their probability estimates were consistently poorly calibrated across all settings and all models tested. This means the models might, for example, assign only a 20% probability of disease to a patient who actually has a 60% chance of having it, which could mislead clinical decision-making. This research suggests that AI tools for eye disease screening cannot simply be transferred from one population or imaging device to another without careful local validation and adjustment. The findings are particularly relevant for remote or underserved populations like those in Greenland, who may have different disease prevalence and demographic characteristics. Ensuring both accuracy and calibration before deploying AI screening tools in new settings appears to be essential for safe clinical use.
Li L, Thambawita V, Byberg S, Hulman A. (2026). Assessing the generalisability of foundation models to ultra-wide field retinal imaging for diabetic retinopathy screening in Denmark and Greenland.. International journal of medical informatics. https://doi.org/10.1016/j.ijmedinf.2026.106503