In the UK, visual acuity is measured by uncertified non-refracting HCP staff in eye clinics. This is the first study to comprehensively investigate and demonstrate the limitations in non-refracting HCP measured VA when compared to the best corrected visual acuity as measured by an optometrist with subjective refractive error correction on the same day.
The non-refracting HCP measured VA was > 5 ETDRS letters inaccurate in a third of the patients. The limits of agreement on Bland-Altman plot indicates that the non-refracting HCP measured VA is imprecise, measuring VA from 18.39 ETDRS letters higher to -14.66 ETDRS lower than the BCVA. This inaccuracy and imprecision in the non-refracting HCP measured VA may have clinical implications when assessing and managing patients with eye conditions. In our example of AMD patients, a variation of less than ± 5 ETDRS letters in between visits in eyes with nAMD is considered to be clinically non-significant, while a loss of > 5 letters is deemed as potentially clinically significant and an improvement of ≥ 5 ETDRS letters is considered as a ‘good’ response to treatment with the licensed anti-VEGF drugs (9). The seminal clinical trials providing evidence for the efficacy of the licensed anti-VEFG drugs for nAMD, set avoidance of moderate visual loss, i.e. losing fewer than 15 ETDRS letters from the baseline at 12 months as their primary endpoint (16, 17). Our findings indicate that non-refracting VA which is routinely measured by a variety of HCPs in busy AMD clinics may be inadequate in detecting either of the two endpoints given the wide limit of agreement with the BCVA, between 18.39 and − 14.66 ETDRS letters – a range of 33 letters on the ETDRS chart. The non-refracting HCP measured VA showed higher inaccuracy and imprecision at lower levels of BCVA.
Variation between two VA measurements has been reported for the same eye tested by the same examiner, using the same chart, even when the VA was measured within a one-minute interval (18). Despite a likely memorisation effect there was still a difference of ± 2.5 letters between the two measurements on ETDRS vision chart (18). However, in a previous report in a group of patients from the Macular Photocoagulation Study, the BCVA measurement of patients by two certified clinical trial examiners on the same day found a mean difference of -0.75 letters with a standard deviation of 4.25 letters, and a limit of agreement between − 9.08 and + 7.58 letters on ETDRS vision chart (19). Another study also demonstrated considerable intersession variability in BCVA measurement among AMD patients, although measurements were not taken on the same day (20).
A consequence of inaccurate and imprecise VA measurements is eligibility to funded treatment of licensed drugs for ophthalmic conditions. Around half of the eyes with nAMD in our cohort met NICE stipulated VA cut-offs to qualify for the licensed drugs (7, 8). Compared to BCVA, the VA measured by the non-refracting HCP missed about 10% of patients who did qualify for the nAMD licensed drugs, whilst it included 15% of patients who didn’t.
VA outcomes discrepancies between the ‘real world’ and clinical trials, as well as variation between different health systems or centres within the same health system, have been highlighted (12). Accuracy of the “real world” clinic measured VA may be a contributing factor to variation in reported nAMD treatment outcomes when differences cannot be explained otherwise.
Finally, eligibility for visual impairment registration and driving may be affected by accuracy and precision of VA measurements. In this study, when refracted BCVA was carried out, 60% of the eyes had a visual acuity below the UK Driver and Vehicle Licensing Agency (DVLA) driving standard for a car (11). The non-refracting HCP measured VA missed 4% of these eyes, while erroneously including about 10% of eyes that had met driving standards. Although we did not measure binocular vision as required by the DVLA, from the inaccurate VA measurement by HCPs, we can extrapolate that these findings could have serious legal implications.
The strength of this study is that it is a well-powered study for the relevant clinical end points. The study compares a typical clinical setting to the reference standard used in clinical trials.
The main weakness of this work is that it is a single centre study. Different centres will have different training protocols and quality assurance for the non-refracting HCPs measuring VA. Furthermore, it would have been helpful to include a cohort of measurements comparing optometrists measuring VA without and with subjective refraction in order to remove the influence of academic background and training when comparing the two methods. This was logistically beyond our resources.
The non-refracting HCPs were aware that this study was being conducted. Due to the Hawthorne effect, we expect an improvement in their performance in measuring visual acuity.
Although there was an obvious difference in technique, non-refracting vs refracting VA, there could be other factors in play. One could speculate on the technique and the training of the non-refracting HCPs in a busy AMD clinic explaining the difference in VA. For instance, due to time constraints, patients with unstable fixation or a scotoma may not be encouraged to read, guessing where necessary, every single letter on each line of the ETDRS chart. Also, moving the vision chart from 4 metres to 1 metre, for the 4m ETDRS charts when the top line cannot be fully read, could have been considered a time-consuming practice and not uniformly followed. As a result, low visual acuity levels would not be correctly measured, which is suggested by the low sensitivity of clinic measured VA vs BCVA in detecting VA of < 24 letters (Table 3). However, the difference in the VA could be due to the inherent variation in non-refracted VA. To investigate these issues further, appropriate quality assurance procedure needs to be followed and the non-refracting HCPs have to be supported and trained if needed.
The UK does not have a certification system for non-refracting VA assessors. There are local protocols that departments may resort to in order to train their staff. However, an unstructured training coupled with the need to quickly deploy non-refracting VA assessors who have the most diverse knowledge and background lends itself to variation. Under pressure to process as many patients as possible assessors may adapt their VA measurement strategy. We cannot comment on the effect of variation of local standard operational procedures on VA measurements as this was beyond the scope of a single centre service evaluation study. We are highlighting that, potentially, there is an issue with “real world” VA measurements in the NHS. In a national health service that strives to collect VA data for monitoring outcomes, efforts should be made to achieve consistency and quality. This report may encourage revision of the standard operating procedure for VA measurement in ophthalmology departments, promote audits and inform a discussion on whether training national standards and quality assurance processes may be required to achieve a more robust and reproducible measurement of the most important assessment of visual function.