The ‘Real World’ Clinic Measured Visual Acuity Versus the Best Correct Visual Acuity Measurement in Patients Treated for Neovascular Age-Related Macular Degeneration

visual fewer than 15 ETDRS letters from the baseline at 12 months endpoint Our ndings indicate that non-refracting VA which is a HCPs busy AMD may be inadequate in detecting either of the two endpoints given the wide limit of agreement with the BCVA, between 18.39 and − 14.66 ETDRS letters – a range of 33 letters on the ETDRS chart. The non-refracting HCP measured VA showed higher inaccuracy and imprecision at lower levels of BCVA.


Background
The most common method of determining the level of vision is to measure monocular distance visual acuity. Monocular distance visual acuity (VA) is the minimum angle of resolution of two points achieved by the visual system under standard conditions (1). It is commonly measured as the number of highcontrast letters, which progressively reduce in size, that a person is able to read from a standardised chart at a pre-de ned distance. It is a relatively simple and quick way to quantify the level of vision in a patient in both the clinical and research environment, including patients with age-related macular degeneration.
Age related macular degeneration (AMD) is a disease affecting the central area of the retina. This results in loss of central vision and deterioration of the patient's ability to undertake tasks such as reading, recognise faces, drive a vehicle, with a signi cant reduction in the quality of life (2). It is an agingassociated disease and approximately 5% of the people aged 65 and over in the United Kingdom (UK) are affected by the advanced form of the disease (3). Indeed, in England & Wales, over 30 000 people are registered as partially sighted or blind annually, half of whom are due to AMD (4). Neovascular, wet or exudative AMD (nAMD) is the late stage manifestation of AMD more frequently associated with devastating loss of central vision (5).
VA measurement is an important tool for the assessment and management of patients with nAMD (6).
Ranibizumab and A ibercept are the two licensed drugs approved by the National Institute for Health and Care Excellence (NICE) to treat nAMD in England (7,8). A key NICE condition of approval of nAMD treatment with licensed drugs is for the VA in the patient's affected eye(s) to be between 6/12 and 6/96 on a Snellen chart. This equates to a VA score between 24 and 73 letters on the ETDRS vision chart. VA measurement is also the primary outcome of all clinical trials on the treatment of nAMD.(2) Changes in VA are used to measure response to treatment (9). Notably, the loss or gain of three or six lines of vision on the ETDRS chart de nes respectively the outcome of moderate or severe clinically signi cant visual loss or gain (9). In the UK, VA is used to determine the level of visual impairment (10). Patients with full visual elds may be registered as severely or partially visually impaired if the VA in the better eye is less than 24 ETDRS letters (3/60 on Snellen chart) or between 24 and 38 letters (3/60 to 6/60 Snellen) respectively (10). Conversely, one of the main criteria to drive a car in the UK is a binocular VA of at least 6/12 on the Snellen chart or 70 letters on ETDRS chart (11).
The reference or "gold" standard measurement of the VA is the best corrected VA (BCVA). In this work, we de ne BCVA as monocular VA measured after correction of the refractive error under standardised conditions, which is usually performed by an optometrist. However, measuring BCVA in every patient is very resource-intensive, hence not practical in a busy eye clinic. Therefore, the VA in patients with nAMD is usually measured by the non-refracting allied health care professionals (HCPs). HCPs include nurses, technicians and healthcare assistants with a variety of ophthalmic training background. When measuring VA, HCPs are not expected to carry out subjective refraction.
Anecdotal evidence and our clinical experience suggest that on occasions the VA measured by a nonrefracting HCP may differ signi cantly from the BCVA. The magnitude of such discrepancy has both clinical and legal implications. Furthermore, inter-centre variation in nAMD VA outcomes, not fully explained by patient or treatment characteristics, has emerged in the UK (12). Reliability of the "real world" VA outcomes depends on the quality of VA measurements.
This report is the rst to ascertain the accuracy and precision of non-refracting HCP measured VA to BCVA in nAMD patients.

Aims:
Primary outcomes: To ascertain if there is a difference of > 5 ETDRS letters between optometrist measured BCVA and VA measured by non-refracting HCP, which we will refer to as BCVA versus VA To determine the sensitivity and speci city of the non-refracting HCP measured VA for identifying patients with a BCVA between 24 and 73 ETDRS letters (The visual acuity bracket in the UK to be eligible to NICE approved anti-VEGF treatment in nAMD) Secondary outcomes: To determine the sensitivity and speci city of the non-refracting HCP measured VA for identifying patients with a BCVA of < 70 ETDRS letters (Visual acuity not eligible for driving in the UK) To determine the sensitivity and speci city of the HCP measured VA for identifying patients with a BCVA of < 24 ETDRS letters (VA compatible with severe visual impairment registration) Population: We aimed to recruit patients, suffering from nAMD in at least one eye, attending the macula clinic for assessment or treatment. The study was carried out between 3rd March 2019 and 31st January 2020 in the Macula Clinic, Hospital of St Cross, University Hospitals Coventry and Warwickshire NHS Trust, Rugby, UK.

Visual acuity measurement:
For the purpose of this study, the VA was measured prior to any assessment or treatment (13). The VA was measured using the 4m LogMAR ETDRS original charts 1, 2 and R. The charts were mounted on a Precision Vision LogMAR lightbox and retro-illumination provided by two internal daylight 20-watt uorescent tubes (burned in for 100 hours, prior to use in the clinics). The room lighting was standardised. The VA assessment was undertaken (i.e. the non-refracting HCP or optometrist) in no particular order. Both the non-refracting HCP and the optometrist were blinded from each other's measurements. All the non-refracting HCPs were trained to carry out visual acuity assessment.
The recommended method for visual acuity assessment was as follows: I. Each eye was tested separately to obtain monocular visual acuity II. The patient was seated at 4 metres from the chart and instructed to read the smallest line of letters visible III. The patient was required to correctly identify at least 3 letters on a line for testing to continue IV. If < 3 letters were correctly identi ed on a line, the patient was instructed to read the line above to con rm at least 3 letters were read correctly and this process was repeated until at least 3 letters were correctly identi ed on a line V. Following this the patient was instructed to continuing reading the lines below until at least 3 letters were identi ed correctly on a line.
VI. Eccentric xation was encouraged and testing stopped when the patient was only able to identify < 3 letters on a line VII. If < 3 letters from the top line were read at 4m, the testing distance was to be reduced to 1m and testing was performed as detailed in III-VI above Over the period of study, 10 different non-refracting HCPs were involved. To carry out VA assessment, the non-refracting HCPs instructed patients to wear their distance spectacles if they were in possession and a pinhole test was also used. The standard non-refracting HCP practice was to use one ETDRS chart for both eyes. The best measured VA (with or without pinhole), by the HCP, was used for analysis.
The best corrected visual acuity was also measured by a quali ed clinical trial optometrist (AC) to determine subjective refractive error and record the BCVA utilising a separate chart for each eye.
Sample size: The sample size was calculated based on the result of a pilot study (n = 54), our clinical experience and after discussion with a medical statistician.
N for a difference of ≥ 5 ETDRS letters between BCVA and HCP measured VA with an effect size = 0.40, Power of 99.9%, and two-tailed type II error of 0.001 = 260 (calculated for t-test comparing two dependent means (matched pairs)) N for a speci city of 90% with a two-sided con dence interval of 95% for the VA as measured by the non-refracting HCP for identifying patients with a BCVA of 24 to 73 ETDRS letters with a prevalence of a BCVA between 24 and 73 ETDRS letters of 45% = 251 N for a sensitivity of 90% with a two-sided con dence interval of 95% for the VA as measured by the non-refraction HCP for identifying patients with a BCVA of 24 to 73 ETDRS letters with a prevalence of a BCVA between 24 and 73 ETDRS letters of 45% = 307 Therefore N = 307 eyes was chosen Data analysis: The data was collated in a spreadsheet and analysed using IBM SPSS Statistics for Windows, Version 22. The tests were two tailed and type 1 error was set at α = 0.05. Student's t test was used to compare the two dependent variable means (matched pairs) of continuous data. Bland-Altman plot was used to analyse the two methods of VA measurement (15). Pearson correlation was used to test correlation between continuous data.

Results
In this cross-sectional study we recruited 167 patients (324 eyes However, as the Bland-Altman plot (Fig. 1)

Discussion
In the UK, visual acuity is measured by uncerti ed non-refracting HCP staff in eye clinics. This is the rst study to comprehensively investigate and demonstrate the limitations in non-refracting HCP measured VA when compared to the best corrected visual acuity as measured by an optometrist with subjective refractive error correction on the same day.
The non-refracting HCP measured VA was > 5 ETDRS letters inaccurate in a third of the patients. The limits of agreement on Bland-Altman plot indicates that the non-refracting HCP measured VA is imprecise, measuring VA from 18.39 ETDRS letters higher to -14.66 ETDRS lower than the BCVA. This inaccuracy and imprecision in the non-refracting HCP measured VA may have clinical implications when assessing and managing patients with eye conditions. In our example of AMD patients, a variation of less than ± 5 ETDRS letters in between visits in eyes with nAMD is considered to be clinically non-signi cant, while a loss of > 5 letters is deemed as potentially clinically signi cant and an improvement of ≥ 5 ETDRS letters is considered as a 'good' response to treatment with the licensed anti-VEGF drugs (9). The seminal clinical trials providing evidence for the e cacy of the licensed anti-VEFG drugs for nAMD, set avoidance of moderate visual loss, i.e. losing fewer than 15 ETDRS letters from the baseline at 12 months as their primary endpoint (16, 17). Our ndings indicate that non-refracting VA which is routinely measured by a variety of HCPs in busy AMD clinics may be inadequate in detecting either of the two endpoints given the wide limit of agreement with the BCVA, between 18.39 and − 14.66 ETDRS letters -a range of 33 letters on the ETDRS chart. The non-refracting HCP measured VA showed higher inaccuracy and imprecision at lower levels of BCVA.
Variation between two VA measurements has been reported for the same eye tested by the same examiner, using the same chart, even when the VA was measured within a one-minute interval (18). Despite a likely memorisation effect there was still a difference of ± 2.5 letters between the two measurements on ETDRS vision chart (18). However, in a previous report in a group of patients from the Macular Photocoagulation Study, the BCVA measurement of patients by two certi ed clinical trial examiners on the same day found a mean difference of -0.75 letters with a standard deviation of 4.25 letters, and a limit of agreement between − 9.08 and + 7.58 letters on ETDRS vision chart (19). Another study also demonstrated considerable intersession variability in BCVA measurement among AMD patients, although measurements were not taken on the same day (20).
A consequence of inaccurate and imprecise VA measurements is eligibility to funded treatment of licensed drugs for ophthalmic conditions. Around half of the eyes with nAMD in our cohort met NICE stipulated VA cut-offs to qualify for the licensed drugs (7,8). Compared to BCVA, the VA measured by the non-refracting HCP missed about 10% of patients who did qualify for the nAMD licensed drugs, whilst it included 15% of patients who didn't.
VA outcomes discrepancies between the 'real world' and clinical trials, as well as variation between different health systems or centres within the same health system, have been highlighted (12). Accuracy of the "real world" clinic measured VA may be a contributing factor to variation in reported nAMD treatment outcomes when differences cannot be explained otherwise.
Finally, eligibility for visual impairment registration and driving may be affected by accuracy and precision of VA measurements. In this study, when refracted BCVA was carried out, 60% of the eyes had a visual acuity below the UK Driver and Vehicle Licensing Agency (DVLA) driving standard for a car (11). The non-refracting HCP measured VA missed 4% of these eyes, while erroneously including about 10% of eyes that had met driving standards. Although we did not measure binocular vision as required by the DVLA, from the inaccurate VA measurement by HCPs, we can extrapolate that these ndings could have serious legal implications.
The strength of this study is that it is a well-powered study for the relevant clinical end points. The study compares a typical clinical setting to the reference standard used in clinical trials.
The main weakness of this work is that it is a single centre study. Different centres will have different training protocols and quality assurance for the non-refracting HCPs measuring VA. Furthermore, it would have been helpful to include a cohort of measurements comparing optometrists measuring VA without and with subjective refraction in order to remove the in uence of academic background and training when comparing the two methods. This was logistically beyond our resources.
The non-refracting HCPs were aware that this study was being conducted. Due to the Hawthorne effect, we expect an improvement in their performance in measuring visual acuity.
Although there was an obvious difference in technique, non-refracting vs refracting VA, there could be other factors in play. One could speculate on the technique and the training of the non-refracting HCPs in a busy AMD clinic explaining the difference in VA. For instance, due to time constraints, patients with unstable xation or a scotoma may not be encouraged to read, guessing where necessary, every single letter on each line of the ETDRS chart. Also, moving the vision chart from 4 metres to 1 metre, for the 4m ETDRS charts when the top line cannot be fully read, could have been considered a time-consuming practice and not uniformly followed. As a result, low visual acuity levels would not be correctly measured, which is suggested by the low sensitivity of clinic measured VA vs BCVA in detecting VA of < 24 letters (Table 3). However, the difference in the VA could be due to the inherent variation in non-refracted VA. To investigate these issues further, appropriate quality assurance procedure needs to be followed and the non-refracting HCPs have to be supported and trained if needed.
The UK does not have a certi cation system for non-refracting VA assessors. There are local protocols that departments may resort to in order to train their staff. However, an unstructured training coupled with the need to quickly deploy non-refracting VA assessors who have the most diverse knowledge and background lends itself to variation. Under pressure to process as many patients as possible assessors may adapt their VA measurement strategy. We cannot comment on the effect of variation of local standard operational procedures on VA measurements as this was beyond the scope of a single centre service evaluation study. We are highlighting that, potentially, there is an issue with "real world" VA measurements in the NHS. In a national health service that strives to collect VA data for monitoring outcomes, efforts should be made to achieve consistency and quality. This report may encourage revision of the standard operating procedure for VA measurement in ophthalmology departments, promote audits and inform a discussion on whether training national standards and quality assurance processes may be required to achieve a more robust and reproducible measurement of the most important assessment of visual function.

Conclusions
We found non-refracting HCP measured VA in eyes with nAMD to have limitations. The non-refracting HCP measured VA showed less accuracy and precision than refracted BCVA by a certi ed optometrist with a limit of agreement of 33 letters. This has potential implications on patient eligibility to licensed treatments, translation of clinical trial results into clinical practice, evaluation of treatment response and outcomes, certi cation of visual impairment and eligibility to driving for nAMD patients managed in eye clinics in the UK.

Declarations
Ethics approval and consent to participate: This work was carried out as part of service evaluation which was formally approved by the hospital clinical governance and research, innovation & development o ce.As such, research ethical approval was not required. Patients were informed of the study and verbal consent was obtained.The study was conducted in accordance with the declaration of Helsinki.

Consent for publication:
Not applicable.
Availability of data and materials: The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Competing interests: We have read and understood the BMC Ophthalmology policy on competing interests and declare the following interests: SP is paid advisor to Heidelberg Engineering®, Zeiss®, Novartis®, Bayer® and Allergan®.
Author's contributions: OK designed the study, analysed the data, and wrote the manuscript. AC helped design the study, collected the data, and critically appraised the manuscript. SP helped design the study and critically appraised the manuscript. The Bland-Altman plot showing the relationship between BCVA and the BCVA minus non-refracting HCP measured VA. The difference between BCVA and non-refracting HCP measured VA has a limits of agreementbetween 18.39 ETDRS letters and -14.66 ETDRS letters.