Methodological aspects of using a wearable eye tracker in differential diagnostics of disorders of consciousness

Background: The differential diagnosis of disorders of consciousness (DOC) is an important element in prognostication, clinical decision making, and information to families. Diagnosis currently relies to a large extent on clinical assessment using standardized neurobehavioral rating scales. Interpretation of the patient ’s responses may be difficult due to cognitive, linguistic, physical, and/or motor impairments associated with the brain injury. Analysis of eye movements using eye tracking technology has been suggested as a potential support in improving diagnostic accuracy. Methods: The aim of the study was to evaluate the feasibility of incorporating eye tracking in a selection of subtests in the Coma Recovery Scale Revised (CRS-R). A secondary aim was to develop a protocol for comparison of the clinician-assessed findings versus the findings based on the eye movement recording. Four adult patients with a persistent disorder of consciousness were assessed with four test items from the CRS-R while eye movements were recorded with a wearable eye tracker. Results: No major adverse reactions were observed suggesting likely patient acceptability. Calibration issues due to oculomotor dysfunction complicated the analysis. However, distinct eye movements were discernible from the recorded data and analysis of these gave findings considered to support clinical assessment. Feasibility and data analysis issues are discussed. Conclusions: Based on patient responses, and the finding that it was possible to record eye movement patterns even without calibration, we conclude that the method is feasible. Further study will help clarify the full potential but the option to record and replay responses in these easily fatigued patients where assessment sessions need to be brief is of great interest. The current hardware and software limitations can be overcome with manual data processing and analysis; however, significant developments in automizing data processing will be required for broader clinical application. Hardware adaptations and the calibration process appear to be important primary targets in this process.


Background
The differential diagnosis of prolonged disorders of consciousness (PDOC) is an important element in assessing prognosis and subsequent clinical decision making. In particular, the differentiation between Unresponsive Wakefulness Syndrome (UWS, previously called vegetative state) and Minimally Consciousness State (MCS) is of relevance when planning rehabilitation interventions. Diagnosis currently relies largely on clinical assessment where standardized neurobehavioral rating scales play an important role (1). The Coma Recovery Scale Revised (CRS-R) (2) is one of the most established scales (3). In the CRS-R the patient is encouraged to perform tasks involving basic levels of conscious motor responses while the clinician observes and gives ratings. Observations of eye movements play an important role in that the patient is asked to look at certain objects or follow a moving target with the eyes.
The patient's responses during the CRS-R may be hampered by cognitive, linguistic, physical, and motor impairments associated with the brain injury (4). Even if the patient's intention is to respond to an instruction, e.g. to look at an object, the response may be poor and difficult to interpret. Strabism (squint), gaze palsies or impaired volitional motor control over eye movements may add further uncertainty to the assessment. These complications are reflected in previous studies where a 15-50% mismatch has been demonstrated between the findings from neurobehavioral rating scales and tests performed with EEG or fMRI (5). EEG and fMRI have however their own limitations (accessibility, difficulties in interpretation at the individual level, need for sophisticated post processing that is not widely available). Criteria for awarding points on the CRS-R are necessarily fairly demanding, in order to minimize the risk that random occurrences are misinterpreted as intentional. However, the examining clinician in some cases reports that the patient "seems to be trying" to perform the task, even though performance falls short of the scale criteria. Since neurobehavioral assessments are key in the diagnosis it is of interest to develop supportive tools that can be applied in a clinical setting, with the potential to detect more subtle non-random stimulusrelated responses that would contribute to early detection of behavioural signs of consciousness.
Analysis of eye movements using eye tracking technology has been suggested as a potential support in differential diagnosis (6)(7)(8). However, several of the commercially available eye tracking systems use desktop mounted setups that require the subject to be positioned within certain angles of the sensors. The subject must also be able to focus their gaze within a specific spatial area so the sensors can capture the eye movements. These requirements cannot be met for patients with PDOC for whom any communicative ability is minimal and fluctuating and where considerable physical limitations due to paresis and disorders of motor tone are usual. Recent technology does however offer eye tracking units that can be worn like a pair of spectacles and this potentially opens up to a more flexible application. It remains to find a balance between technical requirements, practical solutions and patient-friendliness. The purpose of this study is to study how these issues, such as calibration difficulties in cases of strabism and how data could be analyzed when the standard commercially available analysis tool is not applicable, can be dealt with.
The primary aim of the study was to investigate the feasibility of incorporating eye tracking in a selection of subtests in the CRS-R test paradigm, both in practical and technical terms. A secondary aim was to develop a protocol for comparison of the clinician-rated finding versus findings from eye movement registration.

Methods
The study was carried out as an observational case series with descriptive as well as analytical elements. Four patients were recruited from the rehabilitation clinic of a regional rehabilitation unit. The examinations and tests were performed in an outpatient setting in a hospital environment. A physician (specialist in rehabilitation medicine) performed initial screening with regards to inclusion and exclusion criteria.
Inclusion criteria were (1) adult patients aged 18-65 years who had suffered an acquired brain injury in adulthood and (2) a suspected PDOC. The judgement of a suspected PDOC was based on persisting inability to respond to simple instructions, to use simple objects in an adequate way and absence of functional, interactive communication more than four weeks post injury. The exclusion criteria were as follows; known blindness or deafness, eye disease or severe eye motility restrictions, medical instability, sedation, medical restrictions on neck movements and if family relatives or the patient's trustee opposed the patient's participation in the study.

Consent
Patients with PDOC by definition lack the capacity to give informed consent. Patient's relatives were informed about the study and patients included only if the relatives had no objection. The study was approved by the Swedish Ethical Review Authority

Eye tracking equipment
Eye movements were recorded with a Tobii Pro Glasses 2 eye tracking system (www.tobiipro.com) with a sampling frequency of 100 Hz. This is a wearable eye tracker designed to be worn like a pair of spectacles. Corrective lenses can be added to compensate for refractive error or if reading aids are needed. The sensors and a scene camera (90-degree field of view) are built-in in the frame and the total weight is approximately 45 grams.
In order to track the eye movements as accurately as possible and, importantly, be able to map the eye movements against a visual stimulus, the eye tracking system needs to be calibrated. This procedure commonly involves asking to participant to look carefully at a calibration target with both eyes. Strabismus is common in the PDOC patient group and in these cases, it is not possible for the patient to view a calibration target with both eyes simultaneously. PDOC patients often have a reduced ability to fixate on visual targets. The consequence of failed calibration is that the standard analysis tool of the eye tracking system cannot be used. Another technical issue is that the standard analysis tool of the system does not allow mapping of eye movement recordings against moving targets. Subsequently it was necessary to investigate work-around solutions.

CRS-R and general procedure
The test paradigm included four items from two of the six sub-scales of CRS-R (table 1). In item 1 the objects consisted of a yellow ball and a brown cup of similar size (diameter approximately 65 mm). The auditory stimulus in item 2 was a miniature handbell. In item 3 a hand mirror (diameter 138 mm) was used. The object in item 4 consisted of a yellow ping-pong ball mounted over a mini flashlight. When the flashlight was switched on the ping-pong ball became luminescent.
During the test the patients remained seated in a comfortable position in the wheelchair and was positioned to face a neutral wall. The patient's visual function, i.e. ability to discriminate the visual stimuli, was primarily estimated based on their latest spectacle prescription. If this was not available retinoscopy was performed to estimate if there was a refractive error or a need for near aids. Corrective lenses were snapped on to the eye tracker frame if needed. Before putting the eye tracker frame onto the patient, a pair of standard spectacles were put on to familiarize the patient with the situation and to watch for any adverse reactions.
Eye movements recording began with a calibration procedure. The patient was encouraged to look at a calibration target while the eye tracking system acquired eye position information. If calibration was unsuccessful, most commonly due to strabismus, the subsequent recording of eye movements was performed without calibration. During the preparations, the examiner made a clinical assessment of which eye that tended to fixate most often. The movements of this eye were then primarily used in the analysis. Two common objects are simultaneously presented approximately 40 cm apart within the patient's field of view. The patient is asked to look at the object named. A total of four trials are administered.

Localization to sound
An auditory stimulus is presented for 5 seconds to the patient's right side and then the left side. Four trials.
Visual function scale 3. Visual pursuit A hand mirror is held directly in front of the patient's face.
The patient is verbally encouraged to fixate on the mirror image. The mirror is moved 45 degrees to the right and to the left of the vertical midline and then 45 degrees above and below the horizontal midline. Two trials in each plane.

Fixation
A brightly illuminated object is presented in front of the patient's face and then rapidly moved to the upper, lower, right and left visual fields for a total of four trials.
The test procedure was performed by a psychologist trained in CRS-R assessment. First, spontaneous eye movements were observed for a minute. Then, the four test items in table 1 were assessed with brief pauses in between when the psychologist made notes of their observations and ratings.
Eye movement recordings were analyzed stepwise using Tobii Pro Studio or Origin 2017 (www.originlab.com). In the first step the recording was replayed and divided into events based on the actual test events seen via the scene camera. An event was a defined time span in the recording starting at the point where the patient first received an instruction (item 1 and 3) or was exposed to a stimulus (item 2 and 4). The event ended when the stimuli was removed from view (item 1, 3 and 4) or the sound ended (item 2). In the second step the eye tracking data for each separate event was plotted on a graph for visual inspection of the integrity of data. Data loss was the main concern and occurred when the patient closed the eyes or at extreme gaze angles towards the periphery. In the third step a judgement was made whether or not the patient appeared to respond to the stimuli as instructed. This was mainly judged by comparing the trend between pre-test data and data collected during exposure to a stimulus.

Results
Clinical, practical and eye tracking-related aspects will be presented and discussed first. Next, each test item from the CRS-R will be presented with regards to acquisition and analysis of data as well as differences in approach depending on whether calibration was possible or not.

Retinoscopy
Retinoscopy was performed in the event that no patient history was available regarding visual acuity or need for spectacles. It proved challenging to perform a standard static retinoscopy due to difficulties controlling where the patient focused the gaze. Instead, dynamic retinoscopy was performed where the examiner positioned himself at the same distance from the patient as the testobjects were intended to be presented at. The patient was verbally encouraged by the examiner to look toward the lamp of the retinoscope. If judged necessary, corrective lenses of the appropriate refractive power were mounted on the frame. An allowance of approximately +/-1 diopter was accepted due to uncertainty in measures and also due to that the test objects were large.

Practical experiences of using the eye tracker
The eye tracking frame, despite its fairly light construction, is somewhat different from a normal pair of spectacles. The electronics add some bulk to the frame and limit the field of view to some extent. In order to familiarize the patient with the situation a pair of standard spectacles were put on first.
Care was taken to choose nose pads that appeared to rest comfortably on the patient's nose. It was found that a tiny piece of soft padding under the nose pads improved comfort and helped prevent the frame from sliding down the nose. Another aspect was the fact that the ear-pieces of the frame are quite straight, long and somewhat thicker than normal ear-pieces. This proved to be a challenge considering the neck rests of the patient's wheelchair. A certain amount of trial and error was required when adjusting head position to find a balance between patient comfort and preventing the neck rests from pushing the frame out of position. If needed, a neck strap was used to keep the frame in place. In general, the patients appeared to tolerate the eye tracking frame fairly well. However, in one case it was observed that the patient kept the eyes closed to a greater extent when wearing the eye tracker. This was confirmed by re-testing without the frame on. The finding may be interpreted as a potential comfort issue. Given these experiences from practical use some adaptations of the frame design may prove beneficial.

Calibration
Each test-session began with an attempt to calibrate the eye tracker. An assistant positioned him or herself straight in front of the patient holding the calibration target in the patient's theoretical line of sight. Meanwhile the test-leader started the recording and carefully monitored the calibration procedure which required approximately two seconds of visual fixation on the target. If the patient did not spontaneously look at the calibration target, or had their eyes closed, the assistant tried to attract the patient's attention primarily through verbal encouragement or by patting an arm or knee. If necessary, the assistant changed position to the patient's habitual head and gaze direction. In order to reduce the risk of fatigue impacting on later response, generally not more than three attempts were performed before proceeding with the recording.

Acquisition and analysis of data
The full recording was replayed and divided into episodes based on when the exposure to a stimulus started and stopped. If calibration was successful the visualization tool and metrics available in Tobii Pro Studio were used as basis for judging the response. The recorded eye fixations were superimposed on a snapshot of the patient's view using an automated mapping function. The mapping was manually checked for accuracy and corrected if obvious mis-mappings had occurred (usually less than 10% of fixations). In the event that calibration was unsuccessful the analysis relied on visual inspection of plotted raw vector data as will be described further. In both cases the aim of the analyses was to reach a conclusion of whether or not the patient tried to respond.

CRS-R Test item Consistent movement to command
In this case (patient 2) calibration was successful. Two objects were presented and the patient was asked to look at the ball which in this case was located to the left ( figure 1). An area of interest (AOI) was defined around each object and any visual fixation within this area was counted as a hit (table 2). Based on the distribution of hits in this example it was regarded as an appropriate response.  In the next example (patient 1) the patient had a suspected alternating strabism, that is, the patient sometimes appeared to focus with the right eye and sometimes with the left eye. Apart from complicating the clinical judgement this also meant that calibration was unsuccessful, hence the visualization and metrics tools of Tobii Pro Lab could not be used. It also meant that no absolute eye positioning data could be obtained from the recording. Instead, relative changes in gaze direction were derived based on changes in the gaze direction vector.
When observing the patient before commencing the tests, the gaze was habitually directed to the patient's left corresponding to an approximate horizontal gaze direction vector value of 0.4 ( figure 2). According to the coordinate system of the eye tracking system, any increase in vector value corresponds to a leftward movement while a decrease corresponds to a rightward movement. Starting the test, two objects were presented within the patient's field of view; the ball to patient's left and the cup to the right. The patient was asked to look at the tennis ball ( figure 3). A linear fit (right eye) resulted in an intercept 0.094, R-square 0.132 with a slope (2.847*10 -5 ) that was significantly different from zero. Based on visual inspection of the graph and the difference in slope it was considered plausible that this was an appropriate response, that is, an attempt to look towards the stimulus.

CRS-R Test item Visual pursuit
A mirror was presented at approximately 40 cm in front of the patient's eyes. Once the patient was judged to be looking at the mirror image the mirror was moved smoothly along the horizontal and vertical midline. In the first example (patient 2) the calibration was successful. This meant that the gaze point, left and right eye position averaged, could be plotted against the coordinate system of the scene camera. This coordinate system is depicted in pixels starting from the top left corner as seen from the patient's view. Given the resolution of the scene camera this means that a gaze point at the extreme top left has the coordinates x = 0, y = 0 while a gaze point at the extreme down right has the coordinates x = 1920, y = 1080. The coordinates for a gaze point at the very center are x = 960, y = 540. In order to reduce the complexity of the analysis, horizontal and vertical movements were plotted separately. As in previous cases, eye movement behavior without any stimulus present (figure 4) was compared with eye movement behavior when the actual test was performed ( figure 5). Visual inspection indicated a plausible correct response.  . Visual pursuit where the mirror is moved from left to right. In the first phase (67000-69000 ms) the patient's visual fixation on the mirror image is established. At approximately 69200 ms the mirror is starting to move towards the right (increasing pixel-value) and a smooth visual pursuit can be seen up until the endpoint at 73000 ms.
In the next example (patient 1) calibration was not successful. The first trial when the mirror was moved to the patient's left failed to show any response. The second trial on the other hand, when the mirror was moved to the right, led to a response ( figure 6). Figure 6. The graph shows a plot of the subsequent gaze direction vector while the mirror is moved sideways toward the patient's righthand side. A sequential and consistent decrease in vector value was observed during the exposure to the stimuli. This corresponded to the expected eye movement behavior when the patient's eyes move to the right.

CRS-R Test item Fixation
The third test item was Fixation where the luminescent ping-pong ball, as a starting point, was held centered in front of the patient. It was then swiftly moved between five positions (center-left-centerright-center-up-center-down) while the examiner observed if the patient attempted to gaze at the lamp. This proved to be the visual test item where it was most difficult to discern a response. After repeated trials a plausible response was observed in one of the patients (patient 2) (figure 7). Figure 7. Fixation test. The lamp was moved from center position and downward. According to the coordinate system of the scene camera an increase in pixel value corresponds to a downward movement. In the plot it can be seen that the initial vertical gaze direction (time stamp 166000-167000) is at the center (420-600 pixels). It then increases value, peaking at almost 1080 pixels, after which it returns to center value (time stamp 171000)

CRS-R Test item Localization to sound
The fourth and final test item was Localization to sound. Four trials were performed starting with a five-second ringing of the bell on the patient's right-hand side. The expected response was the patient gazing or turning the head towards the direction of the sound. However, based on the eye movement recording and accelerometer registering head movements none of the trials resulted in what was judged to be such a response.

Discussion
The main aim of this study was to investigate the feasibility of incorporating eye tracking in CRS-R test paradigms. Aspects concerning patient care, practical solutions as well as technical requirements were considered.
From a patient acceptability point of view the method appears feasible. No adverse reactions to wearing the eye tracking frame were observed. However, with the current design of the frame, some adaptation and extra care is necessary. Our experience from this sample is that ensuring the patient's comfort, as far as possible, is key when performing the tests. Without this, the interpretation of patient responses may be further complicated, for example if due to discomfort with the eye-tracker equipment or the situation, the patient partially or completely shuts their eyes.
From a technical point of view, it is important to keep the frame in position for the sensors to capture eye movements properly and also to prevent the frame from obscuring the field of view. The frame in itself limits the field of view, especially upwards, whereby care needs to be taken to prevent it from sliding or being pushed down the nose. This is also key in order to minimize obstruction of the examiner's view of the patient's eyes. Some adaptations of the frame design, nose pads and ear pieces, may prove beneficial for these purposes.
Calibration of the eye tracker was partly possible. For the patients where calibration was indeed possible the visualization and metrics functions that come with the system could be conveniently applied. However, some limitations need to be considered. Even though the patient did keep their eyes properly aligned to allow calibration, observation sometimes suspected that alignment failed in some gaze directions. This may be a source of error since the recorded gaze direction is based on averaging of the direction of each individual eye. It should however be possible to compensate for this uncertainty by separating areas of interest according to the estimated accuracy level. Regarding moving stimuli, it was not possible to do a dynamic mapping of eye movements versus stimulus using the inherent metrics and visualization functions. In this sample the actual change in gaze during exposure to the stimulus was analyzed which of course means there is uncertainty if the patient tracked the actual target. This could potentially be managed by repeating with several trials. However, many patients with PDOC fatigue easily and prolonged sessions are therefore not optimal as testing later in a long session would be expected to be less reliable. In the event that calibration was not successful only relative changes in gaze were available for analysis. Despite this, it was still possible to discern what appeared to be distinct changes that could be compared to the clinical assessment with the CRS-R. Future applications of this method would require technical adaptations to allow one-eyed calibrations and dynamic mapping of eye movements versus stimulus. Preferably, this should be developed in cooperation with eye tracking system providers, rather than local adaptations, to promote general availability.
In three out of four patients, eye movement data could be retrieved from a high percentage of trials. For one patient data could only be retrieved from 21% of trials. The most likely reason being an extreme horizontal gaze deviation which was suspected to be habitual rather than an eye movement restriction. When assessment was indeed possible there was a strong agreement between the clinical and eye tracking assessment. For the other three patients the agreement between assessments ranged between 54-69%. In many cases the eye tracking data indicted a response while the clinical CRS-R assessment did not. We think this illustrates the difficulty of these assessments but also indicates the potential support of recording eye movements. The possibility to record, replay, and document the assessments may provide valuable opportunity for development of the sensitivity of assessments as well as support in assessment skills training.
To our knowledge, this study is the first described attempt to incorporate eye movement recording in CRS-R test paradigms using a wearable eye tracker. It is a small sample and further investigations with larger sample and different levels of recovery are needed to develop the understanding of its potential application. It does however appear feasible with some extra care and adaptations.

Conclusions
Based on the patient responses, and the possibility to discern eye movement patterns, we conclude that the method is feasible. Further study will help clarify its full potential but the option of recording and replaying responses in these easily fatigued patients where the assessment sessions need to be brief is of great interest. The current hardware and software limitations can be overcome with some extra care and detailed analysis; however, significant developments will be required for a clinical application. Hardware adaptations and the calibration process appear to be important primary targets in this process.