Using a pilot study to evaluate an assessment strategy for a future Stage III ecacy trial: Example from tuned lighting demonstration

Poor sleep among residents in nursing homes is common and associated with decreased function and falls. Tunable lighting which mimics both day and night exposure may improve sleep. In this pilot study, we evaluate an assessment strategy to determine the appropriate outcome measure for a future Stage III ecacy trial. This pilot study uses a crossover design, in which three hallways were randomized to tunable LED or static lighting for two months (December 2018-January, 2019) and then switched to the other condition for two months (February-March, 2019). Residents who had been in the nursing home for at least 90 days were eligible. We measured sleep (primary) and agitation (secondary) outcomes using key informant interviews and available administrative data. We measured sleep by interviewing staff using the Sleep Disorders Inventory (SDI) and Minimum Data Set (MDS) resident assessments. We measured agitation by interviewing staff using the Cohen-Manseld Agitation Inventory (CMAI) and MDS assessments. We used McNemar’s test and Wilcoxon matched-pairs signed-ranks tests to compare within-person changes for all measures.

1) Uncertainty regarding feasibility of assessment strategy; 2) Widely available administrative measures of sleep may not adequately capture outcomes; 3) Future e cacy trials should consider augmenting existing administrative data with validated staff interview measuers Background Approximately one in ve nursing home (NH) residents have insomnia 1 and 60 percent experience other types of sleep disturbances. 2 Poor sleep been associated with decreased Activities of Daily Living performance, 3 increased falls, [4][5][6] and even decreased survival. 7 8-10 Disrupted sleep may also increase daytime agitation in residents with dementia. 11,12 Medications used to improve sleep, such as anticholinergics, nonbenzodiazepines, and hypnotics, also increase the risk of injurious falls. Non-drug interventions, such as physical activity, light exposure, and mid-body therapies, provide an alternative for NHs to address the issue of disrupted sleep through behavioral or environmental changes. 13 Tunable LED lighting mimics the color and intensity of natural light throughout the day and night and, thus, may restore natural circadian rhythms without the use of medications. There are only a few studies of the effects of ambient tunable LED lighting on sleep and daytime agitation in the NH setting. In one study, blue-enriched white lighting was installed in communal rooms resulting in increased daytime alertness. 14 Another involved installing a dynamic lighting system in the NH common room, resulting in reduced agitation but inconclusive sleep impact. 15 Limitations of the current literature on permanent light xtures includes: incomplete de nitions of lighting protocols; and the use of permanent xtures only in common rooms, not inclusive of hallway or bedroom lighting.
The purpose of this pilot was to inform the design and implementation of a Stage III real-world e cacy randomized, controlled trial. 16 In addition to documenting a best practice protocol for implementing tunable lighting in the NH setting 17 and determining stakeholder acceptability, 18 we wanted to use our pilot to determine the feasibility of our assessment strategy. 19

Sample & Data
There were two study samples of interest. The rst included all residents who had been in the NH at least 90 of the last 100 days on November 1, 2018. The second sample comprised the subset of all eligible residents who had a dementia diagnosis on November 1, 2018.
There were two sources of study data. The rst data source was resident care planning assessments that are conducted quarterly by NH staff using a nationally-standardized instrument, the Minimum Data Set (MDS) version 3.0. 21,22 Eligible residents were required to have two comprehensive MDS assessments, one between November 1, 2018 and January 31, 2019 and one between February 1, 2019 and April 30, 2019.
The second data source was survey data collected on-site by two trained data collectors (EM and CM) with experience interviewing NH staff. 23 We collected survey data at two time points: January

Resident Characteristics
All resident characteristics were de ned using MDS version 3.0 data. Resident demographic characteristics included age, sex, and race. Resident diagnoses of interest included: Alzheimer's or other dementias; depression; anxiety; and bipolar disorder or schizophrenia. Resident medication use was de ned as the number of days in the past week that the resident received antipsychotics, antianxietals, antidepressants, hypnotics, and opioids. Moderate or severe cognitive impairment was de ned as a score of 3 or 4 on the Cognitive Function Scale. 24 Resident physical dependencies were de ned by requiring extensive or total assistance with: transferring; moving around the NH; and eating. Pain in the last ve days was based on resident self-report or staff reported pain symptoms.

MDS-Based Sleep and Agitation Disturbance Measures
Sleep disturbances are not formally measured in the MDS. However, residents are screened for sleeprelated symptoms of depression using the Patient Health Questionnaire-9 Item (PHQ-9), which asks how often in the past two weeks the resident has had "trouble falling asleep or staying asleep or sleeping too much." Responses included: never or 1 day (0); 2-6 days (1); 7-11 days (2); or 12-14 days (3). Responses are captured by interviewing the resident or asking staff when the resident is not able to be interviewed.
To measure agitation in the past week, we focused on four MDS items: physical behaviors directed toward others; verbal behaviors directed toward others; other behaviors not directed toward others; and rejection of needed evaluation or care. The frequency with which each behavior occurred in the past week was recorded as never (0), 1 to 3 days (1), 4-6 days (2), or daily (3). These four items were summed to create the Agitated and Reactive Behavior Scale. 25

Survey-Based Sleep and Agitation Disturbance Measures
The two data collectors (EM and CM) interviewed nighttime nursing staff who knew the residents well using the Sleep Disorders Inventory (SDI). 26 The SDI assesses the frequency of the following behaviors in the past two weeks: di culty falling asleep; getting up during the night; wandering, pacing or getting involved in inappropriate activities at night; waking up residents during the night; waking up and thinking it is morning; and getting up too early. The frequency of each behavior is recorded as: never, less than once per week (1), 1-2 times per week (2), several times per day but not every night (3), or every night (4).
For each behavior, severity is also assessed as mild (1), moderate (2), or marked (3). Total SDI score is derived by summing the products of the frequency and severity ratings for each behavior.
To assess the frequency of agitated behaviors, the two data collectors (EM and CM) interviewed daytime nursing staff who knew the residents well using the Cohen-Mans eld Agitation Inventory (CMAI). 27 The CMAI asks about 29 verbally-and physically agitated and aggressive behaviors, such as yelling, cursing, kicking, or pacing. Staff were asked how often each of these behaviors occurred in the past two weeks: not applicable (0), never (1), less than once a week (2), once or twice a week (3), several times a week (4), once or twice a day (5), several times a day (6), or several times an hour (7). The total CMAI score is derived by summing the reported frequencies for all behaviors.

Analysis
We used McNemar's test and Wilcoxon matched-pairs signed-ranks tests to compare tuned and static measures of agitation and sleep. These tests were chosen because of the small sample, repeated measurement, and the non-normal distribution of the data. All analyses were conducted using STATA, SE Version 16. This pilot study is not intended to be used to determine sample or effect sizes for future RCTs. 19

Results
There were 68 residents with an eligible assessment in both time periods, 62 of whom had sleep survey data available for both time periods. Characteristics of these 62 residents are provided in Table 1. The average resident age was 89.8 (SD:7.6), 69% were women, and 77% were Asian. Residents experienced signi cant cognitive and physical impairment: over half had a dementia diagnosis and 39% had moderate or severe cognitive impairment; 81% require extensive or total staff support to transfer from a bed to a chair, 53% are unable to move around independently, and 23% require extensive or total staff support to eat. Nearly one-third of residents (29%) were diagnosed with depression and almost a quarter (23%) received an antidepressant in the past week. Residents were also likely to experience pain in the last ve days (44%) and 19% received an opioid in the past week. Characteristics of the 35 residents with dementia were similar to the entire sample, except they had higher levels of moderate or severe cognitive impairment (54%) and more depression (34%).  n = 35 ‡ Item D0200C1="Over the past two weeks, have you had trouble falling asleep or sleeping too much?" § Average of categorical response categories: 0 = never, 1 = less than once per week, 2 = 1-2 times per week, 3 = several times per day but not every night, 4 = every night || Average of categorical response categories: 0 = never or 1 day, 1 = 2-6 days, 2 = 7-11 days, 3 = 12-14 ** Items E0200A-C, E0800 = physical behaviors directed toward others, verbal behaviors directed toward others, other behaviors not directed toward others, and rejection of needed evaluation or care †Average of categorical response categories: 1 = never; 2: less than once a week, but still occurring, 3 = once a week; 4 = several times a week; 5 = once or twice a day; 6 = several times a day; 7 = several times an hour †Average of categorical response categories: 1 = behavior occurred 1 to 3 days; 2 = behavior occurred 4 to 6 days; 3 = behavior occurred daily Sleep Based on interviews with nighttime nursing staff who knew the resident well, there was no difference in the percent of residents with any sleep disturbances under the static and tuned lighting conditions (37% vs 34% respectively). Based on the MDS sleep measure, nine of the 62 residents (15%) had any sleep disturbances under the static lighting condition, compared to three of the 62 residents (5%) under the tuned lighting condition (p < .10). Based on interviews, the frequency of sleep disturbances under the static lighting was 3.6 (SD:7.2) and under the tuned condition 1.8 (SD:3.8) (p < .10). Based on the MDS sleep measure, the frequency of sleep disturbances under the static lighting was .15 (SD:.36) and under the tuned condition .06 (SD:.31) (p < .10). Among residents with dementia, only the frequency of sleep disturbances as measured by staff interview was statistically different between tuned and static groups [4.6 (SD:8.1) and 2.5 (SD:4.5) respectively, p < .10].

Agitation
Based on interviews with daytime nursing staff who knew the residents with dementia well, there was no statistical difference in the percent of residents with any agitation under the static and tuned lighting conditions (49% vs 60% respectively). Based on the MDS agitation measure, there was no statistical difference in the percent of residents with any agitation under the static and tuned lighting conditions (29% vs. 34% respectively). Based on interviews, there were no differences in the frequency of agitated behaviors under the static and tuned lighting conditions [36.9 (SD:9.2) and 36.2 (SD: 7.7) respectively].
Based on the MDS agitation measure, there were no differences in the frequency of agitated behaviors under the static and tuned lighting conditions [.49 (SD:.89) vs. .57 (SD:.95) respectively].

Discussion
These pilot results are important for researchers seeking to conduct real-world e cacy trials of nonpharmaceutical interventions to address sleep disturbances among NH residents. There is no validated sleep inventory embedded in the current version of the MDS; the MDS item used to assess presence and frequency of sleep disturbances is based on one item from the PHQ-9. The MDS version of the PHQ-9 has been shown to have good measurement 28 and criterion validity, 29 but the scale was not developed to measure sleep. Respondents may not report sleep disturbances that they believe are unrelated to depression, which may partially explain why only 15% of residents had any sleep disturbance reported in the MDS compared to 37% of the same residents based on staff interview data.
One limitation of this pilot was an incomplete alignment between the study window and the MDS assessments. For the full trial, we would like to be able to coordinate the staff interviews to occur during the same week as the regularly scheduled MDS assessments. Another limitation was the effect of language barriers on reporting of agitated behaviors. Many of the residents in this NH spoke Mandarin, Cantonese, or Japanese. A few CMAI items are dependent on staff and residents sharing a common language (e.g., questions asking whether residents are repeating the same questions / sentences over and over). Another barrier to administering the CMAI was time constraints. Day staff were required to be off the oor for 15 minutes per interview. Interviewing staff about resident sleep disturbances using the SDI was less burdensome because the interviews were ve minutes each and they were conducted between midnight and 2 am.

Conclusions
Based on the results of this pilot, we recommend that sleep be measured by the gold standard SDI for the Stage III trial. By collecting the MDS and the SDI during the same week, the e cacy trial will help researchers to understand the relationship between these measures and the degree of sensitivity to change in the muted MDS measure. Using administrative data reduces costs and pragmatism of future trials, 30 assuming symptoms are detected at comparable levels to the gold standard measures and the measures are similarly sensitive to change. 31 Depending on budget, the addition of actigraphy may shed further light on sleep disturbances and highlight some of the biases in the existing, recall-based measures. 32 Declarations Ethics approval: Because staff participated in the interviews in a professional capacity and did not provide personal information other than name and title, this analysis is not considered human subjects research or subject to Institutional Review Board approval.