This project involved two phases: Phase 1 was the item generation and development of the assessment scale (cross-sectional design) and Phase 2 was a reliability and validation study for the scale developed in Phase 1 (cross-sectional design).
Phase 1: Item generation and development
Items were generated from two sources for content validity: i) the research team and ii) healthcare professionals. The research team (SWH, AD, JH) generated items from reviewing existing mobility scales, discussion with experts in mobility among people with dementia and a search of healthcare textbooks on clinical skills for mobility aid training. We also conducted two focus groups of healthcare professionals from relevant disciplines in London, Ontario Canada providing care across health settings for people with dementia (n=12; 1 geriatrician, 2 registered nurses, 5 physical therapists and 4 occupational therapists). The healthcare professionals were working in the areas of acute hospital care, community care, long-term care, day hospital and rehabilitation hospital. Eleven (92%) participants had 10 or more years of total clinical experience working in geriatrics. All healthcare professionals provided written informed consent prior to the start of data collection.
Our focus groups identified the 4-wheeled walker as the most common mobility aid used in this patient population. Therefore, items were sought that represented 1) the minimum functional tasks needed to assess and evaluate independence and safety with the use of a 4-wheeled walker, and 2) the components within each task that can be objectively evaluated with respect to function and safe use.
The focus groups and work by the research team generated 11 tasks and components within each task. To render the scale manageable in the clinical and research setting, the authors sought to reduce the number of items. The information was presented to an independent panel of five healthcare professionals (1 registered nurse, 2 physical therapist, and 2 occupational therapists) with more than 10 years of clinical experience in geriatrics and who had not participated in the first set of focus groups. The participants were asked, “Rank how important it is to include each task, regardless of setting, when assessing physical function and safety with a 4-wheeled walker”. The 11 identified tasks were ranked on a 5-point Likert scale with a score of 1 representing “least important” and 5 representing “most important”. The scores for each task were summed and averaged across raters. Tasks were kept if the average score was rated ≥ 4, indicative of the item being rated as either important or most important. (Table 1)
Table 1. Results of ranking tasks generated in focus groups to be considered part of a minimum set of tasks that every person should complete, regardless of living setting, when assessing physical function and safety with a 4-wheeled walker.
|
Scores on 5-point Likert scale
|
Task
|
Median Score
|
Range of Scores
|
Sit to stand
|
5
|
3-5
|
Pivot turn and sit in a chair
|
5
|
4-5
|
Walking on a level surface
|
5
|
3-5
|
Walking with horizontal head turns
|
4.5
|
3-5
|
Walking with concurrent cognitive task
|
4
|
4-5
|
Walking around obstacles (figure of 8)
|
5
|
All scored at 5
|
Walking up a ramp
|
3
|
2-5
|
Walking down a ramp
|
3
|
2-5
|
Walking through an open doorway
|
5
|
1-5
|
Open, walk through and close door that opens away
|
4
|
2-5
|
Open, walk through and close door that opens in
|
4
|
2-5
|
Note: 5-point Likert scale with a score of 1 representing “least important” and 5 representing “most important”.
The evaluation resulted in a final selection of nine tasks with task components that comprise The Safe Use of Mobility Aids Checklist (SUMAC). The research team categorized the components in each task into two separate areas of focus, physical function (PF) which includes items related to an individual’s physical ability to perform the task (e.g., stand independently) and use of the equipment (EQ) which includes items related to an individual’s safety in using their 4-wheeled walker (e.g., brakes engaged on walker). Items for PF and EQ are not equally present across the tasks and reflect to some extent the difficulty of the activity. The distribution of scoring items for each task (number of PF items:number of EQ items) are: sit to stand (3:7); pivot turn to sit in chair (3:7); walking on level surface (11:6); walking with horizontal head turns (1:6); walking with concurrent cognitive task (1:6); walking around obstacles – figure of eight (1:6); walk through an open doorway (1:5); open, walk through and close door that opens away from the person (1:10); and open, walk through and close door that opens in to person (1:10). Physical function items are rated on a 3-point scale (0, 1, 2) and use of equipment is a dichotomous scale, where the item is scored as observed yes (1) or no (0). Total PF scores can range from 0 to 38 and the total EQ score can range from 0 to 63. A higher score in each category indicates better physical function and safety. (The full scale is available in the supplementary material and at www.mobility-in-aging-lab.ca)
Phase 2: Reliability and validity evaluation
This phase of the study involved the recruitment of people living with dementia and a separate sample of health care professionals. The project was conducted in accordance to the Declaration of Helsinki and was approved by the University of Western Ontario Research Ethics Board for Health Sciences Research Involving Human Subjects.
People with dementia who used a 4-wheeled walker for ambulation were recruited from a local day program to be evaluated with the SUMAC. Participants had a diagnosis of probable AD from a geriatrician based on the National Institute of Neurologic and Communicative Disorders and Stroke-AD and Related Disorders Association (NINCDS-ARDRA) criteria.[23] Inclusion criteria were: 50 years of age and older, English proficiency, able to follow instructions, mild to moderate disease severity, able to walk 60 meters without support from another person and have a substitute decision maker (in all cases a family member of the person with dementia) to provide information about health and daily activities. Exclusion criteria were any muscle and/or nerve problem that limited movement. Informed written consent was provided by either the participant or their substitute decision maker and then the participant provided assent to participate in the study.
Demographic and clinical information collected on the participants included age, gender, falls in the previous 12 months, number of prescription medications, number of comorbidities and instrumental activities of daily living, as per the Lawton-Brody Instrumental Activities of Daily Living and Basic Activities of Daily Living scales [24]. Participants also completed the Iconographical-Falls Efficacy Scale, which has been validated in older adults with cognitive impairment [25]. Participants (n=10) were videotaped while performing the nine tasks of the SUMAC.
The raters were a convenience sample of five healthcare professionals (5 physiotherapists). All healthcare professionals provided written informed consent prior to the start of data collection. The healthcare professionals were working in the areas of acute hospital care, community care, and long-term care. Two people (40%) had 10 or more years of total clinical experience working in geriatrics. The inclusion criterion was: registered healthcare professional with experience working with older adults with dementia. Each assessor attended a one-hour one-on-one training session on the use of the SUMAC. In the training session, each person was presented with a description of the development of the tool and the rationale for its creation, received a copy of the assessment tool and was given a detailed instruction of the components and items within the tool. The last activity in the training session was evaluating videos of a person performing the nine tasks of the SUMAC.
Reliability
In the reliability evaluation, each of the healthcare professionals was asked to view each participant’s video and evaluate them using the SUMAC on two occasions. The two visits were set one week apart. In both visits the viewing order of each participant’s video was randomized, but all components of the SUMAC were presented together and in the order in the tool for each participant.
Validity
The evaluation of construct validity was performed by a panel of eight healthcare professionals with clinical experience working with older adults and people with dementia (6 physical therapists, 2 registered kinesiologists) who had not participated in any aspect of the development or reliability evaluations of the SUMAC. All members of the panel had the same training session used in the reliability study. The panel viewed the participant videos in a group setting in a single session with discussion until consensus was reached in scoring the physical function and safe use of the equipment. The scores of the HCP panel were compared to the scores of the individual HCP from the reliability study.
Our sample of five HCP who participated in the reliability study of the SUMAC also completed the gait subscale of the Performance-Oriented Mobility Assessment (POMA) gait subscale on each participant. The HCP viewed videos of each participant walking at their self-selected usual gait speed using their 4-wheeled walker over a distance of a 6-meter. Criterion validity was evaluated by comparing the PF and EQ scores of the SUMAC against the Performance-Oriented Mobility Assessment (POMA) gait subscale scores.[26] The POMA is a reliable tool in people with dementia [27]
Data Analysis
a) Reliability: Values for relative and absolute reliability were calculated. An a priori sample size calculation (α=0.05 and β=0.20) for the reliability study indicated that 10 participants and 5 assessors making 2 evaluations would be needed if a target ICC of 0.90 was desired.[28] This sample size minimized recruitment and participant burden while optimizing the use of multiple healthcare professionals as assessors of the videos.
The relative reliability values of inter-rater and test-retest reliability were calculated for the PF and EQ domain scores of the SUMAC using the intra-class correlation coefficient (ICC). Repeated measurements by different raters on the same day were used to calculate inter-rater reliability, while repeated measurements by the same rater on different days were used to calculate test-retest reliability. An ICC value of greater than 0.90 is considered excellent, between 0.80 to 0.89 is good, 0.70 to 0.79 is fair, and values less than 0.70 are of questionable clinical value [29].
Two measures of absolute reliability were calculated: standard error of measurement (SEM) and minimal detectible change with a 95% confidence interval (MDC95) for the PF and EQ domain scores of the SUMAC. The SEM is the measurement error associated with a single value and is expressed in the same units as the scale [29]. The smaller the SEM, the greater the absolute reliability. The MDC95 is an estimate of the smallest change in the score that can be detected beyond measurement error [30]. It is also measured in the same units as the measurement scale. For the present study, the SEM was calculated using pooled standard deviation (SD) and ICC values for each group. Calculations of SEM and MDC95 were:
b) Validity: Spearman’s rank-order correlations were also used to assess construct validity between each healthcare professional and the separate HCP panel consensus scores. Three scores were compared: each participant’s mean score for each of the nine tasks and a total score using the minimum and maximum scores.
Spearman’ rank-order correlation analysis was used to evaluate criterion validity between the gait component of the POMA, and the SUMAC-PF and SUMAC-EQ scores. We hypothesized that the POMA score would be moderately correlated with the SUMAC-PF score and not correlated with the SUMCA-EQ score.
To interpret Spearman's correlation coefficients related to the assessment of criterion and construct validity, the following thresholds were used: ≥0.50 was deemed strong, 0.31-0.49 was moderate to strong, 0.11-0.30 was weak to moderate, and ≤0.10 was considered a non-existent relationship.[31] All statistical analyses were performed using SPSS version 25.0 (IBM Inc., Chicago, IL, USA). Statistical significance was set at p<0.05.