Validity of the Chinese version of the Montreal Cognitive Assessment (MoCA) 5-minute protocol for patients with stroke


 Background and propose The aim of the present study is to examine the validity of 2 Chinese versions of the short-form Montreal Cognitive Assessment (MoCA) in patients with stroke.Methods A prospective observational study was conducted which included patients with stroke as compared to healthy controls. The Chinese version of the Montreal Cognitive Assessment 5-minute protocol (MoCA 5-min protocol), National Institute for Neurological Disorders and Stroke and Canadian Stroke Network 5-minute Protocol (NINDS-CSN 5-min protocol), and Neurobehavioural Cognitive Status Examination (NCSE) were administered to each participant.Results A total of 54 patients with stroke and 27 healthy controls were enrolled in this study. We identified patients with cognitive impairments using the NCSE, and found that the 5-min protocol was equivalent to the MoCA in differentiating patients with cognitive impairments from those without (area under the receiver operating characteristic curve, AUC, of 0.948 for the MoCA 5-min protocol v.s. 0.984 for MoCA, P = 0.097); however, the NINDS-CSN 5-min protocol was less effective than MoCA in differentiating patients with cognitive impairments from those without (AUC of 0.925 for the NINDS CSN 5-min protocol v.s. 0.984 for MoCA, P = 0.027). These three assessments demonstrated equal performance in differentiating patients with stroke from controls.Conclusions The Chinese version of the MoCA 5-min protocol can be used as a valid screening for patients with stroke.

The Montreal Cognitive Assessment (MoCA) was developed by Nasreddine et al. in 1996, and was initially used and validated to detect mild cognitive impairment (MCI) and mild forms of Alzheimer's disease [1]. Recent evidence has demonstrated that the MoCA is, at the present moment, the best screening tool for MCI (pooled sensitivity, 89%; pooled specificity, 75%) [2], and that it is also an excellent screening tool for post-stroke cognitive impairment (pooled sensitivity, 81%; pooled specificity, 79%) [3]. Although MoCA is primarily designed as a cognitive screening tool, it is still relatively time consuming (taking approx. 15 minutes), with limited feasibility for use with patients with either low education levels, or being in very poor medical condition [4]. In recent years, there has been an emerging interest in concentrating the most useful items from the MoCA into a similarly efficacious, but shorter version of MoCA. In 2006, the National Institute for Neurological Disorders and Stroke-Canadian Stroke Network (NINDS-CSN) proposed a 5-minute protocol that can be adopted as a possible cognitive screening tool for patients with stroke, by extracting three items (i.e. orientation, memory and verbal fluency) from the MoCA.
This assessment protocol is aimed to meet the requirement of busy clinics and also to allow for administration over the telephone [5]. In accordance with the recommendations of the NINDS-CSN, two short versions of MoCA have been developed to date [6,7]. Pendlebury et al. are the first group to validate a short version of MoCA (i.e., NINDS-CSN 5-min protocol) in patients with stroke or transient ischemic attack (TIA), reporting that NINDS 5-min protocol was overall less effective to the full MoCA in screening MCI among patients with stroke or TIA [6]. Another study which used the clinical diagnosis criteria of the American Heart Association/American Stroke Association for post-stroke dementia as the reference standard, found that the NINDS-CSN 5-min protocol has an acceptable discriminative power (sensitivity 82%, specificity 67%), and therefore can be a useful significant predictor of post-stroke dementia (odds ratio [OR], 6.32) [8]. Other evidence, A study by Dong et al. suggested that the NINDS-CSN 5-min protocol could predict the occurrence of late onset MCI (i.e. one-year after stroke), though its predictive validity was slightly poorer than the full MoCA [9]. When compared to other validated cognitive screening tasks, such as the full MoCA [10] as well as the Clinical Dementia Rating scale (CDR) [11], the diagnosis accuracy of NINDS-CSN 5min protocol was found to be high with balanced sensitivity and specificity.
A second short version of MoCA was developed by Wong et al. [7], which was later named as the MoCA 5-minute protocol [4]. This protocol uses the same criteria (i.e. orientation, delayed recall and verbal fluency) from the full MoCA like the NINDS-CSN 5 min protocol, but with minor modifications in the assessment instruction and the grading system. To date, only four studies examining the Chinese version of the MoCA 5-min protocol have been published (2 in Hong Kong and 2 in Nanjing) [4,7,12,13]. This protocol has shown an acceptable level of reliability, which includes test-retest reliability and internal consistency [4,13]. We considered the validity of the MoCA 5-min protocol by first, its ability of the to detect cognitive impairments among patients with stroke [4,13], and second, to distinguish patients with stroke from healthy controls [7,12]. The noted area under curve (AUC) of the MoCA 5-min protocol ranged from 0.76 to 0.86. Furthermore, one of these studies also found that the MoCA 5-min protocol is as equally effective as the full MoCA in differentiating stroke or TIA cohorts that have cognitive impairments (CDR > 0.5) from those without cognitive impairments (AUC: 0.78 v.s. 0.74, P > 0.05) [4].

Inclusion and exclusion criteria
The participants who met all of the following inclusion criteria were included: (1) patients diagnosed with unilateral hemispheric subacute stroke (i.e. more than one month after stroke onset) using neuroimaging evidence from either computer tomography (CT) or magnetic resonance imaging (MRI); (2) patients that were aged between 18 to 80 years; (3) Chinese (Mandarin)-speakers; and (4) patients who are able to communicate and to follow at least one-step commands.
The participants who met any of the following criteria were excluded: (1) patients with severe aphasia, swallowing disorders or other complications post-stroke that prevent them from successfully completing the assessments; (2) patients with any previously known psychiatric disorder or neurological disease excluding stroke; (3) patients who have been assessed by the MoCA or NCSE within the past three months and (4) those who refuse to participate in this study.
Cognitively normal healthy controls were enrolled from the relatives of the patients or staff in the hospital, with an inclusion criterion of: (1) no known history of neurological or psychiatric disorders; (2) no complaints of prior cognitive issues; (3) Chinese (Mandarin)-users and (4) agreement to participate in this study.

Montreal Cognitive Assessment (MoCA)
The MoCA Chinese (Mandarin) version-1 (from http://www.mocatest.org) was used in this study. The total score for the MoCA is 30 and it covers seven domains of cognition: visuospatial/executive functions (trail-making test: 1 point, copy tube: 1 point and clock drawing task: 3 points), naming (3 points), attention (forward digit span: 1 point, backward digit span: 1 point, vigilance: 1 point and serial 7 subtraction: 3 points), language (sentence repetition: 2 points, verbal fluency: 1 point), abstraction (2 points), delayed recall (5 points) and orientation (6 points) [1]. According to previous studies on the MoCA based on the Chinese population, we added one point to the final scores of the MoCA if the participants had 6 years of formal school-based education or fewer [15].

Network 5-Minute protocol (NINDS-CSN 5-min Protocol)
The NINDS-CSN 5-min protocol was performed by extracting three items (i.e. 7 orientation, delayed recall and verbal fluency) from the full MoCA, which results in a total score of 12 [5,6].

Montreal Cognitive Assessment 5-minute Protocol (MoCA 5-min protocol)
The MoCA 5-min protocol was previously developed by Wong et al. [7]. The protocol first starts with an attention task by using the first trial of the 5-word delayed recall test. Second, verbal fluency score was evaluated based on the output of the effective number of words, rather than using a single cut-off value based on the number of animal names. Third, the recall test was conducted with cues (category

Neurobehavioral Cognitive Status Examination (NCSE)
The Chinese (Mandarin) version of NCSE was used in this study [14]. This examination covers 10 sections: orientations, attention, comprehension, repetition, naming, construction, memory, calculation, similarities and judgement. All sections, except orientation and memory, contain 1 screening task and series of metric tests with a gradual increase in difficulty. The total score of the NCSE is 82. In line with prior studies, a cut-off value of 65 was used to define the cut-off for cognitive 8 impairment in patients with stroke [16].

Concurrent validity
Among the 54 stroke patients, subjects with sub-optimal NCSE scores (i.e. NCSE < 65) had poorer performance in the MoCA (t = 9.69, P < 0.001), the MoCA 5-min protocol (t = 9.11, P < 0.001) and the NINDS-CSN 5-min protocol (t = 7.57, P < 0.001), when compared to those within the normal range as defined by NCSE. The  (Fig. 1B). Similarly, the NINDS-CSN 5-min protocol demonstrated high correlation with the full MoCA (r = 0.921, P < 0.001) and the NCSE (r = 0.797, P < 0.001, Fig. 1C and 1D). To examine the contribution from each item of the MoCA 5-min protocol and the NINDS-CSN 5-min protocol, correlation analysis between each item and total scores of the assessments was conducted. Pearson's correlation analysis of the MoCA 5-min protocol indicated that delayed recall had strongest association with the total result (r = 0.897, P < 0.001), followed by verbal fluency (r = 0.858, P < 0.001), as well as two other items which were also associated with the total at a lesser degree (attention: r = 0.850, orientation: r = 0.625, both P < 0.001). Pearson's correlation analysis of the MoCA 5min protocol indicated that orientation had strongest association with the total (r = 0.771, P < 0.001), followed by delayed recall (r = 0.764, P < 0.001), and verbal fluency (r = 0.706, P < 0.001).

Diagnostic accuracy by a known-group method
Known-groups method is a way to support criteria validity, and we used this method to test the diagnostic accuracy of the MoCA, the MoCA 5-min protocol and the NINDS-CSN 5-min protocol in discriminating stroke patients from their healthy counterparts. We found that the MoCA 5-min protocol had the largest AUC (AUC: 0.675, 95% CI, 0.562 to 0.775), followed by the NINDS-CSN 5-min protocol (AUC: 0.656, 95% CI, 0.542 to 0.758), and then the full MoCA (AUC: 0.618, 95% CI, 0.504 to 0.724) (Fig. 2B). However, we could not find any significant differences among the ROC curves (all P > 0.05).

Diagnostic accuracy with reference to NCSE
We further tested the diagnostic accuracy of three assessments in differentiating stroke patients with cognitive impairment (NCSE < 65) from those without cognitive impairment. We found that the MoCA 5-min protocol has a statistically similar AUC 11 to the full MoCA (AUC: 0.948 (0.851 to 0.990) vs. 0.984 (0.904 to 1.000), P = 0.0971) (Fig. 2A). The full MoCA presented with a significantly better AUC than the NINDS-CSN 5-min protocol (AUC: 0.925 (0.819 to 0.979) vs. 0.984 (0.904 to 1.000), P = 0.027). The optimal cut-off value of the MoCA 5-min protocol was 21.5, with a sensitivity of 100.0% and a specificity of 81.48%, while the optimal cut off value for the NINDS-CSN 5-min protocol was 8, with a sensitivity of 88.89% and a specificity of 81.48%. The optimal cut-off value of the full MoCA was 22, with a sensitivity of 100.0% and a specificity of 85.19%.

Discussion
The aim of this study is to examine the validity and diagnostic accuracy of two short-form versions of the MoCA at cognitive screening of a cohort of stroke patients in 2 rehabilitation hospitals. Our study demonstrated that the MoCA 5-min protocol, but not NINDS-CSN 5-min protocol, was comparable with the full MoCA in differentiating patients with cognitive impairment from those without cognitive impairment, when the NCSE is used as the reference criteria. The three assessments performed similarly in differentiating patients with stroke from healthy controls.
Moreover, we compared the two short versions of MoCA with the NCSE and found a strong association. These findings suggest that patients with low scores in the short versions of MoCA tend to have multi-domain cognitive impairments.
In our study, the diagnostic accuracy of the MoCA 5-min protocol was comparable with the full MoCA, which was consistent with previous findings [4]. The AUC of the MoCA 5-min protocol was higher than the results of previous studies (AUC from 0.76 to 0.86) [4,7,12,13], which may be because different reference standards were utilized in these studies. Previous studies have reported lower diagnostic accuracy of the NINDS-CSN 5-min protocol as compared to the full MoCA [6,9], while the MoCA 5-min protocol performed equally well to the full MoCA [4], which is similar to the results from our study. Although the MoCA 5-min protocol only consists of attention tasks, delayed recall tasks, semantic fluency tasks and orientation tasks, the memory and verbal fluency tasks provided more detailed information on the patients' cognitive functions which allows for a broader range of scores for evaluation. Impairments of the frontal-subcortical neuronal circuit have been associated with several neurocognitive impairments, including executive dysfunction and poor memory-search strategies [19]. Memory performance involves three primary processes: encoding, consolidation and retrieval [20]. Poor memory retrieval can be regarded as a subtype of executive dysfunction, which are also known as executive memory deficits [19]. If patients fail to encode new information, spontaneous recall and cued recall will also be both impaired. In contrast, if patients only have memory retrieval problems, cued recall will be preserved to a certain degree. The memory task of the MoCA 5-min protocol by Wong et al. required assessors to examine the patients' performance at both spontaneous recall and cued recall, which may help to enhance the discriminative power of this protocol, as compared to the MoCA and the NINDS-CSN 5-min protocol where only the performance of spontaneous recall was assessed.
In a previous large-scale study using 1523 cases, it was reported that the verbal fluency test was comparable with the full MoCA in detecting cognitive impairments amongst the elderly population [21]. The naming fluency of the MoCA 5-min protocol is scored incrementally rather than using a single cut-off point, which makes it 13 comparable with the full MoCA that contains the backward digit span task, clock drawing task, word similarity task and trail making task used to test executive function. Thus, with the modifications of the delayed recall task and the verbal fluency task, the MoCA 5-min protocol is likely to be a good alternative to the full MoCA as a cognitive screening tool in the Chinese stroke rehabilitation setting.
Stroke patients with suboptimal assessment performance can be subjected to further detailed cognitive assessments.

Limitations
Several limitations are notable in this study. This study was conducted in rehabilitation settings employing only the NCSE, which is a common standardized assessment used by occupational therapists, as the reference standard. Further studies will be needed to explore the validity and diagnostic accuracy of the short MoCA against other established neuropsychological test batteries used by neuropsychologists or even using data from clinical diagnoses conducted by neurologists. Furthermore, we extracted the results of the short MoCA from the full MoCA directly, and therefore were unable to test the feasibility of telephonedelivered assessments in the present study. Further study will need to replicate these results for telephone administration.

Conclusion
The Chinese version of the MoCA 5-min protocol is a valid cognitive screening tool for identifying cognitive impairments in patients with stroke.