Our study suggests our novel EMA app Mindtrax is a feasible method to obtain measures of cognitive functioning in breast cancer survivors and healthy adults. Specifically, our results show that a majority of feasibility criteria were met. Further, Mindtrax demonstrated usability (with self-development, altruism, engagement and functionality as themes) and high convergent construct validity.
Previous EMA research has demonstrated EMA designs are feasible in people with cancer [13, 15]. However, the feasibility of ambulatory cognitive assessments has not been established in both breast cancer survivors and a healthy control group. While our results showed five of the eight feasibility criteria were met, criteria for app compliance, technical smoothness, and usability assessment period were not attained. App compliance (78.84%) was close to attainment (80%) and is consistent with the rate of compliance in prior cognitive task-based EMA studies. A previous meta-analysis of mobile cognitive EMA studies found a mean response rate of 79.2% (58–87%) [12]. Moreover, group-specific compliance rates revealed breast cancer participants did meet the criteria for app compliance (82.11%), consistent with a previous cognitive EMA study in breast cancer survivors [16]. This is particularly encouraging as our sampling period was longer than previous cognitive EMA studies [12, 13], demonstrating the potential for being able to track CRCI in breast cancer survivors over longer time periods.
Regarding technical smoothness, variations in the operating systems of participants’ devices introduced technical complications (such as issues with notifications), which prevented some participants from completing sessions. Though, EMA studies frequently report missing data often due to missed sessions [12, 13]. In particular, EMA studies in people with cancer have found up to 39% of their data missing [13]. Future studies should employ advanced mixed models to deal with the missing data [33]. The final unmet feasibility criterion, participants completing usability questionnaire within seven days, may be explained by the usability questionnaire being sent via email. Future studies should integrate the usability questionnaire in the final session of the app.
Unlike the present study, few studies have employed self-report measures and open-ended opinions on usability for cognitive EMA tasks to explore participants’ experience. We found participants' overall experience of using the app was rated highly, with few participants reporting negative experiences with the app. Our inductive content analysis revealed that while some participants enjoyed the self-reflection and challenge aspects of completing the tasks, some breast cancer survivors felt self-judgment when they felt they performed poorly on a task. Our results are consistent with previous cognitive EMA studies in people with mild cognitive impairments who reported mixed responses when asked if being more aware of their memory was unpleasant [24]. Future studies should be mindful that feelings of self-judgment may arise in clinical groups when completing cognitive EMA. Participants highlighted that they found notifications to be an important factor in completing daily sessions consistently. Ensuring consistent notifications should be a priority in future EMA studies. Several participants highlighted that they enjoyed being a part of research and the potential for the research to help others. Similar altruistic sentiments were highlighted in a recent study exploring the motivation to participate in CRCI research on patients with newly diagnosed aggressive lymphoma [27].
Importantly, our EMA tasks were correlated with in-person neuropsychological assessments (average r > 0.53), consistent with previous cognitive EMA tasks [12, 24, 34]. This suggests the Mindtrax tasks have good convergent construct validity. However, the EMA flanker task congruent/incongruent reaction time ratio did not correlate with the equivalent NIH flanker ratio. This may have been due to several methodological factors such as a smaller condition effect in the ambulatory task, too easy (due to the salience of the stimuli) [35] and ceiling effects. Ease of the task can be addressed by adjusting the salience of the stimuli and/or decreasing the time the stimulus is presented. Our tasks also showed moderate-strong associations with NIH tasks of unrelated constructs. However, it is not uncommon for cognitive assessments to be associated with assessments of other constructs, for example several NIH cognitive toolbox tasks show moderate correlations with other NIH tasks [36]. There was evidence of training effects in our card matching task reaction time due to ceiling effects of the EMA card matching task. All other EMA variables did not show evidence for training effects, supporting the feasibility of EMA cognitive tasks over longer durations of time.
Study limitations
While our study demonstrates the feasibility, usability and validity of our novel EMA app tasks, there were several limitations. Firstly, due to our relatively small sample size and our groups not being well matched with gender and age, we analysed the construct validity of the cognitive tasks as one group. Despite not being well matched on gender and age, our groups were well matched on education and IQ which is more of a confounding factor. We recommend future large-scale studies should be undertaken to comprehensively describe EMA cognitive assessments. Secondly, our app required internet access while completing tasks, which may have negatively impacted participants engagement in the tasks, however our participants did not highlight this as a barrier to completing tasks.
Clinical implications
Our findings revealed that a long sampling period, i.e., daily sampling of cognitive functioning across 30-days, is feasible in breast cancer survivors. This finding suggests that our novel EMA app may enable the investigation of daily cognitive deficits throughout the different stages of cancer treatment and survivorship. Furthermore, the breast cancer survivors (and non-cancer controls) both rated the experience of using the app positively, supporting the usage of cognitive assessments in an app format. The app also enables the engagement of patients in rural or remote communities, thus broadening its geographical scope, scale and reach. App-based cognitive assessments will enhance equity in health-care delivery in future work.