Aims and hypothesis
The overall aim of this project is to develop TapTalkTest – a smartphone test that detects risk of AD pathology and is usable, reliable and validated against blood-biomarkers of AD pathology, cognitive screening tests and clinical diagnosis. We will address the hypothesis: “Hand-speech movement patterns will detect the risk of Alzheimer’s disease pathology in research and clinical cohorts of older adults”, where research cohorts have normal cognition and no cognitive symptoms, and clinical cohorts have cognitive symptoms.
Our interdisciplinary team of clinician-researchers (Neurologist, Speech Pathologist, Physiotherapist, General Medicine Physicians, General Practitioner (GP), Neuropsychologist), computer scientists (specialising in Artificial Intelligence and machine learning), data analysts, and neuroscience researchers (specialising in ageing, dementia and blood-based biomarkers) will work together to address the hypothesis through three studies, with each study addressing one of the following aims:
Aim 1. Determine which combinations of hand-speech movement data most accurately predict preclinical AD.
Aim 2. Develop smartphone capability for TapTalkTest and determine usability, reliability and validity.
Aim 3. Prospectively validate TapTalkTest in people who have cognitive symptoms against gold-standard consensus clinical diagnoses of MCI and AD dementia, with comparison to other screening tools commonly used in clinical settings.
AIM 1
Design
We will undertake a cross-sectional single site study. Overview of the methods is presented in Fig. 1.
Participants
We will recruit 500 eligible participants (see below for inclusion and exclusion criteria) with normal cognition from the ISLAND Project; this project was launched in October 2019 at the University of Tasmania as a 10-year prospective cohort study of Tasmanians ≥ 50 years old with nested interventions to reduce dementia risk (45, 46). To date, > 14,000 people have been recruited to the ISLAND Project; 8,500 research participants (mean age 63 years) have provided detailed demographic and health data, 3,000 have completed the online validated Cambridge Neuropsychological Test Automated Battery (CANTAB) cognitive tests (47), 1,800 have completed online motor-cognitive tests (TAS Test) (38) and more than 1,500 have provided blood samples. Paired Associates Learning (PAL) scores distinguish between older adults with MCI and those who are cognitively healthy with a sensitivity/specificity of 0.83/0.82 (48). The online ISLAND Project surveys are repeated annually with cognitive performance and blood-based biomarkers collected every second year.
Inclusion criteria: Adults ≥ 50 years old who are (i) participants in the ISLAND Project (45) (ii) have provided a blood sample for analysis of blood-based biomarkers, (iii) have normal cognition defined as not ≥ 2 SD above the mean total errors (adjusted for age) on the CANTAB PAL tests, and (iv) have no history of persistent cognitive symptoms reported on ISLAND Project questionnaires (all participants are asked each year: Have you noticed a substantial change in your memory and mental function in recent years? [YES/NO], Have you been told by a doctor that you have dementia? [YES/NO], Have you been told by your doctor that you have a memory impairment but they were uncertain if you have dementia [YES/NO], Have you discussed concerns about your memory and mental function with your doctor or other health professional? [YES/NO]).
Exclusion criterion: Adults < 50 years old or those who have impaired cognition, defined by a diagnosis of dementia or MCI or a validated cut-off score ≥ 2 SD above the mean total errors, adjusted for age, on the CANTAB PAL tests or reporting ‘YES’ on any of the ISLAND annual questions ‘Have you noticed a substantial change in your memory and mental function in recent years?’ Have you been told by a doctor that you have dementia? [YES/NO], Have you been told by your doctor that you have a memory impairment but they were uncertain if you have dementia [YES/NO], Have you discussed concerns about your memory and mental function with your doctor or other health professional? [YES/NO]).
As part of the ISLAND Project, we will measure blood levels of a range of proteins known to be associated with AD pathology, including p-tau 181 (45, 49), glial fibrillary acidic protein (GFAP) and neurofilament Light (NFL) (45, 49). Although minimally invasive, the practicalities and cost of accessing the specialist analytic equipment limit wide accessibility. We will use ultrasensitive Simoa® immunoassays measured using the SR-X™ platform from Quanterix™ for blood biomarker analysis in a dedicated biobank with − 80°C freezers for sample storage.
Based on the latest scientific literature, and recognising emerging work around variability we will either define cut-off ranges for biomarker levels that are indicative of risk for AD dementia or examine associations across continuous measures (50). This ‘AD biomarker’ in the TapTalkTest project is likely to be based on one or more blood tests for validated proteins such as p-tau181, GFAP and NFL (12, 51, 52). The field of AD blood biomarkers is developing rapidly (14), and we anticipate further pathological markers will become available during the life of this project; we will assess such new markers in the samples collected from study participants.
Setting
Most participants will complete the hand and speech-like movement tests online remotely either at home, or another setting of their choosing, using their own laptop or desktop computer. If they cannot access a computer with a computer camera and microphone, they may attend one of the University of Tasmania research sites to complete the tests on a university laptop. The TAS Test software is an online ‘self-test’ program that guides the participant through a series of screens to collect a range of motor, cognitive and speech tests (38). It has already successfully collected finger tapping and hand movement data online (as well as various cognitive tests) from ~ 1,800 ISLAND Project participants at home and ~ 400 participants at University of Tasmania research settings (19, 42). For the TapTalkTest project, the TAS Test software will be adapted to collect new finger tapping test data and include an online version of DDK tasks to collect speech-like data.
The full protocol of TAS Test is described in Alty et al; (38) in brief, when participants log into the TAS Test software, and provide consent online to take part, they are shown an introductory video and general instruction screens. Next, they are asked for permission for the software to switch on their computer camera and microphone. Each individual test (e.g., ‘right hand finger tapping at comfortable pace’ test) within TAS Test has an ‘instruction’ screen followed by a ‘recording’ screen, followed by a “well done, you have completed the test” message. To move to the next screen, the participant clicks the ‘next’ button which means the tests are self-paced and participants can take a break at any time. They are shown their progress through TAS Test by green dots on a ‘progress bar’ at the top of the screen; see Fig. 2.
On each instruction screen, participants will be shown a 5–10 second looped video that demonstrates the hand, or speech-like movement that is required for the task. There are text instructions next to the video and participants may click on an ‘audio’ icon if they would like to hear spoken instructions (the text instructions are read out aloud). When participants are performing the hand or speech-like movements on the ‘recording’ screen, the computer video camera will record the hand movement data and the computer microphone will record the audio data respectively.
Hand movement data collection
For each of the hand movement tests, the participant is asked to hold their hands up about 50 cm in front of the computer camera and adjust their position until they can see an image of their hands on the screen fitting completely inside green ‘data collection’ boxes. This careful positioning of the hands at the start of each test assures that the camera can record the movement patterns accurately. Computer-user interface technologies are embedded into the TAS Test software that respond to the positioning of the participant’s hands and give them on screen text prompts (e.g., “your right hand is correctly positioned”, “move your left hand closer to the screen”, or “well done your hands are in the correct position”) to prompt adjustments in the hand positioning as required. These technologies have been purposefully included to allow TAS Test to collect robust data at home, or any remote setting, without any in-person researcher assistance being required (42).
The hand movement tests comprise tests of repetitive finger tapping (index finger tapping against thumb) tests, and of repetitive ‘sequential’ finger tapping (index, middle then ring finger each in turn tapping against the thumb, then index, middle, ring etc). The sequential finger tapping tests are a commonly used clinical test of motor function chosen in this study for direct comparison with the Sequential Motion Rate analysis in the DDK test. Each test is about 10 seconds duration and completed in a fixed order as follows: (i) at a comfortable pace: right hand finger tapping, left hand finger tapping, right hand sequential finger tapping, left hand sequential finger tapping; (ii) at a maximal pace ‘as big and fast as you can’: right hand finger tapping, left hand finger tapping, right hand sequential finger tapping, left hand sequential finger tapping; (iii) at maximal pace: both hands finger tapping together in phase, both hands finger tapping together out of phase, both hands finger tapping together in phase with a cognitive task, both hands finger tapping together out of phase with a cognitive task.
‘In phase’ means that both hands move simultaneously through the finger tapping cycle, whereas ‘out of phase’ means that each hand moves through the finger tapping cycle in the opposite direction to the other hand (i.e., 180 degrees out of phase so one hand has the finger and thumb opposed while the other hand has the finger and thumb spread apart). The cognitive task will be counting backwards aloud from 100 and constitutes a ‘dual motor-cognitive test’ when performed at the same time as finger tapping (53, 54). Each hand will therefore be tested separately, and together, at a comfortable pace and at maximal pace, allowing for a number of finger-tapping measures to be calculated including, but not limited to, frequency, rhythm, inaccuracies (i.e. the wrong digit tapped on the sequential tests), ‘motor reserve’, calculated as a ratio (the maximal pace parameter divided by comfortable pace parameters), bimanual ‘motor cost’, calculated as the finger tapping parameters of one hand divided by the finger tapping parameters of same hand when both hands tapped together and the ‘dual cognitive-motor cost’, calculated as the finger tapping parameters divided by the finger tapping parameters during the counting backwards aloud from 100 task.
We will shorten this finger tapping protocol to around three to five tasks, aiming for a test duration of 2 to 3 minutes, in the smartphone version of TapTalkTest that will be used in Aim 2 and Aim 3. The choice of which tasks will comprise the shortened protocol will be based on the findings from Aim 1. We have purposefully kept a wide array of tests in Study 1 to clarify which tests combine best together, and with the DDK data, to best discriminate biomarker-defined AD pathology.
Speech-like data collection
It is unclear which tests are most sensitive in preclinical AD. We will replicate the clinical DDK (23, 27) test but under both comfortable and maximal speed conditions. A participant will be instructed to say “pa, pa, pa…” repetitively for 10 seconds at a comfortable pace, then the participant will be instructed similarly for “ta, ta, ta…”, “ka, ka, ka…” and “pa-ta-ka, pa-ta-ka…” all at a comfortable pace. After this, they will be instructed, to repeat each test in turn but this time at maximal pace. These deceptively simple tests assess the motor function of the lips (pa), tongue (ta) and palate (ka) respectively, allowing us to calculate the speed, rhythm, accuracy, and motor reserve (‘maximal pace’ measures divided by ‘comfortable pace’ measures) for each syllable.
Similar to the format of the hand movement tests, there will be an ‘instruction’ screen and a ‘recording’ screen followed by a “well done” screen. The difference compared to the hand movement tests, in terms of the software, is that the recording screen does not show a video image of the participant as they perform the task; see Fig. 3. At the end of the protocol, the participant will be asked to complete a questionnaire on their experience of the tests and suggestions for improvement.
Extracting hand-speech movement data
We will use deep neural networks and other machine learning methods to develop and train two computer programs that will automatically detect, respectively, hand key points (e.g. finger and thumb tips) in the video data, (43) and syllable features in audio recording (Fig. 1). A series of movement features will be extracted in a fixed window period, by 1) experts in hand movement (JA, KL, RSG) and speech-like movements (LG), and 2) deep-learning approach applied directly on processed video data, to analyse their associations with biomarker-defined measures of AD pathology. The processed video data can be the hand-only cropped video or displacement (between fingertip and thumb-tip) over time data extracted from the raw video. Thus, two approaches, expert guided and AI-based ‘deep-learning’, will extract discriminative hand-speech movement features as input data for the multivariable model. The main outcome will be a validated personal computer (laptop and desktop) version of the TapTalkTest that discriminates preclinical AD from normal ageing in cognitively healthy adults ≥ 50 years old with defined sensitivity and specificity.
Statistical analysis
We will use data from 350 of a sample of 500 participants to develop multivariable models that classify data according to risk of preclinical AD pathology. Data from the remaining 150 will determine the externally validated sensitivity/specificity of candidate models in detecting preclinical AD. Hand-speech movement data will be normalised. Participants with blood biomarker levels in the preclinical AD range (e.g., p-tau 181, or a combination of blood biomarkers) will be classified as ‘AD positive’. We will use penalised logistic regression to measure the associations between movement patterns and AD positivity. Covariates may include age, gender, apolipoprotein E ([APOE] a risk gene for AD analysed through blood sampling as part of the ISLAND Project), years of education, and handedness. As a secondary, more agnostic approach, we will use deep learning methods to discover features in movement and speech-like data that map to AD positivity.
We will use cross-validation to avoid overfitting by selecting penalty terms (type and amount of shrinkage applied to coefficients) which optimise the bias-variance trade-off. Bootstrap procedures will be used to estimate model uncertainty. We will externally validate models using data from the remaining 150 participants. Receiver Operating Characteristic (ROC) curves will be plotted against the positive AD cut-off to assess the sensitivity/specificity of movement and speech-like models to discriminate the preclinical AD stage from normal ageing. The model with the highest area under the ROC curve (AUC) will be chosen as the TapTalkTest algorithm.
Sample size justification
The precise combination of blood-based biomarkers used to determine ‘positive’ preclinical AD pathology is yet to be determined as the scientific literature is developing quickly and new biomarkers are being developed regularly. We will review the literature before deciding on the best biomarker test, or combination of tests, that should be used to define AD pathology risk most accurately.
Currently p-tau 181 is considered one of the most accurate blood-based biomarkers for preclinical AD risk and we have therefore based our sample size calculation on this biomarker. As new tests are developed and are likely to be even more sensitive/specific than p-tau 181, the current calculation is a conservative estimate. We estimate that 17% of cognitively normal adults within our cohort have p-tau 181 in the preclinical AD range (55). There are no tests of DDK compared to AD biomarkers, so we have based all calculations on finger tapping data.
We have performed ROC curve analysis using open data from Mollica et al. to calculate sensitivity and specificity of finger tapping speed and variability to predict CSF p-tau 181 (20). This showed area under ROC curve (AUC) of 0.75 for a null linear model including age, gender and years of education, but not finger tapping. We calculated that the sample size to compare a screening test with an AUC > 0.90 against the null model requires 60 preclinical AD cases (positive p-tau181) and 290 healthy ageing controls (negative p-tau181). Thus, we will use 350 in the development dataset (expected 60 [17%] preclinical AD cases) and 150 in the validation dataset (expected 25 [17%] preclinical AD cases) to test cut-offs from the development model. The PROBAST tool (56) for assessment of prediction and diagnostic method studies confirms these development and validation strategies have a low risk of bias.
AIM 2
Design
There will be three sub-studies to address Aim 2 as follows: Study 2.1, a cross-sectional study of 30–40 adults to determine validity of data collection from different types of smartphones; Study 2.2, a prospective cohort study of 50–100 adults ≥ 50 years old to assess usability and reliability, and Study 2.3, a prospective cohort study of ~ 1,000 adults ≥ 50 years old to assess validity against cognitive measures. The methods to addresses Aim 2 are summarised in Fig. 4. The TapTalkTest test protocol on a smartphone will be based upon the findings in Study 1 and comprise around three to five of the most discriminating hand and speech-like movements, aiming for test duration of approximately 2–3 minutes. The TapTalkTest protocol may be further refined after studies 2.1 and 2.2, depending on user feedback and results.
Participants and settings
Study 2.1
We will undertake a cross-sectional validation study of 30–40 participants who will be recruited from staff and students at the University of Tasmania. They will be invited to attend the research centre and use their own mobile phone to video record a member of the research team (who acts as a ‘dummy’ participant) as they perform a series of finger tapping tests whilst wearing Polhemus movement sensors (as the ‘gold standard’ motor data benchmark). The Polhemus electromagnetic tracking system delivers six degrees of freedom including the position (i.e., X, Y and Z coordinates of a space) and orientation (i.e., yaw, pitch and roll), of each object with a 60 Hz sample frequency. Compared to commonly used camera-based tracking systems, the Polhemus technology allow us to collect continuous and non-interrupted data without the issue of light-of-sight occlusions. This technology has been increasingly used in research on quantification of wrist and hand movements (57, 58). Similarly, the participants will be asked to use their own smartphone microphone to record the speech-like sounds when the ‘dummy’ participant performs the DDK tasks. The audio benchmark for this will be data collected from a high-quality microphone (such as the Blue Yeti USB microphone). For each recording, the distance from the recording device (smartphone and microphone) and the dummy participant will be constrained to about 50 cm to allow for fair comparisons between the devices.
Statistical Analysis
We will assess the accuracy of the video and audio data collected from each type of smartphone in comparison to the data collected by the wearable sensor measurements and high-quality microphone respectively, using correlation coefficients and mixed effects regression.
The results from this ‘smartphone validation’ study will: (i) allow a correction factor to be introduced, if necessary, into the data analysis later on for a wide range of smartphones and/or, (ii) provide assurance that the data collected for TapTalkTest on smartphones have little variation across a range of smartphones and in comparison to the movement and audio data collected by the gold -standard wearable sensors and microphone respectively.
Study 2.2
We will undertake a prospective study of 50–100 (randomly selected) ISLAND Project participants who participated in Study 1, own a smartphone, and live within 30km of the University of Tasmania research centre. They will be invited to attend the University of Tasmania Clinical Research Facility in Hobart for approximately 15 minutes and asked to bring their own smartphone. Participants will be asked to install the TapTalkTest app on their own smartphone and complete a consent form. Each participant will then complete the test protocol of hand and speech-like movements in front of the researcher. This will allow the researcher to observe if there are any issues around the usability of installing and operating the TapTalkTest app. Each participant will be asked to complete a User Experience Questionnaire immediately after completing both tests, to further evaluate the usability of each form of the test (see supplementary files). We will invite participants to complete TapTalkTest on a smartphone at least three times at home over the next 7-day period, using the same device each time. They will be sent a reminder email every day over the 7-day period and asked to complete the TapTalkTest once each day if possible, or at least a minimum of 3 times in total. Their performance on each test will be automatically uploaded to a central secure TapTalkTest database.
Statistical Analysis
The responses to the User Experience Questionnaire, assessing usability, will be analysed with summary statistics. We will examine any practice effects (temporal reliability) by checking for increasing correlation between the first test (in the research centre) and repeated tests (at home) using Kendall rank correlation coefficients. A Kendall rank correlation coefficient of ≥ 0.70 is considered strong; 0.50 to 0.69 as good; 0.30 to 0.49 as moderate and < 0.30 as poor. This will be tested separately for each participant’s device.
The outcome of study 2.2 will be a measure of the usability and reliability of TapTalkTest. Based on the results from this pilot study, we will make refinements to the software and/or instructions and then invite all ISLAND participants to complete the TapTalkTest at home as part of study 2.3.
Study 2.3
We will undertake an 18-month prospective study of approximately 1,000 ISLAND Project participants who have completed online cognitive CANTAB tests in 2021. Participants in the ISLAND Project (45, 46) are invited to complete online CANTAB cognitive tests every 2 years. The first testing period was mid-2021 and ~ 3,000 participants completed the tests; these participants will be invited to take part in the TapTalkTest prospective validity study. Every 6 months over an 18-month period (four data collection points in total; T0, T + 6, T + 12, T + 18), eligible participants will be invited via the ISLAND Project website (‘portal’), to complete TapTalkTest at any time over the next one-month period. Participants will be asked to complete TapTalkTest on their own smartphone at home or in a setting of their choice. Consent will be taken at each data collection point. When participants complete the TapTalkTest app they will be asked to complete a questionnaire about user experience and symptoms relating to hand and speech function (see supplementary files). The whole procedure, including the questions and tests, is expected to take about 5 minutes. A reminder email will be sent 1, 2 and 3 weeks after the initial invitation to people who have not yet completed the test.
Statistical Analysis
The data from participants who complete TapTalkTest on two or more occasions over an 18-month period, and CANTAB on two occasions (2021 and 2023) will be included in the analysis. De-identified ISLAND Project participants will be classified as cognitively ‘stable’ or ‘declining’ using longitudinal CANTAB cognitive scores. TapTalkTest scores will be calculated using algorithms developed in Study 1. Logistic regression will be used to estimate the odds of a participant being confirmed as ‘declining’ at time T2 (24 months) conditioned on TapTalkTest score at time T1 (12 months). The overall outcome of Study 2 will be a smartphone app, TapTalkTest, that is usable, reliable and validated against established online CANTAB cognitive tests.
Sample size justification
Using CANTAB cognitive test responses already collected in the ISLAND Project (n = 3,000), we estimate that we will collect > 1,000 complete cases (TapTalkTest app completion twice in 18 months and CANTAB at baseline and 24 months), which is ample for logistic regression with a single (fixed-effect) predictor variable.
AIM 3
Design
We will undertake a prospective study of 200 adults who have cognitive symptoms to validate TapTalkTest against gold-standard clinical consensus diagnosis of MCI and AD dementia. Secondary aims include determining the usability of TapTalkTest in a cohort with cognitive symptoms and comparing the usability and validity of TapTalkTest with commonly used brief screening tools. The study will be undertaken at two sites – the Royal Hobart Hospital (RHH) and the ISLAND Clinic with 100 people recruited from each site; see Fig. 5 for an overview of the study design.
The inclusion criteria are adults aged ≥ 50 years old with at least 3 months of persistent cognitive symptoms (patient- or family-reported).
The exclusion criteria are any adults who are < 50 years old, acutely unwell, have significant impairment of hand function or speech function, lack capacity to consent to the research study, or already have a known diagnosis of MCI or dementia.
Recruitment from the hospital
There are several steps to recruiting eligible participants to the study and these are necessary as we will recruit from a pool of patients who are moving through their medical care at the RHH. Clinicians working at RHH (JA, NF, AB) will oversee the identification of 100 patients from the acute medical/subacute sites who may be eligible for the study. We will assess capacity to consent in line with standards of Good Clinic Practice research governance. Specifically, we will check that the participant has understood the information provided on the study, that they can retain it and can recount the key parts back to the researcher, that they can weigh up the information and that they can freely decide to participate.
After obtaining written consent, a research assistant (RA) will collect demographic and clinical details on the Data Collection Form (see supplementary files). The RA will ask the participant to complete the TapTalkTest assessment using the study smartphone; participants will be invited to hold the phone themselves, or to prop it up on a table, and to complete the TapTalkTest assessments without the RA prompting them (as it is designed as a ‘self-test’). The duration of testing is expected to take about 2–3 minutes. If the participant prefers the RA to help them, the RA will hold the smartphone and/or help guide participants through the tests as requested. They will make a note in the Data Collection Form (see supplementary files) how much help was administered and the indications for this, as this is valuable information for the research team to evaluate when considering usability of TapTalkTest in clinical cohorts.
The RA will then administer the MoCA, a standardised brief cognitive screening test that is commonly used in clinical practice and typically takes about 10–15 minutes to complete (59, 60). The addition of this established cognitive test will allow for validation of the TapTalkTest scores, as well as comparison of the two tests (established cognitive test vs TapTalkTest) compared to the clinical diagnosis (see later, ISLAND Clinic consensus diagnosis section). The MoCA outperforms the long-established MMSE in screening for cognitive impairment (60). However, recent data suggest cultural differences may affect performance. Should the MoCA score suggest cognitive impairment for participants from culturally and linguistically diverse backgrounds or First Nations people, additional screening will be completed using the Rowland Universal Dementia Assessment Scale (RUDAS) (61).
After the participant has completed TapTalkTest and the cognitive test(s), the RA will ask the participant to complete a self-report ‘User Experience Questionnaire’ (see supplementary files). If the participant prefers the RA to read the questions out to them, the RA will do so and make a note of this in the Data Collection Form. This will provide valuable information on usability of TapTalkTest in a clinical cohort in comparison to a standard cognitive test.
After completing the test protocol, the RA will give the participant an information leaflet on the ISLAND Clinic (https://islandclinic.utas.edu.au/) that they can pass on to their GP if they would like to have further assessments. This GP letter will include a numerical research code used during the TapTalkTest assessments so that, if the participant attends the clinic, their data can easily be linked. The ISLAND Clinic protocol has been described previously in Alty et al (62); in brief it is a state-wide cognitive clinic that provides facilities and expertise for clinical diagnosis in a ‘one-stop’ interdisciplinary model where participants will have cognitive, medical and movement assessments performed on one day. They will also have an MRI brain scan, blood tests and an array of other tests. The interdisciplinary team formulates a consensus diagnosis for each participant attending the clinic. One of the clinicians on the team delivers the diagnosis to the participant on the same day and provides a detailed management plan for the participant and GP. The Clinic is supported by research funding and is Medicare bulk-billed for participants with no out-of-pocket costs.
This means that through the standard ISLAND Clinic processes, all participants recruited through the RHH will have the opportunity to have thorough medical and cognitive assessments, including neuropsychological tests, an interdisciplinary consensus diagnosis and management plan for their cognitive symptoms (62). The established diagnosis of ‘cognitively unimpaired’ (objectively ‘normal’ on ISLAND Clinic tests), ‘MCI’ and ‘AD dementia’ (AD and mixed AD/vascular) also allow us to validate TapTalkTest scores against the gold standard clinical diagnosis.
Recruitment from GP referrals via the ISLAND Cognitive Clinic
All patients aged ≥ 50 years who are referred to the ISLAND Clinic by their GP with at least 3 months of cognitive symptoms, and no established diagnosis of MCI or dementia, will be invited to take part in the TapTalkTest study. When participants attend the ISLAND Clinic, they are routinely assessed for their capacity to consent to research as part of ethics approved ISLAND Clinic procedures (62). A researcher will administer the TapTalkTest and MoCA (or RUDAS) in the same way as described for participants recruited via the RHH. The only difference is that the clinical and demographic details described on the Data Collection Form are collected routinely within the ISLAND Clinic so will be cross populated from other assessments. For all participants seen in the ISLAND Clinic who were originally recruited through the hospital, the TapTalkTest, MoCA (or RUDAS), demographic and clinical data will have already been collected. The main purpose of the Clinic for these participants is to provide consent to access data collected during the Clinic, clarity on the diagnosis and a management plan for their cognitive symptoms. From the aspects of the TapTalkTest study, this diagnosis also allows us to validate the smartphone tests against clinical diagnosis.
Statistical Analysis
We will calculate AUC for TapTalkTest and MoCA (or RUDAS) to predict clinical diagnosis of MCI and AD dementia (defined as AD or mixed AD/vascular); 95% confidence intervals will be obtained using bootstrapping. Covariates may include age, gender, APOE4, years of education, and handedness. We will estimate cut-off scores for TapTalkTest and MoCA (or RUDAS) to differentiate between cognitively unimpaired vs MCI, and between cognitively unimpaired vs AD using the Youden index to optimise the trade-off between sensitivity and specificity. Classification accuracy (sensitivity and specificity) using these cut-offs will be compared using McNemar’s test. Participants diagnosed with other types of dementia (such as Dementia with Lewy Bodies or Frontotemporal Dementia) will not be included in this analysis.
Sample size justification
As the conditional dependence between TapTalkTest and MoCA (or RUDAS) is unknown, the nominal power of the study cannot be calculated. However, Roalf et al. compared the diagnostic accuracy of MoCA with MMSE using a comparable sample size (n = 126 MCI and n = 140 healthy controls) and found a significant difference (in favour of MoCA) between the two diagnostic tests (63).
The main outcome of Study 3 will be an externally validated TapTalkTest smartphone test with determined precision for prospectively predicting risk of MCI and AD in adults ≥ 50 years old with cognitive symptoms. Secondly, we will have a measure of the accuracy and usability of the TapTalkTest smartphone test compared to MoCA in adults ≥ 50 years old with cognitive symptoms.
Data Management
All data obtained will be managed via a secure database which is hosted on the University of Tasmania virtual server managed by central Information Technology staff and backed up daily. Server access is restricted to authorised administrators using Secure Shell and Public Key Infrastructure certificates. Direct access to the databases is limited to system administrators and overseen by designated custodians of the data and will enable access to data in a de-identified fashion to research personnel. Data will be maintained in secure University of Tasmania databases for at least 10 years, and/or until 5 years after the final publication relating to this data. Consent will be sought for this long-term storage as well as linkage to extension projects. Specifically, data linkages with the ISLAND Project (45), TAS Test Project (38), and the ISLAND Clinic (62), are intended. Access will be requested through the University of Tasmania and the principal investigators of the studies. The participants in the ISLAND Project, TAS Test Project and the ISLAND Clinic all sign a consent form stating that they agree to their data being used for other unspecified research.
We will request that consent is provided to enable sharing of non-identifiable data with research collaborators external to the University of Tasmania. An open database of video recordings of hands/forearms, and audio recordings of DDK tasks will be developed as a resource for other researchers internal and external to University of Tasmania. Extended consent for future research participation will also be included in the consenting information. This is due to the potential opportunities for long-term follow-up in a specific cohort of participants with high risk of AD pathology. The study sponsor organisation is the University of Tasmania, Hobart, Australia, 7001. The study management group comprising clinicians, neuroscientists, computer scientists, and a statistician, will meet every 2 months to monitor and discuss the progress of the study, and to address any issues that may arise. Protocol deviations will be reported to the Human Research Ethics Committee in line with local recommendations.
If any unexpected adverse events (an unforeseen harmful, unpleasant or undesirable response, reaction or outcome experienced by a research participant or researcher) or serious adverse effects occur, these will be documented, reported in line with Good Clinical Practice, and appropriate referrals made for care. A participant may be discontinued from the study at any time if the participant or the research team feels that it is not in the participant’s best interest to continue. Possible reasons for discontinuation include participant withdrawal of consent, lost to follow-up, or new co-morbidity/diagnosis that would meet the exclusion criteria (e.g., loss of hand function). If the participant has taken part in completing TapTalkTest, and then changes their mind after data collection, the research team will do their best to withdraw all data from the study but if their de-identified data has already been included in the group analysis. it will not be possible to withdraw this data. Reasonable attempts will be made by the research team to provide a reason for participant withdrawals in data collection.