Trial design
The Promotoer trial is designed as a randomized, controlled, assessors blinded, single-centre, superiority trial with 2 parallel groups with 1:1 allocation ratio. The two intervention groups are the BCI-assisted MI training (experimental group, BCI-MI) and the MI training not supported by BCI (control group, Control-MI). Both interventions will comprise a total of 12 sessions with a week delivery of 3 times/week, 45 min duration. Training will be completed in 4–6 weeks. The study interventions are conceived as add-on regimen to standard rehabilitation care (details are specified in the study protocol (10)).
The participants recruitment, intervention delivery and data collection will take place at Fondazione Santa Lucia, IRCCS, Rome Italy.
Randomization
Participants are assigned to either the experimental group (BCI-MI) or the control group (Control-MI) in a 1:1 ratio through a randomized allocation process. This randomization process has been stratified by side of the stroke lesion (left or right hemisphere) and the baseline score of UE-FMA/60, with 60 as the maximum score (< 19, severe; from 20 to 47, moderate) (13).
To carry out the randomization, a list has been generated using permuted blocks of varying sizes through the Ralloc procedure of the Statistical software STATA (https://www.stata.com/). The randomization list was centrally generated by the trial statistician at Istituto Superiore di Sanità (ISS, Rome Italy) and then securely managed. To maintain the integrity of the randomization process, numbered, opaque, sealed envelopes were prepared in order to replicate the randomization list for each specific stratum. These envelopes were transferred to the clinician responsible for participant randomization at FSL Hospital.
When a patient becomes eligible for randomization, the clinician in charge submits a randomization form containing the relevant stratification data and date of the request. Then, the clinician responsible for participant randomization at FSL Hospital assigns to the patient the envelope with the number corresponding to the progressive order of randomization of the patient in his/her stratum.
Sample size
The results of the pilot study (8) demonstrated that the BCI-MI group had significantly higher effectiveness in terms of FMA scores compared to the control group (mean ± SD: 44 ± 34.7 vs. 19.8 ± 19.8). Based on these findings, the sample size calculation was conducted using a one-sided alpha level of 5% for the test of superiority, aiming for a statistical power of 80%, and utilizing a t-test for independent groups. The calculation revealed that each group should consist of 18 participants. To account for potential loss to follow-up, a dropout rate of 25% was considered, leading to the determination that a total of 48 patients would be required for the study. The sample size calculation was conducted using the power twomeans module in the statistical software Stata/MP 17.0.
Framework
The primary objective of the trial is to evaluate the superiority of BCI-MI training (BCI-MI) against a non-BCI assisted MI training (Control-MI) in improving hand motor function outcomes in sub-acute stroke patients admitted to the hospital for their standard rehabilitation care. Thus, the research hypothesis is settled in a superiority framework.
Statistical interim analysis and stopping guidance
An internal pilot study (IPS) for sample-size recalculation was planned to be carried out on data on primary outcome in first 20 participants to check congruity of nuisance parameters used in sample size calculation.
Specifically, estimates of standard deviations of effectiveness of FMA score at T1 by intervention group will be obtained from masked IPS data and used in sample size procedure specified in Sample size section by taking unchanged other assumptions and data. Only upwards adjustments of the initially planned sample size might be allowed, according to restricted IPS (14). R code used for masked calculation of standard deviations is reported in Appendix 1.
No interim efficacy analysis is planned. Pockock method for control of family-wise type I might be implemented by using the software RPACK (Confirmatory Adaptive Clinical Trial Design and Analysis, R package) if an interim analysis is required due to changes in the recruitment conditions.
Timing of final analysis
The final analyses will be performed after completion of the last visit of the last enrolled patient. Time schedule of trial assessments is reported in Table 1. Data for analyses will be extracted from the electronic Case Report Form (eCRF) used for data entry and monitoring during the clinical trial.
Table 1
Time schedule of assessments of baseline and outcomes data
|
Data Collection Instrument
|
Enrollment
|
Randomization
|
Evaluation
T0
|
Training session
|
Evaluation
|
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
8
|
9
|
10
|
11
|
12
|
T1
|
T2
|
T3
|
T4
|
|
Demographical and clinical data
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Token test
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
FMA/60
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Eligibility criteria
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Informed consent
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Randomization
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Edinmburgh handedness inventory
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
EEG
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
●
|
●
|
●
|
|
MEP
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Neuropsychological assessments
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
MRI
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
●
|
|
FMA/66
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
●
|
●
|
●
|
|
NIHSS
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
●
|
●
|
●
|
|
ARAT
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
●
|
●
|
●
|
|
MAS
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
●
|
●
|
●
|
|
MMT
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
●
|
●
|
●
|
|
NRS pain
|
|
|
●
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
●
|
●
|
●
|
|
Training session, run
|
|
|
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
|
|
|
|
|
QCM
|
|
|
|
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
|
|
|
|
|
VAS Mood
|
|
|
|
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
|
|
|
|
|
NASA TLX
|
|
|
|
|
●
|
|
|
|
|
|
|
|
|
|
●
|
|
|
|
|
|
VAS Satisfaction StatisfactionStatSatisfaction
|
|
|
|
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
|
|
|
|
|
QUEST
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
|
|
|
|
|
SUS
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
|
|
|
|
|
Status
|
|
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
●
|
|
Rehabilitation monitoring
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
●
|
●
|
●
|
●
|
| FMA = Fugl-Meyer Assessment |
| EEG = Electroencephalography |
| MEP = Motor Evoked Potentials |
| MRI = Magnetic Resonance Imaging |
| NIHSS = National Institute of Health Stroke Scale |
| ARAT = Action Research Arm Test |
| MAS = Modified Ashworth Scale |
| MMT = Manual Muscle Test |
| NRS = Numeric Rating Scale |
| QCM = Questionnaire on Current Motivation |
| VAS Mood = Visual Analogue Scale of Mood |
| NASA TLX = NASA Task Load Index |
| VAS Satisfaction = Visual Analogue Scale of Satisfaction |
| QUEST = Quebec User Evaluation of Satisfaction with assistive Technology 2.0 |
| SUS = System Usability Score |
| T0 = evaluation at randomization, before intervention |
| T1 = at end of intervention training (within 48h) |
| T2 = at 1 month from end of training |
| T3 = at 3 months from end of training |
| T4 = at 6 months from end of training |
Blinding will be removed in two steps: analysis will be performed upon completion of the study and freezing of the database, using group assignments as group A and group B. When all analyses have been performed and final report drafted, BCI-MI and Control-MI will be un-blinded.
Statistical principles
Confidence intervals and P values
The primary hypothesis testing will be performed at a one-sided alpha of 0.05. The significance level used for the primary outcome is consistent with that used in the sample size calculation. Hypothesis testing on any secondary outcomes will be performed at two-sided alpha of 0.05. Treatment effects will be reported through mean differences for continuous variables and odds ratio for binary variables, with two-sided 95% confidence intervals.
Adjustment for multiplicity will be considered in post-hoc comparisons of secondary analyses, specifically in comparisons among groups in each evaluation time (T1, T2, T3 and T4). Bonferroni’s correction criteria will be applied, and exact p-values will be reported.
Adherence and protocol deviations
Adherence to study interventions (BCI-MI or Control-MI) is fixed as a minimum of 9 training sessions (out of 12) delivered within 6 weeks. Moreover, protocol deviations include loss at follow-up (withdrawal/drop-out/death for any cause) and/or missed assessment for primary outcome in T1. Number of patients with adherence to study intervention and number of patients showing deviations from planned study intervention will be reported for each intervention group. Deviations will be described for each patient by intervention group.
Major protocol deviations are missing adherence to study interventions.
Analysis population
The as randomized (AR) population include all randomized patients, regardless of subsequent withdrawal from training sessions or deviation from study protocol.
The Intention-To-Treat (ITT) population is defined as all randomized participants who will perform (minimum) 1 training session (both for BCI-MI training and Control-MI training intervention groups).
The Per-Protocol (PP) population includes those randomized participants without major protocol deviations, namely patients with adherence to study intervention.
The user experience (UE) population include all patients randomized assessed for user experience.
Trial population
Screening and eligibility
Screening for eligibility and enrolment in the study are detailed in the study protocol (10). Briefly, eligible candidates are patients of any gender, aged 18 to 80 years, with a first ever unilateral stroke, confirmed by Magnetic Resonance Imaging (MRI), occurred from 1 to 6 months since admission to FSL site for rehabilitation care. The eligible patients must show hemiplegia/hemiparesis of the upper extremity, and a UE-FMA/60 score (out of a maximum 60 score) lower or equal to 47 [severe participants, with scores from 0 to 19, and moderate participants, with scores from 20 to 47, will be included]. The main exclusion criteria include dementia, or severe neglect, or severe aphasia, severe spasticity (Modified Ashworth Scale > 4 at shoulder/elbow/wrist), a UE-FMA/60 score greater than > 47, a Token Test (TT) score lower or equal to 29 score, and concomitant neurological disorders. All eligibility criteria must be met by the time of the screening visit.
Recruitment and Withdrawal/Follow-up
Patients who meet the eligibility criteria and have provided informed consent will be randomized and undergo the planned evaluations at T0 (see Table 1). Any participant that enrolled in the study can withdraw from the study at any time, for any reason, without prejudice or consequence. The status of participants will be monitored during each scheduled training session and evaluation visit. In cases where a patient misses a planned evaluation visit, a phone call will be made to check on their status.
A CONSORT flow diagram specifically designed for the Promotoer study (Fig. 1) will be used to summarize number of patients randomized, receiving their allocated intervention, withdrawing/loss to follow, and included in analysis populations at evaluation visits T1 and T4.
The timing of withdrawals and loss to follow-up, whenever possible, will be presented as time-to-event data and analysed using Kaplan-Meier curves, stratified by intervention group. To assess any potential differences between the two curves, the Log-rank test will be employed. Furthermore, Reasons of loss of participants during the study will be described by intervention group.
The data collected from participants who have withdrawn from the study will be included in the analysis sets, unless they explicitly withdraw their consent for the data collected up to that point to be used.
Baseline patient characteristics
According to time schedule reported in Table 1, patients will be described - separately for the two randomized groups - with respect to:
-
demographical data (sex, age at enrolment),
-
clinical data and functional assessments collected at the screening visit for eligibility (type and side of lesion, TT score, FMA/60 upper limb section score)
-
functional assessments completed at evaluation time T0 after the signature of the informed consent (MEP, TMT, TAP, Edinburgh handedness inventory score, UE-FMA 66, NIHSS, ARAT, MMT for shoulder/elbow/wrist, NRS for pain for affected upper limb, and the MAS for Spasticity score for shoulder/elbow/wrist).
All continuous variables will be summarized by descriptive statistics of n, mean, standard deviation, minimum, maximum, median, and interquartile range. Shapiro Wilk’s test will be used to assess normal distribution of variables. Categorical data will be summarized by frequencies and percentages. Test of statistical significance will not be undertaken for baseline characteristics; rather the clinical importance of any imbalance will be noted.
Table of baseline characteristics will be obtained for the ITT population (see section 4.3. Analysis populations). Furthermore, tables of baseline characteristics will be produced also for the AR population, and for the PP population. These tables will be useful to assess the success of randomization by examining the balance of prognostic factors among the different intervention groups, and to verify whether any attrition during the study has introduced selection bias or disrupted the balance that was initially achieved through randomization.
Analysis
Outcomes definition
Primary outcome is the “effectiveness” of the UE-FMA score at T1. The UE-FMA is widely recommended for the evaluation of sensorimotor impairments of upper extremity (motor and sensory part) in stroke rehabilitation research (9, 15, 16). It ranges from 0 to 66 (best score).The effectiveness” of UE-FMA score is defined as the proportion of potential improvement that could be achieved after the intervention and is calculated as score at T1 minus score at T0, divided by the maximum score (66) minus score at T0, multiplied by 100 (as specified in (9)) [Formula: 100* (UE-FMA at T1 - UE-FMA at T0)/(66- UE-FMA at T0)]. Thus, if a patient achieves the highest possible score after the intervention, the effectiveness was 100%. This approach allows to normalize the data, by accounting for baseline data. The secondary outcomes are presented in Table 2. Electroencephalogram (EEG) and MRI data will be considered in a separate document.
Table 2
|
Clinical and functional outcomes
|
|
Outcome
|
Description
|
|
Effectiveness of Fugl-Meyer Assessment of upper extremity (UE-FMA) score at evaluation times T2, T3, and T4
|
Analogously to primary outcome, effectiveness (9,28) of UE-FMA score is the proportion of potential improvement that could be achieved after the intervention in the UE-FMA score:
\(100\times \frac{UE/FMA at {T}_{i}-UE/FMA at {T}_{0}}{66-UE/FMA at {T}_{0}}\)
Where\(i=2, 3, 4.\)
|
|
Minimal Clinically Important Difference (MCID) at evaluation times T1, T2, T3, and T4
|
MCID is the improvement of at least 7 points in UE-FMA, with respect to baseline (29).
|
|
MAS for Spasticity score for shoulder/elbow/wrist stroke at evaluation times T1, T2, T3 and T4
|
MAS score is used to assess muscle tone and spasticity in individuals with neurological conditions, such as cerebral palsy or stroke (30). It evaluates resistance to passive movement in specific muscle groups. It ranges from 0 to 4.
|
|
Numeric Rating Scale (NRS) for pain at evaluation times T1, T2, T3 and T4
|
NRS score is commonly used to assess the intensity of pain experienced by a person. It is a self-report score and ranges from 0 ("no pain") to 10 ("the worst pain imaginable").
|
|
Effectiveness of Action Research Arm Test (ARAT) score for upper limb function at evaluation times T1, T2, T3 and T4
|
ARAT score (31) is used for assessing upper limb function. It ranges from 0 to 57. The effectiveness (28) of ARAT score is the proportion of potential improvement that could be achieved after the intervention:
\(100\times \frac{ARAT at {T}_{i}-ARAT at {T}_{0}}{57-ARAT at {T}_{0}}\)
Where\(i=1, 2, 3, 4.\)
|
|
Effectiveness of the National Institute of Health Stroke Scale (NIHSS) score at evaluation times T1, T2, T3 and T4
|
NIHSS (32) is used to evaluate the severity of a stroke and to measure the level of neurological impairment in stroke patients. It ranges from 0 (no neurological deficit) to 42 (severe neurological deficit). Effectiveness (28) of NIHSS score is the proportion of potential improvement in NIHSS that could be achieved after the intervention:
\(100\times \frac{NIHSS at {T}_{i}-NIHSS at {T}_{0}}{42-NIHSS at {T}_{0}}\)
Where\(i=1, 2, 3, 4.\)
|
|
Effectiveness of Manual Muscle Test (MMT) score for shoulder/elbow/wrist (flexor/extensor muscles), at evaluation times T1, T2, T3 and T4
|
MMT (33) is used to evaluate muscle function and monitor changes in muscle strength over time. The maximum possible score for shoulder/elbow/wrist (flexor/extensor muscles) is 25.
Effectiveness (28) of MMT score is the proportion of potential improvement that could be achieved respect to the baseline:
\(100\times \frac{MMT at {T}_{i}-MMT at {T}_{0}}{25-MMT at {T}_{0}}\)
Where\(i=1, 2, 3, 4.\)
|
|
User experience outcomes
|
|
Questionnaire on Current Motivation (QCM, 4 items) score, at training sessions 2 to 12
|
The QCM (34) score is used to evaluate motivation and adherence to technology.
|
|
Workload NASA-TLX Total score at training sessions 2 to 12
|
The Workload NASA-TLX Total score (35) is used to evaluate mental workload-fatigue during training with technology.
|
|
System Usability Score (SUS) at training session 12
|
The SUS (36) is used to evaluate the usability of technology.
|
|
Quebec User Evaluation of Satisfaction with assistive Technology 2.0 (QUEST 2.0), at training session 12
|
The QUEST 2.0 score (37) is used to evaluate satisfaction with technology.
|
|
Visual analogue scale for Mood (VAS Mood) score at training sessions 1 to 12
|
VAS for Mood is a self-assessment tool used to measure a person's emotional state or mood.
|
|
Visual analogue scale for satisfaction (VAS Satisfaction) score at training sessions 1 to 12
|
VAS for satisfaction is a self-assessment tool used to measure an individual's level of satisfaction with a particular experience, product, service, or intervention.
|
Analysis methods for primary and secondary outcomes
The primary analysis will be ITT population on the primary outcome - effectiveness of UE-FMA at T1. In the secondary analysis, the analysis of the primary outcome will also be carried out in the PP population. The secondary outcomes will be compared between groups both in ITT and PP population.
For all continuous variables (effectiveness of UE-FMA score, of ARAT score, of NIHSS score, of MMT score, NRS, QCM, Workload NASA-TLX, SUS, QUEST, VAS Mood, VAS Satisfaction) descriptive statistics (mean and standard deviation, median and interquartile range) will be reported by intervention group, for each evaluation time/training session. t-test for independent groups will be used for comparisons of BCI-MI group vs Control-MI group in each evaluation time/training session. The Shapiro Wilk’s test will be used to assess normal distribution and Mann-Whitney test will be used for comparison if normality assumption is violated.
Moreover, to define whether and to what extent the efficacy of the BCI intervention on effectiveness of UE-FMA score can be sustained long-term after the end of intervention, the analysis of variance (ANOVA) for repeated measures with group as between subject factor and time as within subject factor will be applied to effectiveness of UE-FMA scores assessed at T1, T2, T3, and T4.
All categorical variables (MCID) will be summarized (by number and frequency) in each intervention group and compared between groups by Fisher’s test, in each evaluation time (T2, T3, T4).
To identify clinical and/or neurophysiological determinants of MCID (Table 2) in participant response to an intervention at T1, a forward-stepwise binary logistic regression will be performed including intervention group, demographical, clinical, and neurophysiological parameters. Odds ratios and 95% confidence intervals will be calculated. To reduce the dimensional space of available variables, variable selection algorithms (e.g., Least Absolute Shrinkage and Selection Operator – LASSO regression) will be used to select the most informative features to be included in the predictive model. A probability score will be defined to assess likelihood of good recovery based on obtained determinants of response. Moreover, a further logistic regression analysis will be performed by including only the BCI-MI group. Same strategies of analyses will be used for evaluation of MCID at later evaluation times (T2, T3, T4).
Statistical analysis of MAS score will be detailed for shoulder, elbow, wrist. Descriptive statistics on frequency distributions will be reported for each time point (T1, T2, T3 and T4) and compared between intervention groups by chi-square test. Moreover, the change in the score from T0 to T1 and the change along the follow up (e.g., T2 vs T0; T3 vs T0, and T3 vs T0) will be computed and defined as improved/stable/deteriorated. Such changes will be compared by intervention groups using a chi-square test or Fisher’s exact test.
Scores obtained in user experience assessments for QCM (separately for Mastery confidence, Incompetence/Fear-to Fall, Challenge and Interest), Workload NASA-TLX, SUS, QUEST, VAS Mood, and VAS Satisfaction, will be described by experimental group and comparisons will be carried by using the methodologies adopted for other continuous variables. Analyses will be carried out by training sessions.
Subgroup analyses
To investigate possible effect of number/amount of training sessions received, comparisons among sub-groups of BCI-MI participants defined according to number of completed training sessions (< 9 vs > = 9), will be carried out on primary and secondary outcomes by using the same statistical methodologies adopted in primary and secondary analyses.
Moreover, to assess robustness of effects of BCI-MI intervention, explorative subgroup analyses (17) will be carried out on primary outcome in subgroups of participants identified by: baseline score of UE-FMA/60 with 60 as maximum score (< 19, severe; from 20 to 47, moderate); side of stroke lesion (left/right hemisphere); age at enrolment ( < = 50 years vs > 50 years); time from stroke to admission to FSL site for rehabilitation care ( < = 3 months vs > 3 months). Estimates and confidence intervals will be obtained for each subgroup and will be presented through forest plots. Moreover, for each subgroup variable, statistical tests for interaction (12, 18) will be carried out by using ANOVA including the intervention group (BCI-MI vs Control-MI) and its interaction with the subgroup covariate. Exact p-values of test of interaction will be reported to allow detection of signals for further inspection.
Missing data
The primary analysis will be performed on all available cases. During the conduction of the trial, data collection will be monitored to minimise missing data for primary outcome. Moreover, sensitivity analyses for primary outcome are planned to assess robustness of findings in relation to missing data (19).
The following descriptions will be obtained separately for each intervention group:
-
proportion of participants with missing data for primary outcome;
-
comparison of baseline characteristics between participants with available data and participants with missing data for primary outcome.
If no important difference is observed in baseline characteristics between participants with available data and participants with missing data for primary outcome, comparisons of intervention groups for the primary outcome will be carried out by using multiple imputation technique for missing data (20). The estimation model will include variables used in stratification of patients at randomization (baseline score of UE-FMA/60 and side of stroke lesion), age at enrolment and gender; 20 imputations will be obtained.
In case of a relevant amount of missing data (percentage superior to 25% of drop-out considered), a revision of the SAP will be required.
Safety analysis
BCI-MI and Control-MI interventions are not expected to cause adverse effects (non-invasive procedure; no drug-administration) and both intervention delivery will be under the care of specialized personnel (physiotherapists, neurophysiology technician experts in patient EEG recordings). Previous experience with BCI training delivery in subacute stroke participants is encouraging since training was well-tolerated by the participants and no dropouts were reported.
Statistical software
Analyses will be carried out by STATA 17, and R software (version 4.3.0).
Data collection and management
Study data are collected and managed using REDCap electronic data capture tools hosted at Istituto Superiore di Sanità (Rome, Italy) (21, 22). An ad hoc REDCap template was developed in line with Case Report Form (CRF) of the Promotoer study. Specifically, there were two sections: 1) the “Baseline and Randomization Section” including demographical and clinical baseline data, randomization data, and training session data; and 2) the “Outcomes Section” including baseline neurophysiological assessments, and clinical and functional outcomes data at planned evaluation times. Section 1 data are filled in by unblinded personnel, and section 2 data are filled by outcome assessors blinded to assigned randomization intervention.
Automated export procedures for seamless data downloads to STATA and R statistical packages are available.