See the published version of this preprint at BMC Medical Research Methodology.
This is a preprint, a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
A new preprint design is coming!
We have improved readability, clarity, accessibility, and opportunities for engagement.
Research article
Mandy Roheger
Universitatsklinikum Greifswald
Corresponding Author
ORCiD: https://orcid.org/0000-0002-6015-3194
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Background
Even though investigating predictors of intervention success (e.g Cognitive Training, CT) is gaining more and more interest in the light of an individualized medicine, results on specific predictors of intervention success in the overall field are mixed and inconsistent due to different and sometimes inappropriate statistical methods used. Therefore, the present paper gives a guidance on the appropriate use of multiple regression analyses to identify predictors of CT and similar non-pharmacological interventions.
Methods
We simulated data based on a predefined true model and ran a series of different analyses to evaluate their performance in retrieving the true model coefficients. The true model consisted of a 2 (between: experimental vs. control group) x 2 (within: pre- vs. post-treatment) design with two continuous predictors, one of which predicted the success in the intervention group and the other did not. In analyzing the data, we considered four commonly used dependent variables (post-test score, absolute change score, relative change score, residual score), five regression models, eight sample sizes, and four levels of reliability.
Results
Our results indicated that a regression model including the investigated predictor, Group (experimental vs. control), pre-test score, and the interaction between the investigated predictor and the Group as predictors, and the absolute change score as the dependent variable seemed most convenient for the given experimental design. Although the pre-test score should be included as a predictor in the regression model for reasons of statistical power, its coefficient should not be interpreted because even if there is no true relationship, a negative and statistically significant regression coefficient commonly emerges.
Conclusion
Employing simulation methods, theoretical reasoning, and mathematical derivations, we were able to derive recommendations regarding the analysis of data in one of the most prevalent experimental designs in research on CT and external predictors of CT success. These insights can contribute to the application of considered data analyses in future studies and facilitate cumulative knowledge gain.
Keywords
prognostic research, simulation study, methodology, regression analysis, cognitive decline, cognitive training
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
The full text of this article is available to read as a PDF.
This is a list of supplementary files associated with this preprint. Click to download.
Loading...
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Published
View the published article at BMC Medical Research Methodology.
No community comments so far
Submission checks complete
On 27 Nov, 2020
Editorial decision: Accept
On 22 Nov, 2020
Version 3
Posted 04 Nov, 2020
No comments provided
Reviewer #1 agreed
On 15 Nov, 2020
Reviewers invited
Invitations sent on 29 Oct, 2020
Editor assigned
On 28 Oct, 2020
Submission checks complete
On 27 Oct, 2020
Editor invited
On 27 Oct, 2020
No comments provided
Editorial decision: Major revision
On 04 Oct, 2020
Review #2 received
Received 30 Sep, 2020
Review #1 received
Received 29 Sep, 2020
Reviewer #2 agreed
On 04 Sep, 2020
Reviewer #1 agreed
On 04 Sep, 2020
Reviewers invited
Invitations sent on 01 Sep, 2020
Editor assigned
On 17 Aug, 2020
Submission checks complete
On 16 Aug, 2020
Editor invited
On 16 Aug, 2020
No comments provided
Editorial decision: Revise before sending to peer reviewers
On 28 Jul, 2020
Submission checks complete
On 15 Jul, 2020
Editor invited
On 15 Jul, 2020
Editor assigned
On 15 Jul, 2020
First submitted
On 14 Jul, 2020
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Subject Areas
Health Economics & Outcomes Research
Learn more about our company.
A new preprint design is coming!
We have improved readability, clarity, accessibility, and opportunities for engagement.
See the published version of this preprint at BMC Medical Research Methodology.
This is a preprint, a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research article
Mandy Roheger
Universitatsklinikum Greifswald
Corresponding Author
ORCiD: https://orcid.org/0000-0002-6015-3194
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Published
View the published article at BMC Medical Research Methodology.
No community comments so far
Submission checks complete
On 27 Nov, 2020
Editorial decision: Accept
On 22 Nov, 2020
Version 3
Posted 04 Nov, 2020
No comments provided
Reviewer #1 agreed
On 15 Nov, 2020
Reviewers invited
Invitations sent on 29 Oct, 2020
Editor assigned
On 28 Oct, 2020
Submission checks complete
On 27 Oct, 2020
Editor invited
On 27 Oct, 2020
No comments provided
Editorial decision: Major revision
On 04 Oct, 2020
Review #2 received
Received 30 Sep, 2020
Review #1 received
Received 29 Sep, 2020
Reviewer #2 agreed
On 04 Sep, 2020
Reviewer #1 agreed
On 04 Sep, 2020
Reviewers invited
Invitations sent on 01 Sep, 2020
Editor assigned
On 17 Aug, 2020
Submission checks complete
On 16 Aug, 2020
Editor invited
On 16 Aug, 2020
No comments provided
Editorial decision: Revise before sending to peer reviewers
On 28 Jul, 2020
Submission checks complete
On 15 Jul, 2020
Editor invited
On 15 Jul, 2020
Editor assigned
On 15 Jul, 2020
First submitted
On 14 Jul, 2020
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Subject Areas
Health Economics & Outcomes Research
License:
This work is licensed under a CC BY 4.0 License. Read Full License
View the published article at BMC Medical Research Methodology.
Background
Even though investigating predictors of intervention success (e.g Cognitive Training, CT) is gaining more and more interest in the light of an individualized medicine, results on specific predictors of intervention success in the overall field are mixed and inconsistent due to different and sometimes inappropriate statistical methods used. Therefore, the present paper gives a guidance on the appropriate use of multiple regression analyses to identify predictors of CT and similar non-pharmacological interventions.
Methods
We simulated data based on a predefined true model and ran a series of different analyses to evaluate their performance in retrieving the true model coefficients. The true model consisted of a 2 (between: experimental vs. control group) x 2 (within: pre- vs. post-treatment) design with two continuous predictors, one of which predicted the success in the intervention group and the other did not. In analyzing the data, we considered four commonly used dependent variables (post-test score, absolute change score, relative change score, residual score), five regression models, eight sample sizes, and four levels of reliability.
Results
Our results indicated that a regression model including the investigated predictor, Group (experimental vs. control), pre-test score, and the interaction between the investigated predictor and the Group as predictors, and the absolute change score as the dependent variable seemed most convenient for the given experimental design. Although the pre-test score should be included as a predictor in the regression model for reasons of statistical power, its coefficient should not be interpreted because even if there is no true relationship, a negative and statistically significant regression coefficient commonly emerges.
Conclusion
Employing simulation methods, theoretical reasoning, and mathematical derivations, we were able to derive recommendations regarding the analysis of data in one of the most prevalent experimental designs in research on CT and external predictors of CT success. These insights can contribute to the application of considered data analyses in future studies and facilitate cumulative knowledge gain.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
The full text of this article is available to read as a PDF.
This is a list of supplementary files associated with this preprint. Click to download.
Loading...