We performed a translation of the OKS-APQ into Dutch and prospectively evaluated the measurement properties of the Dutch version. Measurement properties were evaluated using COSMIN quality criteria(6).
Procedure of translation
The OKS-APQ questionnaire was translated from English to Dutch according to the advised forward-backward translation multi step approach for translation as described by Beaton et al(7,8). During the last step, the definitive version was tested in a subset of 5 preoperative and 5 postoperative TKA patients. These patients completed the questionnaire at home and were asked to make notes if they thought a question was difficult to understand. No issues regarding the items of the OKS-APQ questionnaire were reported by the patients.
Patients
As a rule of thumb, at least 100 patients were required to perform methodological testing(9), and we aimed to include preoperative and postoperative patients. The preoperative study sample was recruited from the waiting list for TKA and the postoperative patients were selected from the outpatient registry. There were no age restrictions. All patients underwent TKA at the department of orthopaedics at the Sint Maartenskliniek in Nijmegen between 2015 and 2018. Patients unable to speak and understand Dutch written language were excluded. The study was assessed by the local review committee, but not by the Medical Ethics Committee because this study was not subject to the Dutch medical research involving human subject act. All patients gave their written informed consent to participate in the study.
Questionnaires
In addition to completing the OKS-APQ, patients completed additional condition-specific questionnaires which are commonly used in pre- and postoperative TKA patients between January 2017 and December 2019. All preoperative patients completed the following four questionnaires: the OKS-APQ, the Oxford Knee Score (OKS)(10), the Short Form-36 (SF-36)(11), and a Visual Analogue Scale for pain(12). Postoperative patients also completed an additional fifth questionnaire, the Forgotten Joint Score (FJS)(13). All patients were asked to complete the questionnaires for a second time, after a minimum of two weeks, which was considered appropriate for the test-retest reliability(6).
Evaluation of measurement properties
Oxford Knee Score - Activity and Participation (OKS-APQ)
The OKS-APQ eight-item questionnaire was developed to measure higher levels of activity and participation and is recommended to be used to complement the OKS as an additional scale(4). Items are scored on a five-point Likert scale, ranging from 0 “strongly agree” to 4 “strongly disagree”. Total summary score ranges from 0 to 32, and scores are converted to a 0 to 100 measure(4). A lower total sum score represents lower levels of activity and participation(4).
Oxford Knee Score (OKS)
The OKS 12-item questionnaire has been developed for patients undergoing TKA to evaluate the patients’ perception of pain and functional impairment in the knee(10). The questionnaire consists of 12 questions and it is possible to derive separate OKS pain and function subscales(10). Responses are scored on a 5-point Likert scale, ranging from 0 “significant disability” to 4 “no problem”, in which the final score is an aggregate, sum score for pain and function(14). The total scores ranges from 0 to 48; a lower OKS sum score represents poor function and more pain. The OKS has good measurement properties(10), however ceiling effects were demonstrated in postoperative patients(15,16).
MOS Short Form 36 (SF-36)
The SF-36 is a 36-item questionnaire assessing health-related Quality of Life (QoL). It consists of eight dimensions that are aggregated to two summary scores: Physical Component Score (PCS) and Mental Component Score (MCS) (both 0-100)(11). The SF-36 is widely used and has shown to be reliable and valid in the Dutch general population(11,17,18). A lower score represents a lower level of QoL (10).
Visual analogue scale for pain (VAS pain)
The VAS for pain is a single item scale assessing the intensity of pain in the knee during the past 2 to 3 days. The 100-mm VAS is simple to use, and has already been applied in different populations and settings(12). The score varies from 0 (no pain), to 100 (worst pain). It has shown to be valid and reliable(12).
Forgotten Joint Score (FJS-12)
The 12-item Forgotten Joint Score (FJS-12) questionnaire evaluates the patients’ ‘joint awareness’ during activities of daily living (i.e. stair climbing, walking and gardening). The responses were scored on a five-point Likert scale, ranging from 0 “never” to 4 “mostly”. Item scores were summed and converted to a 0 to 100 scale, a low total sum score reflects that the patient is not able to forget the affected/replaced joint during activities of daily living(13).
Methodological testing & statistical analysis
Kolmogorov Smirnov test was used to test the normality of the OKS-APQ items, OKS-APQ total score and other PROM total scores. Descriptive statistics were used to summarize the data; mean and standard deviation (SD) or median (25th – 75th percentile) for continuous variables and counts and percentages for categorical variables, and to investigate the frequencies of missing data. All statistical analyses were performed in STATA version 13.0 (StataCorp, College Station, Texas). A P-value < 0.05 was considered statistically significant for all analyses.
Reliability
Reliability is the degree to which the Dutch OKS-APQ is free from measurement error. To evaluate reliability, internal consistency, test-retest reliability, the measurement error and the smallest detectable change were calculated.
Internal consistency
Internal consistency is a measure to evaluate to what extent the eight items of the Dutch OKS-APQ refer to the same underlying construct(9). Internal consistency of the Dutch version of the OKS-APQ was determined by calculating the Cronbach’s alpha(9). A Cronbach’s alpha between 0.7 and 0.9 for the eight items of the OKS-APQ indicates good internal consistency(9). The Cronbach’s alpha was measured on the pooled sample and the separate pre- and postoperative samples.
Test-retest reliability
Test-retest reliability involves the degree to which the results of the Dutch OKS-APQ are consistent across repeated measurements(9). To evaluate the reliability of the Dutch OKS-APQ, we calculated intraclass correlation coefficients (ICCs) with a 95% confidence interval (95% CI). In addition, we provided the different variance components to show the systematic differences between the two timepoints in preoperative and postoperative patients. More specific, we used the ICC two-way random effects model type agreement to measure the reliability(9). An ICC equal to and larger than 0.7 is generally accepted as good(9). ICCs were calculated for the pooled sample and the separate pre- and postoperative samples.
Measurement error & Smallest detectable change
The measurement error is the systematic and random error of a participant’s score that is not attributed to true changes in the construct to be measured(6). The standard error of measurement (SEM) was calculated using the square root of the error variance(9,19).
The smallest detectable change (SDC) reflects the smallest individual change in score that can be interpreted as a real change in one individual (SDCind). This was calculated by the SEM * 1.96 * √2(9,19). The SDCind can be divided by √n (n = sample size) to calculate the SDC in a group of patients (SDCgroup)(9,19). SEM and SDC were calculated for the pooled sample and the separate pre- and postoperative samples.
Validity
Validity is the degree to which the Dutch OKS-APQ measures the construct(s) it purports to measure. To evaluate validity, structural validity, construct validity and content validity were measured.
Structural validity
Confirmatory factor analyses (CFA) was used to validate the 1-factor structure of the original English version of the OKS-APQ(4). We examined the comparative fit index (CFI), the Tucker-Lewis Index (TLI), the root mean square error of approximation (RMSEA) and the standardized root mean squared residual (SRMR) to assess goodness of fit of this model. The following indices of fit were considered satisfactory: CFI: > 0.95; RMSEA (0 to 1): < 0.05 representing close fit, <0.08 good fit, <0.1 satisfactory fit and SRMR: < 0.05(20). CFA was assessed using the pooled sample (pre- and postoperative patients).
Construct validity
Validity is the degree to which the OKS-APQ measures the construct it supposes to measure. Since there is no gold standard in the measurement of PROMs, validity was measured as construct validity(9). Construct validity refers to the extent to which the OKS-APQ was related to other measures based on theoretically derived, predefined hypotheses. Construct validity was supported when at least 75% of the results are in accordance with the predefined hypotheses (Table 1)(9). Construct validity was expressed by assessing Pearson correlation coefficients or the nonparametric Spearman’s correlation coefficients(9). The strength of the correlations was interpreted as “weak” (r = 0.10 - 0.30), “moderate” (r = 0.31 - 0.50) or “strong” (r = 0.51 – 1.00)(21). Predefined hypotheses were formulated for the pooled and separate pre- and postoperative samples.
Content validity
A quality criterion for content validity is the absence of floor and ceiling effects. Presence of floor and ceiling effects on the OKS-APQ may influence the test-retest reliability, and construct validity of the questionnaire(9). Patients with the lowest or highest possible score cannot be distinguished from each other, thus reliability is reduced(9). Floor and ceiling effects, in pre- and postoperative samples separately, were determined by calculating the number of individuals that obtained the lowest (0) or highest (100) scores possible and were considered present if more than 15% of the patients achieved the highest or lowest total summary score(9). In addition, floor and ceiling effects on item-level was determined.
Table 1
Predefined hypotheses for evaluating the construct validity of the Dutch OKS-APQ
Hypothesis
|
A. Strong positive correlation (r >0.50):
|
1. A strong positive correlation between OKS-APQ and OKS (pooled, pre- and postoperative patients);
|
2. A strong positive correlation between OKS-APQ and FJS (in postoperative patients);
|
3. A strong positive correlation between OKS-APQ and PCS (SF-36) (pooled, pre- and postoperative patients);
|
B. Moderate to strong negative correlation (r >0.31):
|
4. A moderate to strong negative correlation between OKS-APQ and VAS pain (pooled, pre- and postoperative patients);
|
C. Weak to moderate positive correlation (r 0.10 – 0.50):
|
5. A weak to moderate positive correlation between OKS-APQ and MCS (SF-36) (pooled, pre- and postoperative patients);
|
Abbreviations: OKS-APQ, Oxford knee score – Activity & Participation Questionnaire; FJS, Forgotten joint score; OKS, Oxford knee score; SF-36, 36-Item Short Form Health Survey Questionnaire; VAS for pain, Visual Analogue Scale; PCS, Physical Component Score; MCS, Mental Component Score.
|