Participants
Patients diagnosed with metastatic spinal disease between January 2016 and January 2020 were recruited. The participant population was advised to be at least 50 for appropriate analysis of reliability, construct validity, as well as ceiling or floor effects, and 100 patients are needed for internal consistency analysis(8). Patients participating in the study were all diagnosed with metastatic spinal disease confirmed by pathology or PET-CT. Eight spine evaluators are at least 6 years of education and ability to read and speak Chinese. Patients excluded were those who reported a history of spinal surgery or whose spinal disease was caused by infection, ankylosing spondylitis, or systemic rheumatologic disease. Complete imaging results and available clinical data were required for all patients. Complete imaging results included sagittal view, axial view and coronal view of magnetic resonance imaging (MRI). Clinical data included demographic characteristics, neurological function, tumor levels, pathology report, complications and surgical treatment. The study was approved by the Ethics Committee of our institution, and all patients signed a written informed consent.
Instruments
SINS
SINS, a comprehensive classification to diagnose neoplastic spinal instability, comprises six individual component scores: spine location, pain, lesion bone quality, radiographic alignment, vertebral body collapse, and posterolateral involvement of the spinal elements. The maximum score is 18, and the minimum is 0. The total score is divided into three categories in terms of stability: stable (0-6 points), potentially unstable (7-12 points), and unstable (13-18 points). In addition, the SINS score can also be analyzed as a binary indicator of surgical referral status: ‘stable’ (0-6 points) or ‘current or possible instability’ (7-18 points). A surgical consultation is recommended for patients with SINS scores greater than 7.
Kostuik classification
The Kostuik classification was used to classify the degree of tumor involvement of the spinal column. The vertebral body is divided into 6 components, and three categories of stability are considered: stable (1-2 partial damages), relatively unsteady (3-4 partial damages) and absolutely unsteady (5-6 partial damages).
Procedure
The study was conducted in two phases: first, the SINS was translated into simplified Chinese; second, the factor structure, internal consistency, test-retest reliability, validity, and floor and ceiling effects of the SC-SINS were assessed. The procedures followed the cross-cultural adaptation guidelines issued by the American Association of Orthopedic Surgeons Outcome Committee.
Stage I: Forward translation into simplified Chinese
The 6-component SINS was translated into simplified Chinese independently by 2 bilingual translators who spoke Chinese as the first language. One translator was a medical professional who knew the concepts related to the index well; the other translator was a professional translator with no medical background and was blind for the objective of the study.
Stage II: Synthesis of the translations
The expert committee, consisting of translators, radiologist and spinal surgeons specialized in metastatic spinal disease discussed the translations and compared them with the original English version SINS. After reaching a consensus, the forward translations were formulated into one single simplified Chinese version.
Stage III: Backward translation into English
Then backward translation was undertaken independently by another professional bilingual translator, a radiologist and a spinal orthopedist. All lacked medical backgrounds and were not aware of the prior translation procedures. They independently and blindly translated the simplified Chinese version back into English. Each English translation was then compared with the original English version and checked for inconsistencies.
Stage IV: Expert committee
The expert committee consolidated all the translations and discussed with all the translators, bilingual experts and spinal surgeons. A consensus was reached on all discrepancies. Then the committee came into an agreement on the equivalence between the original version and the target version. Finally, the SC-SINS was created.
Stage V: Evaluation of the pre-final version
The data of 28 patients were collected for pilot test by evaluators. Each subsequently pointed out their difficulties in completing the classification or understanding the purpose and meaning of each question. The expert committee discussed all the findings and then developed the final version of SC-SINS which was used for further psychometric testing.
Stage VI: Evaluation of the final version
A booklet that included the final SC-SINS and informed consent form was given to all participating patients who met the inclusion/exclusion criteria. We evaluated the internal consistency, test-retest reliability, and floor and ceiling effects of the final version. Each patient’s demographic characteristics were recorded.
Statistical analysis
SPSS18.0 (Chicago, IL, USA) was used for statistical analysis. Data were expressed as the mean ± standard deviation (SD). Values were reported with 95% confidence intervals (CIs) and P-values <0.05 indicated statistical significance.
Internal consistency
Internal consistency reliability was evaluated using Cronbach’s a coefficient for each domain(9). High Cronbach’s a indicated high correlations among the items. Cronbach’s a of ≥0.70 was considered satisfactory. In addition, the item-total correlations of each item were calculated. Levels of agreement for a were graded according to the recommendations of Landis and Koch, with a value of 0.00 to 0.20 considered slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; and 0.81 to 1.00, almost perfect agreement(10).
Test-retest reliability
The test-retest reliability was assessed by comparing the results of the first and final SC-SINS scales. Two-way ANOVA random-effects intra-class correlation coefficient (ICC) was calculated to quantify the test-retest reliability(11, 12). We assessed 60 patients for a second time after the first completion. The sequence of SC-SINS was rearranged to reduce the memory error. A 4-week interval was designed between the two tests. ICC values ranged from 0 to 1, and a higher value indicated higher repeatability. An ICC above 0.7 could be accepted as good and below 0.4 as poor reliability(13). The correlation values were as follows: 0-0.20, poor; 0.21-0.40, fair; 0.41-0.60, moderate; 0.61-0.80, very good; 0.81-1.0, excellent.
Validity
To assess criterion-related validity, we examined construct validity. We evaluated the relationship between the SC-SINS and Kostuik classification using the Pearson correlation coefficients. Correlation values of 0.81-1.0 was considered excellent, 0.61-0.80 very good, 0.41-0.60 good, 0.21-0.40 fair, and 0-0.20 poor.
Structural factor analysis
We used factor analysis to evaluate the factor structure of the SC-SINS, and to confirm the subscales. Since the original SC-SINS indicated that the items were distributed across six subscales, we used principal component analysis rotation to confirm the factor structure of each subscale, rather than exploratory factor analysis. Item loadings on each factor equal to or greater than 0.4 were considered satisfactory. In addition, floor and ceiling effects (defined as the percentage of participants displaying the minimum and maximum possible scores, respectively) were calculated and those with over 15% of respondents achieving the lowest or highest possible total scores were considered significant(11).