The Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist for developing and evaluating the self-nutrition risk screening tool for patients with gastric cancer after gastrectomy (SNRSGC) included the following: steps1 identification of a potential referred nutritional risk screening tool. steps 2 items and scores generation, steps 3 item reduction and steps 4 determination of the validity, reliability and responsiveness. Steps 1 and 2 involved developing a preliminary version of the questionnaire, which is described in the methods section. Step 3 involved testing the individual items and subscales of the preliminary version by analyzing patient responses. Based upon these analyses, a final version of the questionnaire was decided upon. Step 4 involved testing the final version of the questionnaire for validity, reliability and responsiveness. A flowchart of the complete study process is shown in figure 1.
2.1Identification of a potential referred nutritional risk screening tool
At present, research shows that the accuracy of screening tools can be improved by revising the entries of existing screening tools[28,29]. The goal of this instrument is to make postoperative patients with gastric cancer themselves evaluate nutritional risk screening. NRS2002 as convenience for application and less time consumption, has been widely used in clinical practice and recommended by many guidelines for nutritional assessment of patients undergoing abdominal surgery[30,31]. Moreover, compared with other nutritional risk screening tools such as MAN and MUST, it included the risk factors related to the patients’ nutrition such as surgery and chemotherapy. In addition, NRS2002 had a strong capacity to identify patients with cachexia (AUC = 0.916)[25]. NRS2002 were significantly associated with overall survival and nutritional measurement tools (BMI and anthropometric measurements) at six months after surgery[25,32]. Therefore, NRS2002 was selected as a potential referred nutritional risk screening tool for the development of the SNRSGC.
2.2Items and scores generation
In the item and scores generation phase, the following steps was operated:
(1) adjusting to establish the frame of SNRSGC according to the original study about NRS2002 [13].
The original developers of the NRS2002 were contacted to ask for permission to adapt the NRS2002 to develop SNRSGC suitable to patients. The main of some items in the NRS2002 did not understoode by the patients because the users of it was expected to be health care personnel such as calculation of BMI, daily intake and body energy demand. Therefore, the items will be revised or deleted. Moreover, some items related to the situations did not occur in the discharged patient will also be deleted such as intensive care patients, bone marrow transplantation and severe pneumonia. Though the some items in it had to been deleted, the basic framework of the NRS2002 “undernutrition;disease severity;age”was retained.
The determination of the scores in the items refers to the research hypothesis of NRS2002 that patients can benefit from nutritional support. For example, scoring standard: “Score=1, Protein requirement is increased, but in most case it can be met by oral diet or supplements. Score=2, Protein requirement is substantially increased but it can be met, although artificial feeding is required in many cases. Score=3, The protein requirement increases, so that in most cases, even artificial feed cannot be met, but protein breakdown and N loss can be attenuated significantly.”
(2) conducting a systematic review of the literature.
Most scholars suggest selecting an appropriate nutritional risk screening tool for different groups of people, or revising the scale according to the characteristics of different diseases [14]. The systematic review identified existing the specific nutritional indicators of postoperative patients with gastric cancer that were expected to improve the accuracy of screening. The factors that affect the nutritional status of postoperative patients with gastric cancer was chosen as a template for the development of the SNRSGC’s items. A literature search was conducted to obtain the factors using electronic databases Medline, PubMed, China National Knowledge Infrastructure and web of science. The search keywords included: nutrition /malnutrition /gastric cancer /stomach Neoplasms, and a manual review of the references in the selected studies was performed to identify further publications. The determination of the inclusion and exclusion criteria of the studies refers to the research hypothesis of NRS2002 that patients can benefit from nutritional supplement. The inclusion and exclusion criteria of the studies was showed in the table 1.
(3) Updated and refer to the cut-off value according to GLIM.
As malnutrition assessment and diagnosis methods are constantly updated, the development of tools should be revised in conjunction with the latest nutrition guidelines. In addition, some nutritional index cut-off values in NRS2002 are not applicable to Chinese patients, so it is necessary to change the items in it[33] . According to Global Leadership Initiative on Malnutrition (GLIM)[12], easy to measure and necessary nutrition measurement indexes are added in SNRSGC and refer to the cut-off value of nutrition indexes for Asians given therein.
2.3 Item reduction process
Rating of SNRSGC items by an expert panel
The second step involved expert enquiry in the field. In order to allow multi-professional cooperation and comprehensive evaluation of items, the complete item pool will be evaluated by 15 experts with knowledge of nursing, nutrition and medical of postoperative patients with gastric cancer. The experts involved in the study (1) who have more than 10 years working experience in this field (2) who with intermediate title or above (3) who with bachelor degree or above. This process of designing uses questionnaire literature methodology. Experts were required to evaluate each item and scores on the consultation list according to the importance, the basis and familiarity and put forward comments and proposal for relevant indicators. Experts will be asked to rate the relevance of each item on a Likert scale: 1 (not relevant); 2 (item needs some revision); 3 (relevant but needs minor revision); and 4 (very relevant) as well as provide an explanation for their decision and suggestions on any missing items. Respondents could participate in a WeChat or telephone call with the lead author to provide further feedback, if desired. All written and verbal feedback was analyzed, and minor wording revisions were made to produce the final tool. The purpose of the rating of SNRSGC items by an expert panel was to identify relevant items that were missing and to improve the readability and comprehension of the questionnaire.
Based upon the first and second administration of the preliminary SNRSGC version, item reduction was performed using the following strategy, which incorporated both expert and patient's opinions. The item-level content validity index (I-CVI) was calculated for each principle and item (number of respondents giving a rating of either 3 or 4, divided by the total number of respondents)[34]. One weakness of CVI is its failure to adjust for chance agreement. We solved this by translating item-level CVIs (I-CVIs) into values of a modified kappa statistic. Denise F. Polit et al. believe that items with an I-CVI of 0.78 or higher can be considered as evidence of good content validity[34]. Items with I-CVI less than 0.78 should be corrected, and items less than 0.5 should be deleted[34]. If the experts’ ratings of item relevance are quite different, we will do second-round panel and calculate S-CVI.
Piloting
The final step in the item generation process was to interview postoperative patients with gastric cancer and their families individually. The final version was made into a WeChat mini program and distributed to patients or family members. The postoperative patients with gastric cancer and their families in the outpatient clinic were randomly selected and asked to assess the difficulty of each item in the SNRSGC questionnaire, as very easy, easy, not easy-not hard, hard, or very hard. Inclusion criteria: patients were those: Age > 18 years old; the postoperative patients with gastric cancer; patients and/or patients’ families with reading ability, clear consciousness and normal expression ability and using mobile software. Exclusion criteria: Patients with other malignant tumors; Patients with metabolic diseases; Palliative surgery patients. The purpose of the patients and their families interviews was to identify relevant items that were missing, to improve the readability and comprehension of the questionnaire and evaluate the patient cognitive burden of screening methods.
The patient's assessment of the difficulty of SNRSGC items is expressed as a percentage. When the number of people who choose "hard" exceeds 50%, the item should be revised.
2.4 To evaluate the validity, reliability and patient acceptability of SNRSGC.
Methodological testing and evaluation of measurement qualities of SNRSGC using the COSMIN checklist
We evaluated the final SNRSGC measurement properties according to the consensus -based Standards for the selection of health Measurements Instruments (COSMIN) guidelines. The purpose of the COSMIN checklist is to evaluate the methodological quality of studies concerning measurement properties of health-related patient-reported outcomes (HR-PROs) instruments.[35,36]
Participants
Participant were recruited from the inpatient and outpatient department of gastrointestinal surgery in a large tertiary hospital in China. Patients eligible for inclusion were those: Age > 18 years old; The patients with gastric cancer confirmed by histopathological biopsy were operated on; Be able to read the scale; Clear consciousness and normal expression ability; Able to use mobile software. Exclusion criteria; Patients with other malignant tumors; Patients with metabolic diseases; Palliative surgery patients; The patients who could not eat by mouth after operation; Refused to participate in this study through communication.
Caregiver eligible for inclusion were those who provided care for the gastric cancer surgery patients older than 18 years in the past three months and were able to read the scale; Clear consciousness and normal expression ability; Able to use mobile software.
Sample size calculation: According to the minimum sample size of diagnostic experiment, the minimum sample size is 151[32,37]. The minimum sample size for test–retest is 50[38]. In order to ensure the validity of the statistical results, we adopt the stratified sampling method. Stratified according to the postoperative time, divided into three months after surgery and more than three months after surgery, the number of people in each group is greater than 50[36].
According to the usage of NRS2002 and SNRSGC, data collection and inclusion and exclusion criteria, a training slide was made for researchers to conduct training. The researcher provided the questionnaire to the patients in the form of a mini program and collected the data. When the patients filled in the questionnaire, the researcher evaluated the NRS2002 score of the patients.
Internal consistency
Internal consistency or item homogeneity is often used for estimating intra-scale reliability in terms of the item variances and covariances derived from a single
occasion of measurement. A principal component factor analysis was performed on the individual subscales to assess their structural validity. Cronbach’ s α was calculated per subscale and a score above 0.70 was taken as an indication of sufficient homogeneity of the items in the subscale[39,40].
Construct validity
Construct validity refers to the extent to which scores on a particular instrument relate to other measures. This construct validity was determined by cross-sectional comparison of the questionnaires when first administered. We conducted exploratory factor analysis to explore the unidimensionality of SNRSGC item. Items were considered to load on a factor if the factor loading was 0.32 or greater[41]. The tool developed in this research aims to distinguish whether patients have nutritional risks and whether patients can benefit from nutritional supplements[42]. Since there is no gold standard for health self-reporting, we cannot compare NRS2002 as a gold standard with SNRSGC. The expert group of COSMIN checklist recommends that it be regarded as structural validity[36]. Therefore, the area under the receiver operating characteristic (ROC) curve, sensitivity and specificity were determined to investigate the concurrent validity of the SNRSGC considering the original NRS2002 as the reference method. Area under the receiver operating characteristic curve (AUC) >0.7 is considered adequate[38].
Test–retest reliability
Test–retest reliability refers to the similarity of the scores of the same patient after repeated measurement over a period of time. Test–retest reliability is divided into two parts: agreement and reliability. The parameter of agreement in this study uses the description of Bland and Altman. Their limits of agreement equal the mean change in scores of repeated measurements (mean change) ± 1.96 × standard deviation of these changes (SD change). The limits of agreement are often reported because they are easily interpretable. Note that SD change equals √2 × SEM consistency. SEM equals the square root of the error variance of an ANOVA analysis. We used intraclass correlation coefficient (ICC) to evaluate reliability, with Kappa values≥0.7 considered adequate.
Patients reporting scores as ‘unchanged’ were considered stable and included in test–retest reliability analysis. Since gastric cancer patients have large changes in body weight and nutritional status within three months after surgery[26,43],they are not suitable for testing the reliability of test-retest reliability; Therefore, we tested the test-retest reliability of patients three months after operation. The interval between the two surveys was one week.
Interpretability
Interpretability is the degree to which one can assign qualitative meaning to an instrument’s quantitative scores or change in scores.
Interpretability refers to the degree of qualitative significance of the quantitative score of SNRSGC. Interpretability includes the distribution of total scores and change scores in the study sample and in relevant subgroups, floor and ceiling effects, estimates of minimal important change (MIC) and/or minimal important difference (MID)[44]. If the highest or lowest score exceeds 15%, the validity of the tool content is considered limited. Patients with the lowest or highest scores cannot be distinguished from each other, reducing reliability.
Acceptability
The acceptability of the SNRSGC was assessed by the patient with an additional questionnaire. The patient categorically assessed the difficulty of the SNRSGC questionnaire, as very easy, easy, not easy-not hard, hard, or very hard. Furthermore, the patient assessed the time spent on the SNRSGC categorically as <5, 5–10, 10–15, or >15 min.
Data collection included patient's SNRSGC score, time of SNRSGC, NRS2002 score, difficulty of questionnaire, tumor stage, filling person's educational background, postoperative time and so on.