Design and population
This longitudinal study was performed among hospitalised older patients. The first face-to-face interview took place within the first four days of hospitalisation. The follow-up interview was performed three months after discharge by telephone.
Eligible participants were 70 years and older; had either a planned or unplanned hospital admission on medical or surgical wards of a university teaching hospital in the Netherlands, had an expected hospital stay of at least 48 hours; were able to speak and understand Dutch and were without cognitive impairment. Inclusion criteria were verified with the staff nurse. Patients were approached by a trained research assistant and gave signed informed consent.
Questionnaire: P-BAS HOP
The P-BAS HOP is an interview-based questionnaire. The baseline questionnaire consists of two parts: In the first part the interviewer lists subjects and the participant can indicate whether he experiences or expects limitations regarding that subject. In the second part is asked for the subjects that apply to the participant per subject whether it is a goal for the current hospitalisation and, if so, how important the goal is. Answer options are: does not apply to me; not at all important; moderately important; quite important and very important.
At follow-up, the participant is asked per goal to what extent the hospitalisation helped him to achieve that goal. The answer options are: not at all; moderately; quite; very.
With the scores of the baseline and follow-up questionnaire, a Patient Benefit Index (PBI) can be calculated: this is the mean of the benefits, weighted by the importance of the goals:
Other questionnaires and constructs
For the construct validity the following questionnaires or constructs were used:
Dutch VMS screening program (VMS)
The VMS questionnaire, which is developed as part of the Dutch National Hospital Safety Management Programme, consists of four instruments: Activities of daily living (ADL), falls, undernutrition and delirium (5). For the hypotheses to test the validity only the question about appetite is analysed: The participant is asked whether he experienced a decrease of appetite during the last month (yes/no). The questions were asked at baseline and follow-up.
Rotterdam Symptom Checklist (RSCL)
The RSCL was developed to measure symptoms reported by cancer patients participating in clinical research. It consists of a broad list with symptoms concerning psychological and physical distress (6). Originally, the symptoms are on a four points Likert scale, but we dichotomised the symptoms into present or absent on admission day.
Pain and Fatigue Numeric Rating Scale (NRS)
Participants were asked to rate their pain and fatigue as experienced at the moment of interview. The scale runs from 0: no pain/ fatigue at all – 10: the worst imaginable pain/ fatigue.
EQ-5D
The EQ-5D is a standardised, non-disease-specific instrument for describing and valuing health-related quality of life. It consists of five dimensions and a Visual Analogue Scale (VAS). The dimensions are mobility, self-care, usual activities, pain/discomfort and anxiety/depression, with three answer options each: no problems, some problems and extreme problems. The VAS, often referred to as the EuroQol ‘thermometer’, has an endpoint of 100 for best imaginable health state and 0 for worst imaginable health state (7). Participants were asked, during the baseline interview, to indicate their health state two weeks prior to hospital admission and, during the follow-up interview, to indicate their health state at the day of interview.
Admission reason
Admission reason was obtained from the medical record, recorded by the attending physician and retrieved by a medical student. Options were acute/elective; diagnostic/curative/palliative. When a participant gave no informed consent for record insight, this was labelled as ‘unknown’.
Katz-15 scale
The Katz-15 scale consists of fifteen items regarding basic activities of daily living such as the need for help with bathing and Instrumental Activities of Daily Living such as shopping. The answer options are dichotomous (8). Participants were asked, during the baseline interview, to indicate their functioning two weeks prior to hospital admission and, during the follow-up interview, to indicate their functioning at the day of interview.
Maastricht Social Participation Profile (MSPP)
The first half (46%) of the sample answered the MSPP (9). The MSPP is an instrument measuring the actual social participation by older adults. Participation is operationalised in part A: consumptive participation and formal social participation, and part B and C: informal social participation. In the original instrument informal social participation was split into participation with friends or acquaintances (part B) and participation with family (part C). In our study, we combined the items of B and C, resulting in a total of 18 items. For each item it was asked how frequently it was performed during the past four weeks with answer options 0, 1-3, 4-8, 9+ (9). We used individual items of the MSPP for the validation, but computed one sum score: All items concerning a day trip, which are items 3-8 of part A and items 4 and 5 of part B/C, are summed into ‘MSPP-daytrip’.
36-Item Short Form Survey Instrument (SF-36) – Social functioning
The second half of the participants (50%) answered the question ‘During the past four weeks, how much of the time has your physical health or emotional problems interfered with your social activities (like visiting with friends, relatives, etc.)?’, which is part of the SF-36 Health Survey, but is used as a single item in our survey. The answer options were: none of the time, a little of the time, some of the time, most of the time, all of the time (10).
Goals on hospital admission
Open question to the participant at baseline: ‘What do you hope to accomplish with this hospitalisation?’ The goal stated by the participant was repeated at follow-up and asked to what extent he had accomplished his goal with the answer options: ‘not at all’, ‘somewhat’, ‘moderately’ ‘quite’, or ‘very’.
Reliability
Test-retest reliability of the baseline questionnaire was performed with an interval of one to three days, while the participant was still hospitalised. The participant was not notified in advance of the retest, but asked for permission for another test on the other day.
For a better understanding of the difference between test and retest, a short qualitative evaluation was done: a selection of seven participants was asked, after the retest, to explain what caused the discrepancies per item between test and retest.
Test-retest of the follow-up questionnaire was performed in another sample than the baseline test-retest with an interval of seven to fourteen days. At the end of the first follow-up interview, the participant was asked permission to be called back a week later to repeat some questions, without specifying which questions.
Percentage of agreement, Cohen’s Kappa with quadratic weighting and maximum attainable kappa (11,12) were calculated per item for the agreement on importance of the goals on baseline, and the extend the hospitalisation helped to achieve the set goals on follow-up. Both the goal items ‘doesn’t apply to me’ and ‘not at all important’ were valued as zero. For the interpretation of the kappa values, the classification of Landis and Koch (13) was used.
The PBI was calculated for both test and retest of baseline and follow-up and compared with Intraclass Correlation Coefficient (ICC).
Validity
Baseline questionnaire
We developed the following hypotheses to test the construct validity of the baseline questionnaire:
- Participants who indicated a lack of appetite on the VMS and/or the RSCL, are expected to have a higher priority for the goal ‘appetite’.
- Participants who indicated tiredness and/or lack of energy on the RSCL, are expected to have a higher priority for the goal ‘energy’.
- Participants who indicated diarrhoea and/or constipation on the RSCL, are expected to have a higher priority for the goal ‘bowel movements’.
- Participants who indicated shortness of breath on the RSCL, are expected to have a higher priority for the goal ‘reducing shortness of breath’.
- Participants who had an acute admission and/or a diagnostic admission reason, are expected to have a higher priority for the goal ‘wanting to know what is wrong’.
- Participants with a higher NRS pain are expected to have a higher priority for the goal ‘pain’.
- Participants with a higher score on the question ‘During the past four weeks, how much of the time has your physical health or emotional problems interfered with your social activities?’, are expected to have a higher priority for the goal ‘visiting family or friends’.
- Goals that were mentioned after the open question, are, when applicable, indicated as minimum ‘somewhat important’ for the concerning goal.
Analysis. Hypotheses 1 to 5 were evaluated using Cramér’s V statistic. Hypotheses 6 and 7 were evaluated with the Spearman’s rank-order correlation. Since experiencing a symptom or restraint in a certain subject, does not necessarily mean that this goal is a priority for hospital admission, the hypotheses are confirmed if the correlation exceeds ‘small’ as defined by Cohen (14), meaning the correlation > 0.10. The answer option ‘does not apply to me now’ and ‘not at all important’ were coded as 0, the options somewhat important, quite important and very important were coded respectively as 1, 2, 3. Only when the assumptions of Cramér’s V statistic were not met because of too low (expected) cell frequency, categories were combined.
For hypothesis 8, a random selection of 50 cases was made and goals mentioned in the open question were coded using the item names of the P-BAS HOP. When a participant mentioned a goal that was not in the P-BAS HOP, it was coded as ‘other’. The coding was done by two researchers independently and then compared and discrepancies were solved by consensus. Subsequently, the percentage of agreement between the labels and the answers given in the P-BAS HOP was calculated.
The baseline questionnaire was considered valid if a minimum of 75%, thus six, of the first seven hypotheses were confirmed and hypothesis 8 was confirmed in a minimum of 75% of the selected cases.
Follow-up questionnaire
The extent to which the hospitalisation helped to achieve the set goals is compared with the progression or deterioration of items between baseline and follow-up from other known questionnaires. Hence the following hypotheses were formulated:
- Participants who indicated a deterioration on the Katz-15 items bathing and/or getting dressed and/ or the EQ-5D item self-care, are expected to have a lower score on the item ‘wash and dress yourself’.
- Participants who indicated a deterioration on the Katz-15 item walking and/or the EQ-5D item mobility, are expected to have a lower score on the item ‘walking’.
- Participants who indicated a deterioration on the Katz-15 item travelling, are expected to have a lower score on the item ‘driving’.
- Participants who indicated a deterioration on the Katz-15 item shopping, are expected to have a lower score on the item ‘groceries’.
- Participants who indicated a deterioration on the EQ-5D item pain/discomfort, are expected to have a lower score on the item ‘pain’.
- Participants who indicated a lack of appetite on the VMS, are expected to have a lower score on the item ‘appetite’.
- Participants who indicated a deterioration on the MSPP item organised sports and/or the MSPP item ‘done something with others that required considerable physical effort’, are expected to have a lower score on the item ‘sports’.
- Participants who indicated a deterioration on the MSPP item seeing family/acquaintances or the question ‘During the past 4 weeks, how much of the time has your physical health or emotional problems interfered with your social activities?’, are expected to have a lower score on the item ‘visiting family or friends’.
- Participants who moved from independent living to sheltered living or a nursing home, are expected to score lower on the item ‘return back to my home’.
- Participants with an increasing difference score between baseline and follow-up on the EQ-5D thermometer ‘general health’, are expected to have a higher score on the item ‘feeling better’.
- Participants with an increasing difference score between baseline and follow-up on the sum score ‘MSPP-daytrip’, are expected to have a higher score on the item ‘go on outings’.
- Participants with an increasing difference score between baseline and follow-up on the NRS fatigue, are expected to have a lower score on the item energy.
- Accomplishing goals noted on the open question correlate with the score on the P-BAS HOP, if applicable.
Analysis. Hypotheses 1 to 9 were evaluated using Cramér’s V statistic. Hypotheses 10 to 12 were evaluated with the Spearman’s rank-order correlation. Since experiencing a progression or deterioration in a certain subject, does not necessarily mean that this is due to the hospital admission, the hypotheses are confirmed if the correlation exceeds ‘small’ as defined by Cohen (14), meaning the correlation > 0.10.
For hypothesis 13 the same records were used as for hypothesis 8 on baseline. For the dyads with agreement between the code for the open question and the P-BAS HOP item, the Spearman’s rank-order correlation between the answer on the open question and the corresponding P-BAS HOP item was calculated. The hypothesis was confirmed if the correlation >0.50.
The follow-up questionnaire was considered valid if a minimum of 75%, thus nine of the first twelve hypotheses, were confirmed and hypothesis 13 was confirmed.
Responsiveness
The following anchor question was used to validate the PBI: ‘How much have you benefited from the admission?’ With the following answer options: not at all, a little bit, somewhat, much, very much.
The PBI is considered valid when it has a Spearman’s correlation coefficient > 0.50 with the anchor question.
Interpretability
The interpretability is evaluated with the visual anchor-based minimal important change distribution method (11,15). Participants who indicated: ‘not at all’, and ‘a little bit’, were considered as having no important benefit. Participants who indicated ‘very’ or ‘very much’, are considered as having important benefit. As it was not clear whether ‘somewhat benefit’ was considered as important benefit or not, we labelled this as ‘borderline’. The receiver operating characteristic (ROC) curve was used to determine the optimal cut-off points for important and no important benefit.
Missing values
When the P-BAS HOP was not administered, the case was completely deleted. For all other missing values, we used pairwise deletion. The computation of the PBI was based on non-missing items.