Instrument Development
Step 1: item generation
The instrument instructions directed survey respondents to think about the last time that they had a bothersome symptom of their chronic illness and then rate how much each item influenced their decision about what to do in response to that symptom. The initial draft of the instrument included 42 items, with several items for each of the 10 contextual factors. For example, for prior experience, items captured both having experience (e.g., I thought about similar past decisions) and lack of experience (e.g., the symptom was new to me). Each item was rated on a 5-point Likert scale with response options of not at all (1), a little (2), some (3), a lot (4), and a great deal (5).
Step 2: item refinement with patient input
Five adult women, ages 43-71, completed the cognitive interviews. Each had multiple chronic conditions and had been living with at least one symptomatic chronic condition for more than 10 years. Based on the responses of these adults, 23 items were retained as written, 7 items were revised, and 11 items were added. Item revisions were made to improve clarity. For example, “I recognized this from last time” was changed to “I recognized this symptom from last time”. Items were added when participants identified that a factor that influences their decision was not captured by existing items. For example, a participant identified that her decision making is affected by depressive symptoms, so the item “I felt too down, so I put off making a decision” was added. Finally, 12 items were deleted as irrelevant (8 items) or redundant (4 items). The refined draft of the instrument included 41 items.
Step 3a: Delphi survey
Twenty-six experts were invited via email to complete the Delphi survey. There were 12 respondents (9 female, 3 male) in round 1 and all 12 respondents also completed round 2. Experts were from United States (n= 7), Italy (n=4), and Germany (n=1) and reported an average of 13 years of experience caring for adults with chronic illness. Eleven out of 12 experts had a PhD and one had a master’s degree. I-CVI and clarity data for each Delphi round are summarized in Table 1.
Table 1. I-CVI and Clarity Data by Delphi Round
|
# of experts
|
# of items in round
|
I-CVIa range
|
% of I-CVI > 0.78b
|
Clarityc range
|
Items rated as clear by ≤ 75% of expertsd
|
Items deleted
or added in round
|
Round 1
|
12
|
41
|
0.5 – 1.0
|
95%
|
42-100%
|
12 items
|
9 deleted
5 added
|
Round 2
|
12
|
37
|
0.83 - 1.0
|
100%
|
75-100%
|
0 items
|
2 added
|
a Item Content Validity Index (I-CVI) = number of respondents who rated the item as ‘highly relevant’ or ‘quite relevant’ divided by total number of respondents.
b An I-CVI of 0.78 or higher indicates good content validity at the item level
c Clarity = number of respondents who rated the item as clear divided by the total number of respondents
d Items rated as clear by less than 75% of experts required revision
The Delphi survey was closed after the second round as consensus on item relevance and clarity was achieved. The S-CVI/Ave of this 39-item instrument was excellent at 0.92.
Step 3b: cognitive interviews
Five adults (3 female, 2 male), ages 44-70, completed the second round of cognitive interviews. Four adults had multiple chronic conditions, while one adult had only one condition. Two adults had been diagnosed in the last 3 years, while 3 adults reported having at least one symptomatic chronic condition for more than 10 years.
In these cognitive interviews, respondents reported that items were relevant to their experience and the instrument was comprehensive. No new items were suggested. For three items, participants reported confusion about wording and endorsed multiple interpretations of the item. These three items were deleted because there were other items that captured the same contextual factor and were clearer to participants. One item, “I worried about the cost of treatment”, was deleted because it reflected access to insurance coverage, which differs across countries. The instrument instructions were also shortened and simplified based on participant feedback. The anchors of the 5-point scale were change to “No Influence” (1) and “A Lot of Influence” (5). Following content validity testing, the instrument contained 35 items.
Psychometric Testing
Invitations to participate were sent to 1,127 individuals who expressed interested in the study on Researchmatch.org. A total of 431 individuals completed the survey for a response rate of 38.2%. The typical participant was female, White, non-Hispanic, with at least some college education (Table 2).
Table 2: Participant Characteristics (n = 431)
|
n (%)
|
Age mean (sd)
|
54.93 (16.15)
|
Gender (n= 426)
|
|
Female
|
302 (70.1)
|
Race (n=425)
|
|
White
|
375 (87)
|
Black
|
20 (4.6)
|
Native American/Alaska Native
|
2 (0.5)
|
Asian
|
6 (1.4)
|
Native Hawaiian/Pacific Islander
|
1 (0.2)
|
Mixed (two or more)
|
21 (4.9)
|
Ethnicity (n=427)
|
|
Hispanic
|
19 (4.4)
|
Education (n=430)
|
|
High School or Less
|
23 (5.3)
|
Some College
|
74 (17.2)
|
Associate's or Bachelor’s Degree
|
179 (41.5)
|
Master's Degree
|
102 (23.7)
|
Professional or Doctoral Degree
|
45 (10.4)
|
Other
|
7 (1.6)
|
Employment (n=430)
|
|
Full Time
|
122 (28.3)
|
Part Time
|
41 (9.5)
|
Unemployed
|
19 (4.4)
|
Unable to work due to illness/disability
|
97 (22.5)
|
Retired
|
134 (31.1)
|
Other
|
17 (3.9)
|
Finances (n=420)
|
|
Have enough or more than enough to make ends meet
|
331 (76.8)
|
Do not have enough to make ends meet
|
89 (20.6)
|
Step 4: Dimensionality & Recalibration
The 35 Self-Care Decisions Scale items fit best into a 6-factor multidimensional structure in exploratory factor analysis (RMSEA = 0.05, CFI = 0.96, TLI = 0.94, and SRMR = 0.04).
Based on primary item loadings (Table 3) we identified six types of contextual factors that influence self-care decisions about symptoms – there were no alternative factor loadings above a priori thresholds.
[Insert Table 3 about here]
Each represents a distinct and separately scored scale on the Self-Care Decisions Scale. Scales were labeled ‘external,’ ‘urgency,’ ‘uncertainty,’ ‘cognitive/affective,’ ‘waiting/cue competition,’ and ‘concealment’ based on the initial literature review and the content of the items that significantly loaded onto that scale (Table 4). Correlations between scales ranged from 0.35 (urgency and uncertainty) to 0.13 (urgency and concealment).
Table 4. Interpretations of the Six Scales of the Self-Care Decisions Scale
Scale
|
Interpretation
|
External
|
The extent to which input from other people influences self-care decision making. Higher scores indicate that self-care decision making is very influenced by the input of others.
|
Urgency
|
The extent to which the perception of urgency or high stakes influences the patient’s self-care decision making. Higher scores indicate that the patient’s self-care decision making is very influenced by the perception that making a decision about what to do about the symptom is urgent or important.
|
Uncertainty
|
The extent to which uncertainty or ambiguity, from incomplete information and/or difficulty interpreting the symptom, influence decision making. Higher scores indicate that the patient’s self-care decision making is very influenced by being unsure about the cause or meaning of the symptom.
|
Cognitive/Affective
|
The extent to which the patient’s thoughts or feelings influence decision making. Higher scores indicate that that patient’s thoughts and/or feelings interfere with or prevent decision making.
|
Waiting/Cue Competition
|
The extent to which situational factors delay decision making. Higher scores indicate that the patient is more likely to delay making a decision about their self-care because of competing priorities and/or a perception that the decision is not urgent.
|
Concealment
|
The extent to which a desire to hide the symptom from others influences decision making. Higher scores indicate that the patient’s self-care decision making is very influenced by a desire to conceal the symptom from others.
|
Scoring Note: Each scale is a separate standardized score that can range from 0-100.
Four items were associated with the scale we labeled ‘external.’ Although all items were significant discriminators between low and high levels of external factors driving decision-making (Table 5), item 20 “someone else recognized the symptom before I did,” had the lowest value for discrimination, and provided the least information about the influence of external factors (Figure 2). Further, based on category characteristic curves (Additional File 1), there had to be extremely high levels of the external influence (i.e. outside of the 95% confidence interval) for respondents to choose any response option above 1 (i.e. no influence). Therefore, item 20 was dropped from the ‘external’ scale.
Figure 2: Self-Care Decisions Scale Item Information Functions
Figure 2 Legend: Each pre-calibration item is show within the six scales of the Self-Care Decisions Scale. On the x-axis, theta represents the mean observed trait and the scale is standard errors around theta. On the y-axis, items providing more information about the trait with respect to greater discrimination have higher curves; items providing less information about the trait have lower curves, particularly those with a peak less than one.
Table 5: Scale-Specific Item Discrimination
|
External
|
Discrimination within Scale
|
Others gave me advice
|
2.238±0.284, p<0.001
|
Others helped me to make a decision
|
3.094±0.570, p<0.001
|
Different people gave different advice about my symptom
|
1.255±0.171, p<0.001
|
Someone else recognized the symptom before I did
|
1.134±0.180, p<0.001
|
Urgency
|
|
I thought about decisions I made in the past when I had a similar symptom
|
0.493±0.115, p<0.001
|
The symptom got worse suddenly
|
1.536±0.177, p<0.001
|
When I had this symptom, I knew something was wrong
|
1.434±0.168, p<0.001
|
The symptom was severe or bothersome
|
1.941±0.228, p<0.001
|
I felt like something bad was going to happen
|
1.700±0.199, p<0.001
|
I felt I needed to make a decision quickly
|
1.139±0.149, p<0.001
|
Uncertainty
|
|
The symptom was different than what I expected
|
1.492±0.150, p<0.001
|
It wasn't clear to me what was causing the symptom
|
1.574±0.159, p<0.001
|
I didn't know what the symptom meant
|
2.575±0.255, p<0.001
|
I thought the symptom might be due to something else
|
1.509±0.155, p<0.001
|
I wasn't sure how important the symptom was
|
1.621±0.161, p<0.001
|
When I had the symptom, I didn't understand what was happening
|
2.107±0.210, p<0.001
|
The symptom was new to me
|
2.505±0.278, p<0.001
|
I recognized this symptom from the last time I had it
|
-0.717±0.118, p<0.001
|
The symptom was different than the last time I had it
|
1.582±0.166, p<0.001
|
Cognitive/Affective
|
|
I felt too sad to make a decision
|
2.168±0.226, p<0.001
|
My thinking was not clear so I could not make a decision
|
2.352±0.230, p<0.001
|
I felt too anxious to make a decision
|
2.504±0.254, p<0.001
|
I didn't feel well enough to make a decision
|
3.840±0.445, p<0.001
|
I felt too tired to make a decision
|
2.513±0.240, p<0.001
|
I felt uncertain about what to do
|
1.713±0.164, p<0.001
|
Waiting/Cue Competition
|
|
I thought I could wait to make a decision
|
1.400±0.153, p<0.001
|
I felt that the symptom was nothing to worry about
|
1.297±0.153, p<0.001
|
The symptom changed slowly
|
0.695±0.120, p<0.001
|
I thought I could tolerate the symptom
|
1.971±0.212, p<0.001
|
Someone else needed my attention
|
1.120±0.155, p<0.001
|
I thought the symptom would go away on its own
|
2.105±0.231, p<0.001
|
Other things were more important at the time
|
1.458±0.171, p<0.001
|
Concealment
|
|
I felt embarrassed about my symptom
|
2.260±0.333, p<0.001
|
I didn't want to burden my family
|
1.810±0.225, p<0.001
|
I didn't want people to know about my symptom
|
2.112±0.296, p<0.001
|
Six items were associated with the scale we labeled ‘urgency,’ All items were significant discriminators between low and high levels of urgency (Table 5); but item 1, “I thought about decisions I made in the past when I had a similar symptom,” had the lowest value for discrimination and not all response options discriminated significantly. In addition, item 1 provided almost no information about the influence of urgency (Figure 2), and there was a very low threshold for higher probability of respondents choosing higher response options. Therefore, item 1 was dropped from the ‘urgency’ scale.
Nine items loaded on the scale we labeled ‘uncertainty.’ All items were significant discriminators between low and high levels of uncertainty (Table 5); however, there were redundancies with respect to item information, especially involving these items: item 3 “The symptom was different than what I expected,” and item 35 “The symptom was different than the last time I had it” (Figure 2). Additionally, item 33 “I recognized this symptom from the last time I had it” was the weakest discriminator and provided the least information about uncertainty. Items 3, 33 and 35 were omitted from the ‘uncertainty’ scale.
Six items were associated with the scale we labeled ‘cognitive/affective.’ All six items discriminated significantly (Table 5). However, for item 32 “I felt uncertain about what to do”, not all response options were significant discriminators (Additional File 1) and item 32 also provided the least information about the influence of the individual’s cognitive/affective state (Figure 2). Accordingly, item 32 was dropped from the ‘cognitive/affective’ scale.
Seven items loaded on the scale we labeled ‘waiting/cue competition.’ All items discriminated significantly between low and high levels of waiting/cue competition (Table 5). However, items 25 “The symptom changed slowly” and 29 “Someone else needed my attention” had the lowest values for discrimination and provided the least information about the ‘waiting/cue competition’ scale (Figure 2). Hence, items 25 and 29 were omitted from the ‘waiting/cue competition’ scale.
Finally, three items loaded on the scale we labeled ‘concealment.’ All three items were significant discriminators between low and high levels of concealment (Table 5) and all items provided sufficient information about concealment (Figure 2). Accordingly, all three items were retained in the ‘concealment’ scale.
Step 5: Construct Validity
Correlations between the six new Self-Care Decisions Scale scales and the four domains of the Melbourne DMQ were tested (Table 6).
Table 6: Convergent Validity Testing with Melbourne Decision-Making Questionnaire Domains
|
|
External
|
Urgency
|
Uncertainty
|
Cognitive/ Affective
|
Waiting/ Cue Competition
|
Concealment
|
Vigilance
|
-
|
-
|
-
|
-
|
-
|
-
|
Buck Passing
|
0.211
|
-
|
-
|
0.363
|
0.170
|
0.233
|
Procrastination
|
-
|
-
|
0.178
|
0.402
|
0.239
|
0.266
|
Hypervigilance
|
0.185
|
-
|
0.160
|
0.427
|
-
|
0.312
|
Values shown are significant (p<0.05) linear correlations with Bonferroni correction applied
The Self-Care Decisions Scale external scale was modestly associated with buck passing and hypervigilance. The Self-Care Decisions Scale uncertainty scale was modestly associated with procrastination and hypervigilance. The Self-Care Decisions Scale cognitive/affective scale was associated with buck passing, procrastination, and hypervigilance. The Self-Care Decisions Scale waiting/cue competition scale was associated modestly with buck passing and procrastination. The Self-Care Decisions Scale concealment scale was associated with buck passing, procrastination and hypervigilance. No scale on the Self-Care Decisions Scale was significantly associated with the Melbourne DMQ vigilance domain, and the Self-Care Decisions Scale urgency scale was not associated with any Melbourne DMQ domain.
We also evaluated differences in scale scores of the Self-Care Decisions Scale between individuals with adequate and inadequate self-care management (Table 7).
Table 7: Criterion Validity Testing Comparing the Self-Care Decisions Scale with Adequate versus Inadequate Self-Care Management
|
|
Adequate Self-Care Managementa
mean (sd)
|
Inadequate
Self-Care Managementa
mean (sd)
|
t-statistic
|
Effect Size
Hedge’s g
|
External
|
31.56 (26.89)
|
23.76 (22.35)
|
-2.97**
(p = 0.003)
|
0.32
|
Urgency
|
65.14 (20.54)
|
55.40 (24.02)
|
-4.34**
(p < 0.001)
|
0.42
|
Uncertainty
|
37.29 (24.16)
|
30.98 (25.24)
|
-2.49**
(p = 0.013)
|
0.25
|
Cognitive/Affective
|
25.54 (25.42)
|
26.22 (26.52)
|
0.25
(p = 0.8)
|
0.03
|
Waiting/Cue Competition
|
39.82 (23.01)
|
40.14 (23.23)
|
0.13
(p = 0.89)
|
0.01
|
Concealment
|
36.75 (31.39)
|
34.00 (28.53)
|
-0.88
(p = 0.38)
|
0.09
|
a Adequate self-care management is defined as a score ≥ 70 on the SC-CII Management Scale
** p-value < 0.05
There were significant differences in the scores on the external, urgency, and uncertainty scales. This partially supported our hypothesis that the scales of the Self-Care Decisions Scale would correlate with adequate self-care. Individuals with higher urgency had statistically significantly higher self-care management, as hypothesized. However, those with higher uncertainty also had higher self-care management scores.
Step 6: Precision and Reliability
Using IRT, test information function graphs along with plotted standard errors inform the range of underlying contextual factor where the scale is most precise; these data are provided in Figure 3. Using confirmatory factor analysis with recalibrated domains, multidimensional reliability (i.e., factor determinacy score) was high at 0.86.
Figure 3: Recalibrated Test Information Functions for each scale of the Self-Care Decisions Scale
Figure 3 Legend: Each post-calibration scale of the Self-Care Decisions Scale is presented regarding the degree to which the factor items collectively inform the trait (left y-axis - information), and range of underlying trait (x-axis with theta representing the mean observed trait and the scale is standard errors around theta) where the scale is most precise (right y-axis – standard error).
Scoring and Reference Ranges
Separate standardized scoring (fixed score range from 0-100) is recommended for the six scales of the Self-Care Decisions Scale. There is no total score. Mean ± standard deviation of standardized scores were external (26.30±24.28), urgency (58.57±25.38), uncertainty (33.03±25.04), cognitive/affective (26.00±26.04), waiting/cue competition (40.04±23.13), and concealment (34.89±29.48) in this derivation sample (Figure 4).
Figure 4: Standardized Scores on the Self-Care Decisions Scale
Figure 4 Legend: The mean and standard deviation of the standardized scores for each scale of the Self-Care Decisions Scale in the current sample are displayed.
Table 3: Self-Care Decisions Scale Item Geomin Loadings and Multidimensional Structure
|
Self-Care Decisions Scale Item
|
External
|
Urgency
|
Uncertainty
|
Cognitive/ Affective
|
Waiting/ Cue Competition
|
Concealment
|
- I thought about decisions I made in the past when I had a similar symptom
|
|
0.535
|
|
|
|
|
- Others gave me advice
|
0.839
|
|
|
|
|
|
- The symptom was different than what I expected
|
|
|
0.522
|
|
|
|
- It wasn't clear to me what was causing the symptom
|
|
|
0.555
|
|
|
|
- The symptom got worse suddenly
|
|
0.672
|
|
|
|
|
- I didn't know what the symptom meant
|
|
|
0.739
|
|
|
|
- When I had this symptom, I knew something was wrong
|
|
0.611
|
|
|
|
|
- I felt too sad to make a decision
|
|
|
|
0.716
|
|
|
- I thought the symptom might be due to something else
|
|
|
0.582
|
|
|
|
- Others helped me to make a decision
|
0.784
|
|
|
|
|
|
- I felt embarrassed about my symptom
|
|
|
|
|
|
0.510
|
- I didn't want to burden my family
|
|
|
|
|
|
0.463
|
- Different people gave different advice about my symptom
|
0.450
|
|
|
|
|
|
- I wasn't sure how important the symptom was
|
|
|
0.580
|
|
|
|
- My thinking was not clear so I could not make a decision
|
|
|
|
0.780
|
|
|
- I thought I could wait to make a decision
|
|
|
|
|
0.466
|
|
- When I had the symptom, I didn't understand what was happening
|
|
|
0.706
|
|
|
|
- The symptom was severe or bothersome
|
|
0.659
|
|
|
|
|
- I felt like something bad was going to happen
|
|
0.430
|
|
|
|
|
- Someone else recognized the symptom before I did
|
0.333
|
|
|
|
|
|
- I felt too anxious to make a decision
|
|
|
|
0.751
|
|
|
- The symptom was new to me
|
|
|
0.757
|
|
|
|
- I didn't want people to know about my symptom
|
|
|
|
|
|
.684
|
- I felt that the symptom was nothing to worry about
|
|
|
|
|
0.545
|
|
- The symptom changed slowly
|
|
|
|
|
0.225
|
|
- I didn't feel well enough to make a decision
|
|
|
|
0.920
|
|
|
- I thought I could tolerate the symptom
|
|
|
|
|
0.760
|
|
- I felt I needed to make a decision quickly
|
|
0.407
|
|
|
|
|
- Someone else needed my attention
|
|
|
|
|
0.387
|
|
- I thought the symptom would go away on its own
|
|
|
|
|
0.667
|
|
- I felt too tired to make a decision
|
|
|
|
0.808
|
|
|
- I felt uncertain about what to do
|
|
|
|
0.555
|
|
|
- I recognized this symptom from the last time I had it
|
|
|
-0.711
|
|
|
|
- Other things were more important at the time
|
|
|
|
|
0.506
|
|
- The symptom was different than the last time I had it
|
|
|
0.479
|
|
|
|