Figure 2 shows the five themes of rheumatological disease distress formed from the secondary data analysis. Appendix 1 shows examples of relevant accounts from original transcripts illustrating identified themes of distress.
Overall, participants in the PPI group agreed that the themes of distress resonated with their experiences of living with RA (see Appendix 2).
Initial item generation yielded 94 items, categorised in order of the five themes of distress. After removal of repetitive statements, 44 items formed the first draft of the RADS. Items remained grouped in the themes of distress, but those relating to emotional distress were placed before items of physical distress to reflect the journey of patients with RA, following diagnosis, acceptance and impact of the disease. No other item order changes were made, informed by a literature review that indicated that there was no consensus for ordering of items (27, 28).
Appendix 3 shows examples of relevant accounts from Phase 4 PPI group. Participants unanimously reported that the themes and concept of DSD in RA resonated with their experiences of living with the disease. An additional theme of ‘Work-related distress’ was identified, added as a sub-theme of Social Distress (See Figure. 2). A corresponding new item to this was suggested, as was a new item regarding infections.
PPI group feedback was to use word-based scoring and reduce the recall period in the PROM instructions from two months to two weeks, as a shorter duration was seen as the most reasonable amount of time one could remember accurately for scoring. Three additional supplementary questions based on time since diagnosis and disease activity were suggested, considered as contextually important for interpreting the distress score.
After the PPI group, a total of two new items had been created, 11 had been removed, and 23 had been re-phrased.
By the end of Phase 4, three versions of the RADS existed, each with the same 35 items, but differed based on word or number scoring, and the presence of supplementary questions.
Thirteen members of a national RA charity members expressed interest with nine participating. Four could not take part due to scheduling conflicts.
Phase 5 interview data that was uploaded onto NVivo 12 yielded the results presented below. See Appendix 4 for relevant accounts from interview transcripts
(i) The Concept of DSD in RA
Five participants clearly validated DSD as an entity in RA and three inferred it was a valid concept when discussing certain items, for example in relation to pain.
One participant did not seem to have experienced DSD but verbalised she could see how the items were relevant and distressing for others living with RA.
(ii) Item Relevancy and Rephrasing
Participants (n = 9) reported that most of the items were relevant to their personal experiences living with RA or would be to others living with RA. Participants suggested minor rephrasing of 19 items.
(iii) Item Creation
New items were suggested in relation to the theme of treatment-related, physical-related, and healthcare-related distress (n = 5).
(iv) Item Removal or Combination
Five participants reported that two items regarding RA pain were similar and could be combined, and that the word ‘distress’ better explained feelings towards their pain than ‘anger’. Two disagreed.
Seven participants reported that two items concerning test results and healthcare professionals’ explanations of treatments were similar and could be combined.
(v) Item Order
Participants had few strong opinions on the order of items on the PROM and stated that the current sequence made sense. However, six suggested that item ‘I feel overwhelmed living with RA’ could move to the end of the scale.
(vi) Supplementary Questions
Nine, seven and six participants preferred to keep supplementary questions 1, 2 and 3 in the scale, respectively. Participants reported that the questions gave context to scoring the PROM.
(vii) PROM Scoring
Six participants reported preferring to answer the PROM using word scoring, two preferred numbers, and one held no preference.
Those who preferred word labels reported being able to ‘relate’ more to words and that it helped them consider each item more carefully than if scoring on a numerical scale.
(viii) PROM Instructions
Overall (n = 9), participants reported that they understood PROM instructions and they were clear. Regarding the timeframe mentioned in the instructions, they reported that due to the variability of their condition, two weeks may not accurately capture distress levels.
Some participants (n = 3) suggested to extend the timeframe to one month, some (n = 1) to three months, some (n = 2) were unsure, while others (n = 3) agreed with two weeks.
Although demographic data was not formally collected for Phases 2, 4 and 5 participants, some information unintentionally was disclosed; Most participants were female, Caucasian, and revealed their occupations were in the banking, business, or healthcare-sectors. Disease duration varied from two to 40 years in cognitive interview participants.
The psychometrician validated the study design and confirmed word-based scoring to be appropriate for a PROM so long as presented in ordinal format for summation of scoring. The psychometrician also confirmed the supplementary questions as relevant and suggested these to be inserted at the end of the PROM to avoid potentially influencing participants scoring. Finally, the psychometrician suggested consideration of the time recall period in the instructions to ensure consistency and validity of participant answers.
Final Changes to the RADS
Final changes were made to the RADS based on the results and analysis from Phase 5 data presented above (See RADS Final Version in Appendix 5).
A preliminary decision was made for instructions to score based on the previous three months rather than two weeks, ensuring enough timeframe to capture different aspects of distress. Word-based scoring was chosen as it was the predominant preference from Phase 4 and 5 participants. No major differences were noted from gross observation of scoring patterns when alternating the different versions in Phase 5.
In total, 19 items on the PROM were rephrased, two were removed, and six new items were created. The three supplementary questions were retained in response to the participants’ preference, and the psychometrician’s advice.
For item order, new items were inserted into the PROM alongside those from their corresponding theme. Item ‘I feel overwhelmed living with RA’ was moved to the end to become item 39 on the RADS as per the participants’ (n = 6) suggestions. Therefore, after analysis of the data in Phase 5 of the RADS development, the final version consisted of a 39-item scale with three supplementary questions (See Appendix 5).