This study comprised four stages (Fig. 1). In Stages 1 and 2 (concept identification and elicitation, respectively), a conceptual framework for fever distress in children was developed based on scientific literature and feedback from both clinical experts and parents/caregivers of young children with fever. The most common behaviours and signs of fever distress were used to develop concept items for the draft ObsRO instrument (Stage 3). Parents of young children with fever were recruited for participation in cognitive interviews whereby they completed an electronic version of the draft instrument and gave feedback on the usability of the instrument and relevance of the items (Stage 4).
All interviews were conducted on behalf of the sponsor (Reckitt Benckiser, RB) by a third-party agency (Evidera) in the UK by staff trained in qualitative interview techniques. The study protocol and informed consent forms (ICFs) were approved by the UK Health Research Authority. All documentation prepared as part of this work was carried out in accordance with the United States (US) Food and Drug Administration (FDA) patient-reported outcome (PRO) guidance recommendations and standards . The study was conducted in accordance with the principles of the Declaration of Helsinki.
Stage 1: concept identification
Targeted research studies, clinical articles and guidance documents were reviewed to identify fever-related concepts and create a definition of ‘fever distress’. Signs of fever were conceptualised as those confirming the presence of fever (e.g., red cheeks). Fever distress was conceptualised as fever-related changes in behaviour (e.g., loss of appetite). The initial conceptual framework for fever distress included behaviours (restlessness/agitation, apathy/limpness and loss of appetite) and signs (red cheeks and glazed eyes). A conceptual disease model (CDM) was developed to understand fever distress. The authors interviewed three expert clinicians, two from the UK and one from the US, for feedback regarding the above concepts and the CDM.
Stage 2: concept elicitation
Participants were screened for eligibility using ethics committee-approved screening forms. Eligible participants were parents/caregivers of children aged ≥ 3 months to < 6 years who had recently resolved fever (within the previous 2 weeks). Participants had a good command of English, gave informed consent and were willing to be audiorecorded during the interview. Children of the participants were grouped according to age: 1) ≥ 3 months to 1 year; 2) 1 to < 2 years; 3) 2 to < 4 years; 4) 4 to < 6 years.
Telephone interviews were scheduled with eligible participants and an interview packet containing an introductory letter, two copies of the ICF and a sociodemographic form was delivered to the participant by courier.
All participants gave confirmation of written informed consent before the elicitation interview began. Each interview (including time for consent, interview and completion of case report forms) lasted approximately 60 minutes and was audio-recorded. The interviewer discussed parental understanding of fever, recent experience with fever, fever-related signs and behaviour, caregiver fever management, caregiver actions and impact. Participants completed the sociodemographic questionnaire after the interview and returned this, along with the two signed copies of ICFs, to study personnel. A fully-signed ICF was returned to participants for their personal records.
Stage 3: instrument development
Fever signs and behaviours endorsed by ≥ 20% of participants elicited during interviews were selected for item development. These were further evaluated against item writing criteria adapted from the International Society for Pharmacoeconomic and Outcomes Research – Patient Reported Outcome (ISPOR PRO) Good Practice Task Force, Part II  (Supplementary Table 1).
Draft ObsRO instrument
Development of the ObsRO instrument, including creation of questions, recall period, and response choices, was guided by best practice guidance outlined in the ISPOR PRO Good Practice Task Force, Part I and Part II [13, 14]. The language of participants during elicitation interviews (i.e., their descriptions of fever signs and behaviours) guided the wording of the instructions for use, item stems and response options. Potential recall periods and response options were evaluated for best fit across sign and behaviour items. The recall period was determined by the time interval between the child exhibiting the sign/behaviour and it being described by parents and clinical experts (i.e., several days, hours, etc.) and the intended use of the measure in future trials where daily fever assessment was required.
Stage 4: cognitive interview
Cognitive interviews were conducted to test the content validity of the draft ObsRO. Parents of children with recently resolved fever were recruited using the same process as the elicitation interviews. Eligible participants were sent an ICF, sociodemographic form, the ObsRO and an importance questionnaire electronically at the time of the interview.
Cognitive interviews were conducted in person, via webcam or by telephone; all sessions were audio-recorded. Interviews were performed in two rounds; in Round 1, participants completed Version 1 of the draft ObsRO instrument whereas in Round 2, participants completed Version 2. The interviewer sought feedback from participants on their understanding and comprehension of the items in the ObsRO, the timeframe and response options. Participants were also asked about usability.
Qualitative data analysis
Interview audio files were transcribed and transcripts were cleaned, de-identified and evaluated using ATLAS.ti qualitative software version 8.0 .
For elicitation data, a coding framework was developed to organize the text data for the major discussion topics and concepts that emerged during discussions. For cognitive interviews, feedback on instruction, response options and item meaning were coded, as well as emerging issues such as missing or confusing content. Team members were trained to use a codebook that coded words and phrases from interview participants. Two and three team members independently coded elicitation and cognitive interview transcripts, respectively, after which all codes were compared, discussed, and reconciled wherever differences occurred. A random sample of approximately 20% of coded transcripts was reviewed. If problems were identified, additional transcripts or codes were reviewed. Key concepts that emerged from coding of elicitation data were put into a concept frequency table that showed how many participants endorsed a particular concept. Concepts were also plotted on a saturation grid. Concept saturation was reached at a point when no new concepts continued to emerge.
An item tracking matrix was developed to capture participants’ comprehension and relevance of each item on the ObsRO. The matrix also tracked changes made to each item during ObsRO development, providing supportive evidence and rationale for each change.
Quantitative data analysis
Tables summarizing quantitative data (means, standard deviations, and range) were presented for continuous variables and frequency and percentage for categorical variables. Quantitative information was used to characterize the study sample and document representation of the target population in accordance with regulatory guidance. In addition, descriptive statistics were calculated for the ObsRO scores.