Patient Experience with Treatment of Relapsed or Refractory Multiple Myeloma (RRMM): Content Validity of the Satisfaction and Experience Questionnaire (SEQ)-MM

Background: Patients with multiple myeloma (MM) generally progress to being relapse/refractory (RR), having received several lines of treatment with various dosing frequencies and administration modalities, which impact patient satisfaction and experience. While there is still no cure, patients live longer but with potential burden attributable to treatment. A t-for-purpose patient-reported outcome measure (PROM) assessing patient satisfaction and experience is missing for RRMM. Methods: We developed the Satisfaction and Experience Questionnaire (SEQ) for MM to measure patient satisfaction and experience, based on concepts identied from a literature review and nurse/patient perceptions about patient experiences. Online focus groups were conducted with oncology nurses who treat patients with MM to provide expert opinion, followed by concept elicitation and cognitive patient interviews. A preliminary SEQ-MM measure was rst drafted based on desk research. Patient feedback from the cognitive interviews resulted in revisions to the preliminary SEQ-MM’s instructions, item wording, and response options between rounds. Content analysis was conducted to assess concept frequency mentioned during the concept elicitation focus group, and an item tracking table documented the SEQ-MM revisions based on cognitive interviews. Results: Three oncology nurses and 10 patients with RRMM participated in the focus groups. Oncology nurses reported treating 17 patients with MM per week, of which 33% were patients with RRMM. Study patients were primarily male (70%), with mean age of 56 (median range: 47 to 71). The most commonly reported satisfaction and experience concepts were related to: satisfaction with treatment (50%), experience with treatment (50%), relationship with healthcare practitioners (50%), time burden (50%), side effects (40%), and avoiding relapse (40%). The SEQ-MM was revised to include 15 items capturing: treatment burden (ve items), time burden (ve items), travel burden (two items), treatment compliance (one item), general satisfaction (one item), and an overall satisfaction item (one item). Conclusions: The SEQ-MM v1.0 evaluates the important facets of satisfaction and experience in patients with RRMM receiving treatment. This tool can be used to assess facets of treatment in MM that might complement health-related


Background
Multiple myeloma (MM) is an incurable, heterogenous, malignant plasma cell disorder. Most MM cases experience patterns of regression and remission, followed by relapse. In the past few decades, there have been advancements and increases in MM treatment options-speci cally, providing treatment options for patients with relapse or refractory MM (RRMM) [1,2]. The availability of multiple treatment options enables personalized treatment, especially for patients with RRMM aiming to eliminate or reduce the considerable unmet need (since the ve-year survival rates are approximately 50%) [3]. Although patients can nd a variety of different treatments prolonging their survival, there remains a lack of measurement of patient satisfaction and experience.
Patient satisfaction and experience with a medication or medical device has been of increasing interest over the past two decades [4,5], particularly with more offerings from a wide spectrum of RRMM treatments that could potentially better t patient lifestyles, reduce travel burden due to improved scheduling frequency, and provide safer and more e cacious dosing regimens [6]. The FDA's 21 st Century Cures Act and plan for Patient Focused Drug Development includes steps to gathering patient perspectives on burden of treatment [7][8][9].
Assessing current medication satisfaction gives useful insights into the patient perspective, and may aid in differentiation among alternative treatments [10]. Patient satisfaction with treatment is de ned as the individual's rating of important attributes regarding the process and outcomes of their current treatment, and focuses on the interaction of expectations, preferences, and satisfaction with medical treatment [10][11][12]. Enhancing patient satisfaction and experience with treatment modalities such as duration and dosing can improve their health outcomes and quality of life (QoL). Given the evolving changes to cancer treatments and their innovative delivery methods, understanding patients' experience and gauging their satisfaction with their cancer care is important-but di cult-to assess with existing measures which were not MM-speci c and did not capture issues related to treatment modalities such as infusion, duration of administration, and perceptions around e cacy and tolerability in MM. However, there remains a lack of evidence-based data assessing the impact of these therapies on health related QoL (HRQoL) as it pertains to patient satisfaction and experience. Therefore, this knowledge gap must be addressed to better understand treatment burden within this population.
Recent studies suggest that productivity loss, time, caregiver burden, symptoms, direct costs, and treatment costs could be major drivers of patient satisfaction and experience, largely due to treatment modalities (oral vs. injectable regimen) [13,14]. Appropriate assessment tools to assess factors in uencing patient satisfaction and experience could be invaluable in clinical decision-making as it pertains to patient adherence and could in uence better patient outcomes. The implementation of treatment satisfaction assessment tools can identify issues related to treatment dissatisfaction in the complex landscape of RRMM treatments-and as a result, lead to improvements in the treatment experience of patients with RRMM [13].
Health-related quality of life (HRQoL) measures alone may not capture all relevant information about the patients' treatment experience and the treatment-related concepts they deem important, such as treatment satisfaction and convenience [6, 15,16]. Therefore, a validated patient-reported outcome (PRO) measure for evaluating patient satisfaction and convenience is needed, especially in the current environment enwith multiple treatment options and modalities in RRMM.
While medication satisfaction may be broadly applicable, the current study focused speci cally on cancer care for patients receiving RRMM treatments; in parallel, patients from other cancer types on prophylactic treatment for febrile neutropenia were also a focus, but are reported separately [17]. The intent of this study was to develop a tool to support decision-making related to patient satisfaction and experience; and understand what additional patient information may aid in differentiation among alternative RRMM treatments.

Overview
The SEQ-MM was developed during three stages ( Figure 1). During Stage 1, a targeted literature review (TLR) was conducted to examine existing PRO measures related to patient experience and treatment satisfaction. Theoretical models of treatment satisfaction were also reviewed [5,10,18,19], core concepts of interest (COIs) were identi ed, and a preliminary conceptual framework (CF) was drafted. The CF included the COIs guiding the development of a new measure called the Satisfaction and Experience Questionnaire-MM (SEQ-MM).
Following the drafting of the preliminary SEQ-MM v0.1, a cross-sectional observational qualitative study was conducted in two steps: 1) focus groups with oncology nurses to explore patient satisfaction and experience; and 2) focus groups and interviews with RRMM patients to rst understand patient satisfaction and experience, followed by a group cognitive interview review of the draft SEQ-MM v0.1 and v0.2 to assess patient comprehension and feedback.

Stage 1: Targeted Literature Review of Existing Treatment Satisfaction Measures
During Stage 1, a TLR was conducted in two steps: 1) identify published literature reporting PRO measures that assess patient satisfaction and experience in oncology; 2) review PRO measures of patient satisfaction and experience in other indication areas beyond oncology. The search was conducted in September 2017 utilizing PubMed/MEDLINE, Embase, Ovid, and Google Scholar databases focused on English-language articles, involving only human subjects and publications between 2007 to 2017. The key terms used were "Patient satisfaction", "Patient experience", "treatment satisfaction", "treatment experience" and "convenience"." Further information about the literature search terms can be found in the additional les (Additional le 1).
Eligible publications focused on patient-reported satisfaction and experience (including convenience) domains among patients with RRMM. Publications were ineligible if they involved pediatric patients, were not reported in English, had inaccessible full texts, did not report the use of validated PROs (or PROs were not validated among patients with cancer or hematologic malignancies), used proxy-reported outcomes, or measured other types of satisfaction (e.g., sexual satisfaction, survivor's satisfaction with healthcare delivery and accessibility, satisfaction with educational intervention, satisfaction on surgical procedure, etc.). Eligible articles were assessed for: 1) whether a treatment experience, convenience or satisfaction PRO was reported; 2) the PRO was relevant to the oncology population; and 3) methodological rigor and quality.
In parallel, an additional search was conducted using the same search terms on Patient-Reported Outcomes and Quality of Life Instruments Database (PROQOLID) and Functional Assessment of Chronic Illness Therapy (FACIT), which are comprehensive online databases of clinical outcome assessments (COAs) to identify generic and disease-speci c PRO measures. PROs that assessed satisfaction and experience domains were eligible to be included in the search. In addition, existing health models of behavior change related to medication and treatment satisfaction and experience were reviewed. This further informed the COIs to be included in the CF for patient satisfaction and experience related to treatments in RRMM.

Assessment of Existing PRO Measures
Following the literature review, information from each article was extracted, and each instrument was assessed whether it was suitable for use with patients with RRMM, captured COIs, and was relevant for the context of use. An instrument was ineligible if it did not assess the COIs (patient experience and satisfaction domains such as convenience of dosing schedule, and mode of medication administration), was not validated in any population, or was not validated in patients with MM. Please review Results section that reports on brief gap analysis results of existing PRO measures.

Item Development
COIs were identi ed based on the assessment of the identi ed PROs, literature review results, and review of existing health models [5,10,18,19]; and a pool of potential items was developed to capture patient experience and satisfaction domains; tolerability, convenience of dosing schedule, travel, treatment, bothersomeness of treatment, and satisfaction with treatment. A recall period of "most recent MM treatment" and ve-point verbal rating scales (VRS) was developed. The draft items were crafted based on generally-accepted procedures outlined by Streiner and Norman [20], the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Best Practices, and the Food and Drug Administration (FDA) PRO Guidance [21][22][23] (Additional le 2). A Flesch-Kincaid readability assessment was conducted on the instructions and items, and revisions were made.

Stage 2: Focus Group with Registered Oncology Nurses
Registered nurses (RNs) with ve or more years of oncology treatment experience in the United States (US) were recruited through a medical provider panel (www.nielsen.com) to take part in a single, 60minute, online concept elicitation focus group platform (www.focusvision.com). Participants were encouraged to use the video camera feature for easy identi cation and better interaction. A semistructured, standardized discussion guide was used to obtain RNs' assessments about patient satisfaction and experience with chemotherapy, targeted therapies, prophylactic therapies, and mode of medication administration (Additional le 3). The RNs also answered several closed-and open-ended questions, using FocusVision's online polling software, centered on patient satisfaction and experience, and tolerability based on their clinical practice experience.
RN input into patient satisfaction and experience with oncology treatments provided insight into the relevance and comprehensiveness of the concepts, and informed any necessary modi cations to the preliminary SEQ-MM v0.1. The focus group was audio recorded (with permission) and subsequently transcribed. RNs were compensated $260 for one hour of their time.

Stage 3: Focus Groups and Interviews with RRMM Patients
Adult patients who were US resident with a diagnosis of RRMM within the past 12 months and on current or recent (within 12 months of self-reported diagnosis date) RRMM treatment were eligible to participate in this study. Participants were required to provide physician veri cation of their RRMM diagnosis and treatment. These patients participated in hybrid concept elicitation and cognitive interview discussions via an online platform.

Ethics and Recruitment
This study was approved by the Quorum Review Institutional Review Board (protocol number 33649). Identi cation, screening, recruitment, and tracking of the patients with RRMM were managed by a thirdparty recruitment vendor, Nielsen. To participate in the vendor's panel, patients were required to submit a physician's veri cation of cancer diagnosis and chemotherapy medication. Eligible participants were scheduled and invited to join the discussion by the focus group vendor. Focus group participants were sent an invitation to join an online platform utilizing webcams; these interactions were audio recorded. The focus group vendor also sent all study documents to the participants, who were encouraged to use a webcam for better interaction.

Discussion Guide
A semi-structured, standardized discussion guide was used in the patient focus groups (Additional le 4). The focus group discussion guide consisted of two sections: 1) open-ended concept elicitation questions to understand patient experience with RRMM treatment and satisfaction; and 2) a cognitive interview to assess patient impressions of the relevance, comprehension, and speci c feedback on the instructions, recall period, items, and response options of the SEQ-MM. Before the initiation of the cognitive interview discussion, focus group participants completed the SEQ-MM.
A Microsoft PowerPoint presentation based on the patient semi-structured discussion guide was developed for use during the online focus groups and telephone interviews. The PowerPoint presentation allowed for discussion of treatment satisfaction and experience topics. The presentation also allowed for a review of the SEQ-MM v1.0 items and response options, alongside alternative items and response options arising from previous focus groups or interviews for patient review and consideration.

Other Questionnaires
After the patient focus group discussions, the following forms were completed by each participant: 1) the SEQ-MM Item Relevance Questionnaire, which obtained the relevance ratings for each item response options rated on a ve-point VRS (not important, slightly important, moderately important, very important, extremely important); 2) a sociodemographic form, which included questions on age, race, employment, and education; 3) a medical history form, which included questions on disease and treatment history; and 4) a treatment characteristics questionnaire, which ascertained a general rating of treatment satisfaction, the time burden, and indirect costs of traveling to receive treatment. At the end of the focus group, selfreported demographic data was collected from participants, and the focus group vendor disbursed compensation of $175 to study participants.
In the cognitive interview portion of the focus group, patients reviewed the instructions, item content, recall period, and response options. The item intent, meaning, and relevance was assessed for the SEQ-MM v0.2 items. All transcripts were developed from the audio les.

Data Analysis
Descriptive statistics (e.g., n, mean, standard deviation [SD], and/or frequency) for quantitative sociodemographic and medical history data were calculated and summarized in results tables to characterize the sample. SAS v9.4 was used for the analyses of the descriptive statistics [24].
Separate preliminary coding frameworks were developed for Stages 2 and 3 concept elicitation discussions related to the meaning of treatment experience and satisfaction. The coding frameworks were imported into ATLAS.ti v8 software to code transcripts [25]. Structured counts were taken during each focus group for each treatment experience and satisfaction concept mentioned. The concepts arising from the patient concept elicitation discussion were documented and evaluated for saturation. Saturation was de ned as the point at which no substantially new themes, concept descriptions, or terms were introduced as additional discussions are conducted [22,23,26].
To document saturation, the rst mention of each concept was examined, and concepts were evaluated to see which were mentioned early in the concept elicitation interview (or if they arose in later discussion). Saturation was documented using a saturation grid, and displayed the rst instance of emerging concepts in interviews/focus groups. A thematic analysis was conducted for the concept elicitation section of the discussion.
For the Stage 3 cognitive interviews, an item tracking matrix was utilized to track feedback and recommendations for changes to each item/question. Also, a coding framework for the cognitive interviews was developed. ATLAS.ti was used to document speci c discussions related to the SEQ-MM and suggestions for changes, resulting in the nal SEQ-MM v1.0. A content analysis approach was used to analyze and interpret the coded cognitive interview data.

Targeted Literature Review
A schematic diagram reporting the results of the TLR is outlined in Figure 2. A total of 972 published abstracts on oncology treatment include PROs screened for patient experience and satisfaction concepts of patient expectation, convenience, satisfaction with medication, and adherence and tolerability to therapy. After removing duplicates, publications with only abstracts available, reviews, cases studies, or clinical guidance articles, 406 full-text articles were reviewed and assessed for eligibility. Of these, only 23 articles were deemed eligible, as they included oncology trials assessing medication interventions, PROs measureing treatment or medication convenience, tolerability, or satisfaction.
Following the literature review, a brief gap analysis was conducted on the 28 satisfaction-related questionnaires identi ed from the literature and key informationInstrument concepts were reviewed and mapped to concepts of interest and the speci c context of use in cancer trials: convenience of medication dosing schedule, medication satisfaction, and satisfaction with mode of treatment administration. Of these 28 measures, only four were speci cally related to cancer/chronic illnesses, and encompassed patient satisfaction with oncology treatment: Cancer Therapy Satisfaction Questionnaire (CTSQ) [27], Treatment Satisfaction Questionnaire for Medication (TSQM) [28], FACIT-Treatment Satisfaction-General (FACIT-TS-G), and Treatment Satisfaction with Medicines Questionnaire (SATMED-Q) [29].
Based on the review of the four identi ed PROs with potential concepts of interest, none were deemed suitable to assess the treatment and medication experience and satisfaction of patients with RRMM. Instead these tools included multiple domains, including satisfaction with healthcare procedures, quality metrics, information perception by patients, healthcare provider (HCP) appraisal, and provider ability to provide adequate care. Additionally, existing measures did not look at the indirect difference between once-weekly versus twice-weekly administration of an infusion-administered oncology treatment. Infusion-administered oncology treatments require patients to come back to the clinic, and would require customized step-up dose escalation, which may be forgotten or omitted.
Based on this nding, the study team decided to create a new instrument speci cally for use with RRMM therapies [17] in order to more accurately capture experience, convenience, and satisfaction with the current once-weekly administration of RRMM treatment.

Development of Preliminary Conceptual Framework
An item pool of relevant items from the four identi ed PROs was created for potential adaptation and inclusion in the new measure. Items were revised to capture treatment experience, satisfaction, and convenience; and a preliminary CF was developed to include the RRMM treatment COIs of patient experience. Patients' experience and satisfaction with treatment e cacy and safety were posited to be directly related to the experience of improving or worsening symptoms, side effects, patient adherence to treatment, and the overall burden on the patient imposed by factors such as the convenience of schedule, duration, timing of dosing, and treatment cost.

Results from Nurse Focus Group
Five RNs were recruited to participate in an online focus group. Two RNs were unable to participate at the last minute. The three participating RNs reported treating an average of 22 patients with MM per week, of which 33% were patients with RRMM. Oncology nurses described patients' satisfaction and tolerability with RRMM treatment based on their clinical practice experience. Primarily, nurses reported that patients with RRMM derived satisfaction from experiencing a treatment response or indication that the treatment was working, and from good communication with their cancer care team. The nurses stated that patients were willing to tolerate treatment side effects if they saw a treatment response or if their QoL was not signi cantly impacted.

Concept Elicitation Results
Three rounds of hybrid concept elicitation and cognitive interview focus groups were conducted to support attainment of concept saturation.

Page 10/26
The detailed demographics of the patient sample are reported in Table 1a. Ten RRMM study participants participated in three focus groups and one individual interview. Three rounds of focus group discussions and one interview were conducted. Five participants were in each round (Round 1: two focus groups; Round 2: one focus group and one interview). Participants were primarily male (70%), with mean ages of 56±8 years. Responses not mutually exclusive Half of participants reported receiving their cancer diagnosis over three years prior to screening (n=5, 50%). Most RRMM treatments were received subcutaneously (n=7, 70%), and the majority reported receiving lenalidomide (n=8, 80%), dexamethasone (n=7, 70%), or bortezomib (n=6, 60%). Most participants were insured and paid minimal out-of-pocket costs (n=7, 70%). Detailed medical information is reported in Table 1b.

Experiences with Treatment
Generally, participants relayed positive experiences with treatment-knowing their treatment is working (n=8, 80%), not experiencing side effects (n=7, 70%), having a good relationship with their HCP (n=6, 60%), and having a good quality of life (n=3, 30%). Individual participants (10%) described positive treatment experiences as receiving treatment close to home and from prominent HCPs or medical facilities ( Table 2). Participants commented on how a few RRMM treatment experiences can have both positive and negative aspects. Participants reported negative experiences from not seeing a treatment response (n=8, 80%), experiencing side effects (n=7, 70%), or experiencing disease progression (n=3, 30%). Participants also viewed the time burden of treatment, including travel time (n=5, 50%), as a negative experience.

Satisfaction with Care
The majority of participants (90%) rated their cancer care team as seven or higher on an 11-point NRS (worst to best cancer care team). Most participants (70%) felt they received bene t from the treatment, with a few side effects (60%), many side effects (20%), or no side effects (20%). All participants stated that they were able to tolerate the treatment despite the side effects. All participants reported that their cancer care team clearly explained their treatment in person (90%), or in addition, provided written materials (30%).

Time Burden
The majority (70%) reported that o ce/clinic visits were scheduled at times that were convenient for them; 30% said that times were sometimes convenient for them. The instrument development process (including creation of questions, recall period, and response choices) followed published guidelines and procedures [21][22][23]. The study results demonstrate that participants with RRMM were able to understand the SEQ-MM v1.0 instructions, items, and response options. Feedback from 10 patients con rmed the content validity of the item concepts as relevant to their treatment experience and satisfaction. Additionally, cognitive interviews provided supportive evidence of patient satisfaction with treatment bene t; minimal side effects as important to SEQ-MM v1.0 concepts was supported by the additional questionnaire results. Participants reported lengthy oneway travel times and treatment wait times, and many were accompanied by another person predominantly at times convenient to them. Participants all described out-of-pocket indirect costs for transportation, and a majority reported signi cant out-of-pocket treatment costs.

Limitations
The content validity assessment was carried out with a sample of 10 adults with RRMM, which is an adequate sample size for reaching saturation for a relatively uncommon cancer. The study participants reviewing the SEQ-MM included patients with RRMM; these results are not generalizable to other tumor types and diseases. Study participants reported having received several MM treatments, with distinct modes of medication delivery.
Scheduling of focus groups sessions, punctuality, and delays can affect the onset and attendance of focus group sessions. In this study, only three of the ve RNs were able to attend the session. However, online, video-enabled focus groups allowed for easy rapport-building, screen-sharing of visual aids, and participation of diverse patients that may not have participated in the in-person focus groups. The number of participants in each focus group session differed, resulting in fewer topics/questions being covered in focus groups with more participants. While individual in-person cognitive interviews may be ideally suited for detailed cognitive interview feedback, the quality of patient input can still be achieved using focus group design and using online methods. The study only included two rounds of cognitive interview discussions. A third round of cognitive interviews was not possible; however, the feedback for changes in the second round of focus groups and the individual interview was supportive of the nal SEQ-MM revisions.
In the future, the SEQ-MM will need to undergo continued patient testing to assess its generalizability to newly diagnosed MM and other tumor types in relation to their treatment experience and treatment satisfaction. This way, it can be a useful tool in comparative effectiveness or treatment evaluation trials.

Conclusion
This study reports on the development and content validity of the patient-reported SEQ-MM v1.0, which provides insights into the patient's experience of burden of current oncology treatment, as well as satisfaction with that experience. The tool was initially developed based on concepts identi ed from a literature review of oncology measures, interviews with oncology nurses and later revised and tested in a sample of and RRMM patients undergoing treatment. The instrument captures the core aspects of patient cancer care experience, and related overall satisfaction with that experience among patients with RRMM and may complement data gathered from traditional HRQoL measures. This study was reviewed and approved by Quorum Review Institutional Review Board (approval number 33649). All participants signed an informed consent form before any study-related focus group discussions or interview were conducted.