We report preliminary results developing and pilot testing a preference survey assessing factors influencing ASM treatment decisions in patients with well-controlled epilepsy. This pilot study was performed to optimize content and ensure comprehensibility prior to future full-scale data collection. Respondents generally agreed that the survey was understandable and complete, and they described areas for development prior to future full-scale data collection clarifying instructions and content.
One of the most interesting results of our work that will inform future questionnaire development was the presence of ‘inconsistent’ response patterns. While only 3% of BWS blocks had an inconsistent response, we still found that 39% of patients made at least one inconsistent response, and the most common pattern was to make an inconsistent response on BWS questions only. Inconsistencies particularly involved seizure probability and driving duration questions. For example, 7 patients chose a 6-month driving restriction as the least concerning and did not mark a 3-month driving restriction as either least or most concerning. This could have suggested they simply did not see the 3-month option (otherwise they would have logically marked that choice as the least concerning), or else did not understand the instructions. Seizure probabilities were the other main source of inconsistent responses, for example ranking a higher seizure risk as lower concern or vice versa. Patients with epilepsy have lower numeracy (the ability to comprehend and use numbers) than the general population.44 This population is at high risk for cognitive impairment due to seizures, ASMs, and the underlying seizure etiology,45,46 any of which could reduce numeracy. We initially attempted to mitigate this possibility by displaying seizure risk in terms of pictographs (showing 100 person icons, with the number of red person icons representing the number of people out of 100 that would have a seizure under a given percentage) informed by optimal risk communication literature.47 However, both physicians and patients found these charts cluttering and more confusing that displaying percentages only. In the future, we may consider showing fractions (“1 in 10”) instead of percentages (“10%” also informed by risk communication literature47 which could be clearer, and also consider only listing a single level of driving restrictions relevant to the patient’s US state to reduce as much confusion as possible. One could have hypothesized that errors would increase by the end of the survey due to respondent fatigue, but all BWS errors occurred in the first half of the BWS questions suggesting against this. While we did include a static ‘example’ BWS response to show respondents what a correctly filled out item would look like, in the future we may include an interactive BWS question requiring that respondents fill out an example question correctly before proceeding. Other strategies may be either collapsing ‘having a seizure’ into a single item without distinguishing between different risk percentages which may have been confusing to patients, or else to force the survey design to allow only a single seizure risk level per block, i.e., displaying an ‘error message’ warning the respondent that they may wish to revise their choice. We intentionally allowed more than one seizure risk level per block though in this pilot study to evaluate the degree to which patients were understanding the task, and because our a priori goal was to evaluate what seizure risk patients believe outweighs each downside of treatment. Interestingly, inconsistencies were not seen with cost items – it is plausible that people have a much more intuitive sense of dollars compared with risk percentages, thus our future work may modify driving duration or seizure risk items but there is no need to modify the cost items.
It was encouraging that results tended to align between techniques - cost, lab monitoring, and the inconvenience of pill-taking were rated as the lowest concerns, whereas cognitive side effects and the highest tier risk of seizure were rated as the highest concerns, for both VAS and BWS questions. This supports convergent validity between techniques. Our preliminary results also suggested that numerous items had wide variation, particularly for the importance of driving restrictions, and greater variation appeared to exist for VAS than BWS questions. This argues for collecting baseline data from future respondents that may influence the importance of driving, such as whether patients work, have dependents, availability of public transportation, and cognitive ability, to further individualize results across clinically important subgroups.
Patient preference studies have immediate clinical implications, because understanding what range of seizure risk outweighs which side effects and ASM-related inconveniences provides key information towards understanding which patients may benefit from continued treatment. Thus, guideline developers and policy makers need evidence on patient preferences to optimally inform patient-centered care.16 Even if in our small sample of physicians appeared qualitatively similar to patient responses, literature supports that physicians frequently misestimate patient preferences.48–52 Prior work in larger samples has suggested that neurologists rank seizure reduction significantly more important and side effect reduction significantly less important compared with what patients actually value.32 Across common medical decisions providers do not ask about patient preferences in up to two-thirds of encounters before recommending interventions,53 and only one-third of patients with epilepsy seizure-free for > 5 years have ever discussed the possibility of discontinuation with their physician.54 This is particularly problematic because patients frequently do not voice their true concerns during time-limited office visits without a structured process for preference measurement.51 Thus, research developing structured patient preference elicitation exercises are critical to inform the medical community about patient priorities, both in epilepsy and across other medical conditions involving preference-sensitive treatment decisions. BWS results may also inform guideline development, as incorporating systematic studies of patient preferences may help policymakers understand how patients with different values may reach different healthcare decisions.16
Our work has several limitations. It was performed at a single academic center with a predominantly Caucasian patient population and thus results may not generalize well to non-academic centers or other populations. Future data collection could also be expanded to patients with poorly controlled epilepsy, and cases managed by non-specialists. It is also not possible to include all items that may possibly influence decision making within a single survey. In this pilot we did not include ASM-related birth defects which would be relevant to women of childbearing age. Though, no respondent identified this was an important omission, and the only noted possible omission was that two respondents noted ASM-related psychosis could be an item to consider in our future larger-scale survey. This pilot study was not intended to be powered to detect between-patient differences or latent classes of patient preferences which can help clarify if a decision is preference-sensitive, or to formally compare patient versus physician responses. Our future work may more formally compare physician versus patient responses and will be planned to have sufficient power to explore patient heterogeneity. For example, given approximately half of participants’ medical charts did not inform whether the patient was driving, this pilot suggested we should embed this as a future survey question. Finally, the decision to continue or discontinue a medication is a bundled choice profile, e.g., discontinuing implies both a higher risk of seizures and thus a higher risk of driving restrictions and a lower risk of side effects. Thus, splitting each attribute into separate items does not necessarily mimic true clinical practice. Nonetheless, our survey was designed to disentangle the relative importance of each attribute to inform factors influencing the decision-making process.