During the COVID-19 pandemic, critical care professionals had to make decisions under time pressure, with an even greater level of uncertainty, as the disease was previously unknown and the resources necessary for its care (mechanical ventilators, intensive care beds) were unavailable to all patients, and on many cases they had to ration or prioritise said resources. In this context, it is very likely that heuristics were used in this process leading to a possible increase in the existence of biases (29). Three heuristics could have played an important role during the course of critical care response during the course of the pandemic: the possible bias of choosing to maintain an existing situation for fear of causing harm (status quo/omission bias), the bias of diagnosing all cases with certain symptomatology as COVID-19 (representativeness) or that the most striking cases recur much more frequently than their real prevalence rate (availability).
The three selected healthcare systems (Chile, Spain, United States) are only representative of themselves, but the study in three such different countries could provide preliminary information on whether the use of heuristics is widespread in medical practice. Analyzing the decision-making process of critical care physicians, and whether this process led to preferably analytical or intuitive decisions, is difficult to carry out in real conditions. In this case, clinical vignettes are indicated to fulfil this objective.
In order to optimize the construct, internal and external validity of the vignettes, they were subjected to assessment by experts representing both clinical practice and behavioural economics. In both groups of experts, a substantial lack of knowledge was observed with respect to the area of study of the other group: while the lack of familiarity among economists of the care of critically ill patients is perhaps more understandable, the lack of knowledge of the term heuristic and its meaning among the clinicians in the pilot groups in the three countries was striking. This also meant there was a need for an introduction to key study concepts prior to its distribution among clinicians. Sufficiently clear prior instructions about the study is central to successful and accurate completion.
Although it was necessary to adapt the vignettes prepared in Spain to the medical language commonly used in Chile and the United States, there was no questioning of the clinical cases proposed in the vignettes in these two countries.
In the process of constructing the vignettes, we opted for the closed-response question modality with two or three alternatives, as it was considered to facilitate the identification of heuristics better than an open-response questionnaire. We also opted for within-subject administration, in which all participants in the study received the same vignettes instead of establishing groups of vignettes randomly distributed to different groups, to allow for a broader exploration of the decision process.
The presentation of the vignettes in different conditions (hour, age of patients, pandemic wave) increases the diversity of decision scenarios, broadening their complexity and approximating the variety of real-world situations, as a tool to increase their external validity.
By systematically following the recommendations from the existing literature regarding vignette development and the absence of substantial proposed modifications during the validation and piloting process of the vignettes, these vignettes could be considered a useful instrument for the study of clinical decision making in critical care during the COVID-19 pandemic in other contexts after a process of adaptation and validation.
The complexity of the process of requesting review by the ethics committees in each country, each with different requirements and conditions, should be emphasised. It would be advisable to have internationally standardised procedures for studies with these characteristics.
Studies using clinical vignettes have limitations, including those related to their construct and external validity. As they are hypothetical scenarios, they are exempt from the conditioning factors and determinants of real clinical practice, such as time limitations and the pressure which exists at time of decision. This study attempted to minimize this potential limitation. It should be noted that this study, similar to other studies of this type, is subject to the same limitation of non-respondent bias (30).
Vignette studies are also subject to the Hawthorne effect (31). Another possible limitation is the sentinel effect (32): behaviour can be modified by the fact that it is being evaluated. An attempt was made to reduce this possible limitation by guaranteeing complete anonymity of the responses. It was also reiterated that the study was in no way evaluative of clinical knowledge or patient outcomes but was simply for the purposes of gaining insight into the clinical decision-making process.
The evidence available on the validity of vignette studies comes mainly from primary care settings (33), where results using this methodology correlate positively with real-world studies, thus constituting a proxy for evaluating practice. However, studies on conditions such as those seen in critical care, remain scarce (34). Despite the existing limitations of this type of methodology, it is still one of the few available approaches to research clinical decision-making in critical care.