THETA: Machine-learning Algorithm for Evaluating Hybrid Operation Theater Entry of Trauma Patients

The clinical benets of hybrid operating rooms are recognized globally. However, appropriate conditions for entry into such rooms must be urgently established, because they exclusively benet few patients under severe trauma while requiring a signicant amount of resources. This paper presents an algorithm to triage trauma patients into a hybrid operating room. This retrospective observational study was conducted using the Japan Trauma Data Bank database comprising information collected between January 2004 and December 2018. A machine-learning-based triage algorithm is developed using the baseline demographics, injury mechanisms, and vital signs obtained from the database. The analysis dataset comprised information regarding 117,771 trauma patients with abbreviated injury scale (AIS) > 3. The performance of the proposed model was compared against those of other statistical models (logistic regression and classication and regression tree [CART] models) while considering the status quo entry condition (systolic blood pressure < 90 mmHg).


Results
The

Conclusions
An accurate machine-learning-based algorithm is developed to triage patient entry into hybrid operating rooms via a web application, thereby enabling emergency doctors to utilize limited medical resources more e ciently.

Background
Hemorrhaging is the leading cause of trauma death. 1,2 Although immediate open surgical treatments have been considered to be the most effective procedures for hemorrhage control in patients with severe injuries, endovascular treatments, including resuscitative endovascular balloon occlusion of the aorta and angioembolization, have also become highly popular in the past decade. [3][4][5] It is considered ideal to perform resuscitation along with open surgery and endovascular treatments in the same room because patient transfer to operation or angiography rooms might delay hemostasis. Therefore, hybrid operation rooms, designated as the resuscitation with angiographic percutaneous techniques and operative resuscitation (RAPTOR) suite and similar equipment, 3 have been developed for use as advanced trauma resuscitation rooms globally. Recent studies have reported that RAPTOR along with an advanced computed-tomography system available in trauma resuscitation rooms (hybrid emergency rooms) can reduce trauma-patient mortality rate. 6,7 In addition, a retrospective study that explored the potential bene ts of RAPTOR revealed that 7% of patients underwent both endovascular and surgical treatments, suggesting that the entry criteria of persistently hypotensive (systolic blood pressure [SBP] < 90 mmHg) and severely injured (injury severity score [ISS] ≥ 12) patients were not speci c to the requirements of multiple hemostatic procedures. 8 Because RAPTOR is an expensive environment that should be used for patients who need immediate endovascular treatments, it is imperative to establish entry criteria to prevent overtriage and allocate resources e ciently. 9,10 This paper presents an algorithm to identify patients who must be urgently admitted to the RAPTOR suite. This is a challenging task owing to (1) variations in RAPTOR-suite accessibility depending on the speci c institute location, institutional commitment, and competing resources; (2) limited information availability upon patient arrival; and (3) non-availability of triage algorithm in medical settings. We overcome these problems by using a state-of-the-art machine-learning algorithm along with a corresponding web application.

Study Design and Data Collection
This retrospective observational study aimed to develop an algorithm that could accurately predict whether a patient would bene t from being admitted to the RAPTOR suite. The data were obtained from the Japan Trauma Data Bank (JTDB) for the period between January 2004 and December 2018. The JTDB was established in 2004; it receives records from 280 hospitals, 95% of which include governmentapproved tertiary emergency medical centers. Patients with ISS ≥ 9 and burn patients are registered in the database. The JTDB includes information about age, sex, vital signs, Glasgow coma scale (GCS) score upon arrival at the emergency department (ED), mechanism of injury, abbreviated injury scale (AIS) code for each body region (AIS version 1990, updated in 1998), duration of hospitalization, and survival state at discharge from hospitals. The study was conducted in accordance with the Helsinki Declaration and approved by the Ethics Committee of Tokyo Medical and Dental University (approval # 2192). The requirement for informed consent was waived owing to the retrospective design and the use of anonymized patient and hospital data.

Study Population
This study included patients with documented blunt or penetrating trauma. Patients were excluded if their age was under 16 years, if they were in cardiac arrest (de ned by an SBP of 0 mmHg) upon arrival at the ED, or if they were transferred from another hospital. In addition, patients without the AIS or hospital mortality data and those without descriptions of the presence or absence of angiographic techniques/open surgery were excluded.

De nition and Outcomes
We utilized the following criteria to identify patients who could bene t from being admitted into a RAPTOR suite: patients who underwent (1) endovascular treatments or (2) open surgery and died in the ED (potential endovascular treatments were required to save them). In addition, the condition of patients with injuries corresponding to AIS = 6 was considered too severe for treatment in RAPTOR suites; therefore, they were deemed ineligible to enter the suite.

Construction of Computational Model
The trauma hybrid-suite entry algorithm (THETA) was built using machine-learning techniques, including Bayesian ridge regression, 11 XGBoost, 12 multilayer perceptron, 13 linear regression, support vector machine, 14 and K-means clustering techniques. 15 Ensemble methods 16 were utilized to combine these models (a scheme of the model is provided in Supplemental Fig. 1). Patients were admitted into the RAPTOR suite according to the relationship between their predictive scores, calculated using the algorithm, and the prede ned threshold in each hospital; RAPTOR was recommended when the predictive score ≥ threshold. A total of 80% of the JTDB cohort was used for training the model (training dataset), and the remaining 20% was utilized for model validation (test dataset). The test dataset was employed only to evaluate the model; all hyperparameters and the model creation process were adjusted using the training dataset. The model included features (explanatory variables) such as age, sex, mechanism of injury (penetrating or blunt), SBP, heart rate, respiratory rate, and GCS score.

Statistical Analysis
The validation dataset (test data) was used to evaluate the model accuracy. The proposed algorithm was compared with the logistic regression model as well as the classi cation and regression tree (CART) model. 17 The model performance was evaluated based on the receiver operating characteristic (ROC) curve, area under the ROC (AUROC), precision-recall curve (PRC), and precision-recall area under the curve (PR-AUC). We evaluated the PRC and PR-AUC because a previous study demonstrated that the proportion of patients bene ting from RAPTOR suites is small, 8 and in a dataset with such highly biased labels, the PRC plots and PR-AUC are known to be bene cial. 18 Precision and recall are de ned as follows.
These two parameters have a trade-off relationship and are represented as PRC plots.
The relationships between the variables "recall," "precision," and "percentage of patients admitted into the RAPTOR suites" are represented using a plot. This plot is important in our prediction algorithm because the corresponding values were determined individually by considering each institutional circumstance. To evaluate the model visually, the prediction calculated using THETA was visualized by inputting the information of each patient.
All statistical analyses were performed using Python version 3.7.2 (Python Software Foundation) and scikit-learn version 0.20.2 (scikit-learn developers). Furthermore, Heroku with Plotly, version 4.6.0, and Dash, version 1.10.0, were used to present the model prediction results interactively.

Web Application
Because THETA is a complicated model whose calculations cannot be performed manually, we developed a web application that can be used by any medical staff on site. This application calculated whether a patient should be admitted into the RAPTOR suite by entering his or her vital signs and mechanism of injury. The application was developed using the Google Cloud Platform and Bokeh 2.0.0; it can be accessed at http://www.accm.jp/theta/index.html. Supplemental Movie shows a sample video of a scene in which this application is used, and Supplemental Fig. 2 illustrates the expected sequence of events using the proposed method. Figure 1 illustrates the study participant selection process. A total of 3 783 patients (3.2%) were selected as candidates for admission into the RAPTOR suite. Table 1 summarizes the baseline characteristics of each group. The overall mortality rate was 8.4%. Predictive Performance Figure 2 shows the PRC plot of THETA. As evident, the PR-AUC of THETA (0.59) is considerably higher than those of the logistic regression model (0.22) and CART (0.20). In Fig. 2, the cross (precision: 0.17, recall: 0.39) represents the precision and recall when the patients were triaged based on the previously adopted criterion 8 : SBP < 90 mmHg. The ROC curves in Fig. 3

Discussion
This analysis demonstrated that THETA, a machine-learning-based algorithm used to predict patients who could potentially bene t from RAPTOR, obtained substantially better prediction accuracy than the status quo criterion of SBP < 90 mmHg. Moreover, the complex THETA model outweighed the models using the conventional logistic regression approach or CART analysis from the perspective of discrimination power evaluated according to the AUROC and PR-AUC. Therefore, the implementation of THETA in a clinical setting could enable the selection of appropriate patients who should be treated in RAPTOR suites and improve resource allocation at each institution. Moreover, the web application could automate the selection process and help clinicians objectively determine whether a patient should be admitted into a RAPTOR suite.
Critically injured patients would greatly bene t from RAPTOR suites; however, serious patients who cannot bene t from any intervention should not be considered for admission. This complex entry requirement cannot be realized via blood-pressure-based classi cation (e.g., SBP < 90 mmHg) or relatively simple models, such as CART or the logistic regression models (Figs. 2 and 3), alone. The strength of THETA is derived from the two-step prediction model that combines several machine-learning models. In the rst step, multiple machine-learning models are trained to set the indications for RAPTOR admission independently. Subsequently, after the patient information is considered as input, the system identi es the model that provides better accuracy and uses the same for decision making (Supplemental Fig. 1). Accordingly, an accurate model is developed to satisfy the contradictory requirements of including patients who urgently require treatment but excluding those who are too severely injured to be treated in time (http://www.accm.jp/theta/ gure.html).
One of the advantages of THETA is its ability to adjust entry conditions by considering the capacity of the facility. The threshold of acceptable precision differs across hospitals with different resources and therefore may not be practical if an entry condition is not adjustable. THETA can exibly consider this factor and allow clinicians to set ex-ante agreements of the threshold for admittance into RAPTORs using the same model (Supplemental Fig. 2). Thus, we believe that THETA can optimize medical resources in institutions considering the introduction of RAPTOR suites in the future as well as the institutions that already utilize them.
Nevertheless, this study has a few limitations. First, this study was retrospective; therefore, the proposed algorithm should be validated through prospective research. Second, we could not consider pre-hospital interventions for the prediction. Although the analysis yielded fair values of the AUROC and PR-AUC, the inclusion of these features for model development may improve the prediction accuracy. Third, we included patients who underwent either operative or angiographic intervention but died in the ED as candidates who might have bene ted from being admitted into RAPTOR suites; however, it is unclear whether these patients would have bene ted from this treatment. Nevertheless, to the best of our knowledge, this study is the rst in which the RAPTOR suite admission conditions have been examined based on objective data analysis. We believe that this report provides useful information on the utilization of RAPTOR suites in trauma treatment, which is currently in the spotlight and is being employed worldwide.

Conclusions
We developed an accurate algorithm to triage patients for admission into RAPTOR suites that can be used onsite. This algorithm will enable the e cient utilization of limited medical resources and simultaneously rescue critically injured patients. This study was conducted in accordance with the principles of the 1964 Declaration of Helsinki and its later amendments. The Institutional Review Board of the Tokyo Medical and Dental University approved this study (#2192). The requirement for informed consent from each patient was waived given the retrospective design of the study and the use of anonymized patient and hospital data.

Consent for publication
Not applicable

Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests
The authors declare that they have no competing interests.

Funding
No funding was received for this study Authors' contributions AS, EA contributed to the acquisition of data. AS and TK jointly analyzed and interpreted the data. EA con rmed the analysis. EA and YO drafted the manuscript. All of the authors reviewed and discussed the manuscript. EA and TK revised the manuscript for important intellectual content. All of the authors read and approved the nal manuscript.