Comparison of the accuracy of clinicians’ prediction of survival and Palliative Prognostic Score: an East Asian cross-cultural study

No study has been conducted to compare the clinicians’ prediction of survival (CPS) with Palliative Prognostic Scores (PaP) across countries. We aimed to compare the performance of the CPS in PaP (PaP-CPS), the PaP without the CPS, and the PaP total scores in patients with advanced cancer in three East Asian countries. We compared the discriminative accuracy of the three predictive models (the PaP-CPS [the score of the categorical CPS of PaP], the PaP without the CPS [sum of the scores of only the objective variables of PaP], and the PaP total score) in patients admitted to palliative care units (PCUs) in Japan, Korea, and Taiwan. We calculated the area under the receiver operating characteristic curve (AUROC) for 30-day survival to compare the discriminative accuracy of these three models. We analyzed 2,072 patients from three countries. The AUROC for the PaP total scores was 0.84 in patients in Japan, 0.76 in Korea, and 0.79 in Taiwan. The AUROC of the PaP-CPS was 0.82 in patients in Japan, 0.75 in Korea, and 0.78 in Taiwan. The AUROC of the PaP without the CPS was 0.75 in patients in Japan, 0.66 in Korea, and 0.67 in Taiwan. The PaP total scores and the PaP-CPS consistently showed similar discriminative accuracy in predicting 30-day survival in patients admitted to PCUs in Japan, Korea, and Taiwan. It may be sufficient for experienced clinicians to use the CPS alone for estimating the short-term survival (less than one month) of patients with far-advanced cancer. The PaP may help to improve prognostic confidence and further reduce subjective variations.


Introduction
The ability of clinicians to predict survival time is vital for palliative patients and their families. End-of-life care based on prognostication can be important to an individual patient's benefit [1], and the clinicians' prediction of survival (CPS) is a classical tool used for this purpose. However, the CPS is known to be far more optimistic than actual survival [1][2][3][4][5]. A previous study demonstrated that the accuracy of the CPS varied between 23 and 78% [6]. And the CPS could be affected by the patient-clinician relationship even in the case of expert clinicians [7]. Therefore, clinicians are recommended to use the CPS with objective prognostic indicators [1,8,9].
The Palliative Prognostic Score (PaP) is comprised of the CPS, Karnofsky Performance Status (KPS), two symptoms, and two laboratory values from the complete blood count [10]. The PaP aims to predict 30-day survival and has been validated in various clinical settings [11][12][13][14][15][16][17][18]. Thus, the European Association for Palliative Care recommended PaP as an evidence-based prognostic tool [9]. However, inexperienced clinicians may hesitate to approach the PaP because it can be difficult for them to formulate the CPS [19][20][21].
A few studies have investigated whether the CPS enhanced the accuracy of prognostic models [22,23]. The performance of the CPS and the PaP can be influenced by each country's environment including the medical systems and the educational background of the clinicians [6]. In addition, the clinician-patient relationship in each country may Yusuke Hiratsuka and Seok-Joon Yoon contributed equally to this work affect the CPS performance [24]. To the best of our knowledge, no study has been conducted to compare CPS with the accuracy of prognostic models in patients in different countries. We thought that investigating diverse populations would be confirmative since previous studies showed contradictory results [20,25]. Therefore, we aimed to compare the performance of the PaP-CPS (the score of the categorical CPS of PaP), the PaP without the CPS, and the PaP total scores to predict the 30-day survival of patients in three East Asian countries.

Participants
This was a secondary analysis of an international multicenter prospective cohort study called the East Asian collaborative cross-cultural Study to Elucidate the Dying Process (EASED). It aimed to investigate the dying process and the end-of-life care of inpatients with far-advanced cancer in palliative care units (PCUs) nationwide in Japan, Korea, and Taiwan. We consecutively enrolled eligible, newly admitted inpatients in the participating PCUs during the study period.
All observations were performed within the range of routine clinical practice. The inclusion criteria were (1) adults aged ≥ 18 years old in Japan and Korea and ≥ 20 years old in Taiwan, (2) patients with locally extensive or metastatic cancer, and (3) patients newly admitted to a participating PCU. The exclusion criteria were (1) a scheduled discharge within one week and (2) refusal of the patient or her/his family to enroll in the study.

Data collection
The palliative clinicians evaluated the patients' symptoms by direct observation at admission. We followed up with the discharged patients for six months after admission to the PCUs in Japan and Taiwan, and six months after discharge from the PCUs in Korea. We defined survival time as mortality in and outside hospitals calculated by subtracting the admission date from the death date. We collected data on the CPS, KPS, dyspnea, anorexia, leukocyte counts, and lymphocyte percentages to calculate the PaP [10].

Data analysis and statistics
In this study, the variables were defined as (1) PaP-CPS, which was the score of the categorical CPS of the PaP, and was selected by the clinicians (range 0-8.5 points); (2) PaP without CPS, which was the combined score of the five clinical and laboratory variables in the PaP, and was equivalent to the PaP total score minus the PaP-CPS (range 0-9 points); and (3) the PaP total score, which was the PaP-CPS + PaP without the CPS (range 0-17.5 points).
We generated a receiver operating characteristic (ROC) curve using the true positive rate and the false positive rate. An ROC curve examines the performance of a binary outcome (e.g., alive or dead at 30 days). We calculated the area under the ROC curve (AUROC) to compare the discriminative accuracy of the PaP total score, the PaP-CPS, and the PaP without the CPS for the 30-day survival predictions of the patients in three countries.
All analyses were performed using IBM Statistical Package for Social Science (SPSS) Statistics for Windows, version 26.0 (IBM Corp., Armonk, NY, USA). A P-value below 0.05 was considered statistically significant.

Ethics
In accordance with the ethical guidelines for human research of Japan's Ministry of Health, Labor, and Welfare, the requirement for informed patient consent was waived in Japan because of the observational nature of the study. In Korea and Taiwan, informed consent was obtained from the patients or their families (if the patient lacked the capacity to make decisions). This study obtained approval from the local Institutional Review Boards of all participating institutions. The Independent Ethics Committee of the Tohoku University School of Medicine (approval no. 2016-1-689) also approved this study.

Patient characteristics
The data of 2,638 patients from 37 PCUs (22 in Japan, 11 in Korea, and four in Taiwan) were available. The patients were enrolled from January 2017 to September 2018. We excluded 564 patients (474 in Japan, 15 in Korea, and 75 in Taiwan) because laboratory data or CPS was missing to calculate the PaP. Among the eligible 2,074 patients, two Japanese patients were excluded because of missing data on survival time. Thus, 2,072 patients (1,422 in Japan, 320 in Korea, and 330 in Taiwan) were analyzed, which included 1,078 men (726 in Japan (51.1%), 174 in Korea  Table 1. Table 2 shows the characteristics of the 162 clinicians (69 in Japan, 29 in Korea, and 64 in Taiwan) participating in this study. The study included 92 male clinicians (52 in Japan (75.4%), eight in Korea (27.6%), and 32 in Taiwan (50.0%)). According to the countries, their clinical backgrounds such as specialty, clinical experience, and clinical experience with palliative care were diverse. The most common specialty of the Japanese clinicians was palliative care, whereas it was family medicine for those in Korea and Taiwan. The clinical careers and palliative care careers of the Taiwanese clinicians were a little bit shorter than those of the Japanese or Korean clinicians because some Taiwanese resident clinicians participated in about half of the institutions in this study. There was no difference in the number of faradvanced cancer patients treated by the clinicians of the three countries in one year.

PaP total score vs. PaP-CPS
The comparison of the PaP total scores and PaP-CPS results is shown in Table 3 and Figure 1. The AUROCs of the PaP total scores in the patients in all three countries were greater than those of the PaP-CPS but were not statistically significant. The phenomenon was common between the three countries.

PaP total score vs. PaP without the CPS
The AUROCs of the PaP total scores in all three countries were greater than those of the PaP without the CPS and were statistically significant (Table 3, Figure 1).

PaP-CPS vs. PaP without the CPS
The AUROCs of the PaP-CPS in all three countries were also significantly greater than that of the PaP without the CPS (Table 3, Figure 1).

Discussion
The aim of this study was to compare the performance of the PaP-CPS, the PaP without the CPS, and the PaP total score to predict 30-day survival in patients in three East Asian countries. Our study demonstrated that the PaP total score and the PaP-CPS alone consistently showed similar discriminative accuracy in predicting 30-day survival in patients in all three East Asian countries.
The study findings showed that the CPS alone was almost as accurate as the PaP total score in predicting 30-day survival in patients in Japan, Korea, and Taiwan. A previous study showed a similar accuracy of the CPS and the PaP in predicting 30-day, 14-day, and 7-day survival [23]. However, the previous study was a single-center study and thus, further extended studies are needed. The strengths of the current study were that we included a large number of participants from three different East Asian countries. And consistently, the accuracy of the CPS was comparable to that of  the PaP total score in the patients in all three countries. We assume our results are explained in terms of patients' and clinicians' factors. As for patient factors, our study population included advanced cancer patients with a short median survival time (19 days in Japan, 23 days in Korea, and 15 days in Taiwan). The CPS could be relatively accurate in patients with more predictable trajectories and experienced clinicians [26]. Therefore, the patients in the three countries studied may have a more predictable clinical course than those in the recent studies. Consistently, the accuracy of the CPS may be higher in patients with shorter predicted survivals [27]. Regarding the clinician factors, we assume that the participating clinicians were experts in predicting survival.
Experts are known to have smaller errors in prognostic predictions and could predict more accurate prognoses [5,24]. The accuracy of the CPS is influenced by clinician-related factors such as the lack of knowledge on the survival of patients under palliative care, the duration of the clinicianpatient relationship, and training on prognostication [24]. In Japan, all palliative care clinicians are required to be educated through a comprehensive educational program provided by the Japanese Society for Palliative Medicine [28,29]. The PaP-CPS in Japan showed higher accuracy than that of the other two countries. Although the difference was not statistically significant, it can be explained by the following. In Japan, the National Cancer Control Act established in 2007 has actively promoted palliative care for cancer patients immediately after their diagnosis to improve distress, symptom burden, and quality of life. It enables palliative care physicians to manage patients with cancer continuously from an earlier phase of the disease, allowing time to observe the changes in a patient. Notably, the major specialty of clinicians in Japan was palliative care, whereas for those in Korea and Taiwan, it was family medicine. This difference in clinician specialties might have contributed to the higher accuracy of the PaP-CPS in Japan. In Korea, standardized educational programs are mandatory for all palliative clinicians to start their careers in the palliative care field. They also should complete at least two educational programs by the Korean Society of Hospice and Palliative Care per year. In Taiwan, a continuous educational program sponsored by the Taiwan Academy of Hospice Palliative Medicine has been provided for clinicians for palliative care similar to that in Japan. Additionally, although some participating clinicians were resident clinicians in half of the institutions, all of them received supervision from attending physicians in terms of CPS evaluation. The palliative care clinicians in all three countries generally would have provided information to families regarding survival predictions [30], since children regard talking about death to their elderly parents contrary to "filial piety" under the Confucian culture [31]. Our findings showed that the CPS was formulated accurately while palliative clinicians were cognizant of indirect communication with the patients. This study had several limitations. First, this study used laboratory data to calculate the PaP within one week before or three days after study entry when available. However, if the timing of laboratory data collection was extended or shortened, the PaP total score might be different. Second, the median survival of our patients was relatively short at 15-23 days. Because patient survival time is an important factor influencing the accuracy of the CPS, we could not guarantee that the accuracy of our CPS findings would be similar in patients with longer survival times. Third, this study population was in PCUs and may not represent the general population of patients with far-advanced cancer. The CPS can differ according to patients' life expectancies and their places of care. Moreover, the CPS might be formulated by experts demonstrating excellent prognostic performance. Fourth, the subject patients were skewed toward Japan (around 69%). And the gender of the participating physicians was skewed to females (72.4%) in Korea. Thus, the results may be different in different populations. Lastly, our findings may be influenced by the accessibility to palliative care services and PCU admission in each country.

Conclusion
The PaP total score and the PaP-CPS showed consistently similar discriminative accuracy in predicting 30-day survival in patients admitted to PCUs in Japan, Korea, and Taiwan. It may be sufficient for experienced clinicians to use the CPS alone for estimating the short-term survival (less than one month) of patients with far-advanced cancer. The PaP may help to improve prognostic confidence and further reduce subjective variations. project administration, resources, supervision, writing-review and editing Takashi Yamaguchi: Investigation, writing-review and editing Tatsuya Morita: Funding acquisition, investigation, writing-review and editing Satoru Tsuneto: Supervision, writing-review and editing Akira Inoue: Investigation, supervision, writing-review and editing Funding This work was supported in part by a Grant-in-Aid from the Japanese Hospice Palliative Care Foundation; Grant Numbers 16H05212 and 16KT0007.

Data availability
The data that support the findings of this study are available from the corresponding author, Sang-Yeon Suh, upon reasonable request. All authors agree to provide data to the journal for review if needed.
Code availability Not applicable.

Declarations
Ethics approval This study obtained was approved by the local Institutional Review Boards of all participating institutions. The independent ethics committee of Tohoku University School of Medicine (approval no. 2016-1-689) approved this study.

Consent to participate
In accordance with the ethical guidelines for human research of the Ministry of Health, Labor, and Welfare in Japan, patients' informed consent was waived in Japan because of the completely observational nature of the study. Patients were provided the opportunity to opt out. In South Korea and Taiwan, informed consent was obtained from the patients or their families (in case the patient lacked the capacity to decide).

Consent for publication
The authors affirm that human research participants provided informed consent for publication of the article.

Conflict of interest
The authors declare no competing interests.