Yale Observation Scale for Predicting Serious Illness in Japanese Primary Care Settings

Objectives The Yale observation scale (YOS), also called the Acute Illness Observation Scale, was developed to help detect serious illness in febrile children. We conducted this study to evaluate the clinical utility of the YOS for predicting serious illness in Japanese primary care settings. Method We conducted a prospective observational study from June 2015 to January 2018. We enrolled patients younger than 10 years, who attended the primary care centre at the Aichi Medical University Hospital and who had recorded YOS scores. Serious illnesses included invasive bacterial infections and any conditions requiring articial ventilation, surgical treatment, or cardiopulmonary resuscitation. We calculated point estimates for YOS sensitivity and specicity for predicting serious illness and hospitalisation. We evaluated the area under the receiver operating characteristic curve (AUC) by the Youden index. Results The 260 participants (227 outpatients and 33 inpatients) had a median age of 1.75 years (range, 0–10); 80 (30.8%) were younger than 1 year, 131 (50.4 %) between 1 and 3 years, 44 (16.9%) between 4 and 7 years, and 5 (1.9%) between 8 and 10 years. Among the 211 were the most (n Fourteen patients among the 29 with and 19 among those with YOS ≤ 10 were hospitalised. We found 6 patients with serious illnesses: intussusception (n = 1; YOS, 22), encephalitis (n YOS, and urinary tract infection (n 4; YOSs, and The sensitivity and specicity of the YOS for predicting serious illness at a cut-off point of 10 were 0.67 and respectively. The corresponding values for hospitalisation at that cut-off point were 0.42 and 0.93, respectively. The AUCs of the YOS for predicting serious illness and hospitalisation were 0. 82 (excellent) and 0.74 useful to predict severe paediatric diseases in Japanese primary care settings.


Background
Most children attending emergency departments and primary care centres have self-limiting illnesses, but distinguish these from serious illnesses is essential [1]. However, serious illnesses can be rare, with rates ranging from <1% in primary care settings to as high as 25% in emergency departments due to fever of unknown origin (FUO). Hospital attendance is in uenced by the medical insurance systems of a country. In Japan, access to medical care is completely covered by a national insurance system. This results in a scenario in which many children with mild illness present to hospitals after normal consultation hours. Given the insu cient numbers of paediatricians to attend to all these patients, residents with limited experience may have to attend paediatric patients.
Thus, the management of paediatric patients relies on the availability of reliable clinical tools to help predict illness severity. The Yale Observation Scale (YOS), also called the Acute Illness Observation Scale, was developed to help detect serious illness in febrile children. The sensitivity and speci city of the scale vary from poor to good depending on the setting in which it is applied [1][2][3]. The diagnostic value of the YOS has been tested in different groups [4][5][6][7][8][9][10][11][12][13][14][15][16][17], but never in general clinical settings in Japan.
We aimed at determining the utility of the YOS for predicting serious illnesses in children in Japanese primary care setting. We included many patients with non-infectious illnesses, because distinguishing patients with infectious illness from all paediatric patients in clinical settings may be di cult during rst visits. Thus, we also analysed the utility of the scale for predicting severity of non-infectious illnesses.

Patients and setting
We conducted this prospective observational study from June 2015 to January 2018. We enrolled children aged from 7 days to 10 years, who attended the primary care centre (PCC) or the emergency room (ER) at the Aichi Medical University (AMU) hospital after normal consultation hours. This covered all attendances between the hours of 5:00 PM and 8:00 AM from Monday through Friday, and all day on a Saturday, Sunday, or a national holiday. The AMU hospital is located in Nagakute city and is the central hospital of the Aichi prefecture in Japan, serving a population of 569,000 people. In this suburban area only AMU provides medical care after normal consultation hours. The ethical committee of the AMU hospital approved the study protocol (2018-H042). We conducted the study in accordance with the tenets of the Declaration of Helsinki. We collected clinical data prospectively during the normal medical practice, and analysed the data retrospectively without including any personal identifying information to have the need for informed consents waived by the ethical review board. A paediatrician made all decisions about the medical care, treatment and hospitalisation of patients irrespective of the YOS scores.
General patient management and use of the YOS A junior resident ( rst or second year of postgraduate training) was the rst physician to examine new patients. We trained these residents to apply the YOS whenever they encountered patients who seemed ill without a clear diagnosis and when the residents were uncertain of the health of the patients. We asked the residents not to apply the YOS to patients who were clearly ill (e.g., shock or cardiopulmonary arrest) or well (e.g., bug bite only with skin lesion). The residents were free to consult a paediatrician if they thought advanced care was necessary.
All participating residents received standardised face-to-face training on using the YOS correctly. Each resident decided whether to apply the YOS or not, and we did not include any resident as authors of this study. Table 1 shows the six domains of the YOS [4]with each assigned a score of 1, 3, or 5. Although the original YOS has a ve-point scale for each item, we selected a three-point scale based on previous evidence that this avoided unnecessary complexity [17]. We embedded the YOS scoring system into the electronic medical chart system of the AMU hospital. Figure 1 shows the algorithm for the use of the YOS in this study. We considered a YOS score >10 as indicative of a serious illness based on published data [2,5,[15][16][17].

Clinical procedure
The junior residents made a referral to a paediatrician when they considered the child to be ill or to have a possible serious illness after completing their history and physical examination. Further tests, including complete blood counts, serum electrolytes, urinalysis, lumbar puncture, chest X-rays and cultures (blood, urine and cerebrospinal uid), were usually performed at the discretion of the consulting paediatrician. The criteria for hospitalisation included: 1) Need for intravenous rehydration and/or antibiotics with hospital stay for a few days (not including patients who received one dose of antibiotics); 2) need for respiratory management, including oxygen administration and nasal high-ow therapy; 3) need for special care (e.g., intussusception or encephalitis); 4) potential for major surgery requiring general anaesthesia (e.g., appendicitis). A paediatrician reviewed results of laboratory and clinical examinations and made the nal diagnosis.
De nition of fever, infectious and serious illness We de ned fever at body temperatures >37.5°C and infectious illness as that in a febrile patient attributed to a pathogen and excluding heat stroke.
Published de nitions for serious illness depend on the year of studies ( Table 2). We chose a de nition for serious illness that included central nervous system infections (e.g., meningitis and encephalitis); bacteraemia/septicaemia proven by positive blood culture; severe bronchiolitis/pneumonia requiring nasal high-ow cannula therapy; suppurative arthritis; urinary tract infection (UTI) not including isolated cystitis in older children; cellulitis/abscess; and any condition requiring arti cial ventilation, surgical treatment, or cardiopulmonary resuscitation. Urine cultures positive for UTI had a single uropathogen at a density >50,000 colony-forming units (CFUs)/mL (for urine specimens collected by catheterisation or suprapubic aspiration) or >100,000 CFU/mL (for those collected by the midstream, clean-catch method) [18]. Residents diagnosed pneumonia based on chest X-ray in ltrates and a specialist con rmed the diagnosis. Bronchiolitis was diagnosed in patients with tachypnea and wheezing with a chest X-ray showing hyperin ation and peribronchial thickening.

Statistical analysis
We calculated point estimates and 95% con dence intervals for the sensitivity and speci city of the YOS at predicting either serious illness or hospitalisation. We used a ROC curve analysis with the Youden index to obtain a cut-off value that optimised the diagnostic value of the YOS. We considered the area under the ROC curve (AUC) acceptable if >0.7 and ≤0.8, excellent if >0.8 and ≤0.9 and outstanding if >0. 9 [19].

Results
During the study period, 16,324 children visited the AMU hospital after consultation hours. Of those, 14,327 were attended as walk-ins and 1,997 were transported by ambulance. In total, 1,128 children (6.9%) were hospitalised and 260 (1.6%) had a YOS score recorded by a resident. Among those who had a YOS score, 54% were boys and the median age was 1.75 years (range, 0-10 years); 80 (30.8%) were younger than 1 year, 131 (50.4%) were between 1 and 3 years, 44 (16.9%) were between 4 and 7 years, and 5 (1.9%) were between 8 and 10 years. Table 3 shows the associations between the YOS score and wither serious illness or hospitalisation. The nal diagnoses of the 260 children with YOS scores included infectious diseases in 211 children, followed by allergic disease in 11, and trauma in 10. Among the 211 patients with infectious illness, upper respiratory tract infections were the most common (55 children).
We found six children diagnosed as having serious illness (Tables 4 and 5). One patient had intussusception (YOS = 22), one encephalitis (YOS = 16), and four UTIs (YOS = 6-18) ( Table 4). One patient with UTI at the age of 4 months had a YOS of 8. The other patient with UTI at the age of 8 months had a YOS of 6. They had only fever without any symptoms. They had lost their appetite on the day of onset and presented to a paediatric clinic. Blood examinations revealed elevated white blood cell counts (WBC) and C-reactive protein (CRP) (WBC, 17,300/ l, CRP, 11.0 mg/dL in the 5-month-old patient; WBC, 22,600/ l and CRP, 2.6 mg/dL in the 8-month-old patient). They were referred and admitted to the AMU hospital.
Eventually, 33 children were hospitalised (Table 5). Two patients with a YOS ≤10, who were not admitted during the rst visit, returned the following day and were admitted. One patient was a 10-month-old child with a YOS of 6, who had a human metapneumovirus infection. Her general condition was good at initial presentation, but she was later referred and hospitalised for oral feeding di culties.
The other patient was a 6-month-old child with a YOS of 10. She was initially diagnosed as having pharyngitis, but developed a fever of 40°C and became ill the next day. She was hospitalised and diagnosed as having a UTI and bacteraemia.
After applying a cut-off value of 10, the sensitivity and speci city for detecting serious illness were 0.67 and 0.90, respectively (Table  5). Overall, a child who appeared ill based on the score was 15.9 times more likely to have a serious illness than a child who appeared well. On the other hand, the sensitivity and speci city for hospitalisation were 0.42 and 0.93, respectively. Figure 2 shows the ROC curves of the YOS for detecting serious illness and hospitalisation. We maximised the Youden index for detecting serious illness at a YOS cut-off value of 14 (sensitivity, 0.67; speci city, 0.97) ( Table 5). We considered the AUC for detecting serious illness at 0.82 as excellent. The Youden index for hospitalisation was maximal at the established cut-off value of 10 (sensitivity, 0.42; speci city, 0.93); at the cut-off value of 14, the corresponding values were 0.21 and 0.98 (Table 5). We considered the AUC for hospitalisation at 0.74 as acceptable. The YOS was less useful for hospitalisation than for the detecting serious illness.
We also veri ed the utility of the YOS with a cut-off value of 14, because the Youden index for detecting serious illness was maximised at the YOS cut-off value of 14. Seven children with a YOS >14 were hospitalised (their nal diagnosis and dispositions are shown in Table 4); of these, four were diagnosed with serious illness and three were diagnosed with non-serious illness (two with FUO and one with pneumonia). Four children with a YOS <14 were not hospitalised. They were ultimately diagnosed as having non-serious illness; two with FUO, one with upper respiratory tract infection, and one with cyclic vomiting syndrome.

Discussion
Our study indicated that the YOS has diagnostic value in a general clinical setting in a suburban area of Japan. The sensitivity and speci city of the YOS for detecting serious illnesses were good and the AUC was excellent at the established cut-off value of 10, supporting its utility as a screening test. By contrast, the accuracy of the YOS for predicting hospitalisation was only acceptable, and a speci city was hampered by only poor sensitivity. Table 6 lists sensitivities and speci cities of the YOS for detecting serious illnesses in febrile children published by other researchers in different places and focusing on different age populations. The local prevalence of illness should be considered when evaluating the e cacy of the YOS. For example, Bang et al reported excellent results when predicting bacteraemia in febrile children aged 3-36 months (sensitivity, 0.88; speci city, 0.84; AUC, 0.90) [16]. However, their study was conducted in India where the prevalence of bacteraemia was apparently higher (28.2%) than that in developed countries. In our study, serious illness was only present in 2.3% (6/260) of patients. In general, the sensitivity and speci city of the YOS in countries with a high prevalence of serious illness tend to be high. But, in our study, the YOS still had high sensitivity and speci city despite the low prevalence of serious illness in our setting. This indicates that the YOS is effective even in populations with a low prevalence of serious illnesses.
Nigrovic et al reported that the YOS score cannot reliably predict febrile infants with invasive bacterial infections when aged ≤60 days [17]. The sensitivity (12.6%) and AUC (0.53) in their study were quite low, although the speci city was high (88.5%). The difference with our results may be attributable to a younger cohort compared to that in our study. Indeed, the YOS has been reported to have limited diagnostic e cacy when evaluating febrile infants in the rst 2-3 months of life [3]. That is because this group has less-developed repertoire of responses to stimuli compared with older children. In our study, 2 infants with UTI had a low YOS (a 5-month-old with a YOS of 8 and an 8-month-old with a YOS of 6). Both infants had elevated WBC and CRP levels. Laboratory examinations help predict serious illnesses in younger infants and reinforce the usefulness of the YOS.
The high sensitivity of the YOS to predict patients with serious illness in our study was also comparable with that reported in other studies [4,6,12,15]. Three of these were conducted in developed or semi-developed countries (2 studies in USA, 1 study in UK and 1 in Turkey) and included relatively older children (<24 months or <36 months) than those included in other studies (<60 days) [3,17]. These ndings indicate that the YOS may be more useful in children aged >3 months rather than in younger infants.
To determine the usefulness in children >3 months, we evaluated the sensitivity and speci city after excluding younger infants. After excluding the 22 children younger than 3 months from the analysis, the sensitivity and speci city for predicting serious illness decreased to 0.60 and increased to 0.91, from 0.67 and 0.90 for all children in our study. This decrease of sensitivity may be explained by the exclusion of one infant younger than 3 months with UTI, and suggests that the YOS is appropriate for predicting serious illnesses in infants younger than 3 months. Therefore, we cannot be sure that the YOS is indeed more useful in children older than 3 months than in younger infants. For infants younger than 3 months, it is preferable to combine the clinical evaluation with laboratory test results including the WBC, absolute neutrophil count, and serum procalcitonin, and CRP levels to predict severity [10,11,14].
A YOS cut-off value of 10 was reported to be optimal in another study [4]. In this study, the analysis using the ROC curve showed that the optimal cut-off score was 14 for predicting serious illnesses, which gave a sensitivity of 0.67 and a speci city of 0.97. However, the sensitivity and speci city were not largely altered when the cut-off level was set at 10 (0.67 and 0.90, respectively). Given that the YOS needs to function as a screening test in different settings, the cut-off value of 10 is preferable because underdiagnosis of serious illness is likely to be reduced by applying the lower cut-off level. We recommend a cut-off level of 10 for predicting illness severity in children, as is recommended in other countries.
In this study, the ability of the YOS to predict hospitalisation was also examined. YOS was less useful for predicting hospitalisation than for detecting serious illness. This could be related to the fact that the decision of hospitalisation can be in uenced by several factors other than the severity of the illness itself. Parents tend to request hospitalisations because of the low medical costs in Japan. However, the AUC for hospitalisation was acceptable, suggesting that the YOS helps in determining whether a child needs hospitalisation in primary care settings.
We are aware of our study's limitations. First, the YOS was not evaluated in all patients who visited the emergency outpatient department (critically ill or apparently well children were excluded). This reduced the sensitivity and speci city of the YOS in this study. During the study period, 16,324 children visited PCC or ER. The gender ratio of all the children visiting PCC or ER (56% boys) was similar to that of the patients for whom YOS scores were recorded (54% boys). Given that the demographics of these two groups were similar, the patients with recorded YOS scores may have represented all the patients vising the PCC or ER. The mean age of all the patients visiting the PCC or ER (55.5 months) was older than that of the patients with recorded YOS scores (21 months). Therefore, the usefulness of the YOS for detecting severe illness may have been overvalued. For example, the hospitalisation rate was higher in the patients with recorded YOS scores (12.7%) than in all the patients (6.9%). Also, many children with cardiopulmonary arrest had no YOS scores because they appeared clearly ill.
Second, the sensitivity/speci city of the YOS may be impacted by providers with limited paediatric experience, who are not speci cally trained on using the observational scale. However, one thing we can say for sure is that YOS is still useful despite of the limited paediatric experience. Third, the YOS has usually been used for febrile children, and one-third of our patients were afebrile. However, the YOS seems also applicable to children suspected of having serious illness regardless of the presence or absence of pyrexia; and, in our study, the YOS accurately identi ed a case with intussusception without pyrexia. Fourth, we only identi ed few patients with serious illness, and this may have reduced the study's power. Thus, future studies with larger populations from multiple centres are needed to con rm our results.

Conclusion
In summary, clinical evaluation using the YOS showed su cient sensitivity and speci city for discriminating serious illnesses. It also appears to help in predicting hospitalisation. Our results suggest the YOS is useful for predicting the severity of illness in Japanese primary care settings, even if the local prevalence of serious illness is low. We recommend that the YOS can be applied to all children in primary care settings to avoid missing children with serious illnesses.

List Of Abbreviations
AUC, area under a receiver operating characteristic curve; CRP, C-reactive protein; ROC, receiver operating characteristic curves; UTI, urinary tract infection; WBC, white blood cell count; YOS, Yale Observation Scale Declarations Ethics approval and consent to participate This study was approved by the ethical committee at the Aichi Medical University (2018-H042). All study evaluations and procedures were performed in accordance with the tenets of the Declaration of Helsinki. We collected clinical data prospectively during the routine medical practice, but the study was performed retrospectively without personal information. Obtaining informed consents from all patients was impractical and the ethical committee waived the requirement given the study's design. All decisions for the treatment policy and hospitalisations were made by the paediatrician, irrespective of YOS scores. The contents of medical care were not changed based on YOS scores.    #2, Although the exact values were not described in the article, sensitivity and specificity were derived from the numbers of patients in the article.
#3, Serious bacterial illness was defined in cases with bacterial growth of a known pathogen in cultures of blood, spinal fluid, urine, or stool.
#4, Serious bacterial infection was defined in cases of bacteremia, pyelonephritis, lobar pneumonia, meningitis or osteoarthritis.
#5, Serious bacterial infection was defined in cases with isolation of the pathogenic organism from a normally sterile body fluid/tissue. #6, Serious bacterial infection was not defined in the article. Figure 1 Algorithm for the use of YOS in this study Figure 2