Transcutaneous Oxygen Saturation Accuracy in Critically Ill Children CURRENT STATUS:

Pulse oximetry (SpO 2 ) is used to monitor oxygen saturation levels to avoid hypoxaemia in children. Sensor manufacturers claim high sensitivity, specificity and accuracy. Few studies have evaluated accuracy and precision of SpO 2 in children. This prospective, observational study was conducted in a 36-bed mixed medical/surgical paediatric intensive care unit. All children <16 years old with an arterial line were eligible. Paired SpO 2 readings obtained with a Masimo and a Nellcor sensor were prospectively matched and validated to the arterial haemoglobin oxygen saturation (SaO 2 ). Bias between SpO 2 and SaO 2 (SpO 2 -SaO 2 ), accuracy root mean square (A rms ), sensitivity, specificity and kappa agreement were calculated for sensors. Multivariable regression analysis was conducted to determine the relationship between clinical variables and bias in paired sensor readings.


Introduction
Pulse oximetry (SpO 2 ) is a commonly used monitoring tool to assess patient stability, guide emergency airway management, titrate vasoactive medications and fluid management, set fraction of inspired oxygen (FiO 2 ), positive end-expiratory pressure (PEEP) and supplemental oxygen delivery in hypoxaemic children. Significant clinical decisions in the patient's care are made based on readings of 4 the pulse oximeter, such as oxygen administration, transfer to higher level of care or escalation of therapy. Although arterial haemoglobin oxygen saturation (SaO 2 ) is considered the 'true gold standard' measure of oxygenation, it is not always available in clinical settings, particularly in infants and children. Therefore, pulse oximetry readings-as the commonly used surrogate measure to arterial oxygen tension (PaO 2 )-preferentially need to be accurate. The detection of a true hypoxaemia (defined as SaO 2 < 88%) by pulse oximeter, with a high sensitivity and specificity, is one of the most clinically relevant measures to assess the severity of a critically ill child 1 .
As with any recorded variable, an element of bias is expected when comparing to the true value. Bias has been reported between SpO 2 and SaO 2 2,3 . The acceptable bias depends on the clinical setting. As such, a smaller bias (and a higher precision) is required in the critically ill child. Unfortunately, movement artifacts, record-lag, and low perfusion states can reduce measurement reliability.
Numerous commercial pulse oximetry sensors with variable bias and precision are available on the Only a few studies have evaluated the accuracy and precision of SpO 2 in critically ill paediatric patients. Earlier studies indicated that SpO 2 systematically overestimates SaO 2 in paediatric patients, but these studies were limited by small quantities of samples in the hypoxaemic range 3,8,9 . Data obtained in children with cyanotic heart diseases (baseline saturations < 90%) showed that pulse oximetry performed less well in this group 2,3,10 .
The primary objective of this study was to assess the performance of pulse oximetry in critically ill

Bias
Of the 9,382 paired readings, 19·0% of the values were below SaO 2 < 88%. Masimo and Nellcor median bias varied with SaO 2 category and were inversely proportional to increasing SaO 2 categories (Table 2). There was significant difference between Masimo and Nellcor median bias in paired readings, in the entire range and categories of 88-92% and ≥92% ( Table 2). The Bland-Altman plots ( Fig. 1) confirmed the inverse agreement between range of saturation and bias. With the exclusion of outliers (where clinician is likely to ignore values as erroneous), the IQR extended out greater than 3% with wide ranges of variability in all categories of SpO 2 readings with the exception of the 92-100% range (Fig. 2). Intra-individual bias for repeated measurements varied (precision), with a decrease with improved saturations (Appendix, Figure A2a-d). Both sensors failed to achieve FDA's A rms requirement in any of the saturation decile ranges (Table 3) particularly in the range of 90-100% (A rms = 4·5 when both sensors included). Agreement between SaO 2 and SpO 2 was strong (κ = 0·737, p < 0·001; Table 4).

Discussion
Our study has the largest dataset of prospectively sampled and validated paired samples to date.
With an a priori exploratory analysis plan, the study provides insights into the parameters that dictate bias. However, the results cause concern. Both Masimo and Nellcor sensors are not precise enough for the requirements of a paediatric intensive care setting. The low sensitivity of both sensors (71·4%) is alarming since 28·6% of the patient readings (n = 1165) in the true hypoxaemic group (SaO 2 < 88%) were not detected as being hypoxaemia by the saturation sensors. These results corroborate with recent paediatric studies, showing the tendency of saturation sensors to overestimate, particularly at SaO 2 < 88% 2,3 . Given that paediatric patients in our ICU have 15·5% of their total readings of SaO 2 under 80%, this low sensitivity is sub-optimal.
Harris et al. attempted to address sensor precision and bias in paediatric patients with cyanotic congenital heart disease (CCHD). They compared a sensor intended for low saturation scenarios  10 . They observed poor precision and a large bias, which increased in the < 75% range of SaO 2 10 .
We observed in our study-as a measure for accuracy-A rms values greater than 3% in all SaO 2 categories (even SaO 2 > 92%), indicate that Masimo and Nellcor sensors, as they are currently being used in paediatric patients, may require changes to the industrial algorithm to achieve the FDA's bias criteria of A rms <3% in a paediatric population. Achieving these improvements requires further studies of saturation sensors in critical care paediatric populations. We also showed that intraindividual bias (precision) was not constant and the bias decreased with increasing saturation values.
Our study also aimed to describe the disease states and criteria that contribute to sensor bias.
Disease categories, such as respiratory disease, sepsis and post-cardiac arrest were shown to be significantly associated with absolute bias > 3% in our model. There is a strong association between increasing severity of respiratory diseases and poor pulse oximetry accuracy 3 . Some researchers suggest that SpO 2 /FiO 2 ratios could be employed instead of the PaO 2 /FiO 2 ratio in severity prediction of acute respiratory distress syndrome (ARDS) and acute respiratory failure 11,14,15 . The recent paediatric ARDS definitions incorporate SpO 2 /FiO 2 ratios in severity stratification. There are two issues with this approach due to the inaccuracy of pulse oximetry found here. First, patients may get incorrectly stratified into a low-risk group and receive less scrutiny compared to if they were in the high-risk group. Furthermore, research studies based on this stratification will yield erroneous results that might then change practice and disadvantage future patients.

Conclusions
Within the limitations of available technology, both Masimo and Nellcor sensors exhibit suboptimal accuracy, particularly in patients with saturations < 88%. Nellcor has slightly better sensitivity when compared to Masimo but is more biased in multivariable modelling. In disease states such as sepsis, respiratory diseases, post-cardiac arrest and vasoconstrictor use, the clinician may have to be circumspect when basing decisions on peripheral oxygen saturations. Ultimately, further improvements need to be made in industrial algorithms to improve sensor accuracy.

Ethical Approval and Consent to Participate
This study was reviewed and approved by the Queensland Children's Hospital IRB under HREC/15/QRCH/165 and RD005952. Waiver of formal consent was approved by the IRB, given that pulse oximetry (SpO 2 ) and arterial blood gas analysis (SaO 2 ) are standard practice in the PICU.
Additionally, assents from parents or patients (where appropriate) were sought before recording data.

Consent for Publication
Not applicable.

Availability of Supporting Data
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing Interests
There are no conflicts of interest to report.

Funding
This study has been supported by a grant from the Intensive Care Foundation, Australia. The funding body contributed financial support for analysis and interpretation of the data.

Author Contributions
JB and AS conceived and designed this study, acquired and interpreted data, and drafted and revised this work. JB, TW and DP helped instruct nurses on validation and recording of sensor readings. KG and SR made significant contributions to interpretation of the data, as well as drafting and revising this work. NA, KRD, TW, MH and DP conceived and designed this study and revised the manuscript. TP contributed to interpretation of the data and revision of the manuscript. All of the authors have given final approval and accountability for the work herein.

Supplementary Files
This is a list of supplementary files associated with this preprint. Click to download.