Diagnostic Test Accuracy of New Generation Tympanic Thermometry in Children: A Systematic Review and Meta-analysis

Abstract

Background

Body temperature measurement is a routine in the management of sick children for both parents and healthcare providers[1, 2]. An accurate diagnosis of fever is crucial in clinical practice[3, 4] and an inaccurate one could lead to serious complications and improper medical decisions[3, 5]. Core temperature is the gold standard for temperature measurement[3]. However, it is invasive and requires specialized equipment, therefore, is unpractical for daily clinical practice[3, 6]. Ideally, body temperature measurement should be noninvasive, accurate, pain-free, cost-effective and time-efficient[3, 7, 8].

Traditionally, non-invasive methods of body temperature measurement include rectal temperature, oral temperature and axillary temperature. Among these methods, rectal thermometry has been the most reliable for measuring body temperature in children and is considered clinically to be the best estimation of the core temperature[9]. However, it is time-consuming and requires certain level of practice [5, 10]. Furthermore, it may cause emotional distress, and -although very rare- brings possible complications such as perforation or transmission of micro-organisms [5, 10]. And therefore infants, health workers and parents more or less express reluctance to perform it[3].

Tympanic thermometry is a popular alternative for the traditional measures. It detects the radiation of tympanic membrane and the ear canal, which share the blood supply with the hypothalamus, the thermoregulatory center of the human body[11, 12]. Using the ear temperature is less traumatic and allows a faster triage[13],but it has been debated for the low accuracy and reproducibility compared with other measurements[1, 13–17]. Over the past years, however, the infrared tympanic thermometers have been developed and updated, and some older versions of infrared tympanic thermometers have been obsolete, for instance, Thermoscan Pro-3000 (San Diego, CA) and First Temp 2000A (Intelligent Medical Systems, Carlsbad, Calif). It is unreasonable to synthesize studies applying obsolete tympanic thermometers. Furthermore, the cutoffs of the tympanic thermometer used in fever detection are diverse, and the optimal cut-off has no consensus.

The aims of this systematic review were (1) to identify and quantify studies reporting the diagnostic accuracy of the new generation of infrared tympanic thermometers in children; (2) to compare the sensitivity and specificity under different cutoffs of the infrared tympanic thermometers and give the optimal cutoff.

Methods

Search strategies

The perform of this systematic review and meta-analysis was based on the Test Accuracy Working Group of the Cochrane Collaboration and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy Studies (The PRISMA-DTA Statement) statement guidelines[18, 19]. A systematic literature search of multiple electronic databases (PubMed, Web of science, EMBASE) was conducted by two trained reviewers (D.S. and LY.Z.) independently from inception to February 2nd, 2019. The search was restricted to English and original research articles only. The following search terms “tympanic ear thermometer or infrared thermometry or ear thermometry or infrared tympanic thermometer or tympanic scan or tympanic temperature” and “pediatric or child or children or kid or newborn or baby or infant or toddler” in title or abstract were used in this review. The bibliographies of included studies were also searched to identify additional studies.

Study Selection

Observational studies, detecting fever by tympanic and rectal thermometers, were deemed acceptable. Inclusion criteria included (1) the object population is children (age <18 years), (2) diagnostic test accuracy studies, (3) studies detecting fever by infrared tympanic thermometers, and (4) studies using rectal thermometers as the reference standard. Exclusion criteria included (1) animal studies, and (2) studies lacking essential data.

Two reviewers (D.S. and LY.Z.) independently reviewed the titles and abstracts of these studies. Papers deemed to match the predefined inclusion criteria or without consensus were reviewed in full text. Disagreements were resolved through discussions and scientific consultations.

Quality assessment and data extraction

We adopted the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2)[20] and used Review Manager 5.3 for drawing the figure of risk of bias[21] Two independent reviewers (D.S. and LY.Z.) assessed the methodological quality of the included studies independently and disagreements were also resolved through discussions and scientific consultations.

The following data were extracted by two independent reviewers (D.S. and LY.Z)  from the included studies: (1) descriptive aspects: primary author, year of publication, country, setting, age, types of tympanic thermometer and reference standard; (2) statistical aspects: the size, number of observations, the cut-off of tympanic thermometer, the True Positive (TP), the False Negative (FN), the False Positive (FP) and the True Negative (TN), sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV).

Statistical Analysis 

Meta-analyses of TP, FN, FP and TN were performed to compare the test accuracy between tympanic temperature and the gold standard (rectal temperature) by MetaDiSc 1.4[22]. Threshold analysis was firstly conducted to evaluate the threshold effect[23]. The inconsistency index (I²) test was used to estimate heterogeneity between studies and I² >75% was considered to be with high heterogeneity[24]. Data were synthesized by using the random-effects model which was recommended in pooled estimates of diagnostic meta-analyses[25]. The area under the curve (AUC), Youden index and index Q* were used to measure test accuracy[26-28]. 

Results

Selection process

Initially, a total of 1739 articles were retrieved in a systematic search. Secondly, 629 duplicates were removed. Thirdly, the titles and abstracts of the remaining 1110 articles were examined and 1062 articles were excluded for diverse reasons. Finally, 11 articles were selected after the full text review and 1 article[29] was added by reviewing references. The process and outcome of the literature selection are presented in detail in Figure 1.

Risk of bias in included studies

Among these 12 included articles, 4 had a high risk of bias in some item and 8 had low applicability concerns in all domains. The risk of bias and applicability concerns in different domains are showed in Figure 2 and Figure 3.

Characteristics of selected studies

Twelve included studies were published from 2010 to 2018. All these studies applied the tympanic thermometer and set the rectal thermometer as reference standard. The descriptive and statistical characteristics of the 12 studies were presented in Table 1 and Table 2 respectively.

Accuracy of Tympanic Thermometry in children under different cut-offs

The 12 studies involved 4639 children. The cut-off points were various. Among the included articles, 7[5, 8, 15, 29-32] studies set the optimal cut-off and the other 5[3, 13, 33-35] studies analyzed the diagnostic test accuracy of tympanic thermometry under different cut-offs. The range of the cut-off point was from 37.0℃ to 38.0℃. Studies had data under same cut-off were synthesized.

Accuracy under the cut-off of 37.0℃

There was only one study[3] reported diagnostic test accuracy under the cut-off 37.0℃. In this study, for ear temperature (37.0°C), sensitivity, specificity, PPV, and NPV were 0.89, 0.84, 0.91, and 0.81 respectively

Accuracy under the cut-off of 37.25℃

Only one study[30] gave the optimal cut-off 37.25℃ and sensitivity, specificity, PPV, and NPV were 0.83, 0.86, 0.88, and 0.80 respectively.

Accuracy under the cut-off of 37.4℃

There was only one study[33] reported diagnostic test accuracy under the cut-off 37.4℃. In this study, for ear temperature (37.4°C), sensitivity, specificity, PPV, and NPV were 0.96, 0.36, 0.82, and 0.73 respectively.

Accuracy under the cut-off of 37.5℃

The cut-off 37.5℃ was used in 2 studies[31, 33] and a total of 390 pediatric patients were involved. The pooled sensitivity was 0.87 (95% CI 0.79-0.92) and heterogeneity between the articles was high: 87.5% (X²=8.02, P<0.05). The pooled specificity was 0.95 (95% CI 0.92-0.97) and heterogeneity between the articles was high: 97.9% (X²=47.74, P<0.05).

Accuracy under the cut-off of 37.6℃

The cut-off 37.6℃ was used in 4 studies[3, 33-35] and a total of 746 pediatric patients were involved. Spearman’s correlation coefficient of sensitivity and specificity was 0.089 (P = .638) and the ROC plane showed no curvilinear trend, suggesting that there was no heterogeneity from threshold effect. The pooled sensitivity was 0.76 (95% CI 0.71-0.80) and heterogeneity between the articles was high: 94.3% (X²=53.04, P<0.05). The pooled specificity was 0.88 (95% CI 0.84-0.91) and heterogeneity between the articles was high: 92.9% (X²=42.22, P<0.05) (Figure 5). The sROC AUC was 0.93 (SE = 0.02) while Q* value was 0.86 (SE=0.03).

Accuracy under the cut-off of 37.7℃

There was only one study[33] reported diagnostic test accuracy under the cut-off 37.7℃. In this study, for ear temperature (37.7℃), sensitivity, specificity, PPV, and NPV were 0.91, 0.60, 0.87, and 0.68 respectively.

Accuracy under the cut-off of 37.8℃

The cut-off 37.8℃ was used in 3 studies[13, 33, 35] and a total of 1795 pediatric patients were involved. The threshold analysis (r=－0.050 , P = .667) and the ROC plane (Figure) suggested that there was no heterogeneity from threshold effect. The pooled sensitivity was 0.92 (95% CI 0.90-0.94) and heterogeneity between the articles was high: 80.1% (X²=10.07, P<0.05). The pooled specificity was 0.91 (95% CI 0.89-0.92) and heterogeneity between the articles was high: 94.5% (X²=36.68, P<0.05) (Figure 6). The sROC AUC was 0.97 (SE = 0.02) while Q* value was 0.91 (SE=0.03).

Accuracy under the cut-off of 38.0℃

The cut-off 38.0℃ was used in 7 studies[5, 8, 13, 15, 29, 32, 34] and a total of 2783 pediatric patients were involved. The threshold analysis (r=0.429, P = 0.337) and the ROC plane suggested that there was no heterogeneity from threshold effect. The pooled sensitivity was 0.81 (95% CI 0.79-0.84) and heterogeneity between the articles was high: 93.7% (X²=94.51, P<0.05). The pooled specificity was 0.96 (95% CI 0.95-0.97) and heterogeneity between the articles was high: 81.6% (X²=32.56, P<0.05). The sROC AUC was 0.97 (SE = 0.01) while Q* value was 0.92 (SE=0.01).

The diagnostic test accuracy of the tympanic thermometry under different Cut-offs in the detection of pediatric fever is summarized in Table 3. The cut-off 37.8 is with the highest sROC AUC and Youden Index and is deemed to be the optimal cutoff.

Discussion

Although the tympanic temperature provides a good alternative for traditional measurements, it has been debated for the low reproducibility. However, since the ear thermometer came out, it has been constantly updated and upgraded. Some techniques have been used to improve the test accuracy, such as the Braun Welch Allyn Pro 4000 Thermoscan, where a heating element in the sensor heats the probe tip to just below normal body temperature to avoid cooling the ear canal[36]. And the improvements of geometry and algorithms have been developed to ensure that the displayed result reflects the tympanic temperature accurately[11]. Hence, the newer versions of tympanic thermometers might meet the clinicians’ requested improvements of repeatability in noninvasive temperature assessments.

Studies compared the test accuracy of tympanic temperature and other temperatures haven’t reached a consensus[1, 3, 13, 30, 37]. To identify the accuracy of the infrared tympanic thermometers, Zhen et al[38, 39] synthesized articles comparing infrared tympanic thermometer with rectal thermometer. Even if the results could be useful for clinical practice, two major issues in these two articles should be underlined. Firstly, the infrared tympanic thermometers have been developed and updated all the time, and some older versions of infrared tympanic thermometers have been obsolete, for instance, Thermoscan Pro-3000 (San Diego, CA) and First Temp 2000A (Intelligent Medical Systems, Carlsbad, Calif). Secondly, the cutoffs used to detect fever of the included studies were diverse. It is unreasonable to synthesize studies applying obsolete tympanic thermometers and different cutoffs.

We conducted this study to assess the discriminant validity of the new generation of tympanic thermometers for detecting pediatric fever determined by rectal thermometry and to find the optimal cutoff by performing a systematic review and meta-analysis of 12 studies. By new generation, we mean the infrared tympanic thermometers that were still in production and on sale according to the official websites of the manufacturers as we started our study. We included the tympanic thermometers under use and excluded the outdated ones so that the results could provide a reference for current clinical practice.

The cutoffs of tympanic thermometry ranged from 37.0℃ to 38℃ among these 12 included studies. After the synthesis of three studies, including 1795 children, we found the optimal cut-off of tympanic thermometry is 37.8℃. Under this cut-off, pooled sensitivity was 0.92 (95% CI 0.90–0.94), pooled specificity was 0.91 (95% CI 0.89–0.92), sROC AUC was 0.97 (SE = 0.02) and Q* value was 0.91 (SE = 0.03). The diagnostic accuracy in our study was far higher than a former systematic review[23], in which pooled sensitivity was 0.70 (95% CI 0.68–0.72), pooled specificity was 0.86 (95% CI 0.85–0.88), sROC AUC was 0.94, and Q* value was 0.87. Excluding articles applying obsolete tympanic thermometers and analyzing diagnostic test accuracy under different cut-offs may be the major reasons for this gap and the test accuracy of tympanic temperature may be underestimated.

What should be underlined is that the heterogeneity between the articles is very high, from 81.6–94.5%. The study objects of included studies are all children, who age from 0 to 18-year-old. But the age groups are various, for example, Duru et al[31] admitted neonates whose mean age is 6.63 ± 6.98 days, while Allegaert et al[5] enrolled children with a median age of 3.2 years (range 0.02 years to 17 years). The variation of age groups may be the major contribution to the high heterogeneity.

Although the results of our study can provide an important reference for subsequent researches and clinical applications, there are two limitations in our present study. We performed different sub-group meta-analyses based on the different cut-offs used. Unfortunately, in many of these analyses a limited number of studies are included. We concluded that 37.8℃ was the optimal cut-off just based on three studies, which seemed unconvincing. But considering that 1795 subjects were included for analysis under the cut-off 37.8℃, the conclusion was much more convincing.

Conclusion

Tympanic thermometry has a high diagnostic accuracy and is a good alternative for rectal thermometry in pediatric patients. The optimal cut-off of ear temperature for screening fever in children is 37.8℃.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and materials

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Funding

There is no funding source.

Authors' contributions

S.D. took part in the study design, literature research, assessments of research, data analysis and manuscript preparation. LY.Z. took part in the study design, literature research and assessments of research. HX.L. was the guarantor of integrity of entire study and led the study design.All authors read and approved the final manuscript.

Acknowledgements

Not applicable.

Abbreviations

References

Tables

Table 1 Descriptive characteristics of including studies

Table 3 Accuracy of tympanic thermometry under different cutoffs in children