A comparative analysis of the InterVA model versus physician review in determining causes of neonatal deaths using verbal autopsy data from Nepal


 Background: Verbal autopsy is a common method of ascertaining the cause of neonatal death in low resource settings where majority of causes of deaths remain unregistered. We aimed to compare the causes of neonatal deaths assigned by computer algorithm-based model, InterVA (Interpreting Verbal Autopsy) with the usual standard of Physician Review of Verbal Autopsy (PRVA) using the verbal autopsy data collected by Morang Innovative Neonatal Intervention (MINI) study in Nepal. Methods: MINI was a prospective community intervention study aimed at managing newborn illnesses at household level. Trained field staff conducted a verbal autopsy of all neonatal deaths during the study period. The cause of death was assigned by two pediatricians, and by using InterVA version 5. Cohen's kappa coefficient was calculated to compare the agreement between InterVA and PRVA assigned proximate cause of death, using STATATM software version 16.1. Results: Among 381 verbal autopsies for neonatal deaths, only 311 (81.6%) were assigned one of birth asphyxia, neonatal infection, congenital anomalies or preterm-related complications as the proximate cause of death by both InterVA and PRVA, while the remaining 70 (18.4%) were assigned other or non-specific causes. The overall agreement between InterVA and PRVA-assigned cause of death categories was moderate (66.5% agreement, kappa=0.47). Moderate agreement was observed for neonatal infection (kappa=0.48) and congenital malformations (kappa=0.49), while it was fair for birth asphyxia (kappa=0.39), and preterm-related complications (kappa=0.31); but there was only slight agreement for neonatal sepsis (kappa=0.19) and neonatal pneumonia (kappa=0.16) as specific causes of death within neonatal infections. Conclusions: We observed moderate overall agreement for major categories of causes of neonatal death assigned by InterVA and PRVA. The moderate agreement was sustained for the classification of neonatal infection but poor for neonatal sepsis and neonatal pneumonia as distinct categories of neonatal infection. Further studies should investigate the comparative effectiveness of an updated version of InterVA with the current standard of assigning the cause of neonatal death through longitudinal and experimental designs.

Methods: MINI was a prospective community intervention study aimed at managing newborn 28 illnesses at household level. Trained field staff conducted a verbal autopsy of all neonatal deaths 29 during the study period. The cause of death was assigned by two pediatricians, and by using 30 InterVA version 5. Cohen's kappa coefficient was calculated to compare the agreement between 31 InterVA and PRVA assigned proximate cause of death, using STATA TM software version 16.1.   Globally, an estimated four million neonatal deaths occurred annually in low and middle income 53 countries (LMICs), of which the majority occur outside of the formal health care system (1). While 54 the global annual neonatal mortality rate (NMR) decreased by 51%, from 36.6 to 18.0 deaths per 55 1000 live births between 1990 and 2017, it is estimated that 27.8 million neonatal deaths will occur 56 by 2030, if the current rate of reduction continues in each country (2). Availability of nationally 57 representative vital registration data on neonatal mortality is limited to about 60 countries and 58 neonatal mortality rate is higher in LMICs that do not have a high-quality vital registration data 59 (2). However, information on causes of death is vital to researchers, program planners and 60 policymakers working at local, national and international levels to improve infant survival. In order 61 to ensure the best possible utilization of limited resources available in such a setting, reliable and 62 adequate information about the causes of neonatal deaths is essential. A practical and the most 63 commonly used method to determine probable causes of death at population level in such settings 64 where systems for medical certification of causes of death are weak or non-existent is verbal 65 autopsy in which a series of questions are asked to the primary caregivers for specific signs and 66 symptoms of the deceased (3, 4). Verbal autopsy tools have been developed over the last few 67 decades (4-10) and used for assigning causes of neonatal death in numerous settings and contexts 68 and using different methods of assigning cause of death (11-24).

69
After conducting the verbal autopsy, the collected information is analyzed to assign causes of 70 death. Causes of death assigned by Physician Review of Verbal Autopsy (PRVA) is most 71 commonly used as a reference standard although multiple automated methods using data-driven 72 algorithms have been developed and tested for assigning cause of death using information 73 collected from a VA (25). However, PRVA is labor intensive, and prone to inter-observer 74 variation. InterVA (Interpreting Verbal Autopsy), is one of the computer-based probabilistic model 75 based on Bayes' probability theorem that has been compared with PRVA in some settings (24, 26-76 28). InterVA offers a promising alternative to expensive and time-consuming physician review in 77 assigning the cause of death in low resource settings (4) (29). InterVA-5 was developed to 78 harmonize with InterVA-4 and WHO 2016 VA standards, which is important for monitoring long-79 term trends over periods when different VA standards have been used (30). Although InterVA is 80 an affordable and available option to assign causes of death using verbal autopsies, users need to 81 be aware that there is no adequate evidence of equivalence, if not superiority, of its performance      Table 1.  InterVA as an acceptable substitute for PRVA.

269
The verbal autopsy is carried out by medically untrained enumerators from usually illiterate or just 270 literate parents in rural settings. So, defining the cause of death by strict criteria using verbal 271 autopsy data may lead to under-or over-estimation of neonatal deaths due to inability to get the 272 right information, which is sensitive and specific enough to make a diagnosis. In particular, early InterVA-5 offers the COMCAT functionality which categorizes the circumstances of death (30).

286
There may be recall bias on late interviewing after the neonatal deaths. However, verbal autopsy 287 taken between 3 to 12 months resulted in comparable results with those taken within 3 months of 288 death (39). Some interviews in our cohort were repeated due to lack of meaningful information on 289 the first interview, which could have introduced bias. InterVA and PRVA used the same set of VA Availability of data and materials:

339
The datasets used and/or analysed during the current study are available from the corresponding 340 author on reasonable request.

342
The authors (DD, PD, DA, NM, DN) declare that they have no competing interests.

369
The opinions expressed herein are those of the authors and do not necessarily reflect the views of 370 any concerned agency.

371
The authors wish to dedicate this article to the late Professor Peter Byass, for his relentless efforts 372 in developing the InterVA model and providing support for analyzing our data using InterVA-5.