Multi-criteria Decision Analysis for Health Technology Assessment of Intensive Care Ventilators for Pediatric Patients.

The technological complexity and heterogeneity of intensive care ventilator models currently available on the market together with the heterogeneity in pediatric patients (0 to 18 years old), make the choice of the best machine for pediatric healthcare setting crucial. This paper is aimed at addressing all the critical aspects linked to the implementation of intensive care ventilators in a pediatric setting, highlighting the most relevant technical features and describing the methodology to conduct health technology assessment (HTA) for supporting the decision-making process. Four ventilators models were included in the assessment process. A decision-making support tool (DoHTA method) based on Analytic Hierarchy Process, was applied. 28 Key Performance Indicators (KPIs) were identied, dening the safety, clinical effectiveness, organizational, technical, and economic aspects. The Performance scores of each ventilator have been measured with respect to KPIs integrated with the total cost of ownership (TCO) analysis, leading to a nal rank of the four possible technological solutions.


Results
The nal technologies' performance scores re ected a deliver valued, contextualized, and shared outputs, detecting the most performant technological solution for the speci c hospital context. HTA results had informed and supported the pediatric hospital decision-making process.

Conclusions
This study, identifying and discussing the pros and cons of innovative features of ventilators and all the evaluation criteria and aspects to be taken into account during the evaluation process, can be considered as a valuable proof of evidence as well as a reliable and transferable method for conducting a decision making process in a hospital context.

Background
Mechanical ventilation has been considerable progressing over the last 20 years, thanks to the development of more e cient and sophisticated technical features.
It has gradually led to the development of different ventilation modes that had signi cantly improved the clinical effectiveness of ventilation techniques and patient-ventilator synchrony (1), leading in the meantime to reduction of time spent on ventilation, staff workload and intensive care unit (ICU) costs (2) (3).
Even if the variety of currently available ventilation modes and technical features are a great chance to solve respiratory failures, health professionals need to be aware about the technology and the risks that they could bring to the system (4).
For this reason, a comprehensive evaluation of the technical characteristics of ICU ventilators results pivotal to understanding the strengths and weaknesses of each ventilator, particularly because the ventilators' e cacy provided by manufacturers may be signi cantly different from the clinical effectiveness in a real clinical setting.
The complexity of pediatric ventilation is rstly linked to the variable size of patient ranging from premature patient below 1 kilogram to adolescents weighing 70 kilos. For this reason, the heterogeneity in patients' age and size should be carefully taken into account during an evaluation process. For instance, as non-invasive ventilation (NIV) and high ow nasal cannula oxygen (HFNC), are the established therapies within the pediatric ward (5), in order to be able to treat all possible kinds of patients, each ICU bed should be provided with a ventilator that includes NIV and HFNC functions as well.
In addition to the technological complexity and the heterogeneity of patients, the heterogeneity of ventilator machines is another aspect to be carefully analyzed.
Moreover, the presence of several ICUs in the same hospital, with high care complexity and with different medical specialties, makes the theme of technologies' heterogeneity crucial. Major issues are associated to the learning curve of healthcare professionals in transferring the theory into clinical practice. The learning curve theory is usually based on homogeneous products. The higher the heterogeneity of intensive care ventilator models within the same clinical setting is, the higher the di culties in covering the clinical needs with enough skilled health professionals able to manage each of these complex technologies.
Another critical aspect associated with intensive care ventilation are costs. Recent years have been characterized by a signi cant increase of healthcare costs (6,7). Expenses of Intensive Care unit (ICU) usually cover up to 20% of overall hospital's costs (8) especially when considering those patients who require prolonged mechanical ventilation (9).
All the critical aspects mentioned above make the selection of the intensive care ventilator model that best ts with the speci c healthcare setting needs di cult. It exposed the problems in taking a wellinformed decision about a suitable technology throughout an effective and reliable tool for prioritization.
Considering the high complexity of decision-making process for ICU ventilators selection and taking advantage of the renewal plan for the intensive care ventilators, the hospital management of Bambino Gesù Children's Hospital requested a detailed Health Technology Assessment (HTA) study aiming at the replacement of the intensive care ventilators in three clinical departments.
The primary aim of this study, indeed, was to provide an exhaustive overview of the assessment criteria that should be considered when comparing the performances of different ventilators models and to show an effective and reliable evaluation method. The secondary aim was to give clinicians a comprehensive and relevant report on these performances. The paper describes in detail the evaluation process that has been conducted within our hospital context offering a clear examination of the ventilators' impact on the possible risk and clinical bene ts for patients, the entire hospital organization, and the hospital budget.
Thanks to HTA process that represents a multidisciplinary process and a method of evidence synthesis, it was possible to determine the ICU ventilators' main technical characteristics, to measure their performances, to analyze the pros and the cons of different ventilators' models, and to sum up all the information gathered and elaborated in an evidence-based nal recommendation, actively supporting the complex decision-making process. Different aspects such as clinical effectiveness, safety, costs, technical, organizational and social, ethical, and legal aspects of the ventilators were analyzed. According to the literature, complex decision-making processes have been usually carried out through Multi-Criteria Decision Analysis (MCDA) method in several elds, even though very few applications to guide resource allocation decisions in health care have been recorded, so far. As MCDA results an effective and reliable tool able to identify the criteria and their weights for priority setting, resulting in a rank ordering of interventions, we decided to adopt MCDA as an integral part of the HTA evaluation (10), as an analytical quantitative instrument focused on supporting the decision-making process between alternative products.

Methods
As the HTA is a multidisciplinary and multidimensionality process, a working group, composed by professionals with different professional skills, was established to identify all the pertinent aspects to be analyzed in the assessment of different ventilator systems. The working group involved eighteen professionals: twelve medical doctors; ve Biomedical Engineers; one Health Economist. Table 1 illustrates the different responders' professional pro le and hospital's Department, which they are addressed to. The HTA process was carried out integrating the evaluation conducted following the EUnetHTA CoreModel©, as a guideline for HTA processes [10] (11), and the Analytic Hierarchy Process(AHP), as outlined in Decision-oriented Health Technology Assessment (Do-HTA) method (10) and graphically schematized in Fig. 1. This method proposes to organize the collected evidence of the selected technology and the consequences of its use in different Key Performance Indicators (KPIs), which are arranged in a hierarchical decision structure (cf AHP). The integration of the analytical approach (AHP) within the qualitative assessment tool (EunetHTA), allows quantifying opinions and transforming them into a coherent decision model assessing all factors and their interactions in a decision domain, leading to de ne a nal rank of several health interventions. The process was described in the owchart presented in Additional File 1 ( Figure S1). The idea is that a decision problem can be decomposed into a hierarchy of more sub-problems each of which can be measured and analyzed independently. AHP mathematical process using pairwise comparisons, eigenvector method for deriving weights, and a method to verify the "consistency" of judgments, is able to integrate all KPIs evaluation in a nal numerical result, which represents the alternatives' relative ability to achieve the decision goal. The whole process, previously published by the authors under the name of DoHTA, it is characterized by seven steps described in Fig. 1.
The rst phase of the HTA process is aimed at de ning and detailing the decision problem. The aim of the HTA process was to select the ventilator model that best ts with the hospital needs. Within the variety of models and manufactures currently available on the market, the working group drew up the minimum tenders' requirements that the ventilators should have to be considered eligible for the assessment process. More speci cally all the ventilators' models must have a touch screen < 21'' and must be approved for neonatal and pediatric patients. Finally, four ventilators models that meet the inclusion criteria were selected. The HTA process was carried out involving the three hospital departments that took part to the renewal plan: The Intensive Care Unit (ICU), the Emergency Department and the Cardiac ICU. Health professionals tested for a month each model involved in the study.
The aim of the study is to show the method used to carry out the decision making process highlighting the KPIs that were used to assess the ventilators models. The KPIs identi ed and the method presented here will be extremely helpful for other hospitals (pediatric or not) that have to select the intensive care ventilators that best feet with their needs, especially during COVID-19 pandemic.
The entire HTA process will be shown as a case study calling the different ventilators' models as Model 1, Model 2, Model 3 and Model 4.
After de ning the decision problem, a preliminary study was conducted to acquire a general overview of the subject and to de ne the main criteria representing discriminant assessment factors between the new technology under evaluation and the state of art.
The main evaluation criteria (Domains) were identi ed, by the working group, among those singled out by EUnetHTA Core Model© (11), which represents the European guideline for each HTA process and intends to coordinate the collected evidence of the selected technology in different "assessment elements". In this way, the technological innovation, could be analyzed according to safety, clinical effectiveness, costs, social, ethical, legal, and organizational aspects.
Each evaluation domain was better de ned throughout several KPIs, in order to detail the decision problem and describe which aspect of each evaluation area, the working group intends to take into account. To outline a clear, detailed, and proper de nition of the indicators, a literature search was carried out. It aimed at identifying the detailed indicators through which the safety, e cacy, costs, organizational and technical aspects of intensive care ventilators could be evaluated. More speci cally, studies that contain relevant information on safety ventilators related aspects, on clinical effectiveness and economic evaluations were considered for the inclusion. According to AHP, the KPIs identi ed were arranged in a hierarchy decision tree. More speci cally, the "goal of the decision" was split into the main evaluation criteria (i.e. the domains), each one is divided in turn, into KPIs, covering all the pertinent aspects to be analyzed in the assessment of the different ventilators' systems.
All indicators and domains were judged in terms of performances' values (Assessment phase) and relative weights (Appraisal phase) through pairwise comparisons. Taking into account the evidence gathered from the literature, together with the professionals' expertise, after the ventilators' test period, each member of the working group provided his judgement on the ventilators' performances (in terms of KPIs) as well as on the relative weights of each assessment element of the decision tree (12). Results of AHP method reveals the global weight of each KPIs, indicating the relevance of each assessment element within the overall evaluation as well as their nal performance values, providing a priority ranking of all alternatives technologies in terms of their overall preferences.

Economic evaluation
The economic evaluation was carried out considering all business proposals and comparing all purchase costs. The analysis of Total Cost of Ownership (TCO) has also been carried out. It is a nancial estimation focused on helping managers to determine direct and indirect costs of a product or system. This analysis informed the economic and evaluation domain results.

Sensitivity analysis
Finally, a sensitivity analysis was performed to test the stability and the robustness of the alternatives' ranking. The analysis was conducted comparing the models with the highest performance value, to identify the elements representing source of uncertainty and to determine the impact of this variability on the stability of the assessment results. It was applied to both weights and performance system. Sensitivity analysis was carried out by calculating the minimum changes on values of each criterion needed to reverse the current ranking of alternative technologies (13).

Evidence gathering
Throughout the critical review of the literature, integrated with expert opinion and information from devices' manufacture's companies and tenders' speci cations, the working group selected 28 KPIs which represent the main factors for discriminating the technologies under evaluation.

Hierarchy construction
The working group decided to include in the analysis ve domains: safety, clinical aspects, costs and economic evaluation, technical characteristics, organizational aspects, whereas Legal, social and ethical domains (which, usually, are included in the Core Model©) were not included because they do not represent discriminant factors between the technologies under evaluation. Each domain was then described by a subgroup of the 28 KPIs previously identi ed. A decision tree was designed and showed in  More speci cally, as the aim of this HTA process was to compare the performances of four ventilators models, each KPI was described by a number of tenders' speci cations, each of which is measurable and intrinsically provides an effect within the related domain. To give an example, it is believed that better complications rate (patient safety), might be reached if the ventilator's model guarantees the minutevolume ventilation mode or if it provides the compliance and resistance monitoring system. Detailed information and description of Lev 1 and 2 KPIs are shown in Table 2.  Table 3 and graphically represented in Figs. 3 and 4. More speci cally, the ring plot (Fig. 3), represents the uni ed weights' system considering the results of all the involved professionals' judgements, pertaining to the "domains" layer of the decision tree, as gathered from pairwise comparisons and mathematical calculations in accordance with the AHP method. Results showed that the most important aspects to be considered when comparing different ventilators' models are the safety and the clinical effectiveness reaching respectively 45,95% and 31,47% of the total weight, followed by the organizational aspects (9,51%), technical characteristics (7,80%) and costs & economic evaluation (5,27%). Detailed information about each indicator weight are listed in the Table 3 (2nd  column).   More speci cally, within the safety domain the "patients' safety" weights twice the "technological risks". The "patient safety" KPI, indeed, covers the 70% of the weight of the domain. It is most represented by the "complication during ventilation" that doubled the weight of "Complication post ventilation".
The "alert for ventilators' parameters" represents the most critical KPI regarding the "technological risks". The others KPIs (safety mechanisms, risk related to the heterogeneity of ventilators models and the adverse events/recalls) assume more or less the same relevance within the evaluation.
The clinical effectiveness domain, which represents the second domain in order of importance, is described by a number of technical speci cations which more or less equally affect the clinical outcomes such as the possibility to customize the ventilation, patients comfort related to the availability of noninvasive ventilation mode, the reduction of ventilation time and the weaning time.
The technical characteristics domain was mainly described by different aspects of the technology management and by a number of speci cations emerged from technical documentation of different devices' producers.
Regarding the organizational aspects, the impact that different operating principles of technologies have on the organization doubled the "interface with other healthcare systems or technologies". More speci cally, the former includes the analysis of the "ease of use and ergonomics of technology", the "maintenance", the "versatility to use the technology with pediatric patients" and the availability of homogeneous technologies, compared to the latter that has been measured in terms of availability of beds where the technologies could be installed.
Finally, results on economic evaluation, aimed to compare all business proposals considering all purchase costs and hypothesizing 10-year life-cycle cost estimate per each ventilator. Table 3, as an example showed the comparison within the four models.
The histogram chart (Fig. 4) It resulted that Model 2 provides several technical functions that, limiting the complication during ventilation, improve the patient safety, including trigger mechanisms, apnea alarm, resistance and static dynamic compliance monitoring, lung recruitment tools (PV loops), higher number waveforms simultaneously displayed and adjustable screen for ergonomics. It emerged that the necessity for endotracheal intubation is reduced if the ventilator model provides compensation for large leaks during NIV, trigger ow mechanism and tube compensations. All these aspects are better represented by Model 2 that is able to provide the best performances with respect of these technical characteristics. Model 1 provides control of optimal cuff pressure during the entire ventilation period, helping to prevent the complications post-ventilation such as ventilator associated pneumonia (VAP) and tracheal injuries.
To reduce the technological risks, Model 2 provides more programmable alarms than the other models as well as a system control for potentially dangerous involuntary changes to ventilation settings.
In relation to the clinical effectiveness, the Fig. 4 shows that the Model 2 outperforms the other Models even though performances score is quite like that reached by The technical characteristics domain was described by a number of speci cations emerged from technical documentation of different devices' manufacturers and by different aspects of the technology management. Even regarding this domain, Model 2 resulted the best performant (Fig. 4). In detail, as mentioned before, Model 2 provides more advanced technology features to make easier and more intuitive the visualization of information during ventilation such as adjustable screen and the possibility to simultaneously visualize more than one curve or loop. Model 2 also ensures high level accuracy and the availability of a wide measure range of monitored parameters (e.g., tidal volume, respiratory rate fraction of inspired oxygen, etc..). Moreover, regarding the post-sales service, while the manufacturer of Model 2 offers higher preventive maintenance frequency and continuous clinical and technical assistance than the others, the manufacturers of all four ventilators models offer the same conditions in terms of support services and training for biomedical technicians.
Organizational aspects analyze different aspects of the management of the technology and the impact of its introduction on the existing work ows.
Model 2 also performs better considering the ease of use and the ergonomics of the technology, providing some technical characteristics (i.e. the adjustable screen and the possibility to simultaneously visualize more than one curve or loop technology) that make the ventilator more usable and enhance the human-machine interface aspects. Moreover Model 2 is also the more versatile model for the pediatric use, because of the wider range of tidal volume.
Finally, results on economic evaluation, as showed in Table 4, aimed to compare all business proposals considering all purchase costs and hypothesizing 10-year life-cycle cost estimate per each ventilator.
From this analysis it resulted the Model 4 as the best option with a total cost of ownership of about € 30.742,66. The highest performance score, indeed, is attributed to Model 4 (the most cheaper) with 5.27% in comparison to Model 1, 3 and 2 (2.79%, 1.24% 1.03%, respectively) (Fig. 4). Sensitivity analysis Sensitivity analysis was carried out comparing the models with the highest nal performances value to test the stability and reliability of the solution. The Model 2 seems to be the best option, even if its total performances score is very close to that of Model 3 (89.76% vs. 87.49%, respectively). The more similar the different models' nal scores are, the higher the probability to reverse the nal solution. As the nal performances value of the Model 2 has led only 2.3% over Model 3, the sensitivity analysis was carried out between these two models.
Sensitivity analysis results on performances showed that to reverse the current ranking of the two different models, at least one of the following conditions have to be veri ed: A 3% reduction of Model 2 safety performance and a simultaneous 3% increase of Model 3 safety performance.
A 5% reduction of Model 2 clinical effectiveness performance and a simultaneous 5% increase of Model 3 clinical effectiveness performance.
Results showed the instability of this solution. A minimum variation in the evaluation parameters nal scores (3% in safety and 5% in clinical effectiveness), might reverse the nal decision. It can result in a substantial equivalence between the Model 2 and Model 3.
Results on weights, instead, showed that to reverse the current ranking of different models a 3.5-fold increase of cost and economic evaluation weight and a simultaneous 14% reduction of the other dimensions' weight should be occurred. It revealed, instead, a greater stability of the solution from the weight system point of view because such level of importance to costs, instead of organizational or clinical effectiveness aspects, has been considered inappropriate by those responsible for the decision making process in our hospital.

Discussion
This project was commissioned by the hospital management for the renewal of all hospital's ventilators.
For these reasons, an HTA process was carried out in order to identify the best suitable model of critical care ventilator available on the market by comparing four different models.
The Do-HTA method (10) was used to yield a structured outcome able to support the decision-making process concerning the nal decision on choosing the best technology to be adopted. Therefore, evidence gathered from the literature review and professionals' expertise and judgments per each unit, with their experience and knowledge, were integrated as results of Do-HTA method.
An important point of this study was the identi cation of KPIs useful to compare the ventilator models and the construction of the decisional hierarchy tree. The main informative indicators we found were about safety, clinical effectiveness, technical characteristics, cost & economic evaluation, and organizational aspects. The safety and the clinical effectiveness have been de ned as the most important domains within the assessment. They covered more than 75% weight of the total evaluation.
As regards the safety domain, the most important indicator into the patient safety (Lev-1 KPI) is the "complication during the ventilation", followed by the "complication post-ventilation" and "necessity of endo-tracheal intubation". Data from literature review showed that an inappropriate use of ventilators may cause complication during ventilation called ventilator-induced lung injury (15). To avoid that, several innovative technical functionalities have been improved, including trigger mechanisms, apnea alarm, resistance and static dynamic compliance monitoring, lung recruitment tools (PV loops), higher number waveforms simultaneously displayed and adjustable screen for ergonomics. Regarding the complication post-ventilation, a recent prospective cohort study showed that the hospital mortality occurrence is related to the incidence of postoperative lung complications (16), even the ventilatorassociated pneumonia (VAP), which is common for pediatrics' patients in ICU, signi cantly affects the morbidity and mortality of pediatric patients (17) (18) (19). The evaluation of this KPI (complication postventilation) has been conducted considering the management of endotracheal tube cuff pressure, representing an important element for the prevention of ventilator-associated pneumonia in patients receiving mechanical ventilation.
Some studies showed that endotracheal intubation-related complications are likely to occur in intubated pediatric patients. (20). This issue might be avoided if the ventilator model is able to optimize the NIV (non-invasive ventilation) operation, which is increasingly being used in pediatric units (21). Some studies showed that the use of NIV in pediatric patients, provides a rapid improvement of the respiratory acute injury, avoiding the requirement for endotracheal intubation reducing the related complications (22) (23).
In relation to the technological risks indicator, based on our analysis, the "alerts for ventilators parameters", resulted to be the most important one. It could be observed, according to the literature and even after the evaluation process, that the patient safety could be, even fatally, compromised, when a situation requiring an alarm activation, is not recognized by the ventilator (21). The "technical alarms" KPI is referred to the potential issues arising from ventilator's malfunctions (24) that might be overcome by the availability of programmable alarms. Ventilators also may have safety mechanisms including control system for involuntary ventilator settings changes, calibration of ow and oxygen sensors and RFID technology (25).
The homogeneity of technologies resulted a pivotal aspect in the evaluation. Ensuring that as many ventilators as possible belong to an identical make and model, reduces the learning curve of operators as well as the risk of errors. Finally, while choosing a ventilator model it is important to be aware of the related adverse events and/or recalls. These aspects are evaluated into the "other technological risks for patients".
Comparing the performances of the four ventilators models with respect of these safety indicators, results showed that the Model 2 seems to be the safest one between the four technologies compared.
Evidence helped us to nd out more about the clinical effectiveness domain and its indicators. Its Lev-2 KPIs are: "customization of ventilation", "patients' comfort", "reduction of ventilation time", "reduction of weaning time". According to the literature and based on the assessment, customized patient-ventilator interface may be useful in long-term ventilation, improving the patient-machine interaction. A number of advanced ventilation modes and innovative technical speci cations (resistance and static dynamic compliance monitoring, trigger mechanism, lung recruitment tools -PV loops-, programmability of frequency and amplitude of sigh breath function) guarantee the customization of ventilation. Literature review also showed that the improvement of patient/device interaction is pivotal for a better patient comfort as well as for the reduction of ventilation time. Optimizing the comfort for the patient represents also an important goal (26). The duration of mechanical ventilation has to be as short as possible and to this end, in case of neonatal patient, the ventilator system, throughout the variable support pressure ventilation mode, should provide a synchronous interaction with the infant (27). The optimization of the duration of ventilation time might reduce the incidence of post-operative complications and can improve patients' clinical outcomes (28). Finally, the reduction of weaning time is another important aspect to be considered in the choice of the most appropriate ventilator model. Ventilators with automated weaning modes can overcome the problem that clinicians might under-recognize the patient's ability to breathe without assistance, prolonging ventilation time and increasing the complications incidence (29).
The technical characteristics domain has been characterized by several technical speci cations gotten from the different devices' manufactures and from some websites as ECRI and similar others. Technology features refer to the ventilation parameters visualization (adjustable screen, the possibility to simultaneously visualize more than one curve or loop, ease to visualize information during ventilation) and to other technical aspects as the availability of ventilators accessories and consumables such as nebulizer, the possibility to export data to EHR as well as accuracy and the availability of a wide measure range of monitored parameters. Moreover, also the management of the technology plays a crucial role, considering the importance attributed to the post-sales service, maintenance, and the training course for the users.
In relation to the cost and economic evaluation, all business proposals have been analyzed. In this way it was possible to make an evaluation of the economic trend relative to the purchase of all consumables for the use of the device. The analysis of total cost of ownership has been carried out (30). A detailed analysis of all service intervals and preventive maintenance requirements per each ventilator model was carried out. It was also looked for pricing information for consumables, repair parts, preventive maintenance kits. Based on the information collected and on discussions with companies, health professionals, economists, and biomedical experts, it was estimated the annual cost of ownership value per each ventilator. At this stage, a 10-year life-cycle cost estimate per each ventilator was calculated.
Results showed that Model 2 resulted more expensive than the others, because of its peculiarities.
Our analysis ended with the organizational aspects domain. Such analysis helped us to make a prevision of the impact that the deployment of new ventilators might have on existing healthcare systems (1). In this domain it was possible to enhance two aspects strictly related to the selection of the ventilator model that best meets the hospital needs: the ventilator operating principles that affects the organizational aspects and the impact of the technology on the existing work ows. As for the former, KPIs included the "ease of use and ergonomics of the technology" (technical characteristics that make the ventilator more usable and enhance the human-machine interface aspects, affecting the mental workload), the "maintenance aspects", the "versatility to use the technology with pediatric patients" due to the possibility to arbitrarily set the minimum and maximum Tidal Volume values and the "homogeneity of devices" related to the availability of portable and or neonatal ventilators of the same manufacturer or the availability of compatible accessories, to be used in the same department or in the whole hospital. The latter aspect referred to the availability of ICU beds and to the possibility to decrease the length of ICU stay if a reduction of the mechanical ventilation time occurred (31). This aspect was carefully taken into account and it was associated to the ventilator model's availability of advanced weaning ventilation modes. However, the literature review results showed no signi cant evidence associating with the reduction of weaning ventilation time and/or ICU stay when these advanced ventilation modes are used (32, 33).
To sum it all up, from our analysis it can be summarized that the safety (in particular that one relative to the patient safety) and the clinical effectiveness indicators resulted as the most important aspects.
The evaluation of the performance value relative to all parameters, as above described, has led to de ne a rank between the four models considered in the analysis. Model 2 reached the best performance value, followed by the other models. However, from the economic analysis and from a performances values point of view, even though Model 2 resulted more expensive than the others, because of its peculiarities, it may be considered as a great added value for the Hospital.
As presented in the results section, a substantial equivalence can be observed between Model 2 and 3 performance values: Model 2 has led, indeed, only 2.3% over Model 3. For this reason, the sensitivity analysis was carried out and it emerges that the nal ranking is not robust, because with a minimum parameter variation (3% in safety performances and 5% in clinical effectiveness performances), the nal decision can be inverted, con rming a substantial equivalence of the two products. From the weight system point of view, instead, results revealed a substantial stability of the nal ranking. It means that the weight system elaborated is robust and coherent with the decision problem even if some changes occurred in the professionals' judgements on the relative importance of the evaluation criteria.
Do-HTA method, therefore, has permitted to evaluate the performances of the selected ventilators, providing reliable evidence about the four ventilators models and therefore supporting the hospital decision makers.

Conclusion
This paper provided a general overview of the most relevant features to consider in the assessment of intensive care ventilators for pediatric patients. It discussed the main characteristics of these technologies as well as the potential implications of introducing their different innovations (proposed by the manufacturers involved in this study) into the pediatrics ICU/Emergency units. The Hospital Decision Makers, after the HTA process which has entailed evaluating the weights' system, averaging departments' preferences, and performing sensitivity analysis, rationally decided to conscientiously and objectively assign the rank of best performer to the Model 2.
Our evidence-based approach, identifying and discussing the pros and cons of innovative features of ventilators and all the evaluation criteria and aspects that be considered when a new ventilator model has to be acquired, can be deemed as a valuable proof of evidence that conducting an HTA process before the acquisition of a technology is essential to make the best, most rational and reliable decision. This paper also provides detailed information on the methodology used to carry out the assessment. Thanks to sensitivity analysis, results appear reliable and, if accurately adapted, they could be transfer to similar hospital contexts.