Primary Outcome: Length of hospital stays after the index CPI (LOSTA) as a continuous variable.
Secondary Objectives: Length of Intensive care unit (ICU) stays after the index intervention (LOSICUA) and cost of hospitalization (both continuous) as well as readmissions and mortality (both categorical).
Study Institution and Setting: A 100-bed, tertiary- and acute-care private hospital.
Clinical Pharmacy Team: Three clinical pharmacists with 4, 7 and 8 years of clinical pharmacy experience. All are doctor of pharmacy or master of clinical pharmacy certificate holders. Clinical pharmacist with the shortest experience was also a recent hire.
Study Design: Usual care, eight-month-long, two-phase, non-interventional study.
Study Subjects: Inclusion and Exclusion criteria
Consecutive patients of all ages were included when admitted to the study institution with CPI to outpatients excluded due to the unavailability of outcomes.
Data Collection
Team documented daily usual recommendations using a standardized Microsoft Excel 2010 tool (version 14.0.4374.1000). Variables documented for each CPI can be found in Tables 1 (matched between accepted and rejected CPI) and 2 (unmatched). In the study database each row constituted a CPI. For each patient, there were multiple CPI in a given admission and patients may have been admitted more than once during the study period. Hence, for each patient during a given admission, some variables value may have been merely a repetition with every CPI allowing us to use that value in a consolidated CPI. Other variables varied with every CPI. Therefore, we consolidated the data to a maximum of one accepted and one rejected per patient per admission using arbitrary ordinal scores for the variables changing with every CPI. These scores were calculated with the details provided in the definitions subsection. Medication information from the local pharmacy and stock management databases were confirmed. Similarly, the laboratory information from the local laboratory database was reported. Lengths of hospitalization, readmissions, and costs were collected from the local admissions database. All these databases were built by Microsoft Access 2000 (version 9.0.6926 SP-3) and managed by the information technology (IT) department in the hospital. Status of the patient upon discharge, whether alive or deceased, was provided by the IT department.
Definitions
Phase I, from January 1st to March 31st, 2019, was a pilot to estimate the needed sample size. Phase II, was conducted from April 1st to August 31st, 2019. Again, Table 1 (36 matched variables) and Table 2 (18 unmatched) summarize all the 54 variables included in this model. As the reader may appreciate, some of these were continuous like age in years, total number of CPI, or the length of hospital or intensive care stay before the first index CPI in days (LOSBI and LOSICUBI). However, the majority were categorical. For example, in Table 1, our reader can see that some CPI was made in critical cases in the intensive care unit whereas others made in non-critical medical cases. Most categorical variables are listed such that a given factor is either present (Yes) or absent (No). For example, there were 55 CPI (11%) in the accepted group and 20 CPI (12%) in the rejected group on beta blocker medications. Obviously, beta blockers were matched between accepted and rejected CPI groups. Poly-pharmacy was a dichotomous variable with 1 for ≥ 8 regular medications and 0 otherwise. Previously published literature was used to identify medication and disease-related variables predicting hospitalization due to drug adverse effects (21).
Non-green antibiotics are a special traffic signal classification we use in antimicrobial stewardship programs. For example, meropenem is red, cefepime is orange, and levofloxacin is green. Red antibiotics are the most serious and require rigorous stewardship while green antibiotics undergo only limited control.
After a quick revision of all types of recommendations made, it was found that there were significant differences between safety and efficacy CPI, i.e. those made to prevent an adverse effect and those made to improve the efficacy of a treatment.
Another area of difference was between orders made to add or stop a medication. Differences were absent or little for the other types, and therefore, these were grouped under miscellaneous. Unreported or difficult to categorize CPI were grouped under “Not determined”.
Scores were calculated to consolidate all CPI to a maximum of 1 accepted and/or rejected per patient per admission. These scores took a value between 0 and 1. For example, CPI was considered complex (took the value of ‘1’) if it involved putting two or more pieces of information together. Simple CPI (took a value of ‘0’) if based on one direct piece of information; for example, if the patient was hypotensive, the simple CPI was to stop the antihypertensive drug. Follow-up CPI (took a value of ‘0.5’) would be based on a check of a new investigation and hence follow-up on a previous CPI. Therefore, problem complexity score is a total average % of all; complex, simple, and follow-up CPI made for a given patient during an admission. As the reader may see, the two groups were comparable in the complexity of CPI (1 versus 0.9 for accepted and rejected CPI groups, respectively, and ranges in brackets). Similarly, Problem intention score is a total % average of errors and problems documented for a given patient during the admission. CPI is a problem (took a value of ‘1’) if the physician tried to defend their original plan. It was an error (took a value of ‘0’) if the physician immediately agreed or explicitly clarified that they made a mistake. Clinical domain score is a % average of clinical (value of 1) versus operational (value of 0) CPI for that admission. An example of a clinical CPI is changing a dose. Whereas an example of operational CPI is to reuse a given stable intravenous medication vial for multiple doses. Clinical Prescribing step score is a % average of CPI for that admission made to the prescribing step in the medication use process. The consultancy score is a % average of the clinician approaching the clinical pharmacist (value of ‘1’) versus the clinical pharmacist approaches the clinician (value of ‘0’) during that admission. Outcomes driven score is a % average of CPI made based on outcomes versus those made with guidelines for that admission. Rejection score is % of CPI rejected in that admission. Combined clinical pharmacy (CP) success score is the % average of clinical pharmacist’s successful CPI averaged over that admission. The physician rejection score is the % of physicians’ rejection rates during the whole study period averaged for that admission recommendations. The diagnosis revision score is the % of all diagnoses that were revised or changed by the end of the admission. The acceptance or rejection of the CPI was entered as a dichotomous input variable.
Finally, outcomes studied were LOSTA, length of stay in the intensive care unit (LOSICU), LOSICUA, readmission, mortality, and cost of admission.
Data Analysis
Analysts consolidated the data for a total of one CPI in accepted or rejected groups per admission per patient. Variables for different CPI were simply set to the matching values or calculated scores described in the definitions section. Final inclusion and refinement of scenarios are presented in Figure 1. Our reader can see that a total of 1694 CPI was finally consolidated to 684 CPI, 519 in the accepted and 165 in the rejected groups, respectively. Univariate analyses were conducted to compare the rejected and accepted CPI groups (Table 1 and Table 2). Finally, the research team built, trained, tested, and cross validated ANN for the main and various outcomes.
Statistical Analysis
The data was analyzed using Stat Tool, version 6.3.0 (Palisades Corp) to generate P-values using Mann Whitney U Test unpaired groups for continuous data, Chi-squared (χ2) test for categorical variables, McNemar Test for categorical outcomes during re-assignment ANN analyses, Wilcoxon Mann Whitney Test for continuous paired outcomes during re-assignment.
ANN Model
Inputs included 54 variables, either matched (36 in Table 1) or unmatched (18 in Table 2), per patient. Diagnoses were made by clinicians and documented in the study form. An ANN model (Figure 2) was developed using NeuralTools, version7.6.0 (Palisade Corp., Ithaca, NY). This is a fully connected feed forward ANN. Just like in our former studies, the first hidden layer (1 per training CPI) ensures accurate performance (20). The second hidden layer (2 neurons, one nominator and one denominator) reduces dimensionality to drive ANN toward fast convergence (i.e., an optimal solution that can be reliably used to predict outcomes) (20). Input layer consist of one neuron for each input variable. This would be 54 for the total or just 18 if only unmatched variables are included. An additional categorical input neuron may be added for the status of CPI accepted versus rejected. Cross validation set consisted of random 4 scenarios after initial training (544, 80%) and testing (136, 20%). Sensitivity report figures showed that this distribution of training, testing and cross validation resulted in no over fitting (available upon request). Reassignment and live predictions enabled the study of effect of maximizing acceptance of CPI from 80% to 100%. In addition, model automatically generated variable impacts (VI), an overall % contribution of a given variable to predict outcome. These were simply compared for all variables.
Sample Size Calculation
Authors used The University of California, San Francisco (UCSF) sample size calculator site (URL: http://www.sample-size.net/sample-size-study-paired-t-test/, Accessed November 27, 2019). Using Phase I data, a standard deviation of change with 80% acceptance versus 100% acceptance of CPI of 1.05 was used. At two-tailed-alpha of 5% and power 80%, a sample size of 554 consolidated CPI to detect an effect size of about 3 hours (0.125 days) difference in LOSTA was needed.
ANN model for the matched variables would expertly need 19*10*2 = 380 sample size. But for a larger ANN including unmatched variables the sample needed would go up to 54*10*2 = 1080.