Development of a Prioritization Model to Compare Emergency Psychiatric Coverage Service Options

Background: Reducing Length of Stay (LOS) is an important way for hospitals to improve emergency department (ED) costs and outcomes. Psychiatric patients represent a challenge to reducing LOS when the scarcity of psychiatric specialists leads to longer LOS. Previous literature describes the unique solutions different hospitals have employed across the US, but does not share methods for evaluating or selecting a solution that can be applied to other hospitals. Methods: We conducted a review of hospital ED case data, market research on psychiatry services, and interviews with hospital staff. This information, along with projected return on investment, was aggregated to create a holistic model for evaluating different service options and selecting the one with the best t. Results: To develop a prioritization model that identies the one psychiatric service improving psychiatric LOS and best tting the hospital’s overall priorities and operations, our methodology identied 8 key factors that captured the overall diculty of implementation and benets associated with each service option. Conclusion: The Prioritization Model created in this study was instrumental in selecting the solution for reducing LOS in a way that best meets patients’ and the hospitals’ needs. This model may be applied to other hospitals and service evaluations to provide a holistic review and direct comparison of opportunities.

Despite the wealth of literature focused on how telepsychiatry services have been implemented, only some address nancial costs [6-9, 12, 18-19]. The available models focus primarily on realized costs post implementation without guidance for estimating costs prior to implementation [7,19]. In addition, there is a scarcity of research de ning the attributes and outcomes of a successful telehealth business model. None propose an actionable method for selecting a psychiatry service partner based on nancial concerns or other priorities [20][21][22][23]24]. Hospitals implementing an emergency telepsychiatry service have limited resources available to guide their decision-making process. This paper will propose a speci c methodology for evaluating emergency psychiatry service options and identifying which aligns best with the hospital's needs by translating each option's unique features into two indices that allow for direct comparisons. This paper will also provide a speci c methodology for calculating the return on investment (ROI) of emergency psychiatry service options. Both methodologies may be applied to other hospitals with unique patient demographics and operational work ows.

Setting
This study took place in a community hospital's 18-bed ED. As the ED does not staff a psychiatrist or mental health expert, the ED cannot directly treat or remove IPHs, which require a psychiatrist evaluation. Social workers must coordinate a transfer to a psychiatric facility for all psychiatric holds to be evaluated and lifted. Research consent was deemed unnecessary because the project was determined by Stanford IRB panel IRB-98 not to meet the de nition of human subjects research as de ned in federal regulations 45 CFR 46.102 or 21 CFR 50.3.

Qualitative Data Collection
This qualitative study used 2 rounds of semi-structured interviews to identify causes for psychiatric boarding. The rst round was conducted with ED staff who treat psychiatric patients directly. The second round was with administrators who manage ED projects and nances. These interviews also served as an initial assessment of all staff's motivational readiness to support a new program [25].
A qualitative thematic analysis of these interviews identi ed main barriers to treatment and opportunities for hospital operations to be adjusted to address these barriers.
Quantitative Data Analysis ED case data was analyzed to quantify psychiatric patients' needs and the opportunity to improve their care. ED case volume was collected from Jan 1, 2019 to Feb 29, 2020. A psychiatric ED case was de ned as a case that began with an IPH, ended with a transfer to a psychiatric facility, or both. LOS is the time between the patient's arrival and discharge, and psychiatric LOS improvement means shortening it to the mean LOS for non-psychiatric cases. As seen in Table 1, the opportunity for improvement was sizable: mean psychiatric cases' LOS was 8.5 hours longer than non-psychiatric cases'. The ED's schedule was categorized into peak and non-peak hours. The greatest volumes of patients arrive during peak hours, experiencing longer lengths of stay and greater risk of leaving without being seen (LWBS) by a provider. A "clearance rate" is the percent of all patients arriving in the ED with an IPH that are removed post-evaluation. At the time of intervention, the IPH clearance rate was 0% due to lack of psychiatric providers on staff. Data from two telepsychiatry programs suggested that access to psychiatric care in the ED could raise clearance rates to 25-80% [5,8,26]. This hospital analysis used the IPH clearance rate estimated by its social worker team, 50%.

Developing the Evaluation Framework
Quantitative data was used to develop a framework for evaluating service options. Evaluation required both calculating expected costs to ensure affordability and assessing overall t: "how well does the service option solve our problem?" The method for calculating expected ROI mirrors those that other studies used for post-implementation ROI calculations [7,19]. This alignment allows for actionable pre-post analyses. This study incorporated the generalized nancial considerations suggested by previous literature, such as costs for purchasing technological devices [6-9, 12, 18-19].
ROI was projected for a 5-year time horizon. Since ROI relied on case volume during peak hours, it was calculated for 3 scenarios with different ED case volumes: low, expected, and high volume. Because ROI also depended on the IPH clearance rate, sensitivity analyses were conducted to assess changes in both factors: ED peak capacity and improvement in LOS due to removing IPHs.
Important features for an emergency psychiatry service other than ROI include the ability to meet patients' needs, patient centeredness, smooth processes and operations, strategic alignment, and integration of care [24][25]27]. A "Prioritization Model" was created to categorize all features as either a "bene t" or "implementation di culty" and then score each psychiatry service option on how well it aligned with the community hospital's needs. The prioritization model builds upon these categories outlined by previous literature [24][25]27] and the results from the qualitative analysis. This model allows for categories to be weighted to re ect how important each feature is: for instance, a category with a weight of 2 is twice as important as another category with a weight of 1. The hospital created 2 models with different prioritization weights: one optimizing for nancial performance overall; and one optimizing for partnership and community engagement.

Search for Psychiatry Services
Telepsychiatry vendors and market solutions were found by two mechanisms. First, an online search was conducted using these search terms: "telepsychiatry" OR "psychiatry" OR "telemedicine psychiatry" AND "emergency" OR "hospital" OR "emergency consultation" OR "acute" AND "service" OR "vendor" OR "company" Searches were repeated with "Bay Area" or "California" or "East Bay". Second, opportunities within the hospital network were sought out.

Qualitative Themes
There were 11 rst round interview participants including ED Administrative and Medical Directors; Social Services Director and staff; 3 Hospitalists; and Nursing O cers. The second round of interviews was conducted with 6 administrators from Compliance, Finance, Project Management, and Business Development teams. All ED staff and administrators eager to explore service solutions to address psychiatric boarding.
Four thematic domains emerged as seen in Table 2: the causes of psychiatric boarding, and the impacts that boarding has on patients, providers, and the hospital overall. Although low vacancy rates at other facilities (Domain 1) were beyond the community hospital's direct control, the other causes were within the community hospital's control and could be solved with reliable access to a psychiatrist in the ED.

ROI Model
In the ROI model the expected pro t comprises two sources: direct costs that are currently incurred and will be avoided post implementation; and new revenue from treating ED patients who would have LWBS. Two direct costs associated with delays in psychiatric care (Domain 2) were identi ed: sitters and funded transportation to psychiatric facilities for uninsured patients. These costs were multiplied by the psychiatric case volume and expected LOS improvement rate (50%) to represent direct cost savings. New revenue from treating patients with the time saved from psychiatric cases was calculated as the average contribution margin per ED case multiplied by the number of additional patients that could be treated. This revenue captured the nancial impact to the hospital (Domain 4).
The nancial investment required for each service option was estimated using pricing structures supplied by service options. Each pricing structure comprised 2 fee types: one-time implementation fees including purchase of equipment; and monthly fees for ongoing staff and technical support.
A breakdown of savings and costs included in calculating ROI is found in Fig. 1. The ROI was considered as one factor in the Prioritization Model.

Prioritization Model
Inputs and themes from the two rounds of interviews were used to identify 8 different categories important to have in any psychiatry service: ve categories of bene ts and three categories representing implementation di culty. For all categories, higher scores were favorable. As seen in Table 3, each category was further broken down into more speci c components that could be directly answered with either a number or Yes/No.
Components with percent values were converted into quintiles, with negative percent values assigned a score of 0. For example, an ROI of 65% was given a score of 4. Other numeric values were converted into a percent of the maximum component value and assigned a quintile score. For example, if the greatest LOS improvement across all service options was 4.6 hours, then the service option offering an improvement of 1.4 hours would be assigned a score of 2.
For Yes/No questions, Yes was assigned a score of 3 and No was assigned a score of 0. How many days will it take for the service to be implemented?
How much money will the hospital have to spend up front on implementation?
Logistics Is the partner within the health care organization network?
Will the service provide support for ongoing training, IT concerns, and general questions?
Will the hospital be able to avoid changing its operational systems signi cantly including electronic medical records and full time employee allocation to add this service?

Financial Costs
How expensive is the program over 5 years?
Does the service include revenue management?
Criteria -Bene t

Meets Patient Needs
How many psychiatric patients can have a psychiatric consult scheduled?
What is the average improvement in the time to rst psych consult?

How many Left Without Being Seen (LWBS) patients could be treated with the time we save?
How insensitive is the volume of LWBS opportunities to ED peak time capacity and the LOS improvement rate?
Is the service in person or telemedicine?
Is the service certi ed or accredited by a 3rd party such as the Joint Commission? Is a certain level of clinical quality ensured?

Meets Staff Needs
Do ED staff feel con dent in the psychiatric consult service's outcomes?
Do ED staff feel con dent that the service can integrate into current operations/work ows smoothly?
Do ED staff believe the service will improve employee satisfaction?

Partnership Viability
Is the solution within the network?
Is the solution embedded in the community?

Future Opportunities
Does the partner offer training and fellowship opportunities for current medical trainees? Are there Leadership and Directorship opportunities for current professionals?

Financial Viability
How insensitive is the ROI to ED peak time capacity and the LOS improvement rate?
What is the ROI after 1 year?
What is the ROI after 5 years?
Component scores were averaged to calculate a category score. Each category score was then multiplied by weights determined by the hospital. Weights (0-2) were assigned to each category based on perceived importance. The weighted scores were then summed to create an overall "Bene t" score and "Implementation Di culty" score per service option. Since a higher score is better across bene ts and implementation di culty, these two can be summed together to nd the one service option with the highest total score, indicating best t.
The relative weights used by the hospital in this study are shown in Table 4. This hospital created two prioritization models: a "Financial" model where positive return on investment was considered just as important as meeting patient needs; and a "Community Engagement" model where embedment in the community and health care network were considered just as important as meeting patient needs.
The ability to change category weights allows for generalizability: if another hospital is evaluating programs that improve staff recruitment and retention, for instance, that hospital can weigh "meets staff needs" more heavily in their own model. Putting it All Together: Selecting a Service Option This study identi ed 10 possible service options. Four were within the network and all offered telepsychiatry. Six services provided revenue management, and 8 services offered reconciliation of psychiatrist professional fees.
Community Engagement and Financial Model scores are presented in Table 5 for each of the 10 service options explored.
The average bene t score in the Community Engagement Model was 11.8 (SD = 2.8) and the average implementation di culty score was 5.7 (SD = 1.3).
Option J achieved both the highest bene t score of 16.0 and the highest implementation di culty score of 7.0, making Option J the preferred partner in this model.
The average bene t score in the Financial Model was 10.8 (SD = 3.1) and the average implementation di culty score was 4.7 (SD = 2.5). Option A received the highest bene t score in this model (16.2), while Option B achieved the highest sum score of 20.0 (bene t = 14.0; implementation di culty = 6.0). Both A and B ranked high in prioritization using the Financial Model.

Discussion
This study proposes a structured way of evaluating ED psychiatry service options. Few studies have proposed a generalizable method that can apply to other hospitals. One challenge previously cited is the diversity in health care settings [22,28]. However, this study suggests a method that may be used even when costs and priorities vary, allowing for a direct comparison of options.
The ROI methodology was conservative in the following ways: rst, the de nition of "psychiatric case" excludes cases where LOS was unlikely to improve.
Second, direct costs excluded the costs of non-sitter staff, bus tickets, and other transient costs. Third, new revenue from non-peak hour cases was not incorporated, because it is not con rmed that patients are at risk for LWBS in non-peak hours. Fourth, it was assumed that all psychiatric cases required a full evaluation to shorten their LOS. In November 2020 the County implemented a policy where a consultation with a psychiatrist would likely su ce for placing and removing IPHs. Since consultations are faster than evaluations, it is possible that even greater improvements in LOS could be captured.
The following were not quanti ed but are expected to improve with increased access to psychiatry and decreased psychiatric boarding: patient satisfaction; indirect costs from provider productivity and satisfaction (thematic Domain 3); ability to meet The Joint Commission standards or other quality metrics for accreditation [29]; and reputation within the community.
This study's main limitation is its single-site nature. When other sites choose to use the ROI Model and Prioritization Model, there may be additional considerations: different regulatory landscapes and opportunities within-network for example. With that in mind, the authors propose that other hospitals evaluating psychiatry ED services will still be able to substitute their own costs and priorities to select the service option best tting their speci c needs.

Conclusion
The concerns with psychiatric boarding in the ED are well documented as the impact is felt not only by psychiatric patients but also other ED patients, ED care providers, and hospital networks. Many hospitals have published their own efforts to tackle psychiatric boarding. To our knowledge, this is the rst study to propose a generalizable method for evaluating multiple psychiatry service options and selecting the one best t for the patients' and hospital's speci c needs.
Our model provides a methodology to improve the decision making process for choosing a new service.
Research consent was deemed unnecessary because the project was determined by Stanford IRB panel IRB-98 not to meet the de nition of human subjects research as de ned in federal regulations 45 CFR 46.102 or 21 CFR 50.3.

Consent for publication
Not applicable.

Availability of data and materials
Data on ED case volume from Jan 1, 2019 to Feb 29, 2020 are available from the corresponding author on reasonable request and with permission of the facility.
The pricing model data that support the ndings of this study are from third party vendors but restrictions apply to the availability of these data, and so are not publicly available. A list of vendors is available from the authors upon reasonable request.
The Prioritization and ROI Models discussed in this published article are found in the supplementary les Prioritization Model.xlsx and ROI Model.xlsx.

Competing Interests
One author has invested in an outpatient ADHD startup, which does not interact with inpatient care or emergency departments. The authors declare that they have no other competing interests.

Funding
The authors received no nancial support for the research, authorship, and/or publication of this article.
Authors' contributions DS and KC contributed to the conceptualization of this study. All authors designed the study. JS and KC performed material preparation, data collection, and analysis. JS conducted literature searches and prepared the manuscript. DS and KC provided ongoing feedback and consultation during the manuscript preparation stages. All authors read and approved the nal manuscript.
Sources of Costs and Savings Used to Calculate Return on Investment (ROI) This diagram shows the breakdown of nancial information used to calculate projected costs and savings, and ultimately return on investment. * indicates a cost that may be calculated differently for other hospitals.

Figure 2
Community Engagement Model Bene t versus Implementation Di culty Scores Each bubble represents a service option. The bubbles' size and color indicate the 5-year ROI (blue is + ROI).
Page 12/13 Financial Model Bene t versus Implementation Di culty Scores Each bubble represents a service option. The bubbles' size and color indicate the 5-year ROI (blue is + ROI).