Of the 42 cancer centers from the first two cohorts of the C3I, 15 centers reported on their operating costs. This study used a mixed methods, comparative case study design, in which each participating site (n=15) is conceptualized as a case . Comparing multiple cases leverages variations across sites and allows for the investigation of contextual factors influencing implementation outcomes, such as cost. Consistent with the objectives of case study research, this study relies on multiple methods of data collection, as the convergence of multiple types of evidence enhances the credibility of the analysis . The study is guided by the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) .
The setting for this study consisted of NCI-designated cancer centers participating in the C3I program designed to enhance the routine delivery of evidence-based tobacco treatment services. Participating institutions are required to overcome patient, clinician, practice, and health system barriers to providing evidence-based tobacco treatment services to their oncology patients who smoke, to achieve institutional buy-in that treating tobacco use is a component of organizational “standard of care”, and to create mechanisms to sustain tobacco treatment services beyond the implementation funding period of the initiative. Several factors position the C3I as a unique setting to evaluate the implementation costs of behavioral interventions in cancer care including: the number and diversity in size, structure, and geography of the funded NCI-designated cancer centers, the range of implementation strategies and tobacco treatment approaches utilized, and the available reporting on standardized patient outcomes and cost measures.
Data collection procedures
Tobacco treatment program data were reported by each site to the C3I Coordinating Center for the purposes of program evaluation. The Coordinating Center, based at the University of Wisconsin-Madison Carbone Cancer Center, provides scientific and technical assistance to grantees in integrating evidence-based tobacco treatment services into clinical care  and was responsible for data collection across sites. The Coordinating Center has developed metrics to assess the tobacco treatment programs using standard measures for patient outcomes, including reach and effectiveness. Reporting of these metrics to the Coordinating Center was required of each participating site on a twice-annual basis using Qualtrics forms (Qualtrics, Provo, UT). Additionally, cost reporting has been encouraged for sites, following a similar reporting frequency. Cost reporting was first introduced for the July-December 2018 reporting period and we herein include all sites with operating cost data reported at least once during that interval through the January-June 2020 period. Sites were asked to report their program costs retrospectively using a Qualtrics form that was developed by the lead author in collaboration with the Coordinating Center. Two pilot sites provided feedback on the cost data collection form prior to its rollout to all sites. For qualitative data collection, site visit interviews were conducted by Coordinating Center staff with the director of the cancer center at each site, principal investigator(s) of the C3I project, program staff, clinical or administrative leaders, IT staff, and/or other administrators. The study was determined to be exempt from institutional review board (IRB) approval as there were no patient-level data used, and the study was categorized as program evaluation by the University of Wisconsin-Madison IRB.
Tobacco treatment program components
On the twice-annual surveys, sites report8-ed whether they implemented the following types of evidence-based tobacco treatment components: in-person (individual or group) counseling delivered by a tobacco treatment specialist, telephone-based counseling delivered by a tobacco treatment specialist from the program (i.e., internal, other than referral to the quitline), cessation counseling delivered at the point of care, track and triage services delivered by interactive voice response system (i.e., TelASK), referral to the quitline, smokefreeTXT text messaging service, web resources (e.g., smokefree.gov), and cessation medications offered as part of the tobacco treatment program.
Standardized metrics were used to report on reach and effectiveness. Program reach was defined as the proportion of current smokers who were offered any type of tobacco treatment program, among current smokers with a clinical visit in the 6-month reporting period. Engagement in a program was defined as participation in an individually or group-delivered counseling program, in-person or via phone, fax or e-referral to a quitline, a website, or a text/mobile program, counseling regarding quitting, or prescribing cessation medication. Effectiveness was defined as the number of current smokers at baseline who engaged in tobacco treatment and reported abstinence from smoking for at least 7 days, among responders at 6-month follow-up. Additional details about outcome measures are reported elsewhere .
Estimating resources and costs
The present study focused on operating (maintenance) costs – i.e., costs to maintain the program after it was developed – as they are most relevant to decision makers. Planning/development costs (e.g., EHR modifications) were inconsistently reported by sites and were not included in the analysis. Research-related costs were excluded as they are not relevant to replicating the program in another setting. The following categories of cost measures were reported by centers: program personnel type and effort, medications provided/covered by the program, educational and training materials, software and technology services (e.g., interactive voice response system), equipment (e.g., computers), and office space. As stated previously, participating sites reported costs for at least one 6-month reporting period, up to a maximum of 3 reporting periods from 2018 to 2020.
We calculated the total operating costs for each site within a six-month time period by summing reported expenses across all categories. Costs were reported from the perspective of the health system, which is most relevant to decision makers in the case of tobacco treatment program implementation within cancer care settings, as opposed to the patient or societal perspectives. All costs were expressed in local market terms following the guide to costing behavioral interventions developed by Ritzwoller et al. . For the personnel category, effort dedicated to delivering the tobacco treatment program was reported by type of personnel and multiplied by the average wage rates for each personnel type as provided by the Bureau of Labor Statistics  for the appropriate metropolitan statistical area of the center. This approach was used to minimize the impact that site-specific variations may have had on intervention cost estimates. Fringe benefits were estimated at 30% of total salary costs for each site. Values were adjusted to 2020 U.S. dollars using the consumer price index to account for inflation . For sites that reported costs for more than one time period, we reported the average cost per category across all available time periods. We converted all costs to monthly costs. We then calculated the cost-per-participant and cost-per-quit for each site by dividing total operating costs by: (1) the number of patients that engaged in tobacco treatment program services within a 6-month period, and (2) the number of patients who reported seven-day abstinence from tobacco use over a six-month time period, respectively. To calculate these two ratios, we used the reach and effectiveness measures reported in the most recent time period, for sites that reported them in more than one time period. Cost-per-quit was unavailable for 3 sites that had not yet reported effectiveness data.
Site visit interviews
Researchers from the C3I Coordinating Center with expertise in qualitative methodology, tobacco cessation, and health systems research developed the interview guide, informed by the Consolidated Framework for Implementation Research (CFIR) . Copies of the interview guide are available from the Coordinating Center upon request. Program leaders and staff were asked about program components, the implementation process, barriers and facilitators, and the resources needed to maintain the program over time. Interviews followed a semi-structured format in which respondents were asked the same questions with the opportunity for customized follow up and suggested probes depending on responses. Interviews were transcribed verbatim, cross-checked by another team member, and analyzed by a team of two researchers using NVivo version 11 (QSR International), a qualitative data management program.
Qualitative data analysis
The research team first conducted queries of coded transcripts from sites that reported cost data to extract content relevant to implementation barriers, implementation facilitators, and resources needed for program sustainability. Queries underwent secondary inductive coding for emerging themes, which were compiled and grouped according to constructs in the Consolidated Framework for Implementation Research (CFIR) related to intervention characteristics, inner setting, and implementation process. Themes most relevant to understanding costs and resource needs for implementation were identified and added to the site-level matrix, allowing for a side-by-side comparison of quantitative and qualitative findings on costs specific to each site. Interpretation of findings followed a concurrent mixed-methods approach, whereby qualitative findings complemented quantitative findings with information on contextual factors related to costs of implementation; and (2) intangible resource factors (e.g., time, space) that are difficult to collect in a quantitative format.