Using a longitudinal observational design, we analyzed existing clinical data from participating CCMP practices to understand the process of CoCM implementation from 2014 to 2019. We used the RE-AIM model, a well-established framework for translating research into real-world implementation of evidence-based interventions to frame our data presentation, focusing on proportion of clinics moving into each of the five stages of this model over the years of implementation(10). Data sources included: an OMH clinic tracking database, Medicaid billing applications, quarterly reports of performance and outcomes, and Medicaid claims. The estimated number of primary clinics of all types is 9,548, across the state. To arrive at this number, we used the New York State Department of Health’s Provider Network Data System to see how many individual providers submitted Medicaid claims as of December 2019(11). Of those, we sorted by provider address and eliminated duplicates, which gave us the number of clinics in New York State submitting Medicaid claims for primary care services. We filtered out clinics for which family medicine, internal medicine, or pediatrics were not the primary specialty of the clinic.
All sites that participated in CCMP had access to training and technical assistance (TTA) during CoCM implementation. OMH staff collaborated with the AIMS Center to provide evidence-based implementation plans and processes for all participating primary care settings implementing CoCM. Since 2014, OMH has offered continuous CoCM implementation TTA, free of charge, to all primary care clinics interested in participating in the CCMP, as well as ongoing TTA after clinics begin offering these services to their patients. Data were collected since CCMP began..
OMH partnered with the AIMS Center at the University of Washington to provide TTA for the CCMP. The AIMS center follows a 5- step implementation process: 1) Lay the Foundation, 2) Plan for Clinical Practice Change, 3) Build Your Clinical Skills, 4) Launch Your Care, and 5) Nurture Your Care(12). Step one focuses on education and orientation to the model and the importance of organizational leadership support. Step two involves creating an implementation plan and identification of care team members within the site, often using coaching calls and online training modules to get team members ready to implement. Step 3 focuses on clinical training, often through online and in-person training sessions that providers attend followed with topical webinars to support use of the model. OMH offered free and/or discounted Problem-Solving Treatment (PST) certification training through the AIMS Center. PST is an evidence-based behavioral health intervention particularly well-suited to primary care settings(13). Step 4 is launching the implementation and using the registry to track patients and ensure quality of CoCM. OMH commissioned the AIMS Center to build a customized behavioral health registry program, which was a version of the Care Management Tracking System (CMTS), and contained process and outcomes metrics specific to the Collaborative Care Medicaid Program(14). CMTS is a web-based registry used for systematically tracking behavioral health caseloads and generating reports to facilitate clinical decision-making and quality improvement. OMH offered clinics access to this version of CMTS at no cost for one year and then at an extremely discounted rate after that. Clinics providing CoCM were required to use some type of registry approved by OMH in order to bill the New York State Medicaid codes and the CoCM CPT codes for other payers, but were not required to use CMTS specifically. Ongoing coaching calls, case presentations, and monthly office hours were used to support sites through both steps four and five, which focused on sustainment of the model.
TTA was further enhanced through the utilization of a New York-based AIMS Center external consultant and her team of implementation specialists who, in addition to conducting most of the activities listed above, provided site visits and specialized expertise in New York State licensure and billing laws. In addition, a website was created specifically for CCMP that housed many resources, including a TTA calendar and recordings of webinars. While TTA was offered to all participants, it was not a requirement of participation. Each clinic was free to decide how much or how little assistance they consumed, or whether they consumed any assistance at all. TTA was meant to help clinics get the infrastructure in place and provide on-going support once services began. Some clinics chose to get their TTA through sources other than or in addition to OMH including, but not limited to, Delivery System Reform Incentive Payment (DSRIP) Preferred Provider Systems, New York City Department of Health and Mental Hygiene (NYC DOHMH) Mental Health Service Corps, and other national content experts such as the National Council for Behavioral Health and the American Psychiatric Association.
We included recruitment and participation records data from primary care clinics located in New York State that expressed an interest in participating in the Collaborative Care Medicaid Program (CCMP) between 2014 and 2019. Clinics that participated in the Collaborative Care Initiative from 2012-2014, were grandfathered into this data set. We defined each of the following RE-AIM framework steps as: Reach, ever contacting OMH with interest in CoCM; Effectiveness, how well the program worked in achieving clinical goals of lowering depression and anxiety levels in patients; Adoption, whether clinics received any training and technical assistance. Adoption was further defined by who provided the TTA (OMH only, an outside source only, or both OMH and an outside source) and level of TTA intensity (high if over 10 encounters with a clinic, low if under 10 encounters with a clinic, and unknown if the intensity could not be determined). Variables for implementation of CoCM were defined as: 1) the completion of a CCMP billing application, 2) the submission of at least one quarter of data to OMH, or 3) the submission of Medicaid billing claims. Finally, we looked at maintenance of clinics’ CoCM programs long-term, defined as submitting quarterly data at least one year after implementation of CoCM.
The FQHC dataset consisted of all New York State grantees of the Health Resources and Services Administration (HRSA)’s 2018, Health Center Awardee’s under the Unified Data System that met our inclusion criteria of providing primary care to patients 12 years and older(15). Clinics were excluded if they were homeless shelters, school-based health clinics, church-affiliated services, dental clinics, mobile clinics, and administrative-only offices. A total of 452 FQHC clinics were included in our analyses. OMH kept a record of every FQHC and non-FQHC with which it had contact in its clinic tracking database (n=611), as well as TTA source and intensity data.
Our analyses were organized around the multiple stages of the RE-AIM model. Reach can be defined as the interest expressed by any member of the target population(10). In our analyses, expression of interest was indicated through an initial contact between OMH and an individual clinic or group of clinics. These contacts occurred over the telephone and in person, with OMH providing information about the requirements for participation in the CCMP, along with a CCMP billing application, followed by a discussion to determine the feasibility of the clinic meeting those requirements with or without TTA from OMH or outside sources. The requirements for participation are stated in the CCMP billing application(16). In some cases, organization leaders met with OMH and let individual sites within their organization decide whether to participate. If an organization met with OMH, we marked all clinics under that organization as having an initial contact.
The next phase of RE-AIM is Effectiveness of the program based on clinical improvement outcomes(10). Using the quarterly reports OMH collected from participating clinics, we were able to determine an average depression and anxiety improvement rate. Improvement was defined by OMH in 2018-2019 as “the number and proportion of patients enrolled in treatment for 70 days or greater who demonstrated clinically significant improvement either by (1), a 50% reduction from baseline PHQ-9/GAD-7 or (2), a drop from PHQ-9/GAD-7 to less than 10”(17).
Adoption is the next phase of the RE-AIM framework, which refers to the settings, systems, or communities that decide to use the program . We defined adoption in this analysis as the number of clinics that received training and technical assistance, as TTA was an indicator of engagement in the program. From 2014-2019, the manner in which TTA was delivered to clinics that requested it varied in structure, but the content offered was the same for all clinics. Consumption of TTA was not mandatory and varied greatly among clinics.
The next phase of RE-AIM is Implementation, which is the extent to which the intervention is implemented as intended in the real world. We divided this phase into three variables. The first variable was the completion and submission of a CCMP application, which attested that all of the components of CoCM were in place. Upon application approval, clinics were required to submit quarterly process and outcomes data, called quarterly metrics reports. A clinic’s submission of quarterly metrics data indicates that it has all the key components in place and is currently providing CoCM services. Our second variable of implementation was the number of clinics that ever provided these quarterly metrics data to OMH.
Our final variable of implementation was the number of clinics that were reimbursed by Medicaid for CoCM services. Developing a workflow to bill Medicaid can be quite challenging for clinics, so we considered this to be the most extensive measure of implementation.
The final phase of RE-AIM is Maintenance, defined as the continued use of a program, i.e. sustainability(10). We defined maintenance of this program as the number of clinics that continued to provide quarterly metrics data at least one year after their first submission of data.
All data were analyzed using SPSS version 27(18).