Evaluation of an educational strategy to improve medication reconciliation in ambulatory care.

BACKGROUND
Improper medication reconciliation can result in inaccurate medication lists. When medication lists are inaccurate, it can result in drug-drug interactions, dosing errors, and medication duplication. Interventions targeting medication reconciliation have had varying levels of success.


OBJECTIVE
This study aimed to describe the medication reconciliation educational program, its implementation in a health care system, pharmacist and clinic personnel perception of the program, and its impact on clinic personnel knowledge and practice.


METHODS
Guided by the Conceptual Model of Implementation Research, a partially mixed sequential dominant status evaluation of a pharmacist-led educational program on evidence-based practices for medication reconciliation implemented into all primary care clinic sites by examining implementation outcomes was conducted. The implementation outcomes measured include penetration, fidelity, acceptability, appropriateness, feasibility, and adoption. Data were collected through program data and direct observations, pre- and postsurveys, and semistructured interviews of pharmacists and clinic personnel.


RESULTS
Of 46 primary care sites, 37 primary care sites (80%) implemented the pharmacist-delivered medication reconciliation education from April to June 2021 with representation from each of Geisinger's regions. Ten clinic sites (27%) completed the medication reconciliation educational program as originally designed, with the remainder adapting the program. A total of 296 clinic personnel completed the presurvey, and 178 completed the postsurvey. There were no differences in baseline characteristics between clinic personnel who completed the pre- versus postsurvey. All clinic personnel interviewed felt satisfied with the educational program and felt it was appropriate because it directly affected their job. Clinic personnel felt the educational program was acceptable and appropriate; two major concerns were discussed: a lack of patient knowledge about their medications and a lack of time to complete the medication reconciliation. The adherence rate to the elements of the medication reconciliation that were covered in the education program ranged from 0% to 95% in the 55 observations conducted.


CONCLUSION
An educational program for medication reconciliation was found to be acceptable and appropriate but was often adapted to fit site-specific needs. Additional barriers affected adoption of best practices and should be addressed in future studies.


Introduction
Improper medication reconciliation can result in inaccurate medication lists which can lead to adverse events including hospitalization (1)(2)(3).
Inaccuracies in medication lists are common and may contribute to 100,000 hospitalizations per year (1)(2)(3). There are many factors contributing to an improper medication reconciliation including patients' lack of knowledge of their medications, lack of a standardized process for conducting medication reconciliation by healthcare centers, and a lack of education for healthcare workers on how to conduct a proper medication reconciliation (4). Interventions to improve medication reconciliation often target one or more of these barriers and have had varying levels of success in improving the process.
In a previous study, we identi ed a lack of training on proper medication reconciliation as one of the major barriers to accurate medication lists (4). To address this barrier, we designed and implemented a pharmacist-led educational program for healthcare staff in our ambulatory care clinics. The objective of this paper is to describe the training, its rollout in a healthcare system, pharmacist and clinic personnel perception of the training, and the impact of the training on medical personnel knowledge and practice. Guided by the Conceptual Model of Implementation Research (5, 6), we collected and measured implementation outcomes important to the success of the pharmacist-led educational program within a healthcare system.

Setting
Geisinger providers (physicians and advanced practitioners) serve a network of 138 primary and specialty clinic sites including 79 communitybased primary care clinics in Pennsylvania. Most of these clinics are based in the northeast and central region of Geisinger coverage, but some are in the western region of Pennsylvania.

Study design
Guided by the Conceptual Model of Implementation Research (5, 6), we conducted an evaluation of a pharmacist-led educational program on evidence-based practices for medication reconciliation by examining implementation outcomes. This study was approved by the Geisinger Institutional Review Board.

Medication Reconciliation Educational Program
To improve medication reconciliation practices in our ambulatory care clinics, we developed an educational program designed to be delivered to clinic personnel over a total of four sessions. The content was developed by an existing workgroup within our health system, the Ambulatory Medication Reconciliation and Education subgroup, which was tasked by system leaders with improving medication reconciliation. The workgroup consisted of pharmacists, nurses, and clinical leadership. The workgroup met monthly to discuss issues related to medication reconciliation. Led by two medication reconciliation champions, the educational materials were developed collaboratively by members of the workgroup. The materials drew on both existing literature on best practices, clinical experiences of members, and a needs assessment performed by the study team (7)(8)(9). Speci cally, the educational program highlighted several practices which are consistent with a comprehensive medication reconciliation including: 1) use open ended questions, 2) clarify and/or determine pharmacy used by the patient, 3) ask the patient about their current medications, 4) have the patient state the medication, strength, dose and how they take each medication, 5) ask for clari cation if their response differs from the list in the electronic health record, 6) ask about non-prescription medications the patient is taking, and 7) evaluate medication adherence.
Once the training was developed, members of the workgroup trained pharmacists to deliver the education as part of an annual professional development retreat. The training and other relevant materials were made available through an internal website. The trained pharmacists, who were embedded within primary and specialty care clinics at Geisinger, trained other clinical staff on how to complete a proper medication reconciliation over 4 weeks in 10-15-minute sessions using a combination of presentations and example videos of good and bad medication reconciliations. Implementation strategies with speci cations are available in Table 1 following guidance for specifying and reporting of implementation strategies by Proctor and colleagues (10). Notes: Implementation strategies are only focused on improving provider process not patient process Data collection Implementation outcomes measured include penetration, delity, acceptability, appropriateness, feasibility, and adoption. Table 2 describes the study speci c de nition, measurement, and data source for each implementation outcome. Data was collected through direct observations, administrative data, pre-and post-surveys, and semi-structured interviews.  Pre-and Post-Surveys. We conducted pre-and post-surveys of clinic personnel to explore acceptability and knowledge gained about medication reconciliation before and after deployment of the educational program. Working with clinical leadership and pharmacists hosting the training, we identi ed clinic personnel working in ambulatory clinic sites who were expected to receive the educational strategy, then invited these clinic personnel to participate in a pre-training survey via e-mail. Clinic personnel included physicians, advanced practitioners, nurses, medical assistants, case managers, and community health assistants. The pre-survey had 12-questions consisting of 7 demographic and 5 knowledge and satisfaction questions. After the training we sent a post-survey to these same clinic sites. The post-survey had 16-questions consisting of 8 demographics questions and 8 knowledge and satisfaction questions. Based on pilot testing, completion of both surveys should have taken no more than 5 minutes. Both surveys were administered using Research Electronic Data Capture, or REDCap (REDCap, Nashville, Tennessee), a secure web application for building and managing surveys online and databases (11,12). We sent out one reminder e-mail two days after the initial surveys were sent, and staff received $10 e-gift cards for each survey completed.
Semi-structured interviews. We conducted interviews with the pharmacists who delivered the educational program and clinical personnel that attended the educational program on medication reconciliation to understand their perspectives on, and experience with, the educational program. Speci cally, we assessed the following implementation outcomes: acceptability, appropriateness, and feasibility for clinic personnel who received the educational program and delity of the pharmacists for the delivery of the educational program. Participants included physicians, nurses, and medical assistants. All clinic personnel were recruited using purposive sampling via e-mail. The pharmacists were recruited if they performed at least one training and the clinic personnel were recruited if they attended at least one session of the medication reconciliation trainings at their clinic site. We conducted all interviews using a semi-structured interview guide (Additional File 2). Example

Data analysis
Direct observations. Data from the direct observations were handwritten by observers then manually inserted into a REDCap survey created by a member of the study team. Data were summarized using descriptive statistics with STATA 14.1 (StataCorp, College Station, TX) software. We also reviewed quantitative data for relationships between variables (e.g. between staff type and adherence to behaviors).
Pre-and Post-Surveys. Descriptive statistics (e.g., counts, percentages, means) were used to summarize the data. When performing a hypothesis test to look for differences in baseline characteristics or responses between groups, a chi-squared test was used when comparing percentages (unless there are less than 5 of a particular response, in which case Fisher's exact test was used instead), and a Student's t-test was used when comparing means of continuous variables. All analyses were performed using SAS 9.4 (SAS Institute, Cary, NC) with p < 0.05 considered signi cant.
Semi-structured interviews. Directly after each interview, a summary of the discussion was created, and responses were mapped to the appropriate implementation outcomes of interest. The interviews were audio recorded and transcribed verbatim using the transcription service offered by our institution. A member of the research team (VD) reviewed and de-identi ed the transcripts, then the transcripts were checked for accuracy by two members of the research team (LKJ and VD). A rapid qualitative analysis method was deployed; research staff reviewed interview summaries for important outcomes and then reviewed transcriptions for accuracy and supportive quotations (13).

Penetration of the Medication Reconciliation Educational Program to Clinic Sites
A total of 37/46 (80%) primary care sites implemented the pharmacist delivered medication reconciliation education from April to June 2021 with equal representation from each of Geisinger's regions during the study observations period. Of the 684 clinical personnel that were eligible for the education, 59% completed the entire education over the 4 weeks (or a modi ed version) and 81% completed a portion of the education.
Eight sites achieved 100% attendance of clinical personnel. Site-level demographics and education delivery information is available in Table 3. The initial implementation of the educational program targeted primary care clinics; however, 2 specialty clinics also implemented the program. Interviews were completed with 11 clinic personnel who attended the educational program and 4 pharmacists that delivered the educational program. Detailed demographics for the interviews are available in Table 4. Pharmacists felt the educational program was generally well-received and stimulated good discussion at the clinic sites. Discussion was noted to be more di cult at the clinics which held their training virtually. In addition, the pharmacists were not always able to reach the target group of clinic personnel due to busy schedules, prior engagements, or trainings held over the lunch hour. Pharmacists emphasized how the educational program needs to be ongoing to ensure that everyone is keeping up with new technologies and changes in the electronic health record that affect the medication reconciliation process. Exemplar quotations from pharmacists related to delity to the educational program are listed below.
"I guess maybe timewise, just the clinic being too big, not enough time to hit all of the people that you need to. I know with some of the huddles you don't always have every person there, they come and go. There are some people that just never attended a huddle and that was just in the bigger clinics, so you'd miss some. So, just having the time to actually do it." (106 Pharmacist) "We have all different sorts of specialties here. So, instead of doing it in the weekly increments, just because there are so many providers that are here and they're all here different days, different hours, I presented it to all the nursing staff early morning on a Thursday. All the nursing staff that deals with all the providers collectively. Gave them all separate power points, and we reviewed it together, one by one, and as a whole in one sitting." (358 Pharmacist) "From 1:10 to 1:40 is now considered "lunch" for our providers so they don't typically have patients and nursing a lot of times will kind of start to eat lunch around noon, but a lot of them will kind of peace out during that timeframe because of the fact that there are no providers to be rooming for and there is just a little bit more time, so all of nursing wasn't always able to attend." (696 Pharmacist)

Acceptability of the Medication Reconciliation Educational Program by Clinic Personnel
A total of 296 clinic personnel completed the pre-survey, while 178 completed the post-survey. There were no statistically signi cant differences in baseline characteristics between clinic personnel that completed the surveys. Detailed demographics for the pre-and post-survey are available in Table 4.
The pre-and post-surveys explored clinic personnel acceptability and knowledge regarding the educational program on medication reconciliation (Table 5). When asked to indicate their agreement with the following statement, "I am con dent that I complete medication reconciliation correctly most of the time," the percentage of clinic personnel responding "strongly disagree" increased by 5% (2% vs. 7%, p = 0.01) and the percentage responding "strongly agree" increased by 11% (33% vs. 44%, p = 0.02) from the pre-to the post-survey. Thus, the educational program 'polarized' people and made them either more or less con dent, with fewer people in the middle. For every category of product (except prescription medications, creams/ointments, eye drops, and "none"), there were statistically signi cant increases in the percentages of clinic personnel saying those products were important to include in medication reconciliation. There was also a statistically signi cant increase in clinic personnel correctly selecting the most appropriate behavior consistent with a medication reconciliation, which changed from 78% presurvey to 91% post-survey (p = 0.0003). While post-survey respondents were also more likely than pre-survey respondents to state that improper medication reconciliation can result in admissions or readmissions (96% vs. 90%, p = 0.02), a correct answer, they were also more likely to state that an improper medication reconciliation may lead to loss of insurance coverage, an incorrect answer (42% vs. 28%, p = 0.002). Clinic personnel felt they learned something from the educational program that would apply to their work (86% said agree or strongly agree), and were likely to recommend the program to a friend (on the Net Promoter Scale of 0-10, mean response was 8.2). All clinic personnel interviewed felt satis ed with the educational program. Also, all described the program as informative and liked that it provided examples on how to complete the medication reconciliation within the electronic health record. About a third, mostly providers, felt there was no new information that they learned from the education but felt it was a good refresher on the topic of medication reconciliation. Other clinic personnel discussed new learnings from the education including: how to place in a note to the doctor about the medication reconciliation they conducted; making sure dose, frequency, time of day is recorded for each medication the patient is taking; adding reasons for when a medication has been discontinued; and learning how to add natural supplements to the medication list. Exemplar quotes about acceptability of the medication reconciliation educational program by clinic personnel are listed below.
"It was very informative because there were lots of things, I didn't know myself when I was doing med rec because usually, I would just ag them to be taken off instead of taking them off myself, that I could do that. I didn't know I had to put a reason every time I took one off." (538 Nurse) "I'm doing it constantly now as far as cleaning up the med rec. I mean we always did make Med reconciliation but now we're actually cleaning up the med rec. We're going in, we're discontinuing the meds, we could use med list cleanup, there's those different dropdown options and such, so yes." (102 Nurse)

Appropriateness of the Medication Reconciliation Educational Program by Clinic Personnel
All clinic personnel felt the educational program was appropriate because it directly impacted their job. Clinical personnel identi ed obtaining an accurate medication reconciliation as a very important part of their job. Many nurses discussed how taking a good medication reconciliation helps the prescriber understand what the patient is taking and helps make important decisions about care. Prescribers felt medication reconciliation was important to caring for their patients. Many felt it was appropriate to have refresher education on the medication reconciliation process at some determined point in time or as updates are made to the electronic health record, and to incorporate this type of education into new employee orientation. Exemplar quotes about appropriateness of the medication reconciliation educational program by clinic personnel are listed below.
"I would hope most staff members already know why, but being able to explain to the patient makes them feel more empowered and they're more open to increasing dialogue." (552 Medical assistant) "I think it will be very helpful, especially with new hires, because I think that when you get a new hire, like I will be hiring some new nurses, I will be able to implement this in part of their training for beginners training because I think you need to mold them in the beginning and not do the shortcuts, and I think it will be better. It's just everybody has to stay on the same page." (274 Nurse) "It's hard to do with some patients because they don't even remember what they're taking, and they don't bring a list in, so it's hard to do some of these. For those I just leave a note for the doctor that I tried to do these, and the patient is not very cooperative with it." (538 Nurse) "Yeah, so just having to go through and then, one, I don't see them in person, so I can't really have them bring their pills with them, they kind of tell me my blue pill, so it's a little bit di cult for me to discriminate which ones they're taking, and then just kind of going through, because you can like pull in from like outside sources, like trying to pull that in and stuff like that, so it can be little bit time consuming when I'm doing the telemedicine visits." (638 Physician) "I think the patient gets in the way. They don't listen. They say they don't know what they're taking. They're on a pill pack. They don't bring in the pills, they don't bring in the pill pack label you know I think it's more patient related than it is anything else." (102 Nurse)

Adoption of the Medication Reconciliation Skills from the Educational Program into Practice by the Clinic Personnel
A total of 55 observations of nurses and medical assistants were completed across three of the larger ambulatory clinic sites for their adoption of their learnings from the educational program. We found the adherence rate to the elements of the medication reconciliation which were covered in the education program ranged from 0-95% (Table 6). There was no meaningful difference across the three sites where the observations were conducted.

Discussion
Our study examined the implementation outcomes associated with implementing an educational program to improve ambulatory medication reconciliation. We found the medication reconciliation educational program had high attendance at the targeted primary care clinics and pharmacists adapted the presentation delivery, but not the content, to meet the needs of the individual clinic sites. Recipients of the educational program found it to be acceptable and appropriate and feasible to implement the skills from the educational program into their daily practice.
However, they expressed some concerns related to portions of the medication reconciliation process which were out of their control including time to conduct an accurate medication reconciliation and patients knowing the medications they were prescribed. Adoption of the process detailed in the educational program was variable and dependent on other factors including responses from patients.
Our ndings from the implementation of a medication reconciliation educational program are similar to other programs in both ambulatory and hospital settings (14,15). A hospital-based educational program found an increase in knowledge and self-e cacy for proper medication reconciliation from pre-and post-surveys (16). Improving knowledge is important and may improve the quality of medication lists gathered through medication reconciliation, but the effect on outcomes is less clear (17)(18)(19). Similar to our ndings, others have found that providing education alone is not enough to guarantee consistently accurate medication reconciliation (17)(18)(19). Additionally, a previous study conducted by our group identi ed numerous barriers to medication reconciliation including lack of time, lack of patient knowledge, and lack of a standardized work ow (4). While this work addressed the standardized work ow barrier, lack of time and lack of patient knowledge were again identi ed by participants in the current study as barriers to medication reconciliation (4).However, in implementation work a balance between adaptations to individual context of interventions and implementation strategies are important as long as key components are retained.
Our study is unique in that it mapped the implementation strategies to the Expert Recommendations for Implementing Change (ERIC) compilation (20), de ned these strategies using a standardized guideline (10), and measured the implementation outcomes after deployment of an educational program. The measurement and analysis of these implementation outcomes generated important insights for the construction and implementation of future educational programs for medication reconciliation. Others have begun to map their implementation strategies for medication reconciliation to the ERIC compilation to create tailored strategies for the eld (21). When implementing educational programs, all necessary staff should attend the full training to maximize bene t. Achieving this may require a change in policy or encouragement from clinic leadership. Educational programs should be exible in their delivery methods of the content and recognize that certain methods work well for some clinics, but not for others. Most clinic personnel believe medication reconciliation is an important part of their job and were receptive of the training, but feasibility often hinders its application in practice, because of the barriers of insu cient time and patients not knowing their medications. Additionally, it is important to see if the ndings from the educational program change practice and if these changes are sustainable over time. Our results indicate that while participants felt the training was applicable and appropriate many aspects of the medication reconciliation process were still not consistently adopted.
This study has a few limitations. While we did not observe all participants in the educational program, we conducted observations at three large sites and noted no signi cant differences between the sites. Adoption may be different at other sites or across individuals, but our data indicate more work is necessary to improve overall adoption of medication reconciliation best practices. Another limitation of our implementation strategy was that it targeted only one group of actors (healthcare personnel) and failed to address the other signi cant actor (patients).
Participants in our study, consistent with previous studies (4), noted patient knowledge and behaviors limit the effectiveness of medication reconciliation. Future studies should examine sets of implementation strategies which target both healthcare personnel and patients, ideally addressing both the lack of time and lack of patient knowledge.

Conclusion
An educational program for medication reconciliation was found to be acceptable and appropriate but was often adapted to t site speci c needs. Additional barriers affected adoption of best practices and should be addressed in future studies.

Declarations
Ethics approval and consent to participate. This study was approved by the Geisinger's Institutional Review Board.

Consent for publication. Not applicable
Availability of data and materials. All data generated or analyzed during this study are included in this published article.
Competing interests. The authors declare that they have no competing interests.
Funding. This work was funded by the Geisinger Health Plan through the Geisinger Clinic Quality Pilot Fund program for scal years 2020 and 2021.
Authors' contributions. LKJ conceptualized and designed the study, acquired, analyzed, and interpreted the data, and drafted the initial and revised to nal manuscript. VD helped to design the study, acquire, analyze, and interpreted the data, and helped to draft the initial and provided revisions to the nal manuscript. KMR helped to design the study, interpret the data, and added substantial revisions to the initial and nal manuscript. AF helped to acquire the data and added substantial revisions to the manuscript. JM helped to acquire the data and added substantial revisions to the manuscript. JG analyzed, and interpreted the data and added substantial contributions to the nal manuscript. MRG helped to design the study, acquire, analyze and interpreted the data, and helped to draft the initial and provided revisions to the nal manuscript.
All authors read and approved the nal manuscript.