A Structured Multimodal Program for Improving The Pediatric Discharge Summaries At The University Teaching Hospital

Objective: Discharge Summary (DS) is the vital document for the ongoing patient care. We aimed to improve the quality of DS given from our pediatric ward of tertiary teaching hospital. Methods: We introduced longitudinal multimodal program. Residents preparing the DS were given the education in a small group. Peer audit of the summaries were done by each residents at several occasions followed by feedback from consultant concluding the lesions for improvement. Summaries rechecking by senior residents or consultants was made mandatory once prepared by junior residents. Discussion on discharge processes was emphasized during the ward round including DS documentation points. We didn’t nd suitable pediatric DS assessment tool in the literature. We developed a scoring system evaluating 33 components of DS under seven broad headings. A scoring tool had good inter-rater reliability. We introduced the program in May 2018. 33 summaries were evaluated each from last quarter (October to December) of the calendar year 2017, 2018, and 2019 as a pre-intervention, post-intervention, and retention analysis respectively.

patients had suffered during the previous admissions from the discharge documentation. It made us believe there was a scope for improvement in our discharge documentation. To improve the quality of discharge summary given from the Pediatric ward in our institute, we implemented a longitudinal mixed educational and audit-feedback program.

Methodology:
Study settings: The study was conducted in the pediatric ward of a tertiary university teaching hospital. We have a 60 bedded pediatric ward with an average occupancy of 65%, catering to children aged from 1 month to 18 years requiring non-critical indoor hospital treatment. The Department has three units, each having three consultants and two to three residents (one rst-year resident (R1) and one or two second or third-year residents(R2 or R3)). Total 18 residents got rotated into one or two months postings among wards, neonatal care area including Neonatal Intensive Care Unit (NICU) and Pediatric Intensive Care Unit (PICU). Consultants usually remained in the designated unit unless some compelling situations. We used a computerized discharge summary template (Annexure 1) in the ward long before this study began. We provide a word processor typed and then printed discharge summary, but not an automated one. The patient's details were entered in a soft copy of a template, relevant changes were made, and the summary was printed in two copies. One copy is given to the patient at the actual time of discharge, while the other was kept along with the Medical Records Department (MRD) le. The discharge summary is always prepared by R1 in the unit under the guidance of R2 or R3, not consistently reviewed by a consultant.
As mentioned, we perceived the scope for improvement in the discharge summaries given from the ward. All the concerned consultants agreed upon this need and met to plan improvement in February 2018.
There was no formal education regarding documentation skills either during the under-graduate or postgraduate study in our institute. The effect of a single educational intervention tends to vanish with time.
Thus, we planned longitudinal ongoing educational efforts with necessary system changes.

Interventions:
The following interventions were placed into force from May 2018. The new batch of residents join each year in May in our institute, and their academic year is from May to April.
Small group discussion -Preferably, the consultants in each unit led the small group discussion with the 2 or 3 residents posted in the unit. The residents from other units of the ward or PICU (physically closer) were invited to join if they were free. Before implementation, all the concerned consultants had identi ed the lacuna and areas for improvement in discharge documentation by informal assessment. The content of small group discussion for this purpose was also outlined beforehand. The content included the importance of good quality summary and its components, how we can improve the quality of summary balancing the necessary detailing with the brevity, clarity of language and sentence framing, documentation skill, etc. This session usually lasted for 30 to 40 minutes. This was repeated every one or two months by each unit in a way that each newly rotated resident team in the unit would have one such exposure during their unit posting. On an average academic year, a single resident spent four to ve months in the ward, got rotated in at least two or three units, and that number of exposure to small group discussion he/she would have.
Team-based audit and feedback -This activity was used to arrange few days after the small group discussion. One summary from the previously posted resident team was selected for the audit. The current resident team of the unit was asked to review the summary and gave their impression of it. They were told to identify the weaknesses and strengths in that summary. Then the feedback was given by the consultant, discussing the ways to address the weaknesses and the lesson learned from the strengths. Again, this was also arranged in a way that each resident team would have one such exposure during their one to two months unit posting.
Improvised working practices -1. All consultants were asked to make it a routine to plan and discuss discharge processes, including relevant points in summary documentation during ward rounds.
2. It was then required for R2 or R3 to recheck all the summaries once prepared by R1 individually.
3. Consultants had to check and review the summary of a patient having a complicated or lengthy (≥7days) stay before their discharge.
We made all the possible efforts to adhere to the identi ed interventions and incorporated these steps into our working habits. The better-performing unit was rewarded and appreciated during the departmental meetings. The identi ed weaknesses were addressed for improving the documentation skill of residents, and no personal criticisms were made. Our self-reported compliance for the small group discussion and team-based feedback activity was 80% of all the available occasions among all three units after one year in May 2019.
Discharge Summary assessment tool: We did not nd any standardized pediatric discharge summary assessment tool for the Indian context when we began the study. Earlier studies over pediatric discharge summary assessed the presence or absence of pre-speci ed components before, during, or after the interventions. [16,17] We were more concerned about improving documentation quality than just the presence or absence of the speci ed components. The computerized discharge summary template was already in place, which could ensure the presence of the necessary components. It is challenging to mitigate subjectivity while assessing discharge documents rather than any documentation for that sack. The discharge summary assessment tool was developed in the past for the adult population. Reinke  They used ve points Likert scale for scoring summary elements. Rao et al. [19] had scored based on the summary elements that needed inclusion, and unnecessary elements needed exclusion, and also gave weightage for subjective components like clarity and consistency. Coit et al. [20] scored 22 elements as 0, 1 or 2 out of 1 or 2, and gave different weightage with multiplier 1 or 2 depending upon their deemed importance in summary. They also developed a rubric for scoring. The percentage of possible scores earned was taken into account. This scoring system was later used by Axon et al. [21] with some modi cations.
A novel scoring tool was devised for the quality assessment of a pediatric discharge summary given at our ward. Pediatric patient pro le is quite diverse in many aspects. The communication value of each summary element varies widely depending upon the patient and his diagnosis. Suppose there is not much value in describing perinatal history or developmental history in 17 years old admitted for Viral fever; against these components in the history will become much more critical for a six-month-old admitted for convulsion and developmental delay. Considering this and guided by the previous literature, [2,4,[7][8][9][18][19][20][21] scoring tool components and their weightage were decided. (Annexure 2) A total of 33 components were judged under seven broad headings: Basic Information, History, Physical Examination, Course during Hospitalization, Investigational Data, Diagnosis, and Discharge Plan. Most of the components were scored 0 or 0.5 or 1 out of 1, but the components of importance like complain leading to hospitalization, physical examination at admission and discharge, hospital course, diagnosis, and discharge medication list were given weightage by scoring them 0, 1 or 2 out of 2. History components like past, perinatal and developmental history, and detailed anthropometric measurements could be score out of 1 or 2 depending upon their importance on a case-to-case basis, as exempli ed above. Some of the components may not be applicable in a particular case, like a cross-departmental reference note. If no cross-departmental reference was made in a particular child, that component could be nulli ed by scoring it 0 out of 0. The percentage value of the obtained score was taken into Analysis. Thus, the overall scoring tool skeletal was xed and framed, leaving behind the descriptive liberty for a diverse patient pro le. The scoring tool was repeatedly reviewed and re-evaluated, considering each author's practical experience before the nal version was approved. The scoring rubric or descriptor of components (Annexure 3) was also developed to minimize inter-assessor variation. The consistency in scoring between the assessors for total score was checked based on ten summaries assessment (other than the summaries taken into the nal assessment).
Outcome Measurement: The assessment was done on the discharge summaries of the child admitted for more than 48 hours in the hospital and getting discharged from the Pediatric ward. The child could either got admitted from the OPD, emergency room, or step down from PICU. The summaries of the patient directly getting discharge from the PICU were not included. The summaries of the patient referred or transferred to other hospitals or those taking Discharge Against Medical Advice (DAMA) were also excluded for maintaining uniformity. The summary's soft copy was obtained from the hospital software system and the hard copy from MRD les. Thirty-three summaries were selected from each last quarter (October to December) of the calendar year of 2017, 2018, and 2019. As the program was implemented in May 2018 and then continued, 2017 was assessed as a pre-implementation period, 2018 as a post-implementation, and 2019 for continuation or maintenance effect. The summaries were selected with a random number generator, but each of the three ward units' equal representation was ensured. Then, these were assessed and scored on an excel spreadsheet by the three of the authors. The details of the summaries were cross-veri ed from the indoor le or from the hospital software system. Each author was given summaries from the units other than his/her own to avoid bias.
Sample Size: The total percentage score was considered as the basis for sample size calculations. A moderate effect size of 0.7 was thought to be worthwhile. With this effect size, a sample of size 32 was required to achieve 80% power allowing for 5% type I (alpha) error. A random sample of 33 from each year was drawn with the expectation to review at least 30 summaries from each year.
Statistical Analysis: The scoring consistency between the assessors was evaluated using Cronbach's Alpha with assessors as domains. Descriptive statistics [Mean( SD), Frequency (%)] were used to present the quality of discharge summaries at different time points. The percent mean (SD) domain scores and the total nal score were compared using Analysis of variance (ANOVA) with posthoc comparisons by Scheffe's test as the summaries involved were different at different time points.
Institutional Ethics Committee approved the study, and waiver of the informed consent was obtained, as only hospital records were checked and no patients were directly involved in the study.

Result:
The Cronbach's alpha was good in a pilot testing for total score with assessors as domains (α = 0.72). Out of 33 summaries from 2017, one patient le was not traceable at the time of the study. Similarly, the reviews of 30 and 32 discharge summaries could be done for 2018 and 2019, respectively. is shown as per the number (%) of the precise observations. The term 'Precise' means total score for the components, ie. 1 out of 1, or 2 out of 2. Figure 1 shows Total Mean (SD) % scoring in the Box plot diagram across the years.
Analysis of Variance (ANOVA) revealed that the Mean(SD) total % score improved signi cantly (P<0.001). However, the post-hoc test showed that the improvement was signi cant (p=0.01) from 2017 to 2018, but the score was similar from 2018 to 2019 (p=0.43).
Scores for the 'Basic information' domain were good to start with and remained so. The 'Investigational data' was improved from 2017 to 2018 but was no improvement then after. The 'Discharge Plan' domain was also better to start with, and there was improvement also. Notable improvement was evident in the components of education and counseling advice and Danger signs explanation.
We decided by consensus that a discharge summary that scores 80% or more will be termed as

Discussion:
The pre-intervention total Mean (SD) % scoring for DS in 2017 was 78.50 (8.23), suggesting a good overall standard of DS even before starting, making further improvement more challenging. Nevertheless, with the well-implemented strategies and teamwork, statistically signi cant improvement was demonstrated in 2018, and further improvement in % scoring in 2019. On analyzing the domain's performance, 'Basic Information' was excelling at the start and continued. As the template was already in use, there was no chance of omitting such information, and de ciencies were related to typos and erroneous entries. This further can be recti ed by semi-automation via auto-generating the information from hospital electronic medical records. We had requested the system department to explore such a possibility. 'History' and 'Physical examination' domains were showing consistent improvement in each of the two following years. A dietary history is an essential component of the pediatric patient pro le, and its non-improvement was worrisome, even when it was relevant to note the same. On the root cause analysis, it was evident that our template (Annexure 1) had not given separate space to enter the dietary history. This can be a limitation of the template system whereby the user enters only under the headings and does not add the necessary information or omit the unnecessary one even if liberty is given. It compromises brevity in the absence of useful information. An error in the template prone to get replicated until corrected. Later on, the template was corrected with the inclusion of some of the local language (Gujarati) instructions in the Discharge advice and home care sections (Annexure 4). We cannot understand the reason behind the dip in the performance in 2018 for the component of Course and outcome during hospitalization under the domain 'Course during hospitalization.' It may be a random error in the selection of the summaries.
Nonetheless, its performance was not up to the expectation even in 2019. We assume this component involves lots of descriptive narration, particularly for the patients with lengthy, complicated stays.
Narrative ability varies highly among individuals and cannot be improved readily with educational efforts. On the other way, assessment of such ability is again a subjective phenomenon. It cannot be objecti ed entirely with the scoring of 0, 1 or 2. 'Investigational data' domain had shown due improvement. 'Diagnosis,' even though it was better to start with, being the most important DS component, we were disheartened to note no further improvement. A secondary diagnosis like undernutrition, anemia and other associated health conditions were missed. It was pleasing that the domain of 'Discharge plan' had demonstrated the improvement. An earlier study had demonstrated inadequacies in documenting tests with pending results and follow-up service providers [11], but we were better on both the components for all three years.
Aspects like documentation and communication were traditionally not addressed in medical education, even at the post-graduate level, with the notion that those skills will be gained with experience [3]. Although communication is given importance in the recently adopted new medical education policy for undergraduates in India, the related documentation needs have not been addressed in detail [22]. Earlier, many workers had successfully adopted different strategies to improve discharge documentation. Some of the simple strategies like educational intervention, including didactic lecture or small group teaching, audit and feedback, use of template and reminder, etc., were proven bene cial repeatedly [14][15][16]19], but longitudinal effects were not assessed in most of the studies. Talwarkar et al. had adopted longitudinal monthly workshop provided feedback and opportunity for peer assessment demonstrated continued bene t for two consecutive years [4]. Axon et al. implemented a multi-pronged approach including DS curriculum with online availability of a lecture, template use, individual and team-based feedback with academic detailing. They assessed the pre-post interventional effects [21]. Almidani et al. had focused entirely on work system change by laying down a set of rules to be followed and assessing the number of delinquent records quarterly for three years [17]. Coit et al. had concluded that reducing residents' workload can signi cantly improve the DS quality [20].
We have implemented a mixture of strategies suitable for our unit. We gave education in a small group setting, followed by an opportunity for peer assessment along with team-based feedback. Appropriate work system changes supported these two activities. Interventions were disseminated in the unit-based system, ensuring each resident had exposure to education and peer assessment at least two times a year.
In an academic center, each year novice team enters, and an experienced team of residents leaves. It makes necessary to continue tested interventions for continued quality improvement. However, our perceived experience is that newly entered residents would not take long to demonstrate the desired quality output when the standardized work culture is established. We introduced the unique scoring system for the pediatric discharge summary and tested for inter-rater reliability. We could not nd any such scoring for the pediatric Department in the literature. Though, the scoring was designed aligning with our working and documentation system. This can be explored and tested at the various centers.
There can be certain limitations of the study. By implementing a mixture of strategies simultaneously, the effect of individual strategy could not be tested. As the study aimed overall improvement in the quality of the DS given, all the residents were involved in the study, and the control group could not be a part of this study design. Different units delivered the educational intervention and peer assessment/feedback activities, so the delivery style and preferences could be varied, albeit content was agreed upon at the onset. We had not assessed the con dence and satisfaction of the involved participants subjectively before and after the program. However, it appeared the program was well-received among the residents.
Rather the preparation of the improvised template (Annexure 4) was primarily headed by the group of the residents. One may relate the observed improvement with the 'Hawthorne Effect,' a phenomenon of improved human behavior and performance when aware of being observed [23]. However, such an effect will always be short-lasting and cannot be continued consecutive year after year. Moreover, neither the residents nor the unit consultants knew from which timeframe summaries for the testing would be selected.

Conclusion:
Considerable improvement in the discharge summary quality prepared by the residents in a teaching institute can be achieved with the approaches like small group educational intervention, peer assessment with feedback, and the work system changes.  Figure 1 Box plots depicting Total % scoring of discharge summaries over years