The Global Surgery Research Unit of the Latin American Indicator Research Collaboratory: An Action Model for Grassroots Collection of Lancet Commission on Global Surgery Indicators

Background. The Lancet Commission on Global Surgery (LCoGS) recommends collection of six surgical indicators to inform surgical system strengthening and improve access to surgical care. However, a standardized practice has yet to be described to guide nations towards this endeavor via a sustainable and reproducible grassroots intervention (i.e., developing information management). Methods. A research team structure, termed a “global surgery research unit” (GSRU), was developed between a hospital in Medellin, Colombia and Rutgers-Robert Wood Johnson Medical School. The GSRU implemented a grassroots intervention in a single center over three months to study the feasibility of obtaining LCoGS indicators 1-4 via hospital medical records, administrative surveys, and patient interviews. The intervention was then retrospectively evaluated utilizing a review of implementation outcomes. Results. During the feasibility study period, 84.5% of general/ trauma surgery patients interviewed reported traveling less than two hours. The workforce includes 15 general surgeons, 34 anesthesiologists, and 9 obstetrician-gynecologists. Total surgical volume is estimated to be 15,180 procedures per year, with a 30-day postoperative mortality rate of 1.8%. Evaluation of implementation outcomes revealed grassroots, single center collection of LCoGS indicators was feasible. However, increased workforce and time commitment is required for sustainability. single center. Grassroots interventions may inform practical collection individual hospital indicators hospital system. model ecacy and expand

Four of these six core surgical indicators have since been adopted by the World Bank as World Development Indicators. This milestone in the effort to assess and improve the surgical system globally has focused international attention and aligned national priorities through a common discussion (4)(5)(6)(7). Additional advocacy by the World Health Organization (WHO) through the World Health Assembly Resolution 68.15 has encouraged this data collection to support the development and implementation of National Surgical, Obstetric, and Anesthesia Plans (NSOAPs)(8). Following NSOAP development and implementation, iterative collection of these surgical indicators may facilitate continuous downstream evaluation and monitoring (9).
Although LCoGS proposed these core indicator metrics with targets, no speci c guidance was provided on how to start and iteratively improve grassroots collection of indicator data in settings without coordinated health system information management mechanisms. To globally report these indicators to the World Bank, a su ciently detailed, sustainable, and cost-effective methodology to achieve timely data collection is necessary. Current efforts to collect LCoGS indicator data rely on top-down approaches, requiring involvement of Ministries of Health, national healthcare informatics and database management, and hospital directors interested in population metrics extrapolated from grassroots levels of care, i.e., clinical settings (5,(10)(11)(12)(13)(14). Although top-down methods enable collection of large volume data, incomplete documentation, fragmented data sources, and exclusion of small hospitals during national data sampling may compromise the accuracy and generalizability of results. Additionally, measurement and selection bias may be introduced, particularly in LMICs with variable health information management system technology (5,10,15). As such, a facility-level analysis of hospital surgical capabilities to accurately collect core surgical indicator data is needed, particularly in small hospitals which may otherwise be under-represented in a national evaluation. Additionally, grassroots methods may also inform health system performance speci c to each hospital via continuous data surveillance of these indicators to inform processes such as quality improvement, budget planning, resource allocation, and training needs.
Presently there is a lack of literature describing systematic grassroots collection of LCoGS indicators and a link to national agenda. Ideally a proposed grass-roots data collection effort should be objectively evaluated for its e cacy, effectiveness, and equity. Dissemination and implementation (D&I) research methods are well suited to assess intervention processes using described implementation outcome variables and process appropriate frameworks(16-18). Implementation science has been used successfully to objectively characterize, guide, or evaluate contemporary interventions in health services research and academic global surgery(16-22). Appropriately chosen D&I frameworks enable identi cation of barriers and facilitators to an intervention (i.e., protocol implementation success) through a systematic process, structure, and outcome assessment(16). Proctor and colleagues described eight implementation outcomes (acceptability, adoption, appropriateness, cost, feasibility, delity, penetration, and sustainability) for the systematic assessment of intervention e cacy and effectiveness (Table 2). To date, D&I methodologies have yet to be applied to assess and improve the process and structure of LCoGS indicator data collection, analysis, and interpretation. Perception among stakeholders that a given treatment, service, practice, or innovation is agreeable or satisfactory

Adoption
The intention or action to employ an innovation Appropriateness The perceived t, relevance, or compatibility of the innovation for a given practice setting; and/or perceived t of the innovation to address a particular problem

Cost
The cost impact of an implementation effort

Feasibility
The extent to which a new innovation can be successfully carried out within a given setting

Fidelity
The degree to which an intervention was implemented as it was prescribed in the original protocol or as it was intended by the program developers

Penetration
The integration of a practice within a service setting

Sustainability
The extent to which a newly implemented intervention is maintained within a setting's ongoing operations In this study, we describe and evaluate a participatory action, team-based global surgery research model in Medellin, Colombia for grassroots LCoGS core surgical indicators 1-4 data collection. This model was qualitatively evaluated using D&I methods to determine e cacy and effectiveness using Proctor's implementation outcomes framework to identify promotors and barriers for iterative design improvement(16, 17).

Global Surgery Research Team Model and Setting
The Latin American Indicator Research Collaboratory (LAIRC) was created to advance development of a cost-effective, reproducible, and standardized methodology for surgical system strengthening at the grassroots level between North America and Latin America and the Caribbean (23). Foundational to the LAIRC is a participatory action research and team-based "twinning" partnership between a LMIC and a high-income country (HIC). Each unit of a twinned pair has been termed a global surgery research unit (GSRU), composed of a research fellow(s) and at least one faculty member. The overarching strategy is for the HIC GSRU to partner with the low-income country/ middle-income country (LIC/MIC) GSRU to understand local needs and co-develop research programs and appropriate solutions led by the LIC/MIC.  (24). This created a network composed of hospital, community, and student members who were engaged throughout all stages of the research process and were stakeholders for indicator data collection, emphasizing the community-based participatory approach of the GSRU model (25,26). These MIC and HIC GSRUs worked together to collect LCoGS core surgical indicator 1-4 data using a transnational protocol that was adapted based on iterative input and participation. The shared research protocol was approved by both institutional review boards.

Data Collection
The research protocol employed mixed methods consisting of patient interviews, hospital leadership interviews, and repurposing data from existing hospital databases for LCoGS indicator collection.
De nitions of variables related to surgical indicators 1-4, research methods, and results are summarized in Table 1. Team members from the Rutgers GSRU travelled to Medellin in June 2017 to participate in surgical indicator and patient demographic data collection.

Methods for Surgical Indicator Data Collection
Patient Demographics. Surgical and obstetrics/gynecology patient demographic data were collected for a one-month period via daily in-person patient interviews during morning rounds (Additional le 1).
Time to Hospital. Daily in-person patient interviews of general surgery, trauma surgery, and obstetrics/gynecology patients were conducted to collect data regarding transport time to the study facility. Newly admitted patients that underwent an operation on admission from these services were rounded on daily following their operation (Additional le 1).
Surgeon, Anesthesiologist, and Obstetrician (SAO) Complement. In-person, semi-structured interviews, adapted from the World Health Organization Hospital Assessment Tool (HAT), were conducted with hospital leaders (Additional le 2) (27). These data were used to determine the total number of surgeons, anesthesiologists, and obstetricians at the study facility.
Surgical Volume. Total surgical volume across all surgical specialties during the month of June 2017 was determined retrospectively from a pre-existing hospital administrative database.
Postoperative Mortality. For the month of June in 2017, all cause, non-risk adjusted, in-hospital, postoperative mortality data were collected retrospectively from a pre-existing hospital administrative database.

Implementation Outcomes Analysis
The GSRU model and data collection intervention processes were retrospectively analyzed using a questionnaire ( Table 3) that guided in-person, semi-structured interviews with Colombian GSRU stakeholders regarding the June 2017 experience. Interview questions were developed following the Page 7/18 conclusion of the 2017 experience and categorized into implementation outcomes based on the framework described by Proctor et al., describing acceptability, adoption, appropriateness, cost, feasibility, delity, penetration, and sustainability(16). Throughout the June 2017 experience, informal feedback was also recorded from interactions with Latin American stakeholders, including patients, students, physicians, and hospital administrators, and during weekly team meetings. This formal and informal feedback was similarly analyzed to identify barriers and facilitators to the GSRU implementation and data collection interventions. Were you satis ed with the data collection process?
Did you agree with the process that was taken to obtain approval for the project?
Did you agree with the process that was taken to form a GSRU for data collection?
Adoption Was the implementation of the GSRU for data collection adopted well by the hospital and staff?
What initial thoughts did you have when rst approached about this project?
How have these thoughts changed after implementation of the GSRU to collect indicator data?
Appropriateness Do you feel that the GSRU with multiple team members from both HIC and LMIC systems was the best way to collect data?
Did this intervention t your system?
What other support would increase data collection?
Do you have recommendation for an improved system based on how your hospital system is structured?
Cost What costs were incurred during the data collection process?
Were these costs more or less than expected?
Do you feel these costs may be a barrier to future data collection in other hospitals in Colombia?
Feasibility Do you feel GSRU is feasible to collect indicator data?
Did the GSRU structure t well within your hospital system to collect indicator data?
How can we improve the structure of the GSRU to improve data collection of all indicators simultaneously?
Fidelity Did the GSRU collect indicator data in the methods approved by ethics committee?
What modi cations were made to the process in order to improve the amount of data that was collected?
What future improvements do you suggest to improve GSRU data collection?

Penetration
Was there di culty with any plans in integrating with hospital system where indicator data collection occurred?

Evaluation of Implementation Outcomes
Analysis of the GSRU intervention is summarized using the implementation variables in Table 4. With respect to acceptability, GSRU stakeholders found the research protocol to be satisfactory. Latin American partners were motivated, informed, and readily formed relationships with hospital leaders and research fellows. However, lack of Latin American medical students' integration in the team was perceived as a barrier to implementation due to the need for a more robust research workforce. One Latin American clinical faculty member from the GSRU stated, "It is very important for [Colombian] students here to be participating in the local story" to best advocate for patients in their community. Appropriateness. Overall, both US and Colombian GSRU members felt the intervention to be appropriate, though it could be improved for indicator 1. They recognized that methods for indicator 1 may introduce recall bias during patient interviews. Additionally, some patients were transferred from other hospitals, and this interfacility transport time was captured during the patient interviews. However, we did not obtain transport time from incident location to the original hospital in these patients. The sum of these transport times is the true indicator 1 value.
Costs. The direct costs for the North American medical students and general surgery resident to travel and board in Colombia were funded entirely by the U.S. Agency for International Development-Research Innovation Fellowship Program (USAID-RIFP), with a scholarship award of approximately $2500 per student for the month. There were no additional direct costs associated with data collection. Indirect costs were identi ed as a barrier to long-term implementation of the intervention. However, the indirect costs, including determination of the cost of Latin American clinical faculty absence from clinical responsibilities due to involvement in surgical indicator data research, were not speci cally measured.
Feasibility. The protocol required a signi cant time commitment for GSRU research partners to collect data prospectively throughout the year due to the absence of Colombian medical student participation. When the GSRU teams were working together in tandem, the intervention was most feasible. The protocol requires assistance from and collaboration with hospital leaders to promote longitudinal data collection as well as with medical students to conduct daily interviews with patients (158 total patients interviewed, requiring approximately 15 minutes per patient), representing a signi cant unfunded time burden for both sister GSRUs and members. Additionally, each interview with the hospital leaders took approximately 15 minutes.
Fidelity. The protocol was closely adhered to as originally developed. GSRU members stated that the research protocol could be modi ed to t a range of hospital settings in Colombia, particularly those with hospital databases.
Penetration. This intervention led to collection of baseline data of LCoGS indicators 1-4 while in country.
The protocol was successfully integrated in the hospital setting while Rutgers GSRU members were presented in Medellin but failed to penetrate the hospital system following their departure.
Sustainability. GSRU members stated they were able to execute the research protocol in the short term while the two teams were working together. However, due to lack of su cient workforce and formal Colombian medical student engagement and participation, data collection ceased following the departure of North American partners.

Discussion
A standardized methodology for collecting surgical indicator data, now mandated by the World Health Organization, is necessary to inform the global surgical system situation analysis effort to address profound barriers in access to surgical care (4). Historically, efforts in LICs and MICs have focused on augmenting the health system domains of workforce, service delivery, and nance (15,(28)(29)(30)(31). To our knowledge, the present study is the rst reported effort to quantitatively and qualitatively assess a prospectively designed grassroots data collection intervention for LCoGS indicators 1-4. Our study suggests that this approach, predicated on a participatory action-based twinned-team model between academic units in an LIC/MIC and HIC, can be successfully implemented in an individual facility.
Whereas the majority of the literature to date has focused on indicator efforts in Africa and Asia, this is the rst directed grassroots effort to assess LCoGS indicator collection in Latin America.
Quantitatively, the team successfully collected data with respect to SOA system preparedness. In this busy urban academic bellwether capable facility, 83% of general/ trauma surgery and OBGYN patients reported a mean transport time within two hours (indicator 1) if inter-facility transportation was not required. However, 32.5% of surveyed patients required interfacility transportation for appropriate services, raising concern that a signi cant proportion of the population is outside of the 2 hour-access target set by the LCoGS. This is consistent with previous ndings that between approximately 15-30% of the Colombian population cannot readily access a bellwether capable facility within 2 hours (14,22). Regarding indicator 2, we were able to accurately determine the number of general surgeons, anesthesiologists, and obstetricians/gynecologists. While a true SAO density cannot be calculated because the catchment area for any given facility is not homogenous, in aggregate, this type of data can be used to determine indicator 2 at the municipal or departmental level.
In addition to preparedness, SOA system service delivery was assessed, and annual performance extrapolated. To begin to characterize indicator 3, the team identi ed 1265 operative procedures performed during the study month. Annualized, this corresponds to a total operative volume of 15,180 procedures. Although this projection is approximately 9% less than the actual volume performed (16,710 surgical procedures in 2017), this is an acceptable variance given the short sampling period. During the study period, we calculated an all-cause, non-risk adjusted 30-day, in-hospital postoperative mortality rate (POMR) of 1.8% (indicator 4). While this single center rate is 1.7-fold higher than the aggregate mean for the city of Medellin reported in 2016, it is within the range of 0.02%-4.78% reported across the 185 municipalities with observed post-operative deaths within 30-days. Furthermore, although this observed POMR is 2.4 fold greater than the reported Colombian median in 2016 (0.74%) (22), it is consistent with the observation that an increased POMR is correlated with areas of higher socioeconomic status and SAO density such as Medellin (10,22). While the underlying reason for this observation remains unexplored, the present estimate is subject to sampling bias given the short study period and case-mix index.
Qualitatively, we used Proctor's implementation outcomes framework to objectively evaluate our team construct (GSRU) and protocol for LCoGS indicator data collection. GSRU data collection activities were successfully performed by learners (medical students) with both local and remote academic faculty mentorship. Noteworthy, though not addressed in the present analysis, is the bene t to learners of cultural immersion, access to local mentorship, and international relationship development for future career growth. In this pilot, our Latin American partners did not incur any direct costs as this was borne by the USAID grant and HIC students themselves. Given the demonstrated success with which learners were able to navigate the hospital system and implement data collection protocols, it is anticipated that Colombian learner engagement would allow for career growth while minimizing future direct costs in the future. While this does not address future HIC learner travel expenses (estimated at $2500/learner/month), in balance, when considered through the lens of the average per credit hour education cost to learners in the US ($1537/hour) these direct costs are commensurate with the nototherwise available, unique, and now mandatory LCME/AAMC education experience (32). Nonetheless, identi cation of exogenous funding and cost sharing is an important consideration for long term sustainability of the program. Thus, this is a cost-effective model for initiation of bi-national indicator research with respect to HIC incurred direct expenses. Notwithstanding, the impact of the direct costs incurred, assessment of indirect costs is particularly applicable for the Latin American faculty component of the GSRU as their participation came at the expense of required clinical activities. Local resources to support this aspect of professional development and activity should be identi ed for future sustainability and growth based on the academic goals of faculty and departmental leadership.
Through the implementation outcomes framework, we identi ed the need for greater GSRU workforce. The time burden of data collection, speci cally patient and family interviews to determine indicator 1, was signi cant. The absence of participating Colombian learners in this pilot resulted in signi cant barriers to feasibility and sustainability of the protocol. However, despite this project's lack of MIC student participation in indicator data collection, the Colombian GSRU has since created a medical student group in collaboration with the organization International Student Surgical Network (InciSioN). This expansion of opportunities to Colombian medical students furthers learner engagement, promotes research, and provides an avenue for global partnerships and action.
There are several limitations in our study. This study was conducted at a single, private, academic bellwether facility. Human and material resource availability, as well as barriers and facilitators to research will differ in other facilities and regions. Consequently, the protocol and GSRU model may need to be adapted according to the resources of the next hospitals that apply the model. Implementation science principles may help to guide the iterative development and documentation of these efforts for dissemination. A second limitation was the short GSRU implementation period of approximately one month which may not be representative of the whole year. The implementation period was restricted due to the HIC medical students' academic calendar, which could only accommodate international travel during the described time period. A lack of student workforce in the Colombian GSRU prevented independent long-term data collection. Therefore, dissemination of education and training of HIC and MIC student participants to achieve greater MIC student involvement and more robust institutional engagement are underway. Gratifyingly, a signi cant increase in Colombian GSRU medical students' participation has been observed. Third, data collection was restricted to speci c patient populations due to established partnerships with the general/ trauma surgery and obstetrics-gynecology services in the host institution. This limited the GSRU's ability to perform data collection across all surgical specialties. Fourth, self-reported estimates of length of time to the facility was subject to recall bias resulting in an unmeasurable time variance as no objective documentation existed against which to verify self-reports.
Finally, we were unable to estimate a catchment area for this single hospital to inform an SAO density calculation as there are multiple tertiary hospitals with overlapping service areas in Medellin.
With respect to cost protection, we were unable to assess indicators 5 and 6 in this setting. Previous studies in Africa reporting on indicators 5 and 6 utilized an in-person questionnaire upon a patient's hospital discharge, which was successful as direct medical costs are upfront or otherwise known during hospitalization(28, 29). We did not nd this protocol to be adequate due to human resource limitations and inability to capture direct costs at time of hospitalization. Work is ongoing to develop a reliable and effective survey tool and the research infrastructure with which to accurately assess direct and indirect costs.

Conclusion
This study introduces a successful grassroots intervention in a single hospital setting for LCoGS indicator 1-4 collection using existing hospital resources and structured stakeholder and patient interviews. Further development of the research workforce recruitment paradigm and interview adaptation is needed for successful longitudinal data collection of indicators 1-6. Such grassroots efforts may inform municipal and facility quality improvement processes. Reciprocal or "twinning" partnerships and cultural pro ciency to LICs/MICs research processes, effective community participation, and local championship for research has promoted these early stages of collaborative LCoGS indicator data collection. Methods drawn from implementation outcome evaluation will be continuously utilized to evaluate, monitor, and improve the GSRU model. Establishing local championship (particularly targeting Latin American student research programs) and cross-sectoral involvement (including the Colombian Ministry of Health, Ministry of Education, and medical school programs) is imperative to ensure on-going data collection, analysis and interpretation. By objectively characterizing the facilitators and barriers to success of the GSRU model through the novel application of an implementation framework, this work may inform an iterative process building upon the GSRU incorporating grassroots partnerships in other countries to advance LCoGS indicator informed national public health and surgical health care plans. Availability of data and materials The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests
The authors declare that they have no competing interests.

Funding United States Agency for International Development -Research Fellowship Innovation Program
Authors' Contributions CM, JSH, and GLP made substantial contributions to the conception and design of the work. EEG, CM, and CM collected, analyzed, and interpreted the data. EEG drafted the manuscript. JSH and GLP provided substantial revisions to the manuscript. All authors read and approved the nal manuscript.