Scope
To develop the COS-LEP protocol guidance set out by the Core Outcome Measures in Effectiveness Trials (COMET) initiative, and further specified by the Core Outcome Set-Standardised Protocol Items (COS-STAP) statement, were followed.(1, 9, 10) These provide standardised frameworks for the development of a COS. The COMET database (available at https://www.comet-initiative.org/Studies) was searched for any pre-existing COS protocols for leptospirosis. When it was confirmed that there were none, this protocol was then registered (https://www.cometinitiative.org/Studies/Details/2536). The planned start date of the study is July 2024, and the end date is October 2025. Technical review has been provided as part of the London School of Hygiene and Tropical Medicine (LSHTM) formal institutional Qualifying Exam process for PhD programs.
The population that will be targeted with this COS are any individuals working in healthcare who regularly look after patients diagnosed with leptospirosis disease, researchers and policymakers involved in leptospirosis research, and individuals or groups representing individuals who are at risk of or who have been diagnosed with leptospirosis disease.
The project will be carried out in five distinct phases, summarized in Fig. 1. The combined Phase 1 and 2 will consist of undertaking a systematic review of the literature for reported outcomes from quantitative and qualitative studies on the management of leptospirosis. In Phase 3, focused interviews to explore further outcomes of interest will be undertaken with healthcare providers and people at risk of leptospirosis infection. Phase 4 will consist of a multi-round Delphi survey process to begin constructing consensus around the outcome set. And Phase 5 will finalize the process with a consensus meeting.
The schedule of enrolments and assessments is provided in Fig. 2, as per Standard Protocol Items: Recommendations for Interventional Trials statement (SPIRIT).(11) A SPIRIT checklist is provided as a supplementary document (Supplementary 1).
Figure 2 - SPIRIT figure detailing chronology of activities by study phase.
Phase 1 and 2
Phase 1 and 2 will establish existing knowledge about outcomes and methodology used in previous quantitative and qualitative studies looking at the treatment of human leptospirosis. To explore outcomes, a systematic review of outcomes from previously published quantitative and qualitative studies will be undertaken. The protocol for this systematic review has already been registered with the PROSPERO system in detail (PROSPERO ID CRD42023397461). In addition, systematic Cochrane Library reviews will be completed exploring methodology of all known clinical trials for human leptospirosis treatment, and determine efficacy and safety outcomes reported to date for evaluations of therapeutic interventions.(12–14) The output from all of these initiatives will provide the basis for options to put forward at later agreement stages.
Study selection
The following databases will be included in the search: the Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, LILACS, Web of Science Core Collection, Clinicaltrials.gov, and OpenSIGLE. The latter will be particularly important for the identification of grey literature sources. In each identified study, forward and backward reference searching will be used to identify further relevant publications.
Any study focusing on human leptospirosis will be included, across all age groups in a global distribution. Non peer-reviewed studies, most likely grey literature sources, may be considered. We will assess studies in batches chronologically by descending years in 5-year frames, until we achieve saturation in outcome types.
Full manuscripts will be prioritised, but abstract-only publications may be considered particularly if the full publication is unable to be obtained from the publisher or manuscript author. The search will not be initially limited by language, but studies may be excluded depending on which languages reviewing authors are able to speak. Further requirements, as identified by the PRISMA-S checklist, will be applied.(15)
Data extraction
Studies will be screened using the Covidence systematic review software (https://www.covidence.org/). Title and abstract review, followed by full text review, will be performed by two reviewers. A third reviewer will be consulted if there is a disagreement between the first two reviewers that cannot be resolved through discussion.
Data will be collected on the following;
1) Study design and sample size
2) Study population – demographics, location
3) Intervention
4) Outcomes
5) Outcome measurements instruments and/or definitions provided by the authors for each outcome
6) Study conclusions
Quality Assessment
The primary author will assess the quality of included studies in accordance to the QualSyst tool.(16) Risk of bias in outcome reporting will be assessed according to the Outcome Reporting Bias in Trials (ORBIT) study classification system, and the ultimate goal will be to provide context to outcome measures rather than a basis for study exclusion.(17)
Data Analysis
Data to be collected will be extracted verbatim. These will then be grouped based on terminology into group names, and then further grouped into domains.(1) With respect to outcomes, as names and domains are dependent on the outcomes identified, these will not be defined a priori. A narrative synthesis of the included studies will be performed focused on the types of data collected defined above. The identified groupings will be mapped to the outcome framework proposed by the COMET group.(1) Subgroup analysis will be considered if there are defined populations targeted by a particular outcome.
Phase 2 specific considerations
COMET handbook guidelines recommend the systematic review and identification of studies that specifically explore outcomes important to health service users.(1) These may either be in the form of previously published Patient Reported Outcome Measures (PROMs) or studies employing qualitative methodology.
It is not expected that a significant number of qualitative studies exploring treatment outcomes or studies establishing PROMs for leptospirosis have been previously published. If identified, a similar study selection and appraisal process as described in Phase 1 will be employed, focused on studies with a qualitative or mixed-methods design (where qualitative data collection and analysis methods were employed). With regards to data extraction, previously published methodology for extracting and analysing outcomes from studies reporting PROMs (18) and qualitative literature (19–22) will be used. Concepts and themes from excerpts will be categorized and tabulated to obtain frequency information. Categories will be mapped against the proposed COMET taxonomy, and if multiple domains are appropriate then two domains will be chosen as per guidance and precedence.(1, 20)
Phase 3
Central to the COS-STAP statement is the definition of stakeholder groups who will be involved in the COS development process including how individuals will be identified, and the nature of their involvement.(9) This is particularly important as it is expected that identification of PROMs through the systematic review of qualitative literature may not yield many results.
Therefore Phase 3 will involve exploring outcomes of interest with healthcare providers and end-users directly affected by leptospirosis, to identify outcomes important to these groups, through the use of qualitative methods. This will, in addition, identify target population groups, explore acceptability of therapeutic interventions, and assess feasibility of conducting trial activities.
Participants Recruitment and Sample Size
Participant identification and recruitment will be done from collaborative institutional sites in the COS-LEP network. The institutions in the network are secondary/tertiary hospitals primarily found in areas endemic for leptospirosis – the Philippines, Malaysia, Vietnam, New Caledonia, and Brazil. This may extend to other institutional sites in the same or different countries during the COS study (with appropriate local ethics agreements obtained prior to any enrolment activities).
Participants who are healthcare providers that manage patients with leptospirosis will be identified and approached for recruitment. Additionally those at risk of leptospirosis, or who have made contact with healthcare services and for whom leptospirosis is included in the differential diagnosis, will also be identified and approached for recruitment. They will be provided with a patient information leaflet, as well as a verbal description of the study and its intention. After a 24 hour consideration period, they will be approached again and consented for enrolment.
Purposive sampling will be employed. Sample size calculations is a point of contention in qualitative studies, but methodological literature recommends that the reasoning behind any calculation be made transparent.(23) Recent qualitative studies exploring individual perspectives of participant’s experience with leptospirosis disease have contained < 10 participants(24, 25). Since focused interviews and thematic analysis will be used, and we will aim to recruit from international sites where collaborative institutional research is currently ongoing, we will aim to recruit at least 6 participants from each site. This may be revised should further sites be identified. The overall aim will be to achieve theoretical sufficiency, as defined as a point when a researcher has achieved sufficient or adequate depth of understanding to build a theory.(23, 26) In addition, the total predicted sample size would approximate or supersede that of other COS-related published research using focused interviews.(24, 27)
Data Collection
Focused interviews will be conducted, aided by a general topic guide. These will either be in person or remote/via teleconferencing means, the latter which now has a growing body of evidence supporting its use in qualitative methodology.(28) For those wishing to participate, but have limited fluency in English, attempts will be made to provide an interpreter subject to availability of personnel and/or funding. A very broad topic guide will be used, but the interview will be guided primarily by participants. Interviews will be audio recorded, and then anonymized and transcribed using the NVivo software (QRS International). Once all transcriptions are complete, interview recordings will be securely destroyed.
Data Analysis
Thematic analysis will be employed to summarize transcripts.(29) Features in the transcripts involving outcomes will be coded, subsequent codes collated into potential themes, themes revised and then defined. The generated themes will then be mapped to against the proposed taxonomy as stated in the COMET handbook, although consideration will be made to include if there is no adequate classification. Common themes in the classifications will be identified and outcome definitions that closely approximate each other will be removed.
Phase 4
The next stage of the COS development process will obtain a broad consensus on the proposed set of core outcomes identified in Phase 1–3. The recommended methodology to accomplish this is the Delphi technique targeted at experts and other stakeholders through the use an evolving questionnaire and multiple rounds of consensus surveys.(1)
Participant Recruitment and Sample Size
The stakeholders of interest for participation in surveys are patients (those who experienced leptospirosis disease), infectious diseases healthcare professionals (including medical microbiologist), non-infectious diseases healthcare professionals (including nephrology, pulmonology, intensive care, family medicine or other specialties), allied global health researchers (including epidemiologists, trialists, statisticians, and one health), and policymakers (both governmental and non-governmental). As many stakeholders as possible in a worldwide geographic spread will be included. Guidance on total number of stakeholders to be included is not based on statistical power but a pragmatic choice.(1) We will seek to include sufficient participants per group to achieve good representation, but which will still allow for a consensus within group to be achieved.
Stakeholders will be approached for recruitment using several communication avenues both in-person and virtual. Participation in surveys will be advertised to patients by direct means (at collaborative sites and those identified during Phase 3), dissemination through hospital communications at collaborative sites, through a project-specific website, and through social media. Researchers active in human leptospirosis will be identified from studies collated in Phase 1–2 of the COS process. Healthcare professionals of any specialities will be identified by direct means at collaborative sites and through various professional networks of research teams. Policymakers will be identified through recommendation, existing national/international leptospirosis guidance, and from committee membership or involvement on various international leptospirosis societies. All participants will be requested to disseminate information through their various networks in order to increase uptake.
Potential participants expressing interesting in taking part in the survey will be directed to a study page provided by the Qualtrics platform that will provide study information in lay terms with full definition of medical terminology if used. Basic information such as demographics and suitability to participate will be assessed if potential participants wish to proceed. If these are met, then they will be able to proceed to the Delphi survey, with consent being gained electronically before undertaking the questionnaire.
Data Collection
The COMET Initiative has offered guidance that at least two survey rounds take place in order to incorporate feedback.(1) The minimum number of rounds for this study will be two but will be extended if further consensus is required. The survey will be built using the Qualtrics survey software (https://www.qualtrics.com) with a focus on lay language minimizing medical jargon (or with definitions included). We will attempt to engage with patients/members of the public identified from previous Phases of the COS-LEP study to develop and trial this lay terminology. Translations into other languages for particular groups will be explored depending on the participant groups included. Trials of the survey will be run amongst the study group and colleagues, and feedback incorporated back into the survey design.
The COMET initiative suggests the use of the 9-point Likert scale where outcomes are graded in accordance to their level of importance, which is also the recommended framework by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group.(1, 30) This uses a scale between 1 and 9, with number 1–3 defined as outcome of limited importance, 4–6 outcome of importance but not critical, and 7–9 meaning critical outcome. There will be an additional grade of “Unable to score” should participants feel they do not have the level of expertise to score a certain outcome.(1) The order of outcomes to be scored might introduce a bias, and so outcomes order will be randomized as used in other COS studies.(20) Space will be provided at the point of rating each outcome and at the end of the questionnaire for participants to list additional outcomes not considered or to provide feedback on the list of outcomes. These will be reviewed between rounds by the study group and select patient representatives to determine classification within COMET taxonomy, how different the proposed outcome is from the existing listed outcome, and ultimately whether appropriate for inclusion in next round.
There is wide variability in definition of whether individual outcomes meet a consensus for inclusion or exclusion of subsequent surveys, or at the end of the study. The COMET handbook does not provide a clear framework, but suggests more inclusive criteria be used in subsequent survey rounds should items be excluded in the intervening period.(1) Relevant results from the first survey round will be presented to participants in the second round, presented graphically and by stakeholder group. This will enable participants to compare their prior scores within their stakeholder group and with other groups in the survey process. This study will adopt criteria used in previous COS studies, with less stringent criteria in earlier rounds.(20, 31–33). Common in all rounds will be 1) the definition of consensus out from next round, which is any outcome with ≥70% of all participants rated 1–3 (not important) for an outcome in any round; 2) any score not meeting inclusion or exclusion criteria will defined as no consensus; and 3) A participant’s score will be included for analysis if they complete 70% or more of outcomes.
• Round 1. Inclusion in next round – any outcome rated 7–9 (critical outcome) by > 50% of any participant and 1–3 (limited importance) by no more than 15% of any single stakeholder group.
• Round 2. Inclusion - any outcome rated 7–9 (critical outcome) by > 70% of any participant, and 1–3 (limited importance) by no more than 15% of any single stakeholder group.
Should there remain a large number of outcomes following two rounds as decided by the study group a further third survey round may be conducted, with the outcome inclusion criteria to take forward to a consensus meeting being as for Round 2.
Addressing Attrition
There is a risk of non-response by participants at all stages of the Delphi process, likely made greater by the international scope of the study. In order to decrease rate of non-responses, surveys will be kept open for four weeks and up to three reminder e-mails per round will be sent to participants with incomplete surveys. Details of current response rate and reminders of survey closing dates will be included in each message. If response rates remain low, consideration will be made for additional measures to increase this with consideration to time and funding. We will aim for 80% response per stakeholder group.
Data Analysis
The distribution of outcomes for inclusion moving forward and degree of consensus will be summarized at the end of each round.(1) Means and medians for each outcome will be calculated. The change of scores between rounds will be assessed by change in percentage, reduction in spread of scores by comparing standard deviations, and reductions in interquartile range.(1) Metric will be reported for all participants, and inter-group between stakeholders.
Phase 5
The final phase of the COS development process will involve a consensus meeting where representatives of the various stakeholder groups can convene to discuss the results of the survey and decide on a final core set. Although we will aim for an in-person meeting, given the international scope of this study and the distribution of the stakeholder group, we may explore the possibility of conducting a hybrid approach of virtual and in-person meeting. If using a hybrid meeting approach, we will use guidance published by the COMET Initiative.(34)
Participant Recruitment and Sample Size
Building on standard proposed by the CODECS study, we will aim for 20 participants (4 from each stakeholder group to allow consensus to be achieved based on our revised consensus criteria) will be purposively selected for inclusion in a minimum of one consensus meeting.(20) We will adjust this number depending on interest and the practicalities/logistics of attending an in-person meeting. Although any participant may be included, to provide transparency in selection the final Delphi survey round will include a query of whether a participant is able to attend the consensus meeting. Consideration will also be made to include participants who did not respond or dropped out during the Delphi survey. We will explore the possibility of using break-out groups depending on total group size and availability of facilitators who can be trained to moderate as used in other COS development studies.(35)
The final number of participants and number of meetings will be a pragmatic decision based on expressed interest in participation, feasibility of joining a meeting, and ensuring representative input from all participants.
Data collection
The consensus meeting will consist of an introduction of the aims of the COS-LEP study, a summary of Phase 1–4 activities and results, the discussions of the final set of outcomes, and a final vote on outcomes. The meeting will be recorded and take place over 2–3 hours, but may be longer or divided into sessions depending on the number of participants, availability, and number of outcomes. Outcomes to be discussed will primarily be those that were selected in the Phase 4 Delphi survey, but outcomes which were excluded may be re-introduced and discussed again depending on participant interest.
Within each stakeholder group > 70% votes for either direct inclusion or exclusion of outcomes will be considered consensus. If there are issues with total number of participants or a limited number of stakeholder groups attending, we may consider using a threshold of > 70% across the entire group. These criteria have been proposed by the COMET Initiative as well as other COS developers such as the Outcome Measures in Rheumatology (OMERACT) group.(10, 36, 37) Results will be presented after voting, and reported by stakeholder group as well as across the entire group. Outcomes which are excluded or which do not meet the criteria of consensus can be reviewed by participants where there is opportunity to discuss any reason to disagree with these results. Should the number of included outcomes be large, a method to classify these following previous examples such as composite or tiered outcomes will be discussed in group.(37, 38) Time permitting, the meeting may also be used to agree outcome definitions and define outcome measures.(1) The meeting will be transcribed and analysed using thematic analysis to explore participant’s perspectives on the agreed core outcomes, barriers to implementing, and any recommendation or suggestions for proceeding.(39) The outcome of Phase 5 will be the final core outcome set.
Dissemination of Results
The complete results of the COS study will be reported in international peer-reviewed open-access journals, international and national scientific and policymaking meetings, shared to clinical trial registries, and directly through all study group and participant networks. All study reports will follow reporting standards defined in the COS-STAR statement.(7)