Search Characteristics and Reporting of Systematic Reviews of Animal Experiments: A Cross-Sectional Study

The formulation and implementation of a comprehensive search strategy is the basis for ensuring the quality of a systematic review (SR) and/or meta-analysis (MA). All published SRs/MAs of animal experiments were comprehensively identied, to evaluate how each search strategy was formulated, and how the search process and reporting were implemented, in order to represent a reference and provide suggestions for the future development of search procedures and reporting guidelines for SRs/MAs of animal experiments which should, ultimately, allow translation of the results of animal experiments into clinical practice.

Although complete search guidelines for SRs/MAs of clinical trials [7] currently exist, they are not fully applicable to animal experiments. In addition, researchers in the eld of basic medical research have little understanding of SR/MA methodology [8] . Thus, almost universally, the formulation of search strategies for published SRs/MAs of animal experiments is problematic, causing omissions and misidenti cation to a varying extent [9,10] . For example, Chen et al. [10] found that only 6.63% (12/181) of SRs/MAs of animal experiments in international journals reported the complete search strategy in at least one database.
Vriezen et al. [11] found that only 39.5% (15/38) of reviews additionally supplemented the search with reference lists in the study, and only 36.8% (14/38) supplemented with grey literature. Nesdill et al. [12,13] brie y described the basic characteristics of databases commonly used for searching animal studies but did not elaborate on the speci c methods for formulating the search strategies or implementing the SRs/MAs of animal experiments. In addition, although a number of researchers have set forth suggestions for the search procedure for the SRs/MAs in animal research [14] , they have generally only utilized their accumulated experience of searching without elaborating a standardized search process in detail, nor systematically explaining the problems within the search process and improvements in the integrity of the results. In 2010, Hooijmans et al. from the SYRCLE Animal Experimental Center of Radboud University Medical School in the Netherlands successively published animal experiment search lters for the PubMed [1] and Embase databases [15] . Although these increased the sensitivity of the searches to a certain extent, the speci city for animal experiments requires improvement, in addition to those lters being limited to the PubMed and Embase databases [15] . In addition, no standardized reporting guidance for searching for animal experiment SRs/MAs exists which also affects search quality to a large extent [16] . Therefore, methods of formulating and implementing extensive and comprehensive searches for the SRs/MAs of animal experiments, and standardization of the reporting of the search results remain an important challenge. Therefore, the present study aimed to review the following aspect of all published SRs/MAs of animal experiments: (1) Systematic evaluation of how each search strategy was formulated and its implementation, including the problems within the search process, so that the search methods could be standardized, preventing the relevant literature being overlooked, reducing the risk of bias, and improving the credibility of the evidence reported in animal experiment SRs/MAs; (2) Analysis of the reporting of search on animal experiment SRs/MAs and its problems, and to provide a reference for the development of animal experiment SRs/MAs reporting guidelines in the future, thereby promoting the translation of preclinical evidence to clinical utilization.

Eligibility Criteria
Inclusion criteria Medical-related SRs/MAs of animal experiments published internationally were included in the study, with no restrictions on animal species and publication language. SRs/MAs involving the results of both humans and animals were included. "Animal experiments" were de ned as in vivo experiments in which an intervention was performed on a living animal, without limiting whether the outcome was measured in vivo or not. The de nition of "Systematic review and/or Meta-Analysis" included studies in which "systematic review" or "meta-analysis" was clearly mentioned in the report, or the nature of the study could be determined by reading the results based on the summary of the multiple research reports it was reviewing, the methods used for literature review, and the reporting of exclusion criteria and quality evaluation methods, etc.
Exclusion criteria Studies of basic biology such as genetics studies, cellular molecular studies, physiological mechanisms, or behavior unrelated to human medicine were excluded, as were studies in which duplicate results were published. Conference papers or academic reports were excluded.

Search Strategy
The search strategies of Peters and Korevaar et al. [17,18]

Methodological quality control
Prior to document screening, all researchers (Bing Zhao, Fei Chen, Qianqian Gao, Fan Mei, Li Zhao, Jinwei Yang, Mingyue Jiao, and Yanbiao Jiang) underwent professional training, which included understanding the eligibility criteria, the literature screening process, and data extraction. Data were extracted on 10% of the included studies after training and before the formal search process to evaluate researcher consistency via calculation of the Kappa statistic. If Kappa was > 0.75, the training could be considered effective and researchers suitably quali ed.

Screening and Data Extraction
Three researchers (Bing Zhao, Fei Chen, Qianqian Gao) independently screened and cross-checked the literature in accordance with the eligibility criteria. In the case of disagreement, disputes were resolved through discussion or adjudication by a third party (Ma Bin).
Four researchers (Bing Zhao, Jinwei Yang, Mingyue Jiao, and Yanbiao Jiang) independently extracted and cross-checked the relevant data. The following information was extracted: Epidemiological characteristics, year and country of publication, publication journal, type and number of included studies, sample size of the included studies, disease involved, scope of intervention, and funding, etc. Methodology of search: search characteristics (number of databases, number of search strategies, whether a supplementary search had been conducted, whether the publication language of searched articles had been restricted). Reporting characteristics: whether the search terms/search methodology had been reported, whether complementary search methods had been reported, whether the search strategy of at least one database had been reported, how the search strategy had been presented, or whether a ow chart describing literature screening, etc. had been reported.

Data Analysis
Microsoft Excel® software (2019 version; http://o ce.microsoft.com/zh-cn/) was used for statistical analysis. Count data (for example, whether it is included in SCI, number of authors, number of retrieved databases, etc.) were described statistically using "number of cases (percentage)". Measurement data (such as the number of included studies, size of samples in the included studies, etc.) were described statistically by medians and interquartile range.

Search results
A total of 19,716 potentially relevant studies were initially selected. After excluding duplicates and those that did not ful ll the inclusion criteria, 813 studies were ultimately included in the present review. The selection process and results are displayed in Fig. 1.

Epidemiological characteristics of the included SRs/MAs
A total of 813 animal experiment SRs/MAs were identi ed in 503 journals, from 52 countries and regions, with a trend of increasing numbers year on year. However, compared with similar clinical systematic reviews and meta-analyses, the total number was nevertheless quite small [24] . The included studies covered almost all the diseases categorized by the WHO ICD-11 [20] . The distribution of diseases researched by animal SRs/MAs is essentially similar to those that are clinically research-based [24] , which to some extent re ects the fact that preclinical research tends to provide the evidence for clinical research to be conducted. Therefore, the publication of animal experiments of SRs/MAs should be continuously encouraged so that su cient evidence is provided to support the clinical transformation of animal research in the future.
Numerous investigations have con rmed that registration of studies or publication of protocols may reduce the risk of speci c research being unnecessarily repeated, and reducing or avoiding selection bias in systematic reviews, such as the inclusion of retrospective adjustments, selective reporting bias, such as using good or positive outcome indicators only, and therefore provides the basis for evaluating the results of other researchers' studies [25,26] . In addition, it is also helpful so that researchers can update their studies in the future. [27] However, the present review found that only 13.2% (107/813) of animal experiment SRs/MAs were pre-registered, considerably fewer than clinical trial-based SRs/MAs study protocols [28,29] . We believe that this may be associated with the lack of publication of animal experiment SRs/MAs until recently, the imperfect methodology used, the lack of understanding of the signi cance of registration, and the lack of relevant guidelines. As early as 1993, the Cochrane Collaboration required authors to pre-register clinical SRs/MAs [30] . A study published in 2004 con rmed that the publication of the Cochrane Handbook of Systematic Reviews signi cantly promoted such SR/MA registration [31,32] . However, it was not until 2018 that International prospective register of systematic reviews (PROSPERO) allowed the registration of animal-based SRs/MAs [33,34] , and by the end of 2020, 781 had been registered [33] , although this is still a relatively small number compared with clinical trial-based SRs/MAs registrations. Nevertheless, there is already a focus on the registration of animal experiment SRs/MAs.

Methodological characteristics of searching in the animal experiment SRs/MAs included in the present review
There are a number of limitations in the search methods used by published animal experiment SRs/MAs. Here, we discuss source database searches, the formulation of search strategies, implementation of the search procedure, and supplementary searches, respectively.

The range of searched databases is large
The present study demonstrated that the number of databases searched ranged from 1 to 16. For clinical trial-based SRs/MAs, the Cochrane Handbook of Systematic Reviews [30] requires that Medline, Embase, and the Cochrane Library should be searched. In addition, databases from the country or region of the investigator should also be searched, and speci c additional databases selected relevant to the study topic. Although no standards exist for database searches for animal experiment SRs/MAs, the present study demonstrated that 28.8% (234/813) of the reviews searched fewer than 3 databases, while fewer than 20% (18.9%, 154/813) searched more than 5.
The present study found that PubMed, Embase, and Medline are currently the three most commonly searched English-language databases for animal experiment SRs/MAs, although these databases do not cover all data resources causing some articles to have been omitted. A number of studies have shown that a search of PubMed/Medline or Embase databases will identify only 55% or 49% of relevant studies, respectively [35] . In addition, Biosis Previews is currently the largest life science and biomedicine database [36,37] , with a wide collection of related literature, such as in biology, medicine, pharmacy, etc. The Web of Science can be searched through known relevant source literature and references, and so it represents an important aid to database searching [38] . However, the present study found that only 29.8% (242/813) 813searched Web of Science and 4.6% (37/813) consulted Biosis Previews. Therefore, following a search of PubMed/Medline and Embase databases, it is also recommended that the search is supplemented using Biosis Previews and Web of Science databases so that fewer relevant articles are omitted caused by incomplete searching.

Problematic formulation of search strategies and implementation of search processes
Prior to the implementation of a search, research issues for animal experiment SRs/MAs should be summarized in accordance with PICOS (Participants, Intervention, Comparisons, Outcomes, and Study Design) principles using a methodology similar to that of clinical trial SRs/MAs [39] . Previous studies have demonstrated that a structured PICOS search strategy allows the de nition of a more speci c and detailed research problem and, additionally, improves the accuracy of a search [40] . During the search procedure, sensitivity (recall) and accuracy (positive predictive value) are often used to evaluate the search results [41] . A number of researchers have found that the sensitivity (recall) of the P and I in the PICOS-based element search in SR/MA abstracts of clinical trials were 37-84% and 26-80%, respectively [42][43] . In addition, it has been shown that the sensitivity (recall) of P, I, and O were 62%, 47%, and 75%, respectively [47] when using an entity recognition model tool to analyze 191 biomedical articles. The results above indicate that the most commonly used elements in PICO for searching for clinical trialbased SRs/MAs are P, I, and O. In the present study, we found that the PICOS elements utilized in the search terms of the 813 studies included here were essentially the same as that observed in clinical trialbased SRs/MAs searching [48] , although P and I were higher, while O was lower. This may be associated with the number of outcome indicators for animal experiments. Where a particular outcome is included in the search process, it may lead to the omission of relevant articles. High sensitivity for a target comprehensive search is considered standard practice for systematic reviews [49] although the ideal search strategy should be a balance of recall and precision [50] . As a result, for searches for animal experiments, we still recommend using the PICOS principle to formulate a search strategy, with P and I elements as the basic choice, and depending on the research purpose, adding C or O, prior to implementation of a search.
Irrelevant literature can to a certain extent be ltered out when addressing limitations of study design.
Therefore, appropriate development and use of search lters could improve search e ciency [51] and reduce workload. There has been a rapid growth in the trend for the use of search lters since 2012 [52] . Medline, Embase, PubMed, and Cinahl have developed their own subject search lters, which are different in their implementation. For example, search lters designed for Ovid Medline typically take advantage of the database's advanced search capabilities, such as eld search, subject search, and location operators, but the user must add them precisely, one line at a time, so commonly used search lters are saved to a personal account. However, search tools in individual accounts can only be used personally which prevents their widespread use in research teams [52] . In contrast, PubMed (including the free version of Medline) provides the means to add a long and complex search lter as a single detailed search string [1] that researchers can copy and paste into a database search box. In addition, its open accessibility allows search lters to be constructed behind hyperlinks and embedded into any HTML environment so that clicking a link or selecting a drop-down menu option can trigger a real-time, up-to-date PubMed search.
For animal experiments, only PubMed and Embase have developed such search lters for their respective databases [1,15] , indeed improving search e ciency to a certain extent. The two are not universal for the identi cation of articles when searching, and researchers are not limited to these two databases when searching for animal experiment SRs/MAs, which undoubtedly increases the burden on researchers to identify other relevant databases to search. Therefore, the development of general search lters suitable for all databases for animal experiments is recommended for future research.

Insu cient supplementary searches
When identifying articles for an SR/MA, searching only database resources may result in failing to detect some that are relevant [53,54] . Supplementary searches can reduce this occurrence [336,55] . In addition, scienti c evaluation of publication bias is of considerable importance for the interpretation of the credibility of the conclusions of animal experiment SRs/MAs [56][57] . Insu cient and incomplete searches undoubtedly increase the possibility of publication bias [58] . Furthermore, manual searches are considered the least e cient of all search methods [59] . The results of the present study indicate that only a half (52.2%, 424/813) of the included 813 animal experiment SRs/MAs had conducted a supplementary search. Although no standard for supplementary searches has been published for the eld of basic medicine, Chris Cooper et al. [60] summarized more effective methods of supplementary searches by referring to 4 existing systematic review methodology manuals, which included referring to references and journals, contacting authors, etc. This can also represent a reference for researchers to develop and implement supplementary searches for animal experiment SRs/MAs.
In conclusion, for search methodology for animal experiment SRs/MAs, we suggest that: (1) Databases should be selected that are pertinent and reasonably appropriate for the speci c research problem. In addition to the PubMed and Embase databases, it is necessary to also search the Web of Science and/or Biosis Previews. For studies involving speci c topics such as pharmacological toxicology, TOXNET and other databases should be reviewed [61] ; (2) Using PICOS principles, the P and I should be selected as the main elements for searching, while C or O should be added in accordance with the research purpose when selecting search terms; (3) Development of effective supplementary search strategies and methods that avoid missing relevant literature, while reducing the risk of publication bias; (4) Identify relevant information from experts in the eld or ask library personnel to participate in the entire search process to ensure the accuracy and scienti c nature of the search strategy; (5) Use appropriate animal experiment search lters to improve search e ciency; (6) Development of guidelines for the search procedure for animal experiment SRs/MAs, that will ultimately promote improvements in the quality of SRs/MAs.

Reporting of search strategies of the animal experiment SRs/MAs included in the present study
A complete and transparent search strategy helps readers understand the credibility of SRs/MAs by allowing analysis of the methodology and assessment of quality [62] , but journal editors, often because of page limitations, but also because of the poor understanding of the importance of reporting the full search strategy, allow only a partial list of search words or search methodology, preventing readers ascertaining the full details, which thus affects their ability to properly interpret the results. PRISMA [63] and MOOSE [62] are international reporting guidelines for SRs/MAs for clinical trials which explicitly requires that the complete search strategy of at least one database to be reported when conducting an SR/MA. Additionally, in terms of reporting the search strategy, the sources of search information, search strategies, search restrictions, and other literature must also be reported. In the results section, in accordance with the above guidelines, a ow chart describing the screening of the literature for an SR/MA must also be presented, indicating the total number of animal experiments included in the meta-analysis. Of course, the encouraging thing is that the proportion reporting database names, search terms, and publication dates of searched articles was found to be more than 98%. However, of the 813 SRs/MAs included in the present review, only 47.4% (385/813) reported the search methodology and 31.6% (243/813) described the search strategy, possibly due to limitations of the journals. As a result, the methodology for the majority of databases has not so far been fully presented. However, as there are currently no reporting guidelines speci cally for animal experiment SRs/MAs, the authors only refer to similar published research guidelines (such as PRISMA [61] ) or published formats for SRs/MAs or lists of content for reporting, resulting in incomplete data and large differences in the quality of reporting.
Furthermore, Medline and Embase have different platforms [64,65] , which increases the burden for researchers in terms of implementing speci c search strategies. In addition, the lack of familiarity with multiple search platforms can result in omitting relevant articles and conducting awed searches [66] . In the present review, only 8.1% (67/813) of studies reported the database search platform. The participation and guidance of trained search experts or library personnel will improve the comprehensiveness, accuracy, and reliability of relevant literature searches for SRs/MAs [10,67] . However, the present results demonstrated that only 4.8% (39/813) of animal experiment SRs/MAs reported the quali cations of the researchers. Therefore, we suggest that: (1)

Conclusions
The quality of search methodology and search process reporting of many published animal experiment SRs/MAs is incomplete and not standardized. Therefore, we suggest that: (1) Depending on the research problem, the most targeted and appropriate databases should be selected for searching, in addition to at least Medline, PubMed, and Embase, with supplementation of the searches with Web of Science and Biosis Previews databases, where possible; (2) Develop search lters for animal experimental research suitable for different databases to improve search e ciency, to avoid the omission of relevant literature and reduce the risk of publication bias; (3) Experts in the search of relevant information or library searchers should participate in the entire search process to ensure the integrity and universality of search strategies; (4) Continuing education and information consultation for evidence-based medicine related to medical-related animal experimental research will assist researchers master the skills of literature searching and allow personnel to become technical guarantors of creating high-quality evidence for clinical translation; (5) On the premise of standardization of the search process, the research and development of animal experiment SRs/MAs search and reporting guidelines, standardization of the search process and reporting, will ensure the transparency of research, and promote its introduction in relevant journal contributions, and ultimately promote improvements in the quality of animal experiment SRs/MAs.

Declarations
Ethics approval and consent to participate Not applicable Consent for publication Not applicable Availability of data and materials All data generated or analysed during this study are included in this published article [and its supplementary information les].

Competing interests
All authors declare that no competing interests exist.  Tables   Table 1, 2       Types of diseases included* (* classi cations in accordance with WHO ICD-11 [20]).