Search methods for identification of studies
A comprehensive search strategy with high sensitivity will be adopted to identify all potentially records relevant to the field of primary care and family medicine as previously described (10, 11). The search strategy includes the terms and synonyms for Bayesian factor analysis and primary care as shown in Table 1. The search strategy is developed with a specialized librarian and will be conducted by at least two reviewers independently. Searches of electronic databases with hand searches of reference lists will be combined. The computer-based searches will combine medical subject headings (MeSH) terms, free text and full text.
Table 1-- Medline search strategy through the PubMed interface
Concepts
|
PubMed
|
Bayesian
|
(bayes*[tw] OR "Bayes Theorem"[Mesh] OR Gibbs sampler OR "MCMC" OR "prior distribution" OR "posterior distribution") AND
|
Factor Analysis
|
("Factor Analysis, Statistical"[Mesh] OR factor analys*[tw] OR item domain correlation* OR item response theor* OR structure equation model* OR latent variable* OR factor loading*) AND
|
Primary care
|
("family"[all fields] OR physician*[all fields] OR practice*[tw] OR "primary care"[all fields] OR "Primary Health Care"[mh] OR primary[tw] OR general pract*[tiab] OR gp[tiab] OR gps[tiab] OR clinic*[tiab] OR refer*[tiab] OR visit*[tiab] OR outpatient*[tiab] OR consult*[tiab] OR communit*[tiab] OR ambulatory[tiab] OR centre*[tiab] OR office[tiab])
|
Databases and time frame
Articles from Medline, Embase, Cochrane Library, CINAHL, and Scopus will be identified. All relevant articles published before January 1st, 2020 will be considered.
Searching other resources
Google Scholar will be manually scanned for the first 200 to 300 records for supplementary information (12). Reference lists and the future citation of the retrieved articles will be manually searched with two additional rounds. As currently no well established guidelines exist on conducting methodological reviews, we will follow general recommendations in the literature (13), guidelines under development (14) as well as the PRISMA-P statement for the methodological review (15, 16). Review of reviews articles will serve for identifying literature covered within the reviews.
Types of studies
Quantitative and empirical research studies, methodological studies using Bayesian factor analysis, review articles, conference abstracts and thesis or dissertation documents will be included. Research studies using similar model structures such as structure equation model and latent variable model, as well as item response theory, factor loading and item domain correlation will be included. Some conferences publish full text papers, i.e. conference proceedings alongside with abstracts. Only conference abstracts with respective full text access will be considered in the methodological review.
Inclusion Criteria
The inclusion criteria of the review requires literature to match the following three themes: “In the context of primary care practice or”, “Bayesian methods” and “factor analysis”. The definition of primary care follows that of the American Academy of Family Physicians as being “comprehensive”, “first contact” and “continuing” meanwhile it covers “any undiagnosed sign, symptom, or health concern” (17). The term “Bayesian methods” refers to any inferential method that employs prior distributions in conjunction with observed information to arrive at an estimate for the parameter of interest.
Exclusion Criteria
Editorials, commentaries, book reviews, hypotheses, critical appraisals, reflections, surveys, case reports or studies or studies that do not employ Bayesian methods will be excluded. Studies that include some of the key words but use them under different connotations or references will be excluded. Examples of ineligible use of key words include “primary studies”, “prior to”, “human epidermal growth factor” and “genetic factor”.
Bayesian methods used in other types of analyses, such as Bayes rule, Bayes or Bayesian factor studies, variational Bayes, Bayesian Information Criterion/Criteria, Bayesian random effects models, Bayesian/Bayes network, belief network and Bayes(ian) model or probabilistic directed acyclic graphical model will be excluded. Studies not in family medicine or primary care but using related terminology will be excluded. Examples are: “a family of methods” and “exponential family”.
Data collection and analysis
Selection of studies
Titles and abstracts of studies will be sequentially screened using the search strategy by at least two independent reviewers using the software Rayyan (18). If no information is given in the title or abstract about any of the three inclusion criteria, i.e. no indication about whether the study is applying Bayesian methods, using factor analysis or in primary care, those studies will be included at the initial stage of screening. In indecisive situations, for example when the term “factor analysis” is mentioned but not specified whether it is Bayesian or not Bayesian, the article will be kept for the next round of full-text review. The full text of articles that meet the inclusion criteria will be retrieved and examined independently by seven reviewers, each reviewing one out of the seven portions of the identified articles that are randomly assigned to each reviewer. All articles will be also reviewed by the main author. Any disagreement between the reviewers and the main author about the eligibility of specific studies will be discussed and additional reviewer will be involved if necessary, until consensus is reached. For studies with multiple publication records, the most comprehensive or up-to-date record will be used.
Data extraction and management
Data extraction and data preparation will be facilitated using Microsoft Excel and the statistical software package R. All records will be coded and categorized under the predefined themes in the codebook from the Canadian Institute of Health Research (CIHR) grants and rewards guide (19). Despite existent guidelines and recommendations on reporting of general Bayesian methods, confirmatory factor analysis and questionnaire development, no single comprehensive recommendation was found on the reporting of Bayesian confirmatory factor analysis (20-22). Where applicable, the following data will be extracted: type of journal, publication date, geographical location, sample size, number of items or questions used for the Bayesian factor analysis, number of factors, domains or constructs, reported item-domain correlations, regression parameters, factor loadings, or parameters of structural equation models, use of prior information and assumed prior distributions, and the primary care settings. A standardized predesigned data collection form will be used for data extraction. The assessment criteria below will be followed:
- Did the authors use either Bayesian confirmatory factor analysis or Bayesian exploratory factor analysis or Bayesian latent variable model or Bayesian structure equation modeling?
- If they used (at least) one of the listed methods, what was the parameter of interest they were aiming to estimate: item-to-domain correlation, factor loading or latent model regression parameter? In other words, for which parameter did they impose a prior distribution?
- How did investigators inform their prior distribution of the respective parameter? What was the prevalence of studies that employed non-informative priors?
- If they mention the term “factor loading”, did they explain it and if, how did they interpret it i.e. as item-to-domain correlation or as model parameter (latent variable coefficient)?
- Did they report standardized factor loadings or parameter estimates that exceeded an interval of [-1, 1]?
- Were credible intervals or confidence intervals reported for factor loading, item-to-domain correlation, model parameter, or regression coefficients?
- What software or libraries were used? Were software codes or original data made available? (reproducibility)
The data extraction form used to summarize information obtained from the identified articles will be pilot tested to identify possible sources of error or imprecision. For this purpose, all reviewers involved will extract data from a selected set of articles using the data extraction form. The extracted data will then be compared and sources for potential mismatches or errors discussed and resolved.
Assessment of quality of implementation and reporting of Bayesian methods
Risk of bias in individual studies is not applicable to and will not be assessed in the review since the goal is to summarize the use and reporting of Bayesian questionnaire validation methods, i.e. there is no single effect parameter that is of primary interest. The data collected across studies will indicate the presence or absence of each of the seven criteria for assessing appropriateness of design, conduct and reporting. The quality of implementation and reporting of Bayesian methods for each eligible study will be assessed and rated on an ordinal scale with levels: very low, low, moderate and high on the following aspects: reporting about methodology, Bayesian model, estimated parameters, prior elicitation, and basic contextual information provided. The quality assessment will be conducted independently by two expert statisticians (H.Z. & T.S.) and presented in tables in the final publication of the methodological review. No available critical tools exist to appraise the use of Bayesian factor analysis, however, the proposed quality appraisal (i.e. a methodological ‘peer review’) by the authors will help to identify prevalent issues and initiate discussions of better reporting standards.
Strategy for data descriptions and synthesis
A descriptive-analytical synthesis of the findings from the included studies with graphs and tables will be provided detailing the use of Bayesian factor analysis based on a common analytical framework on authors, years of publication, estimates, the number of publications over time, geographical locations, the study populations, the aims of the study, data types, key information about the data (e.g. sample sizes, number of questions in a questionnaire, or number of domains or factors), the type of Bayesian method used, and different estimation procedures and software routines (e.g. analytical solutions vs. sampling-based solutions).
Statistical Synthesis
Descriptive analyses will be conducted to summarize frequencies of Bayesian factor analysis approaches being used as well as general and specific reporting issues.
Anticipated Results
Description of studies
The current use of Bayesian factor analysis will be summarized through descriptive statistics, for example, frequency distributions displaying the prevalence of the seven pre-defined assessment criteria across studies. A subjective quality appraisal developed in this review will be useful in initiating discussions of better reporting standards based on the review.