We identified 158 HTA organisations. Webpage searches resulted in 156 potentially relevant publications from 63 of these organisations. After deduplication, a total of 151 publications remained and were assessed for eligibility. Seven documents from eight organisations were included in our synthesis. Two organisations (Gesundheit Österreich GmbH (GÖG)/Ludwig Boltzmann Institut (LBI)) share the same methods document and are therefore considered together in the following. The selection process is illustrated in Fig. 1. We provide a list of excluded documents as supplement [see Additional file I].
We included methods documents from the following organisations: Agency for Care Effectiveness (ACE)(16), European Network for Health Technology Assessment (EUnetHTA)(17), GÖG/ LBI(18), Health Information and Quality Authority (HIQA)(19), Health Quality Ontario (HQA)(20), National Institute for Health and Care Excellence (NICE)(21), Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU)(22).
Review Purpose
SR-HEs can have several purposes. Four organisations (16, 20–22) use them to determine the need to conduct a P-HE. They consider a SR-HE sufficient if one or more P-HEs without major limitations and sufficient transferability are identified. If deemed necessary, the P-HEs are updated and/or adapted to the decision context. In contrast, GÖG/LBI questions whether a SR-HE can be used to answer questions on cost-effectiveness, because transferability is often limited. That said, GÖG/LBI(18) acknowledges that SR-HEs are useful to explore how and why interventions may be more or less effective, resource or cost intensive and to collect important information for performing a P-HE. EUnetHTA(17) suggests using SR-HEs if a P-HE cannot be conducted (eg, due to limited resources). In this case SR-HEs can be used for explanatory purposes or to identify the most relevant P-HE for informing a particular decision.
Assessment Approach Concepts
We developed definitions of all assessment approach characteristics and provide these in Table 1. An overview of the recommended assessment approach concepts can be found in Table 3. Further details are available as supplement [see Additional file 2].
Table 3
Assessment approach concepts
Organisations
|
Consideration of transferability for
|
Consideration of transferability in the preparation process
|
Structuring of assessment approach
|
Instructions or item descriptions provided
|
Combination of different assessments
|
Consideration of sensitivity analyses
|
Effectiveness data
|
Cost data
NR
|
In study selection
|
In study assessment
|
non-structured
|
structured
|
yes
|
no
|
Stand-alone
|
combined
|
yes
|
NR
|
ACE(16)
|
NR*
|
y
|
x
|
x
|
|
x
|
|
x
|
x
|
|
x
|
|
EUnetHTA
(17)
|
NR
|
x
|
|
x
|
x
|
|
|
x
|
x
|
|
|
x
|
GÖG/LBI(18)
|
x
|
x
|
x
|
x
|
|
x
|
x
|
|
x
|
|
x
|
|
HIQA(19)
|
x
|
x
|
x
|
x
|
|
x
|
x
|
|
x
|
|
x
|
|
HQA(20) (Applicability)
|
NR
|
x
|
x
|
x
|
|
x
|
|
x
|
x
|
|
|
x
|
HQA(20) (Generalis-ability)
|
NR
|
x
|
|
x
|
x
|
|
|
x
|
x
|
|
|
x
|
NICE(21)
|
x
|
x
|
x
|
x
|
|
x
|
x
|
|
x
|
|
x
|
|
SBU(22)
|
|
x
|
|
x
|
|
x
|
|
x
|
|
x
|
|
x
|
• NR: not reported |
Aspects of transferability can be considered at different steps during the preparation of a SR-HE: At the selection-level, when defining and applying eligibility criteria and at the assessment-level, when assessing the studies quality (and transferability). All included organisations(16–22) recommend a transferability assessment at the assessment level. Five organisations(16, 18–21) additionally recommend to consider aspects of transferability at the selection level. Moreover, NICE(21) proposes to limit the study setting to the UK or countries with similar healthcare systems, if necessary.
The assessment approaches vary regarding the target data for assessing transferability, the assessment structure, the provision of instructions or item descriptions for assessors, the combination of different assessments (transferability and methodological quality) and the consideration of additional information (e.g. derived from sensitivity analyses).
Regarding the target data for assessing transferability, three organisations(18, 19, 21) consider the transferability of cost and effectiveness data. One organisation(22) considers the transferability of cost data. For three organisations(16, 17, 20) it is unclear, whether their recommendations consider only cost data, or both.
Regarding the assessment structure, six(16, 18–22) concepts can be defined as structured approaches, while two concepts are non-structured (17, 20).
Regarding the provision of instructions/item descriptions for assessors, four organisations(16, 17, 20, 22) do not provide any instructions/item descriptions, while three organisations(18, 19, 21) provide some. These mainly include explanations and examples of assessment criteria, explanations and examples why something might have an impact on transferability and/or guidance on what should be considered in the assessment.
Regarding the combination of different assessments, six organisations(16–21) recommend assessing transferability independently from other aspects (standalone assessment), e.g. the methodological quality while one organisation(22) recommends a combined assessment. Specifically, SBU(22) recommends combining the assessment of transferability and methodological quality, including a joint overall judgement. Although NICE(21), HQA(20) and HIQA(19) recommend a standalone assessment, they link the assessments of transferability and methodological quality: NICE(21) and HQA(20) recommend to assess the methodological quality only if the assessed study is sufficiently transferable, while HIQA(19) recommends to assess transferability only for studies with acceptable quality.
Four(16, 18, 19, 21) organisations recommend to consider information derived from sensitivity analyses in the transferability context. For example, according to ACE, “any key drivers of the economic model and areas of uncertainty identified by the sensitivity analysis should be included in the evaluation report.”(16)
Assessment criteria
An overview of all recommended assessment criteria is provided in Table 4. More details can be found as supplement [see Additional file III].
Table 4
Domain
|
Items
|
ACE(16)
|
EUnetHTA(17)
|
GÖG/LBI(18)
|
HIQA(19)
|
HQA(20) (Applic-ability)
|
HQA(20) (Generalis-ability)
|
NICE(21)
|
SBU(22)
|
Population
|
Population (in general)
|
x
|
|
|
|
x
|
|
x
|
|
Demographics
|
|
x
|
x
|
x
|
|
x
|
|
|
Risk factors
|
|
x
|
|
|
|
|
|
|
Life expectancy
|
|
x
|
x
|
|
|
|
|
|
Compliance
|
|
x
|
x
|
|
|
|
|
|
Ethnicity
|
|
|
|
x
|
|
|
|
|
Epidemiology
|
|
x
|
x
|
x
|
|
x
|
|
|
Stage/severity of disease
|
|
x
|
|
|
|
|
|
|
Case mix
|
|
x
|
x
|
|
|
|
|
|
Variation in health state values
|
|
x
|
x
|
x
|
|
x
|
|
|
Intervention
|
Intervention (in general)
|
x
|
|
|
|
x
|
|
x
|
|
Extent and type of care
|
|
|
|
|
|
|
|
x
|
Comparator
|
Comparator (in general)
|
x
|
|
|
|
|
|
|
|
Outcome
|
Health state preferences (e.g. in terms of QALYs)
|
|
|
|
x
|
x
|
|
x
|
|
Appropriateness of measures
|
|
|
|
|
x
|
|
x
|
|
Source of estimates of treatment effects
|
|
|
|
|
x
|
|
|
|
Health system
|
Health system (in general)
|
x
|
x
|
|
|
x
|
|
x
|
|
Availability of health technologies
|
|
|
x
|
|
|
|
|
|
Available treatment options
|
|
x
|
|
|
|
|
|
|
Incentives for healthcare professionals or institutions
|
|
|
|
x
|
|
x
|
|
|
Resource availability
|
x
|
|
|
x
|
|
x
|
|
|
Clinical practice
|
Clinical practice (in general)
|
x
|
|
x
|
x
|
|
x
|
|
|
Clinical guidelines
|
|
|
|
x
|
|
|
|
|
Care pathways
|
|
|
|
x
|
|
|
|
|
Treatment practice
|
|
|
|
x
|
|
|
|
|
Range of treatment
|
|
|
|
x
|
|
|
|
|
Provider characteristics
|
|
x
|
|
|
|
|
|
|
Provision of health services by different professional groups
|
|
|
x
|
|
|
|
|
x
|
Organisation of prescribing
|
|
|
x
|
|
|
|
|
|
Costs
|
Completeness of cost data
|
x
|
|
|
|
|
|
|
|
Unit prices/costs
|
|
x
|
|
|
|
|
|
x
|
Absolut and relative prices/costs
|
|
|
x
|
|
|
|
|
|
Medical costing approach
|
|
|
x
|
|
|
|
|
|
Relative prices/costs
|
|
|
|
x
|
|
x
|
|
|
Value of various cost elements
|
|
|
|
x
|
|
|
|
|
Methodo-logical aspects
|
Perspective
|
x
|
x
|
x
|
x
|
x
|
|
x
|
x
|
Discount rate
|
|
x
|
x
|
x
|
x
|
|
x
|
|
Modelling approach
|
|
|
|
x
|
|
|
|
|
Assuming effects to be equal
|
|
|
|
|
|
|
|
x
|
Extrapolation
|
|
|
|
x
|
|
|
|
|
Other
|
Clinical effectiveness data
|
|
|
|
x
|
|
|
|
|
PICO (Population, Intervention, Comparison, Outcome)
With exception of SBU(22), all organisations consider population characteristics in their recommended assessments(16–21). Four organisations(17–20) take demographic characteristics, epidemiology (e.g., incidence/prevalence) and values of health sate preferences into account. EUnetHTA(17) and GÖG/LBI(18) additionally consider life expectancy, compliance and comorbidities. Other items are risk factors(17), severity of disease(17), ethnicity(19) and case mix(18).
Four organisations consider intervention characteristics. Three of them(16, 20, 21) pose the question, of whether the intervention is appropriate for the research question of the SR-HE. SBU(22) recommends to compare the care and type of intervention with the current intervention in the target context. ACE(16) additionally includes a question regarding the similarity between the study’s comparator and the comparator proposed in the research question.
Three organisations consider outcome characteristics(19–21). All of them take health state preferences in terms of QALYs into consideration. Furthermore, HQA(20) and NICE(21) consider the appropriateness of measures. HQA(20) also checks whether estimates of treatment effects are from the best available source.
Health System
Health system characteristics are considered by six(16–21) organisations. Three of them pose the question of whether the health system, in which the study was conducted, can be compared to that of the context of interest(16, 20, 21). Others recommend to examine health system characteristics like available treatment options and unit prices(17), availability of the health technology of interest(18) or incentives to healthcare professionals and institutions(19, 20).
Clinical practice
Six (16–20, 22) consider variation in clinical practice. Some of them specify aspects including provider characteristics(17), treatment practice(19), clinical guidelines(19), care pathways(19), range of treatments(19), organisation of prescribing(18) and provision of health services by different professional groups(18, 22).
Costs
Six(16–20, 22) out of seven organisations recommend the consideration of cost data. The following items are suggested: completeness of cost data(16), unit prices/costs(17, 22), absolute and relative prices/costs(18), the medical costing approach(18), relative prices/costs(19, 20) or the value of various costing elements (e.g. charges or fees)(19).
Methodological aspects
All organisations consider methodological aspects of the studies (16–22), in particular a study’s perspective which determines the costs and consequences that were considered in the analysis(16–22). Five organisations (17–21) also consider the appropriateness of discounting. Moreover, HIQA(19) considers the appropriateness of the model used to extrapolate data to the context of interest, while SBU(22) questions whether costs and effects were studied or whether effects were assumed to be equal.