Inclusion criteria
The included population in the study was Swedish public sector authorities at the municipal and regional levels or their owned subsidiaries, as well as their representative member association (SALAR) or its owned subsidiaries. Requests for tenders that were included had to involve HWT that was in accordance with the definition stated in the Background section (1). The included requests for tenders were those that proceeded from announcement to procurement decision (regardless of appeal) at any time prior to May 2021 and had documentation in Swedish or English languages.
Exclusion criteria
Pharmaceutical- or medical technologies used for in-patient or hospital-care settings (e.g., in-hospital digital monitoring equipment) and technologies that did not have the health or welfare of individual or organisational users or providers as a primary function (e.g., smartphones, video-conferencing systems) were excluded. Administrative systems such as electronic health records and databases/registries, interoperability applications and infrastructure, identification systems and similar were also excluded. Pre-announcements, dialogues, and requests for information were excluded, as were requests for tenders that were interrupted or cancelled by the procuring organisation prior to completion. Requests for tenders where documentation was not available were also excluded, as were duplicates found in the same or separate databases.
Search strategy
The Mercell Opic procurement announcement database (www.opic.com (17)) was the primary database searched between May and June 2021. Complementary searches were conducted during the same period in the procurement announcement databases www.e-Avrop.com, www.KommersAnnons.se, and www.offentligaupphandlingar.se to check if additional HWT tenders could be found.
Search strings
Searches were conducted in Swedish and English. The following search words and combinations were used to identify HWT-related requests for tenders:
- English: welfare tech*; digital health tech*; digital health*; e-health*, safety alarm*; GPS-alarm*; security camera*; night camera*; nocturnal camera*; digital nocturnal surveillance; self-monitor*; remote monitor*; door alarm*; floor alarm*; absence alarm*; digital care; remote meeting*; digital lock*; robot*; incontinence sensor*; medication reminder*; medication alarm*; medication dispenser*; digital game*; digital rehab*; digital training*; smart*; health app*; digital activation*; epilepsy alarm*; seizure alarm*; digital learning*; digital education*; digital competency*;
- Swedish: välfärdsteknik*; digital hälsoteknik*; digital hälsa; e-hälsa; trygghetslarm; GPS-larm; tillsynskamera; nattkamera; digital nattillsyn; egenmonitorering; fjärrmonitorering; larmmattor; dörrlarm; rörelselarm; avvikelselarm*; digital vård*; distansmöte*; digital* lås; robot*; inkontinens sensor*; medicinpåminnare*; läkemedelspåminnare*; läkemedel dispens*medicinlarm*; läkemedelslarm*; läkemedelsautomat*; läkemedelsdispenser*; digitalt spel*; digital rehab*; digital träning*; smarta*; hälsoapp*; digital aktivering*; epilepsilarm*; digital *lärning; digital *läromedel; digital kompetensutveckling
The search words were in some cases adjusted (through addition or removal of hyphenation and spaces between words) such that several overlapping search terms were included.
Request for tender selection process
Two researchers (MXR and SLS), hereafter referred to as reviewers, conducted the review process which had five steps:
1. Announcement screening
The request for tender announcement titles, summaries and in some cases attached documents of the obtained records were screened for relevance by one reviewer (MXR) with random checks by the other reviewer (SLS). If the reviewer(s) voted that the record fulfilled the inclusion criteria, then the record was saved in the procurement database and sorted by the search term used to identify it. Any conflicts were resolved through dialogue until consensus was achieved.
2. Request for tender documentation screening
For requests proceeding to this step, all tendering documents related the request for tender announcement were downloaded and screened for confirmation of inclusion by one reviewer (MXR) with random checks by the other reviewer (SLS). If the reviewer(s) voted that the request for tender still fulfilled the inclusion criteria, then the documentation was compiled and proceeded to data extraction. Any conflicts were resolved through dialogue until consensus was achieved.
A summary of the results of this selection process can be found in Figure 1.
3. Content screening for evidence-related terminology
The textual content of all documents for included requests for tenders, including eventual awarding decisions and documentation, were then compiled in a text analysis tool (www.voyant-tools.org; (18)) and searched for evidence-related terminology. The terminology searches were conducted in the same language as the included tendering documentation. The following search words and combinations were used:
English: evidence; science; scientific; sensitivity; specificity; study; studies; intervention*; benefit*; effectiveness; effect*; certification*; MDR; CE conform*; MDD; 2017/745
Swedish: Evidens*; vetenskap*; känslighet*; specificitet*; studie*; utfall*; intervention*; *nytta; nytto*; effekt*; certifier*; bevis*, MDR; CE-märk*; MDD; 2017/745
A context for all found terms, consisting of the 15 words to the left and right of identified search term, were copied and exported to a spreadsheet.
4. Content confirmation for evidence-related criteria
The contexts from the content screening for evidence-related terminology were then reviewed in the tendering documentation from which they were extracted by one reviewer (MXR) with random checks by the other reviewer (SLS). The context was included as an evidence-related criterion if it met the following inclusion criteria:
- Clear evaluation- or awarding criteria (in Swedish respectively known as utvärderingskriterier or tilldelningskriterier, often in the form of “shall” or “should” requirements) were used in the tendering documentation regarding evidence for effects or value, consisting of collected or objective data, a certification that is only achieved upon provision of such data, or similar and
- the requirement was specific to the HWT intervention in the actual request for tender.
The context was excluded as a requirement if any of the following exclusion criteria were met:
- the requirement was solely related to the bidder’s overarching organisation (e.g. that the organisation is ISO-certified or uses a quality management system); or
- the requirement is solely related to administration of the intervention (e.g. availability, delivery, user education); or
- the requirement did not address a defined outcome (e.g. “improve effectiveness” without stating what that effectiveness involves or means for the organisation); or
- there was no request for documentation, certification or data to demonstrate fulfilment of the requirement (e.g. a simple “Yes/No” response sufficed as fulfilling the requirement).
The criteria confirmation was conducted by one reviewer (MXR), with random checks by the other reviewer (SLS). Any conflicts were resolved through dialogue until consensus was achieved.
5. Data extraction
Essential information regarding the type of HWT, procuring organisation, duration of the tender, number of bidders and related information was extracted for all included requests for tenders. The use of evidence criteria was also denoted with a description of what manner it was used (e.g., as a qualifier for bidding, for ranking of bids, etc.). If the tender was successfully awarded, then the economic value of the tender and the attributes of the winning bidder were also extracted; if the tender was appealed then this was also noted, along with information regarding the outcome of the appeal, if available.
Outcomes of interest
The main outcomes of interest for the analysis were:
- the prevalence and type of terminology in the tendering documentation related to evidence for effects and/or value
- the prevalence and type of evidence-related criteria when evaluating bids
- the attributes of the procuring organisation, bidders, type of HWT and procurement result in relation to the above outcomes