Scoping reviews have been used widely ‘to identify knowledge gaps, scope a body of literature, clarify concepts or to investigate research conduct’ (26). They are useful ‘when a body of literature has not yet been comprehensively reviewed, or exhibits a complex or heterogeneous nature not amenable to a more precise systematic review of the evidence’ (27, p.141). Scoping reviews can also document research that informs and addresses practice (28). A scoping review does not include aggregation and synthesis of data nor does a scoping review include a quality assessment of the documents included (26).
Thus a scoping review suits our review question of mapping how routine DHIS2 data use been documented. The objectives were to review the literature (peer reviewed and grey) regarding DHIS2 data use, and to categorise key examples of use of DHIS2 data. This scoping review included a review of the peer reviewed literature, key journals and conferences, and thesis produced within the HISP programme. The primary research question was ‘How is DHIS2 data being used routinely for action and decision making within the health system?’ Sub-questions explored to address the primary research question included:
-
In what areas is it reported that DHIS2 data is being routinely used?
-
What examples are reported of DHIS2 data being routinely used for action and decision making?
The following databases were searched for peer reviewed literature: Pubmed, EMBASE and Web of Science, as these are deemed the most relevant for literature related to the topic (see search strategies in Appendix 1). Email alerts containing new articles published since the search was conducted in December 2020 were added to the original search. The time frame for the search was from when the first article was published in a given database to August 2021.
Due to language limitations, we included only English language articles. Hand searching of reference lists of studies deemed to be highly relevant to the review question were checked to identify other relevant studies. Grey literature included the International Federation for Information Processing: Working Group 9.4 (IFIP 9.4) conferences and the Post Graduate (MSc and PhD) thesis from the Health Information Systems Programme (HISP) in the Dept of Informatics at the University of Oslo. A review of evaluations and assessments of DHIS2 internal to HISP were conducted as part of a separate study by the first author (EB) but didn’t reveal many additional detailed examples of data use that were not included in the other publically available documents. These internal reports were not included as part of the scoping review and this meant that as all the material reviewed is publically available and thus no ethical clearance was needed to conduct the review.
The two authors (EB & JS) analysed the abstracts and the full articles for review according to the inclusion or exclusion categories separately. Where there was any conflict the authors met and resolved the conflicts. There were other existing groups within the department that were available if the conflict could not be resolved between the two authors, but most of the conflicts was around ambiguity on the level of detail required for inclusion rather than whether the article met the inclusion or exclusion criteria. In these cases, the article was included in the full text review.
As noted the focus for this review was solely on DHIS2 (and previous versions of DHIS). Inclusion criteria were therefore that research and conference articles were peer reviewed and described how the data from DHIS2 was being used for action or decision making OR that the Grey literature described how the data from DHIS2 was being used for action or decision making. Exclusion criteria were:
-
Articles which focus on use of data for action or decision making not from DHIS2;
-
Articles that evaluate or assess the needs of the health system in relation to DHIS2 or using DHIS2 data;
-
Articles that describe/evaluate quality of data;
-
DHIS2 data used with other data sources with purpose of validating or highlighting deficiencies of the datasets;
-
Articles that describe theoretical or conceptual frameworks only that could improve DHIS2 data use;
-
Articles that describe the analysis and products of data only with no description in relation to how this analysis or product is used
-
Articles that mention data use but provide no examples of how it is used
-
Non-English language studies.
The JBI Guidelines approach of Peters et al., (27) was followed and included the following steps: defining and aligning the objective/s and question/s; developing and aligning the inclusion criteria with the objective/s and question/s; describing the planned approach to evidence searching, selection, extraction, and charting; searching, selecting, charting and summarising the evidence. The protocol was initially shared with the Heritage Project: Designing for Data Use research group at the University of Oslo for input.
The five stage approach of Arksey and O’Malley (28) and progressed by Levac et al., (29) was followed: i) identifying the research question; ii) identifying relevant studies; iii) study selection; iv) charting the data; v) collating, summarising and reporting results.
Duplicates were removed electronically and checked by hand by EB. Both authors independently screened titles and abstracts using covidence software for inclusion/exclusion. Disagreement between coders was resolved between the team members, though as mentioned internal research groups were available to consult with but this was not needed. For full article review both authors agreed on inclusion and exclusion independently and resolved any conflicts - again there was no need to bring in other groups as conflicts were easily resolved.
An extraction template was agreed upon and EB and JS extracted the full articles and grey material using this template. The data extraction template contained: author(s); year of publication; study title; journal/document source; study location; level of health system and health programme; study rationale; and description of use of data. Covidence software was used to assist with the process. Data was charted and exported from Covidence into excel. Standard descriptive information of included texts such as study site, year of publication, type of publication and health level and programme was conducted using this excel spreadsheet. Study rationale and description of data from the charted data was subsequently categorised in relation to the focus of the study in terms of data use purpose, content or process.
The findings from the scoping review were presented to Paper Development Seminar Series at the Department of Informatics, University of Oslo and subsequently shared with other research groups and key individuals external to University of Oslo for comment. This sharing of early drafts was to validate the data that was included and to provide an opportunity for suggestions on any other articles or documents, especially grey literature, that we may have missed. The sharing was also a means to further the discussion on what could be done to document use of DHIS2 data and the different conceptualisations around ‘data use’.
The two authors are currently part of HISP. EB did her PhD related to an earlier version of DHIS and worked with colleagues from HISP for approx. 15 years ago in South Africa. EB joined the HISP global team as a guest researcher for 12 months at the end of 2020. JS has been working with HISP at country level implementation and as a researcher with the UiO for nearly two decades. The implications are that we can be deemed ‘insiders’, but as noted in Byrne et al (30) there are advantages and disadvantages to the ‘insider’ versus ‘outsider’ debate. Additionally, following the systematic approach of a scoping review and sharing findings with key stakeholders has lent rigour to this review that hopefully will give a clear interpretation of the data found.