The complexity of scaling up an mHealth intervention. SMS for Life: A Case Study from a Health Systems Integration Perspective

Background The objective of this study was to apply systems thinking to examine the opportunities and challenges of scaling up and integrating a new electronic health technology into a complex health system. We used the implementation cycle of the supply chain management solution (SMS for Life 1.0) in Tanzania as a case study. SMS for Life was one of the earliest large-scale implementations of mHealth innovations worldwide. It aimed to bring visibility to antimalarial stock-outs through the use of SMS technology. Methods A qualitative case-study approach was used. This included a literature review, a document review of 61 project documents, a timeline of key events and the collection and analysis of 28 interviews with key stakeholders involved in or affected by the SMS for Life programme. Using the WHO mHealth Assessment and Planning for Scale (MAPS) Toolkit as a framework, we carried out a joint thematic analysis and identied the key reasons for the discontinuation of the programme. Results for Life was reliable at scale and raised awareness of stock-outs with real-time monitoring. However, it was discontinued in 2015 after four years of a national rollout. The main reasons identied for the discontinuation were programme’s changes health system, focus


Background
The accelerating role of electronic information and communication technologies (ICT) in low-and middleincome countries (LMICs) is seen as one of the major opportunities for the health sector (1). Increased operational e ciencies, as well as low-cost deliveries, have led to many investments in the eld (2,3). There has been rapid evolution from mobile health (mHealth), which focussed mainly on the use of mobile phones, to electronic health (eHealth), or digital health, which also include further technologies (4,5).
Although the interest of governments and donors has increased and more efforts are being invested in new technologies for the health sector, few programmes have managed to be successfully upscaled in LMICs (2). It remains the case that digital health interventions are usually undertaken in stand-alone, vertical projects and target relatively small populations (2, 6) rather than integrated as part of the health system. As a result, the focus of digital health initiatives has shifted towards scalability, integration, and sustainability of programmes (6, 7). This move has been driven by the desire of governments, partner agencies, and the private sector to invest in initiatives that provide a measurable long-term impact on the delivery of health programmes(8) and the strength of health systems.
The focus of this study is SMS for Life 1.0, a solution developed as part of a public-private partnership to address the issue of stock-outs in sub-Saharan Africa. SMS for Life was designed to make stock-outs in remote facilities more visible and reduce these with the use of SMS technology (9). One health worker in each facility was trained to send weekly SMS with the counts of stock for four ACT and quinine.
Reminders were sent automatically to the health worker's phone. If the health worker responded within 27 hours, they received credit on their phone (TZs 1,000, equivalent to USD 0.70) for personal use (pay for performance) (10). These SMS were free of charge thanks to an agreement with the mobile service  SMS for Life has been described in the literature before (9,10). Barrington et al(9) presented the successful pilot results and Mikkelsen-Lopez et al (10) showed that after two years of scale-up, health facilities continued to experience severe antimalarial stock-outs. Both papers studied the programme using a quantitative approach and did not explore the reasons behind the results. With this qualitative case study, we aimed to bridge this gap and understand the innovations that SMS for Life brought to the Tanzanian health sector and also the challenges that led to its discontinuation. The lessons learned in this study should be instructive for future digital solutions aiming to go to scale and integrate into similar settings.

Methods
This was a qualitative case study of a nationwide mHealth implementation. Case study methodology allows us to look in detail to all aspects of the programme and extract the lessons learned from it (11). To conduct this case study, we used the WHO health system building blocks framework (12) to guide data collection from a health-systems perspective and the WHO mHealth Assessment and Planning for Scale (MAPS) toolkit as a framework to analyse and interpret this data (13).

Study setting
Tanzania's public health supply chain system serves the MOHCDGEC-operated facilities, including 20 regional vaccine stores, 137 district stores, and 6.821 service delivery points (hospitals, health centres, and dispensaries). The value of medicines and other commodities moving through the public health supply chain each year is roughly 300 billion Tanzanian shillings (TSh) (14).
The MOHCDGEC has overall responsibility for ensuring the system remains functional. System logistics are run by the MSD, a semi-autonomous organisation in MOHCDGEC (14,15). At the district level, the forecast and ordering of commodities is done by district pharmacists, whose supervision lies under the President's O ce Regional Administration and Local Government (PORALG), as part of Tanzania's decentralized system(16).

Data collection
This case study was built on the triangulation of three data collection methods: document review, participatory observation and key informant interviews. A total of 61 documents, 3 participatory observations, and 24 interviews with 28 interviewees were included in the thematic content analysis.

Document review
We reviewed published literature, unpublished reports, evaluations, memoranda of agreement, and other available project documents to gather information on inputs, outputs, and outcomes of the programme. A data extraction tool was developed to gather key information from these.

Key stakeholder interviews
Two main groups of participating stakeholders were identi ed, those involved in high-level governance and design of the programme and those at frontline level (district o cials and health workers). Two districts were purposively selected to conduct interviews, a high and a low-performing district. Purposive and snowball sampling were used to select the interviewees. Interview guides were developed to conduct semi-structured interviews with each group. These guides were structured following the WHO health system building blocks (12). Probes were used to elicit additional information based on the course of conversations and interviewer's observations. CS, a female master student of public health from Spain conducted all interviews. DS, a Canadian male professor in health systems, accompanied CS in the interviews conducted in Switzerland and SM, a male senior social scientist holding a Master of Public Health from Tanzania co-led the interviews in Tanzania. SM led the introductions to the interviewees. Information about the study aim was shared with all participants and written or oral informed consent was collected.
28 interviewees (21 male, 7 female) provided free and informed consent. Three additional interviews were aimed at district and national level, but the interviewees argued having no time for the interview or the phone number provided didn't work. Table 1 describes the organization and level of these, de ned by the geographical reach of their work. 21 interviews were conducted face-to-face in the place of work of the interviewees. A con dential and quiet place was always found to discuss the study questions. The other seven participants were interviewed over the phone or Skype. The average duration of these interviews was one hour. Three interviews were conducted with two interviewees simultaneously. No interview was repeated. Sampling was stopped once data saturation was reached.

Data analysis
Interviews were audiotaped, transcribed and translated, when necessary, into English. Researchers' interview notes were typed and organised into summaries of each interview. This collation encouraged immersion in the data to aid analysis and re ection from an early stage during data collection. CS and SM both participated in this process.
Transcripts and interview summaries were shared, when possible, with the study participants.
Nevertheless, no feedback was received from these.
All interviews were coded and analysed by CS, who was supervised by DC and DS. CS conducted a joint thematic analysis of the transcripts and collated documents using a directed approach with prede ned categories of the WHO MAPS toolkit (Table 2) in Nvivo12. Findings were validated by the wider research team.

Ethical issues
The study received approval from ethics committees at the Swiss Tropical and Public Health Institute, the Tanzania IRB (NIMR/HQ/R.8a/Vol.IX/2767) and COSTECH. All participants in the study signed a consent form or gave oral consent after being given a written study information sheet and a verbal explanation of the consenting process.

Results
We used the domains from WHO MAPS toolkit to structure the main ndings of our qualitative study (summarised in Table 3) (13).
The high costs of the programme (50'000 USD/month), combined with its focus on a single disease, its failure to adapt to the arrival of competitor systems to Tanzania, and the lack of continuity between ministerial leaders, were shown to be the main reasons for its discontinuation. Most interviewees expressed that SMS for Life was well aligned with the country's priorities when it was implemented because the malaria burden was high and delivering rst-line drugs was essential and a challenge for the country. However, the interviewees stated that focussing on stock-outs and not on appropriate stock management fell short of solving the problem and caused some political challenges as no-one wanted to report bad news.
In terms of partnerships, SMS for Life aimed to engage the right partners before rolling out the programme. Its internally-approved, not publicly available implementation guidelines stated that: "before the SMS for Life project is rolled out in any country, it is vital that the key stakeholders involved in Malaria Prevention and Control, and the relevant people responsible for drug distribution and management are informed about, and involved in, the project." However, the opinions of the interviewees differ. A substantial number of interviewees mentioned that not all key stakeholders were involved in the design and deployment of the programme. Some examples given that were not engaged were district pharmacists, MSD or o cials at the regional level.
SMS for Life had strong champions during the pilot, but there were many changes in the government during the scale-up, which led to changes in priorities.
"Changes in leadership, changes in the background in the people in charge, changes in the priorities, this all in uences sustainability of programmes." District Medical O cer With changes in leadership, questions were also raised regarding the costs of the programme. Most interviewees agree that the main reason for discontinuation was that the programme was too expensive to cover just one disease and that it was not possible to justify why a substantial share of the MoH's budget should be dedicated to the supply chain of one programme and not others.
"The system was very expensive. The government asked itself should I pay for a system that costs over 500`000 USD/year or buy drugs for that amount of money?" External researcher Despite some challenges being identi ed in the political arena, the programme was well-received among health facility personnel and it demonstrated that health facilities could use SMS technologies easily and provide continuous stock reporting (17).
The simplicity of sending a weekly SMS was a contrast to the complexity and delays of responding to stock-outs using the quarterly paper system. The response by the health system to the stock-outs SMS for Life was reporting, was not properly designed. Interviewees at the highest level of the national institutions explained that the response to stock-outs was a big challenge of SMS for Life. The reasons for the lack of response when a stock-out was reported through SMS for Life are shown in Table 4.
Nevertheless, the evidence found, including the WHO MoH Service and Availability Readiness Assessment (SARA), showed that antimalarial stock-outs were occurring less frequently in most parts of the country during the time of SMS for Life, especially from 2014(18-20). The main reason stated by most interviewees for this improvement were modi cations in the procurement system that were occurring in parallel to SMS for Life. V ertical programmes (e.g. national malaria control programme) and health facilities were allowed to receive funds directly to purchase drugs, instead of having to go through the MSD. The purchase of drugs was decentralised and direct delivery was introduced. As part of this substantial initiative to improve the stock situation, the electronic Logistics Management Information System (eLMIS) was launched and the Logistics Management Unit (LMU), within the MoH, was created to manage the system (20).
In addition, shortly after the scale-up of SMS for Life, other supply chain information systems entered the Tanzanian landscape, including ILS Gateway. This programme had a similar objective and was also an SMS-based solution, but included 20 different products and had lower running costs. However, users reported this second system was more di cult to use.
The coexistence of both programmes with similar objectives but funded by different donors led to competition between them, even if thought to be complementary. ILSGateway and SMS for Life were making each other redundant. In a Global Fund Audit the following was reported: "Although developed for different purposes and meant to be complementary, some of the developed systems have duplicative functions, e.g. the SMS4life and ILS gateway (costing USD 600,000 and 60,000 annually, respectively) collect similar information. There are plans to discontinue SMS4life under the new grants." (20) SMS for Life was a stand-alone system, so the information was not owing to any other information system in the country. This was reported in almost all interviews as a big challenge for the sustainability of the programme.
Several information systems have been implemented since SMS for Life was scaled-up. MSD works with Epicore 9 to track the logistics of their procedures, while the health system has had different independent systems to track stock since the pull system was implemented: Request&Requisition (R&R) paper forms, end-use veri cation forms, tracer forms, ILS Gateway, DHIS2, and, most recently, eLMIS (21).
These systems increased the complexity of taking action against stock-outs, as they were often reporting contradictory results. To harmonize Tanzania's digital systems, including the supply chain information systems, the country launched its national e-health strategy in 2013(21).

Discussion
The overarching purpose of this study was to contribute to a growing body of literature examining digital health interventions, including the challenges involved in integrating and sustaining such interventions in a complex health system. This paper focusses on a long-running national-scale mHealth solution, which was eventually discontinued. Publications detailing such programmes are often biased towards positive results. A recent study by Al-Durra et al (22) examined non-publication rates in digital health and concluded higher non-reporting rates than in other elds. These results are worrying considering the large number of pilots being conducted in digital health that never reach full scale (7). The learnings that can be taken from failed systems integration attempts are crucially important for the success of future programmes.
We chose to study SMS for Life 1.0 because it was one pilot that had managed to overcome the initial challenges and reached full national scale in 2011, a time when digital health was still emerging in LMICs and the majority of pilots failed to reach this scale (1,22). However, despite initial success, the programme was discontinued without a nal evaluation in 2015.
During the rollout, the technology of SMS for Life proved to be technically reliable and well appreciated. The programme collected real-time data in rural health facilities at a time when this was uncommon (9) and ultimately achieved its stated primary objective to provide visibility of antimalarial stock-outs.
Our analysis suggests that this narrow objective fell short of what was required in Tanzania and the programme would have been improved by aiming to reduce stock-outs, rather than just aiding visibility. A change in aim would have encouraged not only the reporting but also the action necessary to solve stockouts by the health system. The literature describes this output-driven set-up as a common shortfall of digital health projects being implemented around 2010 (23). In 2011, and as a response of this, the Bellagio Statement on eHealth Evidence was written, calling for better evaluation and research of digital health programmes (24). SMS for Life was built within the national malaria control programme and failed to consolidate and communicate with other programmes or departments (e.g. MSD, HMIS Department) (25)(26)(27). In the early days of digital health, this was a common shortcoming where many digital health solutions were developed in isolation and focussed on a single challenge (25,28,29). For future digital health programmes, identifying a process that could bene t more than one healthcare problem, would ensure greater stakeholder buy-in and set up the programme for prolonged success.
During the scale-up of the programme, many internal changes occurred in Tanzania that impacted the sustainability of SMS for Life, including changes in government, treatment policies, and supply chain systems. One of the major changes was the introduction of new logistics management information systems. At the health facility level, the coexistence of different systems with similar objectives confused and at national-level it caused competition between programmes.
With time, champions of SMS for Life transitioned to other government positions and thus support for the programme's implementation waned. Meanwhile, the ILS Gateway gained relevance in the new government. ILS Gateway was perceived as a more cost-effective solution, although an economic evaluation was never conducted to compare both systems. Despite this, most interviewees considered SMS for Life a more reliable technology. Iribarren et al (30) report this lack of economic evaluation and comparison within programmes in their systematic review and advocate for the use of economic methods to undertake informed decisions. The more recent digital health guidelines published by WHO also highlight the need to conduct detailed costing exercises (31).
In Tanzania, the MoH had initially committed to take over the costs of SMS for Life but this never transpired. When it came to taking these costs into its own budget, the programme was discontinued.
This has been reported before, as there is often a large gap between the initial commitment and the actual realisation of assuming the costs(23).
Additionally, SMS for Life lacked a monitoring and evaluation scheme, and no implementation research was conducted to assess the performance after scale-up. The WHO digital health guidelines also address this and call for more effectiveness studies to monitor performance and behavioural performance or user engagement for the bene t of future programmes.
Agile management techniques are now gaining momentum and we believe SMS for Life would have bene ted tremendously from their application to adapt to the on-going changes in Tanzania.

Implications for future digital health solutions at scale
This study highlights the need to rethink interventions when going to scale and to highlight these in the literature for future implementation processes to learn from. Agile project management schemes(32), cost evaluations and system approaches are important tools in achieving sustainable programmes (33). We also encourage the use of the MAPS toolkit for project managers to provide them with resources to assess the maturity of their mHealth programme and stage-based strategies for scale-up(13).

Limitations
There is a level of subjectivity involved in qualitative case studies and this should be considered when interpreting the study ndings. By applying different methodologies, we aimed to triangulate results and overcome some of the limitations associated with each methodology.

Conclusion
It is important to recognise the many successes and innovations that SMS for Life brought to Tanzania and to mHealth globally. The programme developed and introduced an SMS-based solution to address the serious antimalarial stock-out problem. However, it failed to address complex systems and individual, interpersonal, and institutional factors in programme planning, which ultimately led to its discontinuation.
To bridge this gap, we suggest the use of holistic methodologies and detailed costing exercises. These would help governments to carefully plan the programme to ensure its affordability right from its early stages. The programme would also have bene ted from a health systems approach that would have guided programme managers to better navigate complex adaptive health systems (34,35). These learnings are echoed in the steps from the WHO view of systems thinking and the most recent WHO digital health guidelines (12,31).
This research may be relevant to policy and decision-makers as well as programme planners, considering the expansion of digital health programmes to regional or national levels of service. We strongly suggest making use of the tools now available and include scale-up frameworks from the beginning of the intervention design.
Finally, despite the discontinuation of SMS for Life 1.0, key learnings can be taken forward to ensure the success of future programmes and ultimately improve healthcare for all. Declarations Ethics approval and consent to participate: Granted by the National Institute for Medical Research in Tanzania (NIMR/HQ/R.8a/Vol.IX/2767). Written or oral informed consent to participate in the study was obtained from all participants, and permission was granted to use non-identi ed qualitative data for publication Consent for publication: not applicable.
Availability of data and materials: The transcripts that were used for the development of this manuscript can be requested from the corresponding author.
Competing interests: None declared.
Funding: This work was made possible by a grant of Novartis Social Business, Switzerland Authors' contributions: CS designed the study, conducted all interviews, led the analysis and writing of this paper. SM co-led the data collection and interpretation of the data, as well as contributed to the review of the manuscript. DC and DDS made substantial contributions to the design of the work, data analysis and interpretation of the data. All authors have read and approved the nal version of this manuscript.   1. No monitoring and evaluation framework 2. One external evaluation conducted in 2013 reported several challenges that were never addressed Table 4: Reasons stated for the lack of action to solve stock-outs at district level 1) Commodities claimed not to be available at MSD, often due to delays in procurement 2) Lack of resources for transport between health facilities or to pick up drugs at MSD 3) Unwillingness by the health facilities to reallocate their surplus stock due to lack of confidence in a replenishment delivery 4) Lack of ownership of district leadership of the programme, which led to a lack of use of the collected data ISSMCOREQChecklist.pdf