Development of a Framework for the Implementation of Synchronous Digital Mental Health: A Realist Synthesis of Systematic Reviews

David Villarreal-Zegarra IPOPS: Instituto Peruano de Orientacion Psicologica Christoper A. Alarcon-Ruiz San Ignacio de Loyola University: Universidad San Ignacio de Loyola GJ Melendez-Torres University of Exeter Medical School Roberto Torres-Puente IPOPS: Instituto Peruano de Orientacion Psicologica Alba Navarro-Flores IPOPS: Instituto Peruano de Orientacion Psicologica Victoria Cavero Red de Investigacion en Servicios de Salud en Enfermedades Cronicas Juan Ambrosio-Melgarejo IPOPS: Instituto Peruano de Orientacion Psicologica Jefferson Rojas-Vargas National University of San Marcos San Fernando School of Medicine: Universidad Nacional Mayor de San Marcos Facultad de Medicina de San Fernando Guillermo Almeida IPOPS: Instituto Peruano de Orientacion Psicologica Leonardo Albitres-Flores Red de Investigacion en Servicios de Salud en Enfermedades Cronicas Alejandra B. Romero-Cabrera Universidad Cientí ca del Sur: Universidad Cienti ca del Sur Jeff Huarcaya-Victoria (  jhuarcayav@usmp.pe ) Hospital Nivel IV Guillermo Almenara Irigoyen https://orcid.org/0000-0003-4525-9545


Information sources
We searched MEDLINE (Ovid), EBM Reviews (Ovid), PsycINFO (Ovid), EMBASE (Elsevier), SCOPUS, CINAHL Complete (EBSCOhost), and Web of Science databases, including Science Citation Index Expanded, Social Sciences Citation Index and Conference Proceedings Citation Index (Clarivate Analytics). Articles published in the last ve years (January 1st, 2015 to April 31, 2020) were included with no language restrictions. Database's search was performed at April 31, 2020.

Search strategy
The search formula was created using thesaurus and entry terms for the following syntaxis: "telemedicine" AND "mental health, anxiety, depression or stress" AND "systematic reviews". The full search strategy for each database is available in the Supplementary Material 2.

Study records Data management
The records retrieved after the search were managed using the Rayyan QCRI free online application (eliminate duplicates, and review titles and abstracts) [20]. Full-text review and data extraction were performed in an Excel template.

Selection process
The records were screened by title and abstract and then by full-text assessment. Records were divided into three groups. Two independent authors previously calibrated (discrepancy less than 5%) assessed each record for each group. Each pair of authors discussed discrepancies, and a third reviewer was included if needed.

Data collection process
For each eligible study, data were extracted independently and duplicated on pre-designed extraction forms. Reviewers solved discrepancies, and a third reviewer evaluated any unresolved disagreement.

Data items
An extraction form was created for the included systematic reviews. We collected the following information: rst author and publication date of the study, characteristics of the participants, main objective, research questions, inclusion criteria for the systematic review, search date, study selection process, quality assessment (if any), main ndings, and limitations. Also, the full-text of the included articles, the tables, and supplementary material were gathered to perform the qualitative analysis of the text.
Our study aimed to conduct a realist review of systematic reviews, using a qualitative strategy to synthesize the information and answer our research question. Therefore, we did not look for a speci c result such as effectiveness, cost-effectiveness, or similar. Instead, we were interested in identifying the fulltext of all studies that answered our research question to perform a grounded theory analysis with an emergent approach [21]. Priority was given in the analysis of those studies with the lowest risk of bias assessed.

Risk of bias in individual studies
To assess the quality of the included systematic reviews, we used the "A Measurement Tool to Assess Systematic Reviews-2" (AMSTAR-2), which has sixteen domains. Seven of these domains are considered critical: 1) protocol registered before the start of the review, 2) adequacy of the literature search, 3) justi cation for the exclusion of individual studies, 4) risk of bias of individual studies included in the review, 5) adequacy of meta-analytic methods, 6) consideration of the risk of bias in interpreting the results of the review, and 7) assessment of the presence and likely impact of publication bias [22].
AMSTAR-2 classi es the quality of systematic reviews into four categories: high (none or one non-critical weakness), moderate (more than one non-critical weakness), low (one critical weakness with or without non-critical weaknesses), and very low (more than one critical weakness with or without non-critical weaknesses). The quality assessment was rated by two trained researchers independently. In case of difference in the overall quality of the systematic reviews, the AMSTAR-2 criteria were discussed among both researchers to reach a consensus.

Data synthesis
We developed a framework informed by a realist analysis of synchronous digital mental health interventions using a grounded theory approach with an emergent approach [23]. The realist synthesis was based on interpreting, integrating, and inferring the evaluation elements to better understanding the implementation of synchronous digital mental health interventions from all the included studies [24]. To answer the question "what makes the implementation of these interventions work?", hypotheses supported by included studies' results were developed and generated through discussion and consensus among the researchers [24]. Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input to assess the implementation of synchronous digital mental health interventions. Therefore, we did not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).
Three researchers followed the three steps established by Thomas and Harden for qualitative syntheses [25]. First, the extracted data was freely coded. The researchers read the full texts of the included articles and coded each text fragment that provided information to answer the research question. Second, the codi ed data was organized, and then grouped based on descriptive aspects using a context-linked causality approach represented as "context + mechanism = outcome" [21]. Finally, the analytical concepts generated in the previous step were grouped in a way in which they were related to each other. The elements that were related to each other were assumed to be part of a hypothesis that would help to answer the research aim.
The selection of the studies for the realist review was based on the AMSTAR-2 score, with the highest quality studies being assessed rst. We assessed all included studies, down to the criterion of theoretical saturation [26]. All qualitative analyses were performed with the NVivo software (version 12, QSR International).

Con dence in cumulative evidence
The Con dence in Evidence from Reviews of Qualitative Research (CERQual) approach, which has four components (Methodological Limitations, Relevance, Coherence, and Appropriateness Data), was assessed by a researcher, and then reviewed by another independent researcher. The CERQual was evaluated to contribute to an overall assessment of each hypothesis resulting from the realist synthesis to determine the level of con dence (high, moderate, low, or very low) and present the overall assessment in a Summary of Qualitative Findings (SoQF) table [27,28].

Study selection
The search strategy retrieved 30,228 records, and after duplicated cleaning, we obtained 14,536 unique records. The evaluation by title and abstract identi ed 374 results that were evaluated at full-text. From those, 353 were excluded. The reasons for exclusion are listed in Supplementary Material 3. Finally, 21 systematic reviews were included in this study (see Figure 1).

Study characteristics
The included systematic reviews included an average of 26.8 studies (range 9-155). Eleven studies reported some form of synchronous digital mental health intervention based on Internet, telephone, or online cognitive-behavioral therapy as the primary intervention [29][30][31][32][33][34][35][36][37][38][39]. The remaining studies reported a mix of digital mental health interventions based on synchronous components (i.e., telephone, videoconferencing) and asynchronous components (i.e., text messages, email, chats, instructional videos, podcasts). Most of the systematic reviews included exclusively randomized-controlled trials (RCTs) as primary studies, two included only non-RCTs, and ve studies included both. Only six studies didn´t include a meta-analysis. About the type of therapy, nine reviews stated as target therapy Cognitive Behavioral Therapy (CBT), one review used the transdiagnosis method and one included Mindfulness-based interventions. The individual characteristics of the included studies are presented in Table 1. It is important to mention that despite having no language restrictions, all the included articles were published in English and the systematic reviews did not include qualitative studies. Risk of bias within studies Cochrane Collaboration tool (n=12; 57%). [30-32, 34, 35, 37, 40-45]. Seven studies used other tools to assess the risk of bias such as the Effective Public Health Practice Project Quality Assessment Tool (n=2; 9%); Grading of Recommendations Assessment, Development and Evaluation (n=1), and others. Only two studies did not report using any risk of bias tool [38,39]. Ten studies did not appropriately account for the risk of bias of the individual studies included when interpreting the results of their review.

Risk of bias across studies
The study by Olthuis et al [44] presented a medium level of con dence and the study by Lewis et al [35] presented low con dence. The rest of the included systematic reviews presented a critically low level of con dence (see Figure 2). On average, the included reviews only met 40% of the AMSTAR-2 risk of bias items. The study by Rees et al [38] failed to accomplish any of the AMSTAR-2 items and the study by Turgoose et al [46], only passed one AMSTAR-2 item.
The AMSTAR-2 items that were the most ful lled (if applicable) were item 15 (critical) of assessing the presence and likely impact of publication bias (93%), and item 12 (non-critical) of assessing the potential impact of risk of bias in individual studies (73%), in case of meta-analysis. The AMSTAR-2 items that were least ful lled were item 10 (non-critical) on whether the review reported the funding sources of the included studies. Only the study by Irvine et al. [33] achieved compliance. Two other items that had a low compliance rate (14%) were item 4 (critical) on the adequate literature search, and item 3 (non-critical) on the justi cation for the decision on the study designs to be included in the review, and only one study met each of these criteria [43].

Realist synthesis
Synchronous digital mental health interventions provide effective clinical outcomes (see Figure 3). Some systematic reviews identi ed that digital mental health interventions based on CBT (i.e., telephone, internet-based, videoconferencing, online) were equally effective as face-to-face CBT in the treatment of speci c mental health conditions (e.g., social anxiety disorder, PTSD, panic, depressive symptoms, body dissatisfaction, insomnia, speci c phobias) [ interventions have shown to be effective in different population groups, such as adults and elder people [30,31,37,42], veterans [44,46], and people with multiple sclerosis [45].
The advantages of interventions using technology are allowing the inclusion of add-ups to the therapy (e.g., written, audio or visual materials to access online or download, diary-keeping, chats [29], emails [29,44], online forums [29,40,43], new platforms or existing ones, such as Skype or Zoom, etc.) [46]. These interventions also promote better coordination of care and early treatment [39,46].
Guided synchronous components are essential elements in digital interventions to reduce anxiety. They are more effective and improve adherence signi cantly compared to unguided interventions or those with only asynchronous components [40]. Also, it is unclear which guided synchronous components are the most effective or whether there are cumulative effects when combining them [40]. Of note, CBT-based and heterogeneous digital mental health interventions (not CBT-based) showed no difference in their effectiveness in reducing PTSD symptoms [35].

Hypothesis 1
Synchronous digital mental health interventions reach populations otherwise unable to have access through face-to-face interventions, since they do not require the physical presence of the therapist nor the patient, thereby tackling geographic barriers posed by in-person therapy (To expand access).
Synchronous digital interventions in mental health reach populations that would not have access through face-to-face interventions, such as children, veterans, refugees, and people living in rural areas [47,49], because these interventions do not require the physical presence of both the patient and the therapist (see Figure 3). We also found that these interventions can reduce geographical barriers to access (e.g. mobilization for several hours). In addition, they can interact in real-time [38], and tackle the geographic barriers of travel required to receive care, thereby being able to access even from remote areas [34,38,39,[43][44][45]47].
Some aspects need to be taken into consideration for the delivery of successful therapy through synchronous digital mental health interventions. First, to nd a quiet area in home or at the usual environment of the patient to receive the session, which could represent a challenge for many [46]. Second, the platform should be as stable as possible since ineffective internet service could lead to withdrawing the therapy [46], and the quality of the image and sound could be associated with satisfaction [36]. Third, the possibility to expand the use of telepsychiatry will require the development or improvement of a software specially designed for that purpose [39]. Finally, the presence of technical support when needed, as one systematic review found that scheduled guidance showed better outcomes on anxiety symptom severity at post intervention and follow-up [40].
The presence of synchronous human support seems to improve the delivery of digital mental health sessions, although the evidence is not conclusive [29,40].
Guided interventions were superior to completely unguided interventions for symptom severity across mental disorders and presented higher treatment adherence [40]. In studies that used local clinics rather than home-based teletherapy, it was recommended to have local staff on hand to assist, such as to receive homework and other materials via fax machine and disseminate them to participants [39]. However, in the future, arti cial intelligence could replace human support to generate computer responses [30].
Additionally, we found some barriers: First, the absence of physical contact. One review identi ed that patients receiving in-person treatment were more likely to complete the home assessments and tasks given [46]. Second, the safety of the patient could be compromised. It is worth noting a potential issue with interventions using technology. The distance between patient and therapist could put patients' safety at risk, who could not receive the necessary care in the event of a crisis or emergency [40]. Some studies also suggested the presence of an extra person to provide in-person support in case of emergencies [40,46], although not all studies showed favorable results [29,30,40]. Finally, the presence of technical issues could impose a potentially modi able barrier. Some aws found during the therapy delivery were limited connectivity, the lack of human resources and telepsychiatry equipment [39], low image resolution, di culties for establishing the connection, slight audio delays, and problems with the internet connection [39]. Moreover, a systematic review assessing mindfulness-based cognitive therapy for stress reduction found that the users' dissatisfaction was linked to technical issues [36].

Hypothesis 2
Synchronous digital mental health interventions reach populations otherwise unable to have access via face-to-face interventions because they can be successfully delivered by non-specialists, which makes them more cost-effective to implement in health services (To expand access).
A second reason why these interventions reach populations that otherwise would not have access to face-to-face interventions is that they are an accessible and cost-effective treatment in the short term [29]. This may lead to reductions in mental health costs, at least in depression [29]. It should be noted that CBTbased digital interventions tend to be slightly more expensive compared to usual treatment at baseline. Because their cost-effectiveness improves when considering their positive effect on quality-adjusted life years [29] and their costs in the long-term, since they require limited interaction between patient and therapist [31,39].
This higher cost-effectiveness is associated with different components. Regarding phone sessions, they adhere to a more structured format and focus on problem solving and tasks, resulting in more e cient and direct sessions [33]; and shorter durations than in-person therapies [31,33]. It should be noted that the session duration of these interventions was not associated with better outcomes in cases of anxiety and depression, although the therapy duration varied from 19 to 150 minutes [37].
Evidence suggests that physicians, psychiatrists, psychologists, or nurses trained for various mental health problems could perform digital interventions such as telepsychiatry or teleconsultations [49]. This enables optimization for using available human resources when there is a reduced number of specialists for large populations, since non-specialists with adequate training and supervision are as effective as specialists for this purpose [38,40]. For this outcome, it is important to consider some barriers. A potential barrier was the provision of care by non-specialists, highlighting the importance of having appropriate training and supervision to provide long-distant care. Training for therapists providing interventions using technology should include contents on good clinical practices [36,49], the use of technology [46] and telepsychiatry [49], the management of risk or crises [40], as well as potential ethical and/or legal con icts [47]. Second, distrust of the health personnel. One study pointed out that therapists showed greater preference for face-to-face interventions compared to online interventions [35], while another found that some professionals may be reluctant to apply electronic interventions using telephones to treat mental health problems, arguing that it could harm the interactions with the user [33]. However, evidence suggests that the use of electronic interventions using telephones does not change interaction patterns in consultations (duration, alliance, disclosure, empathy, attention, and participation) [33].
Some relevant aspects to consider are clinicians' satisfaction, the lack of training for providers, and ethical challenges. For example, a systematic review of teletherapy for veterans with PTSD found high delity to the intervention and good therapist competence, as well high levels of satisfaction among clinicians in terms of their con dence for the delivery of these forms of therapies [46]. However, as mentioned before, proper training is needed for successful delivery [36,46,49], and the ethical and legal aspects should be established [47] .

Hypothesis 3
Synchronous digital mental health interventions are acceptable by patients and show good results in satisfaction, because they require less need of disclosure and provide more privacy, comfortability, and participation, enabling the establishment of rapport with the therapist (User's satisfaction).
Telepsychiatry for PTSD patients shows the advantage of diminishing the risk of stigmatization. Since patients are treated from their own homes and are no longer required to visit a psychiatric facility, they feel more motivated to seek mental health care [39]. One systematic review found that patients presented more active participation at distance-delivered therapies compared to face-to-face interviews. This may be due to the feeling of "safety" that being at a different location from the therapist could produce. They found that neither empathy, attention nor participation diminished when using telephone interventions [46]. Additionally, telephonic interventions offer the patient a potentially immediate, anonymous, and easy to access option [31]. Another author pointed out that patients felt that the therapist could understand them better during face-to-face therapies. However, there were no differences for the ability of the therapist to guide the patients to "open themselves" between modalities [33]. It was reported that the e cacy of interventions was similar across modalities and although the interaction between patient and therapist was lower [36], the therapeutic alliance was able to be achieved without limitations [39,44], except for the di culties at reading corporal language [46].
Telephone and videocall interventions were usually acceptable and e cient for digital mental health [38]. This was probably because more access to care was allowed for children and adults with comorbid psychiatric and complex medical illnesses in various settings, age spans, and demographic characteristics, including rural areas [49]. However, although there is greater satisfaction on the users' side (and therefore an improvement in mood state), this does not imply that there are improvements in the quality of life, since recovery (the relief of depressive symptoms) does not necessarily amount to parallel improvements in quality-of-life measures [29]. In addition, it should be considered that those two outcomes do not follow the same recovery rate.
Worth noting, during telephone therapies, the patients could develop an awareness of their own emotional and affective changes by listening to their own voice. Moreover, since there is no difference in the measure of how "closely" the therapist could be listening as usual in face-to-face communication, the patients could easier feel the "connection" with their therapist and enhance disclosure of feelings and emotions [33]. It was found that the use of technology did not in uence the therapeutic alliance with their patients [36, 44,46]. This could be explained since, in this context, the therapist's validation is not based on non-verbal communication but their listening capacity, their verbal clarity, the tone of voice used by the therapist, and how the patient experiences it [40].
Indeed, telephone therapy could work better for introverted patients because it provides more anonymity, creating a sense of safety [31,40]. Some aspects to consider include barriers such as awkward silence, concerns about privacy and constraint communication. Some patients had expressed their privacy concerns. For instance, veterans with PTSD mentioned questions about the con dentiality of the video transmissions, and the data they shared during the consultation [46]. In that same review, constraint communication for detecting body language and non-verbal communication by clinicians when doing teletherapy for veterans with PTSD was reported. However, they still could develop rapport [46]. Finally, during communications where there is no video of the patient, as in telephone therapy, silences during the patients' speech were more challenging to interpret. [33] One review noted that only two studies reported to providing ongoing technical support during interventions [36]. In addition, none of the studies included in their review mentioned videoconferencing-speci c good practice guidelines, training of facilitators to conduct online psychological interventions, or contingency plans to support remote participants [36]. Also, few studies reported on the frequency of technical problems [36].

Gaps
Limitations of digital mental health reported in reviews Lastly, even though technology interventions have proven to be as effective as in-person and have 2.13 times more probability of achieving an appointment once a month [49], some limitations should be noted. First, their effectiveness will depend on treatment adherence [37]. Second, there is limited information on whether CBT-based electronic interventions maintain their bene cial effects over time; two systematic reviews did not identify su cient evidence to support the bene ts of this therapy after 3-or 6-months post-treatment for PTSD cases [35,44]. Third, most of these studies did not use randomization and their sample sizes were small, therefore more research is needed [29, 32-34, 36, 38, 41-43, 45, 46, 48]. Finally, most of the available evidence comes from highresource countries with integrated health systems and larger research budgets [39]. Hence some results may not be extrapolated to low-or middle-income countries.

Con dence in cumulative evidence
An overall analysis of the CERQual assessment shows that the hypotheses presented have low or very low con dence in the evidence (see Table 2). The main methodological limitations are that the studies come from research with a low or very low con dence level. In terms of coherence, the baseline assumption and hypothesis 1 show adequate coherence between the different ndings, while hypotheses 2 and 3 show moderate concern since some reviews show heterogeneous results. Finally, all hypotheses show the adequacy of the data and relevance of the results. One study with medium con dence, and 12 studies with very low con dence due to lack of exhaustive bibliographic search, absent of review protocol, incorrect methods for selection and data extraction process.
Minor concerns since these reviews agreed about bene ts of SDI in mental health, but not all of them assess the possible new barriers for the under-served population.
Minor concerns since most of these reviews mention the tackling of geographic barriers. But they offered super cial information about this phenomenon, according to few primary studies.
Minor concerns since the review nding is based in systematic reviews which included all recently published data. However, there is still lack of representation of low-middle income countries Very low con dence SDI in mental health can be successfully delivered by nonspecialists, and therefore they are more cost-effective to implement in health services. But there is a need for training and supervision for human resources, and this can create distrust in some health personals (Hypothesis 2) 12 studies: 31,33,35,37,38,39,40,41,42,48,49,51 Serious concerns regarding methodological limitations. One study with low con dence, and 11 studies with very low con dence due to lack of exhaustive bibliographic search, absent of review protocol, and incorrect interpretation of risk of bias of individual studies.
Moderate concerns since some reviews acknowledge the higher cost of SDI but, not all of them agreed about the costeffectiveness of these interventions.
Minor concerns since most of these reviews mention the tackling of geographic barriers. But they offered super cial information about this phenomenon, according to few primary studies.
Minor concerns since the review nding is based in systematic reviews which included all recently published data. However, there is still lack of representation of low-middle income countries Very low con dence SDI in mental health are acceptable by patients and show good results in satisfaction, because they require less need of disclosure and provide more privacy, comfortability, and participation, enabling the establishment of rapport with the therapist (Hypothesis 3). 10 studies: 31,33,35,38,40,41,42,46,48,51 Serious concerns regarding methodological limitations. All 10 studies with very low con dence due to lack of exhaustive bibliographic search, absent of review protocol, incorrect assessment and interpretation of heterogeneity and risk of bias of individual studies.
Moderate concerns since some reviews also highlighted some barriers for these SDI which could in uence in the patients' perceptions about the interventions Minor concerns since most of these reviews mention the tackling of geographic barriers. But they offered super cial information about this phenomenon, according to few primary studies.
Minor concerns since the review nding is based in systematic reviews which included all recently published data. However, there is still lack of representation of low-middle income countries Very low con dence Discussion Implementation science is an emerging and rapidly growing eld that has established frameworks, methods, and strategies to improve the adoption and sustainability of interventions within the real world [50]; it has also identi ed different barriers and facilitators to the implementation of digital mental health interventions [50]. However, strategies speci cally for implementing digital mental health interventions within the healthcare system are still limited [50][51][52]. Our study develops a framework based on three hypotheses and a baseline assumption to understand/explain the implementation of synchronous digital mental health interventions.
The implementation of digital mental health interventions allows overcoming many barriers in health access, such as geographic, human resources, and stigma barriers. These types of interventions allow patients and therapists to remain in their usual, more comfortable, or safer location. Another advantage is that our framework supports that other mental health providers with lesser degrees after appropriate training could deliver digital mental health interventions, which would increase the available human resources pool of therapists [38,40]. In addition, digital mental health interventions could be more attractive than face-to-face therapies, as they present the opportunity to increase privacy and minimize the risk of stigmatization, as they could take place outside of mental health institutions, especially for populations in which the presence of potential social stigma interferes with the decision to attend mental health facilities [39].
Our study provides hypotheses based on systematic reviews, which allow a better understanding for the implementation of synchronous interventions in digital mental health. However, it does not provide speci c steps or strategies to carry out the implementation process. Therefore, to ll this gap, other researchers could use the ERIC project framework, which presents four general phases for implementing digital interventions in the health system: implementation strategy exploration phase, the preparation phase, implementation phase, and sustainability phase [50,53]. It should be noted that other frameworks that systematize the implementation steps could be used to perform the implementation task, as long as they are adapted to the particularities of the context, the health system, the resources, and the willingness of the actors involved. An alternative that has proven to be useful in favoring the implementation of interventions from heterogeneous contexts are the formative studies that allows for the contextualization of these interventions, while evaluating their acceptability, e ciency, and safety within the health system or community [54]. However, this requires greater investment in research by medium and low-resource countries.
There are currently no frameworks explaining the implementation of digital interventions as the main component in mental health care. A systematic review of barriers and facilitators to the implementation of electronic mental health interventions identi ed that the acceptability of electronic interventions depends on 1) patients' and professionals' expectations 2) and preferences about what they would receive and what they provide during care 3) the appropriateness of the electronic intervention to address patients' mental health conditions [55]. In the absence of an integrative framework, our study proposes a technical underpinning of available evidence to enable decision-makers to implement electronic interventions to address mental health. We identi ed different reviews supported by electronic interventions for anxiety, depression, and PTSD, which are equivalent to face-to-face interventions [30-33, 39, 44, 46, 47], and are costeffective in the long term [29].
Despite evidence in favor of digital mental health interventions, there is a considerable difference between the reports from high-income and low-income countries. Some high-income countries had su cient evidence to conduct country-focused effectiveness evaluations. For example, a systematic review from the United Kingdom identi ed 7 out of 48 digital interventions promoted by their health system for depression and anxiety as having a small but consistent effect, and recommended their use [56]. In addition, the disparity in the amount of evidence remains in economic research, where a systematic review of economic studies identi ed that Internet-based digital interventions for anxiety and depression are cost-effective and recommended their use; however, only studies from high-income countries were identi ed [57].
In contrast, no reviews of effectiveness, cost-effectiveness, or acceptability of electronic interventions were identi ed for middle-and low-income countries.
The limited evidence from middle-and low-income countries suggests that their health systems made decisions based on little local evidence, low-quality evidence (i.e., expert review), or make decisions based on evidence from high-income countries (i.e., different contexts). Additionally, material and economic resources and internet access are limited in low-and middle-income countries. Thus, su cient internet access for healthcare providers and users should be assured for implementing these technologies. Other problems that could generate inequity are still limited access to smartphones in rural and low-income areas, low internet speed and network instability, which could generate gaps for an adequate implementation of these technologies.
An additional element to highlight, apart from the effectiveness or cost-effectiveness of electronic interventions, is the positive effects they could have on patients' quality of life. Although quality of life was not an outcome in our study, we found evidence that electronic interventions to treat mental health positively effect in the quality of life [35,37,43]. These results are consistent with other systematic reviews that CBT-based interventions (e.g., face-to-face, internet, or group) improve participants' quality of life [58,59]. Furthermore, this secondary bene t of electronic mental health interventions on quality of life appeared to affect years of life lost due to disability [60]. This explain why this outcome is key for understanding the cost-effectiveness of this type of intervention since its long-term effect is to reduce costs within the health system [29].

Implementation and Public health implications
Decision-makers and researchers could use this relevant information to support the implementation of electronic mental health interventions within their health systems (i.e., teleconsultation network). There is evidence to support digital intervention due to their effectiveness in depression, anxiety, and PTSD, their feasibility and acceptability, their safety, and additional effect on the quality of life of patients [32, 34-37, 41-43, 45, 46, 48]. The treatment models that have the most empirical support are those based on CBT, so they could be the rst type of interventions to be implemented. In addition, evidence supports those electronic interventions are cost-effective, making their implementation within health systems feasible in the long term.
Health systems must develop legislation and basic technological conditions to achieve the implementation of synchronous digital mental health interventions. First, legislation such as privacy policies, terms of use and technological requirements of teleconsultation platforms should be established [4].
All these issues should be covered and regulated by national policies and there should be an entity to enable their regulation. Consequently, healthcare systems should develop an integrated digital health/digital mental health system that is user-friendly for all literacy levels.
Second, there are a need for quality Internet and cell phone services to increase the likelihood of adherence [4,36,39]. Collaboration among public and private sectors is needed. Technical support and access to therapies should be exible in schedules, since participants would adjust the delivery to their own timetables. Hence, night schedules should be considered. On the other hand, training for personnel with minor degrees must be guaranteed in a standardized and systematic way [38, 40,49].
Third, for the implementation and use of electronic interventions, it is necessary to identify the barriers within each health system to achieve the acceptance of the different actors. Lack of access to technology (especially in low-resource countries), limited training in teleconsultation or reluctance of health personnel to use the technology, problems related to patient safety or privacy, and limited legislation on teleconsultation at country-level are necessary elements to evaluate during the planning of electronic interventions in mental health [61].
Fourth, the context of the COVID-19 pandemic has enhanced the use of technologies to provide health care and reduce health care access gaps, and decisionmakers need to take advantage of this context to enhance the implementation and adoption of these types of interventions [3][4][5][6][7]. It should be noted that digital interventions are not only a short-term solution, as the trend is to incorporate them as a key part of cost-effective healthcare systems [29,31,39].

Strengths and limitations
One of the strengths of our study is that we collected information from systematic reviews in a large number of databases, assuring the comprehensiveness of the evidence included. However, our study has limitations. First, the quality of the systematic reviews included was critically low for the most part, so this could limit the con dence in the conclusions of the study. Other studies have already reported the low quality of systematic reviews and clinical practice guidelines in mental health [62][63][64]. Second, the electronic interventions evaluated are very heterogeneous both in the form of delivery (i.e., telephone, internetbased, videoconferencing, online) and in the theoretical models used (CBT classical, CBT Mindfulness, CBT trans-diagnostic, non-speci c). Therefore, there may be variations in effect, safety, and acceptability in the way of delivery and the theoretical model used. Third, most of the research has been conducted in high-income countries, so the results may not be comparable in low-and-middle income countries. Fourth, although a realist review analysis was rigorously carried out, the evidence evaluated has methodological limitations, so that the overall certainty of the evidence is low.

Conclusions
Our study assessed all available evidence for the implementation of synchronous digital mental health interventions and developed a framework for the implementation of synchronous digital mental health based on three hypotheses. Since it is known that digital mental health interventions are clinically effective, we hypothesized that those interventions reach otherwise inaccessible populations since they abolish the need of physical presence and mobilization (H1), or because non-specialist could deliver it by with the additional advantage of reducing expenses (H2), and, that digital interventions are acceptable for those receiving them and maintain the establishment of rapport (H3). Each hypothesis represented important outcomes in the implementation process. In addition, we analyzed the barriers and facilitators for those outcomes and identi ed gaps in the body of evidence that require attention from future researchers.
Our study provides a framework to understand the implementation of synchronous digital mental health interventions, suggests elements to consider at the time of implementation, and establishes gaps. This information will guide decision-makers, researchers, health system managers, and implementation teams.

Declarations
Ethics approval and consent to participate Not applicable.

Consent for publication
Not applicable.

Availability of data and material
Open Science Framework: PRISMA-P checklist for 'Development of a framework for the implementation of electronic interventions in mental health: A protocol for a meta-synthesis of systematic reviews'. https://osf.io/3uh4n/