1. Implementation of E-learning
Implementation of synchronous E-learning
By the end of June 2019, the national-level China-Gates program remote training centre, developed by the Clinical Centre on tuberculosis of Chinese Centre for Disease Control and Prevention, and three provincial training centres had conducted 98 synchronous learning activities, including 47 national-level activities (32 remote training activities and 15 case discussions). The three provincial platforms organized 51 events, and Jilin had organized more activities (20) than Ningxia (13) and Zhejiang (12) (Table 2).
Table 2
Implementation of synchronous E-learning activities through the China-Gates Foundation TB Control Program
Province
|
Times of activities
|
Remote training
|
Case discussions
|
Telemedicine Consultation
|
Subtotal
|
Ningxia
|
13
|
2
|
2
|
17
|
Jilin
|
20
|
0
|
0
|
20
|
Zhejiang
|
12
|
2
|
0
|
14
|
National Centre
|
32
|
15
|
0
|
47
|
Total
|
77
|
19
|
2
|
98
|
Data source: National TB Telemedicine Consultation and Training Platform. |
39 TB institutions in the pilot area have access to the remote synchronous learning platform, including 14 institutions in Jilin, 9 institutions in Ningxia and 16 institutions in Zhejiang. On average, 23.5 TB institutions and 173.2 TB health workers accessed the national-level platform for each synchronous E-learning activity, while an average of 22.5 units and 163.3 TB health workers participated in each case discussion. Jilin had the highest participation rate followed by Ningxia. The participation in Zhejiang was relatively low (Table 3).
Table 3
Participation of synchronous E-learning in three provinces
Province
|
Average number of institutions online
|
Average number of students online
|
Remote training (%)
|
Case discussion (%)
|
All activities (%)
|
Remote training (SD)
|
Case discussion (SD)
|
All activities (SD)
|
Ningxia
|
6.1/9 (67.7)
|
5.3/9 (58.9)
|
5.8/9 (66.7)
|
42.8 (21.1)
|
36.8 (13.2)
|
40.9 (18.9)
|
Jilin
|
10.1/14 (72.1)
|
9.7/14 (69.3)
|
10.0/14 (71.4)
|
106.9 (39.4)
|
106.1 (34.6)
|
106.7 (38.0)
|
Zhejiang
|
7.1/16 (44.5)
|
6.8/16 (42.5)
|
6.9/16 (44.1)
|
23.5 (10.8)
|
20.4 (7.7)
|
22.5 (9.8)
|
Total
|
23.3/39 (59.7)
|
21.8/39 (55.9)
|
22.8/39 (58.5)
|
173.2 (49.8)
|
163.3 (41.2)
|
170.1 (47.4)
|
Data source: National TB Telemedicine Consultation and Training Platform. The numerators were the number of institutions which participated in the synchronous E-learning activities and the denominators were the total number of TB institutions in polit areas. |
Implementation of asynchronous E-learning
By the end of June 2019, there were 23 online asynchronous training modules on the China-Gates online training and qualification system, including 7 modules for public health personnel, 8 modules for doctors, 4 modules for primary care workers and 4 optional courses. At the same time, 906 (66.9%) TB health workers from the pilot area in three provinces registered for the system, and 681 (50.1%) TB health workers have completed one or more asynchronous training sessions and obtained the qualification certificates. Zhejiang has the highest level of participation, followed by Jilin and Ningxia (Tables 4).
Table 4
Participation of asynchronous E-learning activities in three provinces (%)
Province
|
Indicators
|
Doctors
|
Public health physicians
|
Primary care workers
|
Total
|
Ningxia
|
Registered the system
|
56/162 (34.6)
|
22/48 (45.8)
|
26/91 (28.6)
|
104/301 (34.6)
|
Obtained the qualification certificates
|
23/162 (14.2)
|
11/48 (22.9)
|
13/91 (14.3)
|
47/301 (15.6)
|
Jilin
|
Registered the system
|
190/239 (79.5)
|
51/91 (56.0)
|
63/104 (60.6)
|
304/434 (70.0)
|
Obtained the qualification certificates
|
123/239 (51.5)
|
41/91 (45.1)
|
51/104 (49.0)
|
215/434 (49.5)
|
Ningxia
|
Registered the system
|
60/85 (70.6)
|
114/151 (75.5)
|
324/384 (84.4)
|
498/620 (80.3)
|
Obtained the qualification certificates
|
33/85 (38.8)
|
91/151 (60.3)
|
295/384 (76.8)
|
419/620 (67.6)
|
Total
|
Registered the system
|
306/486 (63.0)
|
187/290 (64.5)
|
413/579 (71.3)
|
906/1355 (66.9)
|
Obtained the qualification certificates
|
179/486 (36.8)
|
143/290 (49.3)
|
359/579 (62.0)
|
681/1355 (50.3)
|
Data source: China TB prevention Online Training Website. |
Key informants reported that different provinces have different patterns of training organization. In Zhejiang Province, asynchronous learning activities were mainly organized by the CDCs with the "province-city-county" three-tier administrative management mode, while synchronous learning activities were TB designated hospitals’ responsibility. The organization of synchronous learning activities was led by provincial designated hospital. They faced many challenges: since the provincial TB designated hospital do not have a vertical administrative relationship with the city and county-level designated hospitals, like the CDCs do, they only maintain loose connections in the name of "technological guidance". At the beginning, they didn’t have the information of contacts in other TB hospitals (the information was later provided by the local CDCs). The training remainders also need to be sent to the CDCs first, and then forwarded to the local TB hospitals. In this way, the provincial hospitals could not be effective managers (Fig. 1). In Ningxia and Jilin, all institutions were notified of synchronous/asynchronous learning activities by provincial hospitals (including CDCs, hospitals, and TB dispensaries), while trainees at the township level were notified by county-level CDCs (Fig. 2).
2. Facilitators of E-learning CME
Facilitators of the training format
In the interviews, most of the TB health workers agreed that the E-learning model is economical and timesaving compared to traditional face to face training model. In the past, only a few face-to-face training opportunities were available for TB health workers, especially for those from the city level institutions and below. The TB institutions usually send one or two doctors for training. They were asked to provide retraining for those who were not able to attend. In some cases, they even used meetings as a substitute for on-job training. A lack of trained teachers and inadequate learning materials made their learning quite difficult. E-learning have expanded access to high-quality continuing medical educational resources, and reduced its costs.
Compared with the traditional training model, the online and remote training are timesaving and more efficient. Traditional methods are not able to invite people come over frequently, and most hospitals can't afford the human and financial costs. These two models, the E-learning models, have advantages in terms of convenience and economy. (Provincial TB health professional FGD in Zhejiang)
Facilitators of training content
In general, the training contents of the China-Gates Foundation Program were regarded as high-quality by the TB health workers. Teachers of the capacity building subproject are well-known experts in China, which provided great opportunities for city, county level and primary care health workers. The lower level health workers could receive training and technological guidance from the national experts, which is unlikely in traditional face-to-face training. Different health workers have different preferences: provincial and city-level TB health workers preferred the knowledge about disciplinary frontiers in synchronous E-learning activities, while county-level TB health workers favoured sections about routine clinical practice in asynchronous E-learning activities.
It's really nice to stay at home and have direct access to teachers from Beijing or across the country, teaching us the basic knowledge or cutting-edge ideas. It’s brilliant. (Provincial TB health professional FGD in Jilin)
Barriers in the implementation of E-learning
Barriers in matching the supply and demand of training
Although China-Gates Foundation Program provided TB staff with high-quality training contents, they were not evenly spread out among different types of TB health workers. In general, more courses were available to the clinical doctors: the total training loads including synchronous session once a week (or twice monthly) and nine modules of asynchronous sessions. However, the training loads for public health physicians and primary care workers was relatively small, with only seven and four modules received in three years respectively.
The uneven supply and demand were another key concern. Staff survey showed that clinical doctors thought they needed more training in the clinical diagnosis and treatment of TB (83.5% for multidrug-resistant TB, MDR-TB, 78.4% for TB radiology and 70.1% for common TB), which perfectly matched with their training supplies (Table 5). Although public health physicians need more training in TB infection control (76.2%), planning (73.4%), and surveillance (63.1%), they mostly received training in planning (91.8%), TB management (89.8%) and treatment of common TB (75.5%). The top three training needs for primary care workers were common and MDR-TB (79.8% and 70.4%, respectively) and TB management (66.7%); however, most mentioned training supplies were treatment for common TB (90.2%), TB management (83.5%) and TB case detection (72.6%). Similar results were observed in FGDs. Some TB health workers complained that the training materials were too difficult to learn and some sessions were not related to their daily practices.
Table 5
Supply and demand of training among TB health workers
Type of TB health workers
|
Training demand
|
Training supply
|
Matching
|
Sample size
|
Top 3 most mentioned (%)
|
Sample size
|
Top 3 most mentioned (%)
|
Doctors
|
97
|
Treatment for MDR-TB (83.5)
|
136
|
Treatment for common TB (90.4)
|
Yes
|
TB radiology (78.4)
|
TB radiology (85.3)
|
Yes
|
Treatment for common TB (70.1)
|
Treatment for common TB (85.3)
|
Yes
|
Public health physicians
|
65
|
TB planning (73.4%)
|
49
|
TB planning (91.8)
|
Yes
|
TB infection control (76.2)
|
TB management (89.8)
|
No
|
TB surveillance (63.1%)
|
Treatment of common TB (75.5%)
|
No
|
Primary care workers
|
267
|
Treatment for common TB (79.8)
|
164
|
Treatment for common TB (90.2)
|
Yes
|
Treatment for MDR-TB (70.4)
|
TB case detection (72.6)
|
No
|
TB management (66.7%)
|
TB management (83.5)
|
Yes
|
Data source: TB health worker survey. |
The content of remote (synchronous) training is too deep to be used in clinical practice, which is beyond our competence, especially for county-level personnel. (County-level TB health professional FGD in Jilin)
Barriers in organizational coordination
The first challenge in organizational coordination in training was the leaders did not pay enough attention to training. Interviews with project management personnel in Zhejiang and Ningxia provinces revealed that leaders paid relatively little attention to the capacity-building subprojects, so they could not mobilize adequate resources they needed. On the contrary, the areas where the project went smoothly with the higher level of participation, such as Jilin, tended to have higher levels of leaders’ attention.
In our case, to be honest, we need to take care of everything. We are extremely busy. What do you think? For example, online training (asynchronous E-learning) is only a very small part of the China-Gates program, so we can't put lots of resources on it, the other works such as information technology have already taken up too much time, haven’t they? And electronic medication monitoring (EMM), all of them are implementing right now. We can’t be anywhere; this is our situation. (Key informant interview of CDC leader in Zhejiang)
The remote (synchronous) training allowed us to hear lectures from national experts, not only in our hospital, but also at the city-level and below. The interaction between the Beijing Chest Hospital and our hospitals, including the tele-rounding, also worked very well, and everyone benefited a great deal from it. It's a great way to strengthen the capacity for our hospital and other TB institutions in Jilin. (Key informant interview of provincial TB designated hospital leader in Jilin)
The second challenge in organizational coordination was how the CDCs and TB designated hospitals collaborate on organizing training, providing feedback, and solving problems. Although China developed the new TB service delivery model - the CDCs, TB designated hospitals and primary care institutions work together in what is known as the "trinity model" - there was no coordinated working mechanism for TB continuing medical education, especially at the primary care level. It hindered further expansion of the high-quality learning resources by E-learning.
Barriers in internet technology
E-learning CME have placed higher demands on hardware and software. According to the survey, there were three kinds of hardware equipment for synchronous E-learning in hospitals of three provinces: the first category was to have professional remote equipment, with a special remote centre as a training room, which existed only in a few provincial hospitals; the second category was accessing the remote platform by computer and duplicating the computer's display on the projector to share video with a group, which works for most city-level hospitals; the third category was using a personal computer in the participants office at home - most county hospitals fall into this category. The latter two forms of training space could hold only 3–5 people. In addition, a lot of technical issues arose when TB health workers received training, such as the slow processing speed of E-learning platform, network jams, system errors in learning progress, fuzzy sound and blurry videos. Technical failures have a significant negative impact on clinicians' participation in training.
Because we're not using a special-purpose device, sometimes the internet connection is not working. It happens sometimes that we could not join the (synchronous E-learning) classroom. (County-level TB health professional FGD in Jilin)
"I've finished my study and the quizzes, but the system still showed that I haven’t done my work. It will show you the answer to the quizzes and you can do it for the second time, but when I finished, it still not working. I didn’t see any notice about this problem. The other problem is that the system could not distinguish the sections that you have learned and the sections you have not. You need to log in and find out for yourself. So, I still don’t get the certificate now." (Provincial TB health professional FGD in Zhejiang)
Another challenge came from the technology itself. Firstly, most medical staff feel that E-learning should not be a substitute for face-to-face training. One of the most obvious barriers is teacher-student communication, which is a great challenge in learning applied professions. Secondly, although they didn’t have to leave their work space for E-learning activities, the synchronous learning sessions still required all participants to join the virtual classroom at the same time. Many medical staff reflected that the training schedule often conflicted with their work schedule. For example, in many county-level hospitals, only one or two doctors were in charge of TB outpatient service, so they could not leave their office to participate in training during the working days.
Barriers in motivations
The lack of motivation was another source of barrier, which is deeply rooted in the health system. The incentive mechanism in current TB health service delivery system has a sustained negative impact on the enthusiasm for learning of TB health workers and the quality of the TB workforce. Firstly, the medical institutions in China largely depend on services revenues to cover operational costs. However, according to the national TB control program, the CDCs, TB dispensaries, and TB designated hospitals are funded by the government to provide free medical services, TB health workers have limited capacity to generate revenues. In this way, they have lower income and lack of work motivation. Moreover, participation in training has no association with the promotion of titles and bonuses, which makes medical personnel more reluctant to take part in training. There was a very common phenomenon called "training professional", which means one person in the department was asked to participate all the training, and signed up for every colleagues. Secondly, due to poor income, the TB workforce in the three provinces were generally older health workers with lower professional titles and less education compared with their colleagues in other departments. Their learning capacities and digital literacy were relatively weak. All these factors may lead to a low acceptance of E-learning among TB health workers, which has a negative effect on the implementation of China-Gates training project.
We are now implementing the new so-called “pay for performance” reform, we are redistributing the resources in our department, as known as the “secondary allocation”. To our general section staff, we get even less income (than before). In this situation, your workload is not associated with your actual (revenue). When you are not satisfied with your revenue, you don't have the motivation to work, you just deal with everything, muddling along, that's all. ... our clinical physicians, especially our heads of the clinical departments, they seem to feel that, managing patients, performance, and getting more money - he feels that that's the point, training is not important. (Key informant interview of project management personnel in Ningxia)
Since TB health personnel were more likely to be “passive learners” instead of “active learners”, the E-learning activities in many institutions still rely on external incentives such as continuing medical education credits. Some project management personnel reflected that since the management of continuing medical education credits has recently become more stringent, it was more difficult for them to add some credits to TB health workers who attended the training. They could only try other strategies such as sending repeated notifications to doctors and forcing them to participants. However, neither of them works well.
Barriers in project sustainability
Many stakeholders expressed their concerns about the sustainability of the training during the interviews. This argument was based on two main reasons: first, the capacity to mobilize resources of the training organizers was very limited. Both national and provincial platforms are facing the tight constrains in terms of funding and human resources, which make it difficult to mobilize resources to continue the training after the program is completed. Second, due to the existing charging rules, the China-Gates program funds did not support equipment purchases or payments to teachers. According to the government regulations, hospitals cannot charge patients for telemedicine consultations either. The training organizers could only issue a small service fee to pay teachers through other funding. In this way, the platform has no existing model for funding to sustain itself.