Satisfaction in Family Health Specialization Courses Offered by the Brazilian Public Health System: a National Assessment

Background: This cross-sectional study investigated student satisfaction in specialization courses in family health produced and offered online by the Brazilian Unied Health System’s Open University, along with associated factors. Methods: Health professionals who were enrolled in these courses between 2013 and 2017 responded to online questionnaires collecting data on sociodemographic aspects, student status, and their perceptions about different facets of the courses measured by a Likert scale. Degree of satisfaction was considered as an outcome, and multiple regression analysis was applied. Results: Of the 3,109 respondents, most had successfully completed their course (98.4%), were female (72.5%), self-declared white (62.1%), Brazilians (69%), ≤ 40 years old (59.1%), nurses (42.1%), or physicians (42.4%). Most respondents viewed the courses positively (59.30%); lower satisfaction score was associated with age (p=<0.0001), education (p=0.0031), and failing the course (p=0.0064). Students reported diculties with the pedagogical aspects of the courses, but gave positive scores to the materials, contribution to professional practice, and stirring interest in the area. Conclusions: The participants were seen to be satised with these courses, although some aspects can be improved.


Introduction
Brazilian public policies include guidelines that motivate and orient managers in training personnel to work in the different public health services. Article 200 of the Brazilian Constitution states that the Uni ed Health System (SUS) must authorize such training [1], and Article 27 of the Organic Law states that this applies to all levels of education, including graduate programs [2]. The National Policy on Permanent Health Education (PNEPS) presents theoretical and methodological concepts to guide training institutions in developing teaching practices that allow workers to critically examine their practices and re ect on intervention strategies where they work [3].
Continuing education in health drives higher education institutions in the SUS Open University System (UNA-SUS), which has become rmly established as a provider of educational opportunities across the entire country. This system comprises 34 higher education institutions, and between March 2008 and October 2015 there were 119,000 health professionals (physicians, nurses, dentists, and other multidisciplinary team professionals) enrolled in specialization and professional quali cation courses. These professionals worked in primary health care in urban and rural areas, indigenous regions, and other di cult-to-reach places in 99% of Brazilian municipalities [4].
Learning opportunities have expanded, and by the end of June 2021 UNA-SUS surpassed 5 million enrollments; 2.9 million of these are health workers in specialization, professional master's, improvement, refresher, and professional quali cation courses. Just the specialization courses had 91.627 professionals from 5.298 municipalities enrolled, in all states and regions of the country [5]. The higher education institutions that comprise the UNA-SUS system not only produce and offer free specialization courses and other large-scale learning opportunities, but also participate in programs to provide the public system with health professionals, thus strengthening the PNEPS. The courses encourage meaningful learning, promoting re ections on everyday situations to improve the teams' work process as well as the services they provide the population. To do so, they incorporate active and problematizing methodologies to develop the knowledge, skills, and attitudes necessary for good professional performance [6].
Because of the importance of improving the teaching/learning strategies used in specialization courses, the higher education institutions in the UNA-SUS system conduct surveys to assess the opinions of the main actors involved. Along these lines, Sturmer et al. [7] evaluated concluding re ections by student/professionals on the courses provided by the Porto Alegre Federal University of Health Sciences in Brazil's North, Northeast, and South regions. These authors found that the pedagogical strategies in these courses, such as virtual cities and ctitious complex cases, promoted re ection among participants and consequently improved their practices.
Similarly, in the state of São Paulo in southeastern Brazil, Sanches Marin et al.
[8] interviewed students who had completed specialization courses in family health offered by the Federal University of São Paulo to health professionals from the 62 municipalities comprising the Marília Regional Health District (DRS IX). The results demonstrated that the course had a positive impact on the processes of care, work, and management; participants stated that the educational opportunity provided theoretical underpinnings to shape therapeutic planning, considering an integrated view of patients' health needs and strengthening the bond between patient and caregiver. It also promoted integration between the multiprofessional team, valuing teamwork as well as information management through the organization of records and other documents.
These ndings suggest that professionals participating in specialization courses in the eld of family health have a positive view of the educational offerings from the UNA-SUS network. Although local studies indicate this hypothesis is correct and re ect improvements in the educational processes within this group of institutions [7,8], research on the national level is still needed to reveal the degree of satisfaction among professionals about the family health specialization courses (known locally as lato sensu courses) offered by the UNA-SUS network, as well as associated factors.

Methods
Study type This is a cross-sectional, quantitative, and inferential study.

Ethical aspects
The project was approved by the Fiocruz Brasília Ethics in Research Committee (CAEE 18046519.2.0000.8027, opinion 4.297.369). All normative and ethical procedures were followed, such as obtaining free and informed consent from participants through a signed agreement and obtaining o cial authorization from UNA-SUS to collect data.

Target public and courses
The target of this study was health professionals enrolled in the family health specialization courses offered by Brazilian higher education institutions, as well as the needs for training and permanent education in the SUS. A total of 17 UNA-SUS higher education institutions offered training at the specialization level in family health for the years covered in this present study: 61.635 registered health professionals were enrolled, mainly located in primary health care in the SUS, distributed across Brazilian municipalities between 2013 and 2017.
Brie y, family health specialization courses were offered remotely online, with a minimum duration of one year (360 hours) and accompanied by one educator per class (average 30 students per class). The courses will be referred to throughout the article as online courses, considering that they utilize web technology and virtual learning environments. A certi cate was provided to students who performed the activities throughout the course and completed the nal project with passing scores (70).
To compose the sample, 46.936 health professionals with email addresses registered in the UNA-SUS system were considered. To calculate the sample, heterogeneous distribution was considered, along with an abandonment rate of 13.82%, absolute error of 2%, and con dence coe cient of 99%, resulting in a total of 3.811.

Data collection and development
An online questionnaire was adapted from an instrument used by UNA-SUS to evaluate student perceptions after completing training [9,10]. Although the questionnaire has not been formally validated, this instrument was developed by experts from the UNA-SUS network.
It contains 15 questions requiring an estimated 15 minutes to complete, and inquires about the participants' sociodemographic status, profession, and course status (successfully completed, failed, withdrawn/canceled). Only participants who successfully completed or failed were able to proceed with multiple-choice questions on a Likert scale which ranged from 0 to 4. The questions were related to the students' experiences during the course: perceptions of course quality for professional practice and career advancement; perceptions of course organization; importance of teacher follow-up for learning and to complete the nal project; how often the participant contacted the teacher; applicability of course content to professional practice in the health unit; contribution of discussion forums to learning; frequency of participation in these forums during the course; volume of content covered in the course; quality of course materials (texts, videos, graphics, images, etc.); adequacy of the proposed assessments; interest stirred in the subjects covered in the course; and level of di culty of the course and assessments.
The survey was administered for four months and transported to an online platform (Survey Monkey). The respondent data were subsequently imported into the R statistical software package [11].
A total of 270.083 emails were sent in seven rounds, with an average interval of two weeks between campaigns. During the rst round, 46.936 e-mails were sent: 5.797 were opened, 658 answered, 117 recipients asked to not receive further e-mails (opted out), and 3.481 were not delivered. In the second round, 43.147 e-mails were sent, 8.357 opened, 796 answered, 80 recipients opted out, and 415 e-mails were not delivered. In the third round, 42.267 e-mails were sent, 7.841 opened, 536 answered, 129 recipients opted out, and 399 e-mails were not delivered. In the fourth round, 41.630 e-mails were sent, 9.873 opened, 557 answered, 168 recipients opted out, and 406 were not delivered. In the fth round, 36.103 e-mails were sent, 6.999 opened, 424 answered, 132 recipients opted out, and 391 e-mails were not delivered. In the sixth round, 38.930 e-mails were sent, 3.872 opened, 166 answered, 66 recipients opted out, and 216 e-mails were not delivered. In the nal round, 21.070 e-mails were sent, 6.691 opened, 341 answered, 160 recipients opted out, and 407 were not delivered. In total, there were 3.478 responses; 105 were duplicates and 264 were from professionals who withdrew from the course and were consequently excluded from the sample of the present study. For analysis purposes, 3.109 responses were considered.
One strategy to boost participation was to create a banner graphic with information about the survey, which was forwarded to the UNA-SUS higher education institutions. They in turn sent the image via WhatsApp to alumni as well as former students who did not complete the course, thus encouraging more responses; this was done between the second and third rounds of emails.
Analysis of the data First, the data was analyzed descriptively. Age was categorized by mean. Next, the sample was dichotomized according to the median degree of satisfaction with the course into more or less satis ed, according to the score on the questionnaire. Frequency analyses for each independent variable were then performed in relation to the outcome (satisfaction). Simple (crude) logistic regression analyses were consequently performed between the outcome and each of the independent variables, along with multiple regression analysis. All variables with p<0.20 in the crude analysis were tested in the multiple regression model, and those with p≤0.05 after adjustments remained in the nal model. The model adjustment was evaluated using the Akaike information criterion (AIC) and -2 Log L (log likelihood). For analysis, the R Development Core Team statistics system (R Foundation for Statistical Computing, Vienna, Austria) was utilized.

Satisfaction with the family health specialization course
Most students expressed more satisfaction with the family health specialization course (59.30%), while 40.7% expressed less satisfaction via lower scores on the questionnaire. Of these less satis ed participants, most were male (41.7%), white (41.8%), less than 40 years old (45.7%), Brazilian (41.7%), from different health areas (45.3%) and failed the specialization course (59.2%) ( Table 1). Furthermore, these students had a lower satisfaction score for the level of di culty of assessments and understanding of the content. Lower satisfaction scores were also assigned for the forums and their contribution to learning, and these participants were less satis ed with teacher follow-up during the nal project, as shown in Figures 1 and 2. Satis ed students (those who successfully completed or failed the course) attributed higher scores for the questionnaire items ( Figure 1). This sample reported greater satisfaction with the course in terms of its contribution to professional practice. Average scores in this group were also higher for quality of materials and stirring interest in the area of family health.
Factors associated with lower satisfaction with the family health specialization course Table 1 presents the parameters estimated by individual and multiple regression analysis, adjusted to describe the in uence of the variables on lower satisfaction among respondents. In the nal model, individuals under 40 years of age (p=<0.0001), physicians (p=0.0035), other health professionals (p=0.0031), and those who failed the course (p=0.0064) were more likely to report lower satisfaction with the family health specialization course.

Discussion
This survey showed that most of the students were satis ed with the quali cation offered by the UNA-SUS network. More satisfaction was reported in terms of the course's contributions to professional practice, the quality of the materials made available, and for having stirred interest in the area of family health.
In general, the graduate course in family health has been considered satisfactory by participants in Brazil. A study conducted in Rio de Janeiro that surveyed 102 graduates of a professional master's course in family health via an online questionnaire in 2014 found that 77% of respondents considered this educational process to have made a positive contribution to their professional practice [12]. This nding is corroborated by an online survey conducted in northeastern Brazil with 225 graduates of a master's program in family health, in which most students assigned high scores to the skills they acquired, emphasizing that graduate studies contributed to their professional activity from the perspective of integrated health care [13]. The lato sensu specialization in family health appears to make the same contribution for professionals in terms of improving practices. According to Sturmer et al. [7], the professionals involved in the UNA-SUS family health specialization courses are recent entrants into the primary care system who pursue this quali cation in order to re ne their work through continuing education.
Considering Brazil's expansive size and the signi cant variations in training among higher-level health professionals despite the existence of National Curriculum Guidelines, online specialization courses make it possible for workers in remote regions where there are few or no face-to-face educational offerings to access professional training opportunities. Distance specialization breaks away from the face-to-face relationship between teachers and students and develops other forms of interaction, generating interactivity with the virtual learning environment and developing new cooperative links between students in the same class [15]. This strengthens the students' role in taking responsibility for their own learning, through autonomy and selfmanagement.
The development of new technologies allows new resources to be incorporated into online courses for effective learning that is signi cant for various types of learners, and is re ected in the student's perception of the course and what they learned. This article describes the UNA-SUS network's specialization courses in family health that were offered to physicians, dental surgeons and nurses from primary health care and reached 61.635 enrollments between 2013 and 2017. The respondent pro le is in line with other educational offerings from the UNA-SUS network, in which most students are female, white, Brazilian, and aged 40 years or less [4,12,17].
Different quality indicators are used to evaluate distance learning courses; one such indicator that is recognized worldwide is student retention [18,19]. Dissatisfaction with the teacher, trouble with internet access, complexity of the activities, di culty assimilating into the culture inherent to distance education, failures in course planning, mistaken student expectations, and inadequate technology or inability to use technology correctly have been indicated as the main reasons for student withdrawal or absences, and direct impact the retention rate [20]. For this reason, student perceptions were veri ed via a questionnaire sent by email using the Survey Monkey Platform.
Students who expressed greater satisfaction with the course also recorded higher scores on the questions related to the level of di culty in assessments and understanding of the content. Among those who were unsatis ed, most were younger than 40 years of age, were physicians, and failed the course. Similarly, physicians, nurses, and dentists from the city of Amparo, São Paulo who participated in a UNA-SUS/UNIFESP family health specialization course reported great satisfaction when asked about the learning process in this teaching modality, notably the ability to share what they learned with their work teams and to share knowledge and experiences with their classmates [21].
A similar study that used the COOLES instrument to evaluate perceptions among graduates about their educational experience found the highest satisfaction scores were given for the course stirring interest in the area, the quality of the materials, and course's relation to professional practice. Furthermore, better perception of course material quality was positively associated with greater interest in relation to the knowledge area among the students [22]. This is important, because simply providing materials in the virtual learning environment does not guarantee learning; they must be stimulating, and develop critical thinking and re ection on the part of the student about what they have observed in the territory where they work [23]. The materials produced for use in online courses should be appropriately scaled in terms of content and language, and use various resources in order to allow students to be autonomous and manage themselves, since they are the main mediators in the learning process [24,25].
A successful learning experience can result in satis ed students, and student learning outcomes are a good predictor of student satisfaction in online education [26].
Using forums in online courses dates back to the original Roman forums that were the center of power where political, economic, religious, and administrative issues were resolved, providing interactions and information among the people. Use of this strategy in online courses is consequently intended to generate debate and exchange experiences, contributing to personal and professional growth and civic development among students and teachers [27]. Discussion forums are asynchronous tools that allow participants to re ect, organize their ideas, and express their viewpoints, and participation is continuously encouraged to foment the collective construction of knowledge [28].
In the present study, among the respondents who successfully completed the course, most expressed less satisfaction with the forums and their potential association with learning, suggesting that improvements are needed when this course is offered again in the future. In a study on the use of forums that applied a questionnaire to 132 teachers of various distance education courses at the Federal University of Rio Grande do Norte (UFRN), 52.6% stated that forums were employed to discuss the subjects addressed in the classes to promote learning through class interaction. A total of 31.6% used forums only to answer questions when they arose, and only 10% used them with speci c, delineated and planned learning objectives [29]. In this same study, 78.9% of the teachers considered forum interactions irregular, demonstrating the need for activities to transform and restructure forums into a tool to achieve learning objectives. The same percentage of teachers considered forums a motivating environment for student participation.
Along similar lines, a survey of thirty-four students in a distance course on scienti c methodology found that forums were not particularly effective for interactive practices and the exchange of information and experiences, presenting only isolated individual contributions from students, with little or (in some cases) no interaction between them [30]. A qualitative case study type survey was conducted with students in a distance course on education because there was little interaction in the forums, with participation occurring notably in the nal days or hours of an activity, and because there were many complaints about lack of learning in the course. To improve student participation and satisfaction, some strategies were implemented, such as developing and disseminating learning objectives related to the forum, recommending previous readings, encouraging participation by a scoring system, requiring minimum participation in the forum, rapid feedback, suitable teacher preparation on the content, teacher assistance and empathy. With these strategies, participation doubled in comparison to the previous time the course was offered. Furthermore, the teachers observed an improvement in post quality and in student involvement and participation [31].
In order for forums to be an effective collaborative learning strategy in online courses, some important points need to be considered. Teacher feedback in response to students should generate new contributions on the topic, creating new challenges for students to continue participating in the forum. For this to happen, teachers require certain technical, pedagogical, and affective skills to generate student engagement [32].
In this study, we observed that nearly 60% of the students who reported less satisfaction were from the group that failed the course. Few studies on student satisfaction about online training are available, particularly those addressing failure. In one previous study comparing in-classroom students with those taking a course online, the students taking the online course were less satis ed, mostly because the use of the technology was new for them and they lacked the discipline and skills needed for distance learning [33]. However, this study was conducted over twenty years ago, when much less was known about the use of technology in educational strategies, there was much less connectivity, usability, and user competence, and the software available was less reliable. Even so, this is still the reality in some Brazilian scenarios. Another study that investigated determining factors for obtaining learning results from the point of view of undergraduate students in South Korea and India found that interaction, student motivation, course structure, and instructor knowledge and facilitation positively in uenced satisfaction with online courses [34].
Although less than 2% of the respondent sample failed the course, this lower satisfaction is expected. This has been previously observed in a study evaluating student perceptions of distance courses offered by the Getulio Vargas Foundation, in which students who failed responded to all items on the questionnaire with lower scores compared to students who passed [35]. In a study including students who passed and failed a lato sensu graduate course offered by the Brazilian National Public Administration Training Program (PNAP) to train public managers within Brazil, 17% of the individuals who did not successfully complete the course indicated that di culties with the course were the reason for evasion. However, in this study, the main reason was the lack of time (38%) [36]. In another study on perceptions of an online logistics course, 58.3% of students responded that distance courses are more di cult than face-to-face courses. This response may be justi ed by the fact that 61.5% of the respondents stated they had already failed a distance class during their program, and 65% of these students reported that a lack of mastery of technological resources was one of the reasons that led to failure in distance learning [37].

Conclusions
This study investigated satisfaction among the professionals participating in the family health specialization courses offered by the UNA-SUS network. Most students were found to be satis ed with the training; those who expressed satisfaction most frequently cited the course's contribution to their professional practice, quality of the materials, and the interest it stirred in the area of family health.
Aspects that contributed to less satisfaction could also be identi ed, such as age, training, and failure. Less satis ed participants expressed di culties with assessment, understanding the content, participating in forums, and less satisfaction with teacher follow-up, aspects related to the pedagogical eld. These aspects can be considered as points for improvement in specialization courses offered by the institutions in this network.
Finally, the data obtained rea rm the importance of seeking out different perceptions, which exposes the challenges of the teaching/learning process and reveals the changes that distance education produces among participants.  Figure 1 Mean distribution of the questions as a function of degree of satisfaction among students who successfully completed the specialization course. Mean distribution of the questions as a function of degree of satisfaction among students who failed the specialization course.