se-atlas: Mapping of Health Care Providers for People With Rare Diseases - a Cross-sectional Survey From the User Perspective

Background In rare diseases, only a low number of regionally distributed experts are available in medical care. The health service platform for rare diseases (www.se-atlas.de) provides a search for health care providers and patient organisations in Germany for specic rare diseases and presents the results to patients or physicians. The objective of this study was to examine the background and purpose, user satisfaction and usability when using se-atlas and to receive suggestions on improvements for implementation in the next release of se-atlas. Methods We conducted an online survey over a total period of ve weeks between December 2020 and January 2021. Participants were members of patient organisations of rare diseases and experts of rare diseases centres in Germany. The questionnaire addressed the objectives of this study in 10 questions. We used Likert scales (4 to 6 points), as well as the System Usability Scale to measure usability (range: 0 to 100). The data obtained from the survey was analysed descriptively.


Background
Rare diseases (RDs) are diseases that affect only a small proportion of the population. In the European Union (EU), a disease is de ned as rare if it affects less than one in 2.000 people. This corresponds to about 0.05 % of the population. These diseases often lead to a chronic manifestation and considerable restriction of quality of life [1][2][3]. In the literature, different causes for RDs are discussed. Most of the disorders are of genetic origin (71.9 %), resulting in frequent affected children (69.9 %) [4]. However, other causes like infectious, immunological or environmental factors are observed [5][6][7].
Besides the low prevalence of RDs, specialised experts are also rare and regionally distributed. Thus, affected persons are depended on health care providers (HCPs), like university hospitals or medical-centres, that are often far away from their own residence. In addition, many affected persons and physicians have insu cient information about treatment options or specialised care facilities [8,9]. Therefore, the platform "se-atlas" was developed for the purpose of combining existing collections of HCPs that diagnose and treat RDs and present them in a user-friendly and transparent way. The project was part of the German National Action League for People with Rare Diseases (NAMSE) within the national strategy for RDs in Germany. NAMSE was funded by the German Federal Ministry of Health and included 52 different projects [8,10].
Since se-atlas was published in February 2015, patients with RDs and their relatives, physicians, non-medical staff and people with a general interest are able to get an overview of HCPs and patient organisations regarding to RDs in Germany on the website www.se-atlas.de. For instance, a person affected by a RD can enter the name of the disease in a exible search and subsequently receive all results on existing HCPs, including specialized rare diseases centres (RDCs) and patient organisations in Germany. Furthermore, se-atlas displays information on research networks such as the European Reference Networks (ERNs) and medical societies [8,11]. The search for HCPs and patient organisations is the central function of se-atlas. The search results for a disease are displayed geographically on a map as well as in a list. By clicking on a HCP, the user is derived to its detailed presentation, which provides a description of the HCP, as well as further contact information like address or consultation hours [8]. Furthermore, seatlas supports users in their individual research of RDs.
The information about all institutions that are provided in se-atlas are collected in different ways. Generally, information is integrated from various existing data sources. For example, medical societies or patient organisations can recommend a HCP who is an expert for certain RDs. In addition, experts can also enter themselves into se-atlas. In any case, the data is only released after a comprehensive review by the se-atlas editorial team, which has a medical background and is responsible for the quality assurance of the data. Entries will only be releases after the editorial team has validated the entries with other source [8,11].
However, not only the data quality is an important factor for the success of health information websites like seatlas. User interfaces and the presentation of the respective information contributes to the long-term success of such a website. Outdated user interface designs and poor presentation can decrease the success and the website is not used or recommended to others [12,13]. To describe the effectiveness, e ciency and user satisfaction of a software-system, the term usability is often used. To increase usability, it is necessary to understand the background and purpose of the user, whether the user receives the needed information and satisfaction rate with the system [12]. Furthermore, the participation of the user during the implementation is important to develop a meaningful and useful system [14]. However, there are only few studies available that assess the usability of health information websites [15][16][17].
Since se-atlas was launched 6 years ago, it was visited more than 300.000 times. Therefore, it is necessary and essential to continuously develop the system and to involve users in this process. We performed an evaluation study of se-atlas to get feedback and requirements for further development of the website. The objective of this study was to investigate the background and purpose, as well as user satisfaction when using se-atlas. Furthermore, we evaluated how high the users assess the usability of se-atlas and derived recommendations for optimization. Finally, concrete implementation changes for the new release of se-atlas were created and implemented.

Design
To address our objectives, we performed a cross-sectional survey. This quantitative design was chosen to gather relevant information, opinions and needs of all stakeholders. We applied the STROBE reporting guideline (Strengthening the Reporting of Observational Studies in Epidemiology) to report this study [18]. We considered 20 of 22 STROBE items. A checklist is provided in additional le 1.

Setting and sampling
The target group was selected according to their representation of the se-atlas user group: (a) experts, working in RDCs, and (b) members of patient organisations, which includes patients and their relatives. In Germany, RDCs are part of medical centres (e.g. university hospitals) and responsible for diagnosis, treatment and research for RDs [10].
According to Meuser and Nagel, "experts" have a special knowledge and experience in their working or research area [19]. We considered the following characteristics: Participants are working in a RDC. They have a completed medical degree, as well as a specialist training in human medicine. Based on these characteristics, 33 potential study participants were identi ed, since 33 RDCs are established in Germany [10].
The group of members of patient organisations were recruited with the help of the German National Alliance for Chronic Rare Diseases (ACHSE e.V.), which is the head organisation of all patient organisations for RDs in Germany [20].
The survey was conducted with a questionnaire using the online survey application LimeSurvey over a total period of ve weeks between December 2020 and January 2021 [21]. For recruitment, the experts group was contacted by e-mail. If there was no reply to the e-mail or the survey was not successfully lled within two weeks, the participants were contacted again. Members of patient organisations were invited by ACHSE via e-mail two times. The members were asked to complete the survey, if they have already used se-atlas in the past. A study information letter was sent along to all invited study participants within the e-mail invitation. Before the survey was distributed to the participants, a short internal pre-test was performed, followed by a minimal adaption of the questionnaire.

Instrument
For the evaluation of se-atlas, we developed a self-designed questionnaire, which consists of four parts, regarding to our research questions: background and purpose, user satisfaction, usability and user recommendations. To answer these questions, we created 10 questions as open-ended, single and multiple choice questions, as well as using the Likert scale from 4 to 6 points. To avoid the bias in the questionnaire, recommendations by Choi and Pak were applied, which include e.g. how to deal with problems in wordings or questionnaire structure [22]. The whole questionnaire is shown in additional le 2.
The questions 1 to 4 of the questionnaire belong to the research question "background and purpose", questions 5, 7 and 8 addresses "user satisfaction", whereas question 6 assesses the usability of se-atlas. Furthermore, the questions 9 to 10 were used to derive concrete suggestions to improve se-atlas.
In question 6, the System Usability Scale (SUS), a common questionnaire for assessing the usability of a software system, was used to measure the usability of se-atlas. The SUS includes 10 items, with a 5-point Likert scale (from "1 = strongly disagree" to "5 = strongly agree") [23].

Data analysis and processing
Data collected from the survey were summarized using common descriptive statistics, including frequency, percentage of response, as well as mean, standard deviation (SD) and range, were applicable. We analysed the results for all study participants and separate for both user groups.
For the analysis of the SUS of question 6, we used the approach of Bangor et al., which speci es a range from "0" to "100". This range should not be interpreted as a percentage value and must be normalised. According to Bangor et al., different methods for normalisation are available. We used the following normalization range to interpret how well the usability of the system is to be rated [23]:

Results
In total, 55 participants successfully completed the survey. We contacted 33 experts, whereas 16 responded to our invitation and participated in the study. Furthermore, 39 members of patient organisations participated in the survey.

Results of the questionnaire
In the following section, we present the main results of the survey. The complete and detailed results for all groups are available in the appendix (additional le 3).

Background and purpose
The results of the rst part of the questionnaire regarding "background and purpose" are presented in this section, showing question 1 and 2 in Table 1 and question 3 and 4 in Table 2.
The results of question 1 demonstrate that 34.38 % of participants use se-atlas because of a recommendation from a patient organisation or through a referral via NAMSE (26.56%). In the experts group, most participants became aware of se-atlas via NAMSE (41.18%) or via recommendation by medical staff (29.41 %). Furthermore, in the patient organisation group, the majority of the study participants know se-atlas via one of the patient organisations (46.81 %) or NAMSE (21.28 %).
The results of question 2 reveal that the majority of participants search on HCPs for a speci c disease (24.80 %) or on information on RDCs (24.00 %) as shown in Table 1. Furthermore, the participants were searching on information on a speci c HCP (16.80 %). The participants of the experts group indicated that they mainly search on information on RDCs (33.33 %), HCPs for a speci c RD (27.78 %) and information on a speci c HCP (22.22 %). The group of members of patient organisations often use of the functionality to search for HCPs for a speci c RD (24.14 %), information on RDCs (20.69 %) and information on a speci c rare disease (18.39 %). The information on other recommendation channels are available in Table 1. Regarding the regular use of se-atlas (question 3), the participants indicated that they use the website rarely (36.36 %) or several times a year (25.45 %), see Table 2. Slightly more than a quarter of the participants (29.09 %) reported using the website once a month or once a week. Concerning the experts group, it can be observed that many participants use the website weekly (43.75 %), whereas all results were in a range between several times a week and several times a year. The members of patient organisations mainly use the system rarely (51.28 %) with a range between several times a week and rarely, whereas only one participant use se-atlas weekly. The

User satisfaction
The results of the second part of the questionnaire regarding to user satisfaction and usability are presented in this section. Details are shown in additional le 3.
Overall show that all se-atlas functionalities are considered as important with an average between 3.11 and 3.75. The function "Search for health care providers of a speci c rare disease" was rated as highest (mean = 3.75, SD = 0.48), whereas the function "General search in the eld of rare diseases" was rated as lowest (mean = 3.11, SD = 0.85). Figure 1 shows the combined results based on the mean values of question 7 and 8.

Usability
In question 6 of the questionnaire, the study participants were able to rate the usability of se-atlas. Overall a score of "67.1" could be achieved which corresponds to the rating "OK", according to Bangor et al [23]. The experts group rates the usability of se-atlas with 76.7, which results in the "Good" range. The members of patient organisations rated the usability with 63.1 ("OK").

User recommendations
The results in the third part of the questionnaire show that 40 participants (72.73 %) stated that all functionalities they needed for their visit to se-atlas were available, while the remaining participants denied the statement (27.27 %) (question 9). These participants gave recommendations for improvement of se-atlas (question 10). To illustrate these recommendations, we created the categories "Usability recommendations", "Functionality recommendations" and "Content recommendations" and summarized the responses as shown below. All responses are available in German in additional le 3.

Usability recommendations
Some study participants stated se-atlas should be more user-friendly and the website should have a better overall structure. Furthermore, other participants indicated that they could not nd individual functions (e.g. creation and link of patient organisations).

Functionality recommendations
The study participants made suggestions regarding the functionality of se-atlas, which for example refer to creating of patient organisations in the system. One study participant suggested integrating a comment function to share his opinion on whether or not a HCP has the expertise for a particular RD. In addition, more information on the diseases should be displayed.

Content recommendations
Study participants indicated that they would like to see further increases in entries of diseases as well as regular updates. One study participant also stated that almost all diseases in a speci c disease group (vascular malformations) cannot be found. The patient organisations would like to identify HCPs who no longer have competence for an RD.

Results of the implementation
In this section, we show examples of concrete implementation changes for the new release of se-atlas, which were implemented after analysing and prioritizing the results of the study.
As the search again proved to be one of the central functions of se-atlas (highest relevance with a mean of 3.75) and almost 50 % of the respondents cited the search and information on RDCs as the main reason for their visit, it was further optimized and its speed improved (see Fig. 2). One point of criticism from the survey participants was the lack of diseases and outdated data. Improvements have been made on the technical side, e.g. the latest version of the Orphanet nomenclature and classi cation, which is the largest and complete classi cation of rare diseases, is now used to identify the search results of the diseases [24].
To simplify access se-atlas data, have a clear and simple overall structure and to improve the usability, the design and the overall structure of se-atlas were redesigned. These wishes were also mentioned in the users' recommendations. The starting page was redesigned and the search function has been placed in the foreground (Fig. 2). In addition, the colour design of the website was changed with the aim of improving user-friendliness and the recommendation to improve the basic structure of se-atlas and its usability. The starting page now also includes an integrated slider-menu which leads directly to the central functions of se-atlas, such as creating of HCPs or patient organisations (high relevance: 3.67), as shown in Fig. 2.
Furthermore, to ensure quick access to the overview of RDCs (Fig. 3), a direct access to this subpage has been implemented in the slider-menu (see Fig. 2). Furthermore, this overview of RDCs was new designed and implemented (Fig. 3), since it was one of the main reasons for visits according to our survey (24.0 %).
Since the survey showed generally su cient usability, se-atlas was also further developed with a focus on increasing mobile use in the future. On the technical side, the editorial page was adapted with the new design.

Discussion
This quantitative study investigated with a survey the background and purpose, as well as user satisfaction when using se-atlas. Furthermore, it was assessed how high the users rate the usability. We derived recommendations for optimizing the website and implemented them in a new version of se-atlas.

Background and purpose
The results demonstrate the majority of the study participants became aware of se-atlas through recommendation by a patient organisation, by NAMSE or through members of the medical care team. There was only a smaller proportion of participants who know se-atlas through other channels. The participants' groups in this study differ in their backgrounds, which is re ected in the results. For instance, the recommendation of se-atlas by a patient organisation was thus more strongly represented among the members of a patient organisation. This can be caused due to that patients are active members in patient organisations and these actively try to promote se-atlas among their members [25]. In the experts group, se-atlas was passed on in the professional environment of the members of the medical care team. Furthermore, the dissemination of se-atlas was part of the NAMSE action plan [26], which contributes to the representation of NAMSE in the top three answers for both groups of participants in the survey.
Another difference in the groups of participants can be determined with regard to the question of what information the participants are searching for on the website. Both groups predominantly use the central function of se-atlas in the context of searching for HCPs for a speci c RD. Likewise, information about RDCs is an important aspect for both groups. However, the experts group is less likely to use the information on a speci c disease and the general research options on RDs. In the group of members of patient organisations, these functions are used more frequently. This could be caused due to the experts use se-atlas mainly in the context of patient care, for example to refer patients to other HCPs. This assumption is also available in the results of the question about the frequency of use of se-atlas. The experts group predominantly indicated weekly or monthly use, while the members of patient organisations uses se-atlas rather rarely or only several times a year.
The results on use of se-atlas show that the system is mainly used via laptop or desktop computers and only a smaller proportion of the study participants indicated mobile use (18.17 %), especially the group of members of patient organisations. However, this result is not in line with the increasing trend of mobile device use on the internet and does not decrease the relevance of mobile use [27][28][29].
User satisfaction Most of the study participants are quite satis ed with the system. In the set of questions on satisfaction and relevance of speci c functionalities (questions 7 and 8), the results show of a satisfaction with a mean value of 4.01 (SD: 0.07). Furthermore, the relevance of all functionalities of se-atlas is high (mean: 3.30, SD: 0.11). The central function of searching for facilities for a speci c rare disease seems to be important for the users, from which it can be assumed that se-atlas ful ls its basic function. The members of the patient organisations show a willingness to actively participate in se-atlas through their high rating of the relevance of creating entries for HCPs and patient organisations. In summary, these results are similar to question 5, where users were able to rank their overall satisfaction to the system. On average, users were quite satis ed with se-atlas (mean: 4.30, SD: 1.18).

Usability
With an SUS score of "67.1" according to Bangor et al [23], the overall group of participants achieved a usability rating of "OK". Brooke et al. investigated in a study 500 usability studies of different software-systems and calculated an average SUS score of 68 across all systems (at or around the 50th percentile) which is close to our results. Other patient information websites achieved a score of 72.5 [30] and 92.5 [31]. Furthermore, in another patient portal, the authors noted user-depended results in different user groups with an SUS from 63 to 81 [32]. This is similar to our results where the experts group rated the usability of se-atlas at 76.7 ("good") and patient organisations with 63.1 ("OK"). Tieu et al speci es that a lower SUS for a subgroup of participants with limited health literacy is associated with a greater burden of barriers to use [32]. This may also be the case here, but cannot be reconstructed on the available evidence.

User recommendations and implementation
In the last part of the study, further elds of action for the improvement of se-atlas were gathered by the users.
Some users gave recommendations on usability, other users made suggestions to integrate various new functions. For instance, a need to provide more information on individual diseases was stated. Furthermore, patient organisations noted to identify HCPs who no longer have competence for an RD or to make suggestions for changes. However, this requirement must be linked to a further validation process by the se-atlas editorial team.
Unfortunately, not all comments and results could be implemented in the new version of se-atlas. These are planned for future releases. Since the design of se-atlas was implemented completely new, further studies need to evaluate whether the usability was improved.

Discussion of methods
In this study, we have chosen a quantitative design with a questionnaire since it is easy to implement and allows us to primarily assess usability weaknesses, user satisfaction and suggestions to improvements for se-atlas. In the future, qualitative studies like Thinking-Aloud-Tests (TA-Test) could be performed, which allow to get opinions of the users by recording their thoughts while they interact with the software [33,34]. The advantage of such a qualitative study is a comprehensive insight into the behaviour of the test persons with a software-system. Response and bias tendencies are largely minimized here. However, the transferability of the results to the real usage situation must always be investigated, e.g. through combination with standardized questionnaire interviews. In addition, the effort required for qualitative studies is signi cantly higher [35]. In a further step, other approaches like the Technology Acceptance Model (TAM) could be used. These model is less appropriate to investigate concrete re-design suggestions, but allows to identify implementation barriers [36].

Limitations
The study has several limitations. Due to the design of the study, not all user could be reached. Therefore, the generalisability is limited. In addition, the two groups of study participants were of different sizes. Therefore, the results can only be compared to a limited extent. Furthermore, in the group of members of patient organisations, unfortunately not all organisations could be reached. The group of patients and those affected, who are not active in patient organisations and often only visit se-atlas once, were di cult to recruit. To reach these users, a quantitative survey directly at the se-atlas website could be considered. However, it is di cult to assess whether the desired user group is reached, as every website visitor could participate. Likewise, a survey on the website itself, rather raises the rst impression instead of an overall impression. According to Thielsch et al., website users are attracted by high aesthetics in web design and are more strongly in uenced by perceived aesthetics and usability in the rst impression [37].

Conclusion
This quantitative study involved experts and members of patient organisations of RDs to assess their background, purpose and user satisfaction when using se atlas. Furthermore, the usability of se-atlas was assessed and we derived improvements suggested by the user to implement a new version.
The study is as a snapshot and representation of the need for further development of se-atlas from the perspective of two speci c user groups. The results show that the participant groups in this study differ in their backgrounds and show different user behaviour and purpose of use. Most of the study participants are quite satis ed with the system and its functionalities. This is also re ected in the usability rating, which is according to the SUS, between "OK" and "Good". Furthermore, initial indications and concrete recommendations for se-atlas were expressed in this study, for instance to constantly increase the entries of diseases.
Further studies and considerations should address the continuous optimisation process of se-atlas and the validation of the data to further increase the satisfaction of all user groups and to maintain user loyalty. A further evaluation based on this survey and its adaptations implemented in se-atlas will be targeted.  New starting page of se-atlas.de with slider-menu