Twenty-five users were included in this study, 52% male and 48% female, aged between 21 and 69 years. Sixty percent lived alone and the others lived with a partner or parents. Two users had never completed their education and ten had a low educational level. Twenty users were Dutch and five had other nationalities. Two users were from the nursing and care of elderly sector (two relatives), ten from the care of people with disabilities sector (one relative and nine clients), three from the mental healthcare sector (all clients), seven from the social care sector (all clients), and three from more than one sector (two healthcare professionals and one client).
The qualitative results are presented by the three phases consisting of an overview of the main findings in each category as defined during the analysis (Table 1). A description of the adjustments made after each phase is given after the findings.
Table 1. The categories defined in the three phases of the study, obtained from data analysis.
Categories of content analysis
Look and feel
Phase I – Look and Feel
Visualization consisted of colours, images/icons, and shapes. The colours were received very positively, but extra colours were requested, especially on the homepage.
Client 5: “Looking at this colour here, the blue is very calming. You are not visually overwhelmed, which makes the page nice to look at and people will want to continue to read what it says.”
The development team discussed the colours of the proposition pages in further detail. Every category of the tool was a different colour on the main page, but these colours changed on the proposition pages. The colour composition of each category had to become recognizable throughout the tool. Initially, the ‘finances’ category was green on the main page, but purple on the other pages.
At the meeting with the development team, the images were considered to be too specific for the elderly. In the interviews all users made similar comments; they thought the images should be more neutral and applicable to all end-users. (see Figure 2). The image of the doctor (on the left) was rejected as this tool was not for use in hospital settings. The tool was developed to support clients with their preferences construction in any care sector, therefore, a neutral persona was chosen. The man (image in the middle) was rejected because most clients could not identify with this man. Clients chose a bird (image on the right) as a neutral symbol, and some users compared this bird with freedom.
Professional 1: “Considering the broad target audience, including older people, the elderly, children with a disability, anyone who can use a computer can use this. Then, the bird is a sympathetic symbol with which almost no-one will have negative associations. If you were to choose a person as a symbol, then it would always be the wrong person. A woman would be wrong because it was a woman, and a man would be wrong because it was a man. And if you were to give the woman or the man a colour, then this would be the wrong colour, especially nowadays.”
Client 2: “I associate a bird with freedom, the freedom to choose. I can identify with that thought.”
Professional 1: “Yes, neutral, freedom to choose, those feelings.”
The users were motivated to discuss their perceived meaning of icons. Their thoughts were compared to the actual meaning of the icon. The icon for the category ‘activities’ was a calendar at first, but users associated this with time rather than activities, and some thought that it looked like an apartment building. In addition, the order of the icons appeared to be important, the first icon in the row belonged to the category ‘finances’, which was a euro sign, and gave one user the idea she had to pay to be able to use the tool.
Client 4: “The icon with the euro sign gives me the impression I have to pay for this tool. I would change or replace this icon, because it is possible that I would not understand it and therefore not continue using this tool.”
Figure 2. Development of the images used on the main page of the tool. On the left, the figure started as a person that looked like a doctor. The tool was not developed for doctors or for a hospital setting. Therefore, the suggestion was to change the image into a ‘normal’ person. The ‘older’ man in the middle figure was rejected because most clients could not identify themselves with this man. The last figure (on the right) was a bird. This is a more neutral image and was seen as a symbol of freedom.
The sub-theme first impressions involved the feeling and accessibility of the tool. The members of the development team felt comfortable, and agreed that the tool seemed accessible, friendly, and easy.
Client 1: “I really like the lay-out of the website, its colours and the composition of the bird, it’s fantastic actually! It stimulates me continue to fill it in.”
The feelings of other users were comparable. More negative feelings were expressed about the time indicator at the start of a category. This was intended to inform people about the potential time investment necessary to complete a category; the users agreed that it would induce time pressure instead of putting the user at ease.
The sub-theme layout comprised composition, font size, and form. The circular shape and the composition of the buttons to access different categories were considered pleasant and easy to use. However, the button to access the overview had to be separated from the circle because it was not a category where the users could answer propositions. The users considered the layout of the proposition pages well-ordered, and it was clear where to look.
Client 8: “The page looks very well ordered. I love anything that is well organized. For me personally it is important that it is pleasant to work with and to look at. However, it is not just me. I think everyone would appreciate some organization in order not to become disoriented. We are disoriented enough to start with as we have already lost our homes, and if this tool was also chaotic we would probably say, ‘Sorry but we are not going to use this.’”
They thought the understanding of the answer options for the 7-point Likert scale could be improved by numbering the answer options. They suggested the addition of the answer ‘not applicable’ would be beneficial and should be added to every proposition. The composition of the overview page caused difficulties in understanding where to focus due to the positioning of text blocks. Although the font size was not commented on by most users, some requested a button that could be used to increase the font size.
Adjustments to version I
The adjustments made to this version were based on the findings as presented above, and agreement of the development team. More colours were added to the homepage, and the colour composition of each category was consistently used throughout the tool. The images were changed to more neutral images (Figure 2). The icons were changed in accordance with the suggestions of users to ensure comprehensibility and identification. Concerning, the positions of the icons, moving the euro symbol to the middle did not give the impression that the tool had to be paid for. The time indication at the start of a category was removed, the option ‘not applicable’ was added to every proposition, and the answer options were numbered. Although a button to increase font size was suggested, this was not included in the adjustments as this function is already available on digital devices. On the overview page, the configuration was changed to increase user-friendliness, and the button to access the overview became distinct from the circle (Table 2).
Phase II – Navigation
Logic covered ordering of pages, the clicks, and switching between pages. The development team and the other users thought that navigation was easy and the ordering of the pages was logical. When questioned, the users were positive about these aspects. Some navigational difficulties occurred, especially if users continued without reading a page or wanted to return to the homepage. The home button was a bird, similar to the bird that assists the users to answer propositions, but a house symbol as the home button was thought to be more familiar.
Client 10: “It is very clear where to click, in this circle with the five categories. Also the bird, which appears and tells you how to continue. When you continue, this bar shows you the proposition you are at, and how many propositions will follow, and that is very convenient. As extra confirmation,, the category you are answering is written on top of the page. I thought this was very good.”
Overall, users understood where to click, however, on the homepage it was not clearly visible that there was extra information about the tool below the start button. In addition, the information on the homepage was thought incomplete. Information about the use of the tool by informal caregivers was missing.
This sub-theme included the accomplishment of tasks, comprehensibility of the navigational tools, whether users understood how to use the tool. The task was answering propositions from one or more categories.
Client 12: “ I think the propositions are very good - very easy -, I do not have to think hard about how to answer them. It is clear what I need to do and I can answer immediately.”
The observations made clear that the users were able to complete the task without assistance. Navigation through the propositions and answering was easy. However, users found the assisting text blocks too long. The development team also considered these assisting text blocks as problematic for some clients, and a read-aloud function was recommended.
Professional 2: “What strikes me is, I am stumbling over the word ‘articulate’. Taking the clients who will be using this tool into account, some words are too difficult. Will everyone understand what is meant? For example, this page ‘assists in articulating what matters most to you’. This sentence seems complicated and could also be shortened.”
Researcher: “Could you give us any tips?”
Professional 2: “Yes, I am on to it. I am writing some options and will come back to you in a moment.”
Client 1: “Maybe you could replace ‘articulating’ with ‘telling’? That would work better.”
Adjustments to version II
The text ‘return to home’ was added next to the bird to identify the home button, and on the homepage, a ‘more information’ button was added to show that further information was available. This section was updated with information for informal caregivers. Reviewing of the large text blocks was necessary, sentences were removed or shortened, and spaces were added in between lines (Figure 3). In order to make texts accessible to users with lower reading capacities, a read-aloud function was included (Table 2).
Figure 3. Development of the text block on the category page of the tool. Based on comments made by the end-users, the text on the right is shortened. Shorter sentences are made, each sentence starts on a new line, and more spaces are inserted.
Phase III – Content
The sub-themes of comprehension were language, texts, and category names. Different from the navigation phase, this sub-theme concerns the comprehension of all textual aspects, whether users understood the propositions and what their answers mean. Users found the meaning of the categories to be clear; the propositions were received well and regarded as being understandable. However, although understandable, the language used in the propositions was often difficult. It was suggested that combining similar propositions and adding example answers would improve the comprehensibility.
Professional 2: “Considering the clients, we need to improve the content. For example, not only the language, but also order of the sentence. A sentence should be formulated as simply as possible, without sub clauses or auxiliary verbs, and everything aimed at the user. I think that is the way that most clients would understand.”
Validity included missing or irrelevant propositions. Users were asked to formulate propositions they thought were missing, such as children, living with like-minded people, and well-being.
Client 20: “A personal extra remark relating to the content and the propositions. What helps someone to become happy? There are no actual propositions on well-being. I do not know how to formulate this in a proposition but I’ll try, maybe something like ‘I want to do what makes me happy.’”
When the users answered the propositions, they were asked which they considered irrelevant. The development team was critical of the category ‘finances’. They thought it too confrontational to ask whether someone could afford to spend more money on care. Propositions potentially missing from this category were also discussed. They suggested that the assistance clients need with finances and administration should be included. Other users had a different stance towards the category ‘finances’, they thought that the propositions were useful and suggested adding more propositions, such as assistance with debts.
Practicality concerned the usefulness and possible use of the tool in practice. The members of the development team were enthusiastic when they saw that their suggestions had been applied in new versions. They agreed that the tool had improved a lot and that it could become a useful aid in conversations about long-term care. To increase the use of the tool in practice, information about people and organizations to contact for assistance with the search for long-term care would be beneficial.
Adjustments to version III
In improving, the focus was on easing the language and comprehensibility of propositions, by combining similar propositions and introducing the propositions in a logical order whereby similar topics follow each other. To ease reading, the questioning style of all propositions was unified, and to clarify the purpose of a proposition illustrative example answers were added under an information button. The category ‘finances’ was completely revised. Several proposition topics were added such as debts, children, and well-being. Extra information about organizations users could potentially contact for assistance in long-term care was added to the homepage. The last adjustments were implemented and the finalized tool was developed (Table 2).
Table 2. Summary of adjustments.
Phase I (Look and Feel)
- Change all pictures of the tool into more neutral pictures.
- More colour requested in the starting page (top picture).
- The composition of colours more into one line over all pages of the tool.
- Remove the time indication at the start of a category.
- Add the option ‘not applicable’ to all propositions.
- The button towards the overview needs to become more visible.
- The configuration of the messages on the overview page need to change in order to make the messages more logical.
Phase II (Navigation)
- Visualization is necessary in order to scroll down on the homepage.
- Make the accompanying texts shorter and more personal for the user.
- The category money needs a revision:
- What is the purpose of this category? More information is required.
- New propositions such as “I want to know the costs for my care.” and “I need assistance with my administrative tasks.”
- Clients ask for a read-aloud function.
- The homepage should make clear how people who assist clients during the search should use the tool.
- If someone finished one category, the explanation of the tool should change and include a suggestion to continue.
Phase III (Content)
- The language used should become as easy as possible.
- More examples of answer are useful.
- Similar propositions should be combined.
- Ordering needs more logic.
- Some aspects were missing:
- Happiness and well-being “I need assistance to be happy.”
- Children “When you have children: my children matter to me.”
- Debts “Debts influence my life.”
- Current living environment “I am happy at my own home.”
- More information how to contact people who can assist with the search for long-term care (Independent Care Coordinators).