First, we present a short introduction to the regulatory context of the four countries, before presenting the overall findings relating to involvement methods across countries. We then describe the reported benefits and challenges of involvement based on the evaluations, if available, of experiments included in the study.
In Table 1 we provide a brief overview of the regulatory context of Norway, England, the Netherlands, and Australia as a backdrop for this paper. Some countries use the term regulation, while others use supervision or inspection about their role and activity. This is reflected in the context description in Table 1.
Methods of involvement in regulation across countries
The data synthesis identified a wide variety of methods for user involvement in regulatory practice. In the following, we present the four categories of activities within these. A summary of these results per country can be found in Tables 2-5. (To ease readability of Tables 2-5 and the four involvement methods categories, we present this text without references. All references are included when we present the reported benefits and difficulties of involvement identified in research studies or in the published grey literature).
Individual proactive methods of involvement
Individual proactive methods refer to involvement of individuals with the purpose of collaboration and use of information for setting the future regulatory agenda and planning regulatory activities. The results in this category showed that the regulators in Norway, England, and Australia used some kind of patient and user surveys to collect information as part of setting the regulatory agenda and informing inspection activities. National surveys are a way of collecting information about experiences and outcomes of the health services from broad groups of patients and users. We also identified more targeted surveys. In such cases, surveys were designed for specific groups, such as next of kin, and used to inform future system audits and make these more context-specific and relevant for the target group of regulation (e.g., children). Questionnaires were also utilized as part of a regulatory activity to collect information from a broader audience of users in planned inspections.
The regulators also made use of qualitative methods such as individual interviews and meetings with service users. Regulators spoke with children, adolescents, parents, families, next of kin, social and mental healthcare users, disabled users, and frail older service users. We found these methods were part of planned inspections (not initiated due to an adverse event) in Norway, England, and the Netherlands. Regulators collected information based on users’ experiences with the services and used this information to assess if services were provided according to standards and regulation.
The synthesis identified specific efforts among the regulators to involve and get in contact with vulnerable groups with increased risk of not being heard through routine approaches. For example, in some Australian states, surveys are specifically directed at ‘hard to reach’ groups (with low response rates, however). In other countries, hard to reach groups such as young asylum seekers, young people with autism, people with a migrant background, elderly and people with learning disabilities are targeted through qualitative methods. Such methods included the development of digital tools for communication with children under 13, and experimenting with including interpreters in system audits of under aged refugees in child protective service institutions.
Individual reactive methods of involvement
Individual reactive methods of involvement refer to how regulators involve individuals (patient, user, or next of kin), when they have experienced either an adverse event or filed a complaint about the service provision to the regulator. The Netherlands, Australia and Norway have regulation enabling users, or next of kin, to file complaints to the regulator or an ombudsman, or both. This legal right is one way of involvement in itself, but we also sought to identify if, and how, people were involved in the regulatory process after filing a formal complaint. In the included countries, a complaint can relate to both lack of service provision and being denied a service. In addition, complaints can include reporting experiences of adverse events, near misses, or patient harm. We found examples from Norway where the regulator established meeting arenas between regulatory inspectors, service providers, and users or next of kin. The purpose was to gather the involved parties and through a dialogue-based approach, try to solve each complaint case so it did not proceed to a formal and often long-lasting written information exchange process between the parties before reaching a conclusion.
In this category, we also found methods of involvement in the regulatory investigation of adverse events. A regulatory investigation may be initiated based on patient complaints, or by mandatory reporting of adverse events from the service providers to the regulator. In Norway, results showed several examples of individual involvement methods in the investigation process. In the most severe cases (e.g., deaths), investigated by the NBHS at the national level, the regulator always consults with the family, informs them about the progress and collects their views on the event. Moreover, there were examples at the county level in Norway of regulators organizing formal face-to-face meetings between regulatory inspectors and next of kin to collect information about the adverse event leading to patient death from the next of kin’s perspective. In the Netherlands and Australia, we identified regulatory requirements for service providers to conduct investigations of adverse events, but the regulator does not usually investigate these. However, since both countries require service providers to involve patients and the families in the investigations, this implies that one way of prompting user involvement is by regulating and requiring the service providers themselves to involve patients and family in investigations (as described in the introduction of this paper). In addition, Norway has recently made it mandatory for health service providers to invite patients and users to a meeting after a severe adverse event. In addition, in long-term care, the Dutch inspectorate experiments with checking directly with next of kin whether the service provider has involved them in the investigation process.
Collective proactive methods of involvement
In collective proactive methods of involvement, the involvement is not so much focused on a patient’s specific case or treatment, but more generally. Users involved are expected to represent a group of interests. The purpose is to inform the future regulatory agenda or specific inspections. The collective and proactive methods of involvement demonstrated a wide repertoire in all the included countries. England and the Netherlands were at the forefront in utilizing a range of approaches. Methods identified in this category related to involvement of user panels and user organizations, campaigns, expert-by-experience and mystery guests, data sharing, and commissioning of research.
Across countries, the most common method of this type is using experts-by-experience, peer-inspectors (in inspection processes) or co-surveyors (in accreditation processes). These methods consider users, patients and family members to be experts on care, which warrants them being a part of planned inspection activities (or accreditation surveys in Australia). The degree of involvement varied from being involved in an inspection planning meeting, to being part of the inspection team as co-investigator on site and in the analysis of results. Experts-by-experience were reported to be involved in thematic reviews and system audits. We identified variations on the expert-by-experience approach in different fields. Examples included adolescents involved in the inspection team to interview adolescents in thematic inspections, co-designing regulatory frameworks, or involvement of people with learning disabilities and next of kin in an investigation process. Moreover, the synthesis uncovered examples of older people being trained and included in the inspection process in elderly care homes. Similarly, the “mystery guest” method was applied in the Netherlands. Mystery guests are a subset of the experts-by-experience approach, where potential service users visited service providers “undercover” and evaluated aspects of the services given. For example, people with learning disabilities assessed the accessibility of services as mystery guests. Another example of the ways in which mystery guests were used, was considering whether the information they received from the service provider would be easy to understand for the group they represented. Based on these experiences the people with learning disabilities, service providers and inspectors discussed the accessibility of the services, what information they wanted to keep and what could be improved. In this particular example, managers were also later involved. The regulator used this information in the assessment of the service provider.
Our synthesis found that user panels and user advisory groups were common in the regulatory bodies across countries. In England, the CQC has user panels to provide input and advice regarding the regulation of children and mental healthcare services. In Norway, the NBHS recently established a user panel at the national level to inform all regulatory activity. Several Norwegian regional regulatory offices have established user panels. The Dutch inspectorate organizes user panels on specific regulatory topics and is investigating the possibility of a structural user advisory board. Furthermore, the Dutch inspectorate involved users in developing their multi annual policy plan ‘20-23’, and the Youth Care department of this inspectorate has been advised by a Children’s Council in 2019. Similar methods that focus on involvement in committees and panels are human research ethic committees and clinical governance committees which are established in Australia. In England, the CQC commissioned research with users.
The regulators also collaborated and organized seminars with user groups, councils, charities, and organizations to collect information and experiences as part of the planning of inspection activities. Some of these activities were co-produced by the user organizations and the regulatory bodies. We also found examples of organizing co-investigator workshops to collect experiences and lessons learnt from the experts-by-experience who had been involved in some kind of regulatory activity.
In this category, we also identified virtual involvement initiatives and campaigns using social media, digital marketing, and other online platforms to encourage patients and families to share their knowledge and experiences with the service providers. Other initiatives online related to disseminating general information to user groups about the regulators’ role and responsibility. In the Netherlands, the ratings and reviews of an independent patient rating website are used by inspectors to identify risks and themes for theme based regulation and to prioritize their visits. The Dutch inspectorate also searches social media for signals from users about the quality of care in the health services they visit. Furthermore, we identified an English initiative where CQC had established a public online community for involvement in health policy and service design. This included both a representative panel and self-selected groups. The CQC additionally took advantage of a data sharing partnership with other websites collecting intelligence from users that is then used by the regulator.
The results show collective user involvement for proactive purposes in Australia in accreditation requiring healthcare organizations to partner with consumers in planning, designing, measuring, delivering and evaluating care. A final example in this category is the national recommendations for user involvement in regulations in Norway. Results from Norway suggest a large emphasis on user involvement in regulation over the previous four-year period (2014-2018), where the NBHS made a strategic effort to improve user involvement in regulation by different means, such as funding innovation projects to test new ways of approaching user involvement in regulation. As a result of the four-year involvement program, the NBHS developed national recommendations for user involvement in regulation. Lessons learnt and strategic actions were summarized and the recommendations related to methods, experiences, and how regulators can and should involve users in their own regulatory activities and their training of inspectors. The recommendations underline patient and user involvement as a core value for health service provision and for regulatory bodies.
Collective reactive methods of involvement
This category relates to collective involvement after healthcare has failed, in terms of adverse events or complaints, but implies that it is not related to an individual’s own specific case. The collective reactive methods repertoire was limited in all countries. Nevertheless, we note some direct examples whereby the Dutch inspectorate aggregated the information from complaints collected at the National Healthcare Report Centre by sector and by theme. Also, the Dutch inspectorate performed an explorative pilot on text mining the content of the complaints for relevant topics. Similarly, in England, the CQC analyses complaints and concerns from various sources. This information is used as a signal in risk-based regulation for agenda setting and prioritization in both countries.
Reported benefits of involvement
The above synthesis shows the varied nature of user participation in regulation in terms of the type of regulation and the type of involvement. Not all the initiatives mentioned above have been evaluated. Evaluation practices differ between countries. Comparatively, most practices have been evaluated in the Netherlands where the inspectorate works with researchers in an academic collaborative. The practices that have been evaluated do offer important insights into the main reported benefits and challenges of involvement practice.
First, the primary espoused reason for involvement is improving regulatory work, and by consequence improving the quality and safety of care (5, 16, 17, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38). In this view, patients and families are seen as an additional information source. In case of incident investigations for example, patients and families are able to put incidents into a broader perspective and offer a more holistic understanding of what happened (13, 14, 16, 23, 24, 39). The espoused reasons are similar for the input they provide during thematic inspections (17). Similarly, experts-by-experience can provide important knowledge as they are more able to tap into experiences of patients, especially in the case of sensitive subjects such as alcohol misuse amongst adolescents (12, 15, 40, 41). Experts-by-experience and related methods also offer another view on quality of care by focusing more on relations and ‘softer’ aspects of service provision (such as quality of the food, time spent outside, decorative aspects of living facilities) rather than concentrating on safety (15, 42 43, 44, 45). Also, by analyzing reviews written on an independent patient rating website and social media posts, the regulator may identify risks from the patient’s perspective (26, 27, 28, 46, 47, 48, 49).
Second, involvement is said to legitimize decision-making of the regulator by using information gathered from patients and the family in the regulatory assessment, but also by co-producing inspection criteria (5, 6, 45). This relates to the goal of democratic decision-making. For example, when patients and the families support the findings of the regulator this is added to regulatory reports (6, 12, 45). Patient and family involvement is also related to being transparent about the work of the regulator, and connecting with patients or the public more generally improves the image of, or trust in, the regulator (5, 13, 14, 39). This argument is not only used to support the active involvement of patients and families, but also to send information through, for example social media, about the work of the regulator to the public (46, 47).
Third, involvement is a way of achieving justice for those affected. This especially applies to the involvement of patients and families during incident investigations as shown in Norway, the Netherlands and Australia. Here their involvement is a way of regaining trust and restoring the therapeutic relationship, including the opportunity to apologize (13, 14, 23, 39). Moreover, involvement allows for the provision of information to the patient and family, as well as space to share emotions and provide aftercare (13, 14, 16, 23, 24, 31, 39, 50, 51). Also, some reports mentioned that through involvement escalation of issues to legal claims could be avoided (16, 31). The need for justice for those affected by poor care can also be found in relation to other methods. For example, in a case of conducting interviews with the elderly, it was shown that participants appreciated the genuine interest in their perspective and felt it was positive to be heard (52). This was also found in relation to involvement of next of kin in the regulatory investigation of adverse event when patients had died. Next of kin expected to be involved and evaluation showed that involvement could have a therapeutic effect (13). The regulatory inspectors in this case found involvement to be in accordance with overall political expectation (14).
Fourth, involvement can be a way for users to empower themselves and learn new skills (12, 17, 32, 40, 41, 45). For example, experts-by-experience learn a lot about a specific subject (e.g., elderly care, alcohol misuse) and the work of the inspectorate (17, 40, 43).
Reported difficulties concerning participation
Along with the possible benefits reported above, the results also show a number of difficulties experienced by regulators when putting involvement into practice. First, it can prove difficult to incorporate the input of patients and families into the decisions or reports of the regulator (6, 15, 16, 23, 31, 39). This has partly to do with the perceived lack of legitimacy of patient or family input. It was regularly argued that patients lack the necessary knowledge to contribute, and less weight is given to their input than, for instance, the input of professionals (e.g., in case of incident investigations, theme based inspections or experts-by-experience) (6, 12, 13, 14, 16, 23, 26, 42, 39). As described earlier, the main argument for involvement is that patients and families provide additional information. However, in cases of conflicting input (e.g., different views on what an incident is, what should be the focus of the investigation or what should be considered good quality care) the contribution of patients and family appears to be difficult to incorporate (6, 16, 39). As a result, there is a danger that involvement becomes tokenistic (35, 36). A recent example from Dutch research tackles this conflict by making the client perspective the starting point for regulation. Inspectors followed the client’s perspective and judgements throughout the whole inspection process (45). However, possible conflicts are likely to persist, and the question remains “how will inspectors make informed judgements in such cases in order to do justice to the complexities of regulatory practice”?
Second, not all patients or family members want to be involved or are easy to involve. For example, in case of incident investigations it has been reported that participation is too burdensome as the incident can have a large impact on patients and families (16, 23, 24, 39). In other cases, it is difficult to do justice to the diversity of patients, as some groups are more inclined to participate than others. This leads to questions concerning the representativeness of those involved and for whom they can speak (5, 39). This is also an important difficulty attached to professionalization attempts of participants. For example, experts-by-experience are often trained for their task. This may ultimately diminish the value of the authentic perspective that they can provide (12).
Fourth, organizational procedures can stand in the way of involvement. This includes the language used by inspectors, and is also due to certain protocols (6, 16, 23). For example, incident investigations need to happen in a specific timeframe, the deadline of which can be too soon for the patients or family to be able to participate (16, 23, 24,). Also, the regulatory context can prohibit taking the input of patients on board. The case of youth involvement in a thematic inspection on care for children growing up poor by the Dutch inspectorate is a case in point. The interviewed youths stated that they felt their privacy was very important and therefore professionals should not share information about them with each other. In this case part of this conflict was influenced by the regulatory context as policy makers and regulators put much emphasis on sharing information in response to fatal incidents, which were analyzed as resulting from a lack of sharing information (6).
A final difficulty was identified as dealing with the emotions of those involved. This not only applies to the emotions of patients and family members, but of the professionals and regulators themselves, who can also be affected by participation. Emotions can be felt especially keenly during incident investigations (13, 14, 16, 39). For the next of kin who have lost a close relative, it can be a considerable mental strain to be part of the entire investigation process when they are grieving and sometimes traumatized. The investigation process can repeatedly remind them of the event leading to the death. However, the interpersonal skills of the inspectors may help to reduce this emotional burden. From the inspectors’ point of view, it can be emotionally challenging to involve next of kin in investigation meetings, because the inspectors can worry about attending beforehand and might continue to dwell on things after the meeting, especially in severe cases. Some inspectors have a legal professional background where they were not trained for face-to-face meetings with people in grief, as inspectors with a healthcare background may have been. At the same time, inspectors report that it is a positive experience to offer support to the next of kin and clarify any misunderstandings or questions (13, 14, 39). In other cases emotions can play a role as, for example, a project involving mystery guests, where civil servants providing services to people with learning disabilities reported feeling left out as their perspective was not sufficiently taken into account (45).