From Measuring User Satisfaction to Improving Quality of Care: Community Verifications and Performance-Based Financing in District Hospitals in Mali

doi:10.21203/rs.3.rs-1561279/v1

PDF

Research Article

From Measuring User Satisfaction to Improving Quality of Care: Community Verifications and Performance-Based Financing in District Hospitals in Mali

https://doi.org/10.21203/rs.3.rs-1561279/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted 15 Apr, 2022

You are reading this latest preprint version

Context: User satisfaction in health facilities is increasingly taken into account to improve the quality of care. In Mali, performance-based financing (PBF) aims to improve the quality of care by increasing the motivation, responsibility, and accountability of health workers. Community verification (CV) is one of the main functions of the PBF. CV makes it possible to monitor the effectiveness of the services reported by the health facilities and to conduct satisfaction surveys of the users who have received care. This paper aims to analyze the CV process during the implementation of the PBF in the Koulikoro region of Mali.

Methods: We adopted a qualitative approach based on a multiple case study design corresponding to the district hospitals (DH) and stakeholder groups. We conducted thirty-nine semi-structured interviews with investigators, patients, and health workers in three of the ten DHs in the Koulikoro region. We designed three interview guides and inductively analyzed our data.

Results: The CV process was well perceived and welcomed. However, the vast majority of interviewers reported difficulties locating some patients in the community for several reasons: incorrect spelling of names, illegible cards, lack of telephone contact information and incomplete information on the cards, and cases of homonymity. All of this was due to poor record-keeping at the health facilities. Some investigators fraudulently filled out forms without investigating users in the communities. They justified this by the lack of time available to them to conduct the CV. No feedback on the results of the CV was possible from the health workers.

Conclusion: The CV is an essential component of PBF implementation. The validity, reliability and efficiency of the measurement instruments must be considered in user satisfaction surveys. In addition, emphasis should be placed on data analysis and the use of CV results.

Performance-based financing

Verification

User satisfaction

Patient satisfaction

Data fabrication

Use of results

Mali

User satisfaction assessment of health facilities is increasingly considered for quality improvement in health care (OECD et al., 2019). It is widely used in low- and middle-income countries (LMICs) (Aldana & Piechulek, 2001). However, the definition of user satisfaction is complex and still not consensual (Linder-Pelz, 1982; Williams, 1994; Ng & Luk, 2019). Pascoe (1983) defined user satisfaction as the level of compliance between their expectations and the services they received. Cultural, sociodemographic, cognitive, and affective factors influence the measurement of user satisfaction (Ng & Luk, 2019). In Mali, from July 2016 to February 2017, a performance-based financing (PBF) pilot was implemented in ten health districts in the Koulikoro region (World Bank, 2017). It aimed to improve the quality of reproductive health care by increasing the motivation, responsibility, and accountability of health workers (Ministère de la Santé et de l’Hygiène Publique, 2016). During its implementation, several activities were carried out: training, development of results plans, signing of contracts, and producing results by health facilities, reporting, verifying quantity and quality of care results, paying subsidies, and providing community verification (CV) (Cissouma et al., 2017; KIT et al., 2017). CV is one of the main functions of PBF implementation. It monitors the effectiveness of the services reported by the health facilities to provide them with subsidies and to conduct a satisfaction survey of users who have benefited from health care.

In general, the methods used to conduct user satisfaction surveys are quantitative based on standardized questionnaires administered to users (self-reported or administered), and qualitative (semi-structured interviews and focus groups) (Sitzia, 1999; Quintana et al., 2006; Beattie et al., 2015). For Beattie et al. (2015) and Almeida et al. (2015), regardless of the survey methods and measurement tools chosen, some dimensions are essential to the construction of patient satisfaction measurement tools such as interactions between patients and health professionals, the physical environment, and the management process of the health facility. The community user survey method is often used for user satisfaction surveys during PBF implementation in Africa (Antony et al., 2017; Turcotte-Tremblay et al., 2017; Falisse et al., 2012). For example, in Mali, the community-based user survey was the chosen approach for user satisfaction surveys during the PBF (KIT et al., 2017).

Conducting the CV among users during PBF implementation is not without consequences. In Burkina Faso, Turcotte-Tremblay et al. (2017) noted several unintended consequences of the PBF's CV process: difficulties in identifying patients in the community, overloading of community verifiers, falsification of data by interviewers, and loss of patient confidentiality that led to marital problems between some patients and their spouses. In Burundi, Falisse et al. (2012) indicated that this type of survey does not necessarily lead to a better consideration of the “voices” of the population in improving the quality of care. In addition, the transmission of people's concerns by community-based organizations to health authorities did not lead to change in Burundi. In Malawi, Petross et al. (2020) showed that community members had mixed views on the positive effect of PBF on the quality of care in health facilities. The positive effects of the PBF in Malawi, they noted, were better availability of drugs, improved quality of infrastructure, and better hygiene in health facilities (Petross et al., 2020). However, they argued that some of the negative effects that existed prior to the PBF, such as health worker overload and overcapacity of delivery beds in maternity facilities, have been exacerbated by the PBF (Petross et al., 2020). These studies demonstrated that community-based user satisfaction surveys have unintended consequences and are influenced by the local context (geographic, socio-cultural, health) (Falisse et al., 2012; Antony et al., 2017; Turcotte-Tremblay et al., 2017; Petross et al., 2020).

It is important to consider the socio-cultural context when conducting user satisfaction surveys (Baumann & Amara, 2007). To our knowledge, there is no published scientific study in Mali on the process of client satisfaction during the implementation of the PBF and its impact on improving the quality of care in health facilities. Our article aims to analyze the CV process during the implementation of the PBF in the Koulikoro region of Mali. More specifically, it allows us to describe the CV system, the dysfunctions observed during its implementation, and the perceptions and interactions between the different actors involved.

Description of the PBF community verification process in Mali

In Mali, the PBF CV had several objectives. First, it intended to verify the accuracy of health workers' statements regarding patients reported to have consulted them. Second, it intended to measure the level of user satisfaction with the quality of care received. Finally, the results of the CV were to be used to finalize the payment procedures for the first and only cycle of PBF subsidies to health facilities. The PBF pilot was very short, lasting only eight months (July 2016 to February 2017), which justifies the only PBF subsidies payment cycle in Mali. Apart from the CV, the challenges observed during the PBF implementation process in Mali have been published in several articles (Zitti et al., 2019; Coulibaly et al., 2020; Zitti et al., 2021).

The VC was conducted in February 2017, i.e., at the end of the PBF pilot in Mali. The KIT-Cordaid-CGIC consortium, which acts as the PBF implementing agency (contracting and verification) on behalf of the Ministry of Health and Public Hygiene, recruited a local NGO in each of the 10 health districts to conduct the CV. These NGOs were recruited following a call for tenders. Following the selection process, each NGO selected 10 interviewers who were trained by the local company (Clinique de Gestion et d’Innovation des Connaissances [CGIC]) team on behalf of the consortium. The training lasted two days in January 2017. In order to conduct the CV in a short time frame, some local NGOs had to train additional interviewers themselves. To assist local actors in implementing the PBF, ten technical advisors (five national and five international) were made available to the ten health districts (one advisor per health district). The list of patients for CV was to be provided by the district hospitals (DHs) with the support of the PBF technical advisors. However, numerous shortcomings led the investigators to redo some of the lists from the health facilities' registers. These samples were drawn from the patients present in the health facility registers and who were seen during the PBF implementation period (November to December 2016). To conduct the CV at the DH level, the agency responsible for monitoring PBF implementation (Table 1) selected three indicators.

Table 1

List of indicators considered for CV at the district hospital level
Indicators
Complicated deliveries/Cesarean sections performed by qualified personnel
Patients received from the Community Health Centres (Centres de santé communautaires [CSCom]), who were properly cared for in the DHs and who were redirected to the CSCom
Severe malaria in children properly treated

In order to constitute a sample of patients in each DH, 10 patients were selected for each of the three indicators (i.e., 30 patients per DH). The 10 patients selected per indicator were chosen according to their distance from the health facility: three patients within 0 km, three patients within 1–5 km, two patients within 6–10 km, and two patients within 11–15 km and beyond. Local NGOs had between five and 10 days to conduct the CV.

Search strategy

We adopted a qualitative approach based on a multiple case study design (Yin, 2009) corresponding to the DHs and stakeholder groups involved. We selected three health districts (districts 1, 2, and 3) from among the ten in the Koulikoro region. The criteria for selecting the three health districts are detailed elsewhere (Zitti et al., (2019).

In order to compare the views of the actors on the CV process, we selected different profiles of actors (Table 2) who participated in the CV process. The patients interviewed were expected to benefit from the management of at least one of these three indicators. Health workers were selected on a purposive basis. Purposive sampling is a non-random sampling technique that allows a researcher to find participants based on the experiences and knowledge they can and will provide (Bernard, 2006). Health workers were selected based on their level of involvement in the PBF implementation process in DH, their level of information related to the CV process, and their availability. Based on their availability to answer questions, we chose to survey the investigating agents who had participated in the CV.

Table 2

List of survey participants
Respondents	Sanitary district 1	Sanitary district 2	Sanitary district 3	Total
Investigators from local NGOs	5	9	3	17
Patients	2	8	8	18
Health workers	2	1	1	4
Total	9	18	12	39

Mali's health system is made up of three levels, a primary level Community Health Centres (Centres de santé communautaires [CSCom]), a secondary level (DH or Health Center of Reference), and a tertiary level (university hospital centers and national hospitals) (PRODESS III, 2014). DHs are the first level of referral for patients, so patients treated in a DH come from health facilities in all health areas of the health district. We compiled an exhaustive list of patient charts that the local NGOs would have investigated for our sample. To do this, TZ and AC searched the archives of the patients' CV forms that were at the level of the CGIC team, which is responsible for implementing the PBF in Mali. In this list, we targeted four patients for each of the three indicators for DH patients: one patient within 0 km, one patient within 1-5 km, one patient within 6-10 km, one patient within 11-15 km and more. The distance from the selected patients to the facility was calculated from the patient's place of residence to the nearest facility (CSCom or DH). We selected a total of 12 patients to survey per DH for a total of 36 patients.

Data collection

We developed semi-structured interview guides for each of the survey targets (local NGO interviewers, patients, health workers). We tested the collection tools in a CSCom that implements PBF, but it is not included in this research. There were four research assistants in charge of this survey. AC organized a two-day training for the research assistants. During the training, the semi-structured interview guides were fully translated into the local language (Bambara). The research assistants interviewed patients who had been referred to the DHs and the local NGO interviewers. AC spent an average of three days in each health district to supervise the field surveys.

Data processing and analysis

The research assistants transcribed the data. TZ performed data coding using a code tree constructed by the researchers involved in the research program. TZ used QDA Miner Lite for data coding. For the analysis, we followed an inductive content analysis approach (Bernard, 2006), allowing the empirical themes to emerge from the data, allowing us to consider the influence of socio-cultural values, the attitudes and practices of health agents, as well as the individual logics of the different actors involved in the PBF CV process in Mali.

Sample design and identification of patients in the community

Several investigators noted difficulties related to sampling during the CV. In most health districts, the respondents' samples were not ready at the start of the CV, which led to delays in its implementation. In order to deal with the delay in the transmission of samples, some investigators said that they had to go to the health facilities to compile or complete the samples themselves. For them, going to the health facilities to compile the samples was an additional burden and caused a delay in the CV. According to the investigators, in order to avoid delays during the implementation of the CV, the samples must be available before the start of the surveys:

"We had some complications, often so much so that we were late, the verifier (PBF technical advisor) who was at the district level had not finished his work ... we went with the head of the post (head of the health facility) and our supervisor to choose the people (the respondents). "[Interviewer, local NGO Health District 2]

Before going to the community, the investigators first went to the mayor of the commune that covers the health area in question to have their mission orders signed by the CV. They then went to the village chief to explain the purpose of the survey before being introduced to the community. Once the village chief had given his approval, he delegated people to help the interviewers locate and identify the respondents in the community. During the training of the interviewers, the trainers from the CGIC team advised the interviewers not to use community health workers or community relays during the CV. However, in some villages, community relays sometimes helped the interviewers identify and locate respondents in the community. The involvement of community health promoters in the villages varied. For example, in Health District 1, the investigators did everything possible to avoid involving the community relays, unlike in Health Districts 2 and 3. For some investigators, the help of the community relays was important, and this made the respondents feel more confident and facilitated the survey:

"Their (community relays) presence facilitates our access to these people, because they have a certain credibility. They (community relays) introduce us, they say that these people came in search of certain information, so immediately they (the respondents) are open to us" [Interviewer, local NGO, health district 3].

Tracking patients in the community was difficult, if not impossible, for several reasons: incorrect spelling of names, illegible charts, lack of telephone contact information, and incomplete information on the charts. This difficulty was due to poor record-keeping in the vast majority of facilities. In addition, the homonymity of respondents often caused difficulties in locating and identifying them in the villages. For the interviewers, there were often cases where there were several people with the same name in the same village. When additional information was available (husband's name or child's first name) on the survey form, they were able to identify the person with the help of the community liaison or an accompanying person. In other cases, they spent additional time locating and identifying the respondents or were unable to do so. Some patients identified in the health areas were not necessarily part of these areas. According to some respondents, the nomadic nature of some people in certain villages makes CV more difficult to carry out:

"Often it is the spelling of the name that is misspelled. I remember one case where the child was BANDJAN but it was spelled JINDJAN" [Interviewer, local NGO, health district 3].

"During the dry season, people come to the villages and during the winter period, they return to the hamlets and when they are in the village, they go to the CSCom. When the CSCom asks them, they say that they live in the village and the CSCom is obliged to write in the register according to their information...when you go to look for them in the villages, it will be complicated" [Health agent, Health District 1].

Users', health workers' and investigators' interests in CV

The vast majority of health workers thought that the CV was a good thing and that it was important to consider the opinions of community members in the process of improving the quality of care offered by DHs. No feedback on the results of the CV was available from health workers because it was conducted in the last days of the PBF pilot (February 2017). According to some investigators, collecting patients' opinions on the quality of care received is a positive part of the CV. For some investigators, the PBF CV is an accurate survey compared to other community surveys. The vast majority of interviewers had previously conducted community surveys with NGOs. For them, these other surveys just ask for a number of people to be surveyed, whereas for the PBF CV, the names of the respondents are specific, so it adds precision to the survey. Some investigators believe that if the CV is done well, it would allow for the raise of people's complaints for improved quality of care.

"I have done many surveys, but the community verification survey I was very happy to go to my brothers and sisters asking them what is wrong and thinking that we will solve their problem." [Interviewer, local NGO Health District 3]

"It (CV) is a good thing if we can only take the recommendations into account, not just leave them on paper" [Interviewer, local NGO, Health District 2].

Before the CV, almost all patients had never heard of PBF. However, the CV process was well received and welcomed by the vast majority of respondents. They were happy with the CV, as it was the first time they had been asked for their opinions on the quality of care provided in DH. The CV allowed people to see that health workers could be monitored with respect to the quality of care they provide. In addition, they believe that regular CVs will enable health workers to do their jobs better. According to them, they freely expressed their opinions about the difficulties encountered in the DHs. For example, some of them raised the high cost of care and medicines:

"It is always good to exchange with people to know how they were treated... people will say if they were well received or not! There is nothing to be ashamed of if you say you were not well received, and it can help health care workers improve" [Patient, health district 3].

Nevertheless, other respondents expressed a sense of mistrust, fear, and apprehension at the beginning of the survey. The investigators tried to reassure the respondents and their families about the confidentiality of the CV and their independence from the DHs. However, despite their explanations of the confidentiality and purpose of the surveys, some respondents were still suspicious. According to some, the population is suspicious of community surveys conducted by NGOs because they take place without changing their living conditions or obtaining results:

"At first they are afraid because they think that every time people come to bluff them (illusory promise) without results. It's not just the PBF, all the NGOs every time they come and gather people and say this and that without any follow up, so they thought ... we're going to come and bluff them, we're going to come and waste their time. "[Interviewer, local NGO Health District 2].

"Every time, we answer the questions and we don't know what happens next. If you see that some people are reluctant to answer the questions, it is because of that" [Patient, health district 2].

Challenges faced by investigators during the CV process

For the majority of the investigators, the two-day training was not long enough as it did not allow them to address all aspects of CV in-depth. The training covered the translation of the questionnaires into the local language (Bambara), how to introduce themselves, how to fill out the survey form, and how to conduct the surveys among the population. The majority of the interviews were conducted in Bambara. Translating the forms into Bambara was a real challenge for the interviewers. They found it difficult to translate the PBF and to explain it easily in the local language:

“We have to make the survey forms simple because it is not easy to translate the French that is on the forms. And even during the training, translation was a problem. We had to reach a consensus on the questionnaires and agree on the translation into the national language.” [Interviewer, local NGO health district 2]

The majority of interviewers reported difficulties in identifying certain patients in the community. Several strategies were implemented by the interviewers to overcome these difficulties. Some interviewers replaced the respondent with her husband or other relatives. Others said they replaced the respondent who was not found with another patient chosen from the facility's records. Occasionally, some interviewers went to the health facilities to ask health workers if they knew any respondents. When this was the case, they could help the interviewer find the respondent. However, trainers from the CGIC team advised the interviewers against using health workers to help them:

“Given the distance I traveled, I didn't want to go back without seeing the lady to ask her for some information. It's true that we were asked not to go to the CSCom, .... if I hadn't gone to the CSCom I was going to ... come back without news of the lady.” [Interviewer, local NGO, health district 2].

In addition, although a rare occurrence, some interviewers conducted telephone interviews with respondents when there was difficulty meeting the respondent in person. For example, there was a case where the interviewer obtained the telephone number of the community relay with a nurse in a health facility. He called the Community Relays to get information on a respondent. Thus, some investigators fraudulently filled out some survey forms without investigating the users in the communities. According to these investigators, the "cheating" was justified by the time they were given (5 to 10 days) to complete the CV:

“The people who filled out the forms with imagination, really, it's because we didn't have enough time. They were under a bit of pressure. ... They were really forced to do these 'little cheats'” [Interviewer, local NGO Health District 2].

Cheating on survey completion by interviewers was more common in urban than in rural areas. We observed that the vast majority of survey forms in urban areas did not include a telephone number, and this was true in most health districts. The lack of a telephone number on the survey forms of urban and peri-urban respondents made it impossible to conduct the survey since, without a telephone number, it is unlikely to find a user in an urban or peri-urban area.

Our article is one of the few studies published in Francophone West Africa that focuses on the CV process during the implementation of the PBF. It addresses the challenges of implementing the RBF in the Malian context by showing the complexity, cumbersomeness, and difficulties of the process.

PBF community verification: reliable, valid, and efficient?

In Mali, unlike in other countries (Burundi, Niger), with regard to CV, the percentage of tolerated deviation between the declarations of health facilities and reality has not been specified in the PBF procedure manual (Ministère de la Santé et de l’Hygiène Publique, 2016). This percentage of tolerated discrepancy is intended to reduce fraud due to false declarations by health facilities in order to benefit from more PBF subsidies. This percentage must be less than 5% in Burundi (Ministère de la Santé Publique et de la lutte contre le SIDA, 2014) and 10% in Niger (Ministère de la Santé Publique du NIGER, 2020). While the percentage of fraud tolerated by the health facilities is mentioned in some procedure manuals or CV guides, no mention is made of cases of fraud by the investigators who are supposed to carry out the CV.

With respect to data collected during field surveys in the African context, the issue of fabrication of such data by scientists and non-scientists is an issue that is not widely addressed (Kingori & Gerrets, 2016). The term that sometimes refers to data falsification by field workers in the African context is "data filling under a tree." It is defined as an interviewer sitting under a tree to fill out a survey questionnaire without collecting information from the participants selected for the survey (Fig. 1) (Kingori & Gerrets, 2016). Several factors are thought to be responsible for data falsification by field workers in the African context: physical, economic, and contextual challenges during community surveys; workload perceived as unrealistic by interviewers; dissatisfaction with working conditions; insufficient institutional support; and weak supervision during surveys (Kingori & Gerrets, 2016).

Our study shows that there was fraud in the filling out of survey forms by some investigators. There are several reasons for this: incompleteness of the forms, errors in the information on the forms, difficulties in locating users within the community (absence, displacement), and the short time of the surveys (5 to 10 days). In Burkina Faso, Turcotte-Tremblay et al. (2017) also observed cases of fraud during the CV. According to them, there are several reasons for data falsification by investigators: work overload and difficulties in identifying patients in the community. We believe that the more difficult it is for the interviewer to identify the user, the greater the risk of falsification of data by the interviewer. In order to improve the performance of the CV and to better identify users in the community, it is necessary to ensure that (i) the correct sampling of patients; (ii) the completeness of the information recorded in the health facility registers; and (iii) the completeness of the information recorded on the survey forms.

Does this observed fraud call into question the reliability and validity of the information collected during the CV? In Mali, the falsification of some survey forms (especially in urban areas) may have had an impact on the quality of the data collected during the CV, even if these frauds did not negatively affect the amount of subsidies allocated to the DH. The objectives of the CV in Mali were to verify the care reported by health workers and to measure the level of user satisfaction. Based on these objectives, we can deduce that fraud by some investigators does not allow us to accurately verify declarations or to evaluate the level of user satisfaction. In order to improve the quality and integrity of fieldworkers during research in Africa, it is necessary to (i) increase institutional recognition of the role of fieldworkers; (ii) improve systematic and comprehensive capacity building of fieldworkers; and (iii) foster career development of fieldworkers within institutions (Participants of an International Workshop in Kenya on the Role of Frontline Staff in Biomedical Research, July 2014 & Kombe, 2015)

It is not necessary to have CV surveys with a 0% risk of fraud. However, survey tools and methods should be selected that will reduce fraud to ensure better quality results. Several systematic reviews that focus on instruments for measuring patient experience in hospitals showed that the quality of the methods and tools chosen is variable (Beattie et al., 2015; Nilvér et al., 2017). Beattie et al. (2015) showed in their systematic review that despite the acceptability of the majority of instruments, the efficiency of these measurement instruments (ratio of cost to data reliability) is insufficient due to the resources needed to obtain reliable samples. For a valid, reliable, and efficient measurement of patient experience, the instruments should be more suitable for the objectives of the survey (Nilvér et al., 2017); and more efficient (Sitzia, 1999; Quintana et al., 2006; Beattie et al., 2015).

What is the purpose of the PBF community verification?

According to our results, the majority of users were satisfied with the fact that the CV allowed them to give their opinions on the care received. However, some of them expressed doubts as to whether their opinions would improve the quality of care in the health facilities. For them, the CV is just another community survey that will not have a positive impact on their daily lives. For Olivier de Sardan (1991), in order to improve the living conditions of the population, development projects must consider the opinions of the population. Development anthropology is intended to be the “voice” of the “voiceless” (marginalized and excluded social groups) and should be used to gather and express the opinions of people whose “fate development institutions intend to improve without giving themselves the means to know them” (Olivier de Sardan, 1991, p. 7). Studies that assess the level of satisfaction of users with the care received in health facilities in Africa revealed low levels of satisfaction (Ndziessi et al., 2017; Girma et al., 2020; Balde et al., 2017; Bohren et al., 2017). In Guinea Conakry, for example, Balde et al. (2017) showed in their study that women report abuse by health workers during childbirth (neglect, physical and verbal abuse). In Ethiopia, a study by Girma et al. (2020) showed that community members have a negative perception of maternal health services. The reasons for their negative perceptions are poor attitudes of health workers towards them, lack of drugs in health facilities, long waiting times, lack of privacy, and lack of attention for women during delivery (Girma et al., 2020).

As the CV was conducted at the end of the PBF pilot, the data collected was only used to calculate the PBF subsidies for DHs. The data collected was not analyzed, or only minimally, by the KIT-Cordaid-CGIC consortium, which acts as the PBF implementing agency (contracting and verification) on behalf of the Ministry of Health and Public Hygiene. The CV did not establish a causal link between the PBF intervention and the improvement in the quality of care reported by users. There is little literature that examines the different verification methods in the PBF and the causal link between these verification methods and verification outcomes (Vergeer et al., 2016). In Malawi, Kambala et al. (2017) showed that women who received care during the implementation of the PBF noted an improvement in health service delivery. However, there were no significant effects of the PBF intervention on women's perception of technical care, quality of facilities and interpersonal relationships. In Burundi, Falisse et al. (2012) showed that the results of the community survey did not necessarily take into account the opinions of users and did not lead to changes in the quality of care. In Benin, Antony et al. (2017) showed that verification processes are complex, costly, and time-consuming; and that little or no analysis was done on the data collected during the CV. Furthermore, they showed that the results of the CV did not lead to the sanctioning of benefits that had been involved in fraud in Benin. All this shows us the importance of using the results of the user satisfaction surveys during the PBF CV to improve the quality of care offered to the population.

Beyond the PBF, the question of the use of research results for decision-making to improve the quality of care arises in the African context. Despite a declared interest by health workers in user satisfaction surveys, the results of these surveys are often not used or discussed in decision-making (Boyer et al., 2006). A systematic review on the use of knowledge to improve public health in LMICs showed that there is a wide variety of strategies and conditions for using knowledge (Siron et al., 2015). Evidence of the use of results for decision-making by actors governing the health system is still too scarce in Africa (Ridde & Yaméogo, 2018; Dagenais, 2021; Dagenais et al., 2021). A growing number of studies carried out in Francophone Africa showed that field actors and decision-makers find the results of research useful (Mc Sween-Cadieux et al., 2017; Dagenais et al., 2021). However, in Burkina Faso, many participants in a research dissemination workshop felt that they were not able to make decisions about the application of this knowledge, despite the fact that the majority of them had perceived the usefulness of the results (Mc Sween-Cadieux et al., 2017). Similarly, a study by Dagenais (2021) showed that there is a low level of use of research results within the Ministry of Health for decision-making in Burkina Faso. Several barriers are identified by decision-makers in the Burkina Faso Ministry of Health: difficulty in accessing research results, their own attitudes towards research-based evidence, difficulty in implementing research recommendations, and a lack of an effective communication strategy. Dagenais et al. (2021) believe that for better use of research findings by policymakers, actions need to be taken to produce and sustain long-term effects at the level of policymakers.

Practical implications of the research

To improve CV in the future, we believe it will be necessary to: (i) to seek to reduce its cost by adopting a more suitable survey methodology (less cumbersome, less costly and more efficient); (ii) for health workers to be able to better collect data (surname and first names, telephone number, surname and first names of ascendants/descendants or other family members) in the health facility registers; (iii) to raise awareness among users on the need to provide reliable information that can be traced back; (iv) involve local NGOs in the process of selecting and choosing the people to be surveyed; (v) allow more time for data collection in the field and strengthen the supervision of surveyors, to reduce the risk of fraud during data collection in the field.

Limitations of the study

Our research has some limitations. Firstly, the opinions of the respondents are not necessarily representative of all the actors involved in the CV process. Secondly, the data collection was carried out cross-sectionally, retrospectively, and at the end of the PBF pilot project. Therefore, several difficulties that are revealed in our study during the CV process might have been different if the PBF implementation had been continued. Some of the implementation difficulties, such as the record keeping of DHs, were bound to improve if the PBF continued. In addition, investigators should also improve on the shortcomings observed over time.

The CV is an important component of the implementation of the PBF. Our study has shown that several difficulties have hindered the smooth running of the CV: too short a timeframe, problems with sampling users, lack of precise information that could facilitate the identification and tracking of users within the communities. All these difficulties sometimes led some interviewers to falsify the data collected. The validity, reliability, and efficiency of the measurement instruments must be taken into account when conducting user satisfaction surveys during the PBF CV. Furthermore, these instruments should be adapted and simplified to the local context for maximum effectiveness. Particular emphasis should be placed on the analysis of data and the use of results from the PBF CV in order to make better use of the results of these surveys for decision making and improving the governance of the health system at the local and national level. In Mali, in order to better measure patient satisfaction, we believe that a study should be carried out on patients' expectations regarding the quality of care in order to better adapt the tools for measuring satisfaction to the local context.

CV, Community verification; DHs, District hospitals; LMICs, Low- and middle-income countries; PBF, Performance-based financing.

Ethics approval and consent to participate

The research protocol has been validated by the ethics committee of the National Institute of Public Health Research (INRSP) of Mali (N°24/2015/CEINRSP). All participants were informed in detail about the objectives of our research. We obtained the free and informed consent from all the participants.

Consent for publication

Not applicable.

Availability of data and materials

The data sets used and analyzed in this study are available from the corresponding author upon reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This work was funded by three Canadian agencies: World Agency Canada (WAC), the Canadian Institutes of Health Research (CIHR), and the International Development Research Centre (IDRC). Our study is part of a larger research program on “Results based financing for equitable access to maternal and child health care in Mali and Burkina Faso”. This seven-year program is part of the "Innovation for Maternal and Child Health in Africa" initiative. This research program is implemented by two research NGOs based in the two countries, and researchers from the University of Montreal. In Mali, the team is composed of researchers and research assistants from the NGO MISELI led by Laurence Touré, who is the principal investigator of the program.

Authors’ contributions

The research protocol was developed by VR with the contribution of AC. The collection tools were developed by TZ and AC with the contribution of VR. Analyses were done by TZ with input from VR, and AC. TZ wrote the first draft of the article. All authors read, improved, and approved the final version of the article.

Acknowledgements

Special thanks to Laurence Touré for her advice and support during the conduct of this research. We would also like to thank all the actors and organizations at the national, regional, and local levels that participated in and/or facilitated the completion of this research. Finally, we would like to thank Heather Hickey who is a professional translator for her help in translating the article from French into English.

Aldana, J. M., & Piechulek, H. (2001). Satisfaction des patients et qualite´ des soins dans des zones rurales du Bangladesh. 5, 6.

Almeida, R. S. de, Bourliataux-Lajoinie, S., & Martins, M. (2015). Satisfaction measurement instruments for healthcare service users : A systematic review. Cadernos de Saúde Pública, 31(1), 11‑25. https://doi.org/10.1590/0102-311X00027014

Antony, M., Bertone, M. P., & Barthes, O. (2017). Exploring implementation practices in results-based financing : The case of the verification in Benin. BMC Health Services Research, 17(1). https://doi.org/10.1186/s12913-017-2148-9

Balde, M. D., Diallo, B. A., Bangoura, A., Sall, O., Soumah, A. M., Vogel, J. P., & Bohren, M. A. (2017). Perceptions and experiences of the mistreatment of women during childbirth in health facilities in Guinea : A qualitative study with women and service providers. Reproductive Health, 14(1), 3‑3. PubMed. https://doi.org/10.1186/s12978-016-0266-1

Baumann, M., & Amara, M.-E. (2007). Evaluation de la satisfaction. Prendre en compte les valeurs socioculturelles des usagers. 465, 292‑297. http://hdl.handle.net/10993/1685

Beattie, M., Murphy, D. J., Atherton, I., & Lauder, W. (2015). Instruments to measure patient experience of healthcare quality in hospitals : A systematic review. Systematic Reviews, 4, 97‑97. PubMed. https://doi.org/10.1186/s13643-015-0089-0

Bernard, H. R. (2006). Research methods in anthropology : Qualitative and quantitative approaches (4th ed). AltaMira Press.

Bohren, M. A., Vogel, J. P., Tunçalp, Ö., Fawole, B., Titiloye, M. A., Olutayo, A. O., Ogunlade, M., Oyeniran, A. A., Osunsan, O. R., Metiboba, L., Idris, H. A., Alu, F. E., Oladapo, O. T., Gülmezoglu, A. M., & Hindin, M. J. (2017). Mistreatment of women during childbirth in Abuja, Nigeria : A qualitative study on perceptions and experiences of women and healthcare providers. Reproductive Health, 14(1), 9. https://doi.org/10.1186/s12978-016-0265-2

Boyer, L., Francois, P., Doutre, E., Weil, G., & Labarere, J. (2006). Perception and use of the results of patient satisfaction surveys by care providers in a French teaching hospital. International Journal for Quality in Health Care, 18(5), 359‑364. https://doi.org/10.1093/intqhc/mzl029

Cissouma, L., Sioro, A., Koné, O., Traoré, K., Bulthuis, S., Mommers, P., & Toonen, J. (2017). Rapport de Capitalisation FBR à la Malienne (p. 45).

Coulibaly, A., Gautier, L., Zitti, T., & Ridde, V. (2020). Implementing performance-based financing in peripheral health centres in Mali : What can we learn from it? Health Research Policy and Systems, 18(1), 54. https://doi.org/10.1186/s12961-020-00566-0

Dagenais, C. (2021). Research use at the Ministry of Health in Burkina Faso : The decision-makers’ perspective. Implementation Science Communications, 2(1), 22‑22. PubMed. https://doi.org/10.1186/s43058-021-00126-9

Dagenais, C., Proulx, M., Mc Sween-Cadieux, E., Nikiema, A., Bonnet, E., Ridde, V., & Somé, P.-A. (2021). Collaborative research and knowledge translation on road crashes in Burkina Faso : The police perspective 18 months on. Health Research Policy and Systems, 19(1), 3. https://doi.org/10.1186/s12961-020-00654-1

Falisse, J.-B., Meessen, B., Ndayishimiye, J., & Bossuyt, M. (2012). Community participation and voice mechanisms under performance-based financing schemes in Burundi : Community participation under PBF. Tropical Medicine & International Health, 17(5), 674‑682. https://doi.org/10.1111/j.1365-3156.2012.02973.x

Girma, M., Robles, C., Asrat, M., Hagos, H., G/Slassie, M., & Hagos, A. (2020). Community Perception Regarding Maternity Service Provision in Public Health Institutions in 2018 and 2019 : A Qualitative Study. International Journal of Women’s Health, 12, 773‑783. PubMed. https://doi.org/10.2147/IJWH.S250044

Kambala, C., Lohmann, J., Mazalale, J., Brenner, S., Sarker, M., Muula, A. S., & De Allegri, M. (2017). Perceptions of quality across the maternal care continuum in the context of a health financing intervention : Evidence from a mixed methods study in rural Malawi. BMC Health Services Research, 17(1). https://doi.org/10.1186/s12913-017-2329-6

Kingori, P., & Gerrets, R. (2016). Morals, morale and motivations in data fabrication : Medical research fieldworkers views and practices in two Sub-Saharan African contexts. Social Science & Medicine, 166, 150‑159. https://doi.org/10.1016/j.socscimed.2016.08.019

KIT, Clinique de gestion et d’innovation des connaissances, & Cordaid. (2017). RAPPORT FINAL DU PROJET PILOTE « PRSR/FBR » FBR-Financement Basé sur les Résultats – Approche pour accélérer les résultats en Santé de la Reproduction.

Linder-Pelz, S. (1982). Toward a theory of patient satisfaction. Social Science & Medicine, 16(5), 577‑582. https://doi.org/10.1016/0277-9536(82)90311-2

Mc Sween-Cadieux, E., Dagenais, C., Somé, P.-A., & Ridde, V. (2017). Research dissemination workshops : Observations and implications based on an experience in Burkina Faso. Health Research Policy and Systems, 15(1), 43. https://doi.org/10.1186/s12961-017-0205-9

Ministère de la Santé et de l’Hygiène Publique. (2016). Manuel Opérationnel du Financement basé sur les résultats du projet PRSR-FBR.

Ministère de la Santé Publique du NIGER. (2020). GUIDE DE LA VERIFICATION COMMUNAUTAIRE DANS LE CADRE DE LA MISE EN ŒUVRE DU FONDS D’ACHAT.

Ministère de la Santé Publique et de la lutte contre le SIDA. (2014). Manuel des procédures pour la mise en œuvre du financement base sur la performance a usage au niveau des formations sanitaires.

Ndziessi, G., Bintsene-Mpika, G., & Bileckot, R. (2017). User Satisfaction with Family Planning Services in Government Health Centres in the Congo. African Journal of Reproductive Health, 21(3), 70‑75. https://doi.org/10.29063/ajrh2017/v21i3.6

Ng, J. H. Y., & Luk, B. H. K. (2019). Patient satisfaction : Concept analysis in the healthcare context. Patient Education and Counseling, 102(4), 790‑796. https://doi.org/10.1016/j.pec.2018.11.013

Nilvér, H., Begley, C., & Berg, M. (2017). Measuring women’s childbirth experiences : A systematic review for identification and analysis of validated instruments. BMC Pregnancy and Childbirth, 17(1), 203‑203. PubMed. https://doi.org/10.1186/s12884-017-1356-y

OCDE, Organisation mondiale de la Santé, & World Bank Group. (2019). La qualité des services de santé : Un impératif mondial en vue de la couverture santé universelle. OECD. https://doi.org/10.1787/62f287af-fr

Olivier de Sardan, J. (1991). L’anthropologie du changement social et du développement comme ambition théorique ? Bulletin de l’APAD, 1. https://doi.org/10.4000/apad.296

Participants of an International Workshop in Kenya on the Role of Frontline Staff in Biomedical Research, July 2014, & Kombe, F. (2015). Enhancing quality and integrity in biomedical research in Africa : An international call for greater focus, investment and standardisation in capacity strengthening for frontline staff. BMC Medical Ethics, 16(1), 77. https://doi.org/10.1186/s12910-015-0071-3

Pascoe, G. C. (1983). Patient satisfaction in primary health care : A literature review and analysis. Evaluation and Program Planning, 6(3‑4), 185‑210. https://doi.org/10.1016/0149-7189(83)90002-2

Petross, C., McMahon, S., Lohmann, J., Chase, R. P., Muula, A. S., & De Allegri, M. (2020). Intended and unintended effects : Community perspectives on a performance-based financing programme in Malawi. BMJ Global Health, 5(4), e001894‑e001894. PubMed. https://doi.org/10.1136/bmjgh-2019-001894

PRODESS III. (2014). PROGRAMME DE DEVELOPPEMENT SOCIO-SANITAIRE 2014-2018 (PRODESS III). https://www.prb.org/wp-content/uploads/2018/04/Programme-de-De%CC%81veloppement-Socio-Sanitaire-2014-2018.-Mali.pdf

Quintana, J. M., González, N., Bilbao, A., Aizpuru, F., Escobar, A., Esteban, C., San-Sebastián, J. A., de-la-Sierra, E., & Thompson, A. (2006). Predictors of patient satisfaction with hospital health care. BMC Health Services Research, 6(1), 102. https://doi.org/10.1186/1472-6963-6-102

Ridde, V., & Yaméogo, P. (2018). How Burkina Faso used evidence in deciding to launch its policy of free healthcare for children under five and women in 2016. Palgrave Communications, 4(1), 119. https://doi.org/10.1057/s41599-018-0173-x

Siron, S., Dagenais, C., & Ridde, V. (2015). What research tells us about knowledge transfer strategies to improve public health in low-income countries : A scoping review. International Journal of Public Health, 60(7), 849‑863. PubMed. https://doi.org/10.1007/s00038-015-0716-5

Sitzia, J. (1999). How valid and reliable are patient satisfaction data? An analysis of 195 studies. International Journal for Quality in Health Care, 11(4), 319‑328. https://doi.org/10.1093/intqhc/11.4.319

Turcotte-Tremblay, A.-M., Gali-Gali, I. A., De Allegri, M., & Ridde, V. (2017). The unintended consequences of community verifications for performance-based financing in Burkina Faso. Social Science & Medicine, 191, 226‑236. https://doi.org/10.1016/j.socscimed.2017.09.007

Vergeer, P., Heard, A., Josephson, E., & Fleisher, L. (2016). Verification in Results-Based Financing for Health. World Bank, Washington, DC. https://doi.org/10.1596/26040

Williams, B. (1994). Patient satisfaction : A valid concept? Social Science & Medicine, 38(4), 509‑516. https://doi.org/10.1016/0277-9536(94)90247-X

World Bank. (2017). Mali—Strengthening Reproductive Health Project (N^o ICR4095; p. 81). World Bank Group. http://documents.worldbank.org/curated/en/542131502720462868/Mali-Strengthening-Reproductive-Health-Project

Yin, R. K. (2009). Case study research : Design and methods (4th ed). Sage Publications.

Zitti, T., Fillol, A., Lohmann, J., Coulibaly, A., & Ridde, V. (2021). Does the gap between health workers’ expectations and the realities of implementing a performance-based financing project in Mali create frustration? Global Health Research and Policy, 6(1), 5. https://doi.org/10.1186/s41256-021-00189-0

Zitti, T., Gautier, L., Coulibaly, A., & Ridde, V. (2019). Stakeholder Perceptions and Context of the Implementation of Performance-Based Financing in District Hospitals in Mali. International Journal of Health Policy and Management, 10.

PDF

Version 1

posted 15 Apr, 2022

You are reading this latest preprint version

From Measuring User Satisfaction to Improving Quality of Care: Community Verifications and Performance-Based Financing in District Hospitals in Mali

Status:

Version 1

Abstract

Figures

Background

Methods

Search strategy

Data collection

Data processing and analysis

Results

Sample design and identification of patients in the community

Users', health workers' and investigators' interests in CV

Challenges faced by investigators during the CV process

Discussion

PBF community verification: reliable, valid, and efficient?

What is the purpose of the PBF community verification?

Practical implications of the research

Limitations of the study

Conclusion

Abbreviations

Declarations

References

Status:

Version 1