The Association Between NCQA-PCMH Evaluation and Patients’ Experiences with Community Health Centres: Results from the Greater Bay Area Study, China


 Background

To explore the association between the service quality of community health centres (CHCs) as the supplier evaluation and patients’ experiences of health services as the demand-side evaluation.
Methods

This study was conducted at six CHCs in the Greater Bay area of China. During August–October 2019, 1568 patients were recruited (55.8% women and 44.2% men). We evaluated the service quality of CHCs using the National Committee for Quality Assurance Patient-Centered Medical Home (NCQA-PCMH) recognition questionnaire. We assessed patients’ experiences with medical and health services using the Primary Care Assessment Tools (PCAT).
Results

The global PCAT score was higher at the CHCs with higher NCQA-PCMH levels, showing positive dose-effect trends. This was similar for the PCAT sub-dimensions. For example, the scores of each PCAT sub-dimension of NCQA-PCMH Level 3 were significantly higher than those of Levels 2 and 1. Except for D and F dimensions, the scores of other sub-dimensions of Level 2 were significantly higher than those of Level 1.
Conclusions

Our results indicated that better institutional service quality evaluation determined by the NCQA-PCMH led to better patients’ experiences as determined by the PCAT. Our findings added new evidence in support of better institutional primary healthcare service quality leading to better experiences among patients, and would help further improve the patient-centred primary healthcare service policy and management.


Introduction
The outbreak of the 2019 novel coronavirus (COVID-19) has evolved into a global crisis. As the rst line of defence against pandemic prevention and control, community health centres (CHCs) have played important roles in primary health screening and management globally. [1][2][3] It is urgent to enhance the service quality of CHCs to effectively deal with health emergencies and community health management.
First, the evaluation of the service quality of CHCs should be put in place.
Quality evaluation can be de ned by the three dimensions of 'structure-process-result', and better process quality will lead to better outcome quality, according to Donabedian, the father of quality management in the United States. [4] For the evaluation of primary healthcare service quality, process quality can be determined by the institutional quality evaluation as the supplier evaluation, and outcome quality can be determined by patients' experiences as the demand-side evaluation.
The National Committee for Quality Assurance Patient-Centered Medical Home (NCQA-PCMH) has been o cially recognised as an institutional primary healthcare service quality assessment tool. [5] The PCMH is 'the most economic and suitable' healthcare service model by the World Health Organization and was developed in the United States. [5] As of 2020, about 13,000 primary care institutions and 67,000 physicians in the United States have been accredited so that they can identify and improve service quality problems to improve patient satisfaction according to the assessment of NCQA-PCMH and reduce healthcare costs. [6][7][8] The Primary Care Assessment Tool (PCAT) is a series of scales that have been developed by the Primary Care Policy Center of Johns Hopkins University, which evaluate the primary healthcare service quality based on patients' experiences. [9] Some studies have tried to explore the association of primary healthcare service quality evaluation between the demand (patients) and supplier (healthcare service institution) side. However, the results have been inconsistent. Star eld and his colleagues analysed the differences between consumers' experiences and provider reports in four main areas of primary care ( rst contact, ongoing care, comprehensiveness, and coordination) and three related areas (family centredness, community orientation, and culturally competent care). There were no signi cant differences between consumers and providers. [10] However, other studies found an association between the evaluation of primary healthcare quality services determined by patients and health institutions. The results from a Spanish study indicated that the evaluation of service quality among staff was always higher than that among patients. [11] In China, according to a quantitative study based on the PCAT, patients' evaluation of rst-class hospitals has been better than that of second-and third-class hospitals. [12] Based on PCAT as well, our previous study indicated that CHCs provide better primary healthcare services than secondary and tertiary health institutions in China, even after controlling for socio-demographic and medical characteristics. [13] Different evaluation standards of service quality, various assessment scales, and differences in population characteristics may have contributed to the heterogeneity of the published results. However, a need remains for additional investigation to assess the evidence for an association of primary healthcare service quality evaluation between the demand (patients) and supplier (healthcare service institution) side. Furthermore, the most obvious limitation of the existing studies is the lack of quantitative institutional evaluation.
Although the NCQA-PCMH is o cially recognised as a quantitative assessment tool of institutional evaluation (supply-side), and the PCAT is widely used as a quantitative assessment tool for patient evaluation (demand-side), associations between the NCQA-PCMH and the PCAT have not been explored globally. To help address the pending data gaps, we estimated the association between the NCQA-PCMH and PCAT in six CHCs. We hypothesised that better institutional service quality evaluation determined by the NCQA-PCMH will lead to better patients' experiences as determined by the PCAT.

Study area and study population
This study was conducted in selected urban CHCs in Guangzhou, the biggest metropolis in the Greater Bay Area, southern China. We employed a multi-stage, strati ed clustering sampling protocol. Data were collected from August to October 2019. In our sampling strategy, the city was strati ed into four urban districts: Liwan, Yuexiu, Tianhe, and Haizhu. Two urban CHCs were randomly selected from Tianhe and Haizhu districts, and one urban CHC was selected from Yuexiu and Liwan districts, respectively. As shown in Figure S1, six CHCs were randomly selected: Linhua (LH), Liede (LD), Jianghai (JH), Shayuan (SY), HuangHuagang (HHG), and Hualin (HL). Then, one family physician group was selected randomly from each selected CHC. In the last sampling stage, participants were randomly recruited by the selected family physician group during their visit to CHCs. One participant aged 20 years or older with one year or more of residency in the current district without audio-visual impairment, mental illness, or other di culties in cooperation was recruited.
Written informed consent from each participant was obtained prior to data collection and sample collection. The study procedure was approved by the Human Studies Committee of Sun Yat-sen University in compliance with the Declaration of Helsinki-Ethical Principles for Medical Research Involving Human Subjects (no. IRB2014.9).

Evaluation of service quality of institutions
We used the NCQA-PCMH recognition questionnaire to evaluate the institutional primary healthcare service as the supplier evaluation. The 2014 version of the NCQA-PCMH survey tool was authorised by the American NCQA website [6] and translated into Chinese. The 2014 version of the NCQA-PCMH scale included six sections: PCMH1, patient-centred access; PCMH2, team-based care; PCMH3, population health management; PCMH4, care management and support; PCMH5, care coordination and care transitions; and PCMH6, performance measurement and quality improvement. Each section has 3-7 evaluation elements, each including a necessary element (a total of 27 elements), and 2-11 speci c items for each element, each including a key entry (a total of 178 items). Each element of each section had separate scoring criteria and rules. According to the compliance of the criteria required by the evaluation agency for different items of each element, the percentage of the institution in the element is obtained (if it does not meet the key entry of the element, the individual percentage of the element is 0%). According to the score of each element of the scale, the answer result of the questionnaire is calculated, the score of each section is obtained, and the total score is obtained. The highest score was 100, which can be divided into three levels: 35-59 for Level 1, 60-84, for Level 2, and 85-100, for Level 3. [14] The higher the NCQA-PCMH level of the organisation, the better the service quality and the lower the service cost. [6] Therefore, it can be favoured by patients and guide patients to implement medical insurance purchase decisions based on a cost-quality assessment. [14] The assessment of patients' experiences We used the PCAT consumer-client version to identify the quality of primary care services as the evaluation of the demand-side. The PCAT was originally validated by the Johns Hopkins Primary Care Policy Center in the United States to measure the quality of primary care services. [10] Rather than other questionnaires focusing on subjective satisfaction, the PCAT focuses on the real experiences of obtaining primary care services in CHCs. Thus, the consequence of the PCAT is more objective. [9] The PCAT includes the following 11 dimensions: A. Extent of a liation with a place/doctor, B. First contact in terms of utilisation, C. First contact care in terms of access, D. Ongoing care, E. Coordination of care, F. Coordination of information systems, G. Comprehensiveness of services available, H. Comprehensiveness of services provided, I. Family centredness, J. Community orientation, and K.
Culturally competent care. Each dimension contains 3-5 items. All items in the PCAT were represented by a 4-point Likert-type scale (1 = de nitely not, 2 = probably not, 3 = probably, and 4 = de nitely). The average score for each scale was derived by averaging the values for all the items under each scale. The average score for the overall quality of primary care was derived by averaging the values for all scales. [15] The higher the PCAT score, the better patients' experiences, and the better the quality of primary care.

Covariates
We collected individual information such as socio-demographic, health service satisfaction, medicalrelated, and health status characteristics through a self-report questionnaire. Socio-demographic information consisted of age (years), sex (male vs. female), home address, annual family income, and highest educational attainment. Health status included diabetes and hypertension. Medical-related status included primary type of health insurance and out-of-pocket medical expenditures.

Statistical analysis
Continuous variables were reported as means ± standard deviations, and relative frequencies were calculated for categorical variables. Welch's test was used to compare the PCAT scores of different NCQA-PCMH levels, and Tamhane's T2 test was used to make multiple comparisons afterwards.
Statistical analyses were conducted using SPSS version 21.0 (SPSS Inc., Chicago, IL, USA). All statistical tests were two-sided, and p-values < .05 were deemed signi cant.

Results
Baseline characteristics and evaluation of patients' experiences with medical and health services A total of 1776 patients were invited to complete the PCAT questionnaires and 1744 returned the questionnaire (response rate = 98.2%). After data cleaning, 1568 valid questionnaires were obtained (effective rate = 89.9%), including 537 in SY, 396 in LH, 149 in LD, 213 in JH, 183 in HL, and 90 in HHG.
The baseline characteristics of participants strati ed by CHCs are presented in Table 1. Of the 1568 Page 6/15 participants, 55.8% were women and 44.2% were men. Six NCQA-PCMH questionnaires were collected from each community (effective rate = 100.0%).

Evaluation Of Primary Healthcare Service Quality of Institutions
As shown in Fig. 1 [insert Fig. 1.], according to the score of NCQA-PCMH, Shayuan CHC had the best service quality as a supplier, while Liede CHC had the poorest service quality. The scores of each section of the NCQA-PCMH are presented in Table 2. Overall, the two sections with the highest scores were PCMH3, Population health management (14.96) and PCMH6, Performance measurement and quality improvement (13.97); while the two sections with the lowest scores were PCMH1, Patient-centred access (8.02) and PCMH2, Team-based care (8.19). Associations between patients' experiences and NCQA-PCMH evaluation of institutional service quality As shown in Fig. 2[insert Fig. 2.], the PCAT scores of patients from the Level 3 CHC surrounded the PCAT scores of Level 1 and Level 2, which were signi cantly higher than those of Level 1 and Level 2 (p < .001).
In addition, except for dimensions D and F, the scores of Level 2 were signi cantly higher than those of Level 1 (p < .001). Furthermore, with the increase in NCQA-PCMH level, the PCAT scores of dimensions B, G, H, and K showed an obvious upward trend.
To evaluate the robustness of our estimates, we compared the items of PCAT and NCQA-PCMH. We found that there were many relevant items in the two evaluation questionnaires (Table S1).

Discussion
In the present study, we found that the global PCAT score was higher in the CHCs with higher NCQA-PCMH levels, showing positive dose-effect trends. This was similar for the PCAT sub-dimensions. It was indicated that effective service quality at CHCs may improve patients' health services experiences. Overall, our results provide additional evidence in support of better institutional primary healthcare service quality, leading to better experiences among patients. To our knowledge, this is the rst report on the association between PCAT and NCQA-PCMH globally.

Service quality of institution
Of the six CHCs participating in this study, three were NCQA-PCMH Level 1, two were Level 2, and only one was Level 3, indicating that the service quality of most of the institutions in this study still required improvement, especially concerning patient-centred access and team-based care. Some programmes can effectively help institutions transform into higher-level PCMHs in the United States. For example, a resident physician cooperation programme helps all 20 participating family medicine residency practices achieve NCQA-PCMH-recognised PCMHs, with 17 attaining Level 3 recognition. [16] NCQA-PCMH recognition may be a helpful measure to improve the institutional primary healthcare service in other countries, including China.

Patients' experiences
In this study, the dispersion trend of the scores for all dimensions of PCAT in SY CHC, rated as Level 3, was small; and the scores were signi cantly higher than those of other-level CHCs, which further indicated that the standardised community with the highest recognition level of NCQA-PCMH provided effective primary healthcare service, and the service quality was recognised by patients in all aspects. A, s compared to Level 3, the CHCs of Level 1 and Level 2 scored lower in dimension C, rst contact care in terms of access; D, ongoing care; and J, community orientation. The purpose of 'First contact care in terms of access' is to evaluate whether patients can contact a physician in time when they need medical and health services; while 'Ongoing care' evaluates the continuous relationship between physicians and patients in primary care institutions, and 'Community orientation' evaluates whether CHCs fully consider the needs of patients in the implementation of health services.
The requirements of these three dimensions are consistent with the responsibilities of family physicians. The World Organization of National Colleges, Academies, and Academic Associations of General Practitioners/Family Physicians notes that family physicians should have six core post-competencies: primary care management, patient-centred care, skills to solve speci c clinical problems, comprehensiveness of services available, community-oriented services, and the ability to provide comprehensive services. [17] The Chinese family physician system is still in its infancy. By the end of 2018, the total number of family physicians in Guangzhou was 4272, with 4.8523 million permanent residents and 1.9759 million key groups. [18] It has been suggested that the signing rate of family physicians in Guangzhou needs to be improved. Therefore, we should vigorously promote the family physician system and improve the signing rate. This can help to improve the satisfaction of patients in the dimensions of C, D, and J; and it is an effective way to provide accessible, continuous, and personalised service to patients. Concurrently, the service model of family physicians in China is similar to the American PCMH model: comprehensive, patient-centred, coordination, accessibility, and ensuring the quality and safety of medical services. [8] It has been suggested that the promotion of family physician contract systems is helpful for CHCs to develop to standardised communities, improve the quality of medical services, and enhance patient satisfaction.
The association of the service quality of the institution with patients' experiences The present ndings indicated that the better the NCQA-PCMH evaluation of CHCs, the higher patients' PCAT score. That is, the higher the service quality evaluation of the institution, the better patients' primary healthcare service experiences. Thus, as health service providers, the quality of community health institutions at the meso-level of the health eld has a great impact on the experiences of patients as users of health services at the micro level. This is consistent with the Institutional Analysis and Development framework proposed by Ostrom. [19] Therefore, if we want to improve the service experiences of patients, we need to make positive changes in the meso-level of the health eld. For example, referring to the NCQA-PCMH recognition system, the CHCs in China also conduct service quality evaluation and are recognised by levels. Then, according to different levels, corresponding improvement measures are put into action to encourage each CHC to develop to a higher-level CHC. Simultaneously, long-term quality management should be considered. NCQA requires the recognised PCMH to report annually to recon rm its recognition to determine whether it still maintains excellent service quality. [14] In this study, for the accessibility of PCAT C and PCMH1, the evaluation scores of each NCQA-PCMH level are low, and the contents of the two evaluation scales on the accessibility of medical services are very similar, which suggests that the evaluation of accessibility of medical services was consistent in the supplier and demand-side evaluation. Thus, the accessibility of medical services is the critical de ciency of CHCs in China. In 2015, a study on the relationship between PCMH practices and access to primary healthcare services showed that patients who called PCMH institutions were more likely to be assigned new appointments and get after-work appointments than non-PCMH institutions, indicating that PCMHrecognised institutions have a greater advantage in providing access to medical services and providing medical services in a more timely manner than do their counterparts. [20] This further supports that standard evaluation and recognition of CHCs may be helpful in improving service quality and improving the patients' health service experiences.
Although novel in geographic context and quality evaluation, our study had some limitations; therefore, the results should be interpreted with caution. First, as a cross-sectional study, we could not establish a temporal association between process and outcomes. Second, the use of self-reported data might be susceptible to reporting bias, leading to misclassi cation for some participants. Finally, the sample size was not large because only six CHCs participated.
Conclusion RQ L designed the study, collected and analyzed the data, and revised the manuscript. YF M and N H analyzed the data and drafted the manuscript. JL W participated in the data collection the statistical analysis. XW Y and LY S revised the manuscript. RW H conceptualized the study, arranged the eld investigation, collected the data, and revised the manuscript. RW H was the study's PI and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. All authors read and approved the nal manuscript.