Validation of the primary care experiences survey developed with Indigenous patients with type 2 diabetes in Canada

Background Health disparities experienced among Indigenous populations in Canada arise from complex historical and social causes. Addressing health disparities requires a multi-sectoral approach, including measures for Indigenous patient healthcare experiences. The Canadian Institute for Health Information published Measuring Patient Experiences in Primary Health Care Survey to evaluate care experiences among general populations; it is not known whether this survey tool is appropriate to measure Indigenous people’s experiences of healthcare. As part of a large research project known as Educating for Equity (E4E), we developed an Indigenous patient experiences with diabetes care survey from existing validated primary care assessment tools, adapting items to the domains of the E4E Care Framework that addresses social and cultural factors that impact Indigenous health. This study describes the development of this Indigenous patient experiences survey, assessing the tool’s validity and reliability. Method A consensus approach was used to establish the face validity of the survey and ease of completion. The survey was administered in three on- and off-reserve practices in Northern Ontario during 2015 and 2016. In total, 154 completed surveys were collected for data analysis. Exploratory factor analysis was performed to assess the survey’s structure. Internal consistency, item convergent validity and discriminant validity were assessed to determine internal reliability and correlation of items within each scale and between scales. Results The Indigenous patient experiences survey consists of two sections: a patient demographic section of eight items and healthcare experience section of 36 items on Likert scales. Exploratory factor analysis identified seven main and sub-scales that align with the survey’s conceptual domains. Two items were removed due to insufficient loading scores. All main and sub-scales obtained acceptable internal reliability (Cronbach’s α: 0.7-0.957), item convergent validity (AVE: 0.4-0.7; CR: 0.78-0.99), and discriminant validity (√ AVE: 0.66-0.85). Conclusion The Indigenous patient experiences survey maps out the construct concepts of the E4E Care Framework: provider-patient relationship, provider cultural competence and social sensitivity, patient diabetic-related psychosocial self-efficacy, and patient-centred continuity of care. Study psychosocial self-efficacy among patients with diabetes or chronic diseases. The results of this study show that the revised IPES (plus T2DM) is a valid tool for measuring healthcare experiences of Indigenous patients with type 2 diabetes in Northern Ontario. We hope this survey tool has potential applicability to measuring Indigenous patient experiences with other chronic diseases beyond T2DM.

results indicate this culturally-tailored instrument is reliable and valid for measuring primary care experiences of Indigenous patients with diabetes in Ontario, Canada.

Participants
Patients were recruited from one on-reserve and two off-reserve family medicine clinics located in Northern Ontario. Patients were eligible to participate in the survey if they were 18 years of age or above, had type 2 diabetes, self-identified as Indigenous, and saw the same family physician during the study time from September 2014 to April 2016.

Survey administration and data collection
Prior to a clinical appointment, the practice receptionist identified patients who were eligible to participate in the survey, informed them about the study, and asked if they were willing to participate in the patient survey. If a patient was interested, the receptionist would provide an information sheet outlining the purpose of the study and a consent form with instruction to sign if agreeable to participation. The receptionist collected the signed consent prior to the clinical appointment.
Following the appointment, the patient was instructed to complete the survey. Onsite clinical support staff would provide assistance in reading and interpreting survey questions upon request. At the end of each day, the receptionist or office staff collected completed surveys and submitted them to the site research lead. All completed surveys were mailed via Express Post to a project team member (HH) in Ontario.

Data analysis
Data analysis was based on the shorter version of the IPES (plus T2DM) that was administered in the second round. The first round responses to the 23 questions that were removed for the second round were not included.
The purpose of the validation was to assess congruence between the proposed conceptual scales for the IPES (plus T2DM) and the underlying structure of this survey identified by empirical results.
Validation involved four steps: (1) Exploratory factor analysis (EFA) was used to explore the survey's structure. Principal factor analysis was performed to identify the factor structure and observe if the survey items fell into the conceptual domains based on factor loadings. [34] Three criteria were used to determine retained items and factors. First, an item with loading above 0.35, no secondary loading above 0.35, and its communalities extraction score above 0.4 was considered for retaining. Communalities extraction score between 0-0.4 suggests that the variables may struggle to load significantly on any factor and indicates the candidate variable for removal after examination of the pattern matrix. Second, each factor had a minimum three retained items. Third, all retained items under each factor should share the same concept or construct. [34,35] (2) Internal consistency reliability analysis was performed to assess the internal consistency of items within each scale by Cronbach's coefficient alpha (α). Cronbach's coefficient alpha is based on the covariance among individual items within a scale and the number of items. It ranges from 0 (indicating no internal consistency reliability) to 1 (indicating full internal consistency reliability), where the minimum accepted Cronbach's coefficient alpha is cut at 0.7. [34,36] In addition, item-total correlation, which is the relationship between an item and its scale, was observed. If an item-total correlation is small, then this item is not considered to be measuring the construct that is measured by other items in the same scale. In this study, the cutoff for item-total correlation was 0.3. [34] (3) Item convergent validity and discriminant validity were assessed to identify the correlation levels of each individual item with its conceptual scale and with other conceptual scales. Item convergent validity was observed by average variance extracted (AVE) and composite reliability (CR) [37][38][39] . AVE "reflects the overall amount of variance in the indicators accounted for by the latent construct". [38, p 612] The threshold for AVE value is 0.5. [37,38] CR estimates the extent to which a set of latent construct indicators share in their measurement of a construct, where the AVE is the amount of common variance among the latent construct indicators. [38] The acceptable CR value is set at 0.5. [38] Discriminant validity refers to the extent to which factors or constructs are distinct and uncorrelated. [38,39] Discriminant validity can be determined by comparing the square root AVE of a latent construct and its correlations with other constructs. Discriminate validity is indicated if the square root AVE value of a construct is greater than the correlation values of this construct with other constructs. [39] (4) Descriptive statistics were performed to identify participating patients' demographics, including gender, age, education, household income, and health status. As well, based on the validation results, descriptive statistics were performed on the retained items to describe the property of the revised IPES in terms of means, standard deviation, range, percentile, and skewness and kurtosis.

Participants' demographics
Between rounds 1 and 2, a total of 154 participants completed the IPES (plus T2DM). They were on average 50 years of age, with 72.1% women. Over half of participants (52.6%) were married or living with a partner, 23.9% were separated, divorced or widowed. Over half (52.6%) had education up to high school, close to 41% had post-secondary education or some college level. Only 2.6% received university education. Close to 41% of participants had annual household incomes under $20,000, around 14% of participants between $20,000 and $40,000, and 20% of participants between $40,000 and $60,000. About 17.6% of participants had an annual household income above $60,000.
Although over half of the participants (54.6%) self-rated their health condition from good (40.9%), very good (11.7%) to excellent (1.95%), most participants reported living with one or more types of chronic disease: 64.9% had hypertension, 31.2% had depression and anxiety, 28.6% had arthritis, 22.7% had heart diseases, and 29.9% had other chronic conditions.

Statistical validation results--Psychometric properties of the IPES (plus T2DM)
Exploratory Factor Analysis Results KMO value (0.876) and Bartlett's Test of Sphericity value (p=0.00) indicated that the data collected from the Indigenous patient experience survey were adequate for factor analysis. Principal component analysis was performed on the 36 survey items on Likert scales. Eight (8) factors that had Eigenvalues above 1.0 were extracted. However, the rotated (Promax with Kaiser Normalization) pattern matrix showed that two factors failed to obtain sufficient retained items (a minimum of three items with loading >0.35) under each. When the number of factor extraction was suppressed to six, one factor still did not have an adequate number of retained items. When the number of factors was suppressed to five for extraction, each factor obtained an adequate number of retained items. These five retained factors explained a total of 63.9% of the variance (see Table 3). Table 3 illustrates the item loadings and factor structure; 34 out of 36 survey items were retained and loaded onto five factors. Items 29 and 31 had loadings lower than 0.35, and their communalities extraction scores were below 0.4, [34] indicating that that they had very low variance explained by any factors and should be considered for removal from the survey.

The pattern matrix in
The factor structure showed a close correspondence to the proposed conceptual domains and concepts of the IPES (plus T2DM) (see Table 4). Factor #1 was loaded with 12 items that reflected two sub-scales of concepts: (a) interpersonal communication (9 items), including the provider giving enough time during visit, listening to the patient, eliciting patient concerns, expectations and preference for care, and explaining tests, treatment, and self-care; (b) doctor-patient relationships (3 items), including providing the patient with emotional support and having patient trust. Items 4 and 10 were related to discussion of care plan during office visit, and the factor loadings group them with the communication scale. Factor #2 was loaded with six items on the concepts of patients with diabetes self-efficacy. Factor #3 was loaded with eight items on two subscales of concepts: (a) holistic and continuing care (5 items), and (b) physician cultural sensitivity (3 items). Factor #4 was loaded with four items on the concept of decision making. Factor #5 was loaded with four items on the concepts of physician social sensitivity (discrimination). This indicates the validity of the construct structure embedded in IPES (plus T2DM).
Internal Reliability of the IPES (plus T2DM) Scales Table 5 presents the results of internal reliability analysis of each scale and sub-scale and the descriptive results of survey items. Cronbach's alpha (α) and item-total correlation scores indicated internal reliability of each scale. The Cronbach's α of seven scales (including subscales) ranged from 0.7 to 0.9, indicating that these scales had acceptable and strong internal consistency. The revised item-total correlation scores ranged from 0.31 to 0.85, also indicating from acceptable to strong relationship between the items and their affiliated scales. The original scale of "Social Sensitivity (Discrimination)" had five items (25)(26)(27)(28)(29), one of which, item 29, showed very low internal consistency and weak relationship with the scale (Item-Total correlation =0.04). When item 29 was deleted, the Cronbach's α of the scale increased to 0.7, and the revised item-total correlation scores of other items improved, ranging from 0.31 to 0.60. Also, item 31 was weak in internal consistency and correlation with the scale "Diabetic Patient Self-Efficacy" (Item-Total Correlation =0.13); when this item was deleted, the Cronbach's α of this scale increased to 0.9, and the revised item-total correlation scores of the other six items improved, ranging from 0.47 to 0.83.
Item convergent validity and discriminant validity Table 6 presents the convergent validity and discriminant validity of the revised survey based on the factor loadings and Cronbach's α of each scale. All scales obtained high CR scores (0.78~0.99), indicating strong composite reliability within each scale. The AVE scores of four main scales --Interpersonal Communication/Relationship, Decision Making, Social Sensitivity and Patient Self-Efficacy --were at and above 0.5 (range 0.5 ~ 0.7). The AVE and CR sores of these four scales indicate from acceptable to good convergent validity of the items affiliating to their own scales. The AVE of the scale "Holistic Care and Cultural Competence" was slightly lower than 0.5; however, combining its strong CR score (0.86), this scale might still have acceptable convergent validity.
In Table 6 Competence) are close to 0, ranging from -0.117 to 0.534, indicating a close normality of data distribution on these scales. [40] The Kurtosis values for Social Sensitivity (Discrimination) and Diabetes Patient Self-Efficacy are greater than 3, indicating non-normality of data distribution on these two scales. [40] Discussion The IPES (plus T2DM) adopted the scales and items from the CIHI's Measuring Patient Experiences in Primary Health Care Survey, used to measure the care experience of Canadian general populations, adding the Diabetes Empowerment Scale-Short Form (DES-SF) that has been validated to measure diabetes patients' self-management in the United States. Our EFA results identified five major scales with two sub-scales. The concepts of these scales and sub-scales were empirically consistent with the In total, these psychometric features confirm that the scales and sub-scales used in our survey have adequate content validity, construct validity, internal consistency and reliability.
The development and validation of IPES (plus T2DM) partially answers the health-related Call for Actions of the Truth and Reconciliation Commission (TRC). [41] Call #19 asks "to establish measurable goals to identify and close the gaps in health outcomes between Aboriginal and non-Aboriginal National and international researchers have identified disparities in Indigenous care and cultural and social determinants related to Indigenous health. [9-11, 32, 33, 42-44] Efforts were made here to explore what to measure in Indigenous patient healthcare experiences, and how. [45] The validated IPES (plus T2DM) is a contribution to this pool of knowledge. It has shown capable of capturing the experiences of Indigenous patients living with diabetes in Canada, and it can be adapted to measuring Indigenous patients living with other chronic diseases. For example, the "Diabetes Patient Self-Efficacy" scale can be replaced with a validated measure tool for any other chronic condition. The IPES (plus T2DM) can also be attuned to specifics within Indigenous contexts, such as connecting to traditional healing and treatment as are mentioned in TRC Calls #21 and #22. [41] This tool can help physicians or potentially wider clinical efforts to be meaningfully responsive to Indigenous patient needs and priorities and bring quality of care for Indigenous populations.

Conclusions
The IPES (plus T2DM) was culturally tailored to the healthcare experience of Canadian Indigenous populations. It captures Indigenous experiences in physician-patient relationships, physician cultural competence and social sensitivity, patient-centred decision-making, continuity of care, and psychosocial self-efficacy among patients with diabetes or chronic diseases.       Notes. AVE value>0.5; CR value>0.5.  Figure 1 The E4E Care Framework reveals social and economic resource disparities experienced by Indigenous people and provides two directives for clinicians in providing care, namely: 1) recenter clinical relationships (e.g., address power and authority), and 2) engage with patient social realities (e.g., social barriers, cultural facilitators) as opportunities for change. Each directive is grounded in cultural facilitators for addressing critical social variables arising from the legacy of colonization, which impact both the patient's world and healthcare relationships.

Supplementary Files
This is a list of supplementary files associated with this preprint. Click to download.