Reliability and Validity of a Free Oral Pre-exposure Prophylaxis (PrEP) Adherence Questionnaire Among Patients with Sexually Transmitted Infection and Without HIV Infection in Shanghai


 Objective: In this study, we aimed to evaluate the validity and reliability of the free oral pre-exposure prophylaxis (PrEP) adherence questionnaire among patients with sexually transmitted infection who do not have HIV infection in Shanghai. Methods: We conducted a cross-sectional study among 816 patients from sexually transmitted infection (STI) clinics in Shanghai. The questionnaire included self-reported demographic characteristics, items on adherence to free oral pre-exposure prophylaxis (PrEP), and PrEP uptake behavior measurement. Item analysis was carried out by calculating the critical ratio and item-total correlations. Internal consistency and split-half reliability were measured to assess reliability. Content validity, construct validity, convergent validity and discriminant validity were measured to assess validity. We conducted logistic regression with receiver operating characteristic (ROC) curve analysis to assess the predictive value of the adherence questionnaire for PrEP uptake behavior. Results: All items were considered acceptable in the item analysis (P<0.05). The questionnaire had a Cronbach’s alpha of 0.757 and split-half reliability of 0.633. The scale-level content validity index (CVI) was 0.938 and the item-level CVI of each item ranged from 0.750 to 1. In exploratory factor analysis, we introduced a four-factor model accounting for 79.838% of the aggregate variance with all factors loading more than 0.4, which was validated in confirmatory factor analysis (c2=47.1, degrees of freedom=14, root mean square error of approximation=0.078). According to regression analysis, adding PrEP adherence questionnaire scores contributed to prediction of PrEP uptake behavior (P<0.001). The maximum area under the ROC curve was 0.778 (95% confidence interval: 0.739–0.817), corresponding to a cutoff score of 9.Conclusion: The PrEP adherence questionnaire generally has acceptable reliability and validity. This questionnaire can be used as a tool to evaluate PrEP adherence in patients with STI who do not have HIV infection.


Introduction
Since acquired immunode ciency syndrome (AIDS) was rst reported in 1981, it has quickly become a global epidemic. Knowledge of the epidemiological characteristics of AIDS has evolved considerably.
Populations at high risk for acquiring human immunode ciency virus (HIV) infection have shifted to sexually active groups, including people with sexually transmitted infection (STI). 1,2 Whereas universal antiretroviral therapy for people living with HIV is recommended and does reduce mortality, it is a treatment and not a cure. 3 Fortunately, the antiretroviral drug oral emtricitabine/tenofovir has been introduced as pre-exposure prophylaxis (PrEP), to help reduce HIV transmission.
Before the advent of PrEP, public health measures to prevent HIV infection such as condom use, antiretroviral therapy, male circumcision, and regular HIV testing of high-risk populations achieved great success in preventing HIV transmission. 1 However, the development of PrEP has brought AIDS prevention to another level, and its e ciency and safety has been proven in many clinical trials. [4][5][6][7] The initial clinical trials of PrEP have yielded different estimations of e cacy, which may be explained by varying population adherence. Two studies failed to demonstrate e cacy of PrEP, the Vaginal and Oral Interventions to Control the Epidemic (VOICE) trial with 29% population adherence, and the Preexposure Prophylaxis Trial for HIV Prevention among African Women (FEM-PrEP) trial with 37% population adherence. 8,9 Compared with other clinical trials, such as the Preexposure Prophylaxis Initiative (iPrEx) trial and the Partners Preexposure Prophylaxis (Partner PrEP) Study, where population adherence was above 50%, a low degree if adherence without e cient measures may explain the ndings of the VOICE and FEM-PrEP trials. 4,10 Nevertheless, drug adherence has an important role in the effectiveness of PrEP among people at high risk for HIV infection. 11,12 Various measures have been introduced in recent clinical studies to improve adherence, such as use of short message service, drug level testing. and electronic medicine containers with automated monitoring that remind patients to take their medication. 13 Nevertheless, unlike clinical trials in which drug adherence can be monitored and a more supportive environment is provided by researchers, medication adherence is di cult to predict; however, optimizing adherence is closely tied to PrEP e cacy. It is important to investigate how best to optimize adherence for populations with a high risk of acquiring HIV and to explore individual-level measures to ensure medication adherence. Simple, effective, user-friendly methods, such as questionnaires, are needed to evaluate individual adherence. In our previous study among transgender women sex workers, we assessed the acceptability of daily use of free PrEP, based on the theory of planned behavior; this formed the basis of a questionnaire developed to evaluate PrEP adherence. 14 Whereas many studies have assessed PrEP uptake and adherence as well as the sustainability of providing PrEP services for men who have sex with men or transgender women, patients with STI require equivalent attention, as an identi able population with a high-risk of acquiring HIV infection. 10,14−22 In research by Galvin, the presence of an STI could increase the infectivity and susceptibility of HIV via various biological mechanisms, causing patients with STI to be more vulnerable to HIV infection. 23 In a national review of the HIV infection epidemic trend in China, although the incidence of HIV infection was low among patients with STI, more HIV infections have been caused by heterosexual contact. 2 Great bene ts can be derived from promoting the use of PrEP and improving medication adherence among patients who are vulnerable to becoming infected with HIV.
Although the AIDS epidemic has remained at a low level in China, the overall incidence of AIDS is showing a gradual upward trend. 2,24 What's more, in light of the three UNAIDS targets for AIDS prevention and control by 2020, 25 improvement is needed in China, for which PrEP is a powerful resource. Therefore, the risk of HIV infection among patients with STI and without HIV infection should not be ignored.
However, there is much uncertainty related to PrEP medication adherence among patients with STI. In this study, we aimed to evaluate the reliability and validity of a free oral PrEP adherence questionnaire for assessing PrEP adherence among patients with STI, and to explore the predictive accuracy of the questionnaire. Our ndings can provide suggestions for future improvements in the prevention of HIV infection among the population with STI and other populations with greater risk of HIV infection.

Participants
We recruited outpatients and inpatients who visited STI clinics on Wednesdays and Saturdays at two branch institutes of Shanghai Dermatology Hospital in the Jingan district (located on Qiujiang Road and Baode Road, respectively) from November 2017 to May 2018.
The inclusion criteria were according to the latest revised STI prevention and management measures in China, and included patients aged over 16 years old with syphilis, gonorrhea, condyloma acuminatum, and genital herpes. All diagnoses (including clinical and laboratory diagnoses) were conducted by doctors in the hospital.
The exclusion criteria were as follows: (1) infected with HIV (2) serious psychological or cognitive diseases; (3) unconscious status; (4) unwilling to cooperate with researchers; (5) vision or hearing loss or poor reading ability, leading to little understanding of the purpose and content of this research.

Data collection
Our research team recruited all physicians working at STI clinics in two cooperative Shanghai Dermatology Hospitals. The researchers were in charge of training all doctors in how to select patients who met the inclusion and the exclusion criteria of this study. After training, doctors were responsible for informing eligible patients about the purpose and content of this research and determining their willingness to participate. The interviewer team comprised senior undergraduate or graduate students recruited from Shanghai Jiao Tong university school of medicine who had experience with administering questionnaire surveys. The interviewer team was responsible for conducting face-to-face interviews with patients. All interviews were conducted twice a week over nearly 7 months.
In each interview, one member of the interviewer team invited patients one by one to a quiet room, so as to keep patients focused on completing the questionnaire. The interview comprised three phases; during the rst phase, patients were provided with an informed consent form to sign. Interviewers answered all patients' concerns in principle. During the second phase, patients were required to complete the questionnaire on their own; no private information was collected during this process. In the third phase, interviewers veri ed the completed questionnaire to ensure no items were missed or ignored. At the end of the interview, patients receive a cash stipend (80 RMB, nearly 12 USD) in consideration of their cooperation in this study.

Measurement
The questionnaire included self-reported demographic characteristics, free oral PrEP adherence questionnaire, and PrEP uptake behavior items, to evaluate patients' adherence to PrEP 14 . Free oral PrEP adherence questionnaire used in this research is original, and eight items were included, based on factors found to be associated with the acceptability of free oral PrEP in our previous team work. 14 Those eight items were used to measure participants' attitudes, subjective norms, and perceived behavioral control. Behavioral intention was measured using an additional question separate from the questionnaire, to predict the actual performance of each individual. Considering the conciseness and clarity of the questionnaire items, four subscales were established: bene ts, barriers, peer support, and self-e cacy. Eight items were included, with two items for each subscale. Each item on the questionnaire was scored using a three-point scale: 0 points for "disagree", 1 point for "unclear", and 2 points for "agree"; the total score on the questionnaire was 16 points. Higher scores indicated higher adherence to PrEP. Both positive and negative questions were included, to reduce participants' response bias to a certain extent. (full questionnaire was shown in additional le 1) Participants were asked the following question apart from the adherence questionnaire: Would you be willing to take PrEP on a daily basis during the following 6 months? There were ve responses, corresponding to different degrees of PrEP uptake willingness or behavioral intention (answers: 1 = de nitely not, 2 = unlikely, 3 = neutral, 4 = likely, 5 = de nitely will). Given that not all behavioral intentions translate into action, 26 we selected "de nitely will" as the standard predicting positive behavior, in a conservative manner.

Statistical analysis
For item analysis, we used the critical ratio and Spearman correlation coe cient to test and evaluate the relevance and reliability of all items. In the calculation of critical ratio, participants were divided into two groups based on their total adherence scores: a high-adherence group (participants ranked among the 27% with the highest scores) and a low-adherence group (participants ranked among the 27% with the lowest scores). We conducted a t-test for scores of each item in the high-and low-adherence groups. A Spearman correlation test was conducted between each item score and the total score. For reliability analysis, Cronbach's alpha coe cient was used to evaluate the internal consistency of the questionnaire, and the Spearman-Brown split-half coe cient was used to evaluate reliability of the questionnaire.
A panel of eight experts specializing in population health and health behavior was invited to conduct the content validity analysis, scoring the relevance of each item to the corresponding content concept, the simplicity and clarity of each item, and the ambiguity of each item. In addition, the experts gave their opinions, based on a four-point scale, in evaluating the questionnaire using the content validity index (CVI), including the item-level CVI (I-CVI) and scale-level CVI (S-CVI); For construct validity, factor analysis was performed. Samples were randomly divided into two groups.
Group 1 (406 patients) was used for exploratory factor analysis (EFA) and Group 2 (410 patients) for con rmatory factor analysis (CFA). In the EFA, Bartlett's test of sphericity and the Kaiser-Meyer-Olkin (KMO) value were applied for eligibility; principal component analysis was adopted to extract common factors. In the CFA, the maximum likelihood method was used to verify the tness of the initial model.
Goodness-of-t indices such as c 2 , degrees of freedom (df), p-value, root mean square error of approximation (RMSEA), goodness-of-t index (GFI), adjusted goodness-of-t index (AGFI), and comparative t index (CFI) were adopted.
The value of average variance extracted (AVE) and construct reliability (CR) were used to evaluate convergent validity. We used the square root value of AVE and related analysis results to evaluate discriminant validity.
For predictive value and score translation, univariate logistic regression, hierarchical logistic regression, and receiver operating characteristic curve (ROC) analyses were carried out to assess the predictive value of adherence questionnaire scores in discriminating patients with positive behaviors (high-adherence group) and negative behaviors (low-adherence group), and to determine the cutoff score.

Participant demographic and adherence-related characteristics
A total of 816 participants were recruited in this study. The mean age of participants was 38.56 years, standard deviation 13.00 years. The mean adherence score was 8.79, standard deviation 2.61. Among the 816 participants, 134 (16.42%) were aware of PrEP as a preventive measure for AIDS, and 11 patients (1.35%) had taken PrEP before the study. Table 1 shows participant demographic and adherence-related characteristics and adherence scores between the groups. The total adherence score was signi cantly different according to age, education level, marital status, income, sexual orientation, and knowledge about PrEP (Table 1).

Item analysis
Except for item 3, all items had a critical ratio value higher than 3, with statistical signi cance. Except for items 3 and 4, all items had a Spearman correlation coe cient greater than 0.3, with statistical signi cance. Items 1, 2, and 5-8 were acceptable in the item analysis (Table 2). PrEP, pre-exposure prophylaxis; CR, critical ratio.

Reliability
Cronbach's alpha of total PrEP adherence for internal consistency was 0.757 and the Spearman correlation coe cient of total PrEP adherence for split-half reliability was 0.633, which were acceptable (Table 3).

Content validity
The S-CVI was 0.938, and the I-CVI for each item ranged from 0.750 to 1. In general, the content validity of the questionnaire was acceptable.
Four common factors with eigenvalues greater than 1 were extracted and accounted for 79.838% of the aggregate variance, which was in accordance with our theoretical four-factor structure. No item loads were less than 0.4. Whereas item 6 loaded in two subscales, it loaded heavier in factor 3 (0.728) than in factor 4 (0.455); therefore, item 6 was classi ed into factor 3 ( Table 4).
Con rmatory factor analysis (CFA) After EFA, a four-factor structural model was adopted for CFA, which included the following factors in uencing PrEP adherence among target participants: bene ts, barriers, peer support, and self-e cacy.

Convergent and discriminant validity
All subscales had a CR value and AVE value greater than 0.7 and 0.5, respectively, except for subscale support; however, the CR value of subscale support was nearly 0.7 (0.691), so the convergent validity was acceptable. The square root AVE value of each subscale was greater than the maximum correlation coe cient between the subscale and other subscales; therefore, discriminant validity was acceptable (Tables 6 and 7).  Predictive value and score translation The score of the PrEP adherence questionnaire was a statistically signi cant predictor of PrEP uptake behavior in the initial univariate logistic regression analysis (odds ratio=1.585, P<0.001). In hierarchical logistic regression, entry of the adherence questionnaire score improved prediction signi cantly (P<0.001) ( Table 8). PrEP, pre-exposure prophylaxis; The ROC curve of the self-report PrEP adherence questionnaire score to determine participants' actual PrEP uptake behavior and the area under the ROC curve (AUC) suggested that the PrEP adherence questionnaire performs well in distinguishing patients with high adherence and low adherence. The maximum AUC value under the curve was 0.778 (95% con dence interval: 0.739-0.817), corresponding to a cutoff score of 9 with a speci city of 72.24% and sensitivity of 74.39% (Figure 1).

Discussion
Since the introduction of PrEP as primary prevention for HIV transmission, its e cacy has been evident in rigorous clinical trials. 4,10 Nevertheless, it is important to maximize the use of PrEP to best prevent HIV transmission. Adherence is one factor that has an impact on PrEP e cacy. 1 The aim of this research was to assess the reliability and validity of the self-report PrEP adherence questionnaire among patients with STI who did not have but were at high risk of HIV infection. Our results demonstrated that the PrEP adherence questionnaire had acceptable reliability and validity and could serve as a useful measurement to evaluate differences in PrEP adherence among patients with STI.
We conducted item analysis and evaluated reliability and validity in this study. In item analysis, items 3 and 4 showed disqualifying results; however, the subscales of items 3 and 4 describe the factors that hinder adherence to PrEP, including drug side-effects and HIV stigma. The in uence of these two factors on PrEP has been shown in previous studies. 18,27,28 In Golub's research, PrEP stigma was found to be highly associated with HIV stigma as PrEP is designed to prevent HIV infection. 28 People tend to relate HIV stigma to concerns about being seen taking medicine by family or friends, and such concerns substantially impact adherence. 19,29 In our design of item 4, we mainly considered concerns from sexual partners, which could represent PrEP stigma to some extent. Moreover, the critical ratio value and correlation coe cient of items 3 and 4 were statistically signi cant; therefore, we retained those two items. The relatively lower critical ratio value and correlation coe cient of items 3 and 4 may result from the language or expressions used, which may require more speci city and clarity. Another issue that may account for this situation was that clinical side-effects and HIV stigma were not the main barriers to PrEP among patients with STI. Further investigation into barriers among the population with STI is required. In reliability analysis, results for internal consistency and external reliability showed that the questionnaire is su ciently stable and has acceptable reliability. In validity analysis, we rst carried out EFA. According to the questionnaire design, four common factors had to be extracted during this process. Through principal component analysis, four common factors with an aggregate variance contribution rate up to 79.838% were obtained, indicating that the variance caused by the four common factors can well explain variation in the variables measured in the questionnaire. In addition, the load of each item on the subscale to which it belongs was above 0.7, demonstrating that the classi cation of each item to its corresponding subscale was consistent with our design purpose. Furthermore, CFA indicated that the theoretical model, divided according to the designed four subscales, could well explain the actual data; the above results with acceptable convergent and discriminant validity indicated that the questionnaire has good structural validity. Furthermore, the results of logistic regression and ROC curve analysis demonstrated that the predictive value of the questionnaire has su cient accuracy in practice. When the adherence score obtained by patients with STI reaches 9, we believe that the patient would perform well, with regular PrEP uptake and good adherence. The same methods for evaluating the predictive value of indicators have been used previously. [30][31][32][33] Use of the PrEP adherence questionnaire provides the opportunity to evaluate patients' adherence in advance, such that a patient with low adherence can be detected prior to starting PrEP therapy. Individualbased measures can be used to improve adherence or to detect the need to use prevention measures other than PrEP, which could be more suitable for an individual and yield greater bene ts. An improved PrEP adherence questionnaire considering more in uencing factors has been greatly needed.

Limitations
Owing to the fact that the test-retest reliability of the questionnaire was not measured during the reliability assessment, the integrity of the reliability assessment was reduced to some extent. In addition, we could not directly observe and track participants' drug use behavior in the real world, on the basis of which to determine their adherence. An additional question was asked to assess patients' behavioral intention as representing actual behavior; conservative prediction was adopted simultaneously to offset the uncertainty created during this process. The study population was not randomly sampled, resulting in insu cient sample representation, and the selected participants were limited to patients with STI. Owing to these limitations, whether our PrEP adherence questionnaire could be applicable to other populations and locations outside Shanghai is unknown. However, a total 816 participants were included in this research, which represents the population with STI in the survey area to some extent.

Conclusion
The reliability and validity of the PrEP adherence questionnaire is generally acceptable. The questionnaire includes four subscales: bene ts, barriers, peer support, and self-e cacy, with two items for each. The present PrEP adherence questionnaire can be used as a tool to evaluate PrEP medication adherence in patients with STI. The purpose and potential risks of this study were indicated in the informed consent forms presented to eligible participants, as was a guarantee of personal privacy. All consent forms were signed voluntarily by the included participants, who were informed of their right to opt out at any time during the research.
Compensation of 80-yuan (about $12) was provided to each survey respondent for their participation.
During the entire process of dealing with participants, we strictly complied with the principles of the Declaration of Helsinki. This research obtained approval from the Shanghai Jiao Tong University School of Medicine Public Health and Nursing Ethics Committee (approval number: SJUPN-201702).

Consent for publication
Not applicable Availability of data and materials