Study design
This study has a mixed qualitative-quantitative design, using a methodological approach to validating a measurement instrument. Qualitative, because it aims to establish the cultural validation of the OHAT questionnaire, contemplating phases of translation and evaluation of the coverage and façade of the scale. The quantitative approach is analytical and relational.
Recruitment of participants
The participants were people aged 60 years or more, proceeding from residences or groups for the elderly, health care groups for prostrated and hospitalized patients from different institutions in the La Araucanía Region of Chile, during years 2019 and 2020. Excluded from the study were older people presenting difficulties regarding responsiveness or the ability to follow simple instructions in the context of the application of the instrument, given a situation of severe dependence, dementia or impaired level of consciousness, which were determined via clinical records, the application of the Barthel Index and the Pfeiffer Short Portable Mental State Questionnaire.
Instruments
a) For the sociodemographic characterization and subsequent descriptive statistical analysis, a general information record of the participants was drawn up, which included gender, age, diagnosis of pathologies such as dementia, diabetes or hypertension, level of dependence and type of nutrition, among others.
b) Oral Health Assessment Tool (OHAT): Measurement instrument, which on this occasion was administered by a speech-language therapist. It is made up of eight items (lips, tongue, gums and tissues, saliva, natural teeth, dentures, oral cleanliness and dental pain), the answers to which are organized according to a Likert scale from 0 to 2 points, where 0 indicates absence of oral health issues (healthy) and 2 suggests possible disease (unhealthy). The score of each item reflects a description of the observed structures (healthy, signs of possible disease and unhealthy). For its application, a professional with competence in the discipline or formal training is required. Flashlight, gloves and mask should be used in the case of hospitalized patients or people with poor oral hygiene of orofacial structures. The instrument can also be applied to persons with cognitive alterations but who are responsive to simple instructions (18).
Sample size
The estimation of the sample size required to achieve structural validity was conducted following the criteria proposed by Streiner et al. (28), consisting of 10 individuals participating per item of the measurement instrument, with a minimum of 200 persons when the number of items is small. Consequently, a non-probabilistic convenience sampling was implemented, with a total of 286 participants recruited.
The test-retest reliability estimation was carried out as proposed by Donner et al. (29), with an Intraclass Correlation Coefficient (ICC) of 0.6 as the acceptable minimum and 0.8 expected, a significance level of α = 0.05 and a power (1-β) of 80%, for two measurements with a dropout rate of 10%. In this case, the calculated minimum sample was 49 persons. In this study, 76 persons were finally included for the purpose of this analysis.
Procedure
This study was carried out in two phases. In the first place, the original version of the OHAT was translated into Chilean Spanish and adapted to its culture. In the second phase, its psychometric properties were assessed in a sample of older people. This study received the approval of the scientific ethics committee of the Universidad Católica de Temuco, under resolution No. 40/20. The participation of the subjects was completely voluntary or authorized by a family member or tutor, and they were at liberty to drop out of the study without this involving any detriment to the daily care provided in their respective facilities. Consequently, each participant or tutor signed an informed consent form, evidencing their free and voluntary participation in accordance with the principles of the Helsinki Decalogue (30).
1) Cultural adaptation:
First of all, authorization to adapt the instrument was requested from the Iowa Geriatric Education Center by e-mail. The cultural adaptation consisted of the following (31):
a) Direct translation: undertaken by two independent bilingual translators whose mother tongue is Chilean Spanish. The first addressed the study blind and the second was informed of its objective. In addition to translating, they identified comprehension and translation problems arising from semantic elements that were difficult to understand or confusing. After this, the translators got together to analyze their texts, detect discrepancies between them and produce the consensus version. They were also asked to maintain the conceptual equivalence of terms rather than a literal translation, when necessary.
b) Reverse translation: Translation of the consensus version produced by the bilingual translators back to the original language. This was carried out by a bilingual speech-language therapist whose mother tongue is English. This person rated each of the translated items in terms of (1) Semantic/conceptual equivalence (maintaining most of the linguistic-semantic terms as expressed in the original translation); (2) Functional equivalence (grammatical modification of the original idea, maintaining the conceptual equivalence) and (3) Non-evident equivalence (major departure from the concept). Whenever any translated word or phrase fell into the third category, an alternative tending toward equivalence or a justification for the change was reached by consensus by the experts.
c) Consolidation and final production of the instrument. A committee was formed made up of the translators who generated the consensus version, a speech-language therapist trained in geronto-geriatrics and a dentist, who were presented the two initial versions provided by the translators, the consensus version and the reverse translation submitted by the speech-language therapist. Discrepancies regarding the translation of the instrument were discussed in terms of quality of translation, maintenance of the linguistic or functional equivalence and the modifications associated with contextual pertinence made to arrive at the final instrument.
d) Pre-test: Three speech-language therapists applied the OHAT to a group of 30 persons. After this they were asked for feedback, to allow them to identify difficulties experienced with regard to understanding some item(s) of the instrument, or aspects related to the instructions, semantics, grammar or comprehension regarding the type of answer required.
2) Collection of data for structural validity
To carry out this procedure, the participants were first required to answer a brief questionnaire in order to collect information regarding their sociodemographic background. After this, the Short Portable Mental Questionnaire and the Barthel index were used to complement the general data. Once the base characteristics and eligibility of the subjects had been checked, the OHAT instrument was applied. Given the diversity of contexts, its application took place in a speech-language attention booth in the respective physical space of the participating centers or in the residences themselves, safeguarding the lighting conditions and absence of distractors, and the delivery of clear (protocolized) instructions by the evaluator, who was trained for this purpose.
3) Procedure for obtaining evidence of reliability
After the first application of the OHAT, the participants in the study were asked to answer it on a second occasion, within a maximum period of 7 days, to determine the test-retest reliability.
4) Statistical Analysis
A sociodemographic characterization of the study population was carried out, using descriptive statistics, specifically central tendency measures and dispersion for quantitative variables, and absolute and relative frequencies for categorical variables.
To achieve the objective regarding the structural validity, a Confirmatory Factor Analysis (CFA) (32) was carried out to corroborate the factor structure of the sub-dimension “Disease and condition status”, which is part of oral health as reported by the literature (1). In this case, given that the instrument variables are ordinal in nature, the Diagonally Weighted Least Squares (DWLS) estimator was used, since the data are ordinal and do not meet the assumption of multivariate normality (33). For model assessment, several model fit indices were calculated, including the Comparative Fit Index (CFI), the Tucker-Lewis index (TLI), the Root Mean Square Error of Approximation (RMSEA), in addition to the Standardized Root Mean Square Error of Approximation (SRMSEA). For CFI and TLI, values greater than or equal to 0.95 were considered acceptable (34). In the case of RMSEA, values less than 0.06 and for SRMSEA, less than 0.08 (35).
To obtain evidence of test-retest reliability, the Intraclass Correlation Coefficient (ICC) was estimated with its respective 95% reliability interval, in order to determine the degree of consistency between measurements. In this context, a two-way mixed-effects model was used to achieve measures of “absolute agreement”. Although no consensus was reached regarding the interpretation of this coefficient, some guidelines were established. For the purposes of this study, values higher than 0.75 were considered acceptable reliability values. Values between less than 0.75 and 0.5 were considered moderately reliable and those below 0.5 insufficiently reliable (36).
The internal consistency reliability was obtained through ordinal Cronbach’s Alphaand Mcdonald’s Omega (37). Values higher than 0.7 were considered acceptable for the factor hypothesized.
The above-described processing and subsequent statistical analysis were carried out using Rstudio software, version 1.4.1743-4.