A cross-sectional online survey, with validation of the Death Literacy Index informed and reported according to the COSMIN Study Design checklist for patient-reported outcome measurement instruments (15). The study protocol was pre-registered on the Open Science Framework (https://osf.io/fwxkh/).
Population and Settings
Participants were prospectively recruited via an online crowdsourcing platform managed by Prolific Academic Ltd (http://www.prolific.co). A nationally representative sample of participants representing the target population was recruited from the estimated 41,000 UK residents on the panel, stratified across age, sex and ethnicity in alignment with the proportions reported in the UK Office of National Statistics Census data (16). Prolific establishes the population strata, with a predetermined number of open slots into which eligible participants in the panel can enrol on a first-come basis. Inclusion criteria included: adults (≥ 18 years of age) currently living in the UK, and with capacity to express their opinion. Participants read a participant information sheet and provided explicit informed consent before completing the survey via Qualtrics online platform (17). Responses were collected between 19th October and 3rd November 2020. Median completion time was approximately 10 minutes. A small financial incentive was offered for completion, equivalent to £9.51/hour.
Measures included the Death Literacy Index (DLI; 13) alongside several measures to assess construct validity. Death literacy was expected to be positively associated with death competency, with the Coping with Death Scale (14) included to assess convergent validity, alongside items to assess i) objective knowledge and ii) actions regarding discussion of death and dying. A negative association was expected between death literacy and loneliness, with the Short Revised UCLA Loneliness Scale (18) included to assess discriminant validity. Lastly, information on socio-demographic characteristics were collected, including individual experience working/volunteering or with prior lived experience of death, dying and loss, to assess known group validity.
The Death Literacy Index (DLI, Version 1.0; 13)
A 29-item self-report measure of the construct of death literacy, with a higher-order factor structure composed of four subscales, two of which have two subscales; 1. Practical Knowledge (8 items) including the (i) ‘Talking Support’ subscale (4 items) and (ii) ‘Doing hands on care’ subscale (4 items), 2. Experiential Knowledge (5 items), 3. Factual Knowledge (7 items) and 4. Community Knowledge (9 items) including (i) ‘Accessing Help’ subscale (5 items) and (ii) ‘Support Groups’ subscale (4 items). Responses are on a 5-item Likert scale (from 1 to 5). Subscale scores are computed by summing items and scaling per number of items in subscale (with a range of scores between 0 and 10). Emerging evidence on the psychometric properties of the DLI in a community-based population in Australia is good (13), confirming structural, cross-cultural and construct validity, internal consistency, and interpretability. The measure has also been piloted in one UK community sample (Mildenhall, England as facilitated by St Nicholas Hospice). Leonard and colleagues in correspondence confirmed that in the UK community sample there were no items which participants found difficult or omitted. The scaled mean scores on the subscales/DLI total score ranged from 4.6- 7.5 with evidence of ceiling effects, and with good internal consistency (Cronbach’s alpha for the scale was .927 and sub-scales ranged from .794 to .904).
Coping with Death Scale (14)
A 30-item self-report measure of the construct of death competency. The scale assesses both one’s sense of competence in handling death and concrete knowledge concerning preparation for death. Participants are instructed to indicate the extent to which they agree with 30 statements using a 7-item Likert scale. Items are summed, with a range of scores between 30 and 210. The scale has shown good internal consistency and stability with various samples, as well as some evidence of construct validity in distinguishing hospice volunteers from controls and predicting death preparation behaviours (19). Cronbach’s alpha in the current sample indicates good internal consistency (30 items, α = .94).
Short Revised UCLA Loneliness Scale (18)
A 3-item self-report measure of the construct of loneliness. The scale measures three different aspects of loneliness, (social connectedness, relational connectedness, and self-perceived connectedness). Participants are instructed to indicate how often they feel that way with three statements, using a 3-item Likert scale (from 1 to 3). The items are summed. This is a widely used measure of loneliness, developed for large online surveys, and demonstrates good psychometric properties in relation to the full UCLA scale (20). Good internal consistency (3 items, α = .86) was reported for the current sample.
Objective knowledge items
Developed by the original DLI authors (13), this includes four items to measure the objective knowledge of the death system. An example includes ‘What is palliative care?’ (response options; Care received only by people in the last few weeks or days of life, Care for people aged over 85, Care that aims to improve the quality of life of people with a life-threatening illness). Participants provide categorical answers, and correct items are summed.
Actions regarding discussion of death & dying items
Developed by the original DLI authors (13), this includes two items to measure the attitudes and actions to discussion of death and dying. The items are ‘In my community we discuss death and dying’ and ‘In my family we discuss death and dying’. Participants provide answers using a 5-item Likert scale (from 1 to 5).
Sample size calculation
The sample size estimation was calculated on the basis of the factor analysis. Where factor structure is known a sample size of >200 is recommended (21). A sample size of n=399 meets multiple criteria, with some researchers recommending a sample size of at least 300 (22; 23) and others recommending participant to item ratios ranging from 5 to 10 participants per item (24), with any less than 3 participants per item deemed inadequate (25).
Research ethics approval was provided by the Queen’s University Belfast Engineering and Physical Sciences Faculty Research Ethics Committee (Reference; EPS 20_218) on 11th September 2020. The study was conducted in accordance with the Declaration of Helsinki and participants completed an informed consent statement prior to completion of the survey.
Data were exported from Qualtrics (17), and analysed using the Statistical Package for Social Science for Windows, Version 25 (SPSS Inc., Chicago, IL, USA), an alpha level of p<0.05 was considered statistically significant. The ordinal responses of the DLI were treated as continuous data. There were no missing data as forced responses were used in the survey. The scaled mean of the subscales is used throughout as recommended by the measure’s authors for benchmarking of population level scores, with raw scores used for assessment of interpretability.
The psychometric properties of the DLI were evaluated according to standard methodology as outlined by COSMIN (15; 26)
Dimensionality. The validity of the factor structure identified in the original scale development study (13) was examined in the current study by confirmatory factor analysis (CFA) using Structural Equation Modelling (SEM) in Amos version 23 (SPSS Inc., Chicago, IL, USA). Preliminary analysis to confirm the suitability of the data for factor analysis included inspecting the correlation matrix for at least several moderate-strong inter-item correlations (>.3) and for no perfect multicollinearity (<.9). Sampling adequacy was also assessed by the Kaiser-Meyer-Olkin (KMO) value (threshold >.6) and Barlett’s Test of Sphericity (significance at <.05). Preliminary analyses evidenced sampling adequacy for factor analysis with largely moderate inter-item correlations but no perfect multicollinearity with all inter-item correlations <.83. A KMO value of 0.92 and a significant Barlett’s Test of Sphericity, χ2 (435) =8150.66, p<.001 indicated suitability for factor analysis. Variance-covariance matrix with maximum likelihood (ML) estimation procedure was used for SEM, which is appropriate if there are more than three ordinal categories (27). Assumptions for ML include multivariate normality. The univariate normality of the variables was assessed by kurtosis and skewness values, with recommended thresholds of moderate non-normality of <2 for kurtosis and <8 for skewness (28). All the univariate skewness and kurtosis values were smaller than the recommended thresholds of moderate non-normality. At the multivariate level, multivariate kurtosis=148.37 with a significant Mardia’s coefficient of 34.95, with threshold of <5 indicating multivariate normality (29). This suggested univariate normality and a multivariate departure from normality. The data was inspected for multivariate outliers by Mahalanobis distance value. Removing five true outliers (substantial distances from other cases) reduced the multivariate kurtosis to 127.20 and Mardia coefficient to 29.772. In all subsequent analyses, 394 participants are the focus. The initial model specified was the 29 items of the DLI, loading onto a hierarchical structure with 8 factors. A second model with a new item developed for the UK context (under Factual Knowledge scale) was tested, as specified a priori in the study pre-registration. This item asks about the contribution of ‘funeral home staff’, in place of an item referring to the contribution of ‘cemetery staff’.
Model fit was assessed using a series of indices, according to best practice (30). A non-significant chi square goodness of fit test is indicative of a well-fitting model and was considered but is sensitive to sample size (27). Additional model fit indices used are the normed chi square (Q), the comparative fit index (CFI), the root mean square of approximation (RMSEA), and the standardised root mean square residual (SRMR). Cut-offs of fit indices include; Q; acceptable criteria vary from under 2 (31) to less than 5 (32); CFI: ≥ 0.90 and 0.95 reflect acceptable and excellent fit to the data, respectively (33). RMSEA and SRMR; values between 0.05 and 0.09 indicating adequate model fit and values <0.05 indicating a very good fit (34). Modification indices available in CFA have been used to identify misspecification in the model. Decisions regarding modifications were based on theoretical in addition to psychometric considerations of item and scale content. We planned to eliminate items if they had low factor loadings (i.e., standardized regression coefficients) (<0.40), or if modification indices suggested they had significant loadings (>0.30) with unintended latent factors (27).
Internal Consistency. After determining dimensionality based on theoretical assumptions and model fit according to standard criteria outlined above, items were evaluated for their psychometric properties. This involved examining the reliability of the unidimensional subscales separately by Cronbach's alpha and coefficient omega. Item to total correlations (r > .30 as a minimum criterion (35). A Cronbach’s alpha coefficient between 0.70 and 0.95 indicates good internal consistency without homogeneity (36).
Construct validity is the extent to which scores on an instrument relate to other measures (convergent validity/discriminant validity) or produce expected differences in scores between ‘known’ groups (known-groups validity). It is given a positive rating if at least 75% of the results are consistent with predefined hypotheses. Construct validity of the DLI was tested against items measuring people’s knowledge of the death system, a measure of death competence and for respondents identifying as having professional or lived experience of death, dying and loss. Pearson’s correlation coefficients or ANOVA were undertaken according to predefined hypotheses of convergent/discriminant validity. We define the strength of the correlation as strong (0.7–1.0), moderate (0.4–0.7), weak (0.2–0.4) and absent (0.0–0.2) (37). We define the strength of the ANOVA as small (Eta sq=.01), medium (.06) or large (.14) (37).
H1: Moderate positive association expected between an individual’s objective knowledge of the death system and the DLI and subscale scores.
H2: Moderate positive association expected between items of individual’s scores on the Coping with Death Scale (14) and the DLI and subscale scores.
H3: Moderate positive association expected between items of individual’s actions in relation to discussing death and dying and the DLI and subscale scores.
H4: Moderate positive association expected for individuals with experience working/volunteering or with prior lived experience of death, dying and loss and the DLI and subscale scores.
H5: Moderate negative associations expected between items of individual’s scores on the Short Revised UCLA Loneliness Scale (18) and the DLI and subscale scores.
Interpretability was determined by analysing the distribution of participants’ total scores (median, range, interquartile range), with floor and ceiling effect indicated if 15% of respondents achieved the lowest or highest possible score, respectively.
Objectives 2 & 3:
Descriptive statistics were used to provide a scaled mean score on the DLI and subscales. ANOVA were used to examine the relationship between demographic variables and DLI/subscale scores.