Design
The study used a methodological design comprising of a cross-cultural adaptation procedure to effectively translate the 51-item Swedish ATODAS instrument for use in other cultural and language settings. Specifically, the study considered Brislin’s multi-step approach as best practice (18). An additional Delphi technique procedure was performed on the Flodén ATODAI [North American version] as a complementary adaptation approach to secure higher scientific certainty of the instrument with regard to validity and reliability by testing the items for content relevance, clarity, and domain coverage.
Description of the ATODAS [Swedish version]
Flodén et al. (13) developed the Swedish 51-item ATODAS as a means of psychometric evaluation of measuring ICU nurses’ ATODA, including validation and reliability testing. In addition to the demographic data, the instrument contains three dimensions covering statements about attitudes toward: actions to safeguard the wishes of the potential organ donor; actions for supporting the family of the potential organ donor; and actions that promote OD at an organizational or structural level.
Translation and cross-cultural adaptation of the Flodén ATODAI [North American version]
The procedure to transfer the Swedish ATODAS instrument into an international arena was guided by Brislin's (18) multi-step back-translation approach, complemented by a Delphi technique procedure (Figure 1).
The first step was for a professional and native American English-speaking interpreter and bilingual Swedish translator to translate the Swedish ATODAS instrument into American-English. The second step comprised of back-translation into American-English, as performed by another expert—a native Swedish-speaking bilingual translator. The translation was performed blindly, i.e. without access to the original version of the Swedish ATODAS.
Step three constituted cross-language testing by the international committee, consisting of three OD specialists; one representing Sweden (PI), and two representing the United States of America (OneLegacy).
“Review and synthesis of the translations” was performed to provide consensus regarding the most accurate and easily understood items. Working from the original instrument, as well as the translated versions, a synthesis of these translations was conducted to produce a consolidated instrument. This validity-checking procedure ensured the translated version reflected the same item content as the original. A written report thoroughly documented the synthesis procedure by addressing each of the issues and how they were resolved. The consensus included the translated version of the instrument and the introduction and instruction to the instrument, resulting in the prefinal version of the Flodén ATODAI [North American version]. The described procedure included achieving equivalence between the original version and the translated version.
Study Populations: Steps Four to Six
Step four:
Seven designated ICU or CC nurses in the greater Los Angeles with experiential knowledge of caring for at least one organ donor formed an expert panel. The panel evaluated the content validity of the items, with reference to Lynn’s criteria (17). All seven nurses on the panel were female, aged between 29-55 years with a mean age of 44.2 years, and their work experience in the ICU and/or Emergency Department ranged between 5-31 years. The panel represented nurses from trauma, education, and teaching hospitals: Three worked in the ICU; three in the Emergency Department; and one in the Education Department. Three of the experts were managers/charge nurses, one was a clinical nurse specialist, one a nurse educator, and two were bedside nurses.
Step five:
In total, 50 ICU nurses from two hospitals in the greater Los Angeles area participated in the test and retest; one university-affiliated hospital (with different types of ICUs); and one county or community hospital (one ICU) (Table 1). The inclusion criteria were: Being an ICU or CC nurse; experiential knowledge of caring for at least one organ donor; and currently working in a clinical setting with OD. The exclusion criteria were: Being a nurse who was not currently working and/or being a nurse without experience of caring for organ donors.
Step six:
A preselected panel of 15 nurses in the United States of America, with extensive experiential knowledge of caring for organ donors, comprised the Delphi panel for the purpose of completing the additional Delphi technique procedure (Table 2).
Table 1. Socio-demographics of ICU nurses performing the test-retest of the 46-item Flodén ATODAI [North American version]
ICU or CC nurses
|
N=50
|
Age
|
25-63 years (mean 38 years)
|
Gender:
Men
Women
|
n=13
n=37
|
Work experiences
|
0.1-34 years (mean 10.2 years)
|
Experience of caring for brain-dead patients:
< 5 times
6-10 times
> 10 times
|
n=13
n=6
n=31
|
Table 2. Socio-demographics of panel (N=15) performing the Delphi technique procedure of the 46-item Flodén ATODAI [North American version]
Location:
Greater Los Angeles
Western and South United States
|
n=11
n=4
|
Age
|
23-60 years (mean 46.7 years)
|
Gender:
Female
Male
|
n=12
n=3
|
Ethnicity:
Asian
Caucasian
Afro-American
|
n=7
n=6
n=2
|
Current workplace:
Intensive Care (general)
Emergency Department
Trauma ICU
Cardiac ICU
Neuro ICU
|
n=8
n=4
n=1
n=1
n=1
|
Main position:
Bedside nurse
Charge nurse
|
n=14
n=1
|
Work experience in ICU
|
3-32 years (mean 16 years)
|
Hospital:
Community hospital
University hospital
Trauma hospital
Private hospital
|
n=12
n=2
n=1
n=1
|
Data Collections and Analysis: Steps Four to Six
Step four: First data collection
The first data collection required testing the prefinal version of the Flodén ATODAI [North American version]. The expert panel was given a rating form with the theoretical definition and a delineation of the three dimensions, objectives, and items. They were asked to review the prefinal 51-item version of the Flodén ATODAI [North American version] for content relevance, clarity, and domain coverage and to rate each item on a 4-point scale (from 1=not relevant to 4=very relevant) (17,19).
Step four: First data analysis
Content Validity
The expert panel was formed to estimate the content validity (with reference to Lynn’s criteria (17) of the items. Content was considered valid when an item was rated as either 3 (relevant and needs little revision) or 4 (very relevant) by at least six evaluators (>86%) and, thus, was included in the new scale (17).
The international committee analyzed the content validity rating by the expert panel and weighted the scores, which resulted with the prefinal 51-item instrument being reduced to 46 items. Of the remaining 46 items, five items were re-worded, as guided by the recommendations of the expert panel. A content validity index (CVI) was calculated to indicate the extent of expert agreement, both for the item CVI (I-CVI) and for the scale CVI (S-CVI). An I-CVI was determined by the number of experts who rated an item content as valid (giving it a rating of 3 or 4) divided by the total number of experts, resulting in a proportion of agreement for each item. The S-CVI was determined by the averages of the I-CVIs (17,19).
Step five: Second data collection
The study performed a test-retest procedure to estimate stability (reliability testing) of the adjusted version of the prefinal 46-item version of the Flodén ATODAI [North American version], as developed from the data analysis performed in step four. Fifty ICU nurses agreed to participate by answering the instrument on two occasions, with two weeks in between.
Step five: Second data analysis
Test-Retest Reliability
The intraclass correlation coefficient (ICC) was used to measure the strength of agreement between the test and retest, using ordered categorial data (20). The level of agreement was confirmed via the weighted form of kappa coefficients (ҡ Weight) (21).
Moreover, the sign test tested for whether systematical differences occur in either direction, described by exact agreement. The test was two-sided and conducted at the 0.05 significance level. The ICC, the ҡ Weight, and the sign test analyses were performed using SAS software version 9.4 (SAS Institute Inc., Cary, NC, USA).
Homogeneity and Stability Reliability
Homogeneity reliability was estimated using Cronbach’s alpha coefficient (α) via SPSS 18.0. According to the conventional rule by Nunnally, this coefficient should at least exceed 0.70 (22).
Step six: Third data collection
An additional Delphi technique procedure of the Flodén ATODAI [North American version] was performed to test the items for content relevance, clarity, and domain coverage. The Delphi technique is evidently dependent on the experiential knowledge of its expert panel (i.e. the Delphi panel). In this step, the Delphi panel only reviewed and judged the 22 items which were identified in step five as having an ICC <0.70. The preselected Delphi panel (N=15) was used to further improve the feasibility of the instrument (23). The Delphi panel members were asked to individually review and judge these 22 items in the Flodén ATODAI [North American version] in two occasions, referred to as “round I” and “round II”. The nurses rated each item using a 4-point rating scale (1=not relevant; 2=unable to estimate relevance without item revision or item in need of such revision that it would no longer be relevant; 3=relevant but needs minor alteration; 4=very relevant and succinct) (17). In round II, the Delphi panel members reviewed the adjusted version of the Flodén ATODAI [North American version] after round I.
Step six: Third data analysis
After round I, this study’s primary investigator (PI) summarized and analyzed all participants’ recommendations. The Delphi panel rated all 22 items as either “relevant but needs minor alteration [3]”, or “very relevant and succinct [4]”. Four of the items were recommended to be kept as they were. The PI, in consultation with the co-investigators, adjusted and re-worded the remaining 18 items, guided by the Delphi panel members’ recommendations. After round II, the PI, in consultation with the co-investigators, summarized, analyzed, and adjusted the Flodén ATODAI [North American version], again guided by the recommendations of the panel.