The German version of the self-efficacy questionnaire (SE-12-G) in a sample of healthcare professionals: Translation and psychometric properties

doi:10.21203/rs.3.rs-4836626/v1

Download PDF

Research Article

The German version of the self-efficacy questionnaire (SE-12-G) in a sample of healthcare professionals: Translation and psychometric properties

https://doi.org/10.21203/rs.3.rs-4836626/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

Effective and patient-oriented communication is essential for healthcare professionals (HCPs) to deliver high quality care. Assessing communication skills training effectiveness relies on validated measures, such as the Self-Efficacy Questionnaire (SE-12). Yet, a validated German version is lacking. Thus, we aimed to translate and adapt the SE-12 into German and assess its psychometric properties.

Methods

We translated the original SE-12 into German using the team translation protocol, adapted it to our context and added a subscale on importance of communication skills, resulting in the SE-12-G with two subscales (confidence and importance-scale). We conducted cognitive interviews with six HCPs to assess the comprehensibility of the SE-12-G. Afterwards, n = 152 HCPs completed the SE-12-G at two measurement time-points. We descriptively analyzed completion rate as indicator for acceptance, reliability (Cronbach’s α) and item characteristics (i.e., item difficulties, corrected item-total correlations, inter-item correlation). A confirmatory factor analysis was performed including three a priori hypothesized models including one that represents the factor structure performed by the authors of the original version of the SE-12.

Results

Translation and comprehensibility assessment of the German version of the SE-12-G showed high face and content validity through the cognitive interviews. Completion rate exceeded 98% for all items. Mean item difficulty across all items and subscales was 0.79, (ranging between 0.71 and 0.97), inter-item correlations ranged between 0.02 and 0.73 for the confidence-scale and between 0.044 and 0.533 for the importance-scale. Ceiling effects were present for half of the items on the confidence-scale, and for all items on the importance-scale. Internal consistency yielded a Cronbach’s α of 0.88 for the confidence-scale and 0.83 for the importance-scale. Confirmatory factor analysis supported a one-factor structure for both subscales of the SE-12-G.

Conclusion

The translated and adapted German version of the SE-12 shows good acceptance and reliability. Similarly to the original version, we found high ceiling effects in some items. Compared to the original factors structure, in our sample a one-factor structure was identified for both subscales. The SE-12-G should be evaluated in further studies and modification of some items should be considered.

Trial registration:

DRKS-00015794

Psychometrics

self-efficacy

communication

healthcare professionals

medical education

cancer

oncology

communication skill training

family-centered care

Effective and patient-oriented communication is essential for healthcare professionals (HCPs) to deliver high quality care to patients and their relatives (1, 2). Good patient-provider communication can foster trust and rapport (3–5) (and can also improve patient-relevant outcomes (e.g., satisfaction (6), health (7), understanding (8, 9), stress (10)). Moreover, it can reduce HCPs’ stress experience and emotional burnout and increase job satisfaction (1, 8). Reviews show that HCPs can be trained to increase their communication competencies through communication skills trainings in different clinical settings (1, 11, 12). One central evaluation outcome of communication skills trainings is HCPs’ self-efficacy in communication skills (12–14). In context of healthcare communication, self-efficacy can be defined as one’s belief in one’s own communication skills to handle certain situations when communicating with patients and/or their relatives.

Despite the importance of HCPs’ self-efficacy for the evaluation of communication skills trainings (12), only a few validated self-reported measures exist (15). The majority of measures relies on self-constructed questionnaires with limited validity and reliability (12).

A short and specific measure of self-efficacy, that is applicable to various clinicians, is the self-efficacy questionnaire (SE-12) developed by Axboe et al. (15). The SE-12 is available in Danish, English, Greek, Korean and Spanish (16–18) and measures self-efficacy of clinicians in their clinical communication skills. The original version is a 12-item, unidimensional instrument developed to assess self-reported self-efficacy in physicians and nurses in Denmark before and after participating in a communication skills training. The 12 items represent the key communications skill components of the Calgary Cambridge Guide (19). Axboe and colleagues (15) found the SE-12 to be both comprehensive and comprehensible and a reliable and partly valid instrument, to assess the self-efficacy of clinical communication skills. The original SE-12 has been psychometrically tested by Axboe and colleagues, revealing high internal consistency with a Cronbach’s α of 0.95 (range, 0.94–0.95) and acceptable test-retest reliability for the complete scale, with an ICC agreement of 0.71 (range, 0.66–0.77), with ceiling effects evident in 9 of the 12 items (15). Since its development, the SE-12 has been used, translated or been adapted for various studies (16, 20–22). Two of these studies added an additional scale to assess the perceived importance of each item (19, 20). Currently, there is no psychometrically tested instrument in German to assess HCPs’ self-efficacy in communicating with patients.

Therefore, we translated and adapted the original SE-12 scales into German and investigated the psychometric properties of the translated German version (SE-12-G) in a sample of German HCPs.

Study design and setting

The presented study is a secondary analysis of data from a randomized controlled pilot-study focusing on communication in the context of parental cancer (23, 24), which was conducted at the Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Germany. The study was registered within the German Clinical Trial Register (DRKS-00015794) and approved by the Local Psychological Ethics Committee of the Center for Psychosocial Medicine, University Medical Center Hamburg‐Eppendorf, Germany (LPEK‐001).

The original SE-12 measure

The SE-12 measures self-efficacy of clinicians in their clinical communication. The original version is a unidimensional instrument with 12 items using a 10-point response scale (1: very uncertain, 10: very certain) to measure confidence in their communication skills including an additional „not relevant“ option (15). Each question starts with the words “How certain are you that you are able to successfully …” followed by a specific communication skill (15). Sum score of the 12 items yields the SE-12 scale with higher scores indicating higher self-efficacy of clinical communication skills. The SE-12 has been adapted by adding a further scale measuring the perceived importance of each item by including a 5-point-scale (1: not important at all to 5: very important) (20) (19), so far without reporting on the psychometric properties of this additional scale.

Translation and adaptation of the SE-12-G

In order to assess self-efficacy of healthcare professionals in their communication competencies, the original SE-12 questionnaire (15) was translated into German following the TRAPD team translation protocol (25). The TRAPD (Translation, Review, Adjudication, Pretesting and Documentation) method has been endorsed within the Guideline for Best Practice in Cross-Cultural Surveys (26) complying with best practice translation research (27). An Excel spreadsheet was developed to assist with the translation and review process for each member of the research team and to document the process. Two team members (WF having a bilingual health science and physical therapy background, MLN a psychology background) independently translated the English version of the SE-12. A third blinded team member (CB) with experience in survey translation reviewed these translations, selecting one version or creating a new one as needed. Finally, all authors discussed the translations, until a consensus on a final version of the SE-12-G was reached. As a next step, the newly developed SE-12-G version was pretested by conducting cognitive interviews to assess comprehensibility and feasibility of the measure. Additionally, we adapted it by including a second scale, the perceived importance-scale, rating the same 12 items on a 5-point-Likert-scale (‘How important is it to incorporate this skill into your clinical work? 1: not important at all to 5: very important), with an additional check box ‘not relevant’ for both subscales all together (see Table 3 for the English items of the SE-12-G and Additional file 1 for the German SE-12-G).

In context of the randomized controlled pilot-study (23), cognitive interviews about the applied series of measures were conducted with HCPs working in oncology to assess the comprehensibility and feasibility in the context of content validity. During the cognitive interviews the think-aloud method was combined with the verbal probing technique (28) to review opinions on certain phrases, distinct words and their meanings for various HCPs and certain skills in daily practice (e.g., comprehensive probing “What does the term ‘non-verbal behavior’ mean to you?” or selection probing “When would you choose the answer ‘not relevant’”?). Interviews were audio-recorded and supported by handwritten protocols. After the interviews comments and suggestions from the material was discussed (WF, LJ, LI, CB), however no further adaptions to the SE-12-G were necessary.

Psychometric assessment

Participants and data collection

As this study is a secondary analysis of data from a randomized controlled pilot-study, study participants were part of a sample of various HCPs in Germany participating in this pilot- study. HCPs were included if they were working in oncology (independent of setting, profession or amount of professional experience in oncology) (23). They were recruited by e-mail or mail through existing networks, clinics and lists of cooperation partners. Participants did not receive any incentives for participation and participation was voluntary. In the pilot-study, HCPs received a paper & pencil baseline questionnaire including the SE-12-G and were randomized by group stratification after returning the baseline questionnaire. The post-training questionnaire (t1) was sent after training and in the waitlist-control group 6 weeks after the return of the baseline questionnaire.

The SE-12-G was applied in a series of measures to assess HCPs’ competencies regarding child- and family-related issues in cancer care. Additionally, demographic variables (e.g., gender, age, profession, work experience with cancer patients) and experience with previous communication skills trainings were assessed.

Data collection started in September 2019 and ended in April 2021. Data were entered into SPSS (IBM SPSS Statistics, V.27) including blinded double entry of 20% for quality control.

Data analyses

Descriptive statistics using frequencies for categorical data or mean values and standard deviation (SD) for metric data were calculated to describe the sample. In the following data analysis strategies will be reported, where Table 1 gives a detailed overview on established criteria used to interpret performed data analyses.

Item analysis was performed for each subscale including calculation of item means and standard deviations (SD), corrected item-total correlations and inter-item correlations. Observed floor and ceiling effects were assessed by analyzing the distribution of participants marking the highest as well as lowest possible score per item and scale. The maximum score was 10 for the confidence-scale and 5 for the importance-scale. Item difficulty was assessed for each of the 12 items on both subscales. Internal consistency was estimated by calculating Cronbach’s coefficient alpha for each scale (confidence- and importance-scale). Test-retest reliability was assessed using the data of two measurement points from participants from waitlist-control group only, calculating Spearman correlation coefficient at item level for each of the 12 self-efficacy items for each scale, and calculating Pearson correlation for the sum score of the subscales. Content validity was assessed by exploring whether the SE-12-G reflects on characteristics of participants, namely “working experience in general” and “working experience with cancer patients” (both in years), similar to Axboe et al (15). For this purpose, we tested the relations of the confidence-scale and importance-scale to these two self-developed items by using bivariate correlations. To assess responsiveness to change, Cohen’s d was used to assess change in both intervention groups, analyzing the effect size of the intervention, complemented by an additional t-test to determine significance. For discriminant validity, known-group differences (Kruskal-Wallis test) comparing the four HCP groups at baseline were calculated followed by post-hoc tests using the Dunn's test with Bonferroni correction for multiple comparisons. For these analyses, various data were used: for the responsiveness to change analysis, data from both intervention groups for the two measurement points (baseline and post-training questionnaire) were used; for the test-retest reliability, data from the waitlist-control group at two measurement points (baseline and 6-weeks after baseline) were used; for all other analysis only the baseline data was applied; the completion rate was calculated to assess the acceptance of the measure; the frequencies of missing data were calculated per item as well as for the overall measure and all cases; for all other calculations, missing data were replaced with item means. Cases were excluded if more than 30% of the SE-12-G items were missing (29).

To evaluate the factorial validity of the SE-12-G, we conducted a confirmatory factor analysis (CFA) (30, 31) (32). To test assumptions for performing a factor analysis, Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity were calculated (33, 34). Three a priori models were hypothesized: Model 1, replicating the one-dimensional factorial structure of the original SE-12 measure for the confidence-scale only (M1); Model 2, assuming a two-factor structure with variables from both subscales (confidence and importance) treated as continuous variables and accounting for residual covariance between each item and scales to address methodological variance (M2); and model 3, an ordinal factor model (M3a) assuming a two-factor structure with both subscales, treating importance variables as ordinal due to the non-normal distribution of the data. The third model was further fitted to check modified indices and possible correlations between items on content-level (M3b) (e.g., item 3 (… to encourage a patient to express and discuss their problems and concerns.) and item 5 (… to express their thoughts and feelings.)).

For M1 and M2 full information maximum likelihood estimates were applied (35), for M3 (a and b), the diagonally weighted least squares method, which is specifically designed for ordinal-scaled data with robust estimation (36), was performed. Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy with values > .05 and the Bartlett’s test of sphericity with values < .05 were conducted to test criteria for calculating a CFA (33, 34).

To evaluate the global fit indices, the following were included: (a) the comparative fit index (CFI) and the Tucker-Lewis index (TLI); (b) the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR); and (c) average variance extracted (AVE) (33, 37, 38). Further, factor covariance was explored to understand the relationship between the two latent factors confidence in and importance of these 12 communication skills.

Analysis of demographic data, analysis of completion rate, item analysis was performed using SPSS (IBM SPSS Statistics, Version 29.0.1.0). CFA and model of fit indices calculations were performed using JASP (JASP Team, 2023 Version 0.18.1), an open-source software, whose analyses are written in R.

Table 1

Psychometric analyses performed
Psychometric analyses	Criteria/Description
Item Response Distributions	Evaluates the distribution of responses for each item to identify patterns or irregularities.
Inter-Item Correlations	Assesses the degree of relationship between pairs of items. Values too close to 1 may indicate redundancy; too low values suggest lack of coherence. High inter-item correlations of above 0.80 indicate that items ask the same questions and might be redundant (33, 39).
Corrected Item-Total Correlation	Measures the relationship between each item and the total score excluding the item. Higher values indicate good item consistency. Generally, values > 0.3 are considered acceptable, representing items measuring the same underlying concepts. Items with < 0.30 should be considered to be removed (40).
Floor and Ceiling Effects	Identifies items where a significant portion of respondents score at the lowest (floor) or highest (ceiling) possible value. Cut-off values of > 15% of respondents are considered to be high (41).
Item Difficulties	Quantifies the proportion of correct responses to an item. Ranges from 0 (most difficult) to 1 (easiest). Item difficulties are calculated by dividing item means by the maximal value of the answer range (0–4) and multiplying it with 100. Item difficulty should be near to 50%, and items should not differ much in their difficulty level (33, 42) (34)
Cronbach’s α	Assesses internal consistency; values ≥ 0.70 are considered acceptable for early stages of research, with ≥ 0.80 preferred for established scales (40).
Kaiser-Meyer-Olkin (KMO) Measure	Values > 0.6 suggest adequate sample size for factor analysis (33).
Bartlett’s Test of Sphericity	Tests the hypothesis that the correlation matrix is an identity matrix, indicating suitability for factor analysis. p < 0.05 indicates appropriateness (33).
Comparative Fit Index (CFI)	CFIs is an indicator for model fit. It ranges from 0 to 1 and higher values indicate better fit. Values above .95 indicate a good model fit (37, 43).
Tucker-Lewis Index (TLI)	Values > 0.90 suggest a good fit of the model to the data in confirmatory factor analysis (37).
Root Mean Square Error of Approximation (RMSEA)	Values ≤ 0.05 indicate a close fit, values up to 0.08 represent reasonable errors of approximation in the population (44).
Standardized Root Mean Square Residual (SRMR)	Values ≤ 0.08 are generally considered good, indicating small residuals between observed and model-implied covariances (37).
Average Variance Extracted (AVE)	Represents the average of the squared factor loadings for all observed variables associated with a particular construct with recommended values of ≥ 0.5 (45, 46).

Translation and adaptation of the SE-12-G

Both translators (WF and MLN) and the reviewer (CB) did not differ much in their translations of the items as well as the response scale. Translation differences were found for the introduction (how to best translate the word “skills”) as well as single items, namely item 2 (agenda vs. themes), item 7 (adding e.g., to the examples), item 8 (by using a German synonym for empathy), item 11 (the word “shared” was translated differently) and item 12 (the word “assuring” was translated differently). Additionally, the response format word for “not relevant” differed between both translators. In summary, only the choice of single words differed between the translators without differences in meaning (e.g. the word “patient” also represents the word “client”, which is often used in psycho-oncology, psycho-social and psychotherapy settings). Within the first round of team discussion (adjudication process) all authors reached consensus on a final version.

To test the German version of the SE-12 for comprehensibility, cognitive interviews with n = 6 HCPs (n = 2 physicians, n = 2 nurses, n = 2 psychologists; 50% being female) were conducted by assessing the complete evaluation tool developed for the pilot-study (Johannsen et al 2023). Cognitive interviews for the complete 13-page baseline questionnaire including a series of measurements lasted for about 75 minutes each. As participants had no critical feedback or comprehensibility issues on the instructions and items of the SE-12-G, no further adaptions to the final version were necessary. The final SE-12-G measure used in this study can be found in the Additional file 1, the English version is displayed in Table 3.

Psychometric assessment

Sample characteristics

Data of n = 152 participants were included in this secondary data analysis. The mean age was 44.4 years (SD 11.6) with 88% being female. Most of the participants were psychologists (37.5%), followed by physicians (26.3%), nurses (18.4%) and social workers/others (17.8%). Of their professional experience working with cancer patients, most had > 11 years of experience (39.1%), followed by 1–5 years (30.5%), 6–10 years (21.9%) and 8.6% less than 1 year (see Table 2).

Table 2

Sample characteristics at baseline (t0, n = 152)
	Total (n = 152)
Age, M (SD) [range]	44.42 (11.6) [24–71]
Sex, n (%)
female	134 (88.2)
male	18 (11.8)
Professional group, n (%)
Physician	40 (26.3)
Nurse	28 (18.4)
Psychologist	57 (37.5)
Social Worker/Other	27 (17.8)
Workplace setting^†, n (%)
outpatient	94 (61.8)
inpatient	84 (55.3)
self-employed/registered	25 (16.4)
other	8 (5.3)
Working experience with cancer patients, n (%)
< 1 year	13 (8.6)
1–5 years	46 (30.5)
6–10 years	33 (21.9)
> 11 years	59 (39.1)
Amount of cancer patients per month, No. M (SD) [range], %	67.4 (33.0) [.1-100], n = 143
Amount of cancer patients between 25–55 years old, M (SD) [range] in %	37.52 (22.21) [.1-100], n = 131
Amount of cancer patients parenting minor children, M (SD) [range] in %	21.4 (21.9) [0-100], n = 123
Marital status, n (%)
single	54 (35.8)
married	82 (54.3)
separated/divorced	11 (7.3)
registered partnership	1 (0.7)
widowed	3 (1.9)
Participants having children, n (%)	97 (63.8)
Having participated in communication skills trainings before, n (%)	134 (88.2)

Abbreviations. M: Mean, SD: Standard Deviation; ^†Multiple answers possible

SE-12-G item analysis

Table 3 shows response distribution, distribution of participants with highest score (floor effects), acceptance, corrected item-total correlation and item difficulty of the 12 items.

Depending on the item, between 5.3% and 6.6% of the participants rated the respective item as “not relevant”. Missing values ranged from 1.3-2% per item. Considering all items, participants answered more than 98% of the SE-12-G.

Ceiling effects were present, for the confidence-scale in 6 of the 12 items (items 3, 4, 5, 7, 8, 10) with a range from 15.8% (item 10) to 30.3% (item 8) of respondents, exceeding the >15% set as limit (41) and therefore indicating high ceiling effects. Regarding the importance-scale, all 12 items present ceiling effects with a range from 54.6% (item 9 and 11) to 85.5% (item 8).

Corrected item-total correlation values for the confidence-scale ranged from 0.46 (item 4) to 0.66 (item 3), for the importance-scale values ranged from 0.34 (item 1) to 0.62 (item 10) with Items 3 for the confidence-scale and item 10 for the importance-scale suggesting stronger relationships between the items and the construct. In contrast, item 4 for the confidence-scale and item 1 for the importance-scale indicate weaker associations with the construct with correlation values of <0.30.

The mean item difficulty (Table 3) across all items was 0.79 for the confidence-scale (range 0.71-0.87) and 0.91 for the importance-scale (range 0.84-0.97) indicating that, on average, items tended to be relatively easy for participants.

[add Table 3 around here]

Inter-item correlations (Table 4) for the confidence-scale ranged from 0.019 (item 4 and item 11) to 0.733 (item 3 and item 5), for the importance-scale from 0.044 (item 5 and item 8) to 0.533 (item 2 and item 10), indicating that item 4 and 11 lack coherence.

[add Table 4 around here]

Internal consistency and test-retest reliability

Cronbach’s alpha coefficient for SE-12-G was high with a calculated α of 0.88 for the confidence-scale and α 0.83 for the importance-scale.

The test-retest reliability at item level for the two subscales was good with estimated correlation coefficient for the 12 items for the confidence-scale being r=0.725 with a 95% Bootstrap Confidence Interval (BCI) of 0.484-0.874 and for the importance-scale estimated correlations of r=0.726 with a 95% BCI of 0.479-0.862 (Table 5). For the confidence-scale no item showed a weak correlation, 5 items showed a moderate correlation of 0.430-0.491 (item 2, 3, 7, 11, 12) and 6 items showed strong correlations ranging from 0.538-0.755 (item 1, 4, 5, 6, 8, 9, 10). For the importance-scale, two items had a strong correlation (item 4 r=0.605, item 5 r=0.632), 8 items a moderate correlation ranging from 0.327-0.421 (item 1, 2, 3, 7, 8, 9, 10, 12) and two items a weak correlation (item 6 r=0.160, item 11 r=0.226).

[add Table 5 around here]

Convergent validity

Statistically significant but weak correlations were found between the confidence-scale and the items working experience in general (r=0.203**, range 0.043-0.350) and working experience with cancer patients (r=0.147**, range 0.085-0.406). No significant correlations were found for the importance-scale for working experience in general (r=.035, range -0.116-0.191) and working experience with cancer patients (r=0.147, range -0.093-0.250).

Responsiveness to change

The Cohen’s d for the confidence-scale was 0.77 (95% CI [4.0, 5.5]), indicating a moderate to large effect size, suggesting responsiveness for change after an intervention (Table 6). For the importance-scale Cohen’s d was 0.25 (95% CI [-.167, .275]) indicating a small effect, suggesting that this subscale shows limited sensitivity in detecting changes over time.

Table 6. Responsiveness to change

	N†	Mdiff (SD) [95% CI]	T	df	p	d [CI 95%]
Scale	N†	Mdiff (SD) [95% CI]	T	df	p	d [CI 95%]
Confidence-scale t1-t0	80	3.68 (0.77) [3.5, 3.8]	42.727	79	< 0.000	0.77 [4.0;5.5]
Importance-scale t1-t0	79	0.01 (0.25) [-0.4, 0.07]	0.475	78	0.636	0.25 [-.167,.275]

Abbreviations: † Intervention participants only (face-to-face and E-Learning, two measurement points, baseline (t0) and after-training participation (t1); Mdiff, mean difference (t1-t0); SD, Standard Difference; CI, Confidence Interval; T, df, degrees of freedom; p, p-value; d, Cohen’s d effect.

Discriminant validity

Analyses of group differences between HCP groups revealed statistically significant differences among the groups for several items of the confidence-scale (e.g., urge the patient to express his or her problems/worries, listen attentively to the patient, demonstrate appropriate non-verbal behavior, show empathy, make a plan based on shared decisions) and for two items in the importance subscale (listen attentively to the patient, encourage the patient to express thoughts and feelings). Most differences were between physicians and psychologists as well as nurses and psychologists with psychologists scoring higher in self-efficacy (Table 7).

[add Table 7 around here]

Factor analysis

Requirements for factor analysis were met, with an adequate sample size of n=149 HCPs related to the indicators per factor (47), after three cases were excluded due to >30% missing items. Additionally, further requirements were met, with KMO measure at .0.843 and Bartlett’s test of sphericity yielding X²=829.190, p<.001 (33) (34). Model fit indices of the three a priori models are presented in Table 8. Factor reliability was satisfactory for all three models (M1: ω 0.827; M2: ω 0.891; M3a: ω 0.839 confidence and ω 0.882 importance), and factor loadings for all 12 items for both subscales were medium to high (M1: between 0.477 and 0.799 for the confidence-scale only; M2: 0.395-0.751 for both subscales; M3a: 0.405-0.810 for both subscales; M3b: 0.410-0.837)(see Figure 1 for M3b and Additional file 2 for figures of M1-3a), indicating that the presented items (communication skills) measure the underlying construct (confidence in and importance of these skills). Factor covariance for M2 and M3a was 0.426 and for M3b 0.432, indicating a moderate correlation between the two factors, affirming their distinctness as separate constructs.

[add Figure 1 around here]

Figure 1. Confirmatory factor analysis model for one-factorial structure (model 3b)

Regarding the model fit indices, values indicated a suboptimal fit of the three assumed models (Table 8). The convergent validity of all three models was low, with higher, but still inadequate AVE values for M3a and M3b for both factors (Table 8), indicating that the latent constructs do not adequately capture the variance in its observed indicators (38).

Table 8. Fit indices for the SE-12-G questionnaire.

Model	χ2	df	P value	CFI	TLI	RMSEA [90% CI]	SRMR	AVE
M1	268.346	54	<.001	0.712	0.648	0.163	0.106	0.364
M2	579.761	239	<.001	0.776	0.741	0.097	0.092	C: 0.396 I: 0.331
M3a	366.631	235	<.001	0.888	0.868	0.061	0.098	C: 0.357 I: 0.497
M3b	376.144	235	<.001	0.880	0.859	0.063	0.099	C: 0.359 I: 0.489

Note: M1 = Axboe et al Model, only confidence-scale; M2 = The model which included confidence- and importance-scale for each 12 items applying continuous variables; M3a = The model which included confidence- and importance-scale for each 12 items, applying ordinal variables for the importance-scale; M3b = like M3a but additional modifications for assumed residual covariances for items 3 and 5 as well as 9 and 10.

Abbreviations: χ2, chi-square; df, degrees of freedom; AVE, average variance extracted; CFI, comparative fit index; CI, confidence interval; RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual; TLI, Tucker–Lewis index.

The present study aimed to translate the original SE-12 by Axboe et al (15) into German and assess its psychometric properties. The SE-12-G instrument proved to be a comprehensible and reliable tool and showed acceptable validity despite moderate fit indices.
Translation and assessment of comprehensibility as part of content validity

The translation process yielded a linguistically sound German version of the SE-12-G. Comprehensibility testing through cognitive interviews with healthcare professionals confirmed the clear understanding of the questionnaire, leading to no further adaptations. Inclusion of more participants with various backgrounds (e.g., various migration and cultural backgrounds) for testing the face-validity of the SE-12-G would provide information on the need of further adaptations.

Analyses of SE-12-G items, reliability and validity

The item analysis revealed generally high response rates and a low rate of missing values, indicating good participant engagement and the SE-12-G to be well-accepted by various HCPs.

Corrected item-total correlations are mostly above 0.66, indicating that most items measure the same underlying concept. Inter-item correlations are all above 0.30 indicating that all items are relevant and none should be deleted (40). Criteria for good item difficulties are met as mean item difficulty was 0.79 for the confidence-scale and for the importance-scale 0.91 and all values are above 50% highlighting the overall ease in responding (34). However, multiple ceiling effects were present. This is in line with previous studies applying the original SE-12 confidence-scale only (15, 18, 19), indicating that especially the importance-scale might be redundant. High ceiling effects may impede the identification of potential training effects in evaluations of communication skills trainings for HCPs. Nevertheless, the evaluation study by Johannsen et al (23) found statistically significant improvements in the confidence and importance scores of the SE-12-G after training participation compared to the waitlist-control group at post-training assessment, supporting the SE-12-G sensitivity to detect change over time. As our sample comprises a large proportion of psychosocial HCPs and participation in a communication skills training was motivation for participation, further application of the SE-12-G in mixed samples of HCPs should be conducted to assess ceiling effects. Regarding the internal consistency, the SE-12-G shows excellent values and an overall good test-retest reliability, which indicates that the questionnaire was consistent over the two measurement points (48, 49).

To our knowledge there are no measures that are related to the SE-12-G and could have been used to assess same or similar constructs. Therefore, we used latent constructs similar to Axboe and colleagues to explore the convergent validity of the SE-12-G. Our results show weak correlations between the confidence-scale and working experience (in general and with cancer patients), which is contrary to the findings of Axboe et al (15). However, subsequent studies including other measurements, e.g., Attitudes towards Medical Communication (18), should be conducted to assess convergent validity. Regarding the discriminant validity, results indicate that HCPs’ confidence (d=0.77) and importance (d=0.25) changed after a communication skills training. Known-group differences revealed various significant differences between different HCP subgroups in our sample in responses for specific items on the confidence-scale of the SE-12-G. Especially psychologists differed significantly and more often from physicians and nurses on the confidence-items. These findings highlight potential distinctions in perceptions among HCP groups regarding communication skills, which may be explained by different education, focus of care and time with the patients (e.g. psychologists and physicians).

Factor validity

We a priori hypothesized three models for the SE-12-G: a one-factor model replicating the original structure (M1), a two-factor model with continuous variables and residual covariances to account for methodological variance (M2), and an ordinal two-factor model to address non-normal data (M3a), which was then fitted further to check modified indices and possible correlations (M3b). A confirmatory factor analysis confirmed the one-factorial structure of the original SE-12 with the confidence-scale only (M1), but fit indices were of no good fit. We tested additional a priori models, which slightly improved the model fit (especially M3b), but still could not find acceptable values. As the convergent validity of all three models was low, with higher, but still inadequate AVE values for M3a and M3b for both factors (Table 8), values indicated that the latent constructs do not adequately capture the variance in its observed indicators (38). Yet, employing AVE for convergent validity relies on rules of thumb rather than statistical testing procedures, neglecting sampling errors and limiting the generalizability of conclusions to broader populations (50). Therefore, we prefer M3b (CFI=0.880, TLI=0.859, RMSEA=0.063 (95%CI)), but recommend to explore the CFA again with a.) a larger and more heterogenic sample (51) and b.) additional modifications (e.g., the exclusion of items with high ceiling effects).

Additionally, after further analysis for the SE-12-G and discussions with external experts on psychometric evaluations, methodology adjustments of the importance-scale might be necessary, to decrease its methodological dependence to the confidence-scale (i.e., “How important do you think it is to implement this in everyday working life?“ regarding the term “this”.) Still, as factor covariance lies within the cut off values of 0.30 and 0.70, results indicate that these two scales are distinct constructs despite the methodological dependence regarding the wording.

Limitations and strengths

This study has several limitations. First, divergent validity as part of psychometric parameters could not be analyzed due to the nature of this study being a secondary analysis. Second, the SE-12-G was applied to a selective sample of HCPs participating voluntarily in a communication skills training on child- and family-specific themes, resulting in sampling and volunteer bias. Further validation in different settings without participation in a communication skills training is needed to ensure generalizability. Third, some items exhibit close content-related associations, potentially exerting an influence in the variance within the factor analysis. A careful revision of the translated items might be indicated for these items. Lastly, the SE-12-G was included within a series of measures to evaluate the effects of a communication skills training, taking approximately 30 minutes and therefore possibly influencing scoring of the SE-12-G.

A strength of this study is that we conducted an elaborated translation procedure, aligned with recommended survey translations. Additionally, we used cognitive interviews to explore face validity with various HCPs working in the field of communication and oncology. Further, we explored various models with an adequate sample to robustly perform factor analysis and various psychometric analyses.

Self-efficacy of HCPs in their communication skills should be assessed with valid and reliable measures. So far, an applicable and valid tool for all HCPs in German was lacking. With this study, we provide the first German measure for assessing self-efficacy in communication skills of HCPs including two subscales, the confidence and importance-scale. The German SE-12 (SE-12-G) is a brief measure with good acceptance and reliability. In our sample, psychometric properties were limited regarding the factor analysis and ceiling effects. This could be due to the sample composition (e.g. voluntary bias). In a next step, the SE-12-G should be further modified, and evaluated in a larger and more heterogenic sample of HCPs, that do not take part in an intervention.

Ethics approval and consent to participate

The study is part of the study “Effectiveness of a training program for healthcare professionals on parental cancer: Results of a randomized controlled pilot-study”, which was registered within the German Clinical Trial Register (DRKS-00015794) and approved by the Local Psychological Ethics Committee of the Center for Psychosocial Medicine, University Medical Center Hamburg-Eppendorf, Germany (LPEK-001). The psychometric evaluation of the SE-12-G was a secondary analysis of data gathered in the randomized controlled pilot-study. No administrative permission was required as all data were collected directly by the study team and no medical records were assessed.

Consent for publication

The article does not contain any individual’s details and consent for publication is not applicable.

Availability of data and materials

The dataset collected and analysed during this study is available from the corresponding author on reasonable request and after consultation of the Local Psychological Ethics Committee of the Center for Psychosocial Medicine, University Medical Center Hamburg-Eppendorf.

Competing interests

The authors declare no competing interests.

Funding

Authors' contributions

WF, LJ, LI and CB made substantial contributions to the design and preparation of the study. WF, LJ, LI and CB were involved in the translation and adaptation process. WF and LJ collected the data for psychometric evaluation. WF conducted the analysis. All authors contributed to the interpretation of results. WF drafted the manuscript and LJ, LI and CB were involved in critically revising the manuscript for important intellectual content. All authors gave final approval of the version to be published.

Declaration of Generative AI and AI-assisted technologies in the writing process

During the preparation of this work, the authors used ChatGPT-3.5 in order to improve readability and language of the work. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

Acknowledgement

We would like to thank Mona L. Nasse (MLN) for the team translation of the questionnaire, all HCPs and experts involved in the development of the questionnaire and the participants taking part in this study. Additionally, we would like to thank our colleagues Eva Christalle and Levente Kriston for their methodological expertise and valuable feedback. We would like to thank our student assistants Charlotte von Bremen, Felicitas Schwenzer and Inga Bosch for their support in data entry. Additionally, we would like to thank Professor Jette Ammentorp, Professor Niels Christian Hvidt and Connie Timmermann for giving the opportunity to translate the SE-12 into German, to use it in our implementation study and for providing us with additional information on the SE-12.

Moore PM, Rivera S, Bravo-Soto GA, Olivares C, Lawrie TA. Communication skills training for healthcare professionals working with people who have cancer. Cochrane Database Syst Rev. 2018;7(7):Cd003751.
Kissane DW, Bylund CL, Banerjee SC, Bialer PA, Levin TT, Maloney EK, D'Agostino TA. Communication skills training for oncology professionals. J Clin Oncol. 2012;30(11):1242-7.
Stewart MA. Effective physician-patient communication and health outcomes: a review. Cmaj. 1995;152(9):1423-33.
Mauksch LB, Dugdale DC, Dodson S, Epstein R. Relationship, communication, and efficiency in the medical encounter: creating a clinical model from a literature review. Arch Intern Med. 2008;168(13):1387-95.
Street RL, Jr., Makoul G, Arora NK, Epstein RM. How does communication heal? Pathways linking clinician-patient communication to health outcomes. Patient Educ Couns. 2009;74(3):295-301.
Williams S, Weinman J, Dale J. Doctor-patient communication and patient satisfaction: a review. Fam Pract. 1998;15(5):480-92.
Stewart M, Brown JB, Donner A, McWhinney IR, Oates J, Weston WW, Jordan J. The impact of patient-centered care on outcomes. J Fam Pract. 2000;49(9):796-804.
Fallowfield L, Jenkins V. Effective communication skills are the key to good cancer care. Eur J Cancer. 1999;35(11):1592-7.
Harrington J, Noble LM, Newman SP. Improving patients' communication with doctors: a systematic review of intervention studies. Patient Educ Couns. 2004;52(1):7-16.
Uitterhoeve RJ, Bensing JM, Grol RP, Demulder PH, T VANA. The effect of communication skills training on patient outcomes in cancer care: a systematic review of the literature. Eur J Cancer Care (Engl). 2010;19(4):442-57.
Kerr D, Ostaszkiewicz J, Dunning T, Martin P. The effectiveness of training interventions on nurses' communication skills: A systematic review. Nurse Educ Today. 2020;89:104405.
Mata ÁNdS, de Azevedo KPM, Braga LP, de Medeiros GCBS, de Oliveira Segundo VH, Bezerra INM, et al. Training in communication skills for self-efficacy of health professionals: a systematic review. Human Resources for Health. 2021;19(1):30.
Berg MN, Ngune I, Schofield P, Grech L, Juraskova I, Strasser M, et al. Effectiveness of online communication skills training for cancer and palliative care health professionals: A systematic review. Psychooncology. 2021;30(9):1405-19.
Frerichs W, Geertz W, Johannsen LM, Inhestern L, Bergelt C. Child- and family-specific communication skills trainings for healthcare professionals caring for families with parental cancer: A systematic review. PLOS ONE. 2022;17(11):e0277225.
Axboe MK, Christensen KS, Kofoed PE, Ammentorp J. Development and validation of a self-efficacy questionnaire (SE-12) measuring the clinical communication skills of health care professionals. BMC Med Educ. 2016;16(1):272.
Efthymiou A, Kalaitzaki A, Rovithis M. Validation of the Self-Efficacy Questionnaire (SE-12-Gr) Assessing the Healthcare Professionals’ Self-Reported Communication Skills with Older Healthcare Users in Greece. Health Communication.1-11.
Gil C-r, Sung KM. Validity and Reliability of the Korean Version of Self-Efficacy Questionnaire(KSE-12). Journal of Digital Convergence. 2020;18:337-45.
Escribano S, Juliá-Sanchis R, García-Sanjuán S, Congost-Maestre N, Cabañero-Martínez MJ. Psychometric properties of the Attitudes towards Medical Communication Scale in nursing students. PeerJ. 2021;9:e11034.
Wolderslund M, Kofoed PE, Ammentorp J. The effectiveness of a person-centred communication skills training programme for the health care professionals of a large hospital in Denmark. Patient Educ Couns. 2021;104(6):1423-30.
Hvidt EA, Ammentorp J, Søndergaard J, Timmermann C, Hansen DG, Hvidt NC. Developing and evaluating a course programme to enhance existential communication with cancer patients in general practice. Scandinavian journal of primary health care. 2018;36(2):142-51.
Desai S, Chen F, Boynton-Jarrett R. Clinician Satisfaction and Self-Efficacy With CenteringParenting Group Well-Child Care Model: A Pilot Study. J Prim Care Community Health. 2019;10:2150132719876739.
Kk A, At J, Lø P, Jd L, L L, S EN, et al. Effects of on-site Supportive Communication Training (On-site SCT) on doctor-patient communication in oncology: Study protocol of a randomized, controlled mixed-methods trial. BMC Med Educ. 2024;24(1):522.
Johannsen LM, Frerichs W, Philipp R, Inhestern L, Bergelt C. Effectiveness of a training program for healthcare professionals on parental cancer: Results of a randomized controlled pilot-study. Psycho-Oncology. 2023;32(10):1567-77.
Inhestern L, Frerichs W, Johannsen LM, Bergelt C. Process-evaluation and outcome-evaluation of a training programme for healthcare professionals in oncology to enhance their competencies in caring for patients with minor children: a study protocol for a randomised controlled pilot study. BMJ Open. 2019;9(10):e032778.
Harkness JA, Villar A, Edwards B. Translation, adaptation, and design. Survey methods in multinational, multiregional, and multicultural contexts. Wiley series in survey methodology. Hoboken, NJ, US: John Wiley & Sons, Inc.; 2010. p. 117-40.
Mohler P, Dorer B, De Jong J, Hu M. Translation. guidelines for best practice in cross-cultural surveys. Ann Arbor, MI: survey research center Institute for social research University of Michigan. 2016.
Harkness J, Pennell B-E, Schoua-Glusberg A. Survey Questionnaire Translation and Assessment. Methods for Testing and Evaluating Survey Questionnaires2004. p. 453-73.
Prüfer P, Rexroth M. Kognitive Interviews. 2005 [Available from: https://www.gesis.org/fileadmin/upload/forschung/publikationen/gesis_reihen/hohow/How_to15PP_MR.pdf.
Bannon W, Jr. Missing data within a quantitative research study: How to assess it, treat it, and why you should care. J Am Assoc Nurse Pract. 2015;27(4):230-2.
Flake JK, Pek J, Hehman E. Construct Validation in Social and Personality Research:Current Practice and Recommendations. Social Psychological and Personality Science. 2017;8(4):370-8.
Flora DB, Flake JK. The purpose and practice of exploratory and confirmatory factor analysis in psychological research: Decisions for scale development and validation. Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement. 2017;49(2):78-88.
Rogers P. Best Practices for Your Exploratory Factor Analysis: A Factor Tutorial. Journal of Contemporary Administration. 2021;26(6).
Rattray J, Jones MC. Essential elements of questionnaire design and development. J Clin Nurs. 2007;16(2):234-43.
Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A practical guide to their development and use: Oxford University Press; 2014 01 Jan 2015.
Enders CK, Bandalos DL. The Relative Performance of Full Information Maximum Likelihood Estimation for Missing Data in Structural Equation Models. Structural Equation Modeling: A Multidisciplinary Journal. 2001;8(3):430-57.
Li C-H. Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods. 2016;48(3):936-49.
Hu Lt, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6(1):1-55.
Hair J BW, Babin B, Anderson RE . Multivariate data analysis. Upper Saddle River, NJ: Pearson Prentice Hall; 2009.
Ferketich S. Focus on psychometrics. Aspects of item analysis. Res Nurs Health. 1991;14(2):165-8.
Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Frontiers in Public Health. 2018;6.
McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4(4):293-307.
Bannigan K, Watson R. Reliability and validity in a nutshell. J Clin Nurs. 2009;18(23):3237-43.
Hooper D, Coughlan, J., & Mullen, M. R. Structural Equation Modelling: Guidelines for Determining Model Fit. . The Electronic Journal of Business Research Methods. 2008;6:53-60.
Browne MW, Cudeck R. Single Sample Cross-Validation Indices for Covariance Structures. Multivariate Behav Res. 1989;24(4):445-55.
Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research. 1981;18(1):39-50.
Cheung GW, Cooper-Thomas HD, Lau RS, Wang LC. Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations. Asia Pacific Journal of Management. 2024;41(2):745-83.
Marsh HW, & Hau, K. T. . Confirmatory Factor Analysis: Strategies for Small Sample Sizes. Statistical Strategies for Small Sample Research. Statistical Strategies for Small Sample Research. 1999;1:251-84.
Portney LG, Watkins MP. Foundations of Clinical Research: Applications To Practice: Pearson/Prentice Hall; 2015.
Matheson GJ. We need to talk about reliability: making better use of test-retest studies for study design and interpretation. PeerJ. 2019;7:e6918.
Shiu E, Pervan SJ, Bove LL, Beatty SE. Reflections on discriminant validity: Reexamining the Bove et al. (2009) findings. Journal of Business Research. 2011;64(5):497-500.
Kyriazos TA. Applied Psychometrics: Sample Size and Sample Power Considerations in Factor Analysis (EFA, CFA) and SEM in General. Psychology. 2018;Vol.09No.08:25.

No competing interests reported.

Additionalfile1GermanVersionoftheSE12G.pdf
Additionalfile2CFAModelsincl.values.docx
Additionalfile3Table35720240731.docx
TABLES 3, 4, 5, 7 are included in the Additional file 3

Download PDF

Editorial decision: Revision requested
08 Aug, 2024
Editor assigned by journal
07 Aug, 2024
Submission checks completed at journal
07 Aug, 2024
First submitted to journal
31 Jul, 2024

You are reading this latest preprint version

The German version of the self-efficacy questionnaire (SE-12-G) in a sample of healthcare professionals: Translation and psychometric properties

Status:

Version 1

Abstract

Background

Methods

Results

Conclusion

Trial registration:

Figures

INTROUCTION

METHODS

Participants and data collection

Data analyses

RESULTS

Psychometric assessment

Sample characteristics

DISCUSSION

CONCLUSION

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1