Validation of an instrument to measure community capacity building for prevention

doi:10.21203/rs.3.rs-2471168/v1

Download PDF

Research Article

Validation of an instrument to measure community capacity building for prevention

https://doi.org/10.21203/rs.3.rs-2471168/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 17 Apr, 2023

Read the published version in Journal of Public Health →

Version 1

posted

You are reading this latest preprint version

Aim: In Germany, the CTC-EFF study, a replication study of the U.S. Community Youth Development Study (CYDS), was rolled out in 2021. This article aims to examine the validity of a translated and adapted version of an instrument that measures five constructs of community capacity for prevention.

Subject and Methods: Twenty-two a-priori matched intervention and comparison communities participate in the study. Two to ten community key informants (n = 182) were interviewed for each community. Exploratory (EFA) and confirmatory factor analyses (CFA) were conducted to assess structural validity. Reliability was assessed at the individual level using McDonald’s omega and at the community level using a generalizability coefficient (GC). Inter-rater agreement was measured using an intraclass correlation coefficient (ICC). Concurrent construct validity was assessed using bivariate correlations and mean comparisons.

Results: CFA showed good model fit (CFI = 0.964) for the hypothesized five constructs model. Reliability analyses showed good internal consistency at the individual level (omega = 0.86 – 0.87) and insufficient to moderate reliability at the community level (CG = 0.00 – 0.62). Inter-rater agreement ranged from insufficient to substantial (ICC = 0.02 - 0.32). Consistent with theory, three constructs correlated positively at the individual and community level (r = 0.24 - 0.42). There are no associations between these constructs and interviewer ratings about the respondents’ cooperativeness, trustworthiness, and understanding.

Conclusion: Three of the five constructs were measured with good validity. Two constructs show insufficient reliability. These will be revised until the next CTC-EFF data collection wave. Further validation is indicated.

Trial registration: This study was registered with the German Clinical Trial Register: DRKS00022819 on Aug 18, 2021.

Community Capacity

Community Readiness

Validity

Reliability

Interrater-Agreement

In Germany, a replication study of the U.S. Community Youth Development Study (CYDS) was rolled out in 2021. This controlled study aims to evaluate the effectiveness of the Communities That Care (CTC) community-based prevention system in Germany. For this replication study, the assessment instruments of the CYDS were used. Most of these instruments had already been translated into German several years before the start of the replication study, adapted to the German context, pre-tested, and trialed (Groeger-Roth et al. 2015; Schwendemann et al. 2018; Jonkman 2015). However, some of these previously translated instruments had to be revised again within the replication study for various reasons (e.g., shortening, modifications, measurement errors, and biases). These revised versions were subjected to cognitive pretests and expert evaluations in an iterative process before being used in the replication study. Before the replication study started, we could not subject the revised instruments to another extensive validation study. These instruments include the Community Key Informant Interview (CKI). The CKI was developed by Arthur et al. (Arthur et al. 2001) and measures the short-term outcomes (System Transformation Outcomes) as postulated by the CTC Theory of Change (Fig. 1) (Arthur et al. 2001; Brown et al. 2008; Brown et al. 2014). In the CYDS, empirical validation of this model demonstrated that over 90% of the effect of CTC on risk and protective factors and adolescent problem behavior (Positive Youth Development) was mediated by the construct Adoption of Science-based Prevention (Brown et al. 2014). In a validation study of CKI, this construct was also shown to correlate with the constructs Community Collaboration for Prevention and Community Support for Prevention (Brown et al. 2008). The interpretation of this correlation was that the first construct depended on the other two constructs.

For Germany, we assume that the effect of CTC is probably also entirely mediated by the Adoption of Science-based Prevention. In addition, we expect that the constructs Community Norms and Social Development Strategy do not mediate the effect of CTC in Germany. Therefore, the German version of the instrument does not include these two constructs. Thus, the German version of the CKI includes five System Transformation Constructs. These are described more precisely in the following.

(1) Stage of adoption of science-based prevention: this measure is based on the Prevention Science Framework for Community Prevention Planning and Roger's Stage Model for Diffusion of Social Innovations (Arthur et al. 2001). The former means that science-based community prevention (1) builds on an explicit model of scientifically proven risk and protective factors for health-related behaviors; (2) bases its intervention planning on epidemiological data on the local prevalence of these risk and protective factors and health-related behaviors; (3) selects, implements, and monitors proven effective interventions to address local prevention needs; (4) evaluates the interventions in a data-driven manner and adapts them if necessary. When considering science-based community prevention as a social innovation, the stage model of diffusion of social innovations (Rogers 1995) distinguishes the following five stages: (1) knowledge, (2) persuasion, (3) decision, (4) implementation, and (5) confirmation. On this basis, the construct distinguishes the following six stages (Table 1):

Table 1

Six stages of adoption of science-based prevention
Stage	Description
0	Pre-awareness. Little or no awareness of science-based prevention. Lack of clear understanding of concepts of risk and protective factors or their relevance to strategic prevention planning.
1	Awareness of terminology and concepts of science-based prevention. Understands concepts of risk and protective factors and their basis in longitudinal research.
2	Adoption of the science-based prevention framework as the basis for strategic prevention planning.
3	Collection of epidemiological data on risk and protective factors as well as adolescent problem behaviors.
4	Use of epidemiological data for allocating prevention resources. Prioritization of specific populations and risk and protective factors for preventive action, and selection of evidence-based interventions that address prioritized risk and protective factors.
5	Repeated epidemiological data collection over multiple years for program evaluation, monitoring, and administrative purposes. Feedback of monitoring data into the prevention planning cycle.

(2) Community Collaboration for Prevention improves the effectiveness and efficiency of community health promotion as it leads to broader political advocacy and shared responsibilities for health promotion activities, better resource pooling for health promotion, and a reduction in duplicative structures and because it counteracts jurisdictional wrangling and turf battles (Richard et al. 2002; Dearing et al. 1996; Brown et al. 2008). The construct is captured in the CKI with the following two measures:

(a) Prevention Collaboration: This measure captures the degree to which the local intersectoral health promotion coalition succeeds in networking (e.g., sharing information), coordinating (e.g., changing activities), and collaborating (e.g., sharing resources) (Brown et al. 2008). This conceptualization distinguishes the measure from related constructs, such as that of collaborative capacity (Goodman et al. 1998) and community readiness (Edwards et al. 2000), which primarily only conceptually capture the local context for intersectoral collaboration for health promotion (Brown et al. 2008).

(b) Sectoral-Collaboration: This measure assesses the extent to which stakeholders (e.g., agencies, organizations, volunteers) in the local intersectoral coalition collaborate with stakeholders from other sectors for health promotion (Brown et al. 2008). Ten sectors are included (e.g., schools, churches and other religious institutions, law enforcement, youth services, and media).

(3) Community Support for Prevention refers to local people and local decision-makers believing in the effectiveness of health promotion, knowing about local health promotion efforts, and wanting the community to invest in health promotion.

(a) Community Support: This measure captures the extent to which the community's population is convinced of the effectiveness of health promotion and is aware of local health promotion activities. In the original instrument, this is determined once for the community population and once for the interviewed key persons. The German version of the instrument determines this value only for the community's population.

(b) Financial Support: This measure captures the importance that key respondents attribute to primary prevention and health promotion when allocating a community budget.

The present study aims to evaluate the structural validity, concurrent construct validity, and reliability (internal consistency and interrater agreement) of these constructs using baseline data from the German replication study (CTC-EFF).

Study Design

The following analyses are based on cross-sectional baseline data of the CTC-EFF study. The CTC-EFF is a quasi-experimental study to evaluate the effectiveness of the Communities That Care community-based prevention system in Germany. Small towns and rural communities, as well as city districts of large cities, are participating in the study. These are in Baden-Württemberg, Bavaria, Lower Saxony, and Rhineland-Palatinate and are referred to below as communities. 22 communities implementing CTC (IC) and 22 a-priori matched comparison communities (CC) were recruited that resemble them in demographics, economy, and crime. Details on the study design and recruitment of communities were published by Röding et al. (2021).

Sample and Recruitment

Drawing on the Community Youth Development Study (Hawkins et al. 2008), a two-stage process was used to generate a sample of community key informants (e.g., mayors, school principals, police officers, youth-serving leaders, or health officials) in each of these communities. In Stage 1, the research team used internet searches to identify community key informants for each community and invited them to participate in a survey study via email and telephone. At the end of the survey, in Stage 2, respondents were asked to identify people they thought were most knowledgeable about prevention or health promotion in the community. These persons were also invited to participate in the survey. The goal was to recruit an average of 10 community key informants per community. For sample size calculation, see Röding et al. (2021).

For the baseline survey in 2021, n = 498 community key informants in 22 IC and n = 406 in 22 CC were identified and invited to participate in the survey. Of these, n = 111 persons from 22 IC and n = 89 persons from 21 CC participated. Reasons for non-participation and reasons for exclusion can be found in Fig. 2. The analysis includes n = 182 persons from 18 communities (n = 101 persons from 20 IC and n = 81 persons from 18 CC).

Data Collection

Data was collected using an abridged version of the Community Key Informant Interview (CKI), translated into German and adapted to the German context. First, the 2009 version of the CKI provided by the Social Development Research Group(SDRG 2009) served as the starting point. Details of the original instrument can be found elsewhere (Brown et al. 2007; Rhew et al. 2013). Second, an existing abridged version of the CKI, translated into German and adapted to the German context, which was tested as part of a feasibility study for the implementation of CTC in Germany from 2013 to 2014, served as a reference point (Jonkman et al. 2015). However, this version was not adopted because (1) it was more abridged than would be appropriate for the CTC-EFF study, (2) in some places, it was adapted very specifically to the needs of the feasibility study, and (3) it showed some need for revision (e.g., too little variance, too high item non-response rates, too little internal consistency), as the data collected with this instrument showed.

There were three main reasons we could not use the original unabridged instrument from the U.S.: (1) The foremost of these was that the interview was estimated to have lasted between 90 and 120 minutes without the abridgments. We assumed that such an interview duration would have resulted in disproportionately low response rates and high item non-response rates in the context of the CTC-EFF. (2) The original instrument contains several items that were neither included in the publications of the CYDS we are replicating nor, in our view, essential for the CTC-EFF. (3) Of the constructs relevant to the replication of the CYDS that are assessed with the CKI (see the "System Transformation Constructs" in Fig. 1), it was shown that in the CYDS, the effects of CTC were mediated entirely by one construct and the other constructs were not significant mediators (Brown et al. 2014). After consultation with the Crime Prevention Council of Lower Saxony and the German Prevention Congress, which are the providers of CTC in Germany, we concluded that the constructs Social Development Strategy and Community Norms are also not relevant mediators in the German version of CTC and therefore do not need to be assessed. Regarding the construct Community Norms, there was also concern that the relevant questions might elicit negative attitudes toward the survey among respondents because there are different norms in Germany than in the United States, where the construct was developed. This is particularly apparent regarding tolerance of the consumption of alcohol in small amounts by 16-year-olds in Germany.

Taking into account these necessary modifications, the German questionnaire was created in a team approach based on the TRAPD procedure used in the European Social Survey ESS (Dorer 2021): Translation - in addition to the present German first translation, a second translation was carried out by the authors, Review – discrepancies were clarified in an interdisciplinary discussion with the first translator and subject matter experts, Adjunction – the questionnaire was finalized by the team, Pretest - the translated and abridged version was subjected to cognitive pretests (n = 5), and revised in several loops by the feedback, Documentation – the whole process is documented.

The final version is composed of seven thematic blocks: (1) characteristics of the community (e.g., social capital, openness to change), (2) prevention programs in the community (e.g., planning basis, evaluation, and monitoring), (3) collaboration for prevention (e.g., coordinating agencies, resource sharing), (4) local financial conditions for prevention (e.g., change in financial situation, assessment of the financial situation), (5) local events in the last two years that had an impact on local prevention activities (e.g., natural disaster, pandemic, suicides), (6) sociodemographic characteristics of respondents, (7) other community key informants in terms of snowball sampling. The first five blocks comprise 82 items or 14 constructs (e.g., multi-item scales, indices).

Data collection was primarily by computer-assisted telephone interviews (CATI). As an alternative to CATI, the key informants were provided with a digital version of the instrument in the form of a self-completion questionnaire. This could either be completed using a PDF viewer or printed out (paper-pencil) and sent by post to Hannover Medical School.

Constructs and variables

The constructs presented below are, at the individual level, a measure of how the community key informants, as experts in their community, assess their community. At the community level, they measure the (approximate) actual status of the respective community based on intersubjective assessments by experts. The respective measure for the community level is calculated from the community-specific mean value of the respective individual-level measures.

Construct 1: Adoption of Science-based Prevention

In the CYDS, the indicator adoption of science-based prevention was measured by 20 closed questions and two open questions (Arthur et al. 2001). In the CTC-EFF, the two open-ended questions are not assessed because they refer to clearly titled risk and protective factor concepts that are reasonably well-known in U.S. prevention practice but not in German practice.

Based on the 20 closed questions (e.g., Were risk and protective factors prioritized in your community to be addressed with prevention measures? Did your community decide which programs to implement based on this prioritization? Have prevention programs changed as a result of the evaluation? Has your community made funding decisions based on program evaluations?), a total score between zero and five was calculated for each respondent¹. This value indicates one of the following six levels of adoption of science-based prevention (Table 1). This score, calculated for each respondent, provides a measure of how the respondent ranks their community in terms of adopting a science-based prevention approach.

Construct 2: Prevention Collaboration

Prevention collaboration is measured using a multi-item scale consisting of nine items (e.g., There is a network of people involved in prevention and/or health promotion who are in contact with each other. Prevention providers rarely coordinate their activities with each other. Organizations and agencies share financial and human resources for prevention. In your community, each organization has a clear role in implementing the local prevention plan.). The item response scale is four-point: agree, tend to agree, tend to disagree, disagree. For each respondent, a mean is calculated from their responses to these items. Beforehand, all items are coded so that higher values indicate better cooperation.

Construct 3: Sectoral-Collaboration

Intersectoral collaboration for prevention is measured using a multi-item scale consisting of ten items. In the German version, 22 items were assessed. Respondents are asked: Please estimate to what extent your organization/institution has collaborated regarding prevention and health promotion with the following organizations in the last year. The response scale is four-point: very much, a lot, a little, not at all. The mean value was calculated over all items belonging to the construct. The variables are coded so that higher values represent stronger intersectoral cooperation.

Construct 4: Financial Support for Prevention

This indicator is based on the question: you have a fixed budget for substance abuse reduction. What percentage of 100 would you allocate to the following approaches? (1) law enforcement and prosecution, (2) prevention, and (3) treatment (medical/psychotherapeutic). The indicator reflects the value of the budget share preferred for prevention and can take a value between 0 and 100.

Construct 5: Support for Prevention

The original six-item multi-item scale (Brown et al. 2014) was assessed with only four items in the CTC-EFF baseline survey. This is due to a misinterpretation of the item construct dictionary provided to us by the authors of the CYDS. The four items surveyed read (1) People in [community] believe that problem behavior among children and youth can be reduced through prevention. (2) People in [community] are aware of prevention activities locally. (3) Most people in [community] think that prevention programs for youth don't work. (4) Most people in [community] would be willing to pay more taxes to support youth prevention programs. The last item was modified for the German context. Unlike in the U.S., communities in Germany have hardly any possibility to levy taxes. It is also not yet common in Germany for communities to introduce taxes or tax increases to finance prevention. Therefore, we have replaced this statement with: Most people in [community] believe that all people in the community are responsible for preventing drug abuse. The item response scale is four-point: agree, tend to agree, tend to disagree, disagree. The variables were coded so that higher values represent better support. The measure is formed from the individual mean of the responses to the two items.

Confounders

Gender was assessed using the categories male, female, and non-binary. Current age was collected as well as migration background. Response categories are: Yes, I was not born in Germany; Yes, at least one parent is not from Germany; No. The respondents' educational status was collected according to the "Demographische Standards" (Beckmann et al. 2016). To assess whether the respondents were experts on the respective study community, we asked: how many years have you lived in [community]? One answer option was: live elsewhere. This asked whether the person lived in the study community and if so, for how many years. Furthermore, we asked: How many years have you been working (including interruptions, if applicable) in [community]? One response option was: I do not work in the community. This question was used to assess whether the person works in the study community and, if so, for how many years they had worked there. To have an indicator of whether respondents are familiar with the main topic of the survey, we asked: thinking about your previous professional and volunteer activities, have you worked in the field of prevention and health promotion among children and adolescents? Response options for this are: yes, no, and don't know. Respondents who answered yes to this were considered to have experience in this area.

To assess the validity of the respondents' answers, the interviewers rated the respondents' willingness to cooperate, trustworthiness, and understanding of the questions at the end of the survey. The three items were each measured with a 4-point Likert scale (e.g., item: "Please rate how cooperative the respondent was"; response format: 1 = "friendly and cooperative", 2 = "generally cooperative", 3 = "generally uncooperative", 4 = "uncooperative and hostile").

Missing data

For the five constructs, the item non-response rate was, on average, in the mid-single digits. Details are shown in the results section. These missing values were replaced by the community-specific mean of the item. This imputed value was commercially rounded to a whole number for categorical data.

Statistical Analyses

Analyses were conducted using SPSS 27.0 (IBM, Corp., Armonk, NY) and, in part, Microsoft Excel 2013 and Jamovi 2.3.18.

The five constructs' structural validity were investigated through exploratory factor analyses (EFA) and confirmatory factor analyses (CFA). First, a series of EFA was conducted in an iterative process. In the first step of the analysis, all 27 items of the Sectoral Collaboration item battery, 11 items of the Collaboration for Prevention item battery, the four items on Support for Prevention, the indicator on Financial Support for Prevention, and the index for the degree of adoption of Science-based Prevention (Adoption Score) were included in the analysis. All analyses were conducted as principal component analyses using the varimax method. Only factors with an eigenvalue of at least 1 were extracted. Factor loadings of at least 0.4 were assessed as substantial. Factor loadings below 0.3 were masked out. Initially, the items with the lowest commonality were sorted out step by step. In the subsequent course, in addition to this criterion, the criterion of improving the interpretability of the factors increasingly came into play. In addition, all factors or items that did not belong to our five constructs were eliminated. According to these criteria, items were eliminated until a solution with five factors was identified. Then the factor analysis was continued with the setting that five factors must be extracted. The Kaiser-Meyer-Olkin criterion (KMO) was used for the stepwise elimination or inclusion of items. A value of 0.6 or more is considered sufficient, a value of 0.7 or more satisfactory, a value of 0.8 or more good, and a value of 0.9 or more very good. Another criterion used was the increase in variance explained by the factors. The best-fitting five-factor model identified in this way formed the starting point for the CFA. The CFA was carried out using Jamovi. Evidence of construct validity would be realized if item-factor loadings for the nine indicator variables were large (i.e., > 0.50) and statistically significant (p < 0.05) and if the overall factor model fit the data well. Fit of the baseline CFA model was assessed using criteria suggested by Hu and Bentler (1998): comparative fit index (CFI; Bentler 1990) greater than 0.95, Tucker–Lewis fit index (TLI; Tucker and Lewis 1973) greater than 0.95, and root mean square error of approximation (RMSEA; Browne and Cudeck 1992; Steiger and Lind 1984) less than 0.06. Since the default model, according to which there are no covariances in the residuals of the items, had an insufficient model fit, covariance was accepted in steps, based on modification indices and theoretical considerations, until a sufficient model fit was achieved.

To assess the internal consistency of the two multi-item scales, Cronbach's alpha and McDonald's omega were calculated using a macro distributed by Hayes (Hayes and Coutts 2020). To assess the internal consistency of the five measures at the community level, O'Brien’s (1990) coefficient of internal consistency was calculated to account for the two-level structure of the data (O'Brien 1990). This reliability coefficient includes two variance components: the (estimated) "universal" variance and the observed variance. The first variance component includes the variance between aggregates. The observed variance refers to the dispersion of responses from individuals residing within each aggregate unit. The reliability coefficient is estimated by subtracting the root mean square associated with individual-level responses within the aggregates (i.e., the residual mean sum of squares) from the root mean square of the aggregates (i.e., the explained mean sum of squares). This number is then divided by the root mean square of the aggregates. The resulting coefficient is a ratio between the estimated variance of the aggregate score and the observed variance of the score. Reliability thus depends on the number of respondents in each community and the variance within and between communities. This ratio is bounded between 0 and 1. The closer this reliability coefficient is to 1, the higher the reliability of the indicator at the aggregate level.

In order to assess whether key informants' ratings of their community have substantial intersubjective explanatory value, a measure of inter-rater agreement was calculated using the method proposed by Garner and Raudenbush (1991) and Sampson et al. (1997). For this, hierarchical linear models (HLM) are first used to calculate which parts of the variance in the measurements (measured constructs) are due to differences between respondents (individual level) and differences between communities (community level), respectively. A significance test performed by SPSS for this purpose can also be used to determine whether differences between communities contribute statistically significantly to the variance.

In a second step, the intraclass correlation coefficient (ICC) is then calculated from the two variance components by dividing the part of the variance attributable to differences between the communities by the total variance (sum of the variance attributable to both differences between the communities and differences between the respondents). Thus, the ICC indicates what percentage of the total variance in the measurements is attributable to differences between communities. Since the individual samples within the communities in the baseline survey show differences in the distribution of sociodemographic characteristics of the respondents, it was examined to what extent the part of the variance attributable to differences between the communities can be explained by sociodemographic differences in the composition of the community-specific samples. Therefore, aggregated variables were calculated for the sociodemographic characteristics (e.g., community-specific proportion of women among respondents, community-specific average age of respondents) and included in the HLM as macro regressors (Baltes-Götz 2020), respectively community-specific characteristics.

To assess concurrent construct validity, correlations between the five constructs were examined at the individual and community level. For metric-scaled variables, Pearson's correlation coefficient r, and for ordinal-scaled variables, Spearman's correlation coefficient rho was calculated.

In addition, mean comparisons were used to examine whether respondents' ratings of their community differed systematically according to how they were rated by the interviewers in terms of their willingness to cooperate, trustworthiness, and how well they understood the questions. Since some groups here (e.g., respondents with low willingness to cooperate) had only small single-digit case numbers, only descriptive mean comparisons were made.

¹ A SPSS syntax for calculating the indicator is available from the authors upon request.

Descriptive Statistics

Table 2 describes the sample (n = 182) based on selected characteristics. 56% of the respondents come from IC, 49% are female (no persons are non-binary), 91% have a university entrance qualification, 7% have a migration background, 54% live, and 83% work in the community they were interviewed about, 69% have acquired professional or voluntary experience in prevention or health promotion for children, adolescents, or families. The respondents are, on average, 49.6 years old and have lived in the community they were interviewed about for an average of 15.1 years.

Table 2

Sample description
	% / Mean (N = 182)
Respondents from Intervention Communities (%)	55.5
Female (%)	48.6
Age in years (mean)	49.6
Respondents with higher education entrance qualification (%)	90.7
Respondents with migration background (%)	6.6
Respondents living in the community (%)	54.4
- Duration of residence in years (mean)	15.1
Respondents working in community (%)	83.0
Respondents with experience in prevention (%)	69.0

Table 3 shows the means, standard deviations, and the number of missing values of the items of the five constructs. For 35 of these 42 items, the item non-response rate was in the single digits. For the construct adoption of science-based prevention, six of the 20 items had an item non-response rate between 11% and 26%. Imputation of missing values resulted in only minor changes (changes of no more than 0.01 units) in the mean and standard deviation of the items.

Table 3

Univariate statistics on the items of the five latent constructs
Construct	Label	Before Imputation			After Imputation		Response format
Construct	Label	M	SD	Missing	M	SD	Response format
Science-based Prevention	Knowledge prevention 1	1.73	0.45	2	1.73	0.45	1 = „1“, 2 = „2“
	Knowledge prevention 2	1.70	0.46	2	1.70	0.46	1 = „1“, 2 = „2“
	Knowledge prevention 3	1.25	0.43	3	1.25	0.43	1 = „1“, 2 = „2“
	Knowledge prevention 4	1.01	0.07	3	1.01	0.07	1 = „1“, 2 = „2“
	Knowledge 5	1.03	0.18	3	1.03	0.18	1 = „1“, 2 = „2“
	Risk and protective factors concept in community	1.68	0.47	29	1.69	0.44	1 = „yes“, 2 = „no“
	Usage risk and protective factors concept of other organizations	2.85	0.57	9	2.88	0.58	1 = „all“, 4 = „none“
	Usage risk and protective factors concept of own organization	2.38	0.87	2	2.37	0.87	1 = „(almost) every time“, 4 = „(almost) never“
	Youth or student surveys	1.59	0.49	36	1.60	0.47	1 = „yes“, 2 = „no“
	Usage social indicators	1.57	0.50	48	1.56	0.46	1 = „yes“, 2 = „no“
	Influence of data on risk and protective factors	2.35	0.87	24	2.35	0.83	1 = „a lot of influence“, 4 = „no influence“
	Influence databases	2.27	0.93	19	2.27	0.91	1 = „a lot of influence“, 4 = „no influence“
	Influence regional disparities	2.40	1.07	21	2.40	1.03	1 = „a lot of influence“, 4 = „no influence“
	Prioritization of risk and protective factors	1.29	0.46	8	1.27	0.45	1 = „yes“, 2 = „no“
	Implementation of programs based on risk and protective factors	1.59	0.91	10	1.75	0.71	1 = „yes“, 2 = „no“
	Prioritization of areas for additional prevention measures	1.65	0.48	2	1.66	0.48	1 = „yes“, 2 = „no“
	Prioritization of population groups for additional prevention measures	1.36	0.48	4	1.35	0.47	1 = „yes“, 2 = „no“
	Evaluation of the influence of the programs on risk and protective factors	1.80	0.79	11	1.80	0.79	1 = „yes“, 2 = „no“
	Adaptation programs through program evaluation	1.43	0.54	10	1.31	0.46	1 = „yes“, 2 = „no“
	Funding based on program evaluation	1.40	0.55	14	1.37	0.33	1 = „yes“, 2 = „no“
Prevention Collaboration	Cooperation prevention network	1.92	0.99	11	1.95	0.97	1 = „agree“, 4 = „disagree“
	Coordination prevention work	2.53	0.80	10	2.54	0.78	1 = „agree“, 4 = „disagree“
	Cooperation prevention network in case of problems	2.44	0.78	14	2.45	0.76	1 = „agree“, 4 = „disagree“
	Meetings prevention network	2.33	0.90	15	2.34	0.89	1 = „agree“, 4 = „disagree“
	Information sharing prevention network	2.41	0.86	16	2.46	0.86	1 = „agree“, 4 = „disagree“
	Collaboration prevention network	2.78	0.74	15	2.80	0.73	1 = „agree“, 4 = „disagree“
	Joint decision making	2.77	0.79	17	2.80	0.77	1 = „agree“, 4 = „disagree“
	Resource sharing	2.98	0.86	23	2.99	0.82	1 = „agree“, 4 = „disagree“
	Clear distribution of roles	2.95	0.87	15	2.96	0.83	1 = „agree“, 4 = „disagree“
Sectoral-collaboration	Cooperation child protection	2.39	1.16	3	2.39	1.15	1 = „a lot“, 4 = „not at all“
	Cooperation secondary schools	1.88	1.03	4	1.88	1.02	1 = „a lot“, 4 = „not at all“
	Cooperation youth work	2.47	1.08	4	2.47	1.07	1 = „a lot“, 4 = „not at all“
	Cooperation social/cultural organizations	2.37	0.97	3	2.37	0.96	1 = „a lot“, 4 = „not at all“
	Cooperation Health Department	2.96	1.06	9	2.97	1.04	1 = „a lot“, 4 = „not at all“
	Cooperation voluntary groups	2.88	1.06	7	2.89	1.05	1 = „a lot“, 4 = „not at all“
	Cooperation police, judiciary	2.27	0.99	8	2.26	0.98	1 = „a lot“, 4 = „not at all“
	Cooperation media	2.90	1.02	8	2.89	1.00	1 = „a lot“, 4 = „not at all“
	Cooperation companies, businesses	2.77	1.09	8	2.77	1.08	1 = „a lot“, 4 = „not at all“
	Cooperation religious institutions	2.60	1.12	6	2.60	1.10	1 = „a lot“, 4 = „not at all“
	Cooperation pharmacies	3.54	0.73	3	3.53	0.73	1 = „a lot“, 4 = „not at all“
	Cooperation employment agency	3.05	1.04	4	3.05	1.02	1 = „a lot“, 4 = „not at all“
	Cooperation drug counseling centers	2.65	0.98	2	2.65	0.98	1 = „a lot“, 4 = „not at all“
	Cooperation hospitals	3.42	0.80	2	3.42	0.79	1 = „a lot“, 4 = „not at all“
	Cooperation care facilities	2.87	0.87	3	2.88	0.86	1 = „a lot“, 4 = „not at all“
	Cooperation physicians	3.13	0.89	3	3.13	0.89	1 = „a lot“, 4 = „not at all“
	Cooperation housing associations	3.59	0.79	5	3.61	0.77	1 = „a lot“, 4 = „not at all“
	Cooperation nurseries	2.37	0.98	1	2.37	0.98	1 = „a lot“, 4 = „not at all“
	Cooperation psychiatric institutions, therapists	2.95	1.04	4	2.94	1.03	1 = „a lot“, 4 = „not at all“
	Cooperation primary schools	2.03	0.95	1	2.03	0.95	1 = „a lot“, 4 = „not at all“
	Cooperation self-aid groups	3.37	0.85	7	3.35	0.84	1 = „a lot“, 4 = „not at all“
	Cooperation sport clubs	2.33	1.03	2	2.33	1.02	1 = „a lot“, 4 = „not at all“
	Cooperation physio-, ergo-, logotherapy	3.29	0.86	2	3.29	0.86	1 = „a lot“, 4 = „not at all“
	Cooperation youth centers	2.11	1.01	2	2.10	1.00	1 = „a lot“, 4 = „not at all“
Financial Support	Financial support for prevention	51.11	15.12	0	51.11	15.12	0–100
Support for Prevention_a	Belief in the effectiveness of prevention	1.85	0.67	13	1.87	0.65	1 = „agree“, 4 = „disagree“
Support for Prevention_a	Knowledge about prevention work	2.85	0.67	6	2.85	0.65	1 = „agree“, 4 = „disagree“
M: Mean SD: Standard deviation

_a: The original six-item multi-item scale ((Brown et al. 2014)) was assessed with only two items in the CTC-EFF baseline surveys. This is due to a misinterpretation of the item construct dictionary provided to the authors of the CYDS.

Table 4 shows mean values and standard deviations of the five constructs according to their measurement at the individual and community level. The standard deviations are larger at the individual level than at the community level. The values of constructs one and five have more variance than those of the other three.

Table 4

Means and standard deviations of the constructs by measurement level
	Individual level (n = 182)		Community level (n = 38)
	M	SD	M	SD
(1) Science-based Prevention	1.65	1.26	1.62	0.84
(2) Prevention Collaboration	2.59	0.57	2.60	0.33
(3) Sectoral-Collaboration	2.31	0.72	2.30	0.42
(4) Support for Prevention	2.36	0.53	2.37	0.26
(5) Financial Support for Prevention	51.11	15.12	51.91	8.24

Figure 3 displays the relative frequencies of the six stages of adoption of science-based prevention as reported by key informant respondents. A fixed algorithm was used to assign respondents to the stages based on information respondents provided about their community. 1% of respondents indicate that their community is in stage 0 (no prevention science awareness), 73% of respondents indicate that their communities are in stage 1 (prevention science awareness), 6% indicate that their community is in stage 2 (prevention science prevention planning), 3% of respondents indicate their community is in Stage 3 (plus epidemiological analysis), 12% of respondents indicate their community is in Stage 4 (plus data-driven allocation of prevention resources), and 5% of respondents indicate their community is in Stage 5 (plus monitoring of interventions and replication of approach).

Figure 4 shows the absolute frequencies of how the 38 communities were assigned to the six stages of adopting science-based prevention. The assignment of communities to stages is based on the commercially rounded community-specific mean scores of the key informants surveyed. Using this method, no community was assigned to stage 0 (no prevention science awareness), 18 were assigned to stage 1 (prevention science awareness), 14 were assigned to stage 2 (prevention science prevention planning), 4 were assigned to stage 3 (plus epidemiologic analyses), one was assigned to stage 4 (plus data-driven allocation of prevention resources), and one was assigned to stage 5 (plus monitoring of interventions and replication of approach).

Structural Validity

Based on the theoretical model (Fig. 1) a series of EFA, and then a CFA were conducted to test the structural validity. An EFA with 44 indicators identifies 13 factors with an eigenvalue greater than 1 (online supplement 1). Factors 1, 2, 9, 10, and 12 represent the five constructs with minor imprecisions. An EFA with 21 indicators (Table 5) identifies five factors with an eigenvalue greater than 1. The KMO criterion for this model is 0.854. The commonalities are above 0.5 for 17 of these 21 indicators and between 0.4 and 0.5 for the other four indicators. The five factors explain 60% of the variance in the indicators. The eight indicators of factor 1 represent the remaining items of the construct Sectoral Collaboration, the nine indicators of factor 2 represent the construct Collaboration for Prevention, the two indicators of factor 3 represent the construct Support for Prevention, and the two indicators for Financial Support and Science-based Prevention correspond to factors 4 and 5. Factor 2 corresponds to the multi-item scale used in the CYDS for the Collaboration for Prevention. Of factor 1, only four indicators correspond to the 10-item scale used in the CYDS for Sectoral Collaboration. In addition, Factor 1 includes four indicators that do not appear in the CYDS 10-item scale.

Table 5

Rotated component matrix of an EFA
Item Description	Components
Item Description	1	2	3	4	5
Youth work	0.811
School social work	0.811
Social work services	0.765
Secondary schools	0.743
Drug counseling centers	0.690
Police, judiciary	0.646
Youth center	0.625
Child protection	0.537
Exchange of information		0.808
Coordinated strategies		0.791
Joint problem solving		0.780
Joint meetings		0.736
Coordination of activities		0.611		-0.322
Existing network for prevention		0.600		0.417
Joint financial/personal resources		0.525	0.326
Defined roles		0.476	0.346		-0.468
Participation in community processes		0.459		0.456	-0.323
Belief in effectiveness of prevention			0.831
Knowledge about prevention efforts		0.308	0.587		0.305
Science-based Prevention				-0.813
Financial Support					0.791

A CFA was carried out with the five factors of the final EFA model. All indicators load statistically highly significantly (p < 0.001) on their factor. Seventeen of the nineteen indicators have a factor loading greater than 0.5 (Table 6). For the other two indicators, the factor loading is between 0.36 and 0.49. The global fit indices also support this model (CFI = 0.964; TLI = 0.956; RMSEA = 0.0388). The model assumes correlations between the factors (online supplement 2) and covariance of the residuals of certain indicators (online supplement 3).

Table 6

Factor loadings
Factor (Indicator)	Estimator	SD	p	Standardized Estimator
Collaboration for Prevention
Existing network of prevention	0.536	0.0722	< .001	0.551
Coordinated activities	0.385	0.0582	< .001	0.498
Joint problem solving	0.521	0.0519	< .001	0.695
Joint meetings	0.638	0.0605	< .001	0.729
Exchange of Information	0.663	0.0573	< .001	0.775
Coordinates strategies	0.593	0.0470	< .001	0.819
Participation in community processes	0.426	0.0560	< .001	0.557
Shared financial/personnel resources	0.399	0.0620	< .001	0.490
Defined roles	0.451	0.0612	< .001	0.541
Sectoral Collaboration
Secondary schools	0.724	0.0748	< .001	0.690
School social workers	0.783	0.0717	< .001	0.751
Youth work	0.728	0.0713	< .001	0.714
Youth centers	0.531	0.0755	< .001	0.526
Social work services	0.746	0.0702	< .001	0.732
Drug counseling centers	0.699	0.0680	< .001	0.714
Police, judiciary	0.600	0.0708	< .001	0.612
Child protection	0.551	0.0794	< .001	0.521
Science-based Prevention	1.264	0.0663	< .001	1.000
Support for Prevention
Belief in effectiveness of prevention	0.231	0.0655	< .001	0.355
Knowledge about prevention efforts	0.591	0.1234	< .001	0.905
Financial Support	15.017	0.7871	< .001	1.000

Internal Consistency and Inter-Rater-Agreement

Table 7 contains results of the reliability analyses for the five constructs. The first two columns show measures (Cronbach's alpha and Mc Donald's omega) for the internal consistency of the scales. These values range from 0.86 to 0.87, indicating good internal consistency for these scales. At this point, it should be recalled that the constructs do not represent the respondents' characteristics but the communities for which they were interviewed as experts. However, the internal consistency depicted by the first two ratios does not take this two-level data structure into account. Therefore, an additional measure of the internal consistency of the measurements at the community level is plotted in the third column. Since this measure can be calculated for scales and indices and individual items, values are shown for all five constructs. In the last column, a measure of inter-rater agreement is plotted. The values are from unadjusted HLM. In addition, we ran adjusted HLM with the result that the tested confounders (proportion of women, average age, proportion living in the community, and proportion working in the community) were not relevant to the models. Accordingly, differences in sample composition between the communities do not affect the reliability of the measurement.

Table 7

Results of the reliability analyses
	Internal Consistency			Inter-Rater Agreement
	Cronbach's Alpha	Mc Donald's Omega	Community-Level Internal Consistency	Intraclass Correlation Coefficient (ICC)
Science-based Prevention	NA	NA	0.62	0.32 0.15 0.16 0.07 0.02
Prevention Collaboration	0.86	0.86	0.38
Sectoral-Collaboration	0.87	0.87	0.30
Financial Support	NA	NA	0.00
Support for Prevention	NA	NA	0.00

Concurrent Construct Validity

Table 8 shows results of correlation analyses among the five constructs. Regarding the findings of Brown (2008), we examine whether the constructs (1) science-based prevention, (2) prevention collaboration and (3) intersectoral collaboration are statistically significantly positively correlated with each other at both the community and individual levels. The results show highly significant correlations at both the individual and community levels. We recognize that theoretical justifications can also be made as to why the constructs (4) financial support and (5) community support could or should also be correlated with each other and the other three constructs. According to our data, however, constructs four and five are not correlated, showing correlations only with some of the other three constructs.

Table 8

Construct validity at the individual and community level
	(1)	(2)	(3)	(4)	(5)
	Above Diagonal: Community Level Correlations (N = 38)
(1) Science-based Prevention		r = .30** p < 0.001	r = .35** p < 0.001	r = − .04 p = .595	r = − .17* p = .025
(2) Prevention Collaboration	r = .25** p = .001		r = .42** p < 0.001	r = .14* p = .05	r = .33** p < 0.001
(3) Sectoral-Collaboration	r = .27** p < 0.001	r = .24** p = .001		r = .16* p = .027	r = .30** p < 0.001
(4) Financial Support	r = .05 p = .488	r = .10 p = 0.201	r = .16* p = .027		r = .01 p = .984
(5) Support for Prevention	r = − .07 p = .347	r = .18** p = 0.018	r = 14^* p = .055	r = .11 p = .122
	Below Diagonal: Individual Level Correlations (N = 182)

Table 9 shows mean values for the five constructs (individual level) differentiated according to how the interviewees rated the respondents' willingness to cooperate, trustworthiness and understanding. Inferential statistical analysis for mean differences was impossible because the number of cases of respondents rated as having a low willingness to cooperate, low trustworthiness, or low understanding was too small. Slight descriptive mean differences should be interpreted conservatively at this point. We looked for linear patterns and, if they were found, examined whether they consistently point in the same direction for all five constructs and for three extraneous assessment criteria. Considering the score for science-based prevention separately by respondents' willingness to cooperate, a linear pattern emerges. The lower the respondents' willingness to cooperate, the lower the mean score for science-based prevention. However, this pattern does not emerge for the other four constructs. And when science-based prevention is considered differentiated by respondents' understanding, this gradient turns in the other direction. The differences in the mean scores of the groups are mostly rather small and do not show a consistent pattern. It, therefore, stands to reason that the descriptive differences in mean values considered here are exclusively or at least mainly due to random fluctuations, which can be correspondingly large when the number of cases is small.

Table 9

Mean values of the constructs (individual level) according to interviewer ratings on respondents' willingness to cooperate, trustworthiness, and understanding
	Science-based Prevention	Prevention Collaboration	Sectoral-Collaboration	Financial Support	Support for Prevention
Cooperation Friendly & cooperative (n = 152) Generally cooperative (n = 28) Generally uncooperative (n = 2) Uncooperative & hostile (n = 0)	1.69 1.54 1.00 NA	2.57 2.68 2.62 NA	2.50 2.64 2.06 NA	51.43 50.39 56.00 NA	2.37 2.35 2.33 NA
Trustworthiness Very truthful (n = 139) Somewhat truthful (n = 43) Not very truthful (n = 0) Not at all truthful (n = 0)	1.69 1.56 NA NA	2.55 2.69 NA NA	2.49 2.63 NA NA	51.71 50.05 NA NA	2.36 2.39 NA NA
Understanding Easily understood questions (n = 131) Understood questions with some explanation (n = 49) Understood questions with much explanation (n = 2) Did not seem to understand many questions (n = 0)	1.66 1.61 2.50 NA	2.56 2.66 2.28 NA	2.50 2.58 2.48 NA	52.11 48.82 61.00 NA	2.34 2.41 2.44 NA

This validation study is embedded in the CTC-EFF study evaluating the effectiveness of the Communities That Care community-based prevention system in Germany. The CTC-EFF is an extensive replication of the U.S. Community Youth Development Study (CYDS), which has been ongoing as a cluster RCT since 2003. In contrast to this design, the CTC-EFF is a non-randomized controlled trial (Röding et al. 2021). For the CTC-EFF, the assessment instruments from the CYDS were translated into German and adapted to the German context. In the current validation study, analyses were conducted on the structural validity, internal consistency, and concurrent construct validity of five community capacity measures collected as part of the baseline of the CTC-EFF using the German version of the Community Key Informant Interview (CKI).

Interpretation of the Results

Only partially comparable figures have been published from CYDS and its Dutch replication for our univariate results. For the adoption of science-based prevention, we find very similar values in our data (M = 1.65; SD = 1.26) to those reported byBrown et al. (2014) for the CYDS for the control group in 2004 (M = 1.65; SD = 0.93). Moreover, according to our data, 16% of respondents are from a community that is at least in stage 4. Steketee et al. (2009) report that 20% of the control communities of the CYDS in 2007 (4.5 years after the introduction of CTC) and 11% of the control communities of the Dutch replication study in 2011 (3 years after the introduction of CTC) are at least in stage 4.Brown et al. (2008) report values of M = 1.97 (SD = 0.57) for the construct intersectoral collaboration for the year 2001. For Germany, we find values of M = 2.52 (SD = 0.62) for this construct. Brown et al. (2011) reported a value of about 41% for financial support for prevention in 2001. We find a value of 51%. Overall, we find only minor differences in the point estimates and standard deviations.

Regarding the structural validity of the four interval-scaled constructs, our results of the EFA show, on the one hand, that these are four distinct constructs. Second, the results show that the construct sectoral-collaboration in our data consists of three subfactors. The results of the CFA on the two multi-item scales collaboration for prevention and intersectoral collaboration for prevention further confirm that these scales are two selective constructs. Additionally, the results show that a two-factor model fits our data best. However, it also reconfirms that the intersectoral collaboration factor consists of three subfactors. Our results are broadly consistent with the theoretical model of Brown et al. (2011). Yet, in our model, the sectors included in the Sectoral Cooperation construct differ from those included in the original model from the Community Youth Development Study. We believe that these differences reflect national differences in prevention practice. For example, businesses and religious organizations in Germany are hardly systematically involved in local prevention. On the other hand, in this field of prevention, social work services, school social workers, and drug counseling centers in Germany are regularly and systematically involved in prevention networks. It is also noteworthy that Factor 3 includes only two of the four indicators adopted from the CYDS to measure Support for Prevention. At this point, it should be pointed out once again that due to a misunderstanding, this construct is only incompletely included in the German version of the CKI.

Our analyses of the internal consistency and reliability of the five constructs show that the two multi-item scales have good internal consistency with Cronbach's alpha of 0.86 and 0.87 and Mc Donald's omega of 0.86 and 0.87 at the individual level. In the CYDS (Brown et al. 2011), Cronbach's alpha of 0.77 and 0.92 were reported. For science-based prevention, reliability coefficients were reported from the CYDS to assess reliability at the community level (O'Brien 1990). This ranged from 0.71 to 0.75 (Arthur et al. 2001). We find a value of 0.62 for our surveys. Since the value is highly dependent on sample size within communities and our average cluster size of n = 4.8 was significantly smaller than those of Arthur et al. (2001) at 6.8 and 9.9, respectively, we attribute the differences in this coefficient to this. According to O'Brien (1990), a value between 0.6 and 0.7 corresponds to moderate reliability. We found reliability coefficients for the other four constructs ranging from 0 to 0.30 for the community level. No comparable values are available from the CYDS. Arthur et al. (2001) further report on science-based prevention having an interrater agreement of ICC = 0.23 and ICC = 0.27. For the CTC-EFF, we find an ICC of 0.32 in this regard. No corresponding measures are reported from the CYDS for the other four constructs. For the CTC-EFF, we find ICCs ranging from 0.02 to 0.21 for the other constructs. Some studies that surveyed key informants in communities found similar values ranging from 0 to 0.21 (Garner and Raudenbush 1991; Sampson et al. 1997; Shapiro et al. 2015). With respect to the partially low values we found for community-level reliability and interrater agreement, it should be noted that the effects of CTC on risk and protective factors and problem behaviors found in the CYDS are exclusively and completely mediated by the adoption of science-based prevention in the multivariate model (Brown et al. 2014). That is, the construct science-based prevention is likely to be most relevant to the CTC-EFF. For this construct, we found relatively good values in terms of reliability and interrater agreement.

Correlations between science-based prevention, prevention collaboration, and intersectoral collaboration were analyzed to test for concurrent construct validity. We find correlations between 0.24 and 0.27 at the individual level and between 0.30 and 0.42 at the community level. Brown et al. (2008) report correlations between 0.26 and 0.37 at the individual level and between 0.64 and 0.75 at the community level. Although our data also show positive correlations between these three constructs, they are systematically weaker than in the CYDS. This could be partly because these constructs could be measured with slightly higher reliability in the CYDS. Second, contextual differences could also explain these differences in correlation strengths, which are currently unclear to us. Since theoretical justifications could be made that the other two constructs should also show a positive correlation within themselves and with the other three constructs, we also tested for this. However, the results are less consistent here, although correlations of up to 0.33 at the community level and 0.18 at the individual level are found. Most importantly, the negative correlation between community support and science-based prevention does not fit the pattern. This should be investigated in more detail in the further course of the CTC-EFF. We also examined whether our scores on the five constructs were statistically related to how cooperative, trustworthy, and understanding our interviewers rated the respondents. At this point, we found no evidence of such correlations.

Due to some differences between the USA and Germany, we had adapted some items to the German context. We found no evidence in our results that our measurements were affected by these adaptations. Studies (Repke and Dorer 2021) show that such adaptations are beneficial.

Overall, the results of this validation study indicate that the German version of the CKI is able to measure three important constructs of community capacity for prevention in a valid way. Thus, a new validated instrument for measuring community capacity is available for Germany. This is important as only one instrument for measuring community capacity (Nickel et al. 2013; Nickel et al. 2018) and one for community readiness (Gansefort et al. 2020) were previously available for Germany. However, measuring these constructs is an essential complement to corresponding qualitative assessment methods (Kehl et al. 2021; Wihofszky et al. 2021). Valid instruments for measuring community capacity are needed for effectiveness studies and testing causal models of approaches to community-based prevention and health promotion.

Limitations

Some limitations in this study are worth highlighting. First, as our study was cross-sectional, test-retest reliability and predictive validity of the CKI could not be ascertained. To investigate these properties, longitudinal research designs should be considered. Second, the target sample size of n = 10 could not be achieved in all communities. Although the sample size (N = 182) was considered sufficient for the conducted analyses, a larger sample size would have provided more reliable results. Due to the small sample size, it was also not possible to use multiple imputation procedures for missing data (Donner and Klar 2000; Campbell et al. 2007; Wears 2002). The utilized imputation procedure takes the cluster structure of the data into account but not the characteristics of the respondents. However, since the measured constructs are characteristics of communities and not individuals, only minor possible biases can be assumed. So far, the factor analyses also do not consider the data’s multilevel structure. Corresponding analyses have been planned for the future. However, based on the results by Brown et al. (2008), we assume that this will not lead to substantially different results than those presented in this article. Furthermore, communities were not randomly selected, which may limit the generalizability of the results. Thus, our results on point estimates may not be generalizable to communities in other contexts. However, we would like to point out that this was not the aim of this study. Moreover, we note that survey data from key informants are widely used in community research (Brown et al. 2014; Shinn 1990).

With the current German version of the CKI, three important community capacity constructs can be measured validly. A revision is being conducted for the other two constructs. The instrument is available to non-commercial prevention research free of charge. Thus, this study makes a significant contribution to German prevention research.

Funding

The Federal Ministry of Education and Research (BMBF) supported the current project (funding number: 01EL2006A). The funder was not involved in the study’s design, manuscript writing, or data collection; the funder will not be involved in data analysis, interpretation, or manuscript writing in the future. Open Access funding enabled and organized by Projekt DEAL.

Conflicts of interest/Competing interests

The authors have no competing interests to declare relevant to the article’s content.

Ethics approval

The questionnaire and methodology for this study was approved by the Hannover Medical School’s ethics committee (Ethics approval number: 9739_BO_K_2021).

Consent to participate

All participating communities completed an informed consenting process with a research team member. Written informed consent was obtained from all individual participants included in the study. The respondents were informed that their inclusion in the study was voluntary, and that they were free to withdraw from the study if they were not willing to participate. To ensure respondents’ confidentiality, their names were stored in separate, protected files, and the anonymity of the respondents was maintained.

Consent for publication

Not applicable.

Availability of data and material

The datasets and material used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Code availability

Not applicable.

Authors’ contribution

All authors contributed to the study conception, design, and adaptation of the instrument. Dominik Röding and Vera Birgel performed data collection and analysis. Dominik Röding and Vera Birgel wrote the first draft of the manuscript and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Acknowledgments

We want to acknowledge the former and actual members of the Social Development Research Group at the University of Washington and especially Sabrina Oesterle, Arizona State University, for their advice in conducting the study and John Briney as well as Danielle Woodward, University of Washington, for providing the survey instruments and other materials from the Community Youth Development Study. We also acknowledge Harrie B Jonkman, Verwey-Jonker Institute, Netherlands, for supporting our study. We acknowledge support from the German Research Foundation (DFG) and the Open Access Publication Fund of Hannover Medical School (MHH). The publication is part of the CTC-EFF study (Effectiveness of the Community Prevention System Communities That Care). CTC-EFF Consortium: Ulla Walter, Renate Soellner, Christian Krauth, Dominik Roeding, Maren Reder, Frederick Groeger-Roth.

Compliance with ethical standards

The questionnaire and methodology for this study was approved by the Hannover Medical School’s ethics committee (Ethics approval number: 9739_BO_K_2021). All participating communities completed an informed consenting process with a research team member. Written informed consent was obtained from all individual participants included in the study. The respondents were informed that their inclusion in the study was voluntary, and that they were free to withdraw from the study if they were not willing to participate. To ensure respondents’ confidentiality, their names were stored in separate, protected files, and the anonymity of the respondents was maintained.

Arthur MW, Glaser RR, Hawkins JD (2001) Steps Toward Community-Level Resilience. In: Peters RD (ed) Resilience in children, families, and communities: Linking context to practice and policy. Kluwer Academic/Plenum Publishers, New York, NY, pp 177–194
Baltes-Götz B (2020) Analyse von hierarchischen linearen Modellen mit SPSS. https://www.uni-trier.de/fileadmin/urt/doku/hlm/hlm.pdf. Accessed 28 July 2022
Beckmann K, Glemser A, Heckel, Christiane, von der Heyde, Christian, Hoffmeyer-Zlotnik, Jürgen H.P., Hanefeld, Ute, Herter-Eschweiler, Robert, Kühnen, Carola (2016) Demographische Standards Ausgabe 2016. Eine gemeinsame Empfehlung des ADM Arbeitskreis Deutscher Markt- und Sozialforschungsinstitute e.V., der Arbeitsgemeinschaft Sozialwissenschaftlicher Institute e.V. (ASI) und des Statistischen Bundesamtes. https://www.statistischebibliothek.de/mir/receive/DEMonografie_mods_00003695
Bentler PM (1990) Comparative fit indexes in structural models. Psychological bulletin 107(2):238–246. doi:10.1037/0033-2909.107.2.238
Brown EC, Hawkins JD, Arthur MW, Abbott RD, van Horn ML (2008) Multilevel analysis of a measure of community prevention collaboration. American journal of community psychology 41(1-2):115–126. doi:10.1007/s10464-007-9154-8
Brown EC, Hawkins JD, Arthur MW, Briney JS, Abbott RD (2007) Effects of Communities That Care on prevention services systems: findings from the community youth development study at 1.5 years. Prevention science the official journal of the Society for Prevention Research 8(3):180–191. doi:10.1007/s11121-007-0068-3
Brown EC, Hawkins JD, Arthur MW, Briney JS, Fagan AA (2011) Prevention Service System Transformation Using Communities That Care. Journal of community psychology 39(2):183–201. doi:10.1002/jcop.20426
Brown EC, Hawkins JD, Rhew IC, Shapiro VB, Abbott RD, Oesterle S, Arthur MW, Briney JS, Catalano RF (2014) Prevention system mediation of communities that care effects on youth outcomes. Prevention science the official journal of the Society for Prevention Research 15(5):623–632. doi:10.1007/s11121-013-0413-7
Browne MW, Cudeck R (1992) Alternative Ways of Assessing Model Fit. Sociological Methods & Research 21(2):230–258. doi:10.1177/0049124192021002005
Campbell MJ, Donner A, Klar N (2007) Developments in cluster randomized trials and Statistics in Medicine. Statistics in medicine 26(1):2–19. doi:10.1002/sim.2731
Dearing JW, Rogers EM, Meyer G, Casey MK, Rao N, Campo S, Henderson GM (1996) Social marketing and diffusion-based strategies for communicating with unique populations: HIV prevention in San Francisco. Journal of health communication 1(4):343–363. doi:10.1080/108107396127997
Donner A, Klar N (2000) Design and analysis of cluster randomization trials in health research. John Wiley & Sons, Ltd, Chichester
Dorer B (2021) Advance Translation—The Remedy to Improve Translatability of Source Questionnaires? Results of a Think-Aloud Study. Field Methods 35(1):33–47. doi:10.1177/1525822X211072343
Edwards RW, Jumper-Thurman P, Plested BA, Oetting ER, Swanson L (2000) Community readiness: Research to practice. J. Community Psychol. 28(3):291–307. doi:10.1002/(SICI)1520-6629(200005)28:3<291:AID-JCOP5>3.0.CO;2-9
Gansefort D, Peters M, Brand T (2020) Wie bereit ist die Kommune? Das Community Readiness-Modell und die beispielhafte Anwendung in der kommunalen Gesundheitsförderung. Gesundheitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)) 82(11):868–876. doi:10.1055/a-1119-6181
Garner CL, Raudenbush SW (1991) Neighborhood Effects on Educational Attainment: A Multilevel Analysis. Sociology of Education 64(4):251. doi:10.2307/2112706
Goodman RM, Speers MA, McLeroy K, Fawcett S, Kegler M, Parker E, Smith SR, Sterling TD, Wallerstein N (1998) Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health education & behavior the official publication of the Society for Public Health Education 25(3):258–278. doi:10.1177/109019819802500303
Groeger-Roth F, Frisch JU, Benit N, Soellner R (2015) Risikofaktoren für problematischen Substanzkonsum von Jugendlichen – Zur Anwendbarkeit des Communities That Care Schülersurveys auf kommunaler Ebene. SUCHT 61(4):237–249. doi:10.1024/0939-5911.a000379
Hawkins JD, Catalano RF, Arthur MW, Egan E, Brown EC, Abbott RD, Murray DM (2008) Testing communities that care: the rationale, design and behavioral baseline equivalence of the community youth development study. Prevention science the official journal of the Society for Prevention Research 9(3):178–190. doi:10.1007/s11121-008-0092-y
Hayes AF, Coutts JJ (2020) Use Omega Rather than Cronbach’s Alpha for Estimating Reliability. But… Communication Methods and Measures 14(1):1–24. doi:10.1080/19312458.2020.1718629
Hu L, Bentler PM (1998) Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods 3(4):424–453. doi:10.1037/1082-989X.3.4.424
Jonkman H (2015) Communities That Care in Niedersachsen. Ergebnisse der Implementation von CTC in fünf Standorten (2013-2014)
Jonkman H, Aussems C, Steketee M, Boutellier H, Cuijpers P (2015) Prevention of Problem Behaviours among Adolescents: The Impact of the Communities that Care Strategy in the Netherlands (2008–2011). DEV 9(1):37–52. doi:10.3233/DEV-13121
Kehl M, Brew-Sam N, Strobl H, Tittlbach S, Loss J (2021) Evaluation of community readiness for change prior to a participatory physical activity intervention in Germany. Health promotion international 36(Supplement_2):ii40-ii52. doi:10.1093/heapro/daab161
Nickel S, Süß W, Lorentz C, Trojan A (2018) Long-term evaluation of community health promotion: using capacity building as an intermediate outcome measure. Public health 162:9–15. doi:10.1016/j.puhe.2018.05.008
Nickel S, Süß W, Trojan A (2013) Messung der Kapazitätsentwicklung als Ansatz für Quartiersbeteiligung. Präv Gesundheitsf 8(3):163–170. doi:10.1007/s11553-013-0392-3
O'Brien RM (1990) Estimating the Reliability of Aggregate-Level Variables Based on Individual-Level Characteristics. Sociological Methods & Research 18(4):473–504. doi:10.1177/0049124190018004004
Repke L, Dorer B (2021) Translate Wisely! An Evaluation of Close and Adaptive Translation Procedures in an Experiment Involving Questionnaire Translation. International Journal of Sociology 51(2):135–162. doi:10.1080/00207659.2020.1856541
Rhew IC, Brown EC, Hawkins JD, Briney JS (2013) Sustained effects of the Communities That Care system on prevention service system transformation. American journal of public health 103(3):529–535. doi:10.2105/AJPH.2011.300567
Richard L, Gauvin L, Potvin L, Denis J-L, Kishchuk N (2002) Making youth tobacco control programs more ecological: organizational and professional profiles. American journal of health promotion AJHP 16(5):267–279. doi:10.4278/0890-1171-16.5.267
Röding D, Soellner R, Reder M, Birgel V, Kleiner C, Stolz M, Groeger-Roth F, Krauth C, Walter U (2021) Study protocol: a non-randomised community trial to evaluate the effectiveness of the communities that care prevention system in Germany. BMC public health 21(1):1927. doi:10.1186/s12889-021-11935-x
Rogers EM (1995) Diffusion of Innovations: Modifications of a Model for Telecommunications. In: Stoetzer M-W (ed) Die Diffusion von Innovationen in der Telekommunikation. Springer, Berlin, Heidelberg, pp 25–38
Sampson RJ, Raudenbush SW, Earls F (1997) Neighborhoods and violent crime: a multilevel study of collective efficacy. Science (New York, N.Y.) 277(5328):918–924. doi:10.1126/science.277.5328.918
Schwendemann HE, Frisch JU, Reder M, Mößle T, Soellner R, Bitzer E-M (2018) The Communities That Care (CTC) Family Attachment Scale: Measurement Invariance Across Family Structures. J Child Fam Stud 27(5):1440–1448. doi:10.1007/s10826-017-1009-9
SDRG (2009) Community Youth Development Study, Community Key Informant Survey, Seattle, WA
Shapiro VB, Oesterle S, Hawkins JD (2015) Relating coalition capacity to the adoption of science-based prevention in communities: evidence from a randomized trial of Communities That Care. American journal of community psychology 55(1-2):1–12. doi:10.1007/s10464-014-9684-9
Shinn M (1990) Mixing and matching: Levels of conceptualization, measurement, and statistical analysis in community research. In: Tolan P, Keys C, Chertok F, Jason LA (eds) Researching community psychology: Issues of theory and methods. American Psychological Association, Washington, pp 111–126
Steiger JH, Lind C (1984) Statistically based tests for the number of common factors: Annual meeting of the Psychometric Society, Iowa City, IA
Tucker LR, Lewis C (1973) A reliability coefficient for maximum likelihood factor analysis. Psychometrika 38(1):1–10. doi:10.1007/BF02291170
Wears RL (2002) Advanced statistics: statistical methods for analyzing cluster and cluster-randomized data. Academic emergency medicine official journal of the Society for Academic Emergency Medicine 9(4):330–341. doi:10.1111/j.1553-2712.2002.tb01332.x
Wihofszky P, Hofrichter P, Layh S, Jahnke M (2021) Transfer partizipativer Forschungsergebnisse in die Praxis: Das Beratungsinstrument Standortanalyse in der kommunalen Gesundheitsförderung. Bundesgesundheitsblatt, Gesundheitsforschung, Gesundheitsschutz 64(2):199–206. doi:10.1007/s00103-020-03273-x

Download PDF

Journal Publication

published 17 Apr, 2023

Read the published version in Journal of Public Health →

Version 1

posted

You are reading this latest preprint version

Validation of an instrument to measure community capacity building for prevention

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Methods

Study Design

Sample and Recruitment

Data Collection

Constructs and variables

Construct 1: Adoption of Science-based Prevention

Construct 2: Prevention Collaboration

Construct 3: Sectoral-Collaboration

Construct 4: Financial Support for Prevention

Construct 5: Support for Prevention

Confounders

Missing data

Statistical Analyses

Results

Descriptive Statistics

Structural Validity

Internal Consistency and Inter-Rater-Agreement

Concurrent Construct Validity

Discussion

Interpretation of the Results

Limitations

Conclusion

Declarations

Funding

Conflicts of interest/Competing interests

Ethics approval

Consent to participate

Consent for publication

Availability of data and material

Code availability

Authors’ contribution

Acknowledgments

Compliance with ethical standards

References

Supplementary Files

Status:

Journal Publication

Version 1