Content Validity of A Questionnaire Exploring the Knowledge Translation Network Among Nursing Professionals

Background: For more than four decades, studies have reported the existence of a large gap between research and clinical practice. This means that research results are not contributing to the improvement of care, and there is a high level of variability which complicates deciding on and implementing the “best care” available. In this context, it is important to understand how nurses use the research by generating different knowledge translation networks in clinical practice. Therefore, a tool is required to detect the structure of the links that are generated during the research utilization. Aim: to validate the content of a questionnaire for exploring the knowledge translation networks established among nursing professionals. Methods: A descriptive cross-sectional content validity study of an instrument created to explore the knowledge translation network between nurses was carried out. The participants were divided into two groups: experts in Social Network Analysis (SNA) and practicing nurses. The instrument was a questionnaire with items that collected both the network's variables, and those of the individual. The content of the items of the questionnaire was validated as set out in the Content Validity Index (CVI). Results: The total Content Validity Index (CVI) of the survey was 0.91. All items received an excellent rating (K*>0.81). The modied kappa coecient for the whole scale was 0.9. Conclusions: The validated questionnaire has good content validity for exploring social networks for knowledge translation and research utilization among nursing professionals.


Introduction
For more than four decades, studies have established the existence of a separation between research and clinical practice, suggesting that the results of research do not contribute to improvements in care (1)(2)(3)(4)(5). This is relevant to situations involving nurses and research utilization. In other words, the nursing role, in addition to promoting awareness of and accessibility to evidence, has been taking on unprecedented dimensions, especially through projects with worldwide repercussions, such as Nursing Now. This project considers 2020 as the International Year of the Nurse and Midwife, as designated by the World Health Organization (WHO) and the International Council of Nurses (ICN). Nursing Now focuses on the critical position nursing holds in different health settings, and is responsible for collating solid evidence that can demonstrate the impact of care (6).
However, to generate evidence and analyze its impact, it is important to understand how nurses use the research, and its implementation in clinical practice, in a standardized way. Research utilization is de ned as a process by which nurses combine explicit research-based knowledge with tacit practice-based knowledge (7). Implementation, on the other hand, is the systematic adoption of the results of research into clinical practice (8).
When, in addition to this implementation process, nursing professionals adapt research ndings to make sense of them in the context of their daily work, this is the process of the knowledge translation into clinical practice (9). Knowledge translation is a dynamic process that includes the synthesis, dissemination, sharing, and ethical application of knowledge to improve health, provide more effective health services, and strengthen health systems (2,10,11). However, throughout this process, we must not forget that scienti c knowledge is not only held by individuals, but is also produced and maintained by groups of people working in the same environment (12,13).
Scienti c evidence can be considered an innovation, and its dissemination the process of communicating it to individuals, groups, and organizations (14). Both, the dissemination of innovation and its implementation take place in group settings, where the links and relationships between professionals can in uence (4,15,16)(4)(4)(4) (4)(4)(4)(4)(4) (Neta et al., 2018). Therefore, the links and relationships between professionals, and the networks that may arise between them, are contextual elements to be taken into account during the implementation process (17). The combination of health professional networks and research utilization is the starting point for our research, which aims to answer the questions around which interprofessional links can in uence knowledge translation.
In the eld of healthcare, there are different groups of people (work units or professional groups to which they belong: doctors, nurses, auxiliary care technicians, training specialists, students), which could generate different knowledge translation networks (18). A knowledge translation network is a set of nodes, which may be individuals or other forms of collective organizations, that seek, create, and transmit knowledge, and are interconnected by social relationships that favor or impede the knowledge translation (17,19,20).
Likewise, in this implementation process, there are also people who have a particular in uence on the beliefs and actions of their peers. They are the opinion leaders, who exercise in uence through their authority and status, or through their representativeness and credibility among equals (21,22). A key point for the implementation of evidence is the identi cation of the so-called opinion leaders that can facilitate this process (23). In previous experiences with opinion leaders trained to in uence the behavior of their peers with respect to the use of research, the impact has generally been positive in terms of changing decision making, but their in uence on patient outcomes is still unknown (9,22).
In relation to these types of networks that co-exist in health organizations (24), it would be of interest to have a tool that could make it possible to capture the structure of the links that are generated during research utilization. Considering the in uence of relational contexts on knowledge translation, and the importance of key people in this process, our question focuses on identifying what relational structures or networks are created among nurses in the research utilization and the knowledge translation processes, and establishing the relational pattern of the people who in uence these processes.
To this end, a methodology was chosen that focuses on the analysis of relationships, speci cally Social Network Analysis (SNA). SNA has been widely applied to the analysis of friendship networks (25), together with networks of advice and collaboration (26,27) in the eld of nursing and health. A systematic review carried out by Chambers et al. found that there is little evidence of the application of SNA in the health eld. However, it indicates that it may have the same importance and utility that it has shown in other areas, and recommends future research that implements and evaluates interventions based on how SNA can be applied (28).
Likewise, studies carried out within the framework of health interventions have shown that interventions constitute relational processes; that is, they are scenarios of interactions between participants, which can in uence the implementation process and the results (29,30).
On the basis of the above, it is clear that SNA can be useful in exploring knowledge translation and research utilization, and is useful in the context of health, in particular to analyze relationships between nurses. The analyses for the SNA studies were developed using data obtained through the use of self-completed questionnaires, and some researchers have developed their questionnaires from speci c items to measure the network of advice and collaboration (26), or knowledge management (31). However, no studies on validated tools have been found in network studies, which analyze knowledge translation among nurses. The creation and validation of the content of a tool to explore the knowledge translation and research utilization within a network could standardize the generation of questions, as well as map possible answers and relationships (32).

Aims
To validate the content of a questionnaire to exploring the knowledge translation network among nursing professionals.

Design
Descriptive cross-sectional content validity study of an instrument developed to explore the knowledge translation network among nursing professionals.

Participants
The participants were divided into two groups: experts in Social Network Analysis (SNA), and practicing nurses without knowledge of SNA, as follows.
Experts in SNA. Ten national experts were contacted. The experts were selected on the basis of an intentional non-probability sample. The experts were considered to be professionals in Health Sciences and related disciplines (psychology, anthropology, sociology, and industrial and computer engineering) with research experience in SNA, and in the eld of health and society. The experts came from the University of Leon (7), the University of Southampton (1), the Carlos III Institute of Health in Madrid (1) and a health center in Valencia.
Active nursing professionals with more than 1 year of experience and no knowledge of SNA. Ten nursing professionals were contacted. These professionals were selected from the hospital center (University Hospital of Móstoles, Madrid) in which the questionnaire that is the subject of this research would be applied, after its validation. The professionals selected for validation would be excluded from the subsequent application of the questionnaire once it had been validated.
The selection criteria of the participants are shown in the Table 1.  (14,37).
Each item described the transfer or use of a resource, determining a different network. For example, the item "Which of the following sources do you go to for information?", generates a network called the Sources of Research Utilization Network; and the item "Who do you go to for advice on issues related to research results?" generates the Advice for Knowledge Translation Network. Some of the items were formulated in different directions to test the responses of the respondents more rigorously, by asking who you go to, and in another item, who comes to you.
Individual variables related to the presence of organizational factors in the workplace for research utilization (time to participate in research and funding activities) and socio-demographic and professional variables (age, sex, years of experience, degree, unit in which you work, previous research training) we also collated. The individual variables collating organizational factors were included in the validation process.
In all but one network (variable to whom you give advice), the frequency was taken into account. The original items and the adaptation for the study are shown in Table 2. With the items prepared for the questionnaire, we proceeded to validate the content through the Content Validity Index process (CVI). The CVI is based on the agreement reached by a group of experts regarding the relevance of the items of a scale for the construct to be measured (38) (38). It measures both the agreement reached on the relevance of each item on the scale, and the relevance of the scale as a whole. A correction is also applied for random agreement through the modi ed Kappa index, since it is based on only one type of agreement; the agreement of relevance.
The validation process was carried out on the basis of three phases, as described below: 1. The rst version of the questionnaire was set out from the items selected from the specialized literature review.
2. Content Validity Index. The initial version of the questionnaire was distributed by e-mail to 10 SNA experts.
Each item in the questionnaire was subdivided into the number of response options (sub-items). With this organization, the experts evaluated the relevance of each item and sub-item using a Likert scale from 1 to 4, where 1 was not relevant at all and 4 was very relevant (See Appendix A).
3. Piloting a sample of nursing professionals. Once the validity of the content had been established, and considering the pertinent modi cations, the nal questionnaire was piloted (see Appendix B) with a group of 10 nursing professionals belonging to the target population in which the questionnaire is to be used. The purpose was to assess aspects related to adequate understanding of the questionnaire and the average time of completion.

Data Analysis
The Content Validity Index was drawn up in accordance with the proposal set out by Lynn (1986), Polit & Beck (2006) and , in which the content validity of each item was established with the probability of random agreement (38)(39)(40). The probability of random agreement among experts (pc) was calculated by dividing the number of experts who gave a score of between 3 and 4 to the item, by the total number of participating experts. The assessment criteria for K* values (modi ed Kappa coe cient), were taken from Streiner & Norman (2015) (41); the most desirable outcome is to reach agreement between good and excellent (Table 3).

Description of the Expert Panel and the Pilot Sample of Nursing Professionals
For the 10 SNA experts recruited for this study, seven were women and three men, with an average age of 34.7 (SD: 8). In relation to their academic quali cations, 30% were doctors, 50% were PhD students, and the rest held a degree in nursing and/or a master's degree. A total of 90% had carried out research in SNA and 10% in health areas. All (100%) had scienti c publications in the eld of SNA.
The 10 pilot participants were nurses from a resuscitation unit who were not active at the time; 100% were women, and they were an average age of 32.9 years old (SD: 3.38).

Content Validity Index
With the scores given by the experts, the content validity of the survey was calculated, which in accordance with the content validity index (CVI) was 0.91. The validity by choice of answer was 0.91 for question 1; 0.90 for question 2; 1 for questions 3, 4 and 6; 0.90 for question 5, and 0.95 for questions 7 and 8. All eight items had an excellent rating (K*>0.81). With these values, we can say that the average of the modi ed kappa coe cient for the whole scale was 0.9 (Table 4). It should be noted that, since the modi cations suggested by the experts were exclusively formal, it was decided not to go through the questionnaire again to modify and evaluate these aspects, and a nal questionnaire was drawn up after including their considerations (see Appendix B). The suggestions incorporated were: Add "social networks, team meetings and informal gatherings" as a place to share research results.
Add "middle managers" as a professional group to be asked for advice.
Improve the wording of one of the items (item 4).
The questionnaire in its nal version was piloted on 10 nursing professionals, who reported no problems in understanding it. The piloting allowed the average time of completion to be established, which was 13.5 minutes (SD: 1.3).

Expert Panel and Piloting in Nursing Professionals
For the validation process of this questionnaire, SNA experts were used; speci cally, professionals with research experience in the context of SNA.
Similar validation studies of questionnaires carried out in the same context as our study, in the hospital environment, considered as "experts" professionals with experience in the subject to be evaluated (14,42). Both the Escobar et al. and Pancorbo et al. studies carried out more than one round of evaluation by the experts. In our study the number of items was considerably lower, and the modi cations suggested by the experts in the rst round were exclusively formal, so it was decided not to carry out a second round of evaluation of the questionnaire. In the mentioned studies, the piloting of the questionnaire with nursing professionals was carried out in the context of studying the same application.

Content Validity Index
The values of the modi ed Kappa coe cient, according to the criteria of Streiner et al. (2015), obtained in this study were excellent for all the items in the questionnaire. Therefore, the content validity of the questionnaire can be considered good (41).
Some previous studies on networks have incorporated speci c items (advice, help, knowledge management, use of research) previously used by other authors, and adapted them in accordance with the objectives of their proposed study (43,44). It is true, on the one hand, that these designs have been made from scaled items whose reliability and validity had been previously contrasted in the literature, but isolated items were used and not the whole tool. On the other hand, SNA aims to create tools that capture the relationships between subjects; therefore, when choosing items, relevance criteria may prevail. A certain degree of subjectivity must be added to this process, both when choosing the items that nally make up the scale, and in the assessments made by the person answering the questionnaire.
In studies with other types of methodologies (validation or adaptation of questionnaires or tools in quantitative research), instruments have been validated to measure the variables, or new tools created and validated as an element of quality that enhances the internal and external validity of the study (14,42,45,46). However, the objective of this type of study differs from that of SNA. More theoretical studies on the use of SNA in the context of knowledge translation in the eld of nursing considered SNA an important tool in determining how some professionals contribute to knowledge translation (24,47).
For this study, in addition to using individual items that had previously been validated jointly (25,26), the calculation of content validity was added; an aspect that can contribute more consistency to the items used to explore networks, and quality to the questionnaire created; in the same way, a questionnaire with a suitable CVI can standardize the generation of questions to explore speci c networks. However, it is necessary to take into consideration that all measures are based on subjective perceptions, an aspect that has to be taken into account despite the quality of the results obtained in the validity analyses performed.
The creation of this questionnaire was part of a larger study, the SUMAMOS EXCELENCIA ® project (48), which aims to analyze the effectiveness of a model for the implementation of evidence, based on quality audit cycles.
This national project aims to create a new knowledge translation network around each participant, each unit, and each participating center, and for this network to help change behavior with regards to research utilization and its maintenance. These new inter-professional links can in uence the speeding up of the dissemination of knowledge, behavior, and evidence. Following the application of this model, an SNA study using the Knowledge Translation Network (KTN) Questionnaire has been carried out in several hospital units that participated in this project, together with a control unit that did not participate. The intention of this mixed analysis was to try to add an approach that can enrich the quantitative character of this study, and to look for an explanation for certain phenomena related to research utilization and the implementation of knowledge that have to do with networks and with the context in which the stakeholders are related. The SNA approach aims to capture relational data that can explain social behaviors related to the knowledge translation process: translating intentions into changes.

Limitations of the study
The limitations of the study presented here are related to the limitations of the CVI calculation itself (40). The consideration of the Content Validity Index as a value for analyzing an instrument is relatively new in the eld of SNA, so more studies that replicate this method are required. The CVI is a subjective method, so there may be biases around the aforementioned subjectivity of the experts. In the case of the questionnaire which was the object of this validation, it is necessary to take into account that the information it generates comes from unique social networks, which prevents its generalization, and that a node may be in uenced by memory bias, since the participants have to remember certain relational aspects that have occurred in the past.
It may be interesting to create future lines of research that go into greater depth as to which interprofessional relational structures are suitable for the knowledge translation, and the role of opinion leaders in knowledge implementation.

Conclusion
This study aimed to validate the content of a questionnaire created to explore knowledge translation and research utilization through the social networks established among nursing professionals. The most relevant conclusions are listed below: The validated questionnaire has good content validity for exploring the knowledge translation and research utilization networks among nursing professionals.
The Content Validity Index used in questionnaires for SNA studies can add more consistency to the items used to explore these networks.