The unit external to the ODP managing organization was excluded from the present study in order to focus the evaluation on intra-organizational processes. We collected the present data in March 2015, 16 months after the end of the ODP. The time gap was because the analyses of the final summative inquiry and mixed-methods study, both administered to the frontline therapists in spring 2014, revealed a need to complement our understanding about the realization of the ODP. The present authors’ connections to the ODP and the present study are presented in Table 2: Their connections to the managing organization and their mutual professional relationships are presented in Additional File 1.
[insert Table 2 here]
The Rapid Assessment Process model (RAP) (30) and the Revised Socratic Approach for Health Technology Assessment (HTA) (31) were used as technical guides while creating the data collection protocol and interview guides, which are presented below. Normalization Process Theory (NPT) (32,33) was applied for the deductive qualitative content analysis (34). These three instruments are presented and their use is explained in Additional File 2. In reporting the study, we adhered to the 32-item checklist of consolidated criteria for reporting qualitative studies (COREQ), which is presented in Additional File 3 (35).
Formation and description of the study group
We assembled the study group according to the purposeful sampling strategy ‘complete target population’ (36). We emailed the invitation to the whole ODP project group and all team leaders of the target units, 14 individuals in total. Only one recipient involved in the project group declined the invitation due to compelling personal reasons, thus resulting in a study group of 13 individuals. We informed the study group in advance about the purpose, setting and course of the study as well as the principles for handling the data. This included information about the videotaping of the interviews and the assurance that no interviews would be transcribed due to the sensitive nature of the material and further the assurance that each participant’s identity would be protected as far as possible during processing and utilization of the information obtained. Recipients were assured that participation in the study was voluntary and would in no way affect their status within the organization. All members of the study group gave verbal consent to participate.
The study group was divided into two Focus Groups (FG 1 and FG 2) according to each member’s relation to the ODP (for more detail, see Table 2). All members of the study group and the researchers had been permanently employed in the organization for years before the launching of the ODP, thus their relationship was established prior to the present evaluation.
Interview protocol and guides
We interviewed FG 1 twice (FGI 1.1 and FGI 1.2) and FG 2 once (FGI 2) (Figure 2). Each interview lasted three hours and was divided into two parts with a short break between them. Four members out of five in FG 1 and five out of eight in FG 2 attended the group interviews in person. Four individuals were unable to attend the group interviews in person due to pressure of work but provided the desired information in alternative ways: The FG 1-enrolled associate executive was interviewed separately immediately after FGI 1.1 and the information obtained was included in the respective report. One FG 2-enrolled person provided written feedback before the FGI 2, and this information was presented to FG 2 during the interview. The remaining two FG 2 -enrolled people who were unable to attend in person had discussed the issues beforehand with their attending colleague.
[insert Figure 2 here]
Figure 2. Setting for the iterative semi-structured Focus Group Interviews and gathering the raw data. Abbreviations: FG 1 = Focus Group 1; FG 2 = Focus Group 2; FGI 1.1 = the first interview with FG 1; FGI 2 = interview with FG 2; FGI 1.2 = second interview with FG 1.
The same interview guide was used for the interviews with FGI 1.1 and FGI 2. It comprised five topics and nine guiding questions relevant to the ODP. We had recognised five topics that would need to be evaluated in depth; these concerned the underlying motives and intentions of the ODP, its management, the perspectives of the participating units, the interests of the individuals conducting the present evaluation and creating a quick vision for future developments. To ensure that the interviews yielded sufficient of the information sought, we selected questions relevant to our topics from the Revised Socratic Approach for HTA (31). Finally, we reformulated the original questions to match the ODP and referred to them as guiding questions. The creation of the interview guide is written out in Additional File 4, Table A.
The FGI 1.2 interview guide (Additional File 4, Table B) was compiled according to the principles of the RAP. This includes FG 1’s involvement in reflexively appraising the description of ODP processes that had been outlined by the time of their second interview. The purpose of repeated data collection was to ensure richer and more accurate data.
While creating the interview protocol and guides, the first author had reflective discussions about the mission and procedure with the collaborating researcher, the clinical director and the principal programme executive in the spirit of RAP. Due to the setting, we had no opportunity to pilot the interview protocol and guides in practice.
Forming the raw data
As a base, we had the technical data on the ODP comprising the implementation plan of ODS-I (28), the research plan of the ODS including the protocol for data collection (Table 1) and total executive resources in ODP (Figure 1). We gathered the supplementary information through an iterative and collaborative process of FGIs and finally wrote one, rich narrative on the realization of the ODP that comprised the raw data (more detail in Figure 2).
Qualitative content analysis
The case of our study was the process of running the ODP all the way from its rationales to its completion, and the unit of analysis was the narrative formed by the raw data (36). We analysed the raw data through deductive qualitative content analysis (34) guided by NPT (see Additional File 5 for coding frame) (32,33). Our analysis and extracting the results progressed in four steps: First, we encoded the raw data using different colours and reorganized it according to the main categories. Second, we re-encoded and organized the data further according to the subcategories. We reviewed the relevance of the encoding during the two first steps and readjusted when needed. Third, we condensed and rewrote the information contained in the encoded text pieces into a fluent narrative in terms of each subcategory. Fourth, we answered the research questions on the basis of the data analysed, thereby providing the results of the present study.
The first author performed the coding and extracted the results in close consultation with the second author. Finally, we presented the results to the members of FG1 for appraisal and possible amendments. They suggested some refinements and, after these had been made, they accepted the results presented below. The analysis of the data was processed manually with assistance of Word for Mac 2011.