A total of 11 people participated in the focus group and semi-structured interviews. Six participants (R5, R6, R8, R9, R10, and R11) were employed as experts. Their job titles were Information Manager, Chief Information Security Officer, Patient Communication Specialist, Medical Device Regulation Specialist, Technical Application Manager, and Functional Application Manager. Two participants (R4 and R7) were employees of the hospital and took part in the speed dating concept as idea owners. Finally, three participants (R1, R2 and R3) were employees of the HIL. Participant characteristics are summarized in Tables 1 and 2.
Table 1
Characteristics of participants in the focus group
| R1 | R2 | R3 | R4 | R5 | R6 |
Gender | Male | Male | Female | Female | Male | Male |
Background | HIL | HIL | HIL | Idea owner | Expert | Expert |
Job title | Information Manager | Information Manager | Information Manager | | Chief Information Security Officer | Information Manager |
Table 2
Characteristics of participants in the semi-structured interviews
| R7 | R8 | R9 | R10 | R11 |
Gender | Female | Female | Male | Female | Female |
Background | Idea owner | Expert | Expert | Expert | Expert |
Job title | | Patient Communication Specialist | Medical Device Regulation Specialist | Technical Application Manager | Functional Application Manager |
Five major themes emerged during the thematic analysis of the transcriptions:
-
Preparation before the Sign in
-
Mapping out the process between Sign in and Sign out
-
Defining the roles and deliverables of experts
-
Guiding the idea owner
-
Supporting capacity of resources
Each theme will now be described in further detail. Additionally, Table 3 provides an overview of all the main points raised by respondents per theme.
Preparation before the Sign in
The HIL shares a short description of the innovation with the experts prior to the Sign in. This enables the experts to read up on the innovation and think ahead about questions to ask during the Sign in. Three respondents described being satisfied with this short description, while six respondents claimed that experts still missed important information about the innovation. They would have preferred to have this information available at an earlier stage:
“[…] when we are far into the process: ‘Isn’t it useful to check whether [the innovation] has a digital privacy impact analysis?’ Then I think: ‘Why didn’t we ask that question at the beginning?’.” (R5)
Besides the information intended for the experts, three respondents also described missing information for idea owners prior to the Sign in. This especially concerned the pitch, in which the idea owner presents the innovation to the experts. Several experts experienced that the pitches often differed in their levels of preparation and depth. Two respondents specifically mentioned the need to remind idea owners to present in regular Dutch, not using Latin or English medical terminology and abbreviations.
Two respondents from the HIL did not agree with this need for additional information prior to the Sign in, as presented by the experts and idea owners. They explained that the Sign in was initially intended to only pitch the idea. One respondent warned that over-preparing and informing experts and idea owners would make the speed dating concept feel akin a regular application process, which would be at the expense of the current interaction between experts and idea owners:
“[Otherwise] you will see that the idea owner says: Is that so? If I fill in ‘yes’ here [on a preparatory questionnaire], then: Oh, what a hassle that will be! I’ll fill in ‘no’!.” (R2)
However, three respondents suggested still letting the idea owner complete a “smart form” prior to the Sign in to provide additional information to the experts. This form could serve as a quick scan to analyse and assess DH ideas at an early stage. The group proposed designing a questionnaire to collect details about the innovation idea, such as the target population, motive for development, method of development, and whether certain requirements had already been met. An important question would be whether the innovation will be self-developed or purchased as an existing product. Second to this, it would be helpful to know if the innovation would be considered a medical device and would need to be CE marked:
“To quickly determine ‘he needs that or not’. [To] see what needs to be done […], what risks [there are], ‘does the idea meet certain elements?’ of which we can say: ‘Then can we take the next step?’.” (R5)
In addition to gaining a better understanding of the innovation, it was expected that this questionnaire would enable experts to provide more specific advice. This could also mean attaching a different expert to the innovation than usual, due to specific knowledge or experience. One respondent also expected the information to make it easier to determine whether a similar innovation already existed within the hospital. One respondent suggested designing the smart form as a web application.
Mapping out the process between Sign in and Sign out
All respondents described feeling involved with the innovation during the Sign in. For example, one respondent felt that the Sign in increased cooperation between experts and idea owners, due to the time spent together and getting to know each other at a personal level. Two respondents described that this made it easier to plan follow-up appointments.
Despite this general feeling of involvement (and the expectation that this would be the same during the Sign out), six respondents (mainly experts) indicated that the experts did not feel involved in the phase between the Sign in and the Sign out:
“We have had the Sign in and some follow-up sessions, but now it is quiet. […] I am not aware of the development.” (R10)
Additionally, two respondents (both idea owners) found it difficult to understand which steps needed to be taken between the Sign in and the Sign out:
“[…] then you do have those contacts [but] I don’t know yet […] in which order I should see those names. […] What are the steps I have to go through? And in what order? […] When do I have to deliver something? And which documents?” (R7)
To strengthen the involvement of experts with innovation, four respondents proposed digitally monitoring the progress of projects between the Sign in and Sign out. One respondent suggested doing so using a website, newsletter or Excel file. This would effectively force idea owners to provide updates and have the HIL employees share these with the experts. This would make it easier for experts to intervene when necessary:
“Maybe […] with a line with dots. That you see ‘[these expert fields are] connected… we’ll try this in a month’s time’. [That] you know: it is going well, stops for a while or the work is in progress. […] with colours you can indicate a kind of stage of progress.” (R8)
One respondent expected that it should be feasible for an idea owner to regularly write a summary of the progress. It was mentioned that the HIL could facilitate this by developing a standard reporting format. One respondent also expected that digital monitoring would make it possible for experts to perform a risk analysis of the innovation more efficiently:
“[That] certain expert fields already say [to] the rest of the experts: […] ‘I will get started with it, you take it easy for a while.” (R2)
To support idea owners in understanding which steps exist between the Sign in and Sign out, three respondents suggested creating a general roadmap, visualizing when an idea owner should contact a certain expert or when a certain deliverable becomes essential, such as a contract. It was noted, however, that this might be difficult to realize, because each new idea differs sufficiently from the previous ideas for the steps to be different as well.
Defining the roles and deliverables of experts
All respondents described that the speed dating concept successfully facilitated contact between idea owners and experts. This in turn led to a clearer understanding of which deliverables (such as documents, designs, risk assessments, and recommendations etc.) were needed per expert field in preparation of the Sign in and subsequent pilot phase. Four respondents remarked that this had made their work more efficient. Nevertheless, four other respondents still felt that it was unclear for idea owners what the division of roles and responsibilities between them and the experts was exactly. This example shows how a required document (in this case a digital privacy impact analysis) should have been drafted by a different expert in advance, but because this was not done, the question now lands with the idea owner, who is insufficiently equipped to complete this task:
“Then [an expert asked]: Has a digital privacy impact analysis been prepared already? [The idea owner responds:] Never heard of it. What do you need from me?” (R6)
Three respondents indicated an additional uncertainty. Some experts were unaware of who exactly the other experts were and what services they delivered during the innovation process. This sometimes resulted in confusion and was mainly explained by the freedom of the innovation process. Five respondents suggested that this was partly due to an overlap in the services provided by the various experts. One respondent did not agree though and suggested that it was due to the patient missing as an expert during the Sign in. The Communication expert effectively taking the place of the patient further contributed to the indistinct roles.
Various suggestions for improvement for the speed dating concept were mentioned, mainly focussing on combatting the lack of clarity. One idea owner suggested creating a list of names, job titles, and short descriptions of the deliverables per expert field:
“So that you have an idea why [for example] someone from Contract Management is present and what he wants to know about me [as an idea owner] and that I can take care of that.” (R7)
Two experts proposed organizing masterclasses about the expert fields, deliverables, and expected questions for the idea owner during the Sign in. This could be supplemented with information about the mission and vision of the speed dating concept. A document containing this information would make it easier for colleagues to familiarize themselves with it. This might benefit both idea owners as experts.
Three respondents from the HIL did not agree with the suggested forms of improvements to clarify the roles and deliverables of the experts, as presented by the experts and idea owners. They explained that the roles and deliverables of the experts can only be clarified by cooperation between experts and idea owners. One respondent noted that in this cooperation it is up to the experts themselves to clarify to the idea owners what their roles and deliverables are and therefore also when they are required to proceed:
“Experts are experts for a reason. […] They know better than anyone else when they are needed in the entire development process. […] but […] this [clarification] takes time.” (R2)
Guiding the idea owner
One respondent specifically pointed out being satisfied with the concept of idea owners itself, because this makes it clear who is responsible for the development process.
More practically speaking, two respondents described being satisfied with the guidance idea owners receive during the Sign in. An idea owner, for example, found it very helpful that notes are made containing all the names of the experts who ask questions and with whom follow-up appointments are to be made.
Experts, however, indicated that the guidance idea owners receive might still be lacking, because the pitches often differed greatly in the amount of information being presented. One respondent also experienced that idea owners were often insufficiently aware of the responsibilities of an idea owner. Additionally, the complexity of project leadership is regularly underestimated:
“[It] will depend on the project and idea owner, [but] often they are inexperienced people who have a nice idea, but don’t actually know about the process. […] We are not trained in that.” (R7)
Despite this desire among experts and one idea owner to expand the current guidance of idea owners, three respondents from the HIL warned about the risk of idea owners then simply transferring tasks to others, at the expense of the idea owner concept:
“I think the HIL is precisely intended to […] make use of the innovative power we have in this organization!” (R3)
Four respondents proposed introducing an innovation coach; someone who might assist the idea owner in the process of managing projects and making contacts. Three respondents suggested that the innovation coach should also regularly evaluate the progress of the innovation among the experts. It was expected that this would result in even stronger cooperation between experts and a clearer picture of what the speed dating concept entails, especially for experts that work outside the hospital walls:
“'(So) a person who collects what everyone has done and who contacts the other experts in the meantime: 'how are you doing?', 'have you already been in contact?' […] 'can I assume that you will take care of that? ' [And] also providing feedback to the Health Innovation Lab: 'these are the agreements made’’. (R11)
Two respondents suggested that the Sign in and the Sign out should be chaired by the innovation coach in the future to provide a clear overview of the involved stakeholders. The innovation coach could also manage time during the Sign in and take final responsibility for forwarding the pitch presentation afterwards. An additional suggestion was to let the innovation coach invite previous idea owners to the Sign in to remind experts of past successes and maintain their enthusiasm.
Supporting capacity of resources
Four respondents described being satisfied with the current workload and being able to handle it well with the available capacity of resources, such as finances, time, and employees. However, six respondents voiced concern that these resources will not be sufficient in the long term when two or three innovations per month will need to be processed. According to four respondents, this insufficiency was already becoming noticeable, since some experts are regularly absent from the Sign in, hurting the innovation process:
“If two or three expert fields are missing, the added value decreases enormously.” (R6)
For this reason, two respondents suggested organizing the speed dating concept at a higher organizational level within the hospital, with the aim of using the resources of different expert departments more efficiently. For example, one respondent suggested adding the Chief Information Officer or the Director of Patient Care as a sponsor to the speed dating concept to ensure approval of the resources requested. Alternatively, four respondents suggested letting the heads of the idea owner’s or experts’ departments act as sponsors in case of capacity issues. The reasoning being that this sponsor would be able to realize more specific resources, since C-level executives and directors are already responsible for a large number of projects and are generally less directly involved in the development process. In addition to adding a sponsor, it was suggested by one respondent to continue planning the Sign in for when as many experts as possible would be available.
Table 3
Structured overview main results
| Positive experience | Negative experience | Suggestions for improvement | Probable risks |
Preparation before the Sign in | Satisfaction about the HIL sharing a short description of the innovation with the experts prior to the Sign in | Experts still missed important information about the innovation Missing information for idea owners prior to the Sign in | Let the idea owner complete a “smart form” prior to the Sign in to provide additional information to the experts. For example: questionnaire, web application | Over-preparing and informing experts and idea owners would make the speed dating concept feel akin a regular application process, which would be at the expense of the current interaction between experts and idea owners |
Mapping out the process between Sign in and Sign out | Feeling of involvement with innovation during the Sign in (the Sign in increased cooperation between experts and idea owners and made it easier to plan follow-up appointments) and same expectation for the Sign out | Experts did not feel involved in the phase between the Sign in and the Sign out Idea owners found it difficult to understand which steps needed to be taken between the Sign in and the Sign out | Monitor the progress of projects between the Sign in and Sign out digitally Create a general roadmap, visualizing when an idea owner should contact a certain expert or when a certain deliverable becomes essential | A general roadmap might be difficult to realize, because each new idea differs sufficiently from the previous ideas for the steps to be different as well |
Defining the roles and deliverables of experts | The speed dating concept successfully facilitated contact between idea owners and experts Clearer understanding of which deliverables were needed per expert field in preparation of the Sign in and pilot phase. This has made work more efficient | Feeling that it was unclear for idea owners what the division of roles and responsibilities between them and the experts was exactly Some experts were unaware of who exactly the other experts were and what services they delivered during the innovation process. This sometimes resulted in confusion | Create a list of names, job titles, and short descriptions of the deliverables per expert field Organize masterclasses about the expert fields, deliverables, and expected questions for the idea owner during the Sign in (supplement this with information about the mission and vision of the speed dating concept) Create a document containing the information of the masterclasses | It is up to the experts themselves to resolve the lack of clarity about the roles and deliverables of experts. They know better than anyone else when they are needed in the entire development process |
Guiding the idea-owner | Satisfaction with the concept of idea owners Satisfaction with the guidance idea owners receive during the Sign in | Experts indicated that the guidance idea owners receive might still be lacking Idea owners were often insufficiently aware of the responsibilities of an idea owner The complexity of project leadership is regularly underestimated | Introduce an innovation coach who might assist the idea owner in the process of managing projects, making contacts, regularly evaluate the progress of the innovation among the experts, chair the Sign in and Sign out, manage time during the Sign in, take final responsibility for forwarding the pitch presentation afterwards and invite previous idea owners to the Sign in | When idea owners can simply transfer tasks to others, this would be at the expense of the idea owner concept and might diminish the innovative power of the organization |
Supporting capacity of resources | Satisfaction with the current workload and being able to handle it well with the available capacity of resources | Concern that the resources will not be sufficient in the long term | Organize the speed dating concept at a higher organizational level within the hospital: add a high-level sponsor Continue planning the Sign in for when as many experts as possible would be available | Since C-level executives and directors are already responsible for a large number of projects and are generally less directly involved in the development process, the heads of the idea owner’s or experts’ departments might be more suitable to act as sponsors in case of capacity issues |