Participants and interviews
Interviews were done by HM (6), HP (4), and TMD (2). Each lasted between 16 and 48 minutes (M = 33, SD = 11). Initial recruitment and the first four interviews were completed at VA’s SimLearn Center (simulation training facility) in Florida, where attendees were learning how to plan and run acute telestroke simulation training sessions. Subsequent recruitment and interviews were done by telephone. Given the program’s small size, we attempted to interview the entire cohort. All 13 eligible neurologists who were approached agreed to be interviewed; 12 (92%) were scheduled and interviewed in the second quarter of 2018.
Key characteristics of NTSP’s virtual service included its membership and organizational context. Eight participants (67%) reported previous practice in telemedicine, while only three participants (25%) knew at least one other NTSP specialist before joining. One-third of the virtual stroke specialists were also exclusive VA clinical providers. One specialist worked exclusively outside of the VA while the rest (58%) of the specialists split their full-time clinical positions between the VA and a university affiliate outside of NTSP. Also notable is NTSP’s flat organization within a larger vertical organization (VA); rather than using intermediaries, NTSP administrators and specialists typically communicated directly with each other and with spoke sites.
Motivation to join NTSP. We identified motivators for joining NTSP and methods for communicating and training. Reasons for joining included personal interest, professional growth, the flexibility of virtual work, and the opportunity to serve more patients.
Team implementation barriers and facilitators
Table 1 summarizes the implementation barriers and facilitators. Onboarding into the team presented challenges. At times, technological hurdles had the potential to stall implementation. Team dynamics and cohesion were viewed as implementation facilitators, as was support from the national leadership. Regular virtual meetings with professional development discussions were valued.
Table 1
NTSP Virtual Stroke Specialists’ Strengths, Weaknesses, Opportunities, and Threats
Strengths Opportunity to interact with other stroke professionals Often more in common with NTSP colleagues than local colleagues Feels supported by National team Faith/trust in fellow specialists Faith/trust in NTSP Medical Director Flexibility of telestroke shifts More meaningful than previous telemedicine practice/experience Stay current with stroke literature via group | Weaknesses Difficult onboarding Continuity of care post-stroke event |
Opportunities Improving onboarding strategies Increasing in-person interaction among specialists Video feed to add visual component to meetings Call routing could be optimized as demand increases | Threats Technology integration |
APPENDIX A |
Virtual Stroke Specialists NTSP Team Interview Guide |
We are doing this interview to better understand your experience, as part of the National Telestroke Program, of being part of a virtual “hub” of Telestroke providers. The arrangement of a virtual hub of stroke specialists, dispersed over a wide geographic area and different home institutions, is unique in the Telestroke literature and experience, and so we want to better understand the characteristics of this virtual team, and what things help or hinder successful virtual team practices. |
By participating in this interview, you are providing verbal assent to the interview process. The NTSP Evaluation is a VA quality improvement initiative, not a research project, and this interview is part of understanding how to improve virtual team experiences and function as this project continues to grow. We will transcribe the interviews, but any identifying information will be removed, and you as an individual will not be identified in any reports or discussion in any way. The interview will take about 30 minutes. Do you have any questions or concerns about the interview? |
Thanks for agreeing to be interviewed! These questions focus around your experiences and practices as part of the virtual team of stroke specialists in the NTSP. |
1. How long have you been a part of the telestroke program at the VA? |
a. Probe: Have you completed any Telestroke consults yet? About how many? |
b. Probe: How many shifts per month do you work as a telestroke hub provider? Do you work during the day or the evening? Do you work on weekends? |
c. Probe: Do you do other clinical work for the VA or is Telestroke your only VA clinical effort? If other VA effort, what clinical care do you provide? |
Probe: About how much of your time on a % basis is outside the VA and how much is within the VA? |
2. What would you say were the main reasons you decided to become a VA Telestroke provider? |
a. Probe: Before joining the telestroke did you know or work with any of the other hub providers. If so, in what context. |
3. What about the NTSP had caused you to keep participating in it? |
Probe: How has your experience as a hub neurologist differed from your original expectations? How has it changed during the course of your involvement? |
4. Have you participated in a “traditional” telehealth or hub/spoke community of practice before becoming a VA telestroke provider? (traditional: hub providers located in the same geographic location) |
a. If so, how is being a VA Telestroke provider like being a Telestroke provider in that other system? How is it different? |
b. What advantages, if any, do you think the VA Telestroke system offers you as a provider over the other system? What advantages, if any, does the other system offer compared to the VA NTSP? |
5. To what extent, as a VA Telestroke provider, do you feel like a part of a virtual healthcare team, or a virtual group practice? |
a. Probe: What’s an example you can share about a time when the telestroke providers worked well together in providing stroke care? What’s an example of a time when more teamwork among the telestroke providers might have been helpful in providing stroke care? |
6. What tools or methods do you use to communicate with other hub neurologists? |
a. Probe: (for each method mentioned): How often do you use that method? What kind of information do you share via that method? Does that method involve all NTSP hub providers or just some? Does that method involve the NTSP national team or just hub providers? |
7. What makes it easy to communicate with other NTSP providers? What makes it difficult? |
8. How does telestroke fit in with your other clinical responsibilities? |
a. Probe: Did becoming a hub provider require you to change anything about your clinical schedule? |
b. Probe: What do other colleagues (VA or non-VA) that are not part of the NTSP feel about your work in this program? |
c. Probe: How does being a VA Telestroke provider affect your overall professional identity as a stroke specialist? Do you see yourself primarily as a university provider or a VA provider or both? |
9. In your opinion–what does the National telestroke team do to facilitate a team environment (making you feel as though you are part of a team) of providers? |
a. Probe: What could they do more of to facilitate a team environment for the hub providers? (Examples if they can’t offer anything: during onboarding, promoting regular communication (in-person, telephone, electronic?), providing other platforms for interactions virtually and in person) |
10. To what extent do you can trust your fellow Telestroke providers in the same way you trust a local stroke colleague? Why or why not? |
a. Probe: Do you trust their clinical judgements? Why or why not? |
b. Probe: Do you trust their communication with you? |
c. Probe: Do you trust their communication with the local VA sites? |
Probe: Are there things that the NTSP could do to increase your trust of your virtual colleagues? |
11. Overall, how satisfied are you with the NTSP? |
a. Probe: What changes in the NTSP would increase your satisfaction as a provider in the virtual hub of stroke neurologists? |
12. Has participating in the NTSP influenced your thoughts about practicing in the VA healthcare system? |
a. Probe: Any other positive influences on your opinion about working in the VA? |
b. Probe: Any negative influences on your opinion about working in the VA? |
What else should we know about the National Telestroke Program that hasn’t come up yet in our discussion? |
Thanks so much for taking the time to talk with me. |
[Table 1 here]
Thematic model. From five related concepts, we identified four themes as phrases36 and summarized our results in a thematic model (Fig. 1). The model links communication, engagement, team cohesion, and tolerance of technology problems. We propose that team cohesion mediates the link between satisfaction and two other concepts: communication and engagement. Further, we propose that team cohesion moderates the potential negative effect of technology challenges on providers’ satisfaction.
Communication
Specialists used scheduled and unscheduled communication for complementary goals. Scheduled communication was used to clarify roles. It centered on a weekly case review conference call led by NTSP clinical leaders. Highlights from this call, attended only by NTSP specialists and administrators, were discussions of challenges, sharing of new research, and reflection on recent notable cases. Communication between weekly calls was a combination of mobile calls, mobile messaging, and electronic mail. Mobile calls were routed to the on-call consultant. Mobile messages used SMS and the mobile chat application WhatsApp (no PII or PHI was communicated). Email services included those provided by VA and other official affiliations. Through these tools, specialists clarified processes and addressed logistics and emergent needs. These tools also connected specialists directly with administrators.
[My] communications with [NTSP’s medical director] are oftentimes by telephone and it’s, it’s a real plus. I know that if anything significant comes up, I can call her and get a hold of her right away. [P12]
Conversely, a participant with scheduling conflicts during the weekly calls expressed decreased feeling of team presence.
Unfortunately, I have my clinic on Monday morning…. But yeah, I think that that’s one of the…. That would probably be one of the things that may diminish what I feel in terms of being part of the team. [P11]
Participants also noticed situations when the communication tool or timing seemed inappropriate for the message (e.g., in urgency).
[If operational changes are coming, then…] thoroughly discuss those in the weekly conference calls and then certain the time in the future will, where they will be implemented, you know, rather than …sending out an e-mail during the week and indicating the changes expected, you know, right away. [P12]
Engagement In Problem Solving
Specialists wore multiple hats to address the program’s growing needs. A factor facilitating team cohesion was engagement in program development. Leadership cultivated team salience by further engaging the virtual stroke specialists in implementation problem solving.
When I am able to participate with meetings and everything, I just get that feeling that everyone has like fully bought in. [P11]
Program administrators led by example, and specialists noticed.
It’s a really ambitious initiative, and obviously, it’s had its kind of challenges as they’ve rolled it out just from a technological and coordination perspective and teaching perspective, but you know. The group that they’ve put together, especially the leadership with [Medical Director] and [Chief], these guys are phenomenal in terms of what their dedication level is to the program. You know. They just really set the tone of just people that are really hard working and are motivated to make this thing as functional and as usable as possible. [P11]
National leadership fostered an inclusive team culture.
[NTSP’s medical director], she’s really really good at making sure that everybody is participating and their voice is being heard. I think she really enables the providers to speak up when there’s something that they’re concerned about. I really like that environment. I think that she was very instrumental in building that kind of comfortable cohesive environment, and it really does start with that weekly meeting. That’s probably the best example of it. [P5]
Engagement in program development included setting up technology for newcomers and aligning schedules. The culture resembled that of a startup company.
I like it when I see the other providers taking on their own leadership niches because I get to see them at something that they’re good at and can learn something from them. [P1]
…the team is very excited about changing the system, improving the system, constantly making it, trying to make it better, and so it’s really exciting working with people that think like that and have that energy, because it gives you energy, gives me energy. [P5]
Cohesion
The shared experience and unique value proposition of a professional niche increased team cohesion.
I think our support system is much better. I think that it’s great that we have our weekly meeting, everybody gets together, we discuss. It’s definitely more a collaborative team. Everybody knows each other. We all talk to each other in our weekly meeting. I think that’s what the VA system is doing better. Also, we have very individual, very personalized feedback with our sites, and when there’s something that needs to be, there’s something that didn’t go as planned or as per protocol, we are comfortable and able to get that information back to the site and constantly improve, so I think the communication is much better. The teamwork is much better. [P5]
At times, the level of team cohesion rivaled that of co-located colleagues. For example, one specialist reported a greater frequency of talking with virtual colleagues than those local.
We are four neurologists there, you know, in the VA in [City]. One person is hospital coverage …which I was doing before, and I moved to - so I see him, but the other two neurologists I haven’t seen for long. Like one person – [Name] – his wife was texting me. I said I had seen [him] like almost like four months back, you know? So this [NTSP] is more interaction. …So at like certain times you don’t see people in your own facility. [P2]
To reach that feeling of finding one’s sense of belonging within the virtual team, common goals may have helped. Recurring reasons to join across participants were personal growth and professional service.
I think in general the group feels like the hardest challenges and the most frustrating hurdles have come from outside the group as opposed to within….You know, technical requirements, security requirements, organizational limitations, or even for example regional culture, like site culture. Like all of those things have been external to our group and so when they come up it feels much more like we’re on the same team trying to find solutions as opposed to anything from within the group causing challenges. [P1]
Participants reported positive effects of membership on their professional identities.
I think it [being a telestroke provider at VA] gave me back my identity as a stroke specialist…So, for nine months when I first joined the VA, I did not do a lot of acute stroke. So, finding out about this program was a great bonus...Because I really, I wasn’t sure, I was looking for that chance to do patient interaction and not just treating patients on the camera. So, I think I have a good balance now, so, a lot happier. [P9]
A neurologist who splits his time between VA and university service (but spends most of his time practicing at the VA) used to consider himself a “university neurologist” but after joining NTSP TeleStroke – reports that he considers himself to be a “VA neurologist.”
I’m academically appointed through [University Affiliate], so you know I think as a clinician I’m probably identify more strongly with the VA …[P10]
Tolerance Of Technical Issues
Experience with other systems increased specialists’ tolerance of technology barriers. Despite technical challenges, specialists reported high levels of satisfaction with the NTSP. Some of this tolerance was shaped by previous telemedicine practice.
But previous practice in telemedicine also served as contrast to NTSP’s problems.
There are just some times when it’s really difficult to do the right thing for the patient because the technology’s interfering with that.… Like I literally have to pull up a list of my passwords every shift and like make sure they all work because if one of them’s broken then I can’t even complete that shift. [P1]
Where were the technology challenges coming from? Rapid changes in software elicited uncertainty.
That it’s been a little bit challenging in the beginning to get the technology streamlined, especially because we’re juggling a lot of different programs. Initially, we were using one video thing, and then we moved to FaceTime, and then just for me as a non-VA provider, getting on-boarded into the kind of VA construct and getting access to CPRS and VistA [electronic health records] and all of that, that was immensely challenging when patient testing. [P11]
However, some of the technical challenges resulted in solutions that providers were interested in spreading outside their NTSP practice. “Yeah, and actually [Apple FaceTime] that’s something that we are looking to maybe fold into our [academic hospital] system as well based on my experience as a possible backup.” [P11].
Shared competence: Most of the virtual specialists had discussed their competences with telemedicine delivery. P4 had set up a different neurological telemedicine program which was deemed “frustrating” but conceded “this [NTSP approach] is the way to do it.”
Almost all of the people that are doing this [NTSP] are fairly technology savvy, you know what I mean, so like emailing and texting is pretty easy. [P8]
Satisfaction
Well, I enjoy the work, and I also enjoy the people, and I have to say that this [NTSP] program runs well, and being a part of this kind of VA program for me was really sort of a breath of fresh air. [P4]
…So I think a big point about telemedicine is that it can be incredibly frustrating. It can be more frustrating than sort of live medicine in a setting like this, but our program [NTSP] has really minimized that. [P4]