Utilizing Inter-Subject Variability to Assess Performance and Brain Activity During Complex Task Learning

With tasks becoming more mentally focused and operators being required to conduct multiple tasks simultaneously, it is important to not only acquire direct measurements from the brain, but also account for changes in performance and brain activity as a function of intersubject variability and task demands. Such methodology is particularly important when evaluating skill acquisition and transfer during training on a complex and ecologically 18 valid task. To evaluate the aforementioned factors, we implemented a search and surveillance task (scanning an 19 assigned area and identifying targets) using a high-fidelity Unmanned Aerial System operator training simulator, 20 acquired brain activity changes via a portable functional near infrared spectroscopy (fNIRS) sensor array, and had 21 novice participants (N = 13) undergo three sessions of easy difficulty followed by two harder difficulty sessions. 22 Behavioral performance results indicated no significant change in scan or target find performance across easy 23 sessions when intersubject variability was not accounted for. However, accounting for intersubject variability 24 indicated that some individuals improved their scan performance, and they deteriorated their target find performance 25 (Attention-focused group), while others deteriorated their scan performance, and they improved their target find 26 performance (Accuracy-focused group). fNIRS results displayed that both groups exhibited a decrease in brain 27 activity across easy sessions within the left dorsolateral prefrontal cortex (LDLPFC) and right anterior medial PFC 28 (RAMPFC), while activity in left anterior medial prefrontal cortex (LAMPFC) increased in the Attention-focused 29 group and decreased in the Accuracy-focused group. In both groups, transitioning to hard sessions resulted in a 30 decrease in performance. The Attention-focused group displayed an increase in brain activity within LDLPFC, 31 RAMPFC and LAMPFC, while the Accuracy-focused group displayed an increase in brain activity within LDLPFC, 32 no change within RAMPFC and a decrease within LAMPFC. These results suggest that the Attention-focused group 33 was able to acquire and transfer the skills needed to efficiently complete the scan task, while remaining engaged in a 34 target find task. Alternatively, the Accuracy-focused group was engaged only on acquiring the skills needed to 35 efficiently complete the target find task. In conclusion, these results suggest that utilizing intersubject variability as 36 relevant information rather than noise improves assessments of skill acquisition and transfer during training on a complex


Introduction
With recent advances in autonomy capability, we expect human-autonomy systems to be safe and efficient. activity leads to an increase in metabolic demands, which results in an increase in oxyhemoglobin (HbO) and deoxy-68 hemoglobin (HbR) concentrations; (2) these hemoglobin chromophores have unique optical properties within the 69 700 to 900 nm wavelength; (3) by examining the manner in which light passes through cortical tissue, 70 concentrations of HbO and HbR can be calculated [8][9][10].

71
Over the last decade, fNIRS has been used extensively to assess workload, quantify mental capacity, and track 72 training in both laboratory and field settings [6,[10][11][12][13][14][15][16]. Majority of these studies have focused on quantifying 73 changes in brain activity within the prefrontal cortex (PFC), which is responsible for executive functions such as 74 working memory, attention, problem solving, decision making, response inhibition, planning, conflict resolution, 75 mental flexibility, and others [17]. In summary, these studies have indicated three important take-aways: (1) 76 additional task load leads to an increase in brain activity during both standard and complex tasks; (2) these brain 77 activity changes acquired by fNIRS are complementary to behavioral metrics; (3) task practice decreases the extent 78 or intensity of brain activity changes, particularly in the attentional and control areas while maintaining higher 79 behavioral or outcome performance.

80
In this paper, we evaluated skill acquisition and transfer during training on a complex ecologically valid task 81 using behavioral and fNIRS measures. We selected an Unmanned Aerial System (UAS) operators' search and 82 surveillance task as the complex task since the task requires an active role of attention and spatial working memory 83 to ensure complete scanning of an assigned area and high accuracy of identifying and tracking targets. The 84 complexity or duality of the task allowed for variation in strategy, therefore enabling modelling of intrasubject 85 variability. Based on intrasubject variability as a factor, we hypothesize that performance will vary across 86 individuals and these changes will be elicited in the prefrontal cortex and acquired by brain activation (fNIRS) 87 measures. We had each participant undergo three sessions of easy difficulty, followed by two sessions of harder 88 difficulty to evaluate skill acquisition and transfer, respectively. Based on previous investigation using a similar

Participants
Thirteen participants between the ages of 19 to 40 (22.92 5.88 years) voluntarily consented to participate in module records cameras zoom level, a logical index true or false representing when target is in FOV or is not in 120 FOV, respectively and the coordinates of the FOV polygon every microsecond. that the aircraft will flying along (1), the area where the scan and target find task are assigned (2), and what region the sensor 125 screen is capturing (3). The payload screen displays real time visual of the landscape being looked at with feedback regarding the 126 zoom level (4). This screenshot also shows how the target (5), a red civilian bus, looks like from a distance and when it is being 127 tracked at a zoom angle of 3 .

129
Hemodynamic changes from PFC were monitored using the fNIR Imager 1200 (fNIR Devices LLC, Potomac,

130
MD) (see Figure 2A). The system operates at a sampling frequency of 10 Hz and measures light intensity during 131 ambient (when no light is shone), 750 and 830 nm wavelengths. The sensor has four surface mount light emitting 132 diodes (red circles in Figure 2B), and twelve silicone photodiodes with integrated trans-impedance preamp (yellow 133 circles in Figure 2B). Ten of the twelve detectors are located 2.5 cm away from each source and enable measurement of cerebral activity from 16 different locations (white circles labelled 1 through 16 in Figure 2B). The

135
remaining two detectors are located 1 cm away from the middle two sources and allow measurement of

Experimental Protocol
Each participant underwent a tutorial session, followed by three easy sessions and two hard sessions (see Figure   145 3). The tutorial session lasted five minutes, during which the participants were shown how to utilize the joystick and 146 the computer mouse to navigate across map and payload screens, how to lock and track a target, and what constituted as proper scanning and target find behaviors, which were defined as completely scanning the assigned 148 region and tracking the target (red civilian bus as shown on the screen in Figure 1B) at or below a zoom level of 15 149 for at least 3 seconds. After instructions were given, participants were allowed to practice utilizing the screens and 150 equipment to execute the tasks for the remainder of the tutorial session. The easy and hard sessions were 151 approximately 12.5 minutes in duration, and all had unique flight paths with inimitable target placements. A 15-152 minute break was given between the easy and hard sessions, during which time the fNIRS was taken off. The

153
primary difference between easy and hard sessions was that the easy sessions occurred at a simulation time of 154 11:00AM, while the hard sessions occurred at 8:00PM and 6:00AM. To make sure there was no bias, easy, and hard 155 scenarios were randomized during their specific time periods. Each session consisted of six subareas, which each 8 159 Figure 3. Experimental protocol began with a tutorial session followed by easy and hard sessions, respectively. Easy 160 sessions consisted of three similar scenarios that occurred at 11:00 AM (simulator time) and were randomly administered. Hard 161 sessions consisted of two different scenarios that occurred at 8:00 PM or 6:00 AM and were also randomly administered. Each 162 scenario was approximately 12 minutes long and consisted of six subareas that each lasted 2 minutes. Within each of these 163 subareas' participants were required to scan the assigned area and find a target. was in FOV during a particular scan, and if scan occurred at a zoom level at or below 15. In subareas that did not 178 have targets, accuracy was set to '1'. An adaptive target find score was calculated by dividing target find score by the number of the subarea. (3) In alignment with the approach previously reported by Izzetoglu et. al.,average HbO and HbR measures each channel [25]. This was performed as the tasks here followed each other continuously to maintain ecological validity, i.e., no resting periods in between, and to wash out any effect from the previous task-hemodynamic 187 response elicited by the preceding task which could be carried over to the present task.

188
To simultaneously evaluate behavioral and hemodynamic measures, relative efficiency and relative involvement 189 measures were calculated using equations 4 and 5 [16,26]. In the equations, P represents standardized performance missing data (determined to be missing at random), linear mixed effects regression (LMER) modelling was used.

200
Models generated investigated the main and interaction effects of Group (Attention-focused vs Accuracy-focused),

204
Since those who performed well on scan tasks did not show improvements in target find tasks and vice versa,

205
interactions between Group and Adaptive Target Find Score were incorporated as a fixed effect. Equation 7 206 describes the model investigated for behavioral measures. A random slope term based on short source detector 207 separation measurements (0 + Short | ID) was added to equation 7 when evaluating mean fNIRS measures. This 208 additional random term allowed for separation of task-induced extracerebral activity from that related to cerebral activity [27,28]. When evaluating relative efficiency and relative involvement measures, equation 7 without the 210 "Group : Adaptive Target Find Score" term was used. A random slope factor accounting for extracerebral relative 211 efficiency and relative involvement were added to the model.

212
DV ~ 1 + Group + Group : Session + Group : Session : Adaptive Target Find Score + (1 | ID) 213 Significance of fixed effect terms were evaluated using likelihood ratio tests, where the full effects model was

Effect of Group, Session and Adaptive Target Find on behavioral performance measures
Interaction between Group and Session was significant for scan ( (8)

266
-0.88), with activity being dominant in the Accuracy-focused group. Lastly, in hard session 2, activity was 267 significant in channel 5 (HbR: adj.p = 0.006, d = 1.50), which was greater in the Attention-focused group.

268
Evaluation of pairwise comparisons between Sessions per Group for "Group : Session" term indicated 269 significant activity across multiple comparisons within channels 2, 5,7,9,11,12,13 and 14 (see Figure 6 and and increased in channels 11, 12 and 13; (iii) hard session 1 to hard session 2 increased in channels 12 and 14 and
Alternatively, in Accuracy-focused group significant changes in relationship between AdpTF and brain activity were

298
Cohen's is negative for HbO and positive for HbR, then this indicates that the activity increased in the second term of the comparison. For example, in Attention-focused performers, channel 13 displayed higher activity in easy session 1 than easy 3.4.

Effect of Group and Session on relative efficiency and relative involvement measures
Significant interaction effects of Group and Session were observed on relative efficiency (RE) and relative

335
With tasks becoming denser for use of mental resources in safety critical domains such as aviation and medicine,

336
there is a need to measure brain activity in conjunction with behavioral performance to evaluate skill acquisition and transfer during training programs. Furthermore, due to complexity of the tasks being performed and since the effect 338 of training varies across individuals due to intrinsic (e.g., age) or extrinsic factors (e.g., cognitive strategy or prior 339 knowledge), there is a concomitant need to evaluate skill acquisition and transfer as a function of intersubject as a confounding factor or noise to evaluate skill acquisition and transfer in novice operators during performance of 342 a complex and realistic task via behavioral and fNIRS measures.  Previous studies have repetitively indicated that brain activity during complex tasks is not localized to one PFC 368 area [10,12,15,18,40,41]. Furthermore, studies have also shown increased intersubject variability when studying 369 a complex task in comparison to a standard or simple task (i.e., Stroop, etc.) [5]. Our results indicated significant 370 brain activity changes within most channels for both HbO and HbR biomarkers (see Figure 5.A). This global effect 371 is likely due to the nature of the task requiring execution and coordination among multiple cognitive processes.

372
However, post hoc results indicated significant differences across numerous comparisons within channels 2, 7, 11, 373 12 and 14. These results indicate that even though most of the PFC was recruited to perform the task, that stronger 374 activity was observed in task-relevant areas. Specifically, fNIRS studies have shown that channel 2 is approximately 375 measuring from the left dorsolateral prefrontal cortex (LDLPFC), which is reported to be involved in spatial 376 working memory or recognizing specific features and task setting [17,24,32,42,43]. Alternatively, channels 11,

377
12, and 14 are the measures from the right anterior medial PFC (RAMPFC), which is known to be involved with 378 attentional control [17,24,32]. Activity within these regions implies that the search and surveillance task employed 379 in this study taxes attention and spatial working memory processes [44]. Furthermore, activations within these 380 channels and regions are in line with other similar fNIRS and fMRI studies evaluating activity during spatial 381 navigation tasks [12,21,37,45,46]. Lastly, channel 7 is overlayed on top of the left anterior medial PFC

382
(LAMPFC) and has been shown by fMRI studies to be involved with task switching [36,39]. Activity within this 383 region likely indicates executive control needed to engage in scan and target find tasks simultaneously. Practice is effective in decreasing brain activity intensity within attentional and control areas from an overly 387 active one to one that is nearly automatic [13,14,37,45,47]. Both Attention-focused and Accuracy-focused groups 388 displayed a decrease in brain activity within RAMPFC and LDLPFC areas. Although these results support neural 15, 16]. Behavioral results indicated that the Attention-focused group improved in scan performance and declined in target find performance, while Accuracy-focused group decreased in scan performance and increased in target find Accuracy-focused group prioritized the target find task. However, activity within LAMPFC increased in the Attention-focused group, while it decreased in the Accuracy-focused group. These results indicate that although the 396 Attention-focused group did not demonstrate improvement in target detection, that they were engaged in both tasks.

397
These engagement results are further supported by the results investigating the effect of adaptive target find score on 398 fNIRS measures from LDLPFC, where over scan measures increased with target find during easy session 1 and 399 decreased during easy session 3 (see. Supplemental Figure 3A), while brain activity increased in easy session 1 and that the Accuracy-focused group was only engaged in the target find task. Specifically, the results showed that the 402 over scan measures decreased with target find during easy session 1 and increased during easy session 3 (see.

403
Supplemental Figure 3A). This shift from negative to positive association, suggests that either the performers 404 stopped scanning after they found the target or were aimlessly wandering. The association between adaptive target 405 find score and fNIRS measures from LDLPFC went from a positive to negative indicating that the subjects went 406 from using more to fewer resources with practice, while finding targets (see. Supplemental Figure 3B). Such 407 associations were not observed in the Attention-focused group, indicating that even though they were task switching 408 they needed additional practice to improve target find performance. recruitment of neural resources varied across PFC region and Group. These results indicate the Attention-focused 418 group prioritized the scan task, while Accuracy-focused group prioritized the target find task. However, assessment 419 of relative efficiency and involvement measures provide further insight. In particular, the Attention-focused group 420 decreased in efficiency and remained involved in scan tasks, while ignoring the target find task. These results

421
validate that the Attention-focused group needed more practice on the target-find task during easy conditions before 422 being able to transfer the skills to the hard condition. Additionally, the Attention-focused group displayed a decrease 423 in relative efficiency across hard sessions, while their relative involvement remained high. This further supports the 424 different prioritization strategies used by the two groups. Unlike the Attention-focused group, the Accuracy-focused 425 group was relatively efficient in the scan task, and they were not when performing the target find task. However, 426 they were relatively involved in the target-find task and not relatively involved in the scan task. As previously 427 described the target find task is a secondary task to the primary scan task, which means that improvement in scan 428 task performance should enable improvement in the target find task. Based on this presumed connection, the 429 Accuracy-focused group could be zooming in further to accommodate for the lack of visibility in the hard condition, 430 therefore they may have utilized scan task performance as way of completing the target find task. This could be the 431 reason why the Accuracy-focused group had increased activity in LAMPFC and LDLPFC, but not in RAMPFC.

432
Lastly, the Accuracy-focused group showed no change in relative efficiency or relative involvement across hard 433 conditions. A possible reason for this could be that they quit.

441
Table 2), the factor does not remove task-evoked extracerebral and systemic activity [27,28,48]. Therefore, future 442 studies will need to incorporate signal processing techniques such as least squares adaptive filters, Kalman filter and activities [49]. The brain activity results must be interpreted with caution, as not all areas of the brain that are fNIRS Devices, LLC., manufactures the optical brain imaging instrument which was utilized in this study. K.I 472 was involved in the technological development and thus offered a minor share in the startup firm, fNIRS Devices,

473
LLC that licensed IP from Drexel University. The remaining authors declare no conflicts of interest.

476
The study received no external funding. We graciously thank Shahar Kosti and Simlat, Inc., of Miamisburg, Ohio, USA, for providing access, licensing, 486 and data extraction from the C-STAR simulator, which made this study possible. We also thank Jaime Kerr, for 487 helping in the data collection process.