A. Behavioural Results
Both accuracy and mean response latencies were examined in the critical trials common to our two goal stimulus conditions, Goal (G) and the simultaneous Novel and Goal (NG). Overall, participants performed well (94% accuracy of goal trials). The proportion of correct responses was analyzed using a 2 way ANOVA. The main effect of condition was not significant across subjects (F(1,11) = 0.43, p = .5136).
A time series analysis using a running average of reaction times was conducted in each participant to explore the basis of these non-significant results and the small effect size (<.01). Running average reaction times in the 12 control participants for conditions G (coloured in black) and NG (coloured in gray) are illustrated in Figure 2.
Solid lines in the upper plots are the means for every condition (black for standard Goal stimuli, gray for the simultaneous Novel and Goal). In the bottom plots the difference of the RTs between the G condition minus the NG condition are shown. There the average and standard deviation calculation of reaction times was run, taking as the centre, the central trial plus and minus 75 trials (condition G) or 15 trials (condition NG) across the whole of the possible accurately answered trials (this explains why the measure does not start from 0 and finish at 400) rendering 151 trials (condition G) and 31 trials (condition NG). This is called running average of Reaction time or running average RT.
Novel distractors slowed RTs in 6 participants (7, 8, 10, 14, 15 & 16), speeded up reaction times in 4 participants (4, 5, 9 & 12) while 2 participants (6 and 11) showed no differences. In Figure 2 the running average RTs for the G and NG conditions are illustrated along with the average difference between the two conditions.
Overall, the lack of significant differences in RT in the two-way ANOVA may be explained by the individual differences in pattern of the running average reaction times in the different conditions. Some individuals clearly show distraction effects while others do not.
B. fMRI results based on the immediately preceding context analysis included in the analysis for ‘distracted’ participants
Continuing with the focus of the condition of the trial immediately prior to the current trial as suggested in controls and schizophrenic participants [3], the classical fMRI analysis was extended. The contextual cases tested in this fMRI analysis were: Z.G vs. G.G, N.G vs. G.G, NG.G vs. Z.G, NG.G vs. G.G, and N.G vs. Z.G.
Common different brain area activations are in the Left Parietal Precuneus, the Right Sub lobar Insula and in the Right Temporal Lobe in the Superior Temporal Gyrus (R STG). In the last case, L STG has different brain activation except for the N.G vs. Z.G contrast (this is discussed in section 3.4.3).
Table 2 lists the differences observed in the contrast between Z.G and G.G. Both hemispheres in frontal, temporal, parietal, occipital and limbic brain areas showed differences strongly biased to the Z.G contextual condition. According to the results, there are no brain areas with the same BA in the positive and negative contrasts, and only the Left Medial Frontal Gyrus with different Brodmann Areas (BA), BA 6 biased to Z.G and the BA 9 biased to the G.G condition. The left and right frontal areas in Inferior and Middle Frontal Gyrus (IFG and MFG) are positive activated. Also, positive differences were found for R MFG, R IFG, and R IPL, and L IPs and R IPs.
Table 3 lists the differences observed in the contrast of sequences N.G and G.G. Both hemispheres in frontal, temporal, parietal and right limbic brain areas showed differences strongly biased to the N.G contextual condition. According to the results, there are no common areas for positive and negative contrast. There are strong frontal differences in R Precentral Gyrus and the R IFG and in 5 other frontal areas. Results showed that the greatest differences measured occurred towards the most frontal area of the brain, with the greatest frontal differences measuring up to 37 mm in the left MFG and up to 28 mm in the right MFG, which means that frontal activation is larger in the left hemisphere when the Novel is presented immediately before the present Goal stimulus. This left lateralisation response is consistent with the present Goal stimulus. Figure 4 has shown this contrast.
Moreover, the PreCentral Gyrus is activated differently between this N.G and G.G contrast, with a clearly right lateralized bias. Bearing in mind that this area was not found in the results for the N and G contrast, thus the Novel before a Goal makes more contribution to different motor area activations. Therefore, this result suggests that attention to the task by the participants produces different motor control in N vs. G contrast and in N.G and G.G contrast. This is addressed in the discussion. Overall these differences in the Prefrontal Cortex by the trial before the G condition in analysis support hypothesis H2.
Table 4 lists the observed differences for the contrast of sequences N.G and NG.G, showing frontal differences in 10 regions. Both hemispheres in frontal, temporal, parietal and limbic brain areas showed differences strongly biased to the N.G contextual condition. According to the results, the Right IFG with BA 13, Right SFG with BA 6 and the Right Cingulate Gyrus with BA 24 are activated with both positive and negative contrast (see the highlighted results in Table 7). In addition, the left Precentral Gyrus is activated differently in this contrast, which informs different motor response than the other contrasts. Again, there are frontal differences in left and right MFG (up to 46 mm and 44 mm respectively). Results showed that the greatest differences measured occurred towards the most frontal area of the brain, with the greatest frontal differences measuring up to 50 mm in the left SFG and up to 56 mm in the right SFG, having the more frontal activation in the right hemisphere. Overall these differences in the Prefrontal Cortex by the trial before the G condition in analysis are supporting hypothesis H2 and suggest the more frontal activation for the switching from simultaneous Novel and Goal to the Goal which is also concordant with Koechlin’s model (2003) of the frontal episodic attention control and with Corbetta’s model (2008) lateralising to the right hemisphere. Figure 4 on the bottom has shown this contrast.
Table 4 also shows the frontal differences in the left and right Anterior Cingulate Cortex (ACC, up to 34 mm and 30 mm respectively), this is consistent with the view that ACC is involved in conflict monitoring (reviewed by van Veen & Carter, 2002) which is the previous context in our analysis.
In Table 5, the contrast of sequences N.G and Z.G is shown. Both hemispheres in occipital and limbic brain areas showed differences strongly biased to the Z.G contextual condition and both hemispheres showed activation for frontal, temporal and parietal in positive and negative contrasts. According to the results, the Left MedialFrontal Gyrus, Left SFG, Right MedialFrontal Gyrus, Right MFG, Right Precentral Gyrus, Right SFG, Left MiddleTemporal Gyrus and Right STG with different BAs are activated with both positive and negative contrast (see the highlighted results in Table 5). Also, Table 5 showed differences in several frontal regions biased to the N.G condition. Again, there are frontal differences in the left and right MFG (up to 46 mm and 44 mm respectively). Results showed that the greatest differences measured occurred towards the most frontal area of the brain, with the greatest frontal differences measuring up to 50 mm in the left SFG and up to 56 mm in the right SFG, having more frontal activation in the right hemisphere. Overall these differences in the Prefrontal Cortex by the trial before the G condition in analysis support hypothesis H2 and suggest more frontal activation for the switching from simultaneous Novel and Goal to the Goal which is also concordant with Koechlin’s model of the frontal episodic attention control and with Corbetta’s model lateralising to the right hemisphere [21].
In Table 4 there are also differences in the left and right Anterior Cingulate Cortex (ACC). This is consistent with the view of ACC in conflict monitoring (van Veen & Carter, 2002) which is the previous context in our analysis.
Table 6 shows the contrast of sequences N.G and Z.G. Both hemispheres in parietal brain areas showed differences strongly biased to the Z.G contextual condition and both hemispheres showed activation for frontal, temporal, occipital and limbic in positive and negative contrasts. According to the results, the Right SuperiorTemporal Gyrus with the BA 22 with both positive and negative contrast (see the highlighted results in Table 6). Further, Table 6 showed frontal differences in two frontal regions biased to the N.G condition. In these contrasts, there are frontal differences in right MFG biased on N.G (up to 37 mm). The other great frontal difference is up to 32 mm in the right IFG. Therefore, the more frontal activation occurs in the left hemisphere. Overall, these differences in the Prefrontal Cortex by the trial before the G condition in analysis support hypothesis H2 and suggest more frontal activation for the switching from Novel to the Goal which is also concordant with Koechlin’s model of the frontal context attention control [5] and with Corbetta’s model lateralising to the right hemisphere [21].
Table 6 also shows the differences in the left and right Anterior Cingulate Cortex (ACC). This is consistent with the view of ACC in conflict monitoring [29], which is the previous context in our analysis.
II. Discussion for Contrasts, Context to Extend Multimodal Task
The first results discussed here focus on the 6 ‘distracted’ participant’s analysis which showed more significant brain activations than found for the whole group of 11 participants.
The analysis of these fMRI data (a) explored the effect of prior context across participants supporting H2 but only for ‘distracted’ participants; (b) explored novel response generators and simultaneous novel and target response generators relative to the standard goal condition supporting H1 but only for ‘distracted’ participants; (c) resulted in a larger recruiting neural response at the prefrontal cortex having less areas for standard goal stimulus and standard previous motor response and (d) attempted to find a possible explanation for the observed smaller than expected Novel sound ERP amplitudes. Last two analyses allowed having a grasp for modeling of auditory and motor function of human brain (H3).
A. RT results suggest that the novelty effect may vary between causing alerting and orienting
The RTs observed in the orienting subgroup were slower (20 to 70 ms) in the simultaneous novel and target (NG) condition suggesting that the focus of attention can be shifted with the introduction of a novel stimulus alongside the target in the mental representation of the auditory scene. In the literature we find this range of reaction times in orienting to alerting stimuli by Fan and colleagues [30]. According to Fan and colleagues, behavioural reaction time differences in alerting would be around 60 ms, orienting around 31 ms and conflict monitoring around 102 ms [30]. Brain areas of specific interest in the number parity decision task.
In the case of the parietal lobes: in the Z vs. G contrast the Right Precuneus were similarly activated only in this contrast; in the NG vs. G contrast the L/R Angular Gyrus, L/R Inferior Parietal Lobule and Left Superior Parietal Lobule (SPL) showed different activations only in this contrast for F-value difference; and in the N vs. G contrast the Left Precuneus showed similar activations only in this contrast while in the motor cortex the Right Paracentral Lobule showed different activations only in this contrast. Therefore, in the NG vs. G contrast IPL and SPL showed different activations. Activation in the Precuneus (p ≤ .0005 uncorrected) is of interest because Precuneus is associated with reaching activity [33]-[34]. Although in the present experiment the hand is not reaching different places, the selected finger (index or middle) is reaching the button for the task, the Goal and Novel stimulus showed an activation similar to the tendency to reach the novel, with different brain activations suppressing the button press in N vs. G more in the right Precuneus and allowing the button press in NG vs. G and Z vs. G in left and right Precuneus. Taking altogether the results for the contrast NG vs. G there is consistent with recent subdural electrodes in humans in the IPS, SPL and Precuneus for reaching a cup from a resting position [35].
On the temporal lobes: in the Z vs. G contrast the Left Sub Gyral area showed similar activations only in this contrast while in the different contrasts the L/R Transverse Temporal Gyrus (TTG) showed different activations. This is consistent with result of the 750 Hz tone which activated more voxels in the medial area of the TTG whereas the 2000 Hz tone activated more voxels in the lateral TTG [36]. Moreover, the Right Superior Temporal Gyrus (STG) has different activations in the different contrasts, which has been reported to be activated more by speech and frequency modulated tones [37]; in the NG vs. G contrast the L/R Angular Gyrus, Left Fusiform Gyrus, L/R Sub Gyral Hippocampus and Right Middle Temporal Gyrus showed different activations only in this contrast. Hippocampus and the different prefrontal areas activated during the task according to the presence of NG appeared by the presence of the novel when there is not an explicit sequence and having several conditions, in spite of Savalia and colleagues findings [9].
In the case of the occipital lobes: in the Z vs. G contrast the Right Fusiform Gyrus showed different activations only in this contrast; in the NG vs. G contrast the Right Cuneus/Precuneus Right Lingual Gyrus and Right Superior Occipital Gyrus showed different activations only in this contrast; and in the N vs. G contrast the Left Cuneus/Precuneus showed similar activations only in this contrast. FusiformGyrus activation reduces with repeated presentations, also when the performance of the participant is better [38]. In the present results, the L FusiformGyrus is more activated in the Novel than the Z and NG conditions, having clear differences at Goal as an object identification. However, there is no clear difference in the contrast of different conditions N vs G and N.G vs G.G. This supports the view that the orienting response is sensitive to the degree of familiarity with the experiment [39].
B. Prefrontal cortex and motor responses in the preceding trial (H2).
Results showed that the Precentral Gyrus (PrG) motor area was activated differently in Z vs. G, N vs. G and NG vs. G contrasts. Activations were more ventral with relatively greater activations for the N condition (BA 43), and with relatively greater activations in different BAs in the NG vs. G contrast, in the left BA 6 for the NG condition and right BAs 4, 6 and 44 with relatively greater activations for the G condition. Moreover, taking into account the contextual contrasts, activations for Z.G vs. G.G contrast produced larger activation in the Right PrG (BAs 4 and 6) and for the N.G vs. G.G contrast had relatively greater activations for the N.G condition on the Left PrG (BA 4) and Right PrG (BA 4, 44 and 6). Therefore, overall all these results different prefrontal control is seen at PrG
Although motor response is usually activated in the contralateral side, in this experiment the right hand was used in the parity decision task whilst some ipsilateral responses in the Left PrG were activated for N.G condition over G.G condition. Considering the change of the fundamental frequencies between N and G conditions, this left ipsilateral result to the right hand of response is consistent with frequency changes greater than 30 Hz observed for harmonic tones [40]. Thus, the Novel before a Goal makes more contribution to different motor area activations and similar activations than the NG conditions. Therefore, the ‘distracted’ participants showed a stronger attention to the task than to the motor control in N vs. G contrast and the motor control switch between N.G and G.G conditions, which is similar to the conflict motor control switch between NG and G conditions. Therefore, the motor response may be used in explaining the prefrontal control in the light of H2. This part of the discussion is expanded in the next part of the discussion which studies context from the point of view of the previous trial.
C. Prefrontal cortex and context given by the immediately previous trial (H2).
Tables 3 and 4 show that there are more differences in NG.G vs. N.G than in G.G vs. N.G, consisting of more frontal areas and towards to the front as well for NG.G vs. N.G, which is consistent with the different frontal activations in the contextual approach of the hypothesis H2.
More insights derived from the results driven by hypothesis H2 are analysed in Table 7. This shows the comparison of the five contrasts analysed (first column). From Z.G vs. G.G to N.G vs. Z.G contrasts, it looks like the effect of a previous Novel stimulus is to increase the activation of the prefrontal areas. When both contrasts are compared to the N.G vs. G.G contrast, this increased activation of additional prefrontal areas is corroborated, and also the change of motor response results analysed in the previous section in the activation of additional prefrontal areas. In Table 7, when the first and third row are compared with the fourth and fifth row, respectively, a similar increase of the number of areas in the prefrontal region is shown. Result suggested, in Table 7, when instead of G is NG part of the increased are because of the recruiting of the brain areas closer to the ACC.
ACC activation was shown in both hemispheres (see Tables 7 and 8) related to NG.G (versus N.G and Z.G) and in the left hemisphere (see Table 3) related to N.G (versus G.G). First, this ACC activation is consistent with the view that the ACC facilitates control of attention [29]. These results showed consistency with conflict monitoring being more frontal and deeper for NG.G vs. N.G contrast, see Left ACC at (-10, 34, -10) mm and the Right ACC at (3, 30, 0) mm in Table 7). Alongside the comparison in the Table 7, these results in frontal areas are not only consistent with the prefrontal control proposed by Koechlin and colleagues [4], but the R SMG is also consistent with the model of control of attention proposed by Corbetta and colleagues [7].
D. fMRI for ‘distracted’ participants showed left and right brain areas for contextual conditions in the attention model (H1 & H2).
First, the results of the Z.G vs. G.G contrast showed different right parietal activation and no different occipital areas as the signature of this contrast. The results are summarized in the graphic in Figure 4 and they have shown consistency with the visual stimulus-driven attention network model of Corbetta and Shulman [7] as shown for the left hemisphere in the dotted rectangle in yellow. Although, the positive contrast results are not exactly consistent with the reorienting of attention of Corbetta and colleagues [21], the activations in Brodmann Areas 7, 19 and 39 may be related to activity in the R IPs. However, the FEF is not clearly activated. In addition, the negative contrast only showed significant activation of the left Medial Frontal Gyrus without a clear different activation of the control of attention for the G.G condition. Of course, this can be explained because the current trial (G) has mostly the same properties of the frequently previous trial type (G). These interpretations suggest that the Z.G is evoking an interaction of the stimulus and goal driven network differently to the pattern orienting of attention, while the IPs is suggested to be related to BAs 7, 19 and 39 (see dotted rectangle in green).
Second, when the N.G and G.G contextual conditions are more involved in a different frontal control of attention: The results of the N.G vs. G.G contrast showed different left and right parietal activation and no differences in occipital areas as the signature of this contrast. The results are summarized in the graphic in Figure 5. The results support right and left (see dotted rectangle in yellow) hemispheres in the the stimulus driven attention network of Corbetta and Shulman [7] suggesting the control of attention in the N.G sequence. Although, the positive contrast results are not exactly consistent with the reorienting of attention of Corbetta and colleagues [21], but the Brodmann Areas 7, 40 and 39 may be enclosing the activity in the R IPs. Further, the negative contrast only did not show significant activation of the cortex; again, this can be explained because the current trial (G) has mostly the same properties of the previous trial (G). These interpretations suggest that the N.G is evoking an interaction of the stimulus and goal driven network similar to the pattern orienting of attention (see dotted rectangle in green).
E. fMRI and ERP comparison and the Anterior Cingulate Cortex.
Comparing fMRI and ERP results in the ‘distracted’ subgroup: (a) The Anterior Cingulate Cortex (ACC) is not activated differently between Z and G conditions (Table 2) and the ERP deflection around 200 ms, biased for Z condition negatively to the left frontal electrode F7 and positively to the right frontal electrode F8 in Figure 1 of supplemental material; (b) Right ACC is activated differently between NG and G conditions (Table 4 of supplemental material) being more frontal for NG condition in the right ACC (BA 32) and more posterior for the G condition (BA 32) and the negative ERP deflection around 200 ms in the right electrode F8 (in Figure 1 of supplemental material) and stronger Left ACC is activated differently between NG and G conditions (Table 4 of supplemental material) being with relatively greater for the NG condition in the left ACC (BA 32) and the negative ERP deflection around 200 ms is stronger to the left frontal electrode F7 (in Figure 1 of supplemental material); (c) difference between N and G conditions (Table 3 of supplemental material) and no clear difference around the ERP at 200 ms (F7 and F8 in Figure 1 of supplemental material). These results suggest that ACC is linked to N200 for NG condition in both hemispheres. On the other hand, in the N vs. G contrast positive and negative activation differences in ACC were observed and no clear ERP different deflections around 200 ms, namely MisMatch Negativity. This analysis is consistent with the view of N200 and ACC in conflict monitoring studies [29]. However, but, because of MMN, it is not clear about the Novel effect.
Moreover, ACC activation was shown to be different across the other contextual contrasts (Z.G vs. G.G, N.G vs. G.G, N.G vs. NG.G, NG.G vs. Z.G and N.G vs. Z.G) and the relatively greater activation was shown not only for novel but also for Zero condition. Therefore, ACC relative activations were sensible to contextual changes depending on Goal (G), Non-Goal (Z and N) and Novel (N and NG) signal.
In the ‘distracted’ participants, the contrast between NG.G and N.G was evaluated for the ACC. Results shown relatively greater activation for the N.G condition in the Left (BA 32) and Right (BA 24) ACC. This suggests that ACC produces different activations depending of the previous context for stimulus-driven network and the conflict monitoring effect. When the contrast between NG.G and N.G conditions in ‘all the participants’ was evaluated, there were no significant differences in ACC activation and this suggests that ACC in the alerting state does not produce different activations for the different Novel trials presented before the current Goal trial. These differences between the ‘distracted’ and the ‘all participants’ would explain the difference of the analysis of the ERP at N200 in Potter’s study [41] and ACC in fMRI in the present analysis of the ‘distracted’ subgroup.
Another possible comparison would be a further eye field activation in fMRI and beta waves in EEG such as was found for higher arousal levels [42]. The present analysis may accommodate the role of the FEF in attention when the Corbetta’s model of attention is considered. Therefore, a further limitation in the present analysis is that this was the third task in the participants and possibly the results for FEF in the ‘distracted’ participants added to the inhibition of return for Z vs G contrast were related with the arousal level to keep the answer to the task in the auditory attention task.
In practical use to add in this discussion, this experimental discussion may have a theoretical extension to be used by BCI systems that involve the management of neural network and learning systems architectures. This was addressed in the following conclusion.
Limitations of twelve participants were compensated by a FDR analysis and bearing in mind current theory of attention and a similar auditory paradigm which explored context with EEG in schizophrenic participants (Mugruza-Vassallo and Potter, 2019).
III. Conclusion: Improve of modeling novel response due to previous motor response
Given results and discussion, the sequence of stimulus studied has shown different activation of the hippocampus areas which have been in favor of the theory or cortical and subcortical loop for sequence proposed by Savalia and colleagues [9]. Moreover, the present results have reported when a sequence is interrupted by a novel (simultaneously) the subcortical loop with the hippocampus is also activated. This has extended Mugruza-Vassallo and Potter studies of temporal stimulus sequence with EEG [3] to fMRI brain regions and following their analysis and extension of management of novel stimulus modulated by the previous motor answer a model was proposed in Figure 6 solving part of the puzzle proposed by Livnet and Zador [15]. These consistencies make it of interest to explore another experiment to study the EEG results in more detail and combine with the fMRI analysis to seek for the explanation of these partial consistencies.
Bearing in mind eye movement research in response to auditory experiment has shown results in pupil dilation response [45], the present findings on motor modulation of attentional processing would we extended by a broader motor response. Moreover, the model would modify the Information Dynamics of Thinking (IDyOT) model for language and music of Forth and colleagues [44] may bear in mind previous motor response and unexpected external stimulus. Forth and colleagues proposed a mechanism for predicting when a perceptual event will happen, given an existing sequence of past events, which may be musical or linguistic [44].
Evolutionary multitasking computation [16] maybe best based on multi-objective optimization of cortical prefrontal cortex for different incoming stimulus employing stimulus properties for objective functions (f) for vectors of decision variables (y) in the search space (Y) following equation 3, considering 4 conditions.
maximize(y∈Y) f(y) = | f1(y) ; f2(y) ; f3(y); f4(y) | (3)
Then for K=4 different tasks (T1, T2, T3, T4) the MOP in terms of the populations would follow equation 3, but bearing in mind the different responses due to previous motor command. In this way fk(y) will depend on the neural processing of previous motor response and the current motor response, as seen in (4).
fk(y) = gk,1(y(s)) gk,2(y(m(nT),y(m(nT-T)))) (4)
Therefore, an additional input would be needed to maximize decision variables going for at least m = {0, 1}, 0 for no motor response and 1 for motor response in (5).
4 4
max(y∈Y) Σ ∫ z fk(z) . [ Σ wjk,m(k-1) .pj,m(k-1)(z)].dz (5)
{wjk.pj(z)} k=1 j=1
Main limitation for this proposal is to ignore possibly conflict when one tends to think about a bad previous response. In the present experiment errors were less than 10% in most of the participants. This would open to study motor response with error response in decision-making responses and improve current learning systems in BCI.
This motor response recruiting prefrontal areas would support the idea the learning modelling of the task has not a linear function influenced by the learning parameter, the greater the maze size for goal-task the more steps to get an optimal pathway [43]. Moreover, the model proposed may help in the future to find compensatory effects in Parkinson’s disease by recruitment of more brain area in the prefrontal cortex and extend not only the present work but also work of Martin and colleagues at planning and executing motor employing different hands might be studied simplifying their experiment with an additional condition of motor planning [46].
Another area of further test maybe on multitask switching on dyslexia, considering our results mainly on right pulvinar which is close to LGN, our experimental results suggest an asymmetry for brain processing. Bearing this result on our auditory number parity decision task, language multitask switch maybe explored as well, as LGN asymmetry was reported by proton density with MRI recently by Giraldo-Chilca and Schneider [48].
Finally, bearing in mind discussion of multitask experiment [3] discussed in use of person identification with reliable decoders [2] and re-identification using different visual views [1] in systems with different interfaces. These interfaces may involve not only EEG but also precise electrodes position inferred or combined with fMRI or fNIRS as occipital images, as the present work suggest.