Medial temporal lobe high-frequency activity at feedback signals error and predicts subsequent memory during spatial learning

: The ability to incorporate information about feedback is critical for associative learning. The medial temporal lobe (MTL) and prefrontal cortex (PFC) are thought to be involved in processing feedback as new associations are learned. However, the relative contributions of these regions to feedback processing and subsequent memory performance in humans are poorly understood. To address this question, we tested pre-surgical epilepsy patients with depth electrodes implanted in the MTL and PFC using a spatial memory task in which subjects learned object-location associations over time. During encoding, subjects were shown objects at random locations along the circumference of an invisible circle. For each training block, the same objects were shown at the top of the circle and subjects used a mouse wheel to rotate the object to where it appeared during encoding. After subjects finished placing each object, the object was shown in the correct location for one second as feedback. We found increased high-frequency activity (HFA; 40-100 Hz), thought to reflect local excitatory activity, in the MTL and dorsolateral PFC (dlPFC) at feedback for high error trials. In the MTL, this HFA error signal predicted greater trial-by-trial decreases in error from one training block to the next indicating that these signals are involved in updating memory representations or modifying incorrect associations during learning. The opposite pattern of activity was observed during retrieval, with greater MTL and dlPFC HFA predicting lower error, replicating previous results from our group. Overall, these data suggest putative mechanisms for the learning of object-location associations. to update based is critical Our results suggest an important role for the MTL in signaling error at feedback and in representations of object-location associations. Future work should aim to identify the by which these integrated into mnemonic and to whether these mechanisms extend to other types of associations in episodic


Abstract:
The ability to incorporate information about feedback is critical for associative learning. The medial temporal lobe (MTL) and prefrontal cortex (PFC) are thought to be involved in processing feedback as new associations are learned. However, the relative contributions of these regions to feedback processing and subsequent memory performance in humans are poorly understood. To address this question, we tested pre-surgical epilepsy patients with depth electrodes implanted in the MTL and PFC using a spatial memory task in which subjects learned object-location associations over time. During encoding, subjects were shown objects at random locations along the circumference of an invisible circle. For each training block, the same objects were shown at the top of the circle and subjects used a mouse wheel to rotate the object to where it appeared during encoding. After subjects finished placing each object, the object was shown in the correct location for one second as feedback. We found increased highfrequency activity (HFA; 40-100 Hz), thought to reflect local excitatory activity, in the MTL and dorsolateral PFC (dlPFC) at feedback for high error trials. In the MTL, this HFA error signal predicted greater trial-by-trial decreases in error from one training block to the next indicating that these signals are involved in updating memory representations or modifying incorrect associations during learning. The opposite pattern of activity was observed during retrieval, with greater MTL and dlPFC HFA predicting lower error, replicating previous results from our group.
Overall, these data suggest putative mechanisms for the learning of object-location associations.
with no feedback. Performance was measured in angular error, or the distance (in degrees) 27 between where subjects placed the object and the correct location. We predicted that MTL and 28 dlPFC high-frequency activity (40-100 Hz) at feedback would predict error, and that the 29 magnitude of this response for specific object-location associations would predict subsequent 30 performance on that association. Using a variant of this task in which subjects were given only 31 one shot to learn the object-location associations (i.e. no training), we previously found a 32 negative correlation between MTL and dlPFC HFA, thought to reflect local excitatory activity, 33 and error at retrieval, indicating that activity within these regions tracks representational fidelity 34 (Stevenson et al., 2018). Here, we predicted that we would replicate these results, showing 35 increased MTL and dlPFC high-frequency activity (HFA) at retrieval for low error trials. 36 37

Results: 38
Task performance. Subjects (21 sessions from 9 patients) performed a spatial learning task as 39 we recorded intracranial electroencephalogram (EEG). During the encoding phase, 30 objects 40 were presented, one at a time, at random positions around the circumference of an invisible 41 circle ( Fig. 1). Prior to the encoding phase, subjects were told that they would be tested on the 42 location of each object. Following encoding there were three training blocks during which each 43 object was shown again, this time at the top of the circle, and subjects were instructed to use 44 the mouse wheel to rotate the object to where it appeared during encoding. Subjects were 45 asked to wait 1 second (until text that read 'Wait…' disappeared from the screen) before 46 beginning to rotate the object and to press the space bar when they were finished placing the 47 object. After each trial, the object appeared on the screen in the correct location for 1 second as 48 feedback. During the final test phase, subjects were again asked to rotate each object to the 49 correct location, though this time no feedback was given. The inter-trial interval (ITI) was 1.2 +/-50 0.2 seconds and the interstimulus interval (ISI), meaning the time between when the subjects 51 finished placing the object and when it was shown in the correct location as feedback was fixed 52 at 0.5 seconds for subjects 1-2 and variable (0.4 +/-0.2 seconds) for subjects 3-9. Performance 53 was measured in angular error, or the distance (in degrees) between where subjects placed the 54 object and the correct location. If subjects performed more than one session, new objects and 55 locations were used. No more than one session was performed on each day. training blocks, the same objects were presented at the top of the screen. After a 1 second wait 60 period, subjects rotated the object to where it appeared during encoding. After subjects finished 61 placing the object, it was shown in the correct location for 1 second as feedback. No feedback 62 was given during the final test phase. B. Example of a mixture model fit of all trials for subject 1.

63
The cut off for high error trials across blocks was derived from the cumulative distribution 64 function of the von Mises distribution, i.e. 90% of trials estimated to be remembered fell within 65 +/-38 degrees of the correct location. C-D, Mean guess rate (C), and SDMem across sessions 66 for training blocks 1-3 (T1-T3) and test. 67 68 Figure 1B shows the distribution of error across all trials for subject 9 (session 21) (see SI  1B) (Sutterer & Awh, 2016;Zhang & Luck, 2008). The uniform distribution reflects trials on which the subject guessed randomly. The von Mises distribution reflects trials on which the 73 subject remembered the location of the object with varying precision. We used the cumulative 74 distribution function of the von Mises distribution estimated for each session to determine which 75 trials to place in the high error condition. Trials that had less than a 10% chance of being 76 remembered with some degree of precision were put into the high error condition. For example, 77 in subject 1, the middle 90% of the von Mises distribution spans +/-38°, so trials with error 78 greater/less than +/-38° were designated as high error trials (Fig. 2B). The remaining trials were 79 sorted by error and split evenly into the medium and low error conditions. 80

81
We used the MemFit function of Memtoolbox in MATLAB (Suchow, Brady, Fougnie, & Alvarez, 82 2013), to obtain an estimate of two parameters describing these distributions: the guess rate (g), 83 which reflects the area under the uniform distribution, and the standard deviation of the von 84 Mises distribution (SDMem), which reflects the overall precision of remembered responses. 85 Figure 1C-D shows the mean value of these parameters for T1-3 and for the final test (see SI 86 high-resolution anatomical atlas with manual tracings of MTL subregions (Zheng et al., 2017). 94 Five subjects had electrodes localized to the MTL and 9 subjects had electrodes localized to the 95 dlPFC (see Table 1 for number of MTL and dlPFC contacts for each subject). Only data from recordings contralateral to the seizure source or outside of the seizure onset zone were used in 97 subsequent analyses. period after the object appeared on the screen but before subjects were able to start moving it. 112 The feedback window was defined as the one second period during which the object was shown 113 in the correct location. Using a cluster-based permutation approach to correct for multiple 114 comparisons across time points, we found that HFA decreased across blocks in the MTL at both 115 retrieval and feedback (SI Fig. S2A, H) and in the dlPFC at feedback (SI Fig. S2N) (ANOVA p < 116 0.05, cluster-corrected). At feedback, decreases in HFA were observed in nearly all brain regions we recorded from, including the lateral temporal, orbitofrontal cortex, and anterior 118 cingulate cortex (see SI Fig. S2 for results from all regions at retrieval and feedback), though 119 there were regions (e.g. the insula) and time periods where there was no significant difference 120 in HFA across blocks, suggesting that these changes were not driven by non-neural sources 121 such as changes in electrode and impedance over time. However, there are a variety of 122 cognitive processes that could contribute to these decreases in HFA signal, including increases 123 in familiarity, decreases in novelty, as well as learning of the object-location associations. 124

125
In order to control for these nonspecific changes in HFA across blocks, we first normalized HFA 126 within each block (T1-3 and test). Replicating previous results from our group, we found greater 127 HFA for lower error trials in the dlPFC at retrieval (ANOVA, p < 0.05, cluster-corrected) 128 (Stevenson et al., 2018). We found the opposite pattern of activity during feedback, with greater 129 HFA for high error trials in both regions . No other region we recorded from showed a 130 significant difference in HFA between high, medium, and low error trials during the retrieval 131 window (SI Fig. S3). However, multiple regions, including the anterior cingulate cortex and 132 caudal prefrontal cortex, showed increased HFA for high error trials at feedback, indicating that 133 this effect was relatively widespread (ANOVA, p < 0.05, cluster-corrected) (SI Fig. S3 Since there were fewer high error trials then there were medium or low error trials, we 137 performed control analyses to make sure observed differences between conditions were not 138 due to differing numbers of trials across conditions. To address this issue, we sorted trials within 139 each block by error and split them into thirds, putting the top 1/3 trials with the highest error from 140 each block into the high error condition, the middle 1/3 trials into the medium error condition, 141 and the bottom 1/3 trials with the least error from each block into the low error condition. We 142 observed the same pattern of effects in both regions for both retrieval and feedback when trials 143 were sorted evenly in this way. We additionally performed control analyses to ascertain that the 144 gamma effects in the MTL and dlPFC at retrieval and feedback were not associated with the 145 distance the object was moved on the screen (p > 0.05, cluster corrected). 146 147 Testing for correlations between HFA and error within blocks in the MTL and dlPFC we found 148 negative correlations between HFA and logged error in both regions at retrieval ( Figure 3A-B) 149 (Pearson, p < 0.05, permutation corrected), consistent with previous results from our group 150 (Stevenson et al., 2018). At feedback, we found the opposite pattern of activity in both regions, 151 with increasing HFA predicting greater error ( Fig. 3A-B). dlPFC (B) HFA and current error for at retrieval (blue), and at feedback (red), and partial 155 correlations between HFA at feedback and the subsequent change in error for high error trials 156 from one task block to the next (green, controlling for current error). The start and end of the 157 retrieval/feedback window are indicated by vertical dotted lines. 158 159 MTL HFA at feedback predicts accuracy on subsequent trials. We then wanted to 160 determine if the observed error signals improved performance on subsequent trials. We found a 161 significant partial correlation between MTL HFA at feedback for high error trials and the change 162 in error from one block to the next (e.g. T1 to T2) for specific object-location associations, 163 controlling for error on the trial occurring in the earlier block (current error) (partial correlation, 164 Pearson, p < 0.05, permutation corrected) (Fig. 3A-B). This was a negative correlation, 165 indicating that the bigger the HFA error signal in the MTL was, the bigger the decrease in error 166 from one block to the next. Importantly, this partial correlation controls for the amount of current 167 error, indicating that this effect is not due to ceiling/floor effects stemming from the amount of 168 current error (i.e the larger the current error is, the more room there is for decreases in error). 169 170

Discussion:
Prior work has shown that MTL and dlPFC activity at feedback signals trial outcome (correct vs. 172 incorrect). However, the contributions of these signals to performance on subsequent trials (i.e. 173 learning) are poorly understood. Here, we found that increased MTL and dlPFC HFA, thought to 174 reflect local excitatory activity, signaled increased error at feedback in an object-location 175 associative learning task. Increased MTL activity at feedback also predicted greater decreases 176 in error from one training block to the next for specific object-location associations, indicating 177 that MTL error signals are involved in updating incorrect or imprecise associations during 178 learning. termed a 'match/mismatch' signal (Duncan, Ketz, Inati, & Davachi, 2012)). These 185 'match/mismatch' signals might contribute to the observed error signals if subjects' placement of 186 an object was less accurate than they had expected. Similarly, prediction errors signal 187 discrepancies between actual and expected outcomes and are thought to drive learning by 188 updating expectations to make predictions more accurate (Mattar & Daw, 2018;Stachenfeld, 189 Botvinick, & Gershman, 2017). Given that it is generally preferable to do well on any given task, 190 reward prediction errors, or the difference between an actual and expected reward might 191 contribute to the observed error signals. In this case, the reward would be good performance on 192 the task (low error), and a reward prediction error would occur if subjects' placement of an 193 object was less accurate than they had expected. The increased activity for high error trials 194 might also reflect encoding (or re-encoding) of the correct location if the location of the object 195 was forgotten, misremembered, or imprecise. Regardless of whether subjects had a specific prediction about where the object would be when it was shown in the correct location at 197 feedback, there would likely be a measure of increased surprise and novelty for higher error 198 trials since the object would appear at an unexpected location. As such, attentional effects that 199 accompany surprise or novelty might also contribute to these signals (Corbetta & Shulman, 200 2002). However, the error signal observed in the MTL at feedback predicted subsequent 201 performance on specific object-location associations, indicating that this signal is not just 202 reflecting surprise or novelty but that it is contributing to associative learning. 203 204 At retrieval, we found negative correlations between increased MTL and dlPFC HFA and error, 205 replicating results from our previous study (Stevenson et al., 2018). These results provide 206 additional evidence that MTL and dlPFC activity does not just reflect a binary signal of retrieval 207 success versus failure, but rather that activity within these regions tracks representational 208 fidelity. This adds to a growing body of literature implicating the extended hippocampal network 209 in spatial memory precision (Koen, Borders, Petzold, & Yonelinas, 2017;Kolarik et al., 2016;210 Nilakantan, Bridge, Gagnon, VanHaerents, & Voss, 2017). 211 212 Previous studies in monkeys have found MTL cells that increase firing at feedback for both 213 correct and error trials (Brincat & Miller, 2015;Wirth et al., 2009). However, the ways in which 214 error signaling drives population activity as recorded by local field potential (LFP), intracranial 215 EEG, or BOLD fMRI in animals and humans is poorly understood. In a cross-species study that 216 examined feedback signals in both monkeys (using LFP) and humans (using fMRI), increased 217 MTL HFA (30-100 Hz) was found to predict error (incorrect > correct) in monkeys, while the 218 BOLD fMRI signal in humans at feedback showed the opposite polarity, with increased MTL 219 beta values for correct trials (correct > incorrect) (E. L. Hargreaves, A. T. Mattfeld, C. E. Stark, & 220 W. A. Suzuki, 2012). High-frequency activity is thought to be generated by increased spiking, gamma oscillations, or by a combination of these two processes (John F. Burke, Ramayya, & 222 Kahana, 2015). In each case, this would mean that the HFA error signals reflect increased local 223 excitatory activity, as noted above. Although the BOLD signal has been found to be correlated 224 with the local field potential, this signal is thought to be generated by complex neurovascular 225 coupling, making the polarity of this signal difficult to interpret (Logothetis, 2001). The results 226 from the current study support the idea that, at a population level, MTL excitatory activity in 227 humans is driven by increased error during associative learning at feedback, though other task 228 designs could result in differing effects. 229

230
We also observed decreases in MTL and dlPFC HFA across task blocks at retrieval and 231 feedback. Although these decreases in activity paralleled decreases in error across blocks, low 232 error trials within blocks were associated with increased MTL and dlPFC activity at retrieval, 233 indicating that these shifts in power across blocks were not driven solely by object-location 234 associative learning. Interestingly, multiple other studies have shown decreases in gamma 235 activity over the course of experimental sessions, linking these gradual shifts to decreases in 236 novelty and increases in familiarity as well as to decreases in attention (Park et al., 2014;237 Sederberg et al., 2006;Serruya, Sederberg, & Kahana, 2014). Decreases in HFA have also 238 been linked to repetition suppression, where activity elicited by a stimulus decreases from the 239 first to the second presentation (Rodriguez Merzagora et al., 2014). Additionally, these 240 decreases in activity could be related to increases in processing efficiency or learning of the 241 task environment and/or stimuli. Future work will be needed to determine how shifts in HFA 242 relate to changes in novelty, attention, processing efficiency, or item recognition. 243

244
A limitation of the current study is that the research was conducted with patients with epilepsy, 245 whose brains may undergo epilepsy-related changes. However, in line with recommendations 246 outlined in a review by human and nonhuman primate intracranial researchers, we excluded 247 trials that contained epileptiform discharges and only included recordings from non-epileptic 248 tissues (Parvizi & Kastner, 2018). 249

250
The ability to update incorrect or imprecise associations based on negative feedback is critical 251 for the formation of accurate representations of the environment that can be used to guide 252 behavior. Our results suggest an important role for the MTL in signaling error at feedback and in 253 updating representations of object-location associations. Future work should aim to identify the 254 mechanisms by which these signals are integrated into mnemonic representations and to 255 assess whether these mechanisms extend to other types of associations in episodic memory. 256 257

Materials and methods: 258
Participants 259 Subjects were 9 patients (5 female, 4 male, age 21-69) who had stereotactically implanted 260 intracranial depth electrodes (Integra or Ad-Tech, 5-mm inter-electrode spacing) placed at the 261 University of California, Irvine Medical Center to localize the seizure onset zone for possible 262 surgical resection. Informed consent was obtained from each subject prior to testing and the 263 research protocol was approved by the IRB of the University of California, Irvine. Electrode 264 placement was exclusively guided by clinical needs. 265 266 267 Spatial learning task 268 The task was presented on a laptop computer screen set at a comfortable distance from the 269 patient. Three hundred and sixty locations were generated along the circumference of a circle 270 centered on the screen with a spacing of 1°. Thirty images of common objects were selected 271 from a set previously used by our group (Stark, Stevenson, Wu, Rutledge, & Stark, 2015). Prior to encoding, subjects were told that they would be shown objects at different locations on the 273 computer screen and were asked to try to remember the location of each object. During 274 encoding, the 30 objects appeared one at a time at pseudorandomly assigned circle locations 275 (Figure 1; 1.2 +/-0.2 second ITI). During each of the three training blocks, subjects were shown 276 the same objects again in pseudorandom order, this time at the top of the screen. Subjects were 277 instructed to wait 1 second (until text that read 'Wait…' disappeared from the screen) before 278 using a mouse wheel to move the object to where it appeared during encoding. Subjects 279 pressed the space bar to indicate that they were finished placing the object. The object was 280 then shown in the correct location for one second as feedback. After the third training block, 281 there was a final test block in which no feedback was given. There was a short (<1 minute) 282 break after the encoding phase and between each training/test block during which task 283 instructions were read. If subjects performed more than one session, new objects and locations 284 were used. No more than one session was performed on each day. 285 286

Behavioral analysis 287
Error on the spatial learning task was measured as the number of degrees between where 288 subjects placed the object and the correct location. Histograms were used to examine the 289 distribution of error values across and within task blocks. We used mixture modeling, as 290 implemented by the MemFit function of Memtoolbox (Suchow et al., 2013), to obtain an estimate 291 of two parameters describing these distributions: the guess rate (g), which reflects the area 292 under the uniform distribution, and the standard deviation of the von Mises distribution 293 (SDMem). We first fit the model using trials from all training and test blocks. Trials that had less 294 than a 10% chance of being remembered with some degree of precision based on the von 295 Mises distribution of this model were placed in the high error condition. The remaining trials 296 were sorted by error and split evenly into the medium and low error conditions. We then fit the 297 model using trials within each block (i.e. the model was fit separately for training blocks 1 and 2) 298 to determine how the guess rate and precision (SDMem) changed across task blocks. 299 300

Electrode localization 301
The electrode localization was performed using pre-and post-implantation structural T1-302 weighted 1mm isotropic MRI scans as well as post-implantation CT scans. For each participant, 303 the post-implantation MRI and CT scans were registered to the pre-implantation scan using a 6-304 parameter rigid body transformation implemented with Advanced Normalization Tools -ANTs 305 (Avants et al., 2011). Electrodes were localized within MTL subregions using a high-resolution 306 (.55 mm) in-house anatomical template with manual tracings of hippocampal subfields and 307 parahippocampal gyrus subregions (Yassa & Stark, 2009). Hippocampal subfield segmentation 308 followed our previously published protocols (Yassa & Stark, 2009). The labeled template was 309 resampled and aligned to each subject's pre-implantation scan using ANTs Symmetric 310 Normalization, so that the labels could be used to guide localization. Each electrode location 311 was determined by examining the co-registered pre-and post-implantation MRIs and identifying 312 the ROI that corresponded to the center of the electrode artifact in the post-implantation MRI 313 and CT. Cases in which electrodes were on the border between ROI's or between gray matter 314 and white matter were noted as such. Outside the MTL, electrode localization was guided by a 315 FreeSurfer cortical parcellation of the pre-implantation MRI (Fischl et al., 2004). 316 317

Data collection and preprocessing 318
Intracranial EEG data were recorded using a Nihon Khoden recording system, analog-filtered 319 above 0.01 Hz and digitally sampled at 5000 Hz. After acquisition, data were demeaned and 320 band-pass filtered from 0.3 Hz to 350 Hz using a two pass zero phase delay Butterworth infinite 321 impulse response (IIRR) filter. Power spectra were examined to identify line noise and a Butterworth notch filter was used to remove 60 Hz noise and harmonics. All electrodes were re-323 referenced to a white matter electrode located on the same depth electrode probe. A neurologist 324 (J.L.) with subspecialty training in epilepsy visually inspected continuous recordings from each 325 session to identify all data with interictal epileptiform discharges. Data were also inspected for 326 excessive noise, including broadband electromagnetic noise from hospital equipment. To avoid 327 potentially biasing the results, the neurologist was blinded to trial information (e.g. stimulus 328 onset and behavioral performance) as well as to electrode location. Only data from recordings 329 contralateral to the seizure source or outside of the seizure onset zone were used in 330 subsequent analyses. 331

HFA Analyses 333
Intracranial recordings were broken into event-related epochs (3 seconds pre-stimulus onset 334 and 3 seconds post-stimulus offset) and convolved with complex Morlet wavelets, implemented 335 using the FieldTrip toolbox, to obtain a measure of instantaneous power (Oostenveld,Fries,336 Maris, & Schoffelen, 2011). Center frequencies ranged from 1 to 150 Hz, with a spacing of 1 Hz 337 and a variable cycle number of 4-15. Power was baseline corrected to the average pre-stimulus 338 power (0.5 to 0.2 prior to stimulus onset), resulting in a measure of the relative change in power 339 per frequency at each time point. For analyses that examined power across training blocks, 340 power was z-transformed separately within each session to account for differences in power 341 and noise across sessions. For analyses that examined power within training blocks, power was 342 z-transformed within each block. We then averaged normalized power over our a priori gamma 343 frequency range of 40-100 Hz, based on prior literature showing MTL and dlPFC gamma activity 344 in this range (J. F. Burke et al., 2014;Greenberg, Burke, Haque, Kahana, & Zaghloul, 2015;345 Sederberg et al., 2007;Stevenson et al., 2018). Next, we averaged HFA across electrodes 346 located within the MTL, including electrodes in the hippocampus as well as the entorhinal, perirhinal, and parahippocampal cortices, and within the dlPFC (Broadman area (BA) 9/10/46). 348 To examine the specificity of effects, we also ran analyses on regions outside of the MTL and 349 dlPFC including the lateral temporal cortex, the insula, the caudal prefrontal cortex (BA 6/8), 350 orbitofrontal cortex, and anterior cingulate cortex. 351

352
We used a cluster-based permutation approach implemented using the FieldTrip toolbox to test 353 for differences in HFA across conditions (high, medium, and low error) at each time point within 354 each region (Oostenveld et al., 2011). In order to test for correlations between HFA and error, 355 we averaged HFA over 100ms sliding windows and calculated the Pearson correlation between 356 HFA and angular error at intervals of 20ms. For this analysis, angular error was logged to 357 account for the non-normal distribution of error. We used permutation testing to ensure that the 358 observed correlations were not driven by outliers or other biases in the data. A null distribution 359 of r values was created by shuffling the trial labels between conditions 1000 times. We derived 360 p-values for the observed r values using the cumulative distribution function of these 361 distributions. We used partial correlations to test for associations between HFA at feedback for 362 high error trials and the subsequent change in error from one block the next, controlling for error 363 on the earlier task block (i.e. the association between HFA at feedback on a high error trial at T1 364 and the change in error for that object-location association from T1 to T2, controlling for error on 365 that trial at T1). P-values for the observed rho values were obtained via permutation testing as 366 described above.