The neuromorphic Mosaic: re-congurable in-memory small-world graphs

Thanks to their non-volatile and multi-bit properties, memristors have been extensively used as synaptic weight elements in neuromorphic architectures. However, their use to deﬁne and re-program the network connectivity has been overlooked. Here, we propose, implement and experimentally demonstrate Mosaic, a neuromorphic architecture based on a systolic array of memristor crossbars. For the ﬁrst time, we use distributed non-volatile memristors not only for computation, but also for routing (i.e., to deﬁne the network connectivity). Mosaic is particularly well-suited for the implementation of re-conﬁgurable small-world graphical models, with dense local and sparse global connectivity - found extensively in the brain. We mathematically show that, as the networks scale up, the Mosaic requires less memory than in conventional memristor approaches. We map a spiking recurrent neural network on the Mosaic to solve an Electrocardiogram (ECG) anomaly detection task. While the performance is either equivalent or better than software models, the advantage of the Mosaic was clearly seen in respective one and two orders of magnitude reduction in energy requirements, compared to a micro-controller and address-event representation-based processor. Mosaic promises to open up a new approach to designing neuromorphic hardware based on graph-theoretic principles with less memory and energy. triangular times and of at to V for SET V gate . V top peak . in of [ State (LRS) of the RESET gate . a . of conductive ﬁlament, out wider [ 80 1000 ] k Ω operation by the V to the gate voltage at 5 a a pulse width of 1 and rise/fall type of stereotypical spiking of a Spiking the


Introduction
Graphs are omnipresent data structures which capture interactions (i.e., edges) between multiple units (i.e., nodes). They are 2 the backbone of many computational systems that represent relational information between their interacting entities 1 . Neural 3 networks are an example of a graph. Graphs can be used to study and represent both biological and artificial neural networks, 4 where neurons correspond to the nodes of a graph and the connections between them (i.e., weights or synapses) correspond to 5 edges. Biological nervous systems, shaped over millions of years of evolution, have developed many computational principles 6 that can be captured using graphical networks. Therefore, building computing architectures based on the same organizational 7 principles is a promising path towards realizing powerful artificially intelligent systems. 8 One such important organizing principle is "small worldness" which is found extensively in empirical studies of structural 9 and functional biological neural networks 2, 3 (Fig. 1a). In such a structure, short paths connecting neighboring nodes (neurons) 10 are more common than long-range connections, which are sparse (Fig. 1b). The mix of dense local and sparse distal connectivity 11 gives rise to efficient global coordination and information flow based on local interactions 4 . A connectivity matrix of an 12 example small-world graph is plotted in Fig. 1c. It is characterized by the heavy connectivity along the matrix diagonal, with 13 increasingly fewer connections between the further off-diagonal neuron pairs. 14 Crossbars of conductive memory elements have often been proposed as a means of realizing such models on hardware 15 (Fig. 1d) [5][6][7][8][9][10][11] . In these structures, a memory element connects a series of vertically running metal lines (i.e., columns) with 16 orthogonal ones (rows). The conductance state of each memory corresponds to the synaptic weight parameter of a neuron, 17 which is located at the end of each row. Such architectures perform matrix multiplication, the core operation of a neural 18 network, in-memory and in an analog fashion. Relative to a von-Neumann architecture, this dramatically reduces the volume of 19 data movement which in turn largely reduces the energy required to run neural network models [12][13][14][15][16][17] . 20 Resistive Random Access Memory (RRAM) devices, otherwise referred to as memristors, have emerged as a promising 21 memory element for such in-memory crossbar architectures [18][19][20][21][22][23] . They can be programmed with multiple discrete conductance 22 levels 24 corresponding to different synaptic weight values in the connectivity matrix of Fig. 1c. Moreover, RRAMs retain 23 information in a non-volatile fashion, which eliminates the static power consumption related to the storage of neural network 24 weights 25 . In particular, biologically inspired Spiking Neural Networks (SNNs) are well matched to RRAMs since the devices 25 in the crossbar are read asynchronously and sparsely -thus dynamic power is also reduced.   However, scaling this to large SNNs requires a very large crossbar of memristors. Problems such as current sneak-paths, 29 parasitic resistance and capacitance of the metal lines, as well as excessively large read currents limit their maximum size 30 in practice 26, 27 . Moreover, a single large crossbar would result in a wasteful utilization of the off-diagonal devices in the 31 implementation of bio-inspired graphs with small-world properties ( Fig. 1(c)).

32
To implement artificial spiking small-world graphs more efficiently, we propose and experimentally demonstrate a new 33 re-configurable neuromorphic computing architecture called the "Mosaic" (Fig. 1(e)). The Mosaic is a two-dimensional systolic 34 matrix of distributed "tiles", each based on a small crossbar of RRAM, that can serve either as analog spiking or spike routing 35 elements. Effectively, the Mosaic dices up one large crossbar into numerous smaller tiles with different functions (Fig. 1(e)).

36
Importantly, the Mosaic uses RRAM not only to store synaptic weights and carry out neural processing, but also to define the 37 routing patterns linking up neighboring tiles.

38
The Mosaic lends itself to the implementation of small-world networks more efficiently, resulting in a better utilization of the  In this Article, we first present the Mosaic architecture and mathematically quantify its memory footprint savings while 43 implementing small-world neural networks compared to a single large memristor crossbar. We then report electrical circuit 44 measurements from tiles that we designed and fabricated in 130 nm CMOS technology co-integrated with Hafnium dioxide-45 based RRAM devices. Calibrated on these measurements, we apply a simulation of the Mosaic to run a RSNN applied to 46 the detection of arrhythmic heart beats from Electrocardiography (ECG) recordings. We compare our approach to equivalent 47 implementations using a microprocessor and an AER-based neuromorphic processor. Per heartbeat we find that Mosaic achieves 48 reductions in the total signal routing energy of two and one order of magnitudes respectively.

50
The Mosaic architecture is illustrated in Fig. 2a as an array of tiles which are distributed in a two-dimensional systolic fashion. 51 Each of the tiles consist of a small memrsitor crossbar which can receive and transmit spikes to and from their neighboring tiles 52 to the North (N), South (S), East (E) and West (W) directions ( Supplementary Fig. S1). The green squares represent "neuron 53 tiles" and correspond to small crossbars (Fig. 1e) that store the synaptic weights of several Leaky Integrate and Fire (LIF) 54 neurons. These neurons are implemented using analog circuits and are located at the termination of each row, emitting voltage 55 spikes at their outputs 31 . These spikes are communicated between neuron tiles through a mesh of blue squares which represent 56 "routing tiles". Routing tiles encompass small crossbars that determine the connectivity patterns between neuron tiles. The state 57 of each device in the crossbar determines the output direction (i.e., N, S, E,W ) towards which its input spike propagates, i.e. 58 steering it towards its intended target neuron elsewhere in the Mosaic. Together, the two tiles give rise to a continuous mosaic 59 of neuromorphic computation and memory for realizing spiking small-world neural networks.

60
An example small-world neural network topology, obtained by randomly programming memristors in a computer model of 61 the Mosaic (see Methods) is shown in Fig. 2b. The resulting graph exhibits an intriguing set of connection patterns that reflect 62 those found in many of the small-world graphical motifs observed in animal nervous systems. For example, central 'hub-like' 63 neurons with connections to numerous nodes, reciprocal connections between pairs of nodes reminiscent of winner-take-all 64 mechanisms, and a number of heavily connected local neural clusters 3 . If desired, these graph properties could be adapted 65 on-the-fly by the re-programming the RRAM states in the two tile types ( Supplementary Fig. S2). For example, a set of 66 desired small-world graph properties can be achieved by randomly programming the RRAM devices into their High-Conductive 67 State (HCS) with a certain probability ( Supplementary Fig. S3). Random programming can for example be achieved elegantly 68 Each neuron tile in the Mosaic is composed of multiple "neuron columns"; a circuit that models a LIF neuron and its synapses.

79
A neuron column circuit is shown in Fig. 3a. It has N parallel one-transistor-one-resistor (1T1R) RRAM structures at its input.

80
The synaptic weights of each neuron are stored in the conductance level of the RRAM devices in one column.

4/13
The functionality of the neuron column is summarized in the three insets of Fig. 3a. Three input pulses of V in < 0 >, < 82 1 >, < N > are applied in sequence to the gate of the three 1T1R structures. This results in three current pulses, i bu f f (green), 83 proportional to the device conductance state. The currents are then injected to a circuit that models biological synaptic dynamics 84 (see Supplementary S7a). This in turn injects an exponentially decaying current into a circuit modelling a biological neuron 32 . 85 The injected current integrates as a voltage, V mem , on the neuron's membrane capacitor ( Supplementary Fig S7b). After the 86 neuron circuit has integrated three input spikes, V mem exceeds its firing threshold (V th ) and the circuit emits an output voltage 87 spike. 88 We fabricated the neuron column of Fig. 3a in a 130 nm CMOS technology integrated with RRAM devices 33 . In the 89 fabricated circuit, the memristor corresponding to G 0 was programmed using a sweep of SET currents -resulting in a range of 90 conductance values (Fig. 3b). After programming each device, we applied an input pulse to V in < 0 > and measured the signal 91 V mem which is plotted in Fig. 3c. This experimental result illustrates that the increase in RRAM conductance increases the peak 92 voltage value resulting from a single input pulse, and thus serves well as a programmable synaptic weight element. A layout of 93 this column circuit can be found in Supplementary Fig. S4.

94
To realize a network using such circuits, these neuron columns are agglomerated into a 'tile'. This is done through stacking 95 consecutive columns side-by-side and connecting their gates row-wise to common input lines (i.e., a crossbar architecture). A 96 simple neuron tile, composed of only two neuron columns receiving two inputs, is shown in Fig. 4a. The top two rows of the 97 crossbar represent the neurons' synaptic weights corresponding to external inputs, while the bottom two represent those of the 98 recurrent connections between neurons within the tile. Following a systolic organization 34 , each input or output spike can enter 99 from, and exit towards, the neighboring N, S, E,W tiles ( Supplementary Fig. S1). 100 We mapped a simple network topology onto a fabricated neuron tile circuit depicted in Fig. 4a. Two devices highlighted 101 in bold black were programmed to be in their HCS while the gray shaded ones were programmed in their Low-Conductive 102 State (LCS). We then applied a train of input voltage spikes to V in < 0 >. The experimental measurements are plotted in Fig. 4b 103 whereby the membrane potential of neuron 0 is observed to periodically increase upon the arrival of each pulse. After the 6 th 104 input pulse, V mem exceeds the threshold V th , and the circuit generates an output spike. Because of the recurrent connection 105 between the two neurons defined in the neuron tile, the membrane of neuron 1 integrates an excitatory post-synaptic potential 106 at the same instant (shown in orange). Neuron 0 then enters a refractory period, during which it does not integrate incoming 107 spikes.

108
Routing tile circuits: connecting small-worlds 109 A routing tile circuit is shown in Fig. 4c. It acts as a flexible means of configuring how spikes emitted from neuron tiles 110 propagate locally between small-worlds. The functional principles of the routing tile circuits are similar to the neuron tiles. The 111 only difference is the replacement of the biological synapse and neuron circuit models (shown in blue in Fig. 3a) with a simple 112 current comparator circuit. On the arrival of a spike on the column, it compares the device read current (i bu f f in Fig. 3a) to a 113 reference. If it is greater than this reference, it generates an output spike. Otherwise the output remains at zero. Therefore, the 114 state of the device serves to either pass or block input spikes: in Fig. 4c, each device determines whether input spikes arriving 115 from different input ports (N, S,W, E) are propagated, or not, to each output port.

116
Using a fabricated routing tile circuit, we demonstrate its functionality experimentally. Two devices (colored in green and 117 red in Fig. 4c) were programmed in respective HCS and LCS. The other devices were left in the pristine state. This has the 118 effect of allowing incoming pulses from N to propagate out to E, while blocking pulses coming from S direction. A pair of 119 pulses were applied to N and S input ports of the fabricated circuit, plotted respectively in solid and dashed blue lines in Fig. 4d. 120 While the E output port remains at zero due to the incoming pulses from the S input port, it switches to a high voltage as a 121 result of incoming pulses from the N input port. This output pulse propagates on-wards to the next tile. Note that in Fig. 4d the 122 output spike does not appear as rectangular due to the large capacitive load of the probe station (see Methods). To allow for 123 greater configurability, more channels per direction can be used in the routing tiles (see Supplementary Fig. S5).

124
Application to ECG anomaly detection 125 RSNNs 35-37 are networks of recurrently connected spiking neurons, whose internal dynamics are a function of the history of 126 their input. They have been demonstrated to be able to process temporally changing sensory information as a result of their 127 internal dynamics 38-40 .

128
Here, we apply a small-world RSNN implemented on the Mosaic to the detection of arrhythmic heartbeats from ECG 129 signals 41 (see Methods). First, we encode the continuous ECG time-series into trains of spikes using a delta-modulation 130 technique, which describes the relative changes in signal magnitude 42, 43 . These spikes are then fed as input into the Mosaic 131 small-world RSNN. As outputs, we designated two sub-populations of neurons within two pairs of the Mosaic's neuron tiles. 132 Elevated spiking activity in either sub-population denotes a normal heart beat (black), or an anomalous one (red) (Fig. 5a). 133 We train the RSNN in an ex-situ fashion 11 , using Backpropagation Through Time (BPTT) 44 with surrogate gradient 134 approximations of the derivative of a LIF neuron activation function 45 (see Methods). We then transferred the resulting 135 here correctly identifying the heartbeat as arrhythmic.

141
The accuracy over the test set for 100 iterations of training, transfer and test is plotted in Fig. 5c  software model (97.0%). Thus it illustrates that not only the imposed small-world structure of the Mosaic does not have 144 a negative effect on the accuracy, but that the model is also robust to a severe degradation in the precision of the weights. 145 Although, due to the variability in the transfer process, the gap between software and experimental models can sometimes as 146 high as 1%. For further comparison, a non-spiking artificial Recurrent Neural Networks (RNN) was also applied to the same 147 task, obtaining a median accuracy of 96.1%. This lower software accuracy compared with the experimental Mosaic's model, 148 further confirms the Mosaic's computational power. This result is consistent with other observations whereby RSNNs have 149 outperformed non-spiking equivalents 46 .

150
Using estimates obtained from SPICE simulations of our fabricated test circuits and statistics from the Mosaic experiments 151 (see Methods), the average energy per routing operation is estimated to be 60 pJ. Given the average number of spikes per 152 heartbeat and the average number of routing tiles traversed between source and destination, the total energy required to process, 153 and make a prediction regarding one heartbeat using the Mosaic is 150 nJ.

154
To gain a perspective on the energy efficiency of Mosaic relative to other hardware approaches, we compare this figure 155 7/13 to the energy required for running the same neural network model on a conventional microprocessor, and on an AER-based 156 Complementary Metal-Oxide-Semiconductor (CMOS) neuromorphic processor 30 . While the energy required for a single 157 routing operation on a microprocessor, assumed to be equivalent to one Static Random Access Memory (SRAM) access, is 158 only 8 pJ, the total energy required per heartbeat is much greater -116µJ. This difference is in large part due to asynchronous 159 nature of memory access in the Mosaic approach relative to a microprocessor -where all variables are required to be updated 160 on each timestep of a numerical simulation. In an AER-based neuromorphic processor, 7.7 nJ is required to generate and route 161 an event between a source and a destination 30 (see Methods). Over the course of a full heartbeat, the total required energy is 162 therefore be 4.8µJ.

163
Based on these estimations, the Mosaic achieves a reduction of two and one orders of magnitude in total routing energy 164 per heartbeat, relative to a microprocessor and an AER-based neuromorphic processor, respectively. This can be attributed not only for computation, but also to route spikes. 172 We showed mathematically that, particularly as network size increases, the Mosaic offers a means of implementing small-173 world graphical models with less memory, and therefore energy, than previous approaches based on single large memristor 174 crossbars.

175
The two fundamental circuit blocks of the Mosaic, the neuron tile and the routing tile, were designed, fabricated and 176 experimentally demonstrated using a hybrid technology co-integrating 130 nm CMOS technology with resistive memory 177 devices. Based on the measurements of these circuits, a mixed hardware-software simulation of the Mosaic was developed and 178 the task of detecting arrhythmic heartbeats from ECG signals was addressed. The Mosaic was able to achieve an accuracy close 179 to that of an equivalent high-precision software model.  Mosaic is thus a timely dedicated hardware architecture optimized for a specific type of graph that is abundant in nature. Both neuron and routing column share a common circuit in Fig. 3a which reads the conductances of the RRAM devices.

192
The RRAM bottom electrode has a constant DC voltage V bot applied to it and the common top electrode is pinned to the 193 voltage V x by a rail-to-rail operational amplifier (OPAMP) circuit. The OPAMP output is connected in negative feedback to 194 its non-inverting input (due to the 90 degrees phase-shift between the gate and drain of transistor M 1 in Fig. 3a) and has the 195 constant DC bias voltage V top applied to its inverting input. As a result, the output of the OPAMP will modulate the gate voltage 196 of transistor M 1 such that the current it sources onto the node V x will maintain its voltage as close as possible to the DC bias 197 V top . Whenever an input pulse V in < n > arrives, a current i in equal to (V x −V bot )G n will flow out of the bottom electrode. The 198 negative feedback of the OPAMP will then act to ensure that V x = V top , by sourcing an equal current from transistor M 1 . By 199 connecting the OPAMP output to the gate of transistor M 2 , a current equal to i in , will therefore also be buffered, as i bu f f , into 200 the branch composed of transistors M 2 and M 3 in series. In the routing tile, this current is compared against a reference current, 201 and if higher, a pulse is generated and transferred onwards. The current comparator circuit is composed of two current mirrors 202 and an inverter (see Supplementary Fig. S6). In the neuron column, this current is injected into a CMOS differential-pair turn feeds back and shunts the capacitor to ground such that it is discharged. Further circuits were required in order to program 208 the device conductance states. Notably, multiplexers were integrated on each end of the column in order to be able to apply 209 voltages to the top and bottom electrodes the RRAM devices.

210
Fabrication/integration 211 The circuits described in Section have been taped-out in 130 nm technology at CEA-Leti, in a 200 mm production line. The 212 Front End of the Line, below metal layer 4, has been realized by ST-Microelectronics, while from the fifth metal layer upwards, 213 including the deposition of the composites for RRAM devices, the process has been completed by CEA-Leti. RRAM devices 214 are composed of a 5 nm thick H f O 2 layer sandwiched by two 5 nm thick TiN electrodes, forming an TiN/H f O 2 /Ti/TiN 215 stack. Each device is accessed by a transistor giving rise to the 1T1R unit cell. The size of the access transistor is 650 nm wide. 216 1T1R cells are integrated with CMOS-based circuits by stacking the RRAM cells on the higher metal layers. In the cases of the 217 neuron and routing tiles, 1T1R cells are organized in a small -either 2x2 or 2x4 -matrix in which the bottom electrodes are 218 shared between devices in the same column and the gates shared with devices in the same row. In this way, the devices can be 219 accessed in a parallel manner. The circuits integrated into the wafer, were accessed by a probe card which connected to the 220 pads of the dimension of [50x90]µm 2 .

236
Inputs and outputs are analog dynamical signals. In the case of the input, we have alternated two HP 8110 pulse generators 237 with a Tektronix AFG3011 waveform generator. As a general rule, input pulses had a pulse width of 1µ s and rise/fall time of 238 50 ns. This type of pulse is assumed as the stereotypical spiking event of a Spiking Neural Network. Concerning the outputs, a 239 1 GHz Teledyne LeCroy oscilloscope was utilized to record the output signals.  The Mosaic model used in this Article was composed of routing tiles of 16 × 16 devices in a Mosaic of 11 × 11 tiles. 249 Devices in the routing tiles were programmed to be in the HCS with a probability of 0.07. Neuron tiles were realized in a 250 20 × 4 array -this allows four signals from each of the four neighbouring tiles to be received independently, as well as four 251 neurons to connect recurrently amongst themselves within a tile.

252
The resulting skeleton connectivity matrix was then exported to a PyTorch model of an RSNN model to be trained on the 253 MIT-BIH heart arrhythmia dataset 41 . Specifically, all of the heartbeats of one patient (labelled as 201 in the dataset) were delta 254 modulated into four spike train channels. These spike trains then served as an effective spiking input layer of the model.

255
Data points were presented to the model in mini-batches of sixteen. Two populations of neurons in two neuron tiles were 256 used to denote whether the presented ECG signals corresponded to a healthy or an arrhythmic heartbeat. The softmax of the 257 total number of spikes generated by the neurons in each population was used to obtain a classification probability. The negative 258 log-likelihood was then minimized using the categorical cross-entropy with the labels of the signals. The derivative of the 259

9/13
Heaviside step function, that is used to rectify the membrane voltage of the LIF neurons into a zero or a one, was approximated 260 using the function 1/abs(V mem −V th) 2 -inline with surrogate gradient training methods 45 . 261 After training, the synaptic weights were transferred into a an array of 16 kb resistive memory devices co-integrated onto a 262 130 nm CMOS technology. The synaptic weight of each synapse was defined by the subtraction of two conductance states of 263 two devices. The process of transferring the high-precision software weights to the conductance states of the devices in the array 264 was achieved using an iterative closed-loop multilevel programming algorithm. It is based on adapting the SET programming 265 compliance current to obtain a conductance within a target range 24 and programming a device until its conductance falls within 266 a pre-defined margin of tolerated error. Such an approach allows each device to be programmed with ten non-overlapping 267 conductance levels.

268
The delta modulated test data was processed by a mixed hardware-software Moasic model. Whenever a pre-synaptic neuron   To evaluate the MF for one large crossbar array, the total number of devices required to implement any possible connections 289 between neurons can be counted -allowing for any Spiking Recurrent Neural Networks (SRNN) to be mapped onto the system.

297
The number of routing tiles which connects all the neuron tiles depends on the geometry of the Mosaic systolic array. Here, 298 we assume neuron tiles assembled in a square, each with a routing tile on each side. We consider R to be the number of routing 299 tiles with 4k 2 devices in each. This brings the total number of devices related to routing tiles up to MF RoutingTiles = R × (4k) 2 .

300
The problem can then be re-written as a function of the geometry. Considering Fig.2a, let i be an integer and (2i + 1) 2 the total number of tiles. The number of neuron tiles can be written as T = (i + 1) 2 , as we consider the case where neuron tiles form the outer ring of tiles. As a consequence, the number of routing tiles is R = (2i + 1) 2 − (i + 1) 2 . Substituting such values in the previous evaluations of MF NeuronTiles + MF RoutingTiles and remembering that k < N × T , we can impose that MF Mosaic = MF NeuronTiles + MF RoutingTiles < MF MF re f . This results in the following expression: MF Mosaic = MF NeuronTiles + MF RoutingTiles < MF re f erence (1) (i + 1) 2 4 × k 2 + [(2i + 1) 2 − (i + 1) 2 ]((4k) 2 ) < (k(i + 1) 2 ) 2 This expression can then be evaluated for i, given a network size, giving rise to the relationships as plotted in Fig.2c  In state-of-the-art event-based neuromorphic chips, the information is communicated through the AER scheme 29 . Whenever a 304 spiking neuron in a chip (or module) generates a spike, its "address" (or any given ID) is written on a high speed digital bus and 305 sent to the receiving neuron(s) in one (or more) receiver module(s). In our Mosaic structure, we have distributed the routing 306 information in a two-dimensional matrix along with the computing units.

307
To compare the routing energy and latency of Mosaic with the AER systems, we have calculated the energy per spike 308 routing in the best and worst case scenarios in both systems.

309
For AER-based systems, we are using the energy and latency numbers reported in Dynap-SE, as one of the most recent and 310 optimized AER routing schemes 30 . It is a multi-core neuromorphic comprising four cores; each core includes 256 neurons. It has 311 a hierarchical asynchronous routing, combining a source-based routing mesh architecture with a destination-based hierarchical 312 tree routing method. SRAM cells store the routing structure in the tree and the Content Addressable Memory (CAM) cells 313 store the tag of the source address to which each neuron is connected.

314
Therefore, once a spike is generated, the least energy consumption happens in a scenario where the events should be routed 315 locally, and thus 256 10-bit CAM cells are accessed. In the worst case, events have to travel from the first-level router to the 316 higher levels and thus the energy of reading SRAM cells are added. Therefore, the energy of routing one spike in Dynap-SE 317 can be calculated by the following equation: Where E Spike is the energy to generate one spike, E Pulse is the energy of the pulse extender circuit, E en is the energy to 319 encode one spike and append destination, E BC is the energy to broadcast the event to the same core, RT is 1 if the spike has to 320 be routed to other cores, otherwise zero, and E RT is the energy to route the events to other cores. If RT = 0, total energy to 321 route the event to the core sums up to 7.68 nJ. In case of the event routing to other cores, multiples of 360 pJ should be added to 322 the energy consumption (energy required for reading SRAM at each hierarchical router level.

323
For the case of the microprocessor, the equivalent of routing an event would be to load into the arithmetic logic unit memory 324 from an SRAM containing the synaptic weights and perform addition and multiplication operations to update neuron states and 325 outputs before writing this back into SRAM. We assume that this is dominated by the SRAM access, and so take figures from 326 the literature that give SRAM access energy figures 50 .