Unit device characteristics of neuron and synapse
N-channel single transistor neuron and synapse have the same SONOS structure, as shown in Fig. 1b. The intercalated charge trap nitride (Si3N4) in the multi-layered gate dielectrics allows multi-states according to the amount of trapped charges. They can perform two functions: (i) enable excitatory/inhibitory function or tuning the VT,firing in the neuron and (ii) control weight update in the synapse. Like the homotype, the neuron and the synapse have the same structure but operate differently, as shown in Fig. 1c. For neuron operation, input current (Iin) collected from the pre-synapses is applied to a n+ drain (or source) electrode, and output voltage (Vout) is produced from the same n+ drain (or source) electrode. For synapse operation, the voltage transferred from the pre-neuron (Vin) is applied to the gate electrode of the synapse, and output current (Iout) is flown from the n+ source (or drain) electrode. These neurons and synapses were fabricated on an 8-inch wafer by using the same standard Si CMOS process, and were connected to each other through metallization for a monolithically integrated neuromorphic system, as shown in Fig. 1d. The fabrication details are described in Supplementary Information 1.
As mentioned earlier, the excitatory/inhibitory state of the neuron is determined by electron trapping in the nitride of the SONOS structure. An inhibitory function that disables the firing of the neuron is necessary, because it can improve the energy efficiency of the neuromorphic system by selectively firing a specific neuron. Hence it can realize effective learning and inference through the winner-takes-all (WTA) mechanism.28-30 As shown in Fig. 2a, unless the electrons are trapped in the nitride, the neuron is at a low-resistance state (LRS). Thus, current flows through the channel when the Iin is applied. As a consequence, charges are not integrated and a leaky integrate-and-fire (LIF) function is inhibited. Otherwise, the neuron is at a high-resistance state (HRS) when trapped electrons in the nitride raise a potential barrier between n+ source and p-type channel referred to as a p-n built-in potential. Accordingly, charges are integrated until the firing. For the neuron operation, the gate of the neuron transistor is a kind of a pseudo-gate, unlike a conventional actual-gate. It is used not for the LIF operation but for charge trapping. For electron trapping in the nitride, a positive voltage pulse is applied to the pseudo-gate. Afterwards, it is sustained in a floating state for the neuron operation. Due to non-volatility of the trapped charges even without gate biasing, energy consumption is much smaller compared to our previous study, which required additional and continuous gate voltage control.31,32
Fig. 2b shows output characteristics of the fabricated n-channel single transistor neuron, which is represented by the drain current versus drain voltage (ID-VD). Its gate length (LG) and channel width (WCH) are 880 nm and 280 nm, respectively. Before the electron trapping, ID flows regardless of VD. After the electron trapping with gate voltage (VG) of 12 V and pulse time of 100 ms, the ID does not flow at a low VD. However, a large amount of ID abruptly flows beyond a critical VD; this is called latch-up voltage (Vlatch). This is known as a phenomenon of single transistor latch (STL) and serves as a threshold switch.33,34
Fig. 2c shows the Vout versus time when a constant Iin was applied to the drain electrode of the single transistor neuron, before and after the electron trapping. The Vout was measured at the same drain electrode. Before the electron trapping, the applied Iin directly flowed through the channel toward the source, and charge accumulation (integration) was not allowed. As a result, the inhibitory function was enabled, unlike the two-terminal based memristor neuron. After the electron trapping, the applied Iin did not flow out toward the source and charges accumulated in a parasitic capacitor (Cpar). According to this integration process, VD equivalent to Vout was increased prior to the VT,firing. Simultaneously, iterative impact ionization was induced by the increased VD, and holes accumulated in the body. When the Vout reaches Vlatch, which is the same as the VT,firing, the accumulated charges in Cpar are suddenly discharged by STL. This is a firing process. Therefore, spiking of the neuron was mimicked. Fig. S2 shows the energy band diagram during the LIF operation, which was extracted by a TCAD device simulation. Note that at the moment of the firing, the energy barrier between the n+ source and p-type body is lowered enough to allow the integrated charges to escape toward the source. The measured spiking frequency (f) was increased as the Iin was increased. In addition, leaky characteristics appeared, as described in Supplementary Information 3. The single transistor neuron shows typical LIF operation.
In addition to the control of the excitatory/inhibitory state, the VT,firing was tunable by controlling the trapped charge density in the nitride. This tunable property of the VT,firing is important to implement a reliable neuromorphic system.25-27 If the conductivity of the synapse is unsuitably low or high owing to process-induced variability and endurance problems, the targeted number of firings cannot be achieved. To suppress this instability, a tunable VT,firing is required. As shown in Supplementary Information 4, the VT,firing was increased as the number of pulses that can control the trapped charge density was increased. This tendency is because fewer electrons were injected over the built-in potential of the n+ source and the p-type body due to the greater amount of trapped charges, and thereby the Vlatch was increased. In summary, the demonstrated multi-state single transistor neuron harnesses both controllability of the excitatory/inhibitory and tunability of the VT,firing.
The f of the LIF neuron can be modeled as follows:
where Roff is the resistance at HRS during the integration. As the VT,firing decreases, the f increases because the firing occurs at the lower voltage. It should be noted that the VT,firing, which corresponds to the Vlatch in Fig. 2b, is determined by various parameters such as LG (Supplementary Fig. 5), body doping concentration, and energy band gap.34,35 As the Iin increases, charging speed becomes faster, the f tends to be increased. Besides the VT,firing and Iin, the Cpar plays an important role in controlling the f. From the above equation, the f is increased as the Cpar is reduced because it takes shorter time to charge the smaller parasitic capacitor (Supplementary Information 6). Accordingly, energy consumption per spike (E/spike) is also decreased as the Cpar is reduced. Power consumption was compared between the single transistor neuron and the memristor neuron. The peak power consumption was extracted from the multiplication of peak current and peak Vout (Supplementary Information 7). It was found that the single transistor neuron consumed 120 nW, which was 10 to 104-fold smaller than the consumption of the memristor neuron, owing to a small cross-sectional channel area for current flowing due to high scalability of the nano-CMOS fabrication. On the other hand, it is noteworthy that the single transistor neuron has a bidirectional characteristic, in which the spiking operation is possible in both the drain input/output (I/O) and source I/O (Supplementary Information 8). This bidirectional characteristic can provide more degrees of freedom in designing a neuromorphic system. Thus, we employed both methods to construct a neuromorphic system.
Since the synapse device has the same SONOS structure as the neuron, the weight of the synapse can be adjusted by controlling the trapped charge density in the nitride. For example, if the electrons are trapped by applying a positive bias to the gate, the threshold voltage (VT) is shifted rightward and the channel conductance is decreased at the same read voltage, as depicted in Fig. 2d. This is a kind of depression. Otherwise, VT is shifted leftward and the channel conductance is increased at the same read voltage. This is a kind of potentiation. Fig. 2e shows transfer characteristics of the fabricated n-channel single transistor synapse, which is represented by the drain current versus gate voltage (ID-VG). Its LG and WCH are 1880 nm and 180 nm, respectively. VT was adjusted by the applied gate voltage that controls the trapped charge density. The potentiation-depression (P-D) curve in Fig. 2f shows the conductance change (weight update) according to the number of applied pulses with an identical amplitude and duty cycle. Both VG and VD for the reading operation were set as 1 V. The VG for potentiation and depression was set as -11 V with a pulse width of 100 ms and 11 V with a pulse width of 10 ms, respectively. As a result, 32 levels (5 bits) of conductance states were secured.
Co-integration of neuron and synapse
If a neuron and a synapse are homotypic, they can be integrated on the same plane at the same time with the same fabrication. Thereafter they can be connected by metal interconnections. This co-integration is demonstrated for two layers in a neural network. One is a pre-layer composed of a pre-synaptic neuron and a transmitted synapse. The other is a post-layer comprising a transmitting synapse and a post-synaptic neuron. Figs. 3a-c show the co-integrated pre-synaptic neuron and transmitted synapse as the pre-layer. Referring to the circuit schematic of Fig. 3a, a constant input current (Iin,neuron) is applied to the drain electrode of the neuron, and the drain is connected to a gate of the synapse to apply the output voltage from the pre-synaptic neuron (Vout,pre-neuron). Note that this configuration employs the abovementioned drain I/O scheme. Therefore, when spiking of the neuron occurs, the corresponding drain current (ID) flows through the channel of the synapse. Its magnitude is modulated by the synaptic weight. In the case of a three-terminal synapse such as a MOSFET, input resistance to the gate is huge. On the contrary, in the case of a two-terminal synapse such as a memristor, the input resistance is too small to suppress the loading effect where a neuronal output is influenced by the resistance of the synapse when it is directly connected to the neuron.36,37 This is a great advantage for large-scale co-integration of pre-synaptic neurons and transmitted synapses, as explained in Supplementary Information 9. Fig. 3b shows the fabricated pre-synaptic neuron and transmitted synapse interconnected through metallization. As shown in Fig. 3c, the spike-shaped output current of the transmitted synapse (Iout,syn) was increased according to the Vout,pre-neuron of the excitatory pre-synaptic neuron in order of weight: w1<w2<w3. It should be noted that the f of the Iout,syn was determined by the Iin,neuron. The spiking was inhibited when it was connected to the inhibitory pre-synaptic neuron, as shown in Supplementary Information 9. Note that stable inference operation is allowed unless the tunneling oxide thickness of the SONOS-based synapse is reduced (Supplementary Information 10). This is because the synaptic weight would not be changed by Vout,pre-neuron, which is small compared to the voltage of potentiation/depression. Figs. 3d-f show the co-integrated post-layer composed of the transmitting synapse and the post-synaptic neuron. As shown in the circuit schematic of Fig. 3d, a constant gate voltage (Vin,syn) is applied to the transmitting synapse, and the drain of the synapse is connected to the source of the post-synaptic neuron. Iout,syn is thus applied to the post-synaptic neuron. The output voltage is measured at the source of the post-synaptic neuron. In other words, it adopts the source I/O scheme. If the Iout,syn is applied from the source of the transmitting synapse to the drain of the post-synaptic neuron (drain I/O scheme), the source voltage of the transmitting synapse is floated. This is the reason why the post-synaptic neuron is selected to have the source I/O scheme. Fig. 3e shows the fabricated transmitting synapse and post-synaptic neuron interconnected through metallization. As shown in Fig. 3f, the f of the output voltage from the excitatory post-synaptic neuron (Vout,post-neuron) is increased according to the increment of Iout,syn from the transmitting synapse in order of weight: w1<w2<w3.
Another way to connect the transmitting synapse and the post-synaptic neuron is suggested in Supplementary Information 11, where a current mirror is used. The current mirror is composed of two NMOSFETs and two PMOSFETs. In this case, the drain I/O scheme of the post-synaptic neuron is available by reversing the direction of the Iout,syn. This configuration is also attractive to modulate the Iout,syn over a wide range by changing the channel width of the current mirror. In addition to the current mirror that can be used for analog circuitry, an inverter composed of an NMOSFET and a PMOSFET, which is a fundamental block to construct digital logic circuitry that controls the neural network for collecting, processing, and transporting data, was also fabricated on the same plane with co-integration of the neuron and synapse at the same time (Supplementary Information 12).
Gain modulation and coincidence detection
Using the co-integrated neurons and synapses, spatio-temporal neural computations such as gain modulation and coincidence detection were carried out. In biology, gain modulation is observed in many cortical areas and is thought to play an important role in maintaining stability.38-41 Herein gain modulation was realized by co-integration of two transmitting synapses and one post-synaptic neuron, as shown in the circuit diagram of Fig. 4a. Two types of pre-synaptic inputs are applied to the gate electrodes of two synapses. A driving input (VG,S1) enables the post-synaptic neuron to fire and a modulatory input (VG,S2) tunes the effectiveness of the driving input, as illustrated in Fig. 4b. As shown in Fig. 4c, the f of the post-synaptic neuron was modulated by the VG,S2 for the fixed VG,S1. This is because the Iin applied to the post-synaptic neuron was increased as the VG,S2 was increased. Fig. 4d shows the secondary data that the f was increased as the VG,S2 was increased at various VG,S1.
Coincidence detection is another important neural computation that encodes information by detecting the occurrence of temporally close but spatially distributed input signals. It has been found that coincidence detection is significant for highly efficient information processing in auditory and visual systems.42-45 By the co-integration of neuron and synapses, coincidence detection is also possible. When two inputs were applied at the same time, the f was increased because the Iin applied to the post-synaptic neuron was increased, as illustrated in Fig. 4b. Accordingly, it is possible to determine whether two inputs are simultaneously applied. Fig. 4e shows the corresponding data. When the two input signals applied at the same time, the f of the neuron was larger than the other cases of the two signals that were not synchronized. In addition, when two input signals overlapped for a certain period of time, the f of the neuron increased only in the overlap region.
Letter recognition with hardware circuit simulation
The neuromorphic system is commonly used to recognize images such as letters, numbers, objects, and faces. Pattern recognition of a letter was demonstrated with the aid of SPICE circuit simulations that were based on the measured neuron-synapse characteristics. As a simple model, the neuron is composed of a threshold switch and a parasitic capacitor connected in parallel. As a result, the simulated electrical properties are similar to the measured characteristics from the fabricated neuron, as shown in Supplementary Fig. 3. The synapse was implemented with a three-terminal MOSFET, and the weight of the synapse was controlled by adjusting the VT. We implemented two types of neural networks: a classifier based on a single-layer perceptron (SLP) and an auto-encoder based on a multi-layer perceptron (MLP). First, a neural network for the classifier was constructed to distinguish the letters ‘n’, ‘v’, and ‘z’, which was composed of 3×3 black-and-white pixels (Fig. 5a). It was composed of 9 input layers labeled with ‘i1’ to ‘i9’, which correspond to each pixel and 3 output layers labeled with ‘On’, ‘Ov’ and ‘Oz’ that are corresponding to each letter (Fig. 5b). The circuit diagram for the classifier is shown in Supplementary Fig. 13. Note that the output neurons were connected to each other to enable the lateral inhibition. According to the output voltage of the output neurons, each letter was identified. First spiking occurred in the first neuron for the input of ‘n’, the second neuron was for the input of ‘v’, and the third neuron was for the input of ‘z’. It should be noted that the multi-state properties of the single transistor neuron play an important role in recognizing a pattern. First, it was confirmed that the unwanted spiking was inhibited by the inhibitory neurons prior to reaching the VT,firing, which can enhance the energy efficiency of the neural network. Second, it was verified that the pattern was well recognized by appropriately tuning the VT,firing, even if the synaptic weight was changed abnormally. This feature can enhance the reliability of the neural network (Supporting Information 14).
In order to improve the recognition rate of an image, an auto-encoder is commonly used.46 The auto-encoder can remove the effect of noisy input and reconstruct the image by encoding the image and decoding it again. As shown in Fig. 5c, we implemented the auto-encoder by use of the MLP network with one middle layer. The input layer and the output layer were composed of 9 neurons, and each layer represented each pixel. After encoding three letters in the first perception, the information of each pixel was newly decoded in the second perception. A circuit diagram for the auto-encoder is shown in Supplementary Fig. 15. It should be noted that the inhibitory function of the single transistor neuron allowed the auto-encoder operation. In more detail, the middle neurons were connected to each other to enable lateral inhibition, and hence the noisy signal could be removed. Receiving the signal from the middle neurons, some output neurons expressed spiking while others were inhibited. The excited output neuron was decoded as a black pixel, while the inhibited output neuron was decoded as a white pixel, as shown in Fig. 5c. As a result, noisy input images became clearer via the image reconstruction by the auto-encoder.
Face recognition with software simulation
Using the hardware-based circuit simulation, off-chip learning that is applicable to inference operation with fixed weights of the synapses was implemented. On the other hand, on-chip learning is also possible by using additional circuits. With the aid of a MATLAB software simulation, a network capable of face recognition through on-chip learning was explored. A fully connected two-layer spiking neural network (SNN) consisting of 32×32 input neurons, 20 neurons in a middle layer, and three output neurons was designed, as shown in Fig. 6a. The measured neuron-synapse characteristics were reflected to the simulation based on the circuit diagram of Fig. 6b. From the Yale Face Database, nine training images composed of 32×32 pixels were selected (Fig. 6c).47 After clustering from an unsupervised crossbar, the classification was evaluated by a supervised crossbar. Neuronal output was converted through a waveform generator to make a proper pulse shape (Supplementary Fig. 16a). In addition, synaptic weight updates, which depend on the time difference between the pre-synaptic pulse (Vpre) and post-synaptic pulse (Vpost) according to a learning rule of spike timing dependent plasticity (STDP), were made.26,30 It should be emphasized that such circuits for waveform generation can be co-integrated on the same plane with neurons and synapses by standard CMOS fabrications. Also, it is noteworthy that lateral inhibition of the output neurons was enabled for efficient learning and inference by the WTA. After the training, the conductance of the synapses was determined, as shown in the visual map diagram of the synapse array (Fig. 6d). The recognition rate was evaluated with a test set containing 24 images of three people. As a result, a recognition rate of 95.8 % was achieved for ‘after training with the lateral inhibition’ and that of below 60 % was observed for ‘after training without the lateral inhibition,’ as shown in Fig. 6e. Unless the lateral inhibition was applied, a high level recognition was not performed because the global weight updates were performed via the firing of all engaged neurons. In addition, even though the conductance of the synapses were abnormally changed by process-induced variability or endurance problems, the recognition failure was prevented by the VT,firing modulation (Fig. S16d). These results prove that the reliable neural network can be implemented by the multi-state single transistor neuron.