Floating-gate silicon-oxygen-nitrogen-oxygen-silicon (SONOS) transistors can be used to train neural networks to ideal accuracies that match those of floating-point digital weights on the MNIST ...handwritten digit data set when using multiple devices to represent a weight or within 1% of ideal accuracy when using a single device. This is enabled by operating devices in the subthreshold regime, where they exhibit symmetric write nonlinearities. A neural training accelerator core based on SONOS with a single device per weight would increase energy efficiency by 120×, operate 2.1× faster, and require 5× lower area than an optimized SRAM-based ASIC.
Recently, a Cambrian explosion of a novel, non-volatile memory (NVM) devices known as memristive devices have inspired effort in building hardware neural networks that learn like the brain. Early ...experimental prototypes built simple perceptrons from nanosynapses, and recently, fully-connected multi-layer perceptron (MLP) learning systems have been realized. However, while backpropagating learning systems pair well with high-precision computer memories and achieve state-of-the-art performances, this typically comes with a massive energy budget. For future Internet of Things/peripheral use cases, system energy footprint will be a major constraint, and emerging NVM devices may fill the gap by sacrificing high bit precision for lower energy. In this paper, we contrast the well-known MLP approach with the extreme learning machine (ELM) or NoProp approach, which uses a large layer of random weights to improve the separability of high-dimensional tasks, and is usually considered inferior in a software context. However, we find that when taking the device non-linearity into account, NoProp manages to equal hardware MLP system in terms of accuracy. While also using a sign-based adaptation of the delta rule for energy-savings, we find that NoProp can learn effectively with four to six 'bits' of device analog capacity, while MLP requires eight-bit capacity with the same rule. This may allow the requirements for memristive devices to be relaxed in the context of online learning. By comparing the energy footprint of these systems for several candidate nanosynapses and realistic peripherals, we confirm that memristive NoProp systems save energy compared with MLP systems. Lastly, we show that ELM/NoProp systems can achieve better generalization abilities than nanosynaptic MLP systems when paired with pre-processing layers (which do not require backpropagated error). Collectively, these advantages make such systems worthy of consideration in future accelerators or embedded hardware.
Bulk 14-nm FinFET technology was irradiated in a heavy-ion environment (42-MeV Si ions) to study the possibility of displacement damage (DD) in scaled technology devices, resulting in drive current ...degradation with increased cumulative fluence. These devices were also exposed to an electron beam, proton beam, and cobalt-60 source (gamma radiation) to further elucidate the physics of the device response. Annealing measurements show minimal to no "rebound" in the ON-state current back to its initial high value; however, the OFF-state current "rebound" was significant for gamma radiation environments. Low-temperature experiments of the heavy-ion-irradiated devices reveal increased defect concentration as the result for mobility degradation with increased fluence. Furthermore, the subthreshold slope (SS) temperature dependence uncovers a possible mechanism of increased defect bulk traps contributing to tunneling at low temperatures. Simulation work in Silvaco technology computer-aided design (TCAD) suggests that the increased OFF-state current is a total ionizing dose (TID) effect due to oxide traps in the shallow trench isolation (STI). The significant SS elongation and ON-state current degradation could only be produced when bulk traps in the channel were added. Heavy-ion irradiation on bulk 14-nm FinFETs was found to be a combination of TID and DD effects.
A well‐posed physics‐based compact model for a three‐terminal silicon–oxide–nitride–oxide–silicon (SONOS) synaptic circuit element is presented for use by neuromorphic circuit/system engineers. Based ...on technology computer aided design (TCAD) simulations of a SONOS device, the model contains a nonvolatile memristor with the state variable QM representing the memristor charge under the gate of the three‐terminal element. By incorporating the exponential dependence of the memristance on QM and the applied bias V for the gate, the compact model agrees quantitatively with the results from TCAD simulations as well as experimental measurements for the drain current. The compact model is implemented through VerilogA in the circuit simulation package Cadence Spectre and reproduces the experimental training behavior for the source–drain conductance of a SONOS device after applying writing pulses ranging from −12 V to +11 V, with an accuracy higher than 90%.
A well‐posed physics‐based compact model of a three‐terminal silicon–oxide–nitride–oxide–silicon (SONOS) synaptic circuit element is presented for neuromorphic circuit designs. Based on technology‐computer‐aided design (TCAD) simulations, a fundamental compact model requiring a memristor was formulated. The model was verified by simulation in Cadence Spectre with VerilogA, which yielded quantitative agreement to experimentally measured channel currents.
The radiation response of TaOx-based resistive memory (RRAM) devices fabricated in academic (Set A) and industrial (Set B) settings was compared. Ionization damage from a 60 Co gamma source did not ...cause any changes in device resistance for either device type, up to 45 Mrad(Si). Displacement damage from a heavy ion beam caused a decrease in resistance at 1× 10 21 oxygen displacements per cm 3 in Set B devices in the high-resistance state (HRS); meanwhile, Set A devices did not exhibit any decrease in resistance due to displacement damage. Both types of devices exhibited an increase in resistance around 3 × 10 22 oxygen displacements per cm 3 , possibly due to the damage at the oxide/metal interfaces. These extremely high levels of damage represent near-total atomic disruption, and if this level of damage was ever reached, other circuit elements would likely fail before the RRAM devices in this article. Overall, both sets of devices were much more resistant to radiation effects than the similar devices reported in the literature. Displacement damage effects were only observed in the Set A devices once the displacement-induced oxygen vacancies surpassed the intrinsic vacancy concentration in the devices, suggesting that high oxygen vacancy concentration played a role in the devices' high tolerance to displacement damage.
The domain wall-magnetic tunnel junction (DW-MTJ) is a versatile device that can simultaneously store data and perform computations. These three-terminal devices are promising for digital logic due ...to their nonvolatility, low-energy operation, and radiation hardness. Here, we augment the DW-MTJ logic gate with voltage-controlled magnetic anisotropy (VCMA) to improve the reliability of logical concatenation in the presence of realistic process variations. VCMA creates potential wells that allow for reliable and repeatable localization of domain walls (DWs). The DW-MTJ logic gate supports different fanouts, allowing for multiple inputs and outputs for a single device without affecting the area. We simulate a systolic array of DW-MTJ multiply-accumulate (MAC) units with 4-bit and 8-bit precision, which uses the nonvolatility of DW-MTJ logic gates to enable fine-grained pipelining and high parallelism. The DW-MTJ systolic array provides comparable throughput and efficiency to state-of-the-art CMOS systolic arrays while being radiation-hard. These results improve the feasibility of using DW-based processors, especially for extreme-environment applications such as space.
We evaluate the sensitivity of neuromorphic inference accelerators based on silicon-oxide-nitride-oxide-silicon (SONOS) charge trap memory arrays to total ionizing dose (TID) effects. Data retention ...statistics were collected for 16 Mbit of 40-nm SONOS digital memory exposed to ionizing radiation from a Co-60 source, showing good retention of the bits up to the maximum dose of 500 krad(Si). Using this data, we formulate a rate-equation-based model for the TID response of trapped charge carriers in the ONO stack and predict the effect of TID on intermediate device states between "program" and "erase." This model is then used to simulate arrays of low-power, analog SONOS devices that store 8-bit neural network weights and support in situ matrix-vector multiplication. We evaluate the accuracy of the irradiated SONOS-based inference accelerator on two image recognition tasks-CIFAR-10 and the challenging ImageNet data set-using state-of-the-art convolutional neural networks, such as ResNet-50. We find that across the data sets and neural networks evaluated, the accelerator tolerates a maximum TID between 10 and 100 krad(Si), with deeper networks being more susceptible to accuracy losses due to TID.
PDF
