Results for “outer product update”

DOE OSTI2021

In situ Parallel Training of Analog Neural Network Using Electrochemical Random-Access Memory

In-memory computing based on non-volatile resistive memory can significantly improve the energy efficiency of artificial neural networks. However, accurate in situ training has been challenging due to the nonlinear and stochastic switching of the resistive memory elements. One promising analog memory is the electrochemical random-access memory (ECRAM), also known as the redox transistor. Its low write currents and linear switching properties across hundreds of analog states enable accurate and massively parallel updates of a full crossbar array, which yield rapid and energy-efficient training. While simulations predict that ECRAM based neural networks achieve high training accuracy at significantly higher energy efficiency than digital implementations, these predictions have not been experimentally achieved. In this work, we train a 3 × 3 array of ECRAM devices that learns to discriminate several elementary logic gates (AND, OR, NAND). We record the evolution of the network’s synaptic weights during parallel in situ (on-line) training, with outer product updates. Due to linear and reproducible device switching characteristics, our crossbar simulations not only accurately simulate the epochs to convergence, but also quantitatively capture the evolution of weights in individual devices. The implementation of the first in situ parallel training together with strong agreement with simulation results provides a significant advance toward developing ECRAM into larger crossbar arrays for artificial neural network accelerators, which could enable orders of magnitude improvements in energy efficiency of deep neural networks.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

The organic redox transistor for neuromorphic computing

Inspired by the in-memory computing architectures of biological systems, neuromorphic computing using crossbar arrays of artificial synapses based on non-volatile memory (NVM) devices with variable conductances has emerged as a new paradigm to enable massively parallel and ultra-low power computing hardware for data centric applications. Although inference has been demonstrated successfully using crossbars based on a variety of NMV technologies, efficient learning and scaling to large arrays (>10 6 elements) remains a challenge due to the synaptic elements' non-ideal electrical characteristics which degrades ANN accuracy. A further challenge is that in the conductive state memristors draw large currents >μA resulting in significant voltage drops in the interconnect wires and increased probability of failure in scaled arrays. We suggest the organic polymer redox transistor (RT) is an alternate approach that could solve many of these challenges, enabling both inference and parallel outer product updates, as recently demonstrated by Fuller et al. An RT consists of redox-active channel and gate electrodes in contact with a liquid or solid electrolyte. lon insertion through the electrolyte controls the channel electronic conductivity, while electron transfer through an external circuit maintains overall charge neutrality. Unlike a rechargeable battery, in the RT the voltage built-up across the electrolyte is kept to a minimum (typically <100 mV) by using the same material for the gate and channel. Elimination of the voltage offset simplifies integration of the RT into programmable arrays by enabling the use of various selectors. RTs based on inorganic and organic materials have been recently demonstrated with conductance tuning occurring at potentials of just a few mV and hundreds to thousands of linearly and symmetrically programmable conductance states, enabling near ideal accuracy in neural network simulations. Introduced in the 1980's, redox transistors with metallic gate electrodes and organic channel materials, also known as organic electrochemical transistors (OECTs), have been explored for a variety of applications such as chem- and bio-sensing, neural interfaces, and low cost printed circuits. A typical channel material for OECTs is the conducting polymer poly(3,4-ethylenedioxythiophene) doped with poly(styrene sulfonate) (PEDOT:PSS). PEDOT is a p-type semiconducting polymer with mobile positively charged polarons that hop chain-to-chain.

Talin, Albert Alec↗

DOE OSTI2020

PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-Efficient ReRAM

The wide adoption of deep neural networks has been accompanied by ever-increasing energy and performance demands due to the expensive nature of training them. Additionally, numerous special-purpose architectures have been proposed to accelerate training: both digital and hybrid digital-analog using resistive RAM (ReRAM) crossbars. ReRAM-based accelerators have demonstrated the effectiveness of ReRAM crossbars at performing matrix-vector multiplication operations that are prevalent in training. However, they still suffer from inefficiency due to the use of serial reads and writes for performing the weight gradient and update step. A few works have demonstrated the possibility of performing outer products in crossbars, which can be used to realize the weight gradient and update step without the use of serial reads and writes. However, these works have been limited to low precision operations which are not sufficient for typical training workloads. Moreover, they have been confined to a limited set of training algorithms for fully-connected layers only. To address these limitations, we propose a bit-slicing technique for enhancing the precision of ReRAM-based outer products, which is substantially different from bit-slicing for matrix-vector multiplication only. We incorporate this technique into a crossbar architecture with three variants catered to different training algorithms. To evaluate our design on different types of layers in neural networks (fully-connected, convolutional, etc.) and training algorithms, we develop PANTHER, an ISA-programmable training accelerator with compiler support. Our design can also be integrated into other accelerators in the literature to enhance their efficiency. Our evaluation shows that PANTHER achieves up to 8.02×, 54.21×, and 103× energy reductions as well as 7.16×, 4.02×, and 16× execution time reductions compared to digital accelerators, ReRAM-based accelerators, and GPUs, respectively.

42 ENGINEERING↗

DOE OSTI2021

Modeling Summary of ASNF in DOE Sealed Standard Canisters

A pathway for road-ready and final disposition packaging configurations for the aluminum-clad spent nuclear fuel (ANSF) fuel dictates storage within helium backfilled sealed DOE standard canisters. The typical packaging configuration for the 15-foot DOE canisters places 10 advance test reactor (ATR) elements within a Type 1a basket, and three baskets are loaded within each DOE canister. During in-reactor operations and cooling pond storage conditions, oxyhydroxide layers form on the surface of the aluminum clad fuel. These layers produce hydrogen gas over time due to the fuel’s radiation field. As part of the packing procedure, the ATR fuel should be dried to remove any residual physio-/chemi- sorbed water bound to the surface. The results of this modeling will include results at fully saturated and fully dried conditions. In addition, fuels that are currently stored at the Savannah River Site were also studied for their potential for hydrogen and pressure build up. These two additional fuels modeled were the Missouri University Research Reactor (MURR) fuel, which is packaged in the same configuration as the ATR, but with a 10-foot-tall DOE standard canister. This was selected due to its relatively high decay heat compared to other DOE-managed ASNF. The second additional fuel studied with the modeling effort was the High Flux Isotope Reactor (HFIR) fuel. This fuel is modeled as two separate DOE canisters with the inner and outer annulus split for storage. The HFIR was selected for study due to its high aluminum cladding surface area. In the associated experimental work, Task 2 - Oxyhydroxide Layer Radiolytic Gas Generation Resolution, additional tests were completed in a helium environment, and updated G-values for the radiolytic production of hydrogen from the oxyhydroxide layers were provided. These values were 2.92 ×10 -4 µmol/J at 50% relative humidity and 4.12 ×10 -4 µmol/J at 100% relative humidity. These values are lower than the value for Argon that was used in prior modeling results. In addition, prior modeling results have been completed with the G-value applied to just the mass of the corrosion layer, and this has been updated to apply the G-value for hydrogen generation to the full mass of the fuel. These two effects combine to show much smaller pressure and hydrogen build up for the sealed canister model. For a nominal scenario of stored ATR fuel, after 50 years the model results give a 1.36 atm total pressure, 7% mole percent hydrogen, for the upper decay heat, 1.51 atm total pressure, 16% mole percent hydrogen, and for upper decay heat with undried fuel 2.6 atm total pressure, 15% mole percent hydrogen. For the MURR nominal case, the model results give 1.34 atm and 6% hydrogen, for upper decay heat this gives 1.41 atm total pressure with 10.8 % hydrogen, and for upper decay heat with undried fuel, this gives 2.38 atm total pressure with 9.9% hydrogen. The nominal scenario for HFIR fuel gives 1.39 atm total pressure with 9.9% hydrogen, the upper decay heat case gives 1.43 atm with 12.1% hydrogen, and the upper decay heat with undried fuel gives 2.17 atm total pressure with 11.9% hydrogen. These results confirm the ATR scenario bounds the other intact ASNF modeled here. No case modeled yields significant oxygen, and the lower decay heat cases for all fuels modeled have hydrogen concentrations that are under the 4% flammability limit after 50 years of storage. In addition, the modeled pressures for all cases are all significantly below the 500-psi limit for the DOE standard sealed canister.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2021

Gamma Spectrometry Examination of the AGR-3/4 Irradiation

The results from gamma spectrometry examination of the different components from the combined third and fourth U.S. Advanced Gas Reactor (AGR) TRISO-coated particle fuel irradiation tests (AGR-3/4) have been analyzed. This experiment was designed to provide information about in-pile fission product migration. In each of the 12 capsules, a single stack of four compacts with designed-to-fail (DTF) particles surrounded by inner and outer graphite and/or graphitic matrix rings and a graphite sink ring were irradiated in the Advanced Test Reactor (ATR) at Idaho National Laboratory (INL). Gamma spectrometry has been used to evaluate the gamma-emitting fission product inventory of compacts from the irradiation and evaluate the burnup of these compacts based on the activity of the radioactive cesium isotopes (i.e., Cs-134 and Cs-137) in the compacts. Burnup from gamma spectrometry compares well with predicted burnup from simulations. The inner and outer rings were also examined by gamma spectrometry to evaluate the total fission product inventory of the rings, and gamma emission computed tomography (GECT) was used to investigate the spatial distribution of gamma-emitting fission products within the rings. This report focuses on results obtained via the Precision Gamma Scanner (PGS) for the compacts and inner and outer rings. These non-destructive gamma measurements are currently being compared to recent destructive measurements of spatial fission product distributions in the rings and to predictions made via mathematical modeling. A separate report discusses the entire AGR-3/4 fission product mass balance for all irradiation capsule components including the inner and outer rings, graphite sink rings, capsule spacers, capsule foils, through tubes, and felts. This report is similar to the conference proceedings: Harp, J.M., Demkowicz, P.A., and Stempien, J.D., “Initial gamma spectrometry examination of the AGR-3/4 irradiation,” International Topical Meeting High Temperature Reactor Technology (HTR 2016), Las Vegas, NV, USA, November 2016, Paper HTR2016-18593, but it has been updated to reflect all the data that has been collected on AGR-3/4 using PGS. The Appendices contain extensive data from the collected gamma spectra.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2020

Gamma Spectrometry Examination of the AGR-3/4 Irradiation

The results from gamma spectrometry examination of the different components from the combined third and fourth U.S. Advanced Gas Reactor (AGR) TRISO-coated particle fuel irradiation tests (AGR-3/4) have been analyzed. This experiment was designed to provide information about in-pile fission product migration. In each of the 12 capsules, a single stack of four compacts with designed-to-fail (DTF) particles surrounded by inner and outer graphite and/or graphitic matrix rings and a graphite sink ring were irradiated in the Advanced Test Reactor (ATR) at Idaho National Laboratory (INL). Gamma spectrometry has been used to evaluate the gamma-emitting fission product inventory of compacts from the irradiation and evaluate the burnup of these compacts based on the activity of the radioactive cesium isotopes (i.e., Cs-134 and Cs-137) in the compacts. Burnup from gamma spectrometry compares well with predicted burnup from simulations. The inner and outer rings were also examined by gamma spectrometry to evaluate the total fission product inventory of the rings, and gamma emission computed tomography (GECT) was used to investigate the spatial distribution of gamma-emitting fission products within the rings. This report focuses on results obtained via the Precision Gamma Scanner (PGS) for the compacts and inner and outer rings. These non-destructive gamma measurements are currently being compared to recent destructive measurements of spatial fission product distributions in the rings and to predictions made via mathematical modeling. A separate report discusses the entire AGR-3/4 fission product mass balance for all irradiation capsule components including the inner and outer rings, graphite sink rings, capsule spacers, capsule foils, through tubes, and felts. This report is similar to the conference proceedings: Harp, J.M., Demkowicz, P.A., and Stempien, J.D., “Initial gamma spectrometry examination of the AGR-3/4 irradiation,” International Topical Meeting High Temperature Reactor Technology (HTR 2016), Las Vegas, NV, USA, November 2016, Paper HTR2016-18593, but it has been updated to reflect all the data that has been collected on AGR-3/4 using PGS. The Appendices contain extensive data from the collected gamma spectra.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2025

Assessment of buffer-IPyC thermomechanical debonding behavior using new experimental strength data in BISON

TRIstructural ISOtropic (TRISO) fuel is a nuclear fuel commonly used in High Temperature Gas-cooled Reactors (HTGRs). A single sub-millimeter-diameter TRISO fuel particle consists of a spherical fuel kernel surrounded by four coating layers: a low-density pyrocarbon buffer layer, an inner pyrolytic carbon (IPyC) layer, a silicon carbide (SiC) layer, and an outer pyrolytic carbon (OPyC) layer. The kernel is commonly made of UO2 or a mixture of uranium carbide and uranium oxide (UCO). During reactor operation, the TRISO coating layers are subjected to irradiation-induced dimensional changes and the associated thermomechanical behavior of each layer. One of the observed behaviors is gap formation between the buffer and IPyC layer due to the porous buffer’s irradiation-induced shrinkage exceeding that of the IPyC layer. Not all irradiated particles will experience buffer-IPyC gap formation. The debonding may be partial, or it may be nearly total. However, from post-irradiation examination of UCO TRISO fuels irradiated as part of the Advanced Gas Reactor (AGR) Fuel Development and Qualification Program, it was concluded that partial buffer-IPyC debonding was the most common type of buffer-IPyC interaction. To predict TRISO thermomechanical performance, multi-physics models have been built that are being continually updated and refined. The BISON code is a finite element-based nuclear fuel performance code that may be used for 1D, 2D, and 3D TRISO particle simulations. This code is used to calculate fuel temperature, kernel swelling, buffer densification, thermal and irradiation creep, fracture, and fission gas production and release. One of the recent additions to the BISON code is the ability to model the process of layer debonding. This paper will focus on the simulation results of the improved BISON debonding model that will utilize updated strengths measured from irradiated AGR TRISO fuel particles. The new experimental strength data from micromechanical tests of irradiated TRISO fuel samples were exercised in the BISON simulations and compared to baseline strength data to assess their applicability in the models. This also includes updated buffer-IPyC bond strengths to simulate layer delamination. Based on current experimental observations it is noted that the buffer-IPyC separation occurs not exactly at the junction of these two layers, but more on the side of the buffer layer. That observation is also implemented in the TRISO interface debonding model. This improved modeling approach using experimental strength data to characterize buffer-IPyC debonding and its potential subsequent cracking will be presented in the paper along with comparisons to available experimental observations.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗