Biological Dynamics Enabling Training of Binary Recurrent Networks
We demonstrate a method for the training of binary activation recurrent neural networks
Chapman, George William↗
SEARCH · Search NASA
Results for “binary network”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
BitGNN: Unlocking the Performance Potential of Binary Graph Neural Networks on GPUs
Graph Neural Networks (GNNs) have shown compelling results in many graph-based learning tasks. They are, however, time-consuming. Recent work has shown a promising direction in improving GNN speed and shrinking the size — network binarization, which binarizes network values and operations. Prior work, however, mainly focused on algorithm designs, leaving it open on how to fully materialize the performance potential. This work fills the gap by proposing techniques to best map binary GNNs and their computations to fit the nature of bit manipulations, optimizations and algorithms to maximize BSpMM kernel efficiency, and solutions to other factors influencing the end-to-end time on GPUs. Results on real-world graphs show that the proposed techniques outperform state of-the-art binary GNN implementations by 21-67× with little accuracy loss.
Chen, Jou-An↗
Biological Dynamics Enabling Training of Binary Recurrent Networks
Explore the source record for details and available documents.
Chapman, George William↗
The Interplay of Binary and Quantitative Structure on the Stability of Mutualistic Networks
Synopsis Understanding how the structure of biological systems impacts their resilience (broadly defined) is a recurring question across multiple levels of biological organization. In ecology, considerable effort has been devoted to understanding how the structure of interactions between species in ecological networks is linked to different broad resilience outcomes, especially local stability. Still, nearly all of that work has focused on interaction structure in presence-absence terms and has not investigated quantitative structure, i.e., the arrangement of interaction strengths in ecological networks. We investigated how the interplay between binary and quantitative structure impacts stability in mutualistic interaction networks (those in which species interactions are mutually beneficial), using community matrix approaches. We additionally examined the effects of network complexity and within-guild competition for context. In terms of structure, we focused on understanding the stability impacts of nestedness, a structure in which more-specialized species interact with smaller subsets of the same species that more-generalized species interact with. Most mutualistic networks in nature display binary nestedness, which is puzzling because both binary and quantitative nestedness are known to be destabilizing on their own. We found that quantitative network structure has important consequences for local stability. In more-complex networks, binary-nested structures were the most stable configurations, depending on the quantitative structures, but which quantitative structure was stabilizing depended on network complexity and competitive context. As complexity increases and in the absence of within-guild competition, the most stable configurations have a nested binary structure with a complementary (i.e., anti-nested) quantitative structure. In the presence of within-guild competition, however, the most stable networks are those with a nested binary structure and a nested quantitative structure. In other words, the impact of interaction overlap on community persistence is dependent on the competitive context. These results help to explain the prevalence of binary-nested structures in nature and underscore the need for future empirical work on quantitative structure.
Zoology↗
Designing Glass and Crystalline Phases of Metal–Bis(acetamide) Networks to Promote High Optical Contrast
Owing to their high tunability and predictable structures, metal–organic materials offer a powerful platform to study glass formation and crystallization processes and to design glasses with unique properties. In this work, we report a novel series of glass-forming metal–ethylenebis(acetamide) networks that undergo reversible glass and crystallization transitions below 200 °C. The glass-transition temperatures, crystallization kinetics, and glass stability of these materials are readily tunable, either by synthetic modification or by liquid-phase blending, to form binary glasses. Pair distribution function (PDF) analysis reveals extended structural correlations in both single and binary metal–bis(acetamide) glasses and highlights the important role of metal–metal correlations during structural evolution across glass–crystal transitions. Notably, the glass and crystalline phases of a Co–ethylenebis(acetamide) binary network feature a large reflectivity contrast ratio of 4.8 that results from changes in the local coordination environment around Co centers. These results provide new insights into glass–crystal transitions in metal–organic materials and have exciting implications for optical switching, rewritable data storage, and functional glass ceramics.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Accelerating cavity fault prediction using deep learning at Jefferson Laboratory
Abstract Accelerating cavities are an integral part of the continuous electron beam accelerator facility (CEBAF) at Jefferson Laboratory. When any of the over 400 cavities in CEBAF experiences a fault, it disrupts beam delivery to experimental user halls. In this study, we propose the use of a deep learning model to predict slowly developing cavity faults. By utilizing pre-fault signals, we train a long short-term memory-convolutional neural network binary classifier to distinguish between radio-frequency (RF) signals during normal operation and RF signals indicative of impending faults. We optimize the model by adjusting the fault confidence threshold and implementing a multiple consecutive window criterion to identify fault events, ensuring a low false positive rate. Results obtained from analysis of a real dataset collected from the accelerating cavities simulating a deployed scenario demonstrate the model’s ability to identify normal signals with 99.99% accuracy and correctly predict 80% of slowly developing faults. Notably, these achievements were achieved in the context of a highly imbalanced dataset, and fault predictions were made several hundred milliseconds before the onset of the fault. Anticipating faults enables preemptive measures to improve operational efficiency by preventing or mitigating their occurrence.
43 PARTICLE ACCELERATORS↗
Next-generation large-scale binary protein interaction network for Drosophila melanogaster
Generating reference maps of interactome networks illuminates genetic studies by providing a protein-centric approach to finding new components of existing pathways, complexes, and processes. We apply state-of-the-art methods to identify binary protein-protein interactions (PPIs) for Drosophila melanogaster. Four all-by-all yeast two-hybrid (Y2H) screens of > 10,000 Drosophila proteins result in the ‘FlyBi’ dataset of 8723 PPIs among 2939 proteins. Testing subsets of data from FlyBi and previous PPI studies using an orthogonal assay allows for normalization of data quality; subsequent integration of FlyBi and previous data results in an expanded binary Drosophila reference interaction network, DroRI, comprising 17,232 interactions among 6511 proteins. We use FlyBi data to generate an autophagy network, then validate in vivo using autophagy-related assays. The deformed wings (dwg) gene encodes a protein that is both a regulator and a target of autophagy. Altogether, these resources provide a foundation for building new hypotheses regarding protein networks and function.
59 BASIC BIOLOGICAL SCIENCES↗
Compact Ferroelectric Programmable Majority Gate for Compute-in-Memory Applications
In this study, a compact and novel ferroelectric (FE) programmable majority gate is proposed and its novel application in Binary Neural Network (BNNs) is investigated. We demonstrate: i) by integrating N metal-ferroelectric-metal (MFM) capacitors on the gate of a transistor (1T-N-MFM structure), a nonvolatile and programmable majority (MAJ) gate that performs MAJ of AND between the gate input and polarization is realized; ii) validation the functionality of our 3-input MAJ of AND gate through comprehensive theoretical and experimental investigations; iii) a compact implementation of 3-input MAJ of XNOR gate that leverages only five of our 3-input MAJ of AND gates connected in parallel; iv) application of MAJ of XNOR gates to replace the XNOR gates and the first layer of the adder tree in the BNNs for up to 21x area saving on top of eliminating the energy-hungry memory accesses due to the compute-in-memory nature.
97 MATHEMATICS AND COMPUTING↗
Uncontrolled Learning: Codesign of Neuromorphic Hardware Topology for Neuromorphic Algorithms
Neuromorphic computing has the potential to revolutionize future technologies and our understanding of intelligence, yet it remains challenging to realize in practice. The learning-from-mistakes algorithm, inspired by the brain's simple learning rules of inhibition and pruning, is one of the few brain-like training methods. This algorithm is implemented in neuromorphic memristive hardware through a codesign process that evaluates essential hardware trade-offs. While the algorithm effectively trains small networks as binary classifiers and perceptrons, performance declines significantly with increasing network size unless the hardware is tailored to the algorithm. This work investigates the trade-offs between depth, controllability, and capacity—the number of learnable patterns—in neuromorphic hardware. This highlights the importance of topology and governing equations, providing theoretical tools to evaluate a device's computational capacity based on its measurements and circuit structure. The findings show that breaking neural network symmetry enhances both controllability and capacity. Additionally, by pruning the circuit, neuromorphic algorithms in all-memristive circuits can utilize stochastic resources to create local contrasts in network weights. Through combined experimental and simulation efforts, the parameters are identified that enable networks to exhibit emergent intelligence from simple rules, advancing the potential of neuromorphic computing.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Simulations of Glass Transition and Mechanical Behavior of Off-Stoichiometric Crosslinked Polymers
Here, this work explores the influence of blend composition, network architecture, and hydrogen bonding on the material properties of crosslinked epoxy networks, focusing on the glass transition temperature (T g ) and Young’s modulus (Y). We used coarse-grained molecular dynamics simulations to simulate varying compositions of stiff and flexible components in epoxy monomer blends with varying excess of curative. We find that, without hydrogen bonding, networks of any composition show a monotonically increasing T g with decreasing excess curative, consistent with theory. In contrast, we find that when hydrogen bonding is introduced, the binary blend networks show significant enhancement in T g for lightly crosslinked systems. This result contributes to an explanation of the anomalous T g behavior observed experimentally in these systems. We further find that Y is generally enhanced by hydrogen bonds, especially below T g , demonstrating that hydrogen bonding has a significant influence on mechanical properties and can allow access to other desirable dynamic behavior, especially self-healing.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Quantum-Inspired Bayesian Sampling for Uncertainty Quantification and Machine Learning (Final Technical Report)
With increasing simulation and measurement data, machine learning and artificial intelligence have been widely used in computational decision-making of complex engineering systems. The resulting tools, such as uncertainty quantification solvers, reinforcement learning, and physics-informed machine learning, have achieved great success in critical DOE tasks such as material discovery and design, energy system modeling and control, and numerical weather and climate prediction. A core topic in scientific machine learning and artificial intelligence is Bayesian inference: given an observed data set, people want to estimate the posterior distribution of a (possibly large) number of hidden parameters. Due to the flexibility and weak assumptions, Bayesian sampling has been the mainstream Bayesian inference solvers despite the rapid progress of approximate Bayesian inference. Classical Bayesian sampling methods such as Markov-chain Monte Carlo suffer from a low-acceptance rate due to the random walk nature, therefore state-of-the-art techniques use Hamiltonian Monte Carlo and its variants to efficiently draw posterior samples in a high dimension. The key idea of Hamiltonian Monte Carlo and its variants is to simulate the Hamiltonian dynamics of a classical particle with a fixed mass, and their performance significantly degrades when the posterior distribution is highly spiky or has multiple modes. Leveraging the idea of quantum physics, this project has investigated new theory, algorithms and applications of Bayesian inference (especially Bayesian sampling). The main results include: (1) novel quantum-inspired Bayesian sampling methods that can lead to better accuracy for challenging multi-modal or spiky distributions, (2) more scalable machine learning framework leveraging tensor-compressed Bayesian inference, and (3) Bayesian and sampling approaches for verifying the robustness of continuous and binary neural networks.
97 MATHEMATICS AND COMPUTING↗
Effect of rare earth size on network structure and glass forming ability in binary aluminum garnets
Rare earth aluminate glasses are potentially useful for optical, luminescence, and laser applications. As reluctant glass formers, these materials exhibit unconventional atomic structures. To better understand how their structures correlate with glass formation, we investigate two rare earth aluminum garnet melts, La 3 Al 5 O 12 (LAG) and Yb 3 Al 5 O 12 (YbAG), which represent the relative extremes of good and poor glass forming ability in rare earth aluminates. Structural models have been refined to high-energy X-ray diffraction data over 1340–2740 K. Both melts contain mixtures of AlO 4 , AlO 5 , and AlO 6 polyhedra, with larger fractions of [5] Al and [6] Al in YbAG. Extrapolation of the Al–O coordination distributions to the glass transition match closely with 27 Al nuclear magnetic resonance measurements of (La 1−z Y z ) 3 Al 5 O 12 glasses, z = 0 to 1. During cooling, the mean coordination numbers increase for La–O in LAG from 6.45(8) to 6.98(8) and for Yb–O in YbAG from 6.02(8) to 6.21(8). Linkedness among Al–O polyhedra at ∼2450 K is mostly corner-sharing, with 9% edge-sharing in LAG and 19% in YbAG. Among [4] Al units, both melts have 6% edge-sharing that convert to all corner-sharing upon cooling. Network connectivity is compared using a newly defined metric, K n , that is similar to the Q n distribution but that accounts for the edge-sharing and triply bonded oxygen present in these melts. The lower glass forming ability in YbAG as compared to LAG correlates with more edge-sharing, associated with the larger fractions of [5] Al and [6] Al, and lower connectivity among [4] Al units.
Wilke, Stephen K. [Materials Development, Inc., Ar↗
CMOS-Based Single-Cycle in-Memory XOR/XNOR
Big data applications are on the rise, and so is the number of data centers. The ever-increasing massive data pool needs to be periodically backed up in a secure environment. Moreover, a massive amount of securely backed-up data is required for training binary convolutional neural networks for image classification. XOR and XNOR operations are essential for large-scale data copy verification, encryption, and classification algorithms. The disproportionate speed of existing compute and memory units makes the von Neumann architecture inefficient to perform these Boolean operations. Compute-in-memory (CiM) has proved to be an optimum approach for such bulk computations. The existing CiM-based XOR/XNOR techniques either require multiple cycles for computing or add to the complexity of the fabrication process. Here, we propose a CMOS-based hardware topology for single-cycle in-memory XOR/XNOR operations. Our design provides at least 2× improvement in the latency compared with other existing CMOS-compatible solutions. We verify the proposed system through circuit/system-level simulations and evaluate its robustness using a 5000-point Monte Carlo variation analysis. This all-CMOS design paves the way for practical implementation of CiM XOR/XNOR at scaled technology nodes.
97 MATHEMATICS AND COMPUTING↗
Spiking Markov Reward Process v.0.1
SAND2024-11150O The Spiking Markov Reward Process software is a spiking neural network that streams binary arithmetic and computes the state value function of a Markov reward process. The software will be released to the SpiNNcloud group for development of neuromorphic acceleration. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
Wang, Felix↗
Machine Learning-Assisted Distribution System Network Reconfiguration Problem
High penetration from volatile renewable energy resources in the grid and the varying nature of loads raise the need for frequent line switching to ensure the efficient operation of electrical distribution networks. Operators must ensure maximum load delivery, reduced losses, and the operation between voltage limits. However, computations to decide the optimal feeder configuration are often computationally expensive and intractable, making it unfavorable for real-time operations. This is mainly due to the existence of binary variables in the network reconfiguration optimization problem. To tackle this issue, we have devised an approach that leverages machine learning techniques to reshape distribution networks featuring multiple substations. This involves predicting the substation responsible for serving each part of the network. Hence, it leaves simple and more tractable Optimal Power Flow problems to be solved. This method can produce accurate results in a significantly faster time, as demonstrated using the IEEE 37-bus distribution feeder. Compared to the traditional optimization-based approaches, a feasible solution is achieved approximately ten times faster for all the tested scenarios.
deep neural networks↗
Sharing leaky-integrate-and-fire neurons for memory-efficient spiking neural networks
Spiking Neural Networks (SNNs) have gained increasing attention as energy-efficient neural networks owing to their binary and asynchronous computation. However, their non-linear activation, that is Leaky-Integrate-and-Fire (LIF) neuron, requires additional memory to store a membrane voltage to capture the temporal dynamics of spikes. Although the required memory cost for LIF neurons significantly increases as the input dimension goes larger, a technique to reduce memory for LIF neurons has not been explored so far. To address this, we propose a simple and effective solution, EfficientLIF-Net, which shares the LIF neurons across different layers and channels. Our EfficientLIF-Net achieves comparable accuracy with the standard SNNs while bringing up to ~4.3× forward memory efficiency and ~21.9× backward memory efficiency for LIF neurons. We conduct experiments on various datasets including CIFAR10, CIFAR100, TinyImageNet, ImageNet-100, and N-Caltech101. Furthermore, we show that our approach also offers advantages on Human Activity Recognition (HAR) datasets, which heavily rely on temporal information. The code has been released at https://github.com/Intelligent-Computing-Lab-Yale/EfficientLIF-Net.
60 APPLIED LIFE SCIENCES↗
Accurate and Data‐Efficient Micro X‐ray Diffraction Phase Identification Using Multitask Learning: Application to Hydrothermal Fluids
Traditional analysis of highly distorted micro X‐ray diffraction (μ‐XRD) patterns from hydrothermal fluid environments is a time‐consuming process, often requiring substantial data preprocessing and labeled experimental data. Herein, the potential of deep learning with a multitask learning (MTL) architecture to overcome these limitations is demonstrated. MTL models are trained to identify phase information in μ‐XRD patterns, minimizing the need for labeled experimental data and masking preprocessing steps. Notably, MTL models show superior accuracy compared to binary classification convolutional neural networks. Additionally, introducing a tailored cross‐entropy loss function improves MTL model performance. Most significantly, MTL models tuned to analyze raw and unmasked XRD patterns achieve close performance to models analyzing preprocessed data, with minimal accuracy differences. This work indicates that advanced deep learning architectures like MTL can automate arduous data handling tasks, streamline the analysis of distorted XRD patterns, and reduce the reliance on labor‐intensive experimental datasets.
97 MATHEMATICS AND COMPUTING↗
Characterizing Quantum Classifier Utility in Natural Language Processing Workflows
Quantum Natural Language Processing (QNLP) develops natural language processing (NLP) models for deployment on quantum computers. We explore feature and data prototype selection techniques to address challenges posed by encoding high dimensional features. Our study builds quantum circuit classifiers that includes classical feature pre-processing, quantum embedding and quantum model training. The quantum models are built on 4 or 6 qubits and the quantum neural network (QNN) uses the established bricklayer design. We compare the dependence of model performance (in terms of accuracy and F1 scores) on feature length, embedding gates and parameterized unitary design. We compare the performance of quantum machine learning models to classical convolution neural network model (CNN) on binary and multi-class classification tasks using two datasets of synthetic features and labels. The first is the ECP-CANDLE P3B3 dataset a corpus of synthetically generated cancer pathology reports. The second dataset is extracted from well-known benchmark dataset (MADELON) - features are generated with a combination of informative, repeated and uninformative features. Both datasets are used for binary classification and multi-class classification with 3 classes. We observe robust, accurate performance from all models on the binary classification tasks, but multiclass classification is a challenge for the quantum models-there is a notable decrease in accuracy when using 3 classes. Overall the performance is comparable in terms of recall and accuracy between QNNs and CNNs, even with large datasets. These results provide a point of comparison between quantum and classical models on real-world datasets.
Hamilton, Kathleen↗