Search NASA⌕ Search

SEARCH · Search NASA

Results for “feed forward”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Benchmarking the Performance of Neuromorphic and Spiking Neural Network Simulators

Software simulators play a critical role in the development of new algorithms and system architectures in any field of engineering. Neuromorphic computing, which has shown potential in building brain-inspired energy-efficient hardware, suffers a slow-down in the development cycle due to a lack of flexible and easy-to-use simulators of either neuromorphic hardware itself or of spiking neural networks (SNNs), the type of neural network computation executed on most neuromorphic systems. While there are several openly available neuromorphic or SNN simulation packages developed by a variety of research groups, they have mostly targeted computational neuroscience simulations, and only a few have targeted small-scale machine learning tasks with SNNs. Evaluations or comparisons of these simulators have often targeted computational neuroscience-style workloads. In this work, we seek to evaluate the performance of several publicly available SNN simulators with respect to non-computational neuroscience workloads, in terms of speed, flexibility, and scalability. We evaluate the performance of the NEST, Brian2, Brian2GeNN, BindsNET and Nengo packages under a common front-end neuromorphic framework. Our evaluation tasks include a variety of different network architectures and workload types to mimic the computation common in different algorithms, including feed-forward network inference, genetic algorithms, and reservoir computing. We also study the scalability of each of these simulators when running on different computing hardware, from single core CPU workstations to multi-node supercomputers. Our results show that the BindsNET simulator has the best speed and scalability for most of the SNN workloads (sparse, dense, and layered SNN architectures) on a single core CPU. However, when comparing the simulators leveraging the GPU capabilities, Brian2GeNN outperforms the others for these workloads in terms of scalability. NEST performs the best for small sparse networks and is also the most flexible simulator in terms of reconfiguration capability NEST shows a speedup of at least 2x compared to the other packages when running evolutionary algorithms for SNNs. The multi-node and multi-thread capabilities of NEST show at least 2x speedup compared to the rest of the simulators (single core CPU or GPU based simulators) for large and sparse networks. We conclude our work by providing a set of recommendations on the suitability of employing these simulators for different tasks and scales of operations. We also present the characteristics for a future generic ideal SNN simulator for different neuromorphic computing workloads.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Effects of a 4.3T superconducting wiggler on the electron beam optics and dynamics at NSLS-II

A superconducting wiggler (SCW) with a peak field of 4.3 T has recently been installed at NSLS-II. The wiggler generates a high-flux X-ray beam with a photon energy range from 20 keV to 200 keV for the High Energy Engineering X-ray Scattering beamline. This device induces significant distortions in the electron beam orbit and optics. Here, we have corrected these adverse effects using feed-forward tables. The effects of the SCW on the electron beam emittance, energy spread, and bunch length were calculated and measured. The beam-induced heating of the cryogenic vacuum chamber was also studied. Finally, the superconducting wiggler has been commissioned for user operations.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Deep spectral CNN for laser induced breakdown spectroscopy

This work proposes a spectral convolutional neural network (CNN) operating on laser induced breakdown spectroscopy (LIBS) signals to learn to (1) disentangle spectral signals from the sources of sensor uncertainty (i.e., pre-process) and (2) get qualitative and quantitative measures of chemical content of a sample given a spectral signal (i.e., calibrate). Once the spectral CNN is trained, it can accomplish either task through a single feed-forward pass, with real-time benefits and without any additional side information requirements including dark current, system response, temperature and detector-to-target range. Our experiments demonstrate that the proposed method outperforms the existing approaches used by the Mars Science Lab for pre-processing and calibration for remote sensing observations from the Mars rover, ‘Curiosity’.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Scalable algorithms for physics-informed neural and graph networks

Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networks from additional information obtained by employing the physical laws and evaluating them at random points in the space–time domain. Such PIML integrates multimodality and multifidelity data with mathematical models, and implements them using neural networks or graph networks. Here, we review some of the prevailing trends in embedding physics into machine learning, using physics-informed neural networks (PINNs) based primarily on feed-forward neural networks and automatic differentiation. For more complex systems or systems of systems and unstructured data, graph neural networks (GNNs) present some distinct advantages, and here we review how physics-informed learning can be accomplished with GNNs based on graph exterior calculus to construct differential operators; we refer to these architectures as physics-informed graph networks (PIGNs). We present representative examples for both forward and inverse problems and discuss what advances are needed to scale up PINNs, PIGNs and more broadly GNNs for large-scale engineering problems.

42 ENGINEERING↗
DOE OSTI2025

Multioutput Convolutional Neural Network for Improved Parameter Extraction in Time-Resolved Electrostatic Force Microscopy Data

Time-resolved scanning probe microscopy methods, like time-resolved electrostatic force microscopy (trEFM), enable imaging of dynamic processes ranging from ion motion in batteries to electronic dynamics in microstructured thin film semiconductors for solar cells. Reconstructing the underlying physical dynamics from these techniques can be challenging due to the interplay of cantilever physics with the actual transient kinetics of interest in the resulting signal. Previously, quantitative trEFM used empirical calibration of the cantilever or feed-forward neural networks trained on simulated data to extract the physical dynamics of interest. Both these approaches are limited by interpreting the underlying signal as a single exponential function, which serves as an approximation but does not adequately reflect many realistic systems. Here, we present a multi-branched, multi-output convolutional neural network (CNN) that uses the trEFM signal in addition to the physical cantilever parameters as input. The trained CNN accurately extracts parameters describing both single-exponential and bi-exponential underlying functions, and more accurately reconstructs real experimental data in the presence of noise. This article demonstrates an application of physics-informed machine learning to complex signal processing tasks, enabling more efficient and accurate analysis of trEFM.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Stability Analysis of Substituted Cobaltocenium [Bis(cyclopentadienyl)cobalt(III)] Employing Chemistry-Informed Neural Networks

Cobaltocenium derivatives are promising components of the anion exchange membranes due to their excellent thermal and alkaline stability under the operating conditions of a fuel cell. Here we present an efficient modeling approach of assessing the chemical stability of substituted cobaltocenium CoCp 2 + based on the computed electronic structure enhanced by machine learning techniques. Within the aqueous environment, the positive charge of the metal cation is balanced by the hydroxide anion through formation of the CoCp 2 + OH¯ complexes, whose dissociation is studied within the implicit solvent employing density functional theory. The data set of about 118 species based on 42 substituent groups characterized by a range of electron- donating (ED) and electron-withdrawing (EW) properties is constructed and analyzed. Given 12 carefully chosen chemistry-informed descriptors of the complexes and relevant fragments, the stability of the complexes is found to strongly correlate with the energies of the highest occupied and lowest unoccupied molecular orbitals, modulated by a switching function of the Hirshfeld charge. The latter is used as a measure of the electron withdrawing-donating character of the substituents. Based on this observation from the conventional regression analysis, two fully connected, feed-forward neural network (FNN) models with different unit structures, called the chemistry-informed (CINN) and the quadratic (QNN) neural networks, are developed. Both models predict the bond dissociation energies of the cobaltocenium complexes with mean relative errors less than 5.40% and average absolute errors less than 0.94 kcal/mol. The results show the potential of QNN to efficiently capture more complex relationships. Here, the concept of incorporating the domain (chemical) knowledge/insight into the neural network structure paves the way to applications of machine learning techniques with small data sets, ultimately leading to better predictive models compared to the conventional regression analysis.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Accurate Machine Learning for Predicting the Viscosities of Deep Eutectic Solvents

Deep eutectic solvents (DESs) are emerging as environmentally friendly designer solvents for mass transport and heat transfer processes in industrial applications; however, the lack of accurate tools to predict and thus control their viscosities under both a range of environmental factors and formulations hinders their general application. While DESs may serve as designer solvents, with nearly unlimited combinations, this unfortunately makes it experimentally infeasible to comprehensively measure the viscosities of all DESs of potential industrial interest. To assist in the design of DESs, we have developed several new machine learning (ML) models that accurately and rapidly predict the viscosities of a diverse group of DESs at different temperatures and molar ratios using, to date, one of the most comprehensive data sets containing the properties of over 670 DESs over a wide range of temperatures (278.15–385.25 K). Three ML models, including support vector regression (SVR), feed forward neural networks (FFNNs), and categorical boosting (CatBoost), were developed to predict DES viscosity as a function of temperature and molar ratio and contrasted with multilinear and two-factor polynomial regression baselines. Further, quantum chemistry-based, COSMO-RS-derived sigma profile (σ-profile) features were used as inputs for the ML models. The CatBoost model is excellent at externally predicting DES viscosity, as indicated by high R 2 (0.99) and low root-mean-square-error (RMSE) and average absolute relative deviations (AARD) (5.22%) values for the testing data sets, and 98% of the data points lie within the 15% of AARD deviations. Furthermore, SHapley additive explanation (SHAP) analysis was employed to interpret the ML results and rationalize the viscosity predictions. The result is an ML approach that accurately predicts viscosity and will aid in accelerating the design of appropriate DESs for industrial applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Delineating Variabilities of Groundwater Level Prediction Across the Agriculturally Intensive Transboundary Aquifers of South Asia

We report the groundwater depletion in South Asia’s Himalayan, transboundary Indus-Ganges-Brahmaputra-Meghna (IGBM) rivers basin is among the highest globally. Given the high irrigation demand and population, groundwater sustainability requires an improved understanding of groundwater systems for the accurate prediction of groundwater levels (GWLs). However, the prediction of groundwater system behaviors is a significant challenge since it is dominated by spatiotemporal and subsurface depth-dependent drivers. Earlier studies that address the challenges are mainly based on the short spatial and temporal extent and/or do not separate the renewable (i.e., shallow) vs nonrenewable (i.e., deeper) groundwater signals. Here, we first identified the variable importance of spatial and depth-dependent drivers on GWL in the IGBM basin. Our results indicate a greater influence of anthropogenic factors (i.e., widespread pumping and increased population) in most parts of the IGBM basin, except in the precipitation-dominated basin of the Brahmaputra. Our next purpose was to delineate a multifactorial approach for GWL prediction using the two most used machine learning models (i.e., support vector machine and feed-forward neural network) in the literature. In general, the machine learning model outputs show a good match in comparison to the GWL from the observation wells (n = 2303 distributed across India and Bangladesh) with some limitations in areas with increased groundwater irrigation. We separately compared the results from shallow (<35 m) and deep (>35 m) observation wells, emphasizing the significance of deep groundwater pumping. Our approach highlights the importance of spatiotemporal to multidepth factors in GWL prediction and can be adopted in other parts of the globe to predict GWLs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

High-Throughput Screening and Accurate Prediction of Ionic Liquid Viscosities Using Interpretable Machine Learning

Ionic liquids (ILs) are a novel group of green solvents with great promise for various industrial applications, including carbon capture and lignocellulosic biomass deconstruction. However, the use of ILs at the industrial scale remains challenging due to their high viscosities at ambient temperatures. To develop ILs with lower viscosities, a systematic study of their quantitative structure–property relationship (QSPR) is desirable. Here, we developed four machine learning (ML) models to predict viscosity at various temperature and pressure ranges, trained over a wide range of ILs consisting of various cationic and anionic families. ML methods including two-factor polynomial regression (two-factor PR), support vector regression (SVR), feed-forward neural networks (FFNN), and categorical boosting (CATBoost) were developed based on features that have proven useful in previous ML studies: COSMO-RS (conductor-like screening model for real solvents)-derived surface screening charge densities (sigma profiles). FFNN and CATBoost were the most accurate in predicting IL viscosities with lower average absolute relative deviation and higher R2 values on the test set. Tanimoto similarity scores were calculated to characterize the chemical space and structural similarity of the investigated ions. Furthermore, SHapley Additive exPlanation (SHAP) analysis was employed to interpret the ML results. Temperature, the polar area of ILs, and the nonpolar regions of ions are key features that influence the viscosity predictions. Importantly, the IL viscosity prediction here is the most accurate reported to date.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Event‐Based Training in Label‐Limited Regimes

Abstract The distribution of attributes assigned using data on independent sensors for a specific source, for example, magnitude, can be richly descriptive for final event characterization and associated uncertainty. Attribute distributions can also provide powerful context for event characterization in the absence of comprehensive annotation. This work develops a way to leverage distributional information across a set of sensors in the absence of comprehensive annotation as a domain‐informed regularization term applied during gradient‐based learning. The regularization term is the basis of event‐based training which I show can be a powerful semi‐supervised learning (SSL) approach. I first use a simple feed forward neural network and a toy data set to outline how data set structure interacts with the assumptions inherent to many semi‐supervised learning approaches. I then demonstrate the effectiveness of event‐based training using a deep convolutional neural network for seismic event classification in Utah, which increases SSL accuracy from 92% to 97% on event classification with a limited number of training labels.

Linville, Lisa M.↗
DOE OSTI2022

Non‐Linear Dimensionality Reduction With a Variational Encoder Decoder to Understand Convective Processes in Climate Models

Abstract Deep learning can accurately represent sub‐grid‐scale convective processes in climate models, learning from high resolution simulations. However, deep learning methods usually lack interpretability due to large internal dimensionality, resulting in reduced trustworthiness in these methods. Here, we use Variational Encoder Decoder structures (VED), a non‐linear dimensionality reduction technique, to learn and understand convective processes in an aquaplanet superparameterized climate model simulation, where deep convective processes are simulated explicitly. We show that similar to previous deep learning studies based on feed‐forward neural nets, the VED is capable of learning and accurately reproducing convective processes. In contrast to past work, we show this can be achieved by compressing the original information into only five latent nodes. As a result, the VED can be used to understand convective processes and delineate modes of convection through the exploration of its latent dimensions. A close investigation of the latent space enables the identification of different convective regimes: (a) stable conditions are clearly distinguished from deep convection with low outgoing longwave radiation and strong precipitation; (b) high optically thin cirrus‐like clouds are separated from low optically thick cumulus clouds; and (c) shallow convective processes are associated with large‐scale moisture content and surface diabatic heating. Our results demonstrate that VEDs can accurately represent convective processes in climate models, while enabling interpretability and better understanding of sub‐grid‐scale physical processes, paving the way to increasingly interpretable machine learning parameterizations with promising generative properties.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Machine learning-accelerated discovery of heat-resistant polysulfates for electrostatic energy storage

The development of heat-resistant dielectric polymers that withstand intense electric fields at high temperatures is critical for electrification. Balancing thermal stability and electrical insulation, however, is exceptionally challenging as these properties are often inversely correlated. A traditional intuition-driven polymer design approach results in a slow discovery loop that limits breakthroughs. Here we present a machine learning-driven strategy to rapidly identify high-performance, heat-resistant polymers. A trustworthy feed-forward neural network is trained to predict key proxy parameters and down select polymer candidates from a library of nearly 50,000 polysulfates. The highly efficient and modular sulfur fluoride exchange click chemistry enables successful synthesis and validation of selected candidates. A polysulfate featuring a 9,9-di(naphthalene)-fluorene repeat unit exhibits excellent thermal resilience and achieves ultrahigh discharged energy density with over 90% efficiency at 200 °C. Its exceptional cycling stability underscores its promise for applications in demanding electrified environments.

Li, He↗
DOE OSTI2023

Predictive understanding of the surface tension and velocity of sound in ionic liquids using machine learning

Knowledge of the physical properties of ionic liquids (ILs), such as the surface tension and speed of sound, is important for both industrial and research applications. Unfortunately, technical challenges and costs limit exhaustive experimental screening efforts of ILs for these critical properties. Previous work has demonstrated that the use of quantum-mechanics-based thermochemical property prediction tools, such as the conductor-like screening model for real solvents, when combined with machine learning (ML) approaches, may provide an alternative pathway to guide the rapid screening and design of ILs for desired physiochemical properties. However, the question of which machine-learning approaches are most appropriate remains. In the present study, we examine how different ML architectures, ranging from tree-based approaches to feed-forward artificial neural networks, perform in generating nonlinear multivariate quantitative structure–property relationship models for the prediction of the temperature- and pressure-dependent surface tension of and speed of sound in ILs over a wide range of surface tensions (16.9–76.2 mN/m) and speeds of sound (1009.7–1992 m/s). The ML models are further interrogated using the powerful interpretation method, shapley additive explanations. We find that several different ML models provide high accuracy, according to traditional statistical metrics. The decision tree-based approaches appear to be the most accurate and precise, with extreme gradient-boosting trees and gradient-boosting trees being the best performers. However, our results also indicate that the promise of using machine-learning to gain deep insights into the underlying physics driving structure–property relationships in ILs may still be somewhat premature.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Emulation of the calculations of final r -process abundance patterns with a neural network

This work explores the construction of a fast emulator for the calculation of the final pattern of nucleosynthesis in the rapid neutron capture process (the r-process). An emulator is built using a feed-forward artificial neural network (ANN). We train the ANN with nuclear data and relative abundance patterns. We take as input the β-decay half-lives and the one-neutron separation energy of the nuclei in the rare-earth region. The output is the final isotopic abundance pattern. In this work, we focus on the nuclear data and abundance patterns in the rare-earth region to reduce the dimension of the input and output space. We show that the ANN can capture the effect of the changes in the nuclear physics inputs on the final r-process abundance pattern in the adopted astrophysical conditions. We employ the deep ensemble method to quantify the prediction uncertainty of the neural network emulator. The emulator achieves a speed-up by a factor of about 20 000 in obtaining a final abundance pattern in the rare-earth region. The emulator may be utilized in statistical analyses such as uncertainty quantification, inverse problems, and sensitivity analysis.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

A Low-Power 1 Gb/s Line Driver with Configurable Pre-Emphasis for Lossy Transmission Lines

A line driver with configurable pre-emphasis is implemented in a 65 nm CMOS process. The driver utilizes a three-tap feed-forward equalization (FFE) architecture. The relative delays between the taps are selectable in increments of 1/16th of the unit interval (UI) via an 8-stage delay-locked loop (DLL) and digital interpolator. It is also possible to control the output amplitude and source impedance for each tap via a programmable array of eight source-series terminated (SST) drivers. The entire design consumes 9 mW from a 1.2 V supply at 1 Gb/s.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Reducing uncertainty of high-latitude ecosystem models through identification of key parameters

Abstract Climate change is having significant impacts on Earth’s ecosystems and carbon budgets, and in the Arctic may drive a shift from an historic carbon sink to a source. Large uncertainties in terrestrial biosphere models (TBMs) used to forecast Arctic changes demonstrate the challenges of determining the timing and extent of this possible switch. This spread in model predictions can limit the ability of TBMs to guide management and policy decisions. One of the most influential sources of model uncertainty is model parameterization. Parameter uncertainty results in part from a mismatch between available data in databases and model needs. We identify that mismatch for three TBMs, DVM-DOS-TEM, SIPNET and ED2, and four databases with information on Arctic and boreal above- and belowground traits that may be applied to model parametrization. However, focusing solely on such data gaps can introduce biases towards simple models and ignores structural model uncertainty, another main source for model uncertainty. Therefore, we develop a causal loop diagram (CLD) of the Arctic and boreal ecosystem that includes unquantified, and thus unmodeled, processes. We map model parameters to processes in the CLD and assess parameter vulnerability via the internal network structure. One important substructure, feed forward loops (FFLs), describe processes that are linked both directly and indirectly. When the model parameters are data-informed, these indirect processes might be implicitly included in the model, but if not, they have the potential to introduce significant model uncertainty. We find that the parameters describing the impact of local temperature on microbial activity are associated with a particularly high number of FFLs but are not constrained well by existing data. By employing ecological models of varying complexity, databases, and network methods, we identify the key parameters responsible for limited model accuracy. They should be prioritized for future data sampling to reduce model uncertainty.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Lightweight jet reconstruction and identification as an object detection task

We apply object detection techniques based on deep convolutional blocks to end-to-end jet identification and reconstruction tasks encountered at the CERN large hadron collider (LHC). Collision events produced at the LHC and represented as an image composed of calorimeter and tracker cells are given as an input to a Single Shot Detection network. The algorithm, named PFJet-SSD performs simultaneous localization, classification and regression tasks to cluster jets and reconstruct their features. This all-in-one single feed-forward pass gives advantages in terms of execution time and an improved accuracy w.r.t. traditional rule-based methods. A further gain is obtained from network slimming, homogeneous quantization, and optimized runtime for meeting memory and latency constraints of a typical real-time processing environment. We experiment with 8-bit and ternary quantization, benchmarking their accuracy and inference latency against a single-precision floating-point. We show that the ternary network closely matches the performance of its full-precision equivalent and outperforms the state-of-the-art rule-based algorithm. Finally, we report the inference latency on different hardware platforms and discuss future applications.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Many-body expansion based machine learning models for octahedral transition metal complexes

Abstract Graph-based machine learning (ML) models for material properties show great potential to accelerate virtual high-throughput screening of large chemical spaces. However, in their simplest forms, graph-based models do not include any 3D information and are unable to distinguish stereoisomers such as those arising from different orderings of ligands around a metal center in coordination complexes. In this work we present a modification to revised autocorrelation descriptors, a molecular graph featurization method, for predicting spin state dependent properties of octahedral transition metal complexes (TMCs). Inspired by analytical semi-empirical models for TMCs, the new modeling strategy is based on the many-body expansion (MBE) and allows one to tune the captured stereoisomer information by changing the truncation order of the MBE. We present the necessary modifications to include this approach in two commonly used ML methods, kernel ridge regression and feed-forward neural networks. On a test set composed of all possible isomers of binary TMCs, the best MBE models achieve mean absolute errors (MAEs) of 2.75 kcal mol −1 on spin-splitting energies and 0.26 eV on frontier orbital energy gaps, a 30%–40% reduction in error compared to models based on our previous approach. We also observe improved generalization to previously unseen ligands where the best-performing models exhibit MAEs of 4.00 kcal mol −1 (i.e. a 0.73 kcal mol −1 reduction) on the spin-splitting energies and 0.53 eV (i.e. a 0.10 eV reduction) on the frontier orbital energy gaps. Because the new approach incorporates insights from electronic structure theory, such as ligand additivity relationships, these models exhibit systematic generalization from homoleptic to heteroleptic complexes, allowing for efficient screening of TMC search spaces.

Meyer, Ralf (ORCID:0000000322360261)↗