Search NASA⌕ Search

SEARCH · Search NASA

Results for “Variational neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Using Machine Learning Tools to Predict Compressor Stall

Clean energy has become an increasingly important consideration in today’s power systems. As the push for clean energy continues, many coal-fired power plants are being decommissioned in favor of renewable power sources such as wind and solar. However, the intermittent nature of renewables means that dynamic load following traditional power systems is crucial to grid stability. With high flexibility and fast response at a wide range of operating conditions, gas turbine systems are poised to become the main load following component in the power grid. Yet, rapid changes in load can lead to fluid flow instabilities in gas turbine power systems. These instabilities often lead to compressor surge and stall, which are some of the most critical problems facing the safe and efficient operation of compressors in turbomachinery today. Although the topic of compressor surge and stall has been extensively researched, no methods for early prediction have been proven effective. This study explores the utilization of machine learning tools to predict compressor stall. The long short-term memory (LSTM) model, a form of recurrent neural network (RNN), was trained using real compressor stall datasets from a 100 kW recuperated gas turbine power system designed for hybrid configuration. Two variations of the LSTM model, classification and regression, were tested to determine optimal performance. The regression scheme was determined to be the most accurate approach, and a tool for predicting compressor stall was developed using this configuration. Overall, results show that the tool is capable of predicting stalls 5–20 ms before they occur. With a high-speed controller capable of 5 ms time-steps, mitigating action could be taken to prevent compressor stall before it occurs.

42 ENGINEERING↗
DOE OSTI2022

Use of Event-Time Embeddings via RNN to Discern Novel Event Sequences in EHRs

In highly configurable health information technology (HIT) systems, such as VistA of the Veterans Health Administration, the variations in how the system is used among different healthcare facilities and how the data are recorded can be significant. Despite the successful standardization of care efforts, some of these variations can be indicative of HIT hazards and demand further investigation. In this work, we implemented a recurrent neural network (RNN) architecture to learn clinical provider order sequences and their temporal dynamics while predicting the orders' terminal state. We demonstrate model performance and provide a use case for the model discerning novel event sequences. This model is proposed to find novel event sequences in an operational environment.

Ozmen, Ozgur↗
DOE OSTI2023

Quantum-inspired tempering for ground state approximation using artificial neural networks

A large body of work has demonstrated that parameterized artificial neural networks (ANNs) can efficiently describe ground states of numerous interesting quantum many-body Hamiltonians. However, the standard variational algorithms used to update or train the ANN parameters can get trapped in local minima, especially for frustrated systems and even if the representation is sufficiently expressive. We propose a parallel tempering method that facilitates escape from such local minima. This methods involves training multiple ANNs independently, with each simulation governed by a Hamiltonian with a different "driver" strength, in analogy to quantum parallel tempering, and it incorporates an update step into the training that allows for the exchange of neighboring ANN configurations. We study instances from two classes of Hamiltonians to demonstrate the utility of our approach using Restricted Boltzmann Machines as our parameterized ANN. The first instance is based on a permutation-invariant Hamiltonian whose landscape stymies the standard training algorithm by drawing it increasingly to a false local minimum. The second instance is four hydrogen atoms arranged in a rectangle, which is an instance of the second quantized electronic structure Hamiltonian discretized using Gaussian basis functions. We study this problem in a minimal basis set, which exhibits false minima that can trap the standard variational algorithm despite the problem’s small size. We show that augmenting the training with quantum parallel tempering becomes useful to finding good approximations to the ground states of these problem instances.

Albash, Tameem↗
DOE OSTI2022

Application of Quantum Machine Learning to High Energy Physics Analysis at LHC Using Quantum Computer Simulators and Quantum Computer Hardware

Machine learning enjoys widespread success in High Energy Physics (HEP) analyses at LHC. However the ambitious HL-LHC program will require much more computing resources in the next two decades. Quantum computing may offer speed-up for HEP physics analyses at HL-LHC, and can be a new computational paradigm for big data analyses in High Energy Physics.We have successfully employed three methods (1) Variational Quantum Classifier (VQC) method, (2) Quantum Support Vector Machine Kernel (QSVM-kernel) method and (3) Quantum Neural Network (QNN) method for two LHC flagship analyses: ttH (Higgs production in association with two top quarks) and H->mumu (Higgs decay to two muons, the second generation fermions). We shall address the progressive improvements in performance from method (1) to method (3).We will present our experiences and results of a study on LHC High Energy Physics data analyses with IBM Quantum Simulator and Quantum Hardware (using IBM Qiskit framework), Google Quantum Simulator (using Google Cirq framework), and Amazon Quantum Simulator (using Amazon Braket cloud service). The work is in the context of a Qubit platform (a gate-model quantum computer). Taking into account the present limitation of hardware access, different quantum machine learning methods are studied on simulators and the results are compared with classical machine learning methods (BDT, classical Support Vector Machine and classical Neural Network). Furthermore, we do apply quantum machine learning on IBM quantum hardware to compare performance between quantum simulator and quantum hardware. The work is performed by an international and interdisciplinary collaboration with the Department of Physics and Department of Computer Sciences of University of Wisconsin, CERN Quantum Technology Initiative, IBM Research Zurich, IBM T.J. Watson Research Center, Fermilab Quantum Institute, BNL Computational Science Initiative, State University of New York at Stony Brook, and Quantum Computing and AI Research of Amazon Web Services. This work pioneers a close collaboration of academic institutions with industrial corporations in the High Energy Physics analyses effort. Though the size of event samples in future HL-LHC physics and the limited number of qubits pose some challenges to the Quantum Machine learning studies for High Energy Physics, more advanced quantum computers with larger number of qubits, reduced noise and improved running time (as envisioned by IBM and Google) may outperform classical machine learning in both classification power and in speed.Although the era of efficient quantum computing may still be years away, we have made promising progress and obtained preliminary results in applying quantum machine learning to High Energy Physics. A PROOF OF PRINCIPLE.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRSDate not supplied

An Online Reinforcement Learning Controller Design For Mars Ascent Vehicle

This paper presents a neural network (NN) approximator-based online reinforcement learning (ORL) controller design for Mars Ascent Vehicle (MAV) under parametric variation and significant external disturbances. The ORL controller, which does not require any offline training, involves two NNs where an action NN produces optimal short-term control performance while a critic NN evaluates the performance of the action NN using an approximated cost function. The simulation example with comparisons against baseline Proportional-Integral-Derivative (PID) and gain scheduled pole-placement PID (GS-PP-PID) controllers show the proposed controller’s effectiveness and robustness under parametric variation and high external disturbances.

Han Woong Bae↗
DOE OSTI2024

Errant Beam Prognostics with Machine Leaning at SNS Accelerator

Particle Accelerators are complex machine with many pieces of equipment running in synchronization to deliver required beam. However, faults in particle accelerators reduce the availability of the beam for experiments affecting the overall science output. To avoid these faults, we apply anomaly detection techniques to predict any unusual behavior and perform preemptive actions to improve the total availability. Many researchers have adopted semi-supervised Machine Learning (ML) methods such as auto-encoders and variational auto-encoders for such tasks. However, supervised ML techniques designed for similarity learning such as Siamese Neural Network (SNN) can outperform semi-supervised or unsupervised methods for anomaly prediction. One of the challenges associated with application of ML models to particle accelerators is the variability in observed data over time due to system configuration changes. We employ conditional models such as Conditional Siamese Neural Networks (CSNN), and Conditional-VAE (CVAE) to learn the variability in the data by using beam configuration parameters as conditional input. We apply these models for errant beam prediction at Spallation Neutron Source accelerator under different system configurations and compare their performance. We demonstrate that CSNN outperforms CVAE in our application. This talk will present the data source, collection, analysis, data-preparation, model development, hyper-parameter studies and the results.

Rajput, Kishansingh↗
DOE OSTI2024

Structured Neural Network Modeling for Developing Digital Twins Models of Hydropower Generation Units

Dynamic modeling is a key part in the development of digital twin (DT) for dynamic systems. This is true for hydropower systems, where whole system modeling including penstock, turbine and generators, etc is important in realizing actuate modeling for the real systems. On the other hand, in response to the large variations of the power demand due to increased penetration of renewables such as wind and solar, hydropower systems are now required to operate in a large power generation range. This situation triggers the nonlinear characteristics of the generation unit with respect to its models. As such, it is imperative to use data driven modeling such as neural networks to learn the nonlinear dynamics of the hydropower generation unit. To achieve this objective, this study constructs a modeling and learning algorithm integrated with multiple structured neural network models for the modeling of turbine shaft speed, penstock pressure, and generator power output based on the generator power control setpoint, field current, and field voltage. In addition, the study uses the hydropower data from Tacoma Public Utilities to train and validate the proposed neural network algorithm. The results have shown that this structured neural network modeling approach can learn the system dynamics effectively by using the real-time data collected from the hydropower system with the desired modeling results.

Wang, Hong↗
DOE OSTI2022

Using neural network ensembles to separate ocean biogeochemical and physical drivers of phytoplankton biogeography in Earth system models

Abstract. Earth system models (ESMs) are useful tools for predicting and understanding past and future aspects of the climate system. However, the biological and physical parameters used in ESMs can have wide variations in their estimates. Even small changes in these parameters can yield unexpected results without a clear explanation of how a particular outcome was reached. The standard method for estimating ESM sensitivity is to compare spatiotemporal distributions of variables from different runs of a single ESM. However, a potential pitfall of this method is that ESM output could match observational patterns because of compensating errors. For example, if a model predicts overly weak upwelling and low nutrient concentrations, it might compensate for this by allowing phytoplankton to have a high sensitivity to nutrients. Recently, we demonstrated that neural network ensembles (NNEs) are capable of extracting relationships between predictor and target variables within ocean biogeochemical models. Being able to view the relationships between variables, along with spatiotemporal distributions, allows for a more mechanistically based examination of ESM outputs. Here, we investigated whether we could apply NNEs to help us determine why different ESMs produce different spatiotemporal distributions of phytoplankton biomass. We tested this using three cases. The first and second case used different runs of the same ESM, except that the physical circulations differed between them in the first case, while the biological equations differed between them in the second. Our results indicated that the NNEs were capable of extracting the relationships between variables for different runs of a single ESM, allowing us to distinguish between differences due to changes in circulation (which do not change relationships) from changes in biogeochemical formulation (which do change relationships). In the third case, we applied NNEs to two different ESMs. The results of the third case highlighted the capability of NNEs to contrast the apparent relationships of different ESMs and some of the challenges it presents. Although applied specifically to the ocean components of an ESM, our study demonstrates that Earth system modelers can use NNEs to separate the contributions of different components of ESMs. Specifically, this allows modelers to compare the apparent relationships across different ESMs and observational datasets.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2020

A Machine Learning Examination of Hydroxyl Radical Differences Among Model Simulations for CCMI-1

Hydroxyl radical (OH) plays critical roles within the troposphere, such as determining the lifetime of methane (CH4), yet is challenging to model due to its fast cycling and dependence on a multitude of sources and sinks. As a result, the reasons for variations in OH and the resulting CH4 lifetime (TCH4), both between models and in time, are difficult to diagnose. We apply a neural network (NN) approach to address this issue within a group of models that participated in the Chemistry-Climate Model Initiative (CCMI). Analysis of the historical specified dynamics simulations performed for CCMI indicates that the primary drivers of TCH4 differences among ten models are the flux of UV light to the troposphere (indicated by the photolysis frequency JO1D) due mostly to clouds, mixing ratio of tropospheric ozone (O3), the abundance of nitrogen oxides (NOx≡NO+NO2), and details of the various chemical mechanisms that drive OH. Water vapor, carbon monoxide (CO), the ratio of NO:NOx, and formaldehyde (HCHO) explain moderate differences in TCH4, while isoprene, CH4, the photolysis frequency of NO2 by visible light (JNO2), overhead O3 column, and temperature account for little-to-no model variation in CH4. We also apply the NNs to analysis of temporal trends in OH from 1980 to 2015. All models that participated in the specified dynamics historical simulation for CCMI demonstrate a decline in CH4 during the analysed timeframe. The significant contributors to this trend, in order of importance, are tropospheric O3, JO1D, NOx, and H2O, with CO also causing substantial interannual variability in OH burden. Finally, the identified trends in TCH4 are compared to calculated trends in the tropospheric mean OH concentration from previous work, based on analysis of observations. The comparison reveals a robust result for the effect of rising water vapor on OH and CH4, imparting an increasing and decreasing trend of about 0.5% decade(exp -1), respectively. The responses due to NOx, O3 column, and temperature are also in reasonably good agreement between the two studies, though a discrepancy in the CH4 response highlights a need for further examination of the CH4 feedback on the abundance of OH.

Julie M Nicely↗
DOE OSTI2022

InClass nets: independent classifier networks for nonparametric estimation of conditional independence mixture models and unsupervised classification

Abstract Conditional independence mixture models (CIMMs) are an important class of statistical models used in many fields of science. We introduce a novel unsupervised machine learning technique called the independent classifier networks (InClass nets) technique for the nonparameteric estimation of CIMMs. InClass nets consist of multiple independent classifier neural networks (NNs), which are trained simultaneously using suitable cost functions. Leveraging the ability of NNs to handle high-dimensional data, the conditionally independent variates of the model are allowed to be individually high-dimensional, which is the main advantage of the proposed technique over existing non-machine-learning-based approaches. Two new theorems on the nonparametric identifiability of bivariate CIMMs are derived in the form of a necessary and a (different) sufficient condition for a bivariate CIMM to be identifiable. We use the InClass nets technique to perform CIMM estimation successfully for several examples. We provide a public implementation as a Python package called RainDancesVI.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

The clustering of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey quasar sample: testing observational systematics on the Baryon Acoustic Oscillation measurement

ABSTRACT Baryon Acoustic Oscillations (BAO) are considered to be a very robust standard ruler against various systematics. This premise has been tested against observational systematics, but not to the level required for the next generation of galaxy surveys such as the Dark Energy Spectroscopic Instrument (DESI) and Euclid. In this paper, we investigate the effect of observational systematics on the BAO measurement of the final sample of quasars from the extended Baryon Oscillation Spectroscopic Survey Data Release 16 in order to prepare and hone a similar analysis for upcoming surveys. We employ catalogues with various treatments of imaging systematic effects using linear and neural network-based non-linear approaches and consider how the BAO measurement changes. We also test how the variations to the BAO fitting model respond to the observational systematics. As expected, we confirm that the BAO measurements obtained from the DR16 quasar sample are robust against imaging systematics well within the statistical error, while reporting slightly modified constraints that shift the line-of-sight BAO signal by less than 1.1 per cent. We use realistic simulations with similar redshift and angular distributions as the DR16 sample to conduct statistical tests for validating the pipeline, quantifying the significance of differences, and estimating the expected bias on the BAO scale in future high-precision data sets. Although we find a marginal impact for the eBOSS QSO data, the work presented here is of vital importance for constraining the nature of dark energy with the BAO feature in the new era of big data cosmology.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Extending Parsimonious Bayesian Inference

Parsimonious Bayesian inference is a theoretical framework for efficient data assimilation that seeks to balance increased consistency between predictions and training data against corresponding increases in model complexity. Within this framework, over-training is understood as optimization that encodes excessive information within model parameters while only achieving small improvements between predictions and training data. This project aims to develop practical methods of limiting excess model information during optimization. One key observation is that practical heuristics for parsimonious learning in high-dimensions must balance expressivity, i.e. the ability of the model to capture diverse predictions with only a few non-zero parameters, against discoverability, i.e. the ability to train the model with gradient-based optimization and drive parameters to low information states. As such, we developed logical activation functions that are able to adaptively approximate arbitrary truth tables that define Boolean logic operations within a probabilistic framework. These functions have demonstrated the ability to learn exclusive disjunction (XOR) and conditioned disjunction (if [condition] then [result_if_true] else [result_if_false]) within a single layer of a neural network. To efficiently exploit these activation functions to drive parsimonious learning required several other advances within the domain of variational inference. The most efficient form of complexity suppression is structured sparsification, driving most model parameters to zero while achieving the structural coherence among nonzeros needed for bandwidth reduction. Such models are not only far more efficient at suppressing information-theoretic complexity, they also reduce the other forms of complexity (computations, communication, storage, and the number of dependencies needed to evaluate predictions). Aiming to support enhanced sparsification, this project examined new approaches to high-dimensional variational inference that allow us to calibrate and control parameter uncertainty during optimization. By identifying which parameters can sustain sparsifying perturbations with little impact on prediction quality, we can develop better pruning strategies by framing them as approximate Bayesian inference. These advances also open paths to mitigate concerns with deploying advanced learning methods in resource-constrained environments, such as running models on power-limited or communication-limited devices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Automated Waterbox Inspection for Nuclear Power Plants Using Computer Vision - Based Change Detection

Nuclear power plant waterboxes require regular inspection for leaks, missing components, and structural damage during maintenance outages. Traditional manual inspection is time-consuming and poses safety risks from confined space entry. We developed an automated computer vision system for drone-based waterbox inspection in partnership with Florida Light and Power. Our approach uses feature detection and matching to identify critical changes between baseline and current inspection images, automatically flagging additions (leaks/debris), removals (missing plugs), and translations (displaced components) while compensating for drone movement and environmental variations. We systematically evaluated six feature matching methods, from classical approaches (SIFT+BF) to state-of-the-art neural networks (SuperPoint+SuperGlue), using both standard benchmarks (HPatches) and waterbox-specific validation with real-world augmentations. SuperPoint+SuperGlue achieved superior performance with 7.82 pixels RMSE and 100% success rate—2.8x better accuracy than our baseline. While the pre-trained model has commercial licensing restrictions for nuclear deployment, our findings validate this architecture for custom training. We implemented a real-time GUI demonstrating the SIFT+BF approach for immediate deployment, processing drone feeds at 30 FPS with color-coded change visualization. Future work includes training a custom SuperPoint+SuperGlue model on waterbox data and integrating Vision-Language Models for automated reporting and maintenance guidance.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗
DOE OSTI2024

Convolution Neural Network for Fault Identification in Distribution Feeder with High Penetration Solar PV

Identification and zonal classification of the faults is a decisive factor in the relay’s decision to trip or not. Different types of fault like three-phase, line-to-line-to-ground and single-line-to-ground can occur at various locations in the feeder. These faults are seen as the variation in the instantaneous values of three-phase voltages and currents, i.e., waveforms, that are measured at the relay location. The objective of this work is to develop a machine learning model that can identify a fault and classify it to various protection zones based on measured waveforms. In this work, a data-driven relay based on Convolutional Neural Network (CNN) is proposed for fault identification in distribution feeders with high penetration solar PV. The proposed CNN model takes local current and voltage waveforms as input and classify it into fault, no-fault or a capacitor switching. Further, the CNN also attempts to identify fault zones based on the images of waveforms. The overall testing accuracy of the trained model exceeds 95%.

Ramesh, Meghana↗
DOE OSTI2024

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

36 MATERIALS SCIENCE↗
DOE OSTI2021

A machine learning-based interaction force model for non-spherical and irregular particles in low Reynolds number incompressible flows

In this study, interaction force of non-spherical particles in low Reynolds number gas-solid flow is investigated by neural network approaches. An artificial neural network (ANN) model is developed to correlate the non-spherical particle shape and the flow conditions with the interaction force. To define the particle shape, spherical harmonic expansion is applied. Furthermore, variational autoencoder model is then used to extract latent geometric features. The latent vector is utilized as an input with the Reynolds number for the ANN. The interaction force data, which is used as output data of the ANN, is obtained by particle resolved direct numerical simulation for 5200 non-spherical particles. The proposed model enables unsupervised extraction for non-spherical particle shapes and accurate predictions on the interaction force without heavy computation. This study provides the model that can explain complicated shapes of particles and be applied to a large scale, computational fluid dynamics simulation.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2021

Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems

We develop data-driven methods for incorporating physical information for priors to learn parsimonious representations of nonlinear systems arising from parameterized PDEs and mechanics. Our approach is based on Variational Autoencoders (VAEs) for learning nonlinear state space models from observations. We develop ways to incorporate geometric and topological priors through general manifold latent space representations. We investigate the performance of our methods for learning low dimensional representations for the nonlinear Burgers equation and constrained mechanical systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Spike-and-Slab Shrinkage Priors for Structurally Sparse Bayesian Neural Networks

Network complexity and computational efficiency have become increasingly significant aspects of deep learning. Sparse deep learning addresses these challenges by recovering a sparse representation of the underlying target function by reducing heavily overparameterized deep neural networks. Specifically, deep neural architectures compressed via structured sparsity (e.g., node sparsity) provide low-latency inference, higher data throughput, and reduced energy consumption. In this article, we explore two well-established shrinkage techniques, Lasso and Horseshoe, for model compression in Bayesian neural networks (BNNs). To this end, we propose structurally sparse BNNs, which systematically prune excessive nodes with the following: 1) spike-and-slab group Lasso (SS-GL) and 2) SS group Horseshoe (SS-GHS) priors, and develop computationally tractable variational inference, including continuous relaxation of Bernoulli variables. We establish the contraction rates of the variational posterior of our proposed models as a function of the network topology, layerwise node cardinalities, and bounds on the network weights. Furthermore, we empirically demonstrate the competitive performance of our models compared with the baseline models in prediction accuracy, model compression, and inference latency.

97 MATHEMATICS AND COMPUTING↗