Absence of Barren Plateaus in Quantum Convolutional Neural Networks
Not Available
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
SEARCH · Search NASA
Results for “Neural Machine Learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
A General Spatiotemporal Imputation Framework for Missing Sensor Data
Many applications from precision agriculture, environmental monitoring and transportation networks rely on data collected across space and time over a large geographic area. Missing data poses a significant challenge for any data-driven inference and control tasks. Data imputation or the estimation of missing data can help fill these gaps by utilizing inherent spatial relationships and temporal patterns. A variety of spatiotemporal imputation models have been developed to address missing data in spatiotemporal datasets. However, these classical methods rely on the assumption that the underlying data follows a smooth trend and fail to provide accurate estimates when there is a large number of missing points in the data. Even though there are machine learning driven tensor completion approaches such as convolutional neural network based tensor completion (CoSTCo) that capture the non-linear relationships in the dataset, the transductive nature makes the algorithm less scalable. Thus, existing approaches for estimating the missing information do not effectively capture all dimensions of the spatiotemporal data structure, resulting in erroneous predictions and poor performance. The main contributions of this paper are: (1) We propose a novel inductive framework (G-LSTM) for missing data imputation that integrates a graph neural network with LSTMs to effectively capture both spatial and temporal dependencies. (2) Experimental results on a traffic dataset demonstrate that the proposed GNN integrated with an LSTM framework achieves improved imputation and maintains steady performance even when there are extreme missing conditions in comparison with the state-of-the-art imputation framework (i.e, CoSTCo). (3) The simulation results on a traffic network show up to 69% reduction in mean absolute error and 61% reduction in root mean square error when compared to CoSTCo.
Tharzeen, Aabila↗
Machine Learning-Based Temperature Prediction for Runtime Thermal Management Across System Components
Elevated temperatures limit the peak performance of systems because of frequent interventions by thermal throttling. Non-uniform thermal states across system nodes also cause performance variation within seemingly equivalent nodes leading to significant degradation of overall performance. In this paper we present a framework for creating a lightweight thermal prediction system suitable for run-time management decisions. We pursue two avenues to explore optimized lightweight thermal predictors. First, we use feature selection algorithms to improve the performance of previously designed machine learning methods. Second, we develop alternative methods using neural network and linear regression-based methods to perform a comprehensive comparative study of prediction methods. We show that our optimized models achieve improved performance with better prediction accuracy and lower overhead as compared with the Gaussian process model proposed previously. Specifically we present a reduced version of the Gaussian process model, a neural network-based model, and a linear regression-based model. Using the optimization methods, we are able to reduce the average prediction errors in the Gaussian process from 4: 2 degrees C to 2: 9 degrees C. We also show that the newly developed models using neural network and Lasso linear regression have average prediction errors of 2: 9 degrees C and 3: 8 degrees C respectively. The prediction overheads are 0.22, 0.097, and 0.026 ms per prediction for reduced Gaussian process, neural network, and Lasso linear regression models, respectively, compared with 0.57 ms per prediction for the previous Gaussian process model. We have implemented our proposed thermal prediction models on a two-node system configuration to help identify the optimal task placement. The task placement identified by the models reduces the average system temperature by up to 11: 9 degrees C without any performance degradation. Furthermore, these models respectively achieve 75, 82.5, and 74.17 percent success rates in correctly pointing to those task placements with better thermal response, compared with 72.5 percent success for the original model in achieving the same objective. Lastly, we extended our analysis to a 16-node system and we were able to train models and execute them in real time to guide task migration and achieve on average 17 percent reduction in the overall system cooling power.
97 MATHEMATICS AND COMPUTING↗
Data-Driven Prediction and Optimization of Energy Use for Transit Fleets of Electric and ICE Vehicles
Due to the high upfront cost of electric vehicles, many public transit agencies can afford only mixed fleets of internal combustion and electric vehicles. Optimizing the operation of such mixed fleets is challenging because it requires accurate trip-level predictions of electricity and fuel use as well as efficient algorithms for assigning vehicles to transit routes. We present a novel framework for the data-driven prediction of trip-level energy use for mixed-vehicle transit fleets and for the optimization of vehicle assignments, which we evaluate using data collected from the bus fleet of CARTA, the public transit agency of Chattanooga, TN. We first introduce a data collection, storage, and processing framework for system-level and high-frequency vehicle-level transit data, including domain-specific data cleansing methods. We train and evaluate machine learning models for energy prediction, demonstrating that deep neural networks attain the highest accuracy. Based on these predictions, we formulate the problem of minimizing energy use through assigning vehicles to fixed-route transit trips. We propose an optimal integer program as well as efficient heuristic and meta-heuristic algorithms, demonstrating the scalability and performance of these algorithms numerically using the transit network of CARTA.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DeepAnnotator
Given a collection of pairs ((A) written material such as reports, phrases, bulleted lists, etc., (B) structured labels called annotations) called the training data, we want to create software to take future written material (A) and correctly predict (B). In this case we have used medical reports and medical coder's structured labels as our training data. The software builds a machine learning model of type deep learning, specifically a convolutional neural network (CNN) using the training data so that when new data arrives in the future the annotations can be predicted.
Christian, JamesB.↗
ECabc: A feature tuning program focused on Artificial Neural Network hyperparameters
ECabc is an open source Python package based on an Artificial Bee Colony, used for tuning functions such as hyperparameters of artificial neural networks. The method is modelled after the honey foraging techniques of bees, where the search space for solutions is in as many dimensions as the number of parameters being tuned.
97 MATHEMATICS AND COMPUTING↗
DeepHyper: A Python Package for Massively Parallel Hyperparameter Optimization in Machine Learning
Machine learning models are increasingly applied across scientific disciplines, yet their effectiveness often hinges on heuristic decisions—such as data transformations, training strategies, and model architectures—that are not learned by the models themselves. Automating the selection of these heuristics and analyzing their sensitivity is crucial for building robust and efficient learning workflows. DeepHyper addresses this challenge by democratizing hyperparameter optimization, providing accessible tools to streamline and enhance machine learning workflows from a laptop to the largest supercomputer in the world. Building on top of hyperparameter optimization, it unlocks new capabilities around ensembles of models for improved accuracy and uncertainty quantification. All of these organized around efficient parallel computing.
ensemble↗
Machine Learning Vacancy Formation Energy in Nickel-Based Superalloys
Creep performance plays a key role in nickel-based superalloys for high temeprature applications. Creep behavior depends on many parameters such as strength, dislocations, diffusivity, and microstructural stability in addition to temeprature, applied stress, and oxidation. This work focuses on predicting vacancy formation energy in nickel-based superalloys using machine learning approach. High-throughput density functional theory (DFT) calculations are performed on Ni-based alloys with the addition of various alloying elements to predict the vacancy formation energy and vacancy concentration. Machine learning is performed using various models including graph neural networks.
creep performance↗
Improving robustness for model discerning synthesis process of uranium oxide with unsupervised domain adaptation
The quantitative characterization of surface structures captured in scanning electron microscopy (SEM) images has proven to be effective for discerning provenance of an unknown nuclear material. Recently, many works have taken advantage of the powerful performance of convolutional neural networks (CNNs) to provide faster and more consistent characterization of surface structures. However, one inherent limitation of CNNs is their degradation in performance when encountering discrepancy between training and test datasets, which limits their use widely. The common discrepancy in an SEM image dataset occurs at low-level image information due to user-bias in selecting acquisition parameters and microscopes from different manufacturers. Therefore, in this study, we present a domain adaptation framework to improve robustness of CNNs against the discrepancy in low-level image information. Furthermore, our proposed approach makes use of only unlabeled test samples to adapt a pretrained model, which is more suitable for nuclear forensics application for which obtaining both training and test datasets simultaneously is a challenge due to data sensitivity. Through extensive experiments, we demonstrate that our proposed approach effectively improves the performance of a model by at least 18% when encountering domain discrepancy, and can be deployed in many CNN architectures.
scanning electron microscopy↗
An Intelligent System for Automatic Selection of DC-DC Converter Topology with Optimal Design
In this paper, we present an intelligent system that has the capabilities of automatically selecting topology classes and optimizing circuit parameters of DC-DC power converters for a given design specification. The system, Machine-Learning-enhanced Automated Circuit Configuration and Evaluation of Power Converters (ML-ACCEPT), uses a hybrid of machine learning technologies, decision tree inference, reinforcement learning and deep neural networks. The system gives high accurate recommendations of design topology classes and computationally efficient results in optimizing power efficiencies in power converter design.
Xue, Lincoln↗
Physics-Informed Machine Learning of Dynamical Systems for Efficient Bayesian Inference
Although the no-u-turn sampler (NUTS) is a widely adopted method for performing Bayesian inference, it requires numerous posterior gradients which can be expensive to compute in practice. Recently, there has been a significant interest in physics-based machine learning of dynamical (or Hamiltonian) systems and Hamiltonian neural networks (HNNs) is a noteworthy architecture. But these types of architectures have not been applied to solve Bayesian inference problems efficiently. We propose the use of HNNs for performing Bayesian inference efficiently without requiring numerous posterior gradients. We introduce latent variable outputs to HNNs (L-HNNs) for improved expressivity and reduced integration errors. We integrate L-HNNs in NUTS and further propose an online error monitoring scheme to prevent sampling degeneracy in regions where L-HNNs may have little training data. We demonstrate L-HNNs in NUTS with online error monitoring consider several complex high-dimensional posterior densities and compare its performance to NUTS.
97 MATHEMATICS AND COMPUTING↗
Uncertainty-aware photovoltaic generation estimation through fusion of physics with harmonics information using Bayesian neural networks
We develop an aggregate photovoltaic generation estimation methodology that uses diverse inputs and can reason on its current input-dependent predictive uncertainty. Named PVPHEst, for PhotoVoltaic Physics- & Harmonics-driven Estimator, the resulting tool is intelligently weighing and fusing information carried by the output of physics models, harmonics, and line sensors, using Bayesian neural networks and related techniques aimed at solving machine learning problems with intrinsic uncertainty quantification. As each of the three input classes carries heterogeneous information that only sheds light on one facet of the estimation problem but its value can diminish in the face of diverse grid phenomena, PV-PHEst with its estimation and uncertainty reasoning capabilities perform a nontrivial and potentially mission-critical task of value to grid operators.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Stepwise Dynamic Calibration of a Hydromechanical Simulation Using Time-Lapse Vertical Seismic Profile
This study aims to develop a methodology for calibrating subsurface stress changes through time-lapse Vertical Seismic Profiling (VSP) integration. The selected study site is the 13-10A injector well within the ongoing CO2-EOR operation of the Farnsworth Field Unit. The Time-lapse VSP dataset carries the combined effects of fluid substitution and mean effective stress changes, thereby providing a dataset amenable for the calibration of production and injection-induced stress changes. The concept is similar to calibrating a reservoir simulation model in that the process honor real field data to set up an inverse problem. The solution optimizes the independent and impactful geomechanical parameters that replicate the observed time-lapse seismic velocity changes. This stress calibration is enabled by 4D geomechanical modeling and the VSP Integration workflow. This calibration benefits from extensive geological, geophysical and geomechanical characterization through 3D seismic data, geophysical well logs, and core assessed as part of the 1D MEM conducted on the 13-10A subject well. These data are used to develop a site-specific rock physics model. The Biot Gassmann workflow combines rock physics and reservoir simulation outputs to determine the fluid substitution contribution to seismic velocity change. Additionally, modeled seismic velocity attributed to mean effective stress are determined from the geomechanical simulation outputs, and the stress-velocity relationship developed from the ultrasonic seismic velocity measurements on the extracted Morrow B core. A penalty function is then formed between the modeled seismic velocities and the observed time-lapse VSP dataset. Four independent and impactful geomechanical parameters have been determined. These are the bulk modulus and shear modulus for zero porosity and the shear and compressional seismic velocity to mean effective stress derivatives. The dataset of numerous coupled hydromechanical- geomechanical simulation realizations is built by combining variations of the four stated geomechanical parameters. A machine learning-assisted workflow comprised of an artificial neural network and a particle swarm optimizer are used to converge on the optimal geomechanical parameters. The successful execution of this workflow has affirmed the suitability of acoustic time-lapse measurements for 4D-VSP geomechanical stress calibration pending measurable stress sensitivities within the anticipated effective stress changes and the availability of suitable and reliable datasets for petroelastic modeling.
02 PETROLEUM↗
Complex-valued universal linear transformations and image encryption using spatially incoherent diffractive networks
As an optical processor, a diffractive deep neural network (D2NN) utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing, completing its tasks at the speed of light propagation through thin optical layers. With sufficient degrees of freedom, D2NNs can perform arbitrary complex-valued linear transformations using spatially coherent light. Similarly, D2NNs can also perform arbitrary linear intensity transformations with spatially incoherent illumination; however, under spatially incoherent light, these transformations are nonnegative, acting on diffraction-limited optical intensity patterns at the input field of view. Here, we expand the use of spatially incoherent D2NNs to complex-valued information processing for executing arbitrary complex-valued linear transformations using spatially incoherent light. Through simulations, we show that as the number of optimized diffractive features increases beyond a threshold dictated by the multiplication of the input and output space-bandwidth products, a spatially incoherent diffractive visual processor can approximate any complex-valued linear transformation and be used for all-optical image encryption using incoherent illumination. The findings are important for the all-optical processing of information under natural light using various forms of diffractive surface-based optical processors.
36 MATERIALS SCIENCE↗
AnisONet: A deep neural operator-based anisotropic permeability upscaler from pore to Darcy scale
Directional permeability variations, which govern directional fluid flow in porous media with anisotropy, are important to accurately predict flow behavior, reactive transport, and fluid–solid interactions for various processes such as enhanced geothermal systems, energy storage devices, and biological systems. However, the intricate architecture of porous media makes it difficult to predict directional permeabilities. In this work, we present a novel machine learning (ML) framework, AnisONet, built upon an integration of a convolutional neural network, Swin transformer, and the deep operator network architecture, designed to predict anisotropic permeability and upscale predictions to larger spatial domains. First, AnisONet was evaluated with three classes of two-dimensional (2D) porous media, including synthetic circular and elliptical grains and natural sandstone grains from micro-computed tomography images. A lattice Boltzmann model (LBM) was used to calculate directional permeabilities at every 10° angle, producing 19 data points per image of porous media. AnisONet is then trained to predict permeability as a function of rotation angle. AnisONet showed strong predictive capability of directional permeability. Second, we tested our model for five upscaling cases with a large image size in the finite-element method (FEM) for 2D Darcy flow with various permeability tensor construction methods. Overall, upscaled permeability tensors in FEM simulations produce a reasonably good match with LBM results, highlighting the importance of selecting appropriate tensor formation strategies for accurate permeability upscaling. AnisONet, as a directional permeability estimator, could be further developed for more complex geometries, with the potential to develop a foundational ML model for various applications in porous media.
42 ENGINEERING↗
Clarifying trust of materials property predictions using neural networks with distribution-specific uncertainty quantification
It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the adsorption energy predictions, namely k-fold ensembling, Monte Carlo dropout, and evidential regression. The effectiveness of each UQ method is assessed based on accuracy, sharpness, dispersion, calibration, and tightness. Evidential regression is demonstrated to be a powerful approach for rapidly obtaining tunable, competitively trustworthy UQ estimates for heterogeneous catalysis applications when using neural networks. Recalibration of model uncertainties is shown to be essential in practical screening applications of catalysts using uncertainties.
36 MATERIALS SCIENCE↗
Strategies and Software for Machine Learning Accelerated Discovery in Transition Metal Chemistry
Machine learning the electronic structure of open shell transition metal complexes presents unique challenges, including robust and automated data set generation. In this report we introduce tools that simplify data acquisition from density functional theory (DFT) and validation of trained machine learning models using the molSimplify automatic design (mAD) workflow. We demonstrate this workflow by training and comparing the performance of LASSO, kernel ridge regression (KRR), and artificial neural network (ANN) models using heuristic, topological revised autocorrelation (RAC) descriptors we have recently introduced for machine learning inorganic chemistry. On a series of open shell transition metal complexes, we evaluate set aside test errors of these models for predicting the HOMO level and HOMO–LUMO gap. The best performing models are ANNs, which show 0.15 and 0.25 eV test set mean absolute errors on the HOMO level and HOMO–LUMO gap, respectively. Poor performing KRR models using the full 153-feature RAC set are improved to nearly the same performance as the ANNs when trained on down-selected subsets of 20–30 features. Analysis of the essential descriptors for HOMO level and HOMO–LUMO gap prediction as well as comparison to subsets previously obtained for other properties reveal the paramount importance of nonlocal, steric properties in determining frontier molecular orbital energetics. We demonstrate our model performance on diverse complexes and in the discovery of molecules with target HOMO–LUMO gaps from a large 15,000 molecule design space in minutes rather than days that full DFT evaluation would require.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Model-free stabilization via Extremum Seeking using a cost neural estimator
In this paper, a fully model-free architecture for vertical stabilization of thermonuclear plasmas in tokamak experimental reactors is presented. For the first time, an Extremum Seeking control algorithm is combined with neural networks to estimate the Lyapunov function to be minimized, resulting in a fully data-driven control architecture. The performance of different neural networks are compared. Specifically, Multilayer Perceptrons and Extreme Learning Machines are considered. The proposed architecture is tested in simulation to show that it can counteract relevant plasma disturbances, resulting in a significant improvement in terms of the achievable operative space compared to the Extremum Seeking algorithm, which still relies on model-based cost estimator.
42 ENGINEERING↗