Search NASA⌕ Search

SEARCH · Search NASA

Results for “Markov networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Operation Optimization Using Reinforcement Learning with Integrated Artificial Reasoning Framework

In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.

Kim, Junyung↗
DOE OSTI2022

Computational Aspects of Single-Molecule Kinetics for Coupled Catalytic Cycles: A Spectral Analysis

Catalysis from single active sites is analyzed using methods developed from single molecule kinetics. Using a stochastic Markov state description, the observable properties of general catalytic networks of reactions are expressed using an eigenvalue decomposition of the transition matrix for the Markov process. By the use of a sensitivity analysis, the necessary eigenvalues and eigenvectors are related to the energies of controlling barriers and wells located along the reaction routes. A generalization of the energetic span theory allows the eigenvalues to be computed from several activation energies corresponding to distinct barrier-well pairings. The formalism is demonstrated accurately for model problems for a physically realistic mechanism for an alkene hydrogenation reaction on a single atom catalyst.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Network Reconfiguration for Enhanced Operational Resilience Using Reinforcement Learning

This paper proposes a reinforcement learning-based approach for distribution network reconfiguration(DNR) to enhance the resilience of the electric power supply. Resilience enhancements usually require solving large-scale stochastic optimization problems that are computationally expensive and sometimes infeasible. The exceptional performance of reinforcement learning techniques has encouraged their adoption in various power system control studies, specifically resilience-based real-time applications. In this paper, a single agent framework is developed using an Actor-Critic algorithm (ACA) to determine statuses of tie-switches in a distribution feeder impacted by an extreme weather event. The proposed approach provides a fast-acting control algorithm that reconfigures the feeder topology to reduce or even avoid load shedding. The problem is formulated as a discrete Markov decision process in such a way that a system state captures the system topology and its operational characteristics. An action is made to open or close a specific set of tie-switches after which a reward is calculated to evaluate the practicality and advantage of that action. The iterative Markov process is used to train the proposed ACA under diverse failure scenarios and is demonstrated on the 33-node distribution feeder system. Results show the capability of the proposed ACA to determine proper switching action of tie-switches with accuracy exceeding 93%.

actor critic↗
DOE OSTI2022

Quantum advantage for differential equation analysis

Quantum algorithms for differential equation solving, data processing, and machine learning potentially offer an exponential speedup over all known classical algorithms. However, there also exist obstacles to obtaining this potential speedup in useful problem instances. The essential obstacle for quantum differential equation solving is that outputting useful information may require difficult postprocessing, and the essential obstacle for quantum data processing and machine learning is that inputting the data is a difficult task just by itself. In this study, we demonstrate that, when combined, these difficulties solve one another. We show how the output of quantum differential equation solving can serve as the input for quantum data processing and machine learning, allowing dynamical analysis in terms of principal components, power spectra, and wavelet decompositions. To illustrate this, we consider continuous-time Markov processes on epidemiological and social networks. These quantum algorithms provide an exponential advantage over existing classical Monte Carlo methods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Sensitivity-Informed Bayesian Inference for Home PLC Network Models with Unknown Parameters

Bayesian inference is used to calibrate a bottom-up home PLC network model with unknown loads and wires at frequencies up to 30 MHz. A network topology with over 50 parameters is calibrated using global sensitivity analysis and transitional Markov Chain Monte Carlo (TMCMC). The sensitivity-informed Bayesian inference computes Sobol indices for each network parameter and applies TMCMC to calibrate the most sensitive parameters for a given network topology. A greedy random search with TMCMC is used to refine the discrete random variables of the network. This results in a model that can accurately compute the transfer function despite noisy training data and a high dimensional parameter space. The model is able to infer some parameters of the network used to produce the training data, and accurately computes the transfer function under extrapolative scenarios.

42 ENGINEERING↗
DOE OSTI2021

An adaptive Hessian approximated stochastic gradient MCMC method

Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their ability to handle large datasets and the potential to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. Here, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation (SA) to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the samples and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix via SA, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and demonstrates better learning results compared to standard SG-MCMC updating rules. The approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. Numerical experiments are performed on several problems, including sampling from 2D correlated distribution, synthetic regression problems, and learning the numerical solutions of heterogeneous elliptic PDE. The numerical results demonstrate great improvement in both the convergence rate and accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Automated Control of Transactive HVACs in Energy Distribution Systems

Heating, Ventilation, and Air Conditioning (HVAC) systems contribute significantly to a building’s energy consumption. In the recent years, there is an increased interest in developing transactive approaches which could enable automated and flexible scheduling of HVAC systems based on the customer demand and the electricity prices decided by the suppliers. Flexible and automated scheduling of the HVAC systems make it a prime source for participation in residential demand response or transactive energy systems. Therefore, it is of significant interest to identify an optimal strategy to control the HVAC systems. Here, reducing the energy cost while keeping the comfort level acceptable to the users, we argue that such a control strategy should consider both the energy cost and user comfort simultaneously. Accordingly, we develop the control strategy through the solution of an optimization problem that balances between the energy cost and consumer’s dissatisfaction. This optimization enables us to solve a decision-making problem through first price prediction and then choosing HVAC temperature settings throughout the day based on the predicted price, history of the price and HVAC settings, and outside temperature. More specifically, we formulate the control design as a Markov decision process (MDP) using deep neural networks and use Deep Deterministic Policy Gradients (DDPG)-based deep reinforcement learning algorithm to find the optimal control strategy for HVAC systems that balances between electricity cost and user comfort.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Trustworthy Physics-Informed Deep Learning for Predictive Scientific Computing

This project has developed powerful trustworthy physics-informed deep learning (TPiDL) models and methods to fundamentally enhance the scale and power of computational modeling in the scientific and engineering domains. Deep learning (DL) has radically advanced the state-of-the-art in machine learning, computer vision, natural language processing, and also scientific computing. Nevertheless, progress has been driven almost entirely by empirical observations, hacks, and tricks. Under the support of this project, the graph operator learning tools and advanced trustworthy physical informed neural networks have been developed. In addition, stochastic gradient replica-exchange Markov Chain Monte Carlo (MCMC) sampling algorithms have been designed to quantify the uncertainties and speed up the training of large-scale neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Climatic clustering and longitudinal analysis with impacts on food, bioenergy, and pandemics

Predicted growth in world population will put unparalleled stress on the need for sustainable energy and global food production, as well as increase the likelihood of future pandemics. In this work, we identify high-resolution environmental zones in the context of a changing climate and predict longitudinal processes relevant to these challenges. We do this using exhaustive vector comparison methods that measure the climatic similarity between all locations on earth at high geospatial resolution relative to global-scale analyses. The results are captured as networks, in which edges between geolocations are defined if their historical climate similarities exceed a threshold. We apply Markov clustering and our novel Correlation of Correlations method to the resulting climatic networks, which provides unprecedented agglomerative and longitudinal views of climatic relationships across the globe. The methods performed here resulted in the fastest (9.37x10 18 operations/sec) and one of the largest (168.7x10 21 operations) scientific computations ever performed, with more than 100 quadrillion edges considered for a single climatic network. Our climatic analysis reveals areas of the world experiencing rapid environmental changes, which can have important implications for global carbon fluxes and zoonotic spillover events. Correlation and network analyses of this kind are widely applicable across computational and predictive biology domains, including systems biology, ecology, carbon cycles, biogeochemistry, and zoonosis research.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

A Kinetic Monte Carlo Approach for Simulating Cascading Transmission Line Failure

In this work, cascading transmission line failures are studied through a dynamical model of the power system operating under fixed conditions. The power grid is modeled as a stochastic dynamical system where first-principles electromechanical dynamics are excited by small Gaussian disturbances in demand and generation around a specified operating point. In this context, a single line failure is interpreted in a large deviation context as a first escape event across a surface in phase space defined by line security constraints. The resulting system of stochastic differential equations admits a transverse decomposition of the drift, which leads to considerable simplification in evaluating the quasipotential (rate function) and, consequently, computation of exit rates. Tractable expressions for the rate of transmission line failure in a restricted network are derived from large deviation theory arguments and validated against numerical simulations. Extensions to realistic settings are considered, and individual line failure models are aggregated into a Markov model of cascading failure inspired by chemical kinetics. Cascades are generated by traversing a graph composed of weighted edges representing transitions to degraded network topologies. Numerical results indicate that the Markov model can produce cascades with qualitative power-law properties similar to those observed in empirical cascades.

Kinetic Monte Carlo, rare events, Cascading Failur↗
DOE OSTI2022

Classification of Nuclear Reactor Operations Using Spatial Importance and Multisensor Networks

Distributed multisensor networks record multiple data streams that can be used as inputs to machine learning models designed to classify operations relevant to proliferation at nuclear reactors. The goal of this work is to demonstrate methods to assess the importance of each node (a single multisensor) and region (a group of proximate multisensors) to machine learning model performance in a reactor monitoring scenario. This, in turn, provides insight into model behavior, a critical requirement of data-driven applications in nuclear security. Using data collected at the High Flux Isotope Reactor at Oak Ridge National Laboratory via a network of Merlyn multisensors, two different models were trained to classify the reactor’s operational state: a hidden Markov model (HMM), which is simpler and more transparent, and a feed-forward neural network, which is less inherently interpretable. Traditional wrapper methods for feature importance were extended to identify nodes and regions in the multisensor network with strong positive and negative impacts on the classification problem. These spatial-importance algorithms were evaluated on the two different classifiers. The classification accuracy was then improved relative to baseline models via feature selection from 0.583 to 0.839 and from 0.811 ± 0.005 to 0.884 ± 0.004 for the HMM and feed-forward neural network, respectively. While some differences in node and region importance were observed when using different classifiers and wrapper methods, the nodes near the facility’s cooling tower were consistently identified as important—a conclusion further supported by studies on feature importance in decision trees. Node and region importance methods are model-agnostic, inform feature selection for improved model performance, and can provide insight into opaque classification models in the nuclear security domain.

Tibbetts, Jake↗
DOE OSTI2025

Discrete generative diffusion models without stochastic differential equations: A tensor network approach

Diffusion models (DMs) are a class of generative machine learning methods that sample a target distribution by transforming samples of a trivial (often Gaussian) distribution using a learned stochastic differential equation. In standard DMs, this is done by learning a “score function” that reverses the effect of adding diffusive noise to the distribution of interest. Here we consider the generalisation of DMs to lattice systems with discrete degrees of freedom, and where noise is added via Markov chain jump dynamics. We show how to use tensor networks (TNs) to efficiently define and sample such “discrete diffusion models” (DDMs) without explicitly having to solve a stochastic differential equation. We show the following: (i) by parametrising the data and evolution operators as TNs, the denoising dynamics can be represented exactly; (ii) the auto-regressive nature of TNs allows to generate samples efficiently and without bias; (iii) for sampling Boltzmann-like distributions, TNs allow to construct an efficient learning scheme that integrates well with Monte Carlo. We illustrate this approach to study the equilibrium of two models with non-trivial thermodynamics, the d = 1 constrained Fredkin chain and the d = 2 Ising model. Published by the American Physical Society 2025

Causer, Luke (ORCID:0000000194243473)↗
DOE OSTI2023

Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation

Whittle index policy is a heuristic to the intractable restless multi-armed bandits (RMAB) problem. Although it is provably asymptotically optimal, finding Whittle indices remains difficult. In this paper, we present Neural-Q-Whittle, a Whittle index based Q-learning algorithm for RMAB with neural network function approximation, which is an example of nonlinear two-timescale stochastic approximation with Q-function values updated on a faster timescale and Whittle indices on a slower timescale. Despite the empirical success of deep Q-learning, the non-asymptotic convergence rate of Neural-Q-Whittle, which couples neural networks with two-timescale Q-learning largely remains unclear. This paper provides a finite-time analysis of Neural-Q-Whittle, where data are generated from a Markov chain, and Q-function is approximated by a ReLU neural network. Our analysis leverages a Lyapunov drift approach to capture the evolution of two coupled parameters, and the nonlinearity in value function approximation further requires us to characterize the approximation error. Combing these provide Neural-Q-Whittle with convergence rate, where is the number of iterations.

reinforcement learning, structured learning, conve↗
DOE OSTI2022

Efficient quantum state tomography with mode-assisted training

Neural networks (NNs) representing quantum states are typically trained using Markov chain Monte Carlo based methods. However, unless specifically designed, such samplers only consist of local moves, making the slow-mixing problem prominent even for extremely simple quantum states. Here, we propose to use mode-assisted training that provides global information via the modes of the NN distribution. Further, applied to quantum state tomography using restricted Boltzmann machines, this method improves the quality of reconstructed quantum states by orders of magnitude. The method is applicable to other types of NNs and may efficiently tackle problems previously unmanageable.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

36 MATERIALS SCIENCE↗
DOE OSTI2023

HOLISMOKES IX. Neural network inference of strong-lens parameters and uncertainties from ground-based images

Modeling of strong gravitational lenses is a necessity for further applications in astrophysics and cosmology. With the large number of detections in current and upcoming surveys, such as the Rubin Legacy Survey of Space and Time (LSST), it is pertinent to investigate automated and fast analysis techniques beyond the traditional and time-consuming Markov chain Monte Carlo sampling methods. Building upon our (simple) convolutional neural network (CNN), we present here another CNN, specifically a residual neural network (ResNet), that predicts the five mass parameters of a singular isothermal ellipsoid (SIE) profile (lens center x and y, ellipticity e x and e y , Einstein radius θ E ) and the external shear (γ ext, 1 , γ ext, 2 ) from ground-based imaging data. In contrast to our previous CNN, this ResNet further predicts the 1σ uncertainty for each parameter. To train our network, we use our improved pipeline to simulate lens images using real images of galaxies from the Hyper Suprime-Cam Survey (HSC) and from the Hubble Ultra Deep Field as lens galaxies and background sources, respectively. We find very good recoveries overall for the SIE parameters, especially for the lens center in comparison to our previous CNN, while significant differences remain in predicting the external shear. From our multiple tests, it appears that most likely the low ground-based image resolution is the limiting factor in predicting the external shear. Given the run time of milli-seconds per system, our network is perfectly suited to quickly predict the next appearing image and time delays of lensed transients. Therefore, we use the network-predicted mass model to estimate these quantities and compare to those values obtained from our simulations. Unfortunately, the achieved precision allows only a first-order estimate of time delays on real lens systems and requires further refinement through follow-up modeling. Nonetheless, our ResNet is able to predict the SIE and shear parameter values in fractions of a second on a single CPU, meaning that we are able to efficiently process the huge amount of galaxy-scale lenses expected in the near future.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Optimizing High Performance Markov Clustering for Pre-Exascale Architectures

HipMCL is a high-performance distributed memory implementation of the popular Markov Cluster Algorithm (MCL) and can cluster large-scale networks within hours using a few thousand CPU-equipped nodes. It relies on sparse matrix computations and heavily makes use of the sparse matrix-sparse matrix multiplication kernel (SpGEMM). The existing parallel algorithms in HipMCL are not scalable to Exascale architectures, both due to their communication costs dominating the runtime at large concurrencies and also due to their inability to take advantage of accelerators that are increasingly popular. In this work, we systematically remove scalability and performance bottlenecks of HipMCL. We enable GPUs by performing the expensive expansion phase of the MCL algorithm on GPU. Additionally, we propose a CPU-GPU joint distributed SpGEMM algorithm called pipelined Sparse SUMMA and integrate a probabilistic memory requirement estimator that is fast and accurate. Furthermore, we develop a new merging algorithm for the incremental processing of partial results produced by the GPUs, which improves the overlap efficiency and the peak memory usage. We also integrate a recent and faster algorithm for performing SpGEMM on CPUs. We validate our new algorithms and optimizations with extensive evaluations. With the enabling of the GPUs and integration of new algorithms, HipMCL is up to 12.4x faster, being able to cluster a network with 70 million proteins and 68 billion connections just under 15 minutes using 1024 nodes of ORNL's Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Search for low mass dark matter in DarkSide-50: the bayesian network approach

Abstract We present a novel approach for the search of dark matter in the DarkSide-50 experiment, relying on Bayesian Networks. This method incorporates the detector response model into the likelihood function, explicitly maintaining the connection with the quantity of interest. No assumptions about the linearity of the problem or the shape of the probability distribution functions are required, and there is no need to morph signal and background spectra as a function of nuisance parameters. By expressing the problem in terms of Bayesian Networks, we have developed an inference algorithm based on a Markov Chain Monte Carlo to calculate the posterior probability. A clever description of the detector response model in terms of parametric matrices allows us to study the impact of systematic variations of any parameter on the final results. Our approach not only provides the desired information on the parameter of interest, but also potential constraints on the response model. Our results are consistent with recent published analyses and further refine the parameters of the detector response model.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗