Improving Scalability of Multilevel Markov Chain Monte Carlo Using Multiple Proposals
Explore the source record for details and available documents.
Mathematics and Computing↗
SEARCH · Search NASA
Results for “MARKOV”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
HIDDEN MARKOV MODEL APPLICATION FOR UNSUPERVISED MACHINE LEARNING CLUSTERING OF WAVEFORM DATA DURING INDUCED SEISMICITY.
Abstract not provided.
Willis, Rachel Marie↗
Bayesian Inference and Sequential Tempered Markov Chain Monte Carlo for Synthetic Biological Systems.
Abstract not provided.
Catanach, Thomas Anthony↗
Multifidelity Sequential Tempered Markov Chain Monte Carlo for Bayesian Inference.
Abstract not provided.
Catanach, Thomas Anthony↗
A hidden Markov model approach to understanding molecular characteristics in fluorescence microscopy.
Abstract not provided.
Patel, Lekha↗
Constraining neutrino oscillation and interaction parameters with the NOvA Near Detector and Far Detector data using Markov Chain Monte Carlo
This thesis reports a constraint of the neutrino oscillation parameters $\Delta m^{2}_{32}$, $\sin^2 \theta_{23}$, and $\delta_{CP}$ using the NuMI Off-Axis $\nu$ Appearance (NOvA) experiment's Near Detector (ND) data and Far Detector (FD) fake data set simultaneously. This thesis also reports a constraint on NOvA's systematic uncertainty model solely with its Near Detector data. The Hamiltonian Monte Carlo algorithm is used to estimate Bayesian Credible Intervals for the oscillation and interaction parameters. The $1\sigma$ Credible Intervals for $\sin^2 \theta_{23}$ are $(0.44, 0.512)$ $\cup$ $(0.536, 0.56)$, for $\Delta m^{2}_{32}$ $(2.41 \times 10^{-3}$ eV$^2,\ 2.52 \times 10^{-3}$ eV$^2)$, and for $\delta_{CP}$ $(0.74\pi,\ 1.1\pi)$ $\cup$ $(1.38\pi,\ 1.58\pi)$. The statistical power of the ND data constrains NOvA's interaction parameters, while the FD fake data constrains the oscillation parameters. This is the first analysis within NOvA to constrain the ND and FD prediction sim ultaneously, and to investigate the neutrino interaction modeling in the context of constraining the oscillation parameters. To constrain the ND data requires a sophisticated understanding of the neutrino interaction modeling and its uncertainties. The interested reader is advised to focus on Chapters 4 and 6, which discuss the ND selection, uncertainties, and ND-only fits to data. The reader interested in oscillation parameter constraints will find this in Chapter 7.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
A Kinetic Monte Carlo Approach for Simulating Cascading Transmission Line Failure
In this work, cascading transmission line failures are studied through a dynamical model of the power system operating under fixed conditions. The power grid is modeled as a stochastic dynamical system where first-principles electromechanical dynamics are excited by small Gaussian disturbances in demand and generation around a specified operating point. In this context, a single line failure is interpreted in a large deviation context as a first escape event across a surface in phase space defined by line security constraints. The resulting system of stochastic differential equations admits a transverse decomposition of the drift, which leads to considerable simplification in evaluating the quasipotential (rate function) and, consequently, computation of exit rates. Tractable expressions for the rate of transmission line failure in a restricted network are derived from large deviation theory arguments and validated against numerical simulations. Extensions to realistic settings are considered, and individual line failure models are aggregated into a Markov model of cascading failure inspired by chemical kinetics. Cascades are generated by traversing a graph composed of weighted edges representing transitions to degraded network topologies. Numerical results indicate that the Markov model can produce cascades with qualitative power-law properties similar to those observed in empirical cascades.
Kinetic Monte Carlo, rare events, Cascading Failur↗
Cyote-attack Chain Estimator
Attack Chain Estimator (ACE) Application Overview The Attack Chain Estimator (ACE) Application is a sophisticated tool designed for the ingestion, classification, sequencing, and enrichment of cybersecurity threat reports. This application leverages advanced machine learning models and extensive historical data to provide comprehensive insights into cyber threats, specifically targeting Industrial Control Systems (ICS). Purpose The primary functions of the ACE Application include: Ingestion of Cybersecurity Threat Reporting: Capable of ingesting text-based threat reports in markdown or text file format. Supports ingestion of structured data from other sources in STIX/JSON format. Classification of Report’s Text-Based Events: Utilizes a DeBERTa classifier, specifically trained on cybersecurity data, to map the events to MITRE ATT&CK for ICS Tactics and Techniques. Classification is performed using multiple Jupyter notebooks and machine learning workflows hosted as FastAPI microservices: regex_data deberta_base_35_train_hft_classifier_mlflow.ipynb hft_regex_classifier_mlflow.ipynb param_train_hft_classifier_mlflow.ipynb regex_tactic_tech.ipynb Ordering of Tactics, Techniques, and Observable Events: Sequences the identified tactics, techniques, and events to form a coherent attack chain. Enrichment with Historical Attack Chain Details: Enhances the attack chain with details from historical attacks using a Markov model developed from CyOTE Precursor Analysis Report data. The Markov model is available as a FastAPI endpoint for seamless integration. Enrichment with Adversary Emulation Capabilities Data: Integrates adversary emulation capabilities data using MITRE Caldera for OT adversary abilities UUIDs. Export of Output Files: Provides options to export the enriched attack chain in JSON or CSV formats. Routing of Output to Other Applications: Facilitates routing of output to various platforms and applications, including: Threat Intelligence Platforms COREII Scout for Threat Intelligence Analysis COREII Modeling and Simulation for Adversary Emulation Technical Description The ACE Application is an advanced cybersecurity tool designed to provide detailed threat analysis and sequence generation. It is built on a robust architecture that integrates natural language processing, machine learning, and historical data modeling. Key Components: Data Ingestion Module: Handles the input of threat reports and data from various formats, ensuring flexibility in data sources. Classification Engine: Employs DeBERTa-based classifiers hosted as FastAPI microservices to analyze and classify threat report events in accordance with the MITRE ATT&CK framework for ICS. Sequence Generator: Orders the classified events into a logical attack chain, providing clear insight into the sequence of tactics and techniques used in the threat. Enrichment Engine: Integrates historical data and adversary emulation capabilities to enhance the attack chain with valuable context and additional details. The historical data enrichment is powered by a Markov model, which is available as a FastAPI endpoint. Export and Routing Module: Facilitates the export of the enriched attack chain in multiple formats and routes the output to designated applications for further analysis or emulation.
Paul, Tony [Idaho National Laboratory (INL), Idaho↗
Reinforcement learning for adaptive maintenance policy optimization under imperfect knowledge of the system degradation model and partial observability of system states
Maintenance policy optimization usually is faced with challenges that arise from an imperfect knowledge of system degradation models and from the partial observability of system degradation states. Here, this paper proposes a reinforcement learning method to address these two challenges for a class of maintenance problems with Markov degradation processes. The reinforcement learning approach consists of a learning component and a planning component. Using sequentially collected observations, at each step of decision-making the learning component improves the knowledge of system degradation in terms of the probability distributions of the transition rates based on sequential Bayesian inference. Using the updated transition rates, at each step of decision-making the maintenance policy optimization problem is then formulated as a partially observable Markov decision problem, and the planning component computes the optimal maintenance policy that maximizes the expected cumulative reward. The proposed method is illustrated using a numerical example with repair and inspection maintenance actions. The result shows that as more observations are collected, the learning component progressively learns the true system degradation process, and the planning component adjusts the optimal maintenance policy accordingly as well, which leads to increased reward.
42 ENGINEERING↗
Operation Optimization using Reinforcement Learning with Integrated Artificial Reasoning Framework
In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.
99 GENERAL AND MISCELLANEOUS↗
Approximating Nash Equilibrium in Day-ahead Electricity Market Bidding with Multi-agent Deep Reinforcement Learning
In this paper, a day-ahead electricity market bidding problem with multiple strategic generation company (GEN-CO) bidders is studied. The problem is formulated as a Markov game model, where GENCO bidders interact with each other todevelop their optimal day-ahead bidding strategies. Considering unobservable information in the problem, a model-free and data-driven approach, known as multi-agent deep deterministic policy gradient (MADDPG), is applied for approximating the Nash equilibrium (NE) in the above Markov game. The MADDPG algorithm has the advantage of generalization due to the automatic feature extraction ability of the deep neural networks. The algorithm is tested on an IEEE 30-bus system with three competitive GENCO bidders in both an uncongested caseand a congested case. Comparisons with a truthful bidding strategy and state-of-the-art deep reinforcement learning methods including deep Q network and deep deterministic policy gradient (DDPG) demonstrate that the applied MADDPG algorithm can find a superior bidding strategy for all the market participants with increased profit gains. In addition, the comparison with a conventional model-based method shows that the MADDPG algorithm has higher computational efficiency, which is feasible for real-world applications.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Intrepid MCMC: Metropolis-Hastings with exploration
In engineering examples, one often encounters the need to sample from unnormalized distributions with complex shapes that may also be implicitly defined through a physical or numerical simulation model, making it computationally expensive to evaluate the associated density function. For such cases, MCMC has proven to be an invaluable tool. Random-walk Metropolis Methods (also known as Metropolis-Hastings (MH)), in particular, are highly popular for their simplicity, flexibility, and ease of implementation. However, most MH algorithms suffer from significant limitations when attempting to sample from distributions with multiple modes (particularly disconnected ones). Here, in this paper, we present Intrepid MCMC - a novel MH scheme that utilizes a simple coordinate transformation to significantly improve the mode-finding ability and convergence rate to the target distribution of random-walk Markov chains while retaining most of the simplicity of the vanilla MH paradigm. Through multiple examples, we showcase the improvement in the performance of Intrepid MCMC over vanilla MH for a wide variety of target distribution shapes. We also provide an analysis of the mixing behavior of the Intrepid Markov chain, as well as the efficiency of our algorithm for increasing dimensions. A thorough discussion is presented on the practical implementation of the Intrepid MCMC algorithm. Finally, its utility is highlighted through a Bayesian parameter inference problem for a two-degree-of-freedom oscillator under free vibration.
97 - MATHEMATICS AND COMPUTING↗
Identification of hot water end-use process of electric water heaters from energy measurements
This paper presents an algorithm for the identification of parameters for a stochastic hot water end-use process that drives a homogeneous population of thermostatically controlled electric water heaters (EWH). Usually, only metered interval consumption data (kWh) is collected and the hot water end-use process is unobservable to utility and aggregators. However, the availability of EWHs for demand response (DR) is closely coupled with the hot water end-use process. In this context, the hot water end-use process is modeled as a two-state Markov chain (Use / No use), which causes the thermostatic ON-OFF switching process to behave as a Markov renewal process (MRP). A set of first passage-time problems is developed to obtain the moments of the transition probability densities of the MRP. These problems are addressed by establishing a system of coupled partial differential equations characterizing the temperature evolution of the EWH population. A key quantity in the methodology for estimating the parameters is the total time an EWH is ON within a period of interest. It is referred to as the total busy time. Total busy time in this approach is a random variable for which analytical expressions of the moments are developed as a function of the metered window length. The latter expressions become the basis of a hot water demand model identification algorithm which is validated using agent-based simulations of EWHs.
42 ENGINEERING↗
Discovering causal structure with reproducing-kernel Hilbert space ε -machines
We merge computational mechanics’ definition of causal states (predictively equivalent histories) with reproducing-kernel Hilbert space (RKHS) representation inference. The result is a widely applicable method that infers causal structure directly from observations of a system’s behaviors whether they are over discrete or continuous events or time. A structural representation—a finite- or infinite-state kernel ϵ-machine—is extracted by a reduced-dimension transform that gives an efficient representation of causal states and their topology. In this way, the system dynamics are represented by a stochastic (ordinary or partial) differential equation that acts on causal states. We introduce an algorithm to estimate the associated evolution operator. Paralleling the Fokker–Planck equation, it efficiently evolves causal-state distributions and makes predictions in the original data space via an RKHS functional mapping. We demonstrate these techniques, together with their predictive abilities, on discrete-time, discrete-value infinite Markov-order processes generated by finite-state hidden Markov models with (i) finite or (ii) uncountably infinite causal states and (iii) continuous-time, continuous-value processes generated by thermally driven chaotic flows. The method robustly estimates causal structure in the presence of varying external and measurement noise levels and for very high-dimensional data.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Reaction–drift–diffusion models from master equations: application to material defects
We present a general method to produce well-conditioned continuum reaction–drift–diffusion equations directly from master equations on a discrete, periodic state space. We assume the underlying data to be kinetic Monte Carlo models (i.e. continuous-time Markov chains) produced from atomic sampling of point defects in locally periodic environments, such as perfect lattices, ordered surface structures or dislocation cores, possibly under the influence of a slowly varying external field. Our approach also applies to any discrete, periodic Markov chain. Here, the analysis identifies a previously omitted non-equilibrium drift term, present even in the absence of external forces, which can compete in magnitude with the reaction rates, thus being essential to correctly capture the kinetics. To remove fast modes which hinder time integration, we use a generalized Bloch relation to efficiently calculate the eigenspectrum of the master equation. A well conditioned continuum equation then emerges by searching for spectral gaps in the long wavelength limit, using an established kinetic clustering algorithm to define a proper reduced, Markovian state space.
36 MATERIALS SCIENCE↗
Network Reconfiguration for Enhanced Operational Resilience Using Reinforcement Learning
This paper proposes a reinforcement learning-based approach for distribution network reconfiguration(DNR) to enhance the resilience of the electric power supply. Resilience enhancements usually require solving large-scale stochastic optimization problems that are computationally expensive and sometimes infeasible. The exceptional performance of reinforcement learning techniques has encouraged their adoption in various power system control studies, specifically resilience-based real-time applications. In this paper, a single agent framework is developed using an Actor-Critic algorithm (ACA) to determine statuses of tie-switches in a distribution feeder impacted by an extreme weather event. The proposed approach provides a fast-acting control algorithm that reconfigures the feeder topology to reduce or even avoid load shedding. The problem is formulated as a discrete Markov decision process in such a way that a system state captures the system topology and its operational characteristics. An action is made to open or close a specific set of tie-switches after which a reward is calculated to evaluate the practicality and advantage of that action. The iterative Markov process is used to train the proposed ACA under diverse failure scenarios and is demonstrated on the 33-node distribution feeder system. Results show the capability of the proposed ACA to determine proper switching action of tie-switches with accuracy exceeding 93%.
actor critic↗
A Deep Generative Model for Non-Intrusive Identification of EV Charging Profiles
The proliferation of electric vehicles (EVs) brings environmental benefits and technical challenges to power grids. An identification algorithm which can accurately extract individual EV charging profiles out of widely available smart meter measurements has attracted great interests. This paper proposes a non-intrusive identification framework for EV charging profile extraction, which is driven by deep generative models (DGM). First, the proposed DGM is designed as a representation layer embedded into the Markov process and used to model the joint probability distribution of available time-series data. A novel contribution is to approximate posterior distributions by neural networks whose parameters are obtained by variational inference and supervised learning. Second, the EV charging status is inferred from the DGM via dynamic programming. Lastly, the desired EV charging profile can be reconstructed by the rated power of EV models and inferred status. Compared with the benchmark Hidden Markov Models, the proposed framework can better handle noise in data with less computational complexity and better overall accuracy performances with smaller recall. The proposed framework is validated by numerical experiments on the Pecan Street dataset.
33 ADVANCED PROPULSION SYSTEMS↗
Joint Estimation of Behind-the-Meter Solar Generation in a Community
Distribution grid planning, control, and optimization require accurate estimation of solar photovoltaic (PV) generation and electric load in the system. Most of the small residential solar PV systems are installed behind-the-meter making only the net load readings available to the utilities. This paper presents an unsupervised framework for joint disaggregation of the net load readings of a group of customers into the solar PV generation and electric load. Our algorithm synergistically combines a physical PV system performance model for individual solar PV generation estimation with a statistical model for joint load estimation. The electric loads for a group of customers are estimated jointly by a mixed hidden Markov model (MHMM) which enables modeling the general load consumption behavior present in all customers while acknowledging the individual differences. At the same time, the model can capture the change in load patterns over a time period by the hidden Markov states. The proposed algorithm is also capable of estimating the key technical parameters of the solar PV systems. Our proposed method is evaluated using the net load, electric load, and solar PV generation data gathered from residential customers located in Austin, Texas. Testing results show that our proposed method reduces the mean squared error of state-of-the-art net-load disaggregation algorithms by 67%.
behind-the-meter solar generation↗