Spiking Neural Algorithms for Markov Process Random Walk.
Abstract not provided.
Severa, William Mark↗
SEARCH · Search NASA
Results for “Markov process”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Reduced Equations of Motion for Quantum Systems Driven by Diffusive Markov Processes
Not Available
Sarovar, Mohan↗
Pitch angle scattering of relativistic electrons from stationary magnetic waves: Continuous Markov process and quasilinear theory
We develop a Markov process theory of charged particle scattering from stationary, transverse, magnetic waves. We examine approximations that lead to quasilinear theory, in particular the resonant diffusion approximation. We find that, when appropriate, the resonant diffusion approximation simplifies the result of the weak turbulence approximation without significant further restricting the regime of applicability. We also explore a theory generated by expanding drift and diffusion rates in terms of a presumed small correlation time. This small correlation time expansion leads to results valid for relatively small pitch angle and large wave energy density - a regime that may govern pitch angle scattering of high-energy electrons into the geomagnetic loss cone.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Optimal Control of Markov Processes with Age-Dependent Transition Rates
We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterise the value function and optimal controls for both discounted and average cost cases.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Series of semi-Markov processes to model infrastructure resilience under multihazards
Civil infrastructure systems are subjected to multiple hazards, including natural and anthropogenic, that disrupt their function or the level of service offered. Estimating the function recovery of these systems (or how soon normalcy of operations will be restored) when subjected to repeated hazard events by considering the inter-event dependencies is an important problem in multihazard infrastructure resilience. However, this problem has been less addressed in the field. This paper proposes a series of semi-Markov processes model to capture the inter-event dependencies in infrastructure recovery when subjected to successive hazard events. Recovery after each new hazard event is represented by a unique semi-Markov process that models the reduced recovery rates and the increased recovery times caused by the system’s incomplete recovery from the preceding event. Two novel formulations of the inter-event dependency modeling, namely Maximal Effects Dependency (considers the worst impact of two successive hazard events) and Cumulative Effects Dependency (considers the aggregated impacts of two successive hazard events), are proposed and discussed. The model is demonstrated by considering the following applications: Three-state system subjected to deterministic and random occurrences of identical hazard events; and Multihazard resilience of a building in Charleston, SC, considering earthquake and hurricane hazards. Results indicate that considering inter-event dependencies in recovery modeling can lead to lesser-predicted resilience, thereby affecting resilience-based decision-making.
42 ENGINEERING↗
Markov and semi-Markov processes as a failure rate
In this paper the reliability function is defined by the stochastic failure rate process with a non negative and right continuous trajectories. Equations for the conditional reliability functions of an object, under assumption that the failure rate is a semi-Markov process with an at most countable state space are derived. A proper theorem is presented. The linear systems of equations for the appropriate Laplace transforms allow to find the reliability functions for the alternating, the Poisson and the Furry-Yule failure rate processes.
97 MATHEMATICS AND COMPUTING↗
First Passage Moments of Finite-State Semi-Markov Processes
In this paper, we discuss the computation of first-passage moments of a regular time-homogeneous semi-Markov process (SMP) with a finite state space to certain of its states that possess the property of universal accessibility (UA). A UA state is one which is accessible from any other state of the SMP, but which may or may not connect back to one or more other states. An important characteristic of UA is that it is the state-level version of the oft-invoked process-level property of irreducibility. We adapt existing results for irreducible SMPs to the derivation of an analytical matrix expression for the first passage moments to a single UA state of the SMP. In addition, consistent point estimators for these first passage moments, together with relevant R code, are provided.
97 MATHEMATICS AND COMPUTING↗
Markov Process to Evaluate the Value Proposition of a Risk-Informed Predictive Maintenance Strategy
To achieve high-capacity factors, the nuclear fleet has relied on labor-intensive and time-consuming operation and preventive maintenance programs for plant systems. Manually performed inspection, calibration, testing, and maintenance of plant assets at periodic frequencies, along with time-based replacement of assets irrespective of condition, have resulted in a costly, labor-centric business model. Fortunately, there are technologies that can eliminate unnecessary preventive maintenance activities by deploying risk-informed predictive maintenance, enabling the transition to a technology-centric business model. The technology-centric business model will enable plants to optimize and automate maintenance activities, leading to cost reductions since labor is a rising cost and technology is a declining cost. The implementation of scalable technologies and methodologies across plant systems and across the nuclear fleet is critical for successful deployment of a risk-informed predictive maintenance strategy at commercial nuclear power plants. The work presented in the report is being developed as part of a collaborative research effort between Idaho National Laboratory and Public Service Enterprise Group Nuclear, LLC. This report describes the technical basis using the Markov Process to evaluate the value proposition for the risk-informed predictive maintenance strategy for the circulating water system. The plant process data from the Salem nuclear plant’s circulating water system is utilized to develop a Markov chain risk models and formulation to estimate the loss and gain in revenue based on plant availability. The outcomes presented in this report provides the technical basis for extensive quantitative evaluation of a scalable risk-informed predictive maintenance strategy as part of future research.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Efficient analysis of stochastic gene dynamics in the non-adiabatic regime using piecewise deterministic Markov processes
Single-cell experiments show that gene expression is stochastic and bursty, a feature that can emerge from slow switching between promoter states with different activities. In addition to slow chromatin and/or DNA looping dynamics, one source of long-lived promoter states is the slow binding and unbinding kinetics of transcription factors to promoters, i.e. the non-adiabatic binding regime. Here, we introduce a simple analytical framework, known as a piecewise deterministic Markov process (PDMP), that accurately describes the stochastic dynamics of gene expression in the non-adiabatic regime. We illustrate the utility of the PDMP on a non-trivial dynamical system by analysing the properties of a titration-based oscillator in the non-adiabatic limit. We first show how to transform the underlying chemical master equation into a PDMP where the slow transitions between promoter states are stochastic, but whose rates depend upon the faster deterministic dynamics of the transcription factors regulated by these promoters. We show that the PDMP accurately describes the observed periods of stochastic cycles in activator and repressor-based titration oscillators. We then generalize our PDMP analysis to more complicated versions of titration-based oscillators to explain how multiple binding sites lengthen the period and improve coherence. Finally, we show how noise-induced oscillation previously observed in a titration-based oscillator arises from non-adiabatic and discrete binding events at the promoter site.
59 BASIC BIOLOGICAL SCIENCES↗
Projected metastable Markov processes and their estimation with observable operator models
The determination of kinetics of high-dimensional dynamical systems, such as macromolecules, polymers, or spin systems, is a difficult and generally unsolved problem — both in simulation, where the optimal reaction coordinate(s) are generally unknown and are difficult to compute, and in experimental measurements, where only specific coordinates are observable. Markov models, or Markov state models, are widely used but suffer from the fact that the dynamics on a coarsely discretized state spaced are no longer Markovian, even if the dynamics in the full phase space are. The recently proposed projected Markov models (PMMs) are a formulation that provides a description of the kinetics on a low-dimensional projection without making the Markovianity assumption. However, as yet no general way of estimating PMMs from data has been available. Here, we show that the observed dynamics of a PMM can be exactly described by an observable operator model (OOM) and derive a PMM estimator based on the OOM learning.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov's Method
We aim at characterizing domains of attraction for controlled piecewise deterministic processes using an occupational measure formulation and Zubov's approach. Firstly, we provide linear programming (primal and dual) formulations of discounted, infinite horizon control problems for PDMPs. These formulations involve an infinite-dimensional set of probability measures and are obtained using viscosity solutions theory. Secondly, these tools allow to construct stabilizing measures and to avoid the assumption of stability under concatenation for controls. The domain of controllability is then characterized as some level set of a convenient solution of the associated Hamilton-Jacobi integral-differential equation. The theoretical results are applied to PDMPs associated to stochastic gene networks. Explicit computations are given for Cook's model for gene expression.
97 MATHEMATICS AND COMPUTING↗
Efficient simulation of quantum systems driven by diffusive Markov processes.
Abstract Not Provided
Sarovar, Mohan↗
K-Spin Hamiltonian for Quantum-Resolvable Markov Decision Processes
The Markov decision process is the mathematical formalization underlying the modern field of reinforcement learning when transition and reward functions are unknown. We derive a pseudo-Boolean cost function that is equivalent to a K-spin Hamiltonian representation of the discrete, finite, discounted Markov decision process with infinite horizon. This K-spin Hamiltonian furnishes a starting point from which to solve for an optimal policy using heuristic quantum algorithms such as adiabatic quantum annealing and the quantum approximate optimization algorithm on near-term quantum hardware. In arguing that the variational minimization of our Hamiltonian is approximately equivalent to the Bellman optimality condition for a prevalent class of environments we establish an interesting analogy with classical field theory. Along with proof-of-concept calculations to corroborate our formulation by simulated and quantum annealing against classical Q-Learning, we analyze the scaling of physical resources required to solve our Hamiltonian on quantum hardware.
Hamiltonian↗
The exit-time problem for a Markov jump process
The purpose of this paper is to consider the exit-time problem for a finite-range Markov jump process, i.e, the distance the particle can jump is bounded independent of its location. Additionally, such jump diffusions are expedient models for anomalous transport exhibiting super-diffusion or nonstandard normal diffusion. We refer to the associated deterministic equation as a volume-constrained nonlocal diffusion equation. The volume constraint is the nonlocal analogue of a boundary condition necessary to demonstrate that the nonlocal diffusion equation is well-posed and is consistent with the jump process. A critical aspect of the analysis is a variational formulation and a recently developed nonlocal vector calculus. Finally, this calculus allows us to pose nonlocal backward and forward Kolmogorov equations, the former equation granting the various moments of the exit-time distribution.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Analysis of surface and seismic sources in dense array data with match field processing and Markov chain Monte Carlo sampling
Not provided.
Geochemistry & Geophysics↗
Markov Jump Processes Approximating a Non-Symmetric Generalized Diffusion
Consider a non-symmetric generalized diffusion X( Dot-Operator ) in Double-Struck-Capital-R {sup d} determined by the differential operator A(x) = -{Sigma}{sub ij} {partial_derivative}{sub i}a{sub ij}(x){partial_derivative}{sub j} + {Sigma}{sub i} b{sub i}(x){partial_derivative}{sub i}. In this paper the diffusion process is approximated by Markov jump processes X{sub n}( Dot-Operator ), in homogeneous and isotropic grids G{sub n} Subset-Of Double-Struck-Capital-R {sup d}, which converge in distribution in the Skorokhod space D([0,{infinity}), Double-Struck-Capital-R {sup d}) to the diffusion X( Dot-Operator ). The generators of X{sub n}( Dot-Operator ) are constructed explicitly. Due to the homogeneity and isotropy of grids, the proposed method for d{>=}3 can be applied to processes for which the diffusion tensor {l_brace}a{sub ij}(x){r_brace}{sub 11}{sup dd} fulfills an additional condition. The proposed construction offers a simple method for simulation of sample paths of non-symmetric generalized diffusion. Simulations are carried out in terms of jump processes X{sub n}( Dot-Operator ). For piece-wise constant functions a{sub ij} on Double-Struck-Capital-R {sup d} and piece-wise continuous functions a{sub ij} on Double-Struck-Capital-R {sup 2} the construction and principal algorithm are described enabling an easy implementation into a computer code.
97 MATHEMATICS AND COMPUTING↗
Spiking Markov Reward Process v.0.1
SAND2024-11150O The Spiking Markov Reward Process software is a spiking neural network that streams binary arithmetic and computes the state value function of a Markov reward process. The software will be released to the SpiNNcloud group for development of neuromorphic acceleration. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
Wang, Felix↗
Conditions for the Solvability of the Linear Programming Formulation for Constrained Discounted Markov Decision Processes
We consider a discrete-time constrained discounted Markov decision process (MDP) with Borel state and action spaces, compact action sets, and lower semi-continuous cost functions. We introduce a set of hypotheses related to a positive weight function which allow us to consider cost functions that might not be bounded below by a constant, and which imply the solvability of the linear programming formulation of the constrained MDP. In particular, we establish the existence of a constrained optimal stationary policy. Our results are illustrated with an application to a fishery management problem.
97 MATHEMATICS AND COMPUTING↗