Search NASA⌕ Search

SEARCH · Search NASA

Results for “Markov process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Automatic treatment of the variance estimation bias in TRIPOLI-4 criticality calculations

The central limit (CLT) theorem States conditions under which the mean of a sufficiently large number of independent random variables, each with finite mean and variance, will be approximately normally distributed. The use of Monte Carlo transport codes, such as Tripoli4, relies on those conditions. While these are verified in protection applications (the cycles provide independent measurements of fluxes and related quantities), the hypothesis of independent estimates/cycles is broken in criticality mode. Indeed the power iteration technique used in this mode couples a generation to its progeny. Often, after what is called 'source convergence' this coupling almost disappears (the solution is closed to equilibrium) but for loosely coupled systems, such as for PWR or large nuclear cores, the equilibrium is never found, or at least may take time to reach, and the variance estimation such as allowed by the CLT is under-evaluated. In this paper we first propose, by the mean of two different methods, to evaluate the typical correlation length, as measured in cycles number, and then use this information to diagnose correlation problems and to provide an improved variance estimation. Those two methods are based on Fourier spectral decomposition and on the lag k autocorrelation calculation. A theoretical modeling of the autocorrelation function, based on Gauss-Markov stochastic processes, will also be presented. Tests will be performed with Tripoli4 on a PWR pin cell. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Deep Reinforcement Learning for Distribution System Operations: A Tutorial and Survey

Here, the rapid evolution of modern electric power distribution systems into complex networks of interconnected active devices, distributed generation (DG), and storage poses increasing difficulties for system operators. The large-scale integration of distributed energy resources (DERs) and the rapid exchange of measurement data via communication networks present major opportunities for advancing grid operations but also introduce greater uncertainty, higher data dimensionality, more complex network and device models, and challenging control and optimization problems. Deep reinforcement learning (DRL) algorithms are promising in addressing these challenges. However, they have not been effectively adapted for power systems applications, requiring extensive customization for implementation and evaluation. This has resulted in reproducibility challenges and a steep learning curve for researchers new to applying DRL algorithms to the power systems domain. To bridge these gaps, this tutorial aims to serve as a valuable resource for researchers interested in exploring learning-based algorithms to operate active power distribution networks. Specifically, this work presents a generalized process for translating sequential decision-making problems in power distribution systems into Markov decision process (MDP) formulations, illustrated through concrete grid service examples. Additionally, we introduce a simple environment design strategy to develop and evaluate example DRL algorithms for distribution system applications, complete with an included code repository to guide users through environment construction.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Cosmic Ray Propagation through the Magnetic Fields of the Galaxy with Extended Halo

In this project we perform theoretical studies of 3-dimensional cosmic ray propagation in magnetic field configurations of the Galaxy with an extended halo. We employ our newly developed Markov stochastic process methods to solve the diffusive cosmic ray transport equation. We seek to understand observations of cosmic ray spectra, composition under the constraints of the observations of diffuse gamma ray and radio emission from the Galaxy. The model parameters are directly are related to properties of our Galaxy, such as the size of the Galactic halo, particle transport in Galactic magnetic fields, distribution of interstellar gas, primary cosmic ray source distribution and their confinement in the Galaxy. The core of this investigation is the development of software for cosmic ray propagation models with the Markov stochastic process approach. Values of important model parameters for the halo diffusion model are examined in comparison with observations of cosmic ray spectra, composition and the diffuse gamma-ray background. This report summarizes our achievement in the grant period at the Florida Institute of Technology. Work at the co-investigator's institution, the University of New Hampshire, under a companion grant, will be covered in detail by a separate report.

Zhang, Ming↗

Contribution of surface plasmon decay to secondary electron emission from an Al surface

Spectra of secondary electrons (SE) emitted from a polycrystalline Al surface have been measured in coincidence with 500 eV-electrons for energy losses between 10 and 155 eV. The spectra for a given energy loss are qualitatively similar, consisting of surface and volume plasmon decay and a contribution attributable to direct electron-electron scattering. The similarity of the contribution of surface and volume plasmon decay in the SE spectra proves directly that electron multiple scattering is governed by a Markov-type process. The average value of the surface plasmon decay contribution to the SE spectrum amounts to {approx}25%.

36 MATERIALS SCIENCE↗

Mappings of open quantum systems onto chain representations and Markovian embeddings

We study systems coupled linearly to a bath of oscillators. In an iterative process, the bath is transformed into a chain of oscillators with nearest neighbour interactions. A systematic procedure is provided to obtain the spectral density of the residual bath in each step, and it is shown that under general conditions these data converge. That is, the asymptotic part of the chain is universal, translation invariant with semicircular spectral density. The methods are based on orthogonal polynomials, in which we also solve the outstanding so-called “sequence of secondary measures problem” and give them a physical interpretation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A simple stochastic quadrant model for the transport and deposition of particles in turbulent boundary layers

We present a simple stochastic quadrant model for calculating the transport and deposition of heavy particles in a fully developed turbulent boundary layer based on the statistics of wall-normal fluid velocity fluctuations obtained from a fully developed channel flow. Individual particles are tracked through the boundary layer via their interactions with a succession of random eddies found in each of the quadrants of the fluid Reynolds shear stress domain in a homogeneous Markov chain process. In this way, we are able to account directly for the influence of ejection and sweeping events as others have done but without resorting to the use of adjustable parameters. Deposition rate predictions for a wide range of heavy particles predicted by the model compare well with benchmark experimental measurements. In addition, deposition rates are compared with those obtained from continuous random walk models and Langevin equation based ejection and sweep models which noticeably give significantly lower deposition rates. Various statistics related to the particle near wall behavior are also presented. Finally, we consider the model limitations in using the model to calculate deposition in more complex flows where the near wall turbulence may be significantly different.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

SU-D-BRB-05: Quantum Learning for Knowledge-Based Response-Adaptive Radiotherapy

Purpose: There is tremendous excitement in radiotherapy about applying data-driven methods to develop personalized clinical decisions for real-time response-based adaptation. However, classical statistical learning methods lack in terms of efficiency and ability to predict outcomes under conditions of uncertainty and incomplete information. Therefore, we are investigating physics-inspired machine learning approaches by utilizing quantum principles for developing a robust framework to dynamically adapt treatments to individual patient’s characteristics and optimize outcomes. Methods: We studied 88 liver SBRT patients with 35 on non-adaptive and 53 on adaptive protocols. Adaptation was based on liver function using a split-course of 3+2 fractions with a month break. The radiotherapy environment was modeled as a Markov decision process (MDP) of baseline and one month into treatment states. The patient environment was modeled by a 5-variable state represented by patient’s clinical and dosimetric covariates. For comparison of classical and quantum learning methods, decision-making to adapt at one month was considered. The MDP objective was defined by the complication-free tumor control (P{sup +}=TCPx(1-NTCP)). A simple regression model represented state-action mapping. Single bit in classical MDP and a qubit of 2-superimposed states in quantum MDP represented the decision actions. Classical decision selection was done using reinforcement Q-learning and quantum searching was performed using Grover’s algorithm, which applies uniform superposition over possible states and yields quadratic speed-up. Results: Classical/quantum MDPs suggested adaptation (probability amplitude ≥0.5) 79% of the time for splitcourses and 100% for continuous-courses. However, the classical MDP had an average adaptation probability of 0.5±0.22 while the quantum algorithm reached 0.76±0.28. In cases where adaptation failed, classical MDP yielded 0.31±0.26 average amplitude while the quantum approach averaged a more optimistic 0.57±0.4, but with high phase fluctuations. Conclusion: Our results demonstrate that quantum machine learning approaches provide a feasible and promising framework for real-time and sequential clinical decision-making in adaptive radiotherapy.

60 APPLIED LIFE SCIENCES↗

Polychromatic Arm Exponents for the Critical Planar FK-Ising Model

Schramm–Loewner evolution (SLE) is a one-parameter family of random planar curves introduced by Schramm in 1999 as the candidates for the scaling limits of the interfaces in the planar critical lattice models. This is the only possible process with conformal invariance and a certain “domain Markov property”. In 2010, Chelkak and Smirnov proved the conformal invariance of the scaling limits of the critial planar FK-Ising model which gave the convergence of the interface to SLE{sub 16/3}. We derive the arm exponents of SLE{sub κ} for κ∈(4,8). Combining with the convergence of the interface, we derive the arm exponents of the critical FK-Ising model. We obtain six different patterns of boundary arm exponents and three different patterns of interior arm exponents of the critical planar FK-Ising model on the square lattice.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.

97 MATHEMATICS AND COMPUTING↗

Hybrid pathwise sensitivity methods for discrete stochastic models of chemical reaction systems

Stochastic models are often used to help understand the behavior of intracellular biochemical processes. The most common such models are continuous time Markov chains (CTMCs). Parametric sensitivities, which are derivatives of expectations of model output quantities with respect to model parameters, are useful in this setting for a variety of applications. In this paper, we introduce a class of hybrid pathwise differentiation methods for the numerical estimation of parametric sensitivities. The new hybrid methods combine elements from the three main classes of procedures for sensitivity estimation and have a number of desirable qualities. First, the new methods are unbiased for a broad class of problems. Second, the methods are applicable to nearly any physically relevant biochemical CTMC model. Third, and as we demonstrate on several numerical examples, the new methods are quite efficient, particularly if one wishes to estimate the full gradient of parametric sensitivities. The methods are rather intuitive and utilize the multilevel Monte Carlo philosophy of splitting an expectation into separate parts and handling each in an efficient manner.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Probabilistic risk analysis toward cost-effective 3S (safety, safeguards, security) implementation

Probabilistic Risk Analysis (PRA) has been introduced for several decades in safety and nuclear advanced countries have already used this methodology in their own regulatory systems. However, PRA has not been developed in safeguards and security so far because of inherent difficulties in intentional and malicious acts. In this paper, probabilistic proliferation and risk analysis based on random process is applied to hypothetical reprocessing process and physical protection system in nuclear reactor with the Markov model that was originally developed by the Proliferation Resistance and Physical Protection Working Group (PRPPWG) in Generation IV International Framework (GIF). Through the challenge to quantify the security risk with a frequency in this model, integrated risk notion among 3S to pursue the cost-effective installation of those countermeasures is discussed in a heroic manner.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Scientific discovery in a model-centric framework: Reproducibility, innovation, and epistemic diversity

Consistent confirmations obtained independently of each other lend credibility to a scientific result. We refer to results satisfying this consistency as reproducible and assume that reproducibility is a desirable property of scientific discovery. Yet seemingly science also progresses despite irreproducible results, indicating that the relationship between reproducibility and other desirable properties of scientific discovery is not well understood. These properties include early discovery of truth, persistence on truth once it is discovered, and time spent on truth in a long-term scientific inquiry. We build a mathematical model of scientific discovery that presents a viable framework to study its desirable properties including reproducibility. In this framework, we assume that scientists adopt a model-centric approach to discover the true model generating data in a stochastic process of scientific discovery. We analyze the properties of this process using Markov chain theory, Monte Carlo methods, and agent-based modeling. We show that the scientific process may not converge to truth even if scientific results are reproducible and that irreproducible results do not necessarily imply untrue results. The proportion of different research strategies represented in the scientific population, scientists’ choice of methodology, the complexity of truth, and the strength of signal contribute to this counter-intuitive finding. Important insights include that innovative research speeds up the discovery of scientific truth by facilitating the exploration of model space and epistemic diversity optimizes across desirable properties of scientific discovery.

97 MATHEMATICS AND COMPUTING↗

An Online Search Method for Representative Risky Fault Chains Based on Reinforcement Learning and Knowledge Transfer

In the analysis of cascading outages and blackouts in power systems, risky cascading fault chains should be accurately identified in order to do further block or alleviate blackouts. However, the huge computational burden makes online analysis difficult. In this paper, an online search method for representative risky fault chains based on reinforcement learning and knowledge transfer is proposed. This method aims at promoting efficiency by exploiting similarities of adjacent power flow snapshots in operations. After the “representative risky fault chain” is defined, a framework of tree search based on Markov Decision Process and Q-learning is constructed. The knowledge in past runs is accumulated offline and then applied online, with a mechanism of knowledge transition and extension. The proposed learning based approach is verified on an illustrative 39-bus system with different loading levels, and simulations are carried out on a realworld 1000-bus power grid in China to show the effectiveness and efficiency of the proposed approach.

Blackout↗

Assessing the pollution risk of a groundwater source field at western Laizhou Bay under seawater intrusion

Coastal areas have great significance for human living, economy and society development in the world. With the rapid increase of pressures from human activities and climate change, the safety of groundwater resource is under the threat of seawater intrusion in coastal areas. The area of Laizhou Bay is one of the most serious seawater intruded areas in China, since seawater intrusion phenomenon was firstly recognized in the middle of 1970s. This study assessed the pollution risk of a groundwater source filed of western Laizhou Bay area by inferring the probability distribution of groundwater Cl{sup −} concentration. The numerical model of seawater intrusion process is built by using SEAWAT4. The parameter uncertainty of this model is evaluated by Markov Chain Monte Carlo (MCMC) simulation, and DREAM{sub (ZS)} is used as sampling algorithm. Then, the predictive distribution of Cl{sup -} concentration at groundwater source field is inferred by using the samples of model parameters obtained from MCMC. After that, the pollution risk of groundwater source filed is assessed by the predictive quantiles of Cl{sup -} concentration. The results of model calibration and verification demonstrate that the DREAM{sub (ZS)} based MCMC is efficient and reliable to estimate model parameters under current observation. Under the condition of 95% confidence level, the groundwater source point will not be polluted by seawater intrusion in future five years (2015–2019). In addition, the 2.5% and 97.5% predictive quantiles show that the Cl{sup −} concentration of groundwater source field always vary between 175 mg/l and 200 mg/l. - Highlights: • The parameter uncertainty of seawater intrusion model is evaluated by MCMC. • Groundwater source field won’t be polluted by seawater intrusion in future 5 years. • The pollution risk is assessed by the predictive quantiles of Cl{sup −} concentration.

54 ENVIRONMENTAL SCIENCES↗

DOUBLE POWER LAWS IN THE EVENT-INTEGRATED SOLAR ENERGETIC PARTICLE SPECTRUM

A double power law or a power law with exponential rollover at a few to tens of MeV nucleon{sup −1} of the event-integrated differential spectra has been reported in many solar energetic particle (SEP) events. The rollover energies per nucleon of different elements correlate with a particle's charge-to-mass ratio (Q/A). The probable causes are suggested as residing in shock finite lifetimes, shock finite sizes, shock geometry, and an adiabatic cooling effect. In this work, we conduct a numerical simulation to investigate a particle's transport process in the inner heliosphere. We solve the focused transport equation using a time-backward Markov stochastic approach. The convection, magnetic focusing, adiabatic cooling effect, and pitch-angle scattering are included. The effects that the interplanetary turbulence imposes on the shape of the resulting SEP spectra are examined. By assuming a pure power-law differential spectrum at the Sun, a perfect double-power-law feature with a break energy ranging from 10 to 120 MeV nucleon{sup −1} is obtained at 1 au. We found that the double power law of the differential energy spectrum is a robust result of SEP interplanetary propagation. It works for many assumptions of interplanetary turbulence spectra that give various forms of momentum dependence of a particle's mean free path. The different spectral shapes in low-energy and high-energy ends are not just a transition from the convection-dominated propagation to diffusion-dominated propagation.

79 ASTRONOMY AND ASTROPHYSICS↗