Search NASA⌕ Search

SEARCH · Search NASA

Results for “Markov process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Lignin valorization process control under feedstock uncertainty through a dynamic stochastic programming approach

The randomness introduced by reactants is an issue when processing renewable bioresources. In this paper, we apply tools from dynamic stochastic programming theory to the biochemical process. Instead of introducing extra preprocessing units, we consider the inherent randomness of the process and optimize in expectation the performance of the system. In a general setting, this is a multistage stochastic optimization problem and we investigate its approximate solution via two approaches, namely stochastic dual dynamic programming (SDDP) and the finite state, finite action Markov decision process (MDP) framework. These two methods are implemented to a case study of lignin valorization that is crucial to a cost-effective biorefinery process using biomass as the feedstock.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Phenomenological memory-kernel master equations and time-dependent Markovian processes

Do phenomenological master equations with a memory kernel always describe a non-Markovian quantum dynamics characterized by reverse flow of information? Is the integration over the past states of the system an unmistakable signature of non-Markovianity? We show by a counterexample that this is not always the case. We consider two commonly used phenomenological integro-differential master equations describing the dynamics of a spin 1/2 in a thermal bath. By using a recently introduced measure to quantify non-Markovianity [Breuer et al., Phys. Rev. Lett. 103, 210401 (2009)] we demonstrate that as far as the equations retain their physical sense, the key feature of non-Markovian behavior does not appear in the considered memory kernel master equations. Namely, there is no reverse flow of information from the environment to the open system. Therefore, the assumption that the integration over a memory kernel always leads to a non-Markovian dynamics turns out to be vulnerable to phenomenological approximations. Instead, the considered phenomenological equations are able to describe time-dependent and unidirectional information flow from the system to the reservoir associated with time-dependent Markovian processes.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Model of superdiffusion

A locally nonequilibrium model of superdiffusion is proposed that is based on the partition of the set of diffusing particles into groups according to the flight length of these particles. The process of diffusion is described in terms of partial concentrations of particles belonging to different groups. As special limit cases, the model yields equations with fractional time derivative and the so-called porous medium equation. The basic equations of the model are Markov equations; therefore, they easily include reaction terms. The model can be applied to describing the types of diffusion in which the diffusing particles are in free flight most of the time.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Causation entropy from symbolic representations of dynamical systems

Identification of causal structures and quantification of direct information flows in complex systems is a challenging yet important task, with practical applications in many fields. Data generated by dynamical processes or large-scale systems are often symbolized, either because of the finite resolution of the measurement apparatus, or because of the need of statistical estimation. By algorithmic application of causation entropy, we investigated the effects of symbolization on important concepts such as Markov order and causal structure of the tent map. We uncovered that these quantities depend nonmonotonically and, most of all, sensitively on the choice of symbolization. Indeed, we show that Markov order and causal structure do not necessarily converge to their original analog counterparts as the resolution of the partitioning becomes finer.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e

We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at addressing the challenge of maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab). Our primary objective is to regulate the spill process to ensure a consistent intensity profile, with the ultimate goal of creating an automated controller capable of providing real-time feedback and calibration of the Spill Regulation System (SRS) parameters on a millisecond timescale. We treat the Mu2e accelerator system as a Markov Decision Process suitable for Reinforcement Learning (RL), utilizing PPO to reduce bias and enhance training stability. A key innovation in our approach is the integration of a neuralized Proportional-Integral-Derivative (PID) controller into the policy function, resulting in a significant improvement in the Spill Duty Factor (SDF) by 13.6%, surpassing the performance of the current PID controller baseline by an additional 1.6%. This paper presents the preliminary offline results based on a differentiable simulator of the Mu2e accelerator. It paves the groundwork for real-time implementations and applications, representing a crucial step towards automated proton beam intensity control for the Mu2e experiment.

43 PARTICLE ACCELERATORS↗

An Easy Algorithm to Generate Colored Noise Sequences

The need for accurate and efficient simulation of the noise background arises in statistical significance tests for periodic signals buried in colored noise. This paper discusses techniques for generating colored-noise sequences which simulate processes with a given spectral density. Matlab routines based on the fast fractional difference algorithm are presented. These routines can create various stochastic models (e.g., first-order autoregressive (AR(1)), power law (PL), autoregressive fractionally integrated moving average ((1, d, 0)), and generalized Gauss Markov (GGM)) that serve as possible candidate null hypotheses to test against in various scenarios. Allan variance and power spectral density (PSD) show that our algorithms are accurate and efficient, and can be easily implemented for stationary noise models and non-stationary PLs with spectral indices up to 2. Our algorithms can also be extended to produce a non-stationary PL with a range-limited steeper PSD (spectral indices up to about 4) using a GGM approximation with a proper break frequency. The red-noise leakage effect on the periodogram is further discussed. The result shows that our GGM approximation has a potential for alleviating the red-noise leakage in the PSD estimates. The Timmer and Koenig procedure is also included for comparison.

79 ASTRONOMY AND ASTROPHYSICS↗

Reinforcement learning for real-time process control in high-temperature superconductor manufacturing

With high efficiency and low energy loss, high-temperature superconductors (HTS) have demonstrated their profound applications in various fields, such as medical imaging, transportation, accelerators, microwave devices, and power systems. The high-field applications of HTS tapes have raised the demand for producing cost-effective tapes with long lengths in superconductor manufacturing. However, achieving the uniform and enhanced performance of a long HTS tape is challenging due to the unstable growth conditions in the manufacturing process. Although it is confirmed that the process parameters during the advanced metal organic chemical vapor deposition (A-MOCVD) process influence the uniformity of the produced HTS tapes, the high-dimensional process parameter signals and their complicated interactions make it difficult to develop an effective control policy. In this paper, we propose a local measure for the uniformity of HTS tapes to provide instant feedback for our control policy. Then, we model the manufacturing of HTS tapes as a Markov decision process (MDP) with continuous state and action spaces to assess the instant reward in real time in our feedback control model. As our MDP involves continuous and high-dimensional state and action spaces, a neural fitted Q-iteration (NFQ) algorithm is adopted to solve the MDP with artificial neural network (ANN) function approximation. The collinearity of process parameters can restrict our capability of adjusting the process parameters, which is addressed by the principal component analysis (PCA) in our method. The control policy adjusts the PCA of process parameters using the NFQ algorithm. In conclusion, based on our case studies on real A-MOCVD dataset, the obtained control policy increases the average uniformity of tapes by 5.6% and performs especially well on sample HTS tapes with a low uniformity.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Stochastic Games for Continuous-Time Jump Processes Under Finite-Horizon Payoff Criterion

In this paper we study two-person nonzero-sum games for continuous-time jump processes with the randomized history-dependent strategies under the finite-horizon payoff criterion. The state space is countable, and the transition rates and payoff functions are allowed to be unbounded from above and from below. Under the suitable conditions, we introduce a new topology for the set of all randomized Markov multi-strategies and establish its compactness and metrizability. Then by constructing the approximating sequences of the transition rates and payoff functions, we show that the optimal value function for each player is a unique solution to the corresponding optimality equation and obtain the existence of a randomized Markov Nash equilibrium. Furthermore, we illustrate the applications of our main results with a controlled birth and death system.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Tunable superconducting qudit mediated by microwave photons

We have investigated the time-domain characteristics of the Autler-Townes doublet in a superconducting circuit. The transition probabilities between the ground state and the Autler-Townes doublet states are shown to be controlled in a phase-coherent manner using a well-known microwave pulse pattern technique. The experimental results are well explained by a numerical simulation based on the Markovian master equation. Our result indicates that the Autler-Townes doublet states might be useful as a tunable qudit for implementation of quantum information processing, in particular as a multivalued quantum logic element.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Non-Markovian effect on the geometric phase of a dissipative qubit

We studied the geometric phase of a two-level atom coupled to an environment with Lorentzian spectral density. The non-Markovian effect on the geometric phase is explored analytically and numerically. In the weak coupling limit, the lowest order correction to the geometric phase is derived analytically and the general case is calculated numerically. It was found that the correction to the geometric phase is significantly large if the spectral width is small, and in this case the non-Markovian dynamics has a significant impact on the geometric phase. When the spectral width increases, the correction to the geometric phase becomes negligible, which shows the robustness of the geometric phase to the environmental white noises. The result is significant to the quantum information processing based on the geometric phase.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Non-equilibrium effects upon the non-Markovian Caldeira-Leggett quantum master equation

Highlights: > Classical Brownian motion described by a non-Markovian Fokker-Planck equation. > Quantization process. > Quantum Brownian motion described by a non-Markovian Caldeira-Leggett equation. > A non-equilibrium quantum thermal force is predicted. - Abstract: We obtain a non-Markovian quantum master equation directly from the quantization of a non-Markovian Fokker-Planck equation describing the Brownian motion of a particle immersed in a generic environment (e.g. a non-thermal fluid). As far as the especial case of a heat bath comprising of quantum harmonic oscillators is concerned, we derive a non-Markovian Caldeira-Leggett master equation on the basis of which we work out the concept of non-equilibrium quantum thermal force exerted by the harmonic heat bath upon the Brownian motion of a free particle. The classical limit (or dequantization process) of this sort of non-equilibrium quantum effect is scrutinized, as well.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DETECTION OF THERMAL EMISSION FROM A SUPER-EARTH

We report on the detection of infrared light from the super-Earth 55 Cnc e, based on four occultations obtained with Warm Spitzer at 4.5 {mu}m. Our data analysis consists of a two-part process. In a first step, we perform individual analyses of each data set and compare several baseline models to optimally account for the systematics affecting each light curve. We apply independent photometric correction techniques, including polynomial detrending and pixel mapping, that yield consistent results at the 1{sigma} level. In a second step, we perform a global Markov Chain Monte Carlo analysis, including all four data sets that yield an occultation depth of 131 {+-} 28 ppm, translating to a brightness temperature of 2360 {+-} 300 K in the IRAC 4.5 {mu}m channel. This occultation depth suggests a low Bond albedo coupled to an inefficient heat transport from the planetary day side to the night side, or else possibly that the 4.5 {mu}m observations probe atmospheric layers that are hotter than the maximum equilibrium temperature (i.e., a thermal inversion layer or a deep hot layer). The measured occultation phase and duration are consistent with a circular orbit and improves the 3{sigma} upper limit on 55 Cnc e's orbital eccentricity from 0.25 to 0.06.

79 ASTRONOMY AND ASTROPHYSICS↗

Stochastic dynamics of penetrable rods in one dimension: Entangled dynamics and transport properties

The dynamical properties of a system of soft rods governed by stochastic hard collisions (SHCs) have been determined over a varying range of softness using molecular dynamics simulations in one dimension and analytic theory. The SHC model allows for interpenetration of the system’s constituent particles in the simulations, generating overlapping clustering behavior analogous to the spatial structures observed in systems governed by deterministic bounded potentials. Through variation of an assigned softness parameter δ, the limiting ranges of intermolecular softness are bridged, connecting the limiting ensemble behavior from hard to ideal (completely soft). Various dynamical and structural observables are measured from simulation and compared to developed theoretical values. The spatial properties are found to be well predicted by theories developed for the deterministic penetrable-sphere model with a transformation from energetic to probabilistic arguments. While the overlapping spatial structures are complex, the dynamical properties can be adequately approximated through a theory built on impulsive interactions with Enskog corrections. Our theory suggests that as the softness of interaction is varied toward the ideal limit, correlated collision processes are less important to the energy transfer mechanism, and Markovian processes dominate the evolution of the configuration space ensemble. For interaction softness close to hard limit, collision processes are highly correlated and overlapping spatial configurations give rise to entanglement of single-particle trajectories.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A bayesian approach to estimate the size and structure of the broad-line region in active galactic nuclei using reverberation mapping data

This is the first paper in a series devoted to the systematic study of the size and structure of the broad-line region (BLR) in active galactic nuclei (AGNs) using reverberation mapping (RM) data. We employ a recently developed Bayesian approach that statistically describes the variability as a damped random walk process and delineates the BLR structure using a flexible disk geometry that can account for a variety of shapes, including disks, rings, shells, and spheres. We allow for the possibility that the line emission may respond non-linearly to the continuum, and we detrend the light curves when there is clear evidence for secular variation. We use a Markov Chain Monte Carlo implementation based on Bayesian statistics to recover the parameters and uncertainties for the BLR model. The corresponding transfer function is obtained self-consistently. We tentatively constrain the virial factor used to estimate black hole masses; more accurate determinations will have to await velocity-resolved RM data. Application of our method to RM data with Hβ monitoring for about 40 objects shows that the assumed BLR geometry can reproduce quite well the observed emission-line fluxes from the continuum light curves. We find that the Hβ BLR sizes obtained from our method are on average ∼20% larger than those derived from the traditional cross-correlation method. Nevertheless, we still find a tight BLR size-luminosity relation with a slope of α = 0.55 ± 0.03 and an intrinsic scatter of ∼0.18 dex. In particular, we demonstrate that our approach yields appropriate BLR sizes for some objects (such as Mrk 142 and PG 2130+099) where traditional methods previously encountered difficulties.

79 ASTRONOMY AND ASTROPHYSICS↗

An optimal repartitioning decision policy

A central problem to parallel processing is the determination of an effective partitioning of workload to processors. The effectiveness of any given partition is dependent on the stochastic nature of the workload. The problem of determining when and if the stochastic behavior of the workload has changed enough to warrant the calculation of a new partition is treated. The problem is modeled as a Markov decision process, and an optimal decision policy is derived. Quantification of this policy is usually intractable. A heuristic policy which performs nearly optimally is investigated empirically. The results suggest that the detection of change is the predominant issue in this problem.

Nicol, D. M.↗

Operation Optimization Using Reinforcement Learning with Integrated Artificial Reasoning Framework

In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.

Kim, Junyung↗

Inverse Uncertainty Quantification of Reactor Simulation with Polynomial Chaos Surrogate Model

Reliable uncertainty quantification (UQ) and sensitivity analysis (SA) is significant part of modelling and simulation. This is particularly important in the nuclear community due to the fact that the licensing of nuclear installations has shifted from conservative to Best Estimate plus Uncertainty (BEPU) methodologies. UQ can be broadly classified as forward UQ and inverse UQ. Forward UQ is the process of quantifying the uncertainties in simulation outputs by propagating the uncertainties in input parameters. It requires knowledge in the input uncertainty, which is often specified by ad-hoc expert judgment which lacks mathematical rigor. The inverse UQ is the process to determine the uncertainty in input parameters given observations of the output. The purpose of this paper is to replace such ad-hoc expert judgment of the statistical properties of input model parameters in nuclear reactor simulation. This summary presents work on inverse UQ of a nonlinear Point Reactor Kinetics Equations - lumped thermal-hydraulics parameter coupled problem. Bayesian inference and Markov Chain Monte Carlo (MCMC) are used for the inverse UQ. Because MCMC requires a large number of simulations which is impractical for models that are computationally expensive, surrogate models is constructed by generalized Polynomial Chaos (PC) expansion technique. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DISSECTING MAGNETAR VARIABILITY WITH BAYESIAN HIERARCHICAL MODELS

Neutron stars are a prime laboratory for testing physical processes under conditions of strong gravity, high density, and extreme magnetic fields. Among the zoo of neutron star phenomena, magnetars stand out for their bursting behavior, ranging from extremely bright, rare giant flares to numerous, less energetic recurrent bursts. The exact trigger and emission mechanisms for these bursts are not known; favored models involve either a crust fracture and subsequent energy release into the magnetosphere, or explosive reconnection of magnetic field lines. In the absence of a predictive model, understanding the physical processes responsible for magnetar burst variability is difficult. Here, we develop an empirical model that decomposes magnetar bursts into a superposition of small spike-like features with a simple functional form, where the number of model components is itself part of the inference problem. The cascades of spikes that we model might be formed by avalanches of reconnection, or crust rupture aftershocks. Using Markov Chain Monte Carlo sampling augmented with reversible jumps between models with different numbers of parameters, we characterize the posterior distributions of the model parameters and the number of components per burst. We relate these model parameters to physical quantities in the system, and show for the first time that the variability within a burst does not conform to predictions from ideas of self-organized criticality. We also examine how well the properties of the spikes fit the predictions of simplified cascade models for the different trigger mechanisms.

79 ASTRONOMY AND ASTROPHYSICS↗