Search NASA⌕ Search

SEARCH · Search NASA

Results for “Markovian processes”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Phenomenological memory-kernel master equations and time-dependent Markovian processes

Do phenomenological master equations with a memory kernel always describe a non-Markovian quantum dynamics characterized by reverse flow of information? Is the integration over the past states of the system an unmistakable signature of non-Markovianity? We show by a counterexample that this is not always the case. We consider two commonly used phenomenological integro-differential master equations describing the dynamics of a spin 1/2 in a thermal bath. By using a recently introduced measure to quantify non-Markovianity [Breuer et al., Phys. Rev. Lett. 103, 210401 (2009)] we demonstrate that as far as the equations retain their physical sense, the key feature of non-Markovian behavior does not appear in the considered memory kernel master equations. Namely, there is no reverse flow of information from the environment to the open system. Therefore, the assumption that the integration over a memory kernel always leads to a non-Markovian dynamics turns out to be vulnerable to phenomenological approximations. Instead, the considered phenomenological equations are able to describe time-dependent and unidirectional information flow from the system to the reservoir associated with time-dependent Markovian processes.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Quantum Fisher information flow and non-Markovian processes of open systems

We establish an information-theoretic approach for quantitatively characterizing the non-Markovianity of open quantum processes. Here, the quantum Fisher information (QFI) flow provides a measure to statistically distinguish Markovian and non-Markovian processes. A basic relation between the QFI flow and non-Markovianity is unveiled for quantum dynamics of open systems. For a class of time-local master equations, the exactly analytic solution shows that for each fixed time the QFI flow is decomposed into additive subflows according to different dissipative channels.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Stochastic Impulse Control of Non-Markovian Processes

We consider a class of stochastic impulse control problems of general stochastic processes i.e. not necessarily Markovian. Under fairly general conditions we establish existence of an optimal impulse control. We also prove existence of combined optimal stochastic and impulse control of a fairly general class of diffusions with random coefficients. Unlike, in the Markovian framework, we cannot apply quasi-variational inequalities techniques. We rather derive the main results using techniques involving reflected BSDEs and the Snell envelope.

97 MATHEMATICS AND COMPUTING↗

Estimation of stochastic volatility with long memory for index prices of FTSE Bursa Malaysia KLCI

In recent years, modeling in long memory properties or fractionally integrated processes in stochastic volatility has been applied in the financial time series. A time series with structural breaks can generate a strong persistence in the autocorrelation function, which is an observed behaviour of a long memory process. This paper considers the structural break of data in order to determine true long memory time series data. Unlike usual short memory models for log volatility, the fractional Ornstein-Uhlenbeck process is neither a Markovian process nor can it be easily transformed into a Markovian process. This makes the likelihood evaluation and parameter estimation for the long memory stochastic volatility (LMSV) model challenging tasks. The drift and volatility parameters of the fractional Ornstein-Unlenbeck model are estimated separately using the least square estimator (lse) and quadratic generalized variations (qgv) method respectively. Finally, the empirical distribution of unobserved volatility is estimated using the particle filtering with sequential important sampling-resampling (SIR) method. The mean square error (MSE) between the estimated and empirical volatility indicates that the performance of the model towards the index prices of FTSE Bursa Malaysia KLCI is fairly well.

97 MATHEMATICS AND COMPUTING↗

Effective Pore-Scale Dispersion Upscaling with the Correlated Continuous Time Random Walk Approach

We propose a general framework for upscaling dispersion in porous media. A key challenge of the upscaling procedure is to relate the temporal evolution of spreading to the small scale velocity field properties. The representation of the Lagrangian velocity transition process as a Markovian process in space provides a simple way to quantify complex correlation properties, i.e. non-Gaussian velocity distributions. The resulting effective transport model is a correlated CTRW. We use this framework to upscale pore scale dispersion for a periodic pore geometry. The correlated CTRW model is defined by the transit time distribution across one pore and the transition probability density quantifying the correlation between successive transit times. The latter is of central importance since it accounts for incomplete mixing at the pore throats. The predictions of the correlated CTRW model are in good agreement with the pore scale simulations over the pre-asymptotic and asymptotic regimes. We investigate the representation of this effective dispersion model in phase space (position, velocity) in a form similar to a Boltzmann transport equation.

multiscale modeling, dimension reduction, transpor↗

Fast Quantum Algorithm for Predicting Descriptive Statistics of Stochastic Processes

Stochastic processes are used as a modeling tool in several sub-fields of physics, biology, and finance. Analytic understanding of the long term behavior of such processes is only tractable for very simple types of stochastic processes such as Markovian processes. However, in real world applications more complex stochastic processes often arise. In physics, the complicating factor might be nonlinearities; in biology it might be memory effects; and in finance is might be the non-random intentional behavior of participants in a market. In the absence of analytic insight, one is forced to understand these more complex stochastic processes via numerical simulation techniques. In this paper we present a quantum algorithm for performing such simulations. In particular, we show how a quantum algorithm can predict arbitrary descriptive statistics (moments) of N-step stochastic processes in just O(square root of N) time. That is, the quantum complexity is the square root of the classical complexity for performing such simulations. This is a significant speedup in comparison to the current state of the art.

Williams Colin P.↗

Probabilistic Round Trip Contamination Analysis of a Mars Sample Acquisition and Handling Process Using Markovian Decompositions

A method for evaluating the probability of a Viable Earth Microorganism (VEM) contaminating a sample during the sample acquisition and handling (SAH) process of a potential future Mars Sample Return mission is developed. A scenario where multiple core samples would be acquired using a rotary percussive coring tool, deployed from an arm on a MER class rover is analyzed. The analysis is conducted in a structured way by decomposing sample acquisition and handling process into a series of discrete time steps, and breaking the physical system into a set of relevant components. At each discrete time step, two key functions are defined: The probability of a VEM being released from each component, and the transport matrix, which represents the probability of VEM transport from one component to another. By defining the expected the number of VEMs on each component at the start of the sampling process, these decompositions allow the expected number of VEMs on each component at each sampling step to be represented as a Markov chain. This formalism provides a rigorous mathematical framework in which to analyze the probability of a VEM entering the sample chain, as well as making the analysis tractable by breaking the process down into small analyzable steps.

Hudson, Nicolas↗

Markovian transport processes in a heterogeneous, variably saturated watershed: A multi-domain spatial Markov model

Transport processes in watersheds remain challenging to describe accurately with upscaled models because of their intricate complexities at multiple scales that can lead to multi-modal, non-Fickian breakthrough curves. One of the recent advances in solute transport modeling has been the Lagrangian spatial Markov model (SMM), which describes transport using velocity increments and models the correlated transitions between them. The method has been applied previously to saturated porous media and fractured media, but these successful applications suggest it may be useful for describing watershed scale processes as well. An existing, 3-D, heterogeneous variably saturated model of a headwaters catchment and classical random walk particle tracking were used to generate a forward simulation and the resulting transport was analyzed with a spatial Markov framework. Existing SMM methods were used to analyze the transport paths through the complex, unconfined domain and we found that the vadose zone plays an important role in the correlation structure. As such, a new kind of SMM is proposed, termed a multi-domain SMM (MD-SMM), that uses separate correlation models for auto-transitions in the vadose zone and the saturated zone, and for the cross-transitions between them. The MD-SMM improves representation of transport relative to the single domain model and can naturally delineate times when a particle is in the vadose zone or the saturated zone, adding an additional degree of realism to the upscaled model. Lastly, we show that the transition probability matrix for transport in the saturated portion of this unconfined system is approximately Markovian, which is an important validation of the proposed framework.

54 ENVIRONMENTAL SCIENCES↗

Decentralized control of Markovian decision processes: Existence Sigma-admissable policies

The problem of formulating and analyzing Markov decision models having decentralized information and decision patterns is examined. Included are basic examples as well as the mathematical preliminaries needed to understand Markov decision models and, further, to superimpose decentralized decision structures on them. The notion of a variance admissible policy for the model is introduced and it is proved that there exist (possibly nondeterministic) optional policies from the class of variance admissible policies. Directions for further research are explored.

Greenland, A.↗

Stochastic dynamics of penetrable rods in one dimension: Entangled dynamics and transport properties

The dynamical properties of a system of soft rods governed by stochastic hard collisions (SHCs) have been determined over a varying range of softness using molecular dynamics simulations in one dimension and analytic theory. The SHC model allows for interpenetration of the system’s constituent particles in the simulations, generating overlapping clustering behavior analogous to the spatial structures observed in systems governed by deterministic bounded potentials. Through variation of an assigned softness parameter δ, the limiting ranges of intermolecular softness are bridged, connecting the limiting ensemble behavior from hard to ideal (completely soft). Various dynamical and structural observables are measured from simulation and compared to developed theoretical values. The spatial properties are found to be well predicted by theories developed for the deterministic penetrable-sphere model with a transformation from energetic to probabilistic arguments. While the overlapping spatial structures are complex, the dynamical properties can be adequately approximated through a theory built on impulsive interactions with Enskog corrections. Our theory suggests that as the softness of interaction is varied toward the ideal limit, correlated collision processes are less important to the energy transfer mechanism, and Markovian processes dominate the evolution of the configuration space ensemble. For interaction softness close to hard limit, collision processes are highly correlated and overlapping spatial configurations give rise to entanglement of single-particle trajectories.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Equivalence of the measures of non-Markovianity for open two-level systems

Different measures have been presented to depict the deviation of quantum time evolution in open systems from Markovian processes. We demonstrate that the measure proposed by Breuer, Laine, and Piilo [Phys. Rev. Lett. 103, 210401 (2009)] and the two measures proposed by Rivas, Huelga, and Plenio [Phys. Rev. Lett. 105, 050403 (2010)] have exactly the same non-Markovian time-evolution intervals and thus are really equivalent to each other when they are applied to open two-level systems coupled to environments via the Jaynes-Cummings or dephasing models. This equivalence implies that the three measures, in different ways, capture the intrinsic character of the non-Markovianity of quantum evolutional processes. We also show that the maximization in the definition of the first measure can be actually removed for the considered models without influencing the sensibility of the measure to detect non-Markovianity.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Measure for the non-Markovianity of quantum processes

Recently, a measure for the non-Markovian behavior of quantum processes in open systems has been developed, which is based on the quantification of the flow of information between the open system and its environment [Phys. Rev. Lett. 103, 210401 (2009)]. The information flow is connected to the rate of change of the trace distance between quantum states, which can be interpreted in terms of the distinguishability of these states. Here, we elaborate the mathematical details of this theory, present applications to specific physical models, and discuss further theoretical and experimental implications as well as relations to alternative approaches proposed recently.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Distribution of chirality in the quantum walk: Markov process and entanglement

The asymptotic behavior of the quantum walk on the line is investigated, focusing on the probability distribution of chirality independently of position. It is shown analytically that this distribution has a longtime limit that is stationary and depends on the initial conditions. This result is unexpected in the context of the unitary evolution of the quantum walk as it is usually linked to a Markovian process. The asymptotic value of the entanglement between the coin and the position is determined by the chirality distribution. For given asymptotic values of both the entanglement and the chirality distribution, it is possible to find the corresponding initial conditions within a particular class of spatially extended Gaussian distributions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Discrete time modelization of human pilot behavior

This modelization starts from the following hypotheses: pilot's behavior is a time discrete process, he can perform only one task at a time and his operating mode depends on the considered flight subphase. Pilot's behavior was observed using an electro oculometer and a simulator cockpit. A FORTRAN program has been elaborated using two strategies. The first one is a Markovian process in which the successive instrument readings are governed by a matrix of conditional probabilities. In the second one, strategy is an heuristic process and the concepts of mental load and performance are described. The results of the two aspects have been compared with simulation data.

Cavalli, D.↗

Complexity-calibrated benchmarks for machine learning reveal when prediction algorithms succeed and mislead

Abstract Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a “next-generation” reservoir computer was introduced in which the memory trace involves only a finite number of previous symbols. We explore the inherent limitations of finite-past memory traces in this intriguing proposal. A lower bound from Fano’s inequality shows that, on highly non-Markovian processes generated by large probabilistic state machines, next-generation reservoir computers with reasonably long memory traces have an error probability that is at least $$\sim 60\%$$ ∼ 60 % higher than the minimal attainable error probability in predicting the next observation. More generally, it appears that popular recurrent neural networks fall far short of optimally predicting such complex processes. These results highlight the need for a new generation of optimized recurrent neural network architectures. Alongside this finding, we present concentration-of-measure results for randomly-generated but complex processes. One conclusion is that large probabilistic state machines—specifically, large $$\epsilon$$ ϵ -machines—are key to generating challenging and structurally-unbiased stimuli for ground-truthing recurrent neural network architectures.

97 MATHEMATICS AND COMPUTING↗