Search NASA⌕ Search

SEARCH · Search NASA

Results for “MARKOV PROCESS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Calculating infinite-medium {alpha}-eigenvalue spectra with a transition rate matrix method

The time-dependent behavior of the energy spectrum in neutron transport was investigated with a formulation, based on continuous-time Markov processes, for computing {alpha}-eigenvalues and eigenvectors in an infinite medium. For this, a research Monte Carlo code called TORTE was created and used to estimate elements of a transition rate matrix. TORTE is capable of using both multigroup and continuous-energy nuclear data, and verification was performed. Eigenvalue spectra for infinite homogeneous mixtures were obtained and an eigenfunction expansion was used to investigate transient behavior of the neutron energy spectrum. (authors)

97 MATHEMATICS AND COMPUTING↗

A Large Deviations Analysis of Certain Qualitative Properties of Parallel Tempering and Infinite Swapping Algorithms

Parallel tempering, or replica exchange, is a popular method for simulating complex systems. The idea is to run parallel simulations at different temperatures, and at a given swap rate exchange configurations between the parallel simulations. From the perspective of large deviations it is optimal to let the swap rate tend to infinity and it is possible to construct a corresponding simulation scheme, known as infinite swapping. In this paper we propose a novel use of large deviations for empirical measures for a more detailed analysis of the infinite swapping limit in the setting of continuous time jump Markov processes. Using the large deviations rate function and associated stochastic control problems we consider a diagnostic based on temperature assignments, which can be easily computed during a simulation. Here we show that the convergence of this diagnostic to its a priori known limit is a necessary condition for the convergence of infinite swapping. Finally, the rate function is also used to investigate the impact of asymmetries in the underlying potential landscape, and where in the state space poor sampling is most likely to occur.

97 MATHEMATICS AND COMPUTING↗

Discrete-Time Pricing and Optimal Exercise of American Perpetual Warrants in the Geometric Random Walk Model

An American option (or, warrant) is the right, but not the obligation, to purchase or sell an underlying equity at any time up to a predetermined expiration date for a predetermined amount. A perpetual American option differs from a plain American option in that it does not expire. In this study, we solve the optimal stopping problem of a perpetual American option (both call and put) in discrete time using linear programming duality. Under the assumption that the underlying stock price follows a discrete time and discrete state Markov process, namely a geometric random walk, we formulate the pricing problem as an infinite dimensional linear programming (LP) problem using the excessive-majorant property of the value function. This formulation allows us to solve complementary slackness conditions in closed-form, revealing an optimal stopping strategy which highlights the set of stock-prices where the option should be exercised. The analysis for the call option reveals that such a critical value exists only in some cases, depending on a combination of state-transition probabilities and the economic discount factor (i.e., the prevailing interest rate) whereas it ceases to be an issue for the put.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A Piecewise Deterministic Markov Toy Model for Traffic/Maintenance and Associated Hamilton–Jacobi Integrodifferential Systems on Networks

We study optimal control problems in infinite horizon whxen the dynamics belong to a specific class of piecewise deterministic Markov processes constrained to star-shaped networks (corresponding to a toy traffic model). We adapt the results in Soner (SIAM J Control Optim 24(6):1110–1122, 1986) to prove the regularity of the value function and the dynamic programming principle. Extending the networks and Krylov’s “shaking the coefficients” method, we prove that the value function can be seen as the solution to a linearized optimization problem set on a convenient set of probability measures. The approach relies entirely on viscosity arguments. As a by-product, the dual formulation guarantees that the value function is the pointwise supremum over regular subsolutions of the associated Hamilton–Jacobi integrodifferential system. This ensures that the value function satisfies Perron’s preconization for the (unique) candidate to viscosity solution.

97 MATHEMATICS AND COMPUTING↗

Almost sure convergence in quantum spin glasses

Recently, Keating, Linden, and Wells [Markov Processes Relat. Fields 21(3), 537-555 (2015)] showed that the density of states measure of a nearest-neighbor quantum spin glass model is approximately Gaussian when the number of particles is large. The density of states measure is the ensemble average of the empirical spectral measure of a random matrix; in this paper, we use concentration of measure and entropy techniques together with the result of Keating, Linden, and Wells to show that in fact the empirical spectral measure of such a random matrix is almost surely approximately Gaussian itself with no ensemble averaging. We also extend this result to a spherical quantum spin glass model and to the more general coupling geometries investigated by Erdős and Schröder [Math. Phys., Anal. Geom. 17(3-4), 441–464 (2014)].

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

SU-F-T-683: Cancer Stem Cell Hypothesis and Radiation Treatments

Purpose: The tumor control probability in radiation therapy allows comparing different radiation treatments to each other by means of calculating the probability that a prescribed dose of radiation eradicates or controls the tumor. In the conventional approach, all cancer cells can divide unlimited number of times and the tumor control often means eradicating every malignant cell by the radiation. In recent years however, there is a mounting consensus that in a given tumor volume there is a sub-population of cells, known as cancer stem cells (CSCs) that are responsible for tumor initiation and growth. Other or progenitor cancer cells can only divide limited number of times. This entails that only cancer stem cells may nned to be eliminated in order to control the tumor. Thus one may define TCP as the probability of eliminating CSCs for the given dose of radiation. Methods: Using stochastic methods, specifically the birth-and-death Markov processes, an infinite system of equations is set for probabilities of having m cancer stem cells at time t after the start of radiation. The TCP is calculated as the probability of no cancer stem cells surviving the radiation. Two scenarios are studied. In the first situation, the TCP is calculated for a unidirectional case when CSC gives birth to another CSC or a progenitor cell. In the second scenario, a bidirectional model is studied where the progenitor cell gives rise to CSC. Results: The proposed calculations show that the calculated TCP for CSC depends on whether one adopts unidirectional or bidirectional conversion models. The bidirectional model shows significantly lower TCP values for the given dose delivered to the tumor. Conclusion: Incorporating CSC hypothesis into the TCP modeling may notably influence the dose prescription as well as the concept of the expected TCP after the radiation treatments.

60 APPLIED LIFE SCIENCES↗

Calculating Alpha Eigenvalues and Eigenfunctions with a Markov Transition Rate Matrix Monte Carlo Method

For a nuclear system in which the entire α-eigenvalue spectrum is known, eigenfunction expansion yields the time-dependent flux response to any arbitrary source. Applications in which this response is of interest include pulsed-neutron experiments, accelerator-driven subcritical systems, and fast burst reactors, where a steady-state assumption used in neutron transport is invalid for characterizing the time-dependent flux. To obtain the α-eigenvalue spectrum, the transition rate matrix method (TRMM) tallies transition rates describing neutron behavior in a discretized position-direction-energy phase space using Monte Carlo. Interpretation of the resulting Markov process transition rate matrix as the operator in the adjoint α-eigenvalue problem provides an avenue for determining a large finite set of α eigenvalues and eigenfunctions of a nuclear system. Results from the TRMM are verified using analytic solutions, time-dependent Monte Carlo simulations, and modal expansion from diffusion theory. For simplified infinite-medium and one-dimensional geometries, the TRMM accurately calculates eigenvalues, eigenfunctions, and eigenfunction expansion solutions. Applications and comparisons to measurements are made for the small fast burst reactor CALIBAN and the Fort St. Vrain high-temperature gas-cooled reactor. For large three-dimensional geometries, discretization of the large position-energy-direction phase space limits the accuracy of eigenfunction expansion solutions using the TRMM, but it can still generate a fair estimate of the fundamental eigenvalue and eigenfunction. These results show that the TRMM generates an accurate estimate of a large number of α eigenvalues. This is not possible with existing Monte Carlo–based methods.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Parametric sensitivity analysis for stochastic molecular systems using information theoretic metrics

In this paper, we present a parametric sensitivity analysis (SA) methodology for continuous time and continuous space Markov processes represented by stochastic differential equations. Particularly, we focus on stochastic molecular dynamics as described by the Langevin equation. The utilized SA method is based on the computation of the information-theoretic (and thermodynamic) quantity of relative entropy rate (RER) and the associated Fisher information matrix (FIM) between path distributions, and it is an extension of the work proposed by Y. Pantazis and M. A. Katsoulakis [J. Chem. Phys. 138, 054115 (2013)]. A major advantage of the pathwise SA method is that both RER and pathwise FIM depend only on averages of the force field; therefore, they are tractable and computable as ergodic averages from a single run of the molecular dynamics simulation both in equilibrium and in non-equilibrium steady state regimes. We validate the performance of the extended SA method to two different molecular stochastic systems, a standard Lennard-Jones fluid and an all-atom methane liquid, and compare the obtained parameter sensitivities with parameter sensitivities on three popular and well-studied observable functions, namely, the radial distribution function, the mean squared displacement, and the pressure. Results show that the RER-based sensitivities are highly correlated with the observable-based sensitivities.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Resolvent-Techniques for Multiple Exercise Problems

We study optimal multiple stopping of strong Markov processes with random refraction periods. The refraction periods are assumed to be exponentially distributed with a common rate and independent of the underlying dynamics. Our main tool is using the resolvent operator. In the first part, we reduce infinite stopping problems to ordinary ones in a general strong Markov setting. This leads to explicit solutions for wide classes of such problems. Starting from this result, we analyze problems with finitely many exercise rights and explain solution methods for some classes of problems with underlying Lévy and diffusion processes, where the optimal characteristics of the problems can be identified more explicitly. We illustrate the main results with explicit examples.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

graphMC: A package for testing the independence of graphs

graphMC consists of an implementation of a set of statistical tests that ensure that a Markov chain in the space of graphs has converged to its stationary distribution. The tests are applicable when the graphs have labelled nodes. The tests are applied to the binary time-series traced out by an edge between two selected nodes. The software fits models of a first-order Markov process and an independent draws from a binary distribution to the time-series and selects one based on the BIC scores of the two models. The package consists of (1) an implementation of a Markov chain driving a graph rewiring process that preserves the joint degree distribution of the graph, (2) implementation of the statistical test mentioned above and (3) two examples that show how the Markov chain infrastructure and the statistical tests can be used.

Safta, Cosmin↗

graphMC: A package for testing the independence of graphs

graphMC consists of an implementation of a set of statistical tests that ensure that a Markov chain in the space of graphs has converged to its stationary distribution. The tests are applicable when the graphs have labelled nodes. The tests are applied to the binary time-series traced out by an edge between two selected nodes. The software fits models of a first-order Markov process and an independent draws from a binary distribution to the time-series and selects one based on the BIC scores of the two models. The package consists of (1) an implementation of a Markov chain driving a graph rewiring process that preserves the joint degree distribution of the graph, (2) implementation of the statistical test mentioned above and (3) two examples that show how the Markov chain infrastructure and the statistical tests can be used.

Source record↗

Software forecasting as it is really done: A study of JPL software engineers

This paper presents a summary of the results to date of a Jet Propulsion Laboratory internally funded research task to study the costing process and parameters used by internally recognized software cost estimating experts. Protocol Analysis and Markov process modeling were used to capture software engineer's forecasting mental models. While there is significant variation between the mental models that were studied, it was nevertheless possible to identify a core set of cost forecasting activities, and it was also found that the mental models cluster around three forecasting techniques. Further partitioning of the mental models revealed clustering of activities, that is very suggestive of a forecasting lifecycle. The different forecasting methods identified were based on the use of multiple-decomposition steps or multiple forecasting steps. The multiple forecasting steps involved either forecasting software size or an additional effort forecast. Virtually no subject used risk reduction steps in combination. The results of the analysis include: the identification of a core set of well defined costing activities, a proposed software forecasting life cycle, and the identification of several basic software forecasting mental models. The paper concludes with a discussion of the implications of the results for current individual and institutional practices.

Griesel, Martha Ann↗

Blinking statistics and molecular counting in direct stochastic reconstruction microscopy (dSTORM)

Abstract Motivation Many recent advancements in single-molecule localization microscopy exploit the stochastic photoswitching of fluorophores to reveal complex cellular structures beyond the classical diffraction limit. However, this same stochasticity makes counting the number of molecules to high precision extremely challenging, preventing key insight into the cellular structures and processes under observation. Results Modelling the photoswitching behaviour of a fluorophore as an unobserved continuous time Markov process transitioning between a single fluorescent and multiple dark states, and fully mitigating for missed blinks and false positives, we present a method for computing the exact probability distribution for the number of observed localizations from a single photoswitching fluorophore. This is then extended to provide the probability distribution for the number of localizations in a direct stochastic optical reconstruction microscopy experiment involving an arbitrary number of molecules. We demonstrate that when training data are available to estimate photoswitching rates, the unknown number of molecules can be accurately recovered from the posterior mode of the number of molecules given the number of localizations. Finally, we demonstrate the method on experimental data by quantifying the number of adapter protein linker for activation of T cells on the cell surface of the T-cell immunological synapse. Availability and implementation Software and data available at https://github.com/lp1611/mol_count_dstorm. Supplementary information Supplementary data are available at Bioinformatics online.

Patel, Lekha↗

Receiver processing for direct-detection optical communication systems.

A model that is sufficiently general to describe the predominant statistical characteristics of the output of many real optical detectors is formulated. This model is used to study the optimum receiver processing for direct-detection optical communication systems. In particular, the structures of detectors and estimators for filtered doubly stochastic Poisson processes (DSPP) observed in additive white Gaussian noise are considered. Representations for the posterior statistics of a vector Markov process that modulates the intensity of the DSPP are obtained.

Hoversten, E. V.↗

On the first-excursion probability in stationary narrow-band random vibration. II.

The first-excursion probability of a stationary narrow-band Gaussian process with mean zero has been studied. Within the framework of point process approach, series approximations derived from the theory of random points and approximations based on the maximum entropy principle have been developed. With the aid of numerical examples, merits of the approximations proposed previously as well as of those developed in this paper have been compared. The results indicate that the maximum entropy principle has not produced satisfactory approximations but the approximation based on nonapproaching random points is found to be the best among all the approximations proposed herein. A conclusion drawn from the present and the previous studies is that the point process approach produces a number of useful approximations for the first-excursion probability, particularly those based on the concepts of the Markov process, the clump-size, and the nonapproaching random points.

Yang, J.-N.↗

Stochastic Price Generation for Evaluating Wholesale Electricity Market Bidding Strategies

This work presents a novel method for generating electricity price scenarios from statistical properties of past electricity prices using a hybrid statistical and reduced-form stochastic model. Previous work in applying stochastic differential equations (SDE) to model electricity prices has focused on daily average prices. To extend stochastic price generation methods to hourly or sub-hourly pricing, we address several weaknesses in the state-of-the-art: (1) we replace the mean-reversion component of the SDE with an ARIMA process that is better able to characterize the daily and weekly trends; (2) we extend the price-spike, or jump process to account for conditional probabilities of price spikes occurring in consecutive time steps by replacing the traditional Poisson process for modeling jumps with a generalized point process model inspired by brain neuron models; and (3) we replace the traditional method of estimating spike intensity with empirical variance with a Markov process based on observed price spike intensity transitions. The method is demonstrated with electricity prices from the US ERCOT market and a use-case example is provided for bidding an energy storage unit into the day-ahead and real-time energy markets of ERCOT using stochastic optimization methods. Results show that the the synthetic price model out performs a (naive) persistence forecast model by resulting in 24% to 47% more in profits over 168 simulated days.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Quantum advantage for differential equation analysis

Quantum algorithms for differential equation solving, data processing, and machine learning potentially offer an exponential speedup over all known classical algorithms. However, there also exist obstacles to obtaining this potential speedup in useful problem instances. The essential obstacle for quantum differential equation solving is that outputting useful information may require difficult postprocessing, and the essential obstacle for quantum data processing and machine learning is that inputting the data is a difficult task just by itself. In this study, we demonstrate that, when combined, these difficulties solve one another. We show how the output of quantum differential equation solving can serve as the input for quantum data processing and machine learning, allowing dynamical analysis in terms of principal components, power spectra, and wavelet decompositions. To illustrate this, we consider continuous-time Markov processes on epidemiological and social networks. These quantum algorithms provide an exponential advantage over existing classical Monte Carlo methods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Bayesian recursive image estimation.

The enhancement of images that are characterized only by statistical data where the picture contains additive noise is considered. The random process representing the output of the scanner is characterized by the output of a dynamic system with white noise input. The dynamic system describes a first-order vector-Markov process. The procedure of Kalman filtering is then utilized to recursively determine the minimum mean-square error estimate of the image. The result is then extended to obtain the smoothing of the data.

Nahi, N. E.↗