Optimal Discounted Stochastic Control for Diffusion Processes
Optimal cost discounted stochastic control for diffusion processes
Kushner, H. J.↗
SEARCH · Search NASA
Results for “Stochastic processes.”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Adaptive Sampling of Large Deviations
We introduce and test an algorithm that adaptively estimates large deviation functions characterizing the fluctuations of additive functionals of Markov processes in the long-time limit. These functions play an important role for predicting the probability and pathways of rare events in stochastic processes, as well as for understanding the physics of nonequilibrium systems driven in steady states by external forces and reservoirs. The algorithm uses methods from risk-sensitive and feedback control to estimate from a single trajectory a new process, called the driven process, known to be efficient for importance sampling. Its advantages compared to other simulation techniques, such as splitting or cloning, are discussed and illustrated with simple equilibrium and nonequilibrium diffusion models.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Core-level nonlinear spectroscopy triggered by stochastic X-ray pulses
Stochastic processes are highly relevant in research fields as different as neuroscience, economy, ecology, chemistry, and fundamental physics. However, due to their intrinsic unpredictability, stochastic mechanisms are very challenging for any kind of investigations and practical applications. Here we report the deliberate use of stochastic X-ray pulses in two-dimensional spectroscopy to the simultaneous mapping of unoccupied and occupied electronic states of atoms in a regime where the opacity and transparency properties of matter are subject to the incident intensity and photon energy. A readily transferable matrix formalism is presented to extract the electronic states from a dataset measured with the monitored input from a stochastic excitation source. The presented formalism enables investigations of the response of the electronic structure to irradiation with intense X-ray pulses while the time structure of the incident pulses is preserved.
74 ATOMIC AND MOLECULAR PHYSICS↗
A general framework for quantitatively assessing ecological stochasticity
Understanding the community assembly mechanisms controlling biodiversity patterns is a central issue in ecology. Although it is generally accepted that both deterministic and stochastic processes play important roles in community assembly, quantifying their relative importance is challenging. Here we propose a general mathematical framework to quantify ecological stochasticity under different situations in which deterministic factors drive the communities more similar or dissimilar than null expectation. An index, normalized stochasticity ratio ( NST ), was developed with 50% as the boundary point between more deterministic (<50%) and more stochastic (>50%) assembly. NST was tested with simulated communities by considering abiotic filtering, competition, environmental noise, and spatial scales. All tested approaches showed limited performance at large spatial scales or under very high environmental noise. However, in all of the other simulated scenarios, NST showed high accuracy (0.90 to 1.00) and precision (0.91 to 0.99), with averages of 0.37 higher accuracy (0.1 to 0.7) and 0.33 higher precision (0.0 to 1.8) than previous approaches. NST was also applied to estimate stochasticity in the succession of a groundwater microbial community in response to organic carbon (vegetable oil) injection. Our results showed that community assembly was shifted from more deterministic ( NST = 21%) to more stochastic ( NST = 70%) right after organic carbon input. As the vegetable oil was consumed, the community gradually returned to be more deterministic ( NST = 27%). In addition, our results demonstrated that null model algorithms and community similarity metrics had strong effects on quantifying ecological stochasticity.
54 ENVIRONMENTAL SCIENCES↗
An Adaptive Technique for a Redundant-Sensor Navigation System
An on-line adaptive technique is developed to provide a self-contained redundant-sensor navigation system with a capability to utilize its full potentiality in reliability and performance. The gyro navigation system is modeled as a Gauss-Markov process, with degradation modes defined as changes in characteristics specified by parameters associated with the model. The adaptive system is formulated as a multistage stochastic process: (1) a detection system, (2) an identification system and (3) a compensation system. It is shown that the sufficient statistics for the partially observable process in the detection and identification system is the posterior measure of the state of degradation, conditioned on the measurement history.
Chien, T. T.↗
Additive manufacturing: Toward holistic design
Here, additive manufacturing offers unprecedented opportunities to design complex structures optimized for performance envelopes inaccessible under conventional manufacturing constraints. Additive processes also promote realization of engineered materials with microstructures and properties that are impossible via traditional synthesis techniques. Enthused by these capabilities, optimization design tools have experienced a recent revival. The current capabilities of additive processes and optimization tools are summarized briefly, while an emerging opportunity is discussed to achieve a holistic design paradigm whereby computational tools are integrated with stochastic process and material awareness to enable the concurrent optimization of design topologies, material constructs and fabrication processes.
36 MATERIALS SCIENCE↗
Sparse Control Synthesis for Uncertain Responsive Loads With Stochastic Stability Guarantees
In this report, recent studies have demonstrated the potential of flexible loads in providing frequency response services, predominantly due to their availability and cost-effectiveness. However, uncertainty and variability in various weather-related and end-use behavioral factors often impact the reliability of demand-side control performance. This work addresses this problem with the design of a demand-side control to achieve frequency response under load uncertainties. Our approach involves modeling the load uncertainties via stochastic processes that appear as both multiplicative and additive in the power system dynamics. Recently developed mean square exponential stability (MSES) results for continuous-time linear stochastic systems are applied to pose the control synthesis problem which results in an LMI-based optimization problem. Additional costs and constraints are added to the LMI-based controller synthesis to ensure MSES, improve closed-loop transient performance, maximize tolerable uncertainties, and promote sparsity in the controller. Additionally, the fundamental limitations between the tolerable uncertainties and control efforts while ensuring MSES are discussed. Further, the control synthesis problem for the case of the full-state measurement is generalized to the case of partial-state measurements. The proposed control synthesis is illustrated on an IEEE 39 bus system with rigorous studies to demonstrate the role of sparsity, closed-loop transient performance, tolerable uncertainties, and control efforts while ensuring MSES and achieving frequency response.
42 ENGINEERING↗
Investigation of brightness changes of MZ Cas and TZ Cas in B- and V-light
The results are presented concerning statistical processing of two-color observations of MZ Cas and TZ Cas. Light histograms, dispersion and statistical amplitudes are given. Light variations of the variables are represented by normal stochastic processes. Observational data are tabulated.
Lukatskaya, F. I.↗
Monte Carlo Direct Simulation Method for Initiation Probability Problems
During the early period of a supercritical system, the growth of fission chains shows a significant stochastic feature. In multiplying medium, it is a probability issue of fail-pass type for a single neutron to sponsor a divergent fission chain, and the probability of success is named as single neutron initiation probability. Meanwhile, in the presence of a weak neutron source, the time at which fission burst occurs also exhibits a stochastic nature, which can be observed on experiments of pulsed reactors such as GODIVA I and GODIVA II. The initiation probability and burst waiting time reflect the dynamic feature of a supercritical system, which are very important for nuclear criticality safety concerns. It is desirable that a supercritical system has an initiation probability very close to zero and releases limited energy in case of criticality accidents. The stochastic process of neutron population evolution in early age in a multiplying system is governed by Bell's equation in the form of an adjoint neutron transport equation with a non-linear fission term. In order to solve this stochastic neutron transport equation, the deterministic method discretizing the phase space was proposed long ago and has been implemented in several SN codes such as Partisn, Ardra and PANDA. However, the deterministic method is of limited usage, due to its poor ability to precisely describe real-world 3D complex geometries and continuous energy neutron spectrum. Recently, Monte Carlo methods have gradually gained attention for solving Bell's equation. Greenman from LLNL proposed an analog Monte Carlo forward-transport method for calculating initiation probability in 2007. He implemented this method by introducing the maximum age parameter, the time sub-interval and the neutron progenitor tag into Mercury in time-dependent fixed-source simulation mode. This method is only suitable for initiation probability calculations, however not for analyzing the randomness of burst waiting time. Later, GANG in 2011 developed the GSMP (Generalized Semi-Markov Process method) Monte Carlo method according to Bell's equation. While the GSMP method, which can deal with both initiation probability and burst waiting time, is flexible and powerful for zero-dimensional problems, it is not favorable when extended to 3D and continuous energy problems due to severe lack of computational efficiency. In 2015, a direct Monte Carlo simulation method for stochastic systems problems was implemented in MCATK, yet this method has not been applied specifically to the burst waiting time. In this work, a new and efficient 3D Monte Carlo direct simulation method for calculating both the initiation probability and the burst waiting time is developed. By taking random factors during evolvement of fission chains into account in dynamic Monte Carlo simulation, this new method has the potential of simulating the whole process from source neutron injection to exponential growth of the neutron population, and finally to extinction of the neutron pulse. This capability is useful in criticality accidents evaluation and pulsed reactor experiments explanation. (authors)
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
A new stochastic systems approach to structural integrity
This paper develops improved stochastic models for the description of a large variety of fatigue crack growth phenomena that occur in components of considerable importance to the functionality and reliability of complex engineering structures. In essence, the models are based on the McGill-Markov and Closure-Lognormal stochastic processes. Not only do these models have the capability of predicting the statistical dispersion of crack growth rates, they also, by incorporating the concept of crack closure, have the capability of transferring stochastic crack growth properties measured under ideal laboratory conditions to situations of industrial significance, such as those occurring under adverse loading and/or environmental conditions. The primary data required in order to be in a position to estimate the pertinent parameters of these stochastic models are obtained from a statistically significant number of replicate tests. In this paper, both the theory and the experimental technique are illustrated using a Ti-6Al-4V alloy. Finally, important structural integrity, reliability, availability and maintainability concepts are developed and illustrated.
Provan, James W.↗
Planetary circulations in the presence of transient and self-induced heating
The research program focuses on large-scale circulations and their interaction with the global convective pattern. An 11-year record of global cloud imagery and contemporaneous fields of motion and temperature have been used to investigate organized convection and coherent variability of the tropical circulation operating on intraseasonal time scales. This study provides a detailed portrait of tropical variability associated with the so-called Madden-Julian Oscillation (MJO). It reveals the nature, geographical distribution, and seasonality of discrete convective signal, which is a measure of feedback between the circulation and the convective pattern. That discrete spectral behavior has been evaluated in light of natural variability of the ITCZ associated with climatological convection. A composite signature of the MJO, based on cross-covariance statistics of cloud cover, motion, and temperature, has been constructed to characterize the lifecycle of the disturbance in terms of these properties. The composite behavior has also been used to investigate the influence the MJO exerts on the zonal-mean circulation and the involvement of the MJO in transfers of momentum between the atmosphere and the solid Earth. The aforementioned observational studies have led to the production of two animations. One reveals the convective signal in band-pass filtered OLR and compares it to climatological convection. The other is a 3-dimensional visualization of the composite lifecycle of the MJO. With a clear picture of the MJO in hand, feedback between the circulation and the convective pattern can be diagnosed meaningfully in numerical simulations. This process is being explored in calculations with the linearized primitive equations on the sphere in the presence of realistic stability and shear. The numerical framework represents climatological convection as a space-time stochastic process and wave-induced convection in terms of the vertically-integrated moisture flux convergence. In these calculations, frictional convergence near the equator emerges as a key to feedback between the circulation and the convective pattern. At low latitudes, nearly geostrophic balance in the boundary layer gives way to frictional balance. This shifts the wave-induced convection into phase with the temperature anomaly and allows the attending heating to feed back positively onto the circulation. The calculations successfully reproduce the salient features of the MJO. They are being used to understand the growth and decay phases of the composite lifecycle and the conditions that favor amplification of the MJO.
Salby, Murry L.↗
A Stochastic Fractional Dynamics Model of Space-time Variability of Rain
Rainfall varies in space and time in a highly irregular manner and is described naturally in terms of a stochastic process. A characteristic feature of rainfall statistics is that they depend strongly on the space-time scales over which rain data are averaged. A spectral model of precipitation has been developed based on a stochastic differential equation of fractional order for the point rain rate, that allows a concise description of the second moment statistics of rain at any prescribed space-time averaging scale. The model is thus capable of providing a unified description of the statistics of both radar and rain gauge data. The underlying dynamical equation can be expressed in terms of space-time derivatives of fractional orders that are adjusted together with other model parameters to fit the data. The form of the resulting spectrum gives the model adequate flexibility to capture the subtle interplay between the spatial and temporal scales of variability of rain but strongly constrains the predicted statistical behavior as a function of the averaging length and times scales. We test the model with radar and gauge data collected contemporaneously at the NASA TRMM ground validation sites located near Melbourne, Florida and in Kwajalein Atoll, Marshall Islands in the tropical Pacific. We estimate the parameters by tuning them to the second moment statistics of radar data. The model predictions are then found to fit the second moment statistics of the gauge data reasonably well without any further adjustment.
TRMM↗
Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces
Typical generative diffusion models rely on a Gaussian diffusion process for training the backward transformations, which can then be used to generate samples from Gaussian noise. However, real world data often takes place in discrete-state spaces, including many scientific applications. Here, we develop a theoretical formulation for arbitrary discrete-state Markov processes in the forward diffusion process using exact (as opposed to variational) analysis. We relate the theory to the existing continuous-state Gaussian diffusion as well as other approaches to discrete diffusion, and identify the corresponding reverse-time stochastic process and score function in the continuous-time setting, and the reverse-time mapping in the discrete-time setting. As an example of this framework, we introduce “Blackout Diffusion”, which learns to produce samples from an empty image instead of from noise. Numerical experiments on the CIFAR-10, Binarized MNIST, and CelebA datasets confirm the feasibility of our approach. Generalizing from specific (Gaussian) forward processes to discrete-state processes without a variational approximation sheds light on how to interpret diffusion models, which we discuss.
Santos, Javier E.↗
Modeling delay in genetic networks: From delay birth-death processes to delay stochastic differential equations
Delay is an important and ubiquitous aspect of many biochemical processes. For example, delay plays a central role in the dynamics of genetic regulatory networks as it stems from the sequential assembly of first mRNA and then protein. Genetic regulatory networks are therefore frequently modeled as stochastic birth-death processes with delay. Here, we examine the relationship between delay birth-death processes and their appropriate approximating delay chemical Langevin equations. We prove a quantitative bound on the error between the pathwise realizations of these two processes. Our results hold for both fixed delay and distributed delay. Simulations demonstrate that the delay chemical Langevin approximation is accurate even at moderate system sizes. It captures dynamical features such as the oscillatory behavior in negative feedback circuits, cross-correlations between nodes in a network, and spatial and temporal information in two commonly studied motifs of metastability in biochemical systems. Overall, these results provide a foundation for using delay stochastic differential equations to approximate the dynamics of birth-death processes with delay.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Using metacommunity ecology to understand environmental metabolomes
Abstract Environmental metabolomes are fundamentally coupled to microbially-linked biogeochemical processes within ecosystems. However, significant gaps exist in our understanding of their spatiotemporal organization, limiting our ability to uncover transferrable principles and predict ecosystem function. We propose that a theoretical paradigm, which integrates concepts from metacommunity ecology, is necessary to reveal underlying mechanisms governing metabolomes. We call this synthesis between ecology and metabolomics ‘meta-metabolome ecology’ and demonstrate its utility using a mass spectrometry dataset. We developed three relational metabolite dendrograms using molecular properties and putative biochemical transformations and performed ecological null modeling. Based upon null modeling results, we show that stochastic processes drove molecular properties while biochemical transformations were structured deterministically. We further suggest that potentially biochemically active metabolites were more deterministically assembled than less active metabolites. Understanding variation in the influences of stochasticity and determinism provides a way to focus attention on which meta-metabolomes and which parts of meta-metabolomes are most likely to be important to consider in mechanistic models. We propose that this paradigm will allow researchers to study the connections between ecological systems and their molecular processes in previously inaccessible detail.
54 ENVIRONMENTAL SCIENCES↗
UNCERTAINTY ANALYSIS FOR THE TECHA RIVER DOSIMETRY SYSTEM
Uncertainties in the doses estimated for the members of the Techa River Cohort (TRC) are being estimated with a two-dimensional Monte Carlo approach. In order to provide more accurate and precise estimates of individual dose (and thus more precise estimates of radiation risk) for the members of the TRC, a new dosimetric calculation system, the Techa River Dosimetry System-2009 (TRDS-2009) has been prepared. The deterministic version of the improved dosimetry system TRDS-2009D was basically completed in April 2009. Recent developments in evaluation of dose-response models in light of uncertain dose have highlighted the importance of different types of uncertainties in the development of individual dose estimates. These include uncertain parameters that may be either shared (common to some or all individuals) or unshared (a unique value for each person whose dose is to be estimated) within the dosimetric cohort. The nature of the type of uncertainty may be aleatory (random variability of true values due to stochastic processes) or epistemic (due to lack of complete knowledge about a unique quantity). Finally, there is a need to identify whether the structure of the errors is either related to measurement (the estimate differs from the true value by an error that is stochastically independent of the true value; frequently called classical uncertainty) or related to grouping (the true value varies from the estimate by an error that is random and is independent of the estimate; frequently called Berkson uncertainty). An approach has been developed that identifies the nature of the various input parameters and calculational methods incorporated in the Techa River Dosimetry System (based on the TRDS-2009D implementation), and a stochastic calculation model has been prepared to estimate the uncertainties in the dose estimates. This article reviews the concepts of uncertainty analysis, the equations, and input parameters, and then identifies the authors’ interpretations of their general nature. It presents the approach selected so that the stochastic, Monte-Carlo, implementation of the dosimetry system - TRDS-2009MC - provides useful information regarding the uncertainties of the doses.
uncertainty analysis, radiation dosimetry, Techa R↗
Latency considerations for stochastic optimizers in variational quantum algorithms
Variational quantum algorithms, which have risen to prominence in the noisy intermediate-scale quantum setting, require the implementation of a stochastic optimizer on classical hardware. To date, most research has employed algorithms based on the stochastic gradient iteration as the stochastic classical optimizer. In this work we propose instead using stochastic optimization algorithms that yield stochastic processes emulating the dynamics of classical deterministic algorithms. This approach results in methods with theoretically superior worst-case iteration complexities, at the expense of greater per-iteration sample (shot) complexities. We investigate this trade-off both theoretically and empirically and conclude that preferences for a choice of stochastic optimizer should explicitly depend on a function of both latency and shot execution times.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Noise-driven sidebranching in the boundary-layer model of dendritic solidification
In the context of local models of solidification, it is suggested that the sidebranching which appears to be a characteristic feature of dendritic processes may be generated by selective amplification of noise near the tip and that, under some circumstances, thermal fluctuations may be strong enough to explain the observed effects. This proposal is discussed based on the two-dimensional boundary-layer model with kinetic crystalline anisotropy. Experimental methods to distinguish this stochastic process from a deterministic, dynamical sidebranching mechanism are suggested, including the possibility of space-based experiments.
Pieters, R.↗