Search NASA⌕ Search

SEARCH · Search NASA

Results for “stochastic approximation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Multidimensional stochastic approximation using locally contractive functions

A Robbins-Monro type multidimensional stochastic approximation algorithm which converges in mean square and with probability one to the fixed point of a locally contractive regression function is developed. The algorithm is applied to obtain maximum likelihood estimates of the parameters for a mixture of multivariate normal distributions.

Lawton, W. M.↗

Revisiting the ODE Method for Recursive Algorithms: Fast Convergence Using Quasi Stochastic Approximation

Several decades ago, Profs. Sean Meyn and Lei Guo were postdoctoral fellows at ANU, where they shared interest in recursive algorithms. It seems fitting to celebrate Lei Guo's 60th birthday with a review of the ODE Method and its recent evolution. The method has been regarded as a technique for algorithm analysis. It is argued that this viewpoint is backwards: The original stochastic approximation method was surely motivated by an ODE, and tools for analysis came much later (based on establishing robustness of Euler approximations). The paper presents a brief survey of recent research in machine learning that shows the power of algorithm design in continuous time, following by careful approximation to obtain a practical recursive algorithm. While these methods are usually presented in a stochastic setting, this is not a prerequisite. In fact, recent theory shows that rates of convergence can be dramatically accelerated by applying techniques inspired by quasi Monte-Carlo. Subject to conditions, the optimal rate of convergence can be obtained by applying the averaging technique of Polyak and Ruppert. The conditions are not universal, but theory suggests alternatives to achieve acceleration. The theory is illustrated with applications to gradient-free optimization, and policy gradient algorithms for reinforcement learning.

learning and adaptive systems in artificial intell↗

Source Localization using Stochastic Approximation and Least Squares Methods

This paper presents two approaches to locate the source of a chemical plume; Nonlinear Least Squares and Stochastic Approximation (SA) algorithms. Concentration levels of the chemical measured by special sensors are used to locate this source. Non-linear Least Squares technique is applied at different noise levels and compared with the localization using SA. For a noise corrupted data collected from a distributed set of chemical sensors, we show that SA methods are more efficient than Least Squares method. SA methods are often better at coping with noisy input information than other search methods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Estimation of correlation functions by stochastic approximation.

Consideration of the autocorrelation function of a zero-mean stationary random process. The techniques are applicable to processes with nonzero mean provided the mean is estimated first and subtracted. Two recursive techniques are proposed, both of which are based on the method of stochastic approximation and assume a functional form for the correlation function that depends on a number of parameters that are recursively estimated from successive records. One technique uses a standard point estimator of the correlation function to provide estimates of the parameters that minimize the mean-square error between the point estimates and the parametric function. The other technique provides estimates of the parameters that maximize a likelihood function relating the parameters of the function to the random process. Examples are presented.

Habibi, A.↗

Estimation of optical flow in airborne electro-optical sensors by stochastic approximation

The essence of motion or range estimation by passive electrooptical means is the ability to determine the correspondence of picture elements in pairs of image frames and to estimate their coordinates and their disparity (relative shifts) in the image plane of an electrooptical imaging sensor. The disparity can be in successive frames due to self-motion or in simultaneous frames of a stereo pair. A key issue is to provide these estimates on-line. This paper describes the theoretical background of such an interframe shift estimator. It is based on a stochastic gradient algorithm, specifically implementing a form of stochastic approximation, which can achieve rapid convergence of the shift estimate. Analytical and numerical simulation examples for random texture and isolated features validate the feasibility and the effectiveness of the estimator.

Merhav, S. J.↗

Stochastic Approximation for Multi-period Simulation Optimization with Streaming Input Data

We consider a continuous-valued simulation optimization (SO) problem, where a simulator is built to optimize an expected performance measure of a real-world system while parameters of the simulator are estimated from streaming data collected periodically from the system. At each period, a new batch of data is combined with the cumulative data and the parameters are re-estimated with higher precision. The system requires the decision variable to be selected in all periods. Therefore, it is sensible for the decision-maker to update the decision variable at each period by solving a more precise SO problem with the updated parameter estimate to reduce the performance loss with respect to the target system. We define this decision-making process as the multi-period SO problem and introduce a multi-period stochastic approximation (SA) framework that generates a sequence of solutions. Two algorithms are proposed: Re-start SA (ReSA) reinitializes the stepsize sequence in each period, whereas Warm-start SA (WaSA) carefully tunes the stepsizes, taking both fewer and shorter gradient-descent steps in later periods as parameter estimates become increasingly more precise. We show that under suitable strong convexity and regularity conditions, ReSA and WaSA achieve the best possible convergence rate in expected sub-optimality either when an unbiased or a simultaneous perturbation gradient estimator is employed, while WaSA accrues significantly lower computational cost as the number of periods increases. In addition, we present the regularized ReSA, which obviates the need to know the strong convexity constant and achieves the same convergence rate at the expense of additional computation.

Computer Science↗

Eddy Current Probe Parameters Identification Using a Genetic Algorithm and Simultaneous Perturbation Stochastic Approximation

This study tries to identify the coil parameters using numerical methods. The eddy current testing (ECT) is used for evaluation of a crack with the aid of numerical simulations by utilizing the identification of these parameters. In this study, a comparison of the performance of the GA and SPSA algorithms to identify the parameter values of the coil sensors are presented. So, the optimization probe geometry is introduced in the simulation with Three-dimensional finite element simulations (FLUX finite element code) were conducted to obtain eddy current signals resulting from a crack in a plate made of aluminium. The simulation results are compared with experimental measurements for the defect present in a plate.

42 ENGINEERING↗

Validity conditions for moment closure approximations in stochastic chemical kinetics

Approximations based on moment-closure (MA) are commonly used to obtain estimates of the mean molecule numbers and of the variance of fluctuations in the number of molecules of chemical systems. The advantage of this approach is that it can be far less computationally expensive than exact stochastic simulations of the chemical master equation. Here, we numerically study the conditions under which the MA equations yield results reflecting the true stochastic dynamics of the system. We show that for bistable and oscillatory chemical systems with deterministic initial conditions, the solution of the MA equations can be interpreted as a valid approximation to the true moments of the chemical master equation, only when the steady-state mean molecule numbers obtained from the chemical master equation fall within a certain finite range. The same validity criterion for monostable systems implies that the steady-state mean molecule numbers obtained from the chemical master equation must be above a certain threshold. For mean molecule numbers outside of this range of validity, the MA equations lead to either qualitatively wrong oscillatory dynamics or to unphysical predictions such as negative variances in the molecule numbers or multiple steady-state moments of the stationary distribution as the initial conditions are varied. Our results clarify the range of validity of the MA approach and show that pitfalls in the interpretation of the results can only be overcome through the systematic comparison of the solutions of the MA equations of a certain order with those of higher orders.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Universal resources for approximate and stochastic measurement-based quantum computation

We investigate which quantum states can serve as universal resources for approximate and stochastic measurement-based quantum computation in the sense that any quantum state can be generated from a given resource by means of single-qubit (local) operations assisted by classical communication. More precisely, we consider the approximate and stochastic generation of states, resulting, for example, from a restriction to finite measurement settings or from possible imperfections in the resources or local operations. We show that entanglement-based criteria for universality obtained in M. Van den Nest et al. [New J. Phys. 9, 204 (2007)] for the exact, deterministic case can be lifted to the much more general approximate, stochastic case. This allows us to move from the idealized situation (exact, deterministic universality) considered in previous works to the practically relevant context of nonperfect state preparation. We find that any entanglement measure fulfilling some basic requirements needs to reach its maximum value on some element of an approximate, stochastic universal family of resource states, as the resource size grows. This allows us to rule out various families of states as being approximate, stochastic universal. We prove that approximate, stochastic universality is in general a weaker requirement than deterministic, exact universality and provide resources that are efficient approximate universal, but not exact deterministic universal. We also study the robustness of universal resources for measurement-based quantum computation under realistic assumptions about the (imperfect) generation and manipulation of entangled states, giving an explicit expression for the impact that errors made in the preparation of the resource have on the possibility to use it for universal approximate and stochastic state preparation. Finally, we discuss the relation between our entanglement-based criteria and recent results regarding the uselessness of states with a high degree of geometric entanglement as universal resources [D. Gross et al., Phys. Rev. Lett. 102, 190501 (2009); M. J. Bremner et al., Phys. Rev. Lett 102, 190502 (2009)].

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Bounded-Degree Approximations of Stochastic Networks

We propose algorithms to approximate directed information graphs. Directed information graphs are probabilistic graphical models that depict causal dependencies between stochastic processes in a network. The proposed algorithms identify optimal and near-optimal approximations in terms of Kullback-Leibler divergence. The user-chosen sparsity trades off the quality of the approximation against visual conciseness and computational tractability. One class of approximations contains graphs with speci ed in-degrees. Another class additionally requires that the graph is connected. For both classes, we propose algorithms to identify the optimal approximations and also near-optimal approximations, using a novel relaxation of submodularity. We also propose algorithms to identify the r-best approximations among these classes, enabling robust decision making.

97 MATHEMATICS AND COMPUTING↗

Gaussian approximations for stochastic systems with delay: Chemical Langevin equation and application to a Brusselator system

We present a heuristic derivation of Gaussian approximations for stochastic chemical reaction systems with distributed delay. In particular, we derive the corresponding chemical Langevin equation. Due to the non-Markovian character of the underlying dynamics, these equations are integro-differential equations, and the noise in the Gaussian approximation is coloured. Following on from the chemical Langevin equation, a further reduction leads to the linear-noise approximation. We apply the formalism to a delay variant of the celebrated Brusselator model, and show how it can be used to characterise noise-driven quasi-cycles, as well as noise-triggered spiking. We find surprisingly intricate dependence of the typical frequency of quasi-cycles on the delay period.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Comparison of different moment-closure approximations for stochastic chemical kinetics

In recent years, moment-closure approximations (MAs) of the chemical master equation have become a popular method for the study of stochastic effects in chemical reaction systems. Several different MA methods have been proposed and applied in the literature, but it remains unclear how they perform with respect to each other. In this paper, we study the normal, Poisson, log-normal, and central-moment-neglect MAs by applying them to understand the stochastic properties of chemical systems whose deterministic rate equations show the properties of bistability, ultrasensitivity, and oscillatory behaviour. Our results suggest that the normal MA is favourable over the other studied MAs. In particular, we found that (i) the size of the region of parameter space where a closure gives physically meaningful results, e.g., positive mean and variance, is considerably larger for the normal closure than for the other three closures, (ii) the accuracy of the predictions of the four closures (relative to simulations using the stochastic simulation algorithm) is comparable in those regions of parameter space where all closures give physically meaningful results, and (iii) the Poisson and log-normal MAs are not uniquely defined for systems involving conservation laws in molecule numbers. We also describe the new software package MOCA which enables the automated numerical analysis of various MA methods in a graphical user interface and which was used to perform the comparative analysis presented in this paper. MOCA allows the user to develop novel closure methods and can treat polynomial, non-polynomial, as well as time-dependent propensity functions, thus being applicable to virtually any chemical reaction system.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Spectral convergence of the stochastic galerkin approximation to the boltzmann equation with multiple scales and large random perturbation in the collision kernel

In a 2018 article, spectral convergence and long-time decay of the numerical solution towards the global equilibrium of the stochastic Galerkin approximation for the Boltzmann equation with random inputs in the initial data and collision kernel for hard potentials and Maxwellian molecules under Grad's angular cutoff were established using the hypocoercive properties of the collisional kinetic model. One assumption for the random perturbation of the collision kernel is that the perturbation is in the order of the Knudsen number, which can be very small in the fluid dynamical regime. In this article, we remove this smallness assumption, and establish the same results but now for random perturbations of the collision kernel that can be of order one. The new analysis relies on the establishment of a spectral gap for the numerical collision operator.

97 MATHEMATICS AND COMPUTING↗