Design studies of stochastic control systems
Stochastic control system design methodology noting applications to ASW, law enforcement, document retrieval and autopilots
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Stochastic control system design methodology noting applications to ASW, law enforcement, document retrieval and autopilots
An identification algorithm that uses the maximum likelihood technique to identify the unknown time delays, plant parameters, and noise covariances of linear discrete stochastic systems is presented. Cases of additive white noise and colored measurement noises are considered. The likelihood function is evaluated using either a minimum-variance (Kalman) filter or a minimal-order observer. The Kalman filter is used in the identification algorithm to provide minimum-variance estimates. The minimal-order observer is a lower-dimensional and computationally simpler filter, and is advantageous especially for systems with long delays. It provides a less optimal solution to the minimum-mean-square state estimation problem. The colored-noise observer algorithm has the disadvantage of having to compute an extra error covariance matrix of lower order.
This paper outlines the formulation of a robust fault detection and isolation scheme that can precisely detect and isolate simultaneous actuator and sensor faults for uncertain linear stochastic systems. The given robust fault detection scheme based on the discontinuous robust observer approach would be able to distinguish between model uncertainties and actuator failures and therefore eliminate the problem of false alarms. Since the proposed approach involves precise reconstruction of sensor faults, it can also be used for sensor fault identification and the reconstruction of true outputs from faulty sensor outputs. Simulation results presented here validate the effectiveness of the robust fault detection and isolation system.
To enhance the performance of the tracking property , this paper presents a novel control algorithm for a class of linear dynamic stochastic systems with unmeasurable states, where the performance enhancement loop is established based on Kalman filter. Without changing the existing closed loop with the PI controller, the compensative controller is designed to minimize the variances of the tracking errors using the estimated states and the propagation of state variances. Moreover, the stability of the closed-loop systems has been analyzed in the mean-square sense. A simulated example is included to show the effectiveness of the presented control algorithm, where encouraging results have been obtained.
A generalized problem of stochastic control is discussed in which multiple controllers with different data bases are present. The vehicle for the investigation is the finite state, finite memory (FSFM) stochastic control problem. Optimality conditions are obtained by deriving an equivalent deterministic optimal control problem. A FSFM minimum principle is obtained via the equivalent deterministic problem. The minimum principle suggests the development of a numerical optimization algorithm, the min-H algorithm. The relationship between the sufficiency of the minimum principle and the informational properties of the problem are investigated. A problem of hypothesis testing with 1-bit memory is investigated to illustrate the application of control theoretic techniques to information processing problems.
A class of multidimensional stochastic control problems with noisy data and bounded controls encountered in aerospace design is examined. The emphasis is on suboptimal design, the optimality being taken in quadratic mean sense. To that effect the problem is viewed as a stochastic version of the Lurie problem known from nonlinear control theory. The main result is a separation theorem (involving a nonlinear Kalman-like filter) suitable for Lurie-type approximations. The theorem allows for discontinuous characteristics. As a byproduct the existence of strong solutions to a class of non-Lipschitzian stochastic differential equations in dimensions is proven.
Steepest descent method extended to trajectory optimization problem for stochastically disturbed systems with application to Euler equation for white noise problem of reentering vehicle
Nonlinear perturbation theory for optimal estimation and control of time-discrete, stochastic, dynamic systems when measurement data is noisy
We present a heuristic derivation of Gaussian approximations for stochastic chemical reaction systems with distributed delay. In particular, we derive the corresponding chemical Langevin equation. Due to the non-Markovian character of the underlying dynamics, these equations are integro-differential equations, and the noise in the Gaussian approximation is coloured. Following on from the chemical Langevin equation, a further reduction leads to the linear-noise approximation. We apply the formalism to a delay variant of the celebrated Brusselator model, and show how it can be used to characterise noise-driven quasi-cycles, as well as noise-triggered spiking. We find surprisingly intricate dependence of the typical frequency of quasi-cycles on the delay period.
Time evolution of a quantum system which is influenced by a stochastically fluctuating environment is studied by means of the stochastic Liouville equation. The two different types of the stochastic Liouville equation and their relation are discussed. The stochastic Liouville equation is shown to be derived from the quantum master equation of the Lindblad under certain conditions. Relaxation processes of single and bipartite quantum systems which are initially correlated with a stochastic environment are investigated. It is shown the possibility that the stochastic fluctuation can create coherence and entanglement of a quantum system with the assistance of the initial correlation. The results are examined in the pure dephasing processes of qubits, which are caused by the nonstationary Gauss-Markov process and two-state jump Markov process.
In this note, we present a novel data-based approach to investigate the non-Gaussian stochastic distribution control problem. As the motivation of this note, the existing methods have been summarised regarding to the drawbacks, for example, neural network weights training for unknown stochastic distribution and so on. To overcome these disadvantages, a new transformation for dynamic probability density function is given by kernel density estimation using interpolation. Based upon this transformation, a representative model has been developed while the stochastic distribution control problem has been transformed into an optimisation problem. Then, data-based direct optimisation and identification-based indirect optimisation have been proposed. In addition, the convergences of the presented algorithms are analysed and the effectiveness of these algorithms has been evaluated by numerical examples. In summary, the contributions of this note are as follows: 1) a new data-based probability density function transformation is given; 2) the optimisation algorithms are given based on the presented model; and 3) a new research framework is demonstrated as the potential extensions to the existing stochastic distribution control.
Serving as a prerequisite to power system dynamic state estimation, the observability analysis of a power system dynamic model has recently attracted the attention of many power engineers. However, because this model is typically nonlinear and large-scale, the analysis of its observability is a challenge to the traditional derivative-based methods. Indeed, the linear-approximation-based approach may provide unreliable results while the nonlinear-technique-based approach inevitably faces extremely complicated derivations. Furthermore, because power systems are intrinsically stochastic, the traditional deterministic approaches may lead to inaccurate observability analyses. In this work, facing these challenges, we propose a novel polynomial-chaos-based derivative-free observability analysis approach that not only is free of any linear approximations, but also accounts for the stochasticity of the dynamic model while bringing a low implementation complexity. Furthermore, this approach enables us to quantify the degree of observability of a stochastic model, what conventional deterministic methods cannot do. The excellent performance of the proposed method has been demonstrated by performing extensive simulations using a synchronous generator model with IEEE-DC1A exciter and the TGOV1 turbine governor.
A technique for digital control of nonlinear stochastic plants is presented. The development achieves a practical digital algorithm with which the closed-loop system behaves in a classical Type I manner even with gross nonlinearities in the plant structure and low signal-to-noise power ratios. The design procedure is explained in detail and illustrated by an example whose simulated responses testify to the practicality of the approach.
In this paper, a novel hybrid control method is proposed to enhance the control performance of the PI based control system for a class of nonlinear and non-Gaussian stochastic dynamic processes with unmeasurable states. Firstly, to enhance the tracking performance of the PI controller where the PI parameters are fixed in many actual control systems, the compensative signal is formed using the extended Kalman filter(EKF) based state estimator and driven by the radial basis function (RBF) neural network based compensator. In addition, the weights of RBF is trained to minimize the entropy criterion of tracking error as the process is subjected to non-Gaussian disturbances. Meanwhile, since the precise statistical property of noises is hard to obtain for many industrial processes, the kernel density estimation (KDE) technique is employed in this paper to estimate the entropy. The convergence of RBF network is discussed and the stability of the resulted closed-loop hybrid control system is analyzed in mean square sense. Finally, a numerical example and a practical system testing are given to illustrate the effectiveness of the proposed control method.
This paper presents a novel filter design for the single-output stochastic non-linear systems subjected to non-Gaussian noises and the proposed assumptions. Based on a state transformation, the unmeasurable states of the systems can be estimated where non-linear terms in the systems have been eliminated. It has been shown that the estimation error is linearly dynamical regarding to the presented vector-valued filter gain which can be optimised by minimising the entropy-based performance criterion. In addition, the convergence of the presented algorithm is analysed in mean-square sense and a numerical example is given to verify the effectiveness of the presented filtering algorithm. Meanwhile, the extended Kalman filter, unscented particle filter and minimum entropy filter are given for the comparisons of the filtering performance. Following the presented framework, some extensions of the presented filtering algorithm are discussed to indicate the flexibility of the filter design. The contribution of this paper can be summarised as establishing a novel minimum entropy filtering framework which consists of model transformation, entropy optimisation and convergence analysis.
A new implicit dual technique is presented for stochastic adaptive control synthesis. In this technique, denoted as the method of utility costs (MOUC), a new control policy is derived from a known starting control policy by using a single iteration in time-varying policy space. An important result is that the new control policy yields a corresponding improvement in adaptive performance relative to the known starting policy. Since the starting policy can be chosen arbitrarily, the new policy has potentially superior performance relative to any existing design. Unlike other implicit dual designs, the MOUC can be applied to systems having constrained inputs, discontinuities, discrete valued disturbances, as well as in situations where the variances of the associated process statistics are large. A numerical example is given in which a controller is synthesized to provide improved performance relative to the Open-Loop Feedback policy of Dreyfus.
The paper deals with a class of multidimensional stochastic control problems with noisy data and bounded controls encountered in aerospace design. The emphasis is on suboptimal design, the optimality being taken in quadratic mean sense. To that effect the problem is viewed as a stochastic version of the Lurie problem known from nonlinear control theory. The main result is a separation theorem (involving a nonlinear Kalman-like filter) suitable for Lurie-type approximations. The theorem allows for discontinuous characteristics. As a byproduct the existence of strong solutions to a class of non-Lipschitzian stochastic differential equations in n dimensions is proved.
This paper develops improved stochastic models for the description of a large variety of fatigue crack growth phenomena that occur in components of considerable importance to the functionality and reliability of complex engineering structures. In essence, the models are based on the McGill-Markov and Closure-Lognormal stochastic processes. Not only do these models have the capability of predicting the statistical dispersion of crack growth rates, they also, by incorporating the concept of crack closure, have the capability of transferring stochastic crack growth properties measured under ideal laboratory conditions to situations of industrial significance, such as those occurring under adverse loading and/or environmental conditions. The primary data required in order to be in a position to estimate the pertinent parameters of these stochastic models are obtained from a statistically significant number of replicate tests. In this paper, both the theory and the experimental technique are illustrated using a Ti-6Al-4V alloy. Finally, important structural integrity, reliability, availability and maintainability concepts are developed and illustrated.