Search NASA⌕ Search

SEARCH · Search NASA

Results for “MARKOV”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
NASA NTRS1991

A Markov model reduction technique for fault tolerant processor reliability analysis

A fault tolerant processor (FTP) plays a key role in many high performance, safety-critical control system applications. Realistic modeling of an FTP is crucial to gaining a high degree of confidence in the reliability and safety analysis of such a system. While fidelity is clearly a major consideration, a practical model must also be kept to a moderate size to allow its incorporation into the overall system model. This paper presents a systematic reduction technique that starts from a complex, detailed model of a triple, redundant FTP and produces a low order approximation of very high accuracy. The existence of two distinct time scales represents the key to the success of the technique. No eigenvalue solution or coordinate transformation are needed. The reduced model captures all the important features of the detailed model, is amenable to an analytical solution and provides insight into the reconfiguration behavior of an FTP.

Schor, Andrei L.↗
NASA NTRS1993

System identification for Space Station Freedom using observer/Kalman filter Markov parameters

The Modal Identification Experiment (MIE) is a proposed experiment to define the dynamic characteristics of Space Station Freedom. Previous studies emphasized free-decay modal identification. The feasibility of using a forced response method (Observer/Kalman Filter Identification (OKID)) is addressed. The interest in using OKID is to determine the input mode shape matrix which can be used for controller design or control-structure interaction analysis, and investigate if forced response methods may aid in separating closely spaced modes. A model of the SC-7 configuration of Space Station Freedom was excited using simulated control system thrusters to obtain acceleration output. It is shown that an 'optimum' number of outputs exists for OKID. To recover global mode shapes, a modified method called Global-Local OKID was developed. This study shows that using data from a long forced response followed by free-decay leads to the 'best' modal identification. Twelve out of the thirteen target modes were identified for such an output.

Papadopoulos, Michael↗
NASA NTRS1999

Cooperation and Coordination Between Fuzzy Reinforcement Learning Agents in Continuous State Partially Observable Markov Decision Processes

Successful operations of future multi-agent intelligent systems require efficient cooperation schemes between agents sharing learning experiences. We consider a pseudo-realistic world in which one or more opportunities appear and disappear in random locations. Agents use fuzzy reinforcement learning to learn which opportunities are most worthy of pursuing based on their promise rewards, expected lifetimes, path lengths and expected path costs. We show that this world is partially observable because the history of an agent influences the distribution of its future states. We consider a cooperation mechanism in which agents share experience by using and-updating one joint behavior policy. We also implement a coordination mechanism for allocating opportunities to different agents in the same world. Our results demonstrate that K cooperative agents each learning in a separate world over N time steps outperform K independent agents each learning in a separate world over K*N time steps, with this result becoming more pronounced as the degree of partial observability in the environment increases. We also show that cooperation between agents learning in the same world decreases performance with respect to independent agents. Since cooperation reduces diversity between agents, we conclude that diversity is a key parameter in the trade off between maximizing utility from cooperation when diversity is low and maximizing utility from competitive coordination when diversity is high.

Berenji, Hamid R.↗
NASA NTRS1998

An Introduction to Markov Modeling: Concepts and Uses

Kharkov modeling is a modeling technique that is widely useful for dependability analysis of complex fault tolerant systems. It is very flexible in the type of systems and system behavior it can model. It is not, however, the most appropriate modeling technique for every modeling situation. The first task in obtaining a reliability or availability estimate for a system is selecting which modeling technique is most appropriate to the situation at hand. A person performing a dependability analysis must confront the question: is Kharkov modeling most appropriate to the system under consideration, or should another technique be used instead? The need to answer this gives rise to other more basic questions regarding Kharkov modeling: what are the capabilities and limitations of Kharkov modeling as a modeling technique? How does it relate to other modeling techniques? What kind of system behavior can it model? What kinds of software tools are available for performing dependability analyses with Kharkov modeling techniques? These questions and others will be addressed in this tutorial.

Boyd, Mark A.↗
NASA NTRS2002

Protein Kinase Classification with 2866 Hidden Markov Models and One Support Vector Machine

The main application considered in this paper is predicting true kinases from randomly permuted kinases that share the same length and amino acid distributions as the true kinases. Numerous methods already exist for this classification task, such as HMMs, motif-matchers, and sequence comparison algorithms. We build on some of these efforts by creating a vector from the output of thousands of structurally based HMMs, created offline with Pfam-A seed alignments using SAM-T99, which then must be combined into an overall classification for the protein. Then we use a Support Vector Machine for classifying this large ensemble Pfam-Vector, with a polynomial and chisquared kernel. In particular, the chi-squared kernel SVM performs better than the HMMs and better than the BLAST pairwise comparisons, when predicting true from false kinases in some respects, but no one algorithm is best for all purposes or in all instances so we consider the particular strengths and weaknesses of each.

Weber, Ryan↗