Engineering topics

Knuth, Kevin H.

Publications and source records attributed to Knuth, Kevin H..

At least 19 records

NASA NTRS2016

Combining Photometry from Kepler and TESS to Improve Short-Period Exoplanet Characterization

Planets emit thermal radiation and reflect incident light that they receive from their host stars. As a planet orbits its host star the photometric variations associated with these two effects produce very similar phase curves. If observed through only a single bandpass, this leads to a degeneracy between certain planetary parameters that hinder the precise characterization of such planets. However, observing the same planet through two different bandpasses gives much more information about the planet. Here we develop a Bayesian methodology for combining photometry from both Kepler and the Transiting Exoplanet Survey Satellite. In addition, we demonstrate via simulations that one can disentangle the reflected and thermally emitted light from the atmosphere of a hot-Jupiter as well as more precisely constrain both the geometric albedo and day-side temperature of the planet. This methodology can further be employed using various combinations of photometry from the James Webb Space Telescope, the Characterizing ExOplanet Satellite, or the PLATO mission.

photometric variations↗

NASA NTRS2005

Gesture Based Control and EMG Decomposition

This paper presents two probabilistic developments for use with Electromyograms (EMG). First described is a new-electric interface for virtual device control based on gesture recognition. The second development is a Bayesian method for decomposing EMG into individual motor unit action potentials. This more complex technique will then allow for higher resolution in separating muscle groups for gesture recognition. All examples presented rely upon sampling EMG data from a subject's forearm. The gesture based recognition uses pattern recognition software that has been trained to identify gestures from among a given set of gestures. The pattern recognition software consists of hidden Markov models which are used to recognize the gestures as they are being performed in real-time from moving averages of EMG. Two experiments were conducted to examine the feasibility of this interface technology. The first replicated a virtual joystick interface, and the second replicated a keyboard. Moving averages of EMG do not provide easy distinction between fine muscle groups. To better distinguish between different fine motor skill muscle groups we present a Bayesian algorithm to separate surface EMG into representative motor unit action potentials. The algorithm is based upon differential Variable Component Analysis (dVCA) [l], [2] which was originally developed for Electroencephalograms. The algorithm uses a simple forward model representing a mixture of motor unit action potentials as seen across multiple channels. The parameters of this model are iteratively optimized for each component. Results are presented on both synthetic and experimental EMG data. The synthetic case has additive white noise and is compared with known components. The experimental EMG data was obtained using a custom linear electrode array designed for this study.

Wheeler, Kevin R.↗

NASA NTRS2005

Revealing Relationships among Relevant Climate Variables with Information Theory

The primary objective of the NASA Earth-Sun Exploration Technology Office is to understand the observed Earth climate variability, thus enabling the determination and prediction of the climate's response to both natural and human-induced forcing. We are currently developing a suite of computational tools that will allow researchers to calculate, from data, a variety of information-theoretic quantities such as mutual information, which can be used to identify relationships among climate variables, and transfer entropy, which indicates the possibility of causal interactions. Our tools estimate these quantities along with their associated error bars, the latter of which is critical for describing the degree of uncertainty in the estimates. This work is based upon optimal binning techniques that we have developed for piecewise-constant, histogram-style models of the underlying density functions. Two useful side benefits have already been discovered. The first allows a researcher to determine whether there exist sufficient data to estimate the underlying probability density. The second permits one to determine an acceptable degree of round-off when compressing data for efficient transfer and storage. We also demonstrate how mutual information and transfer entropy can be applied so as to allow researchers not only to identify relations among climate variables, but also to characterize and quantify their possible causal interactions.

Knuth, Kevin H.↗

NASA NTRS2005

Accurate Biomass Estimation via Bayesian Adaptive Sampling

The following concepts were introduced: a) Bayesian adaptive sampling for solving biomass estimation; b) Characterization of MISR Rahman model parameters conditioned upon MODIS landcover. c) Rigorous non-parametric Bayesian approach to analytic mixture model determination. d) Unique U.S. asset for science product validation and verification.

Wheeler, Kevin R.↗

NASA NTRS2005

Informed Source Separation: A Bayesian Tutorial

Source separation problems are ubiquitous in the physical sciences; any situation where signals are superimposed calls for source separation to estimate the original signals. In h s tutorial I will discuss the Bayesian approach to the source separation problem. This approach has a specific advantage in that it requires the designer to explicitly describe the signal model in addition to any other information or assumptions that go into the problem description. This leads naturally to the idea of informed source separation, where the algorithm design incorporates relevant information about the specific problem. This approach promises to enable researchers to design their own high-quality algorithms that are specifically tailored to the problem at hand.

Knuth, Kevin H.↗

NASA NTRS2005

Coping with Trial-to-Trial Variability of Event Related Signals: A Bayesian Inference Approach

In electro-neurophysiology, single-trial brain responses to a sensory stimulus or a motor act are commonly assumed to result from the linear superposition of a stereotypic event-related signal (e.g. the event-related potential or ERP) that is invariant across trials and some ongoing brain activity often referred to as noise. To extract the signal, one performs an ensemble average of the brain responses over many identical trials to attenuate the noise. To date, h s simple signal-plus-noise (SPN) model has been the dominant approach in cognitive neuroscience. Mounting empirical evidence has shown that the assumptions underlying this model may be overly simplistic. More realistic models have been proposed that account for the trial-to-trial variability of the event-related signal as well as the possibility of multiple differentially varying components within a given ERP waveform. The variable-signal-plus-noise (VSPN) model, which has been demonstrated to provide the foundation for separation and characterization of multiple differentially varying components, has the potential to provide a rich source of information for questions related to neural functions that complement the SPN model. Thus, being able to estimate the amplitude and latency of each ERP component on a trial-by-trial basis provides a critical link between the perceived benefits of the VSPN model and its many concrete applications. In this paper we describe a Bayesian approach to deal with this issue and the resulting strategy is referred to as the differentially Variable Component Analysis (dVCA). We compare the performance of dVCA on simulated data with Independent Component Analysis (ICA) and analyze neurobiological recordings from monkeys performing cognitive tasks.

Ding, Mingzhou↗

NASA NTRS2004

Deriving Laws from Ordering Relations

The effect of Richard T. Cox's contribution to probability theory was to generalize Boolean implication among logical statements to degrees of implication, which are manipulated using rules derived from consistency with Boolean algebra. These rules are known as the sum rule, the product rule and Bayes Theorem, and the measure resulting from this generalization is probability. In this paper, I will describe how Cox s technique can be further generalized to include other algebras and hence other problems in science and mathematics. The result is a methodology that can be used to generalize an algebra to a calculus by relying on consistency with order theory to derive the laws of the calculus. My goals are to clear up the mysteries as to why the same basic structure found in probability theory appears in other contexts, to better understand the foundations of probability theory, and to extend these ideas to other areas by developing new mathematics and new physics. The relevance of this methodology will be demonstrated using examples from probability theory, number theory, geometry, information theory, and quantum mechanics.

Knuth, Kevin H.↗

NASA NTRS2004

Lattice Duality: The Origin of Probability and Entropy

Bayesian probability theory is an inference calculus, which originates from a generalization of inclusion on the Boolean lattice of logical assertions to a degree of inclusion represented by a real number. Dual to this lattice is the distributive lattice of questions constructed from the ordered set of down-sets of assertions, which forms the foundation of the calculus of inquiry-a generalization of information theory. In this paper we introduce this novel perspective on these spaces in which machine learning is performed and discuss the relationship between these results and several proposed generalizations of information theory in the literature.

Knuth, Kevin H.↗

NASA NTRS2004

Measuring Questions: Relevance and its Relation to Entropy

The Boolean lattice of logical statements induces the free distributive lattice of questions. Inclusion on this lattice is based on whether one question answers another. Generalizing the zeta function of the question lattice leads to a valuation called relevance or bearing, which is a measure of the degree to which one question answers another. Richard Cox conjectured that this degree can be expressed as a generalized entropy. With the assistance of yet another important result from Janos Acz6l, I show that this is indeed the case; and that the resulting inquiry calculus is a natural generalization of information theory. This approach provides a new perspective of the Principle of Maximum Entropy.

Knuth, Kevin H.↗

NASA NTRS2004

Discovering Planetary Nebula Geometries: Explorations with a Hierarchy of Models

Astronomical objects known as planetary nebulae (PNe) consist of a shell of gas expelled by an aging medium-sized star as it makes its transition from a red giant to a white dwarf. In many cases this gas shell can be approximately described as a prolate ellipsoid. Knowledge of the physics of ionization processes in this gaseous shell enables us to construct a model in three dimensions (3D) called the Ionization-Bounded Prolate Ellipsoidal Shell model (IBPES model). Using this model we can generate synthetic nebular images, which can be used in conjunction with Hubble Space Telescope (HST) images of actual PNe to perform Bayesian model estimation. Since the IBPES model is characterized by thirteen parameters, model estimation requires the search of a 13-dimensional parameter space. The 'curse of dimensionality,' compounded by a computationally intense forward problem, makes forward searches extremely time-consuming and frequently causes them to become trapped in local solutions. We find that both the speed and of the search can be improved by judiciously reducing the dimensionality of the search space. Our basic approach employs a hierarchy of models of increasing complexity that converges to the IBPES model. Earlier studies establish that a hierarchical sequence converges more quickly, and to a better solution, than a search relying only on the most complex model. Here we report results for a hierarchy of five models. The first three models treat the nebula as a 2D image, while the last two models explore its characteristics as a 3D object and enable us to characterize the physics of the nebula. This five-model hierarchy is applied to HST images of ellipsoidal PNe to estimate their geometric properties and gas density profiles.

Huyser, Karen A.↗

NASA NTRS2003

Intelligent machines in the twenty-first century: foundations of inference and inquiry

The last century saw the application of Boolean algebra to the construction of computing machines, which work by applying logical transformations to information contained in their memory. The development of information theory and the generalization of Boolean algebra to Bayesian inference have enabled these computing machines, in the last quarter of the twentieth century, to be endowed with the ability to learn by making inferences from data. This revolution is just beginning as new computational techniques continue to make difficult problems more accessible. Recent advances in our understanding of the foundations of probability theory have revealed implications for areas other than logic. Of relevance to intelligent machines, we recently identified the algebra of questions as the free distributive algebra, which will now allow us to work with questions in a way analogous to that which Boolean algebra enables us to work with logical statements. In this paper, we examine the foundations of inference and inquiry. We begin with a history of inferential reasoning, highlighting key concepts that have led to the automation of inference in modern machine-learning systems. We then discuss the foundations of inference in more detail using a modern viewpoint that relies on the mathematics of partially ordered sets and the scaffolding of lattice theory. This new viewpoint allows us to develop the logic of inquiry and introduce a measure describing the relevance of a proposed question to an unresolved issue. Last, we will demonstrate the automation of inference, and discuss how this new logic of inquiry will enable intelligent machines to ask questions. Automation of both inference and inquiry promises to allow robots to perform science in the far reaches of our solar system and in other star systems by enabling them not only to make inferences from data, but also to decide which question to ask, which experiment to perform, or which measurement to take given what they have learned and what they are designed to understand.

Review↗

NASA NTRS2003

Differentially Variable Component Analysis (dVCA): Identifying Multiple Evoked Components using Trial-to-Trial Variability

Electric potentials and magnetic fields generated by ensembles of synchronously active neurons in response to external stimuli provide information essential to understanding the processes underlying cognitive and sensorimotor activity. Interpreting recordings of these potentials and fields is difficult as each detector records signals simultaneously generated by various regions throughout the brain. We introduce the differentially Variable Component Analysis (dVCA) algorithm, which relies on trial-to-trial variability in response amplitude and latency to identify multiple components. Using simulations we evaluate the importance of response variability to component identification, the robustness of dVCA to noise, and its ability to characterize single-trial data. Finally, we evaluate the technique using visually evoked field potentials recorded at incremental depths across the layers of cortical area VI, in an awake, behaving macaque monkey.

Knuth, Kevin H.↗

NASA NTRS2003

Deriving Laws from Ordering Relations

It took much effort in the early days of non-Euclidean geometry to break away from the mindset that all spaces are flat and that two distinct parallel lines do not cross. Up to that point, all that was known was Euclidean geometry, and it was difficult to imagine anything else. We have suffered a similar handicap brought on by the enormous relevance of Boolean algebra to the problems of our age-logic and set theory. Previously, I demonstrated that the algebra of questions is not Boolean, but rather is described by the free distributive algebra. To get to this stage took much effort, as many obstacles-most self-placed-had to be overcome. As Boolean algebras were all I had ever known, it was almost impossible for me to imagine working with an algebra where elements do not have complements. With this realization, it became very clear that the sum and product rules of probability theory at the most basic level had absolutely nothing to do with the Boolean algebra of logical statements. Instead, a measure of degree of inclusion can be invented for many different partially ordered sets, and the sum and product rules fall out of the associativity and distributivity of the algebra. To reinforce this very important idea, this paper will go over how these constructions are made, while focusing on the underlying assumptions. I will derive the sum and product rules for a distributive lattice in general and demonstrate how this leads to probability theory on the Boolean lattice and is related to the calculus of quantum mechanical amplitudes on the partially ordered set of experimental setups. I will also discuss the rules that can be derived from modular lattices and their relevance to the cross-ratio of projective geometry.

Knuth, Kevin H.↗

NASA NTRS2003

Hierarchies of Models: Toward Understanding Planetary Nebulae

Stars like our sun (initial masses between 0.8 to 8 solar masses) end their lives as swollen red giants surrounded by cool extended atmospheres. The nuclear reactions in their cores create carbon, nitrogen and oxygen, which are transported by convection to the outer envelope of the stellar atmosphere. As the star finally collapses to become a white dwarf, this envelope is expelled from the star to form a planetary nebula (PN) rich in organic molecules. The physics, dynamics, and chemistry of these nebulae are poorly understood and have implications not only for our understanding of the stellar life cycle but also for organic astrochemistry and the creation of prebiotic molecules in interstellar space. We are working toward generating three-dimensional models of planetary nebulae (PNe), which include the size, orientation, shape, expansion rate and mass distribution of the nebula. Such a reconstruction of a PN is a challenging problem for several reasons. First, the data consist of images obtained over time from the Hubble Space Telescope (HST) and spectra obtained from Kitt Peak National Observatory (KPNO) and Cerro Tololo Inter-American Observatory (CTIO). These images are of course taken from a single viewpoint in space, which amounts to a very challenging tomographic reconstruction. Second, the fact that we have two disparate and orthogonal data types requires that we utilize a method that allows these data to be used together to obtain a solution. To address these first two challenges we employ Bayesian model estimation using a parameterized physical model that incorporates much prior information about the known physics of the PN. In our previous works we have found that the forward problem of the comprehensive model is extremely time consuming. To address this challenge, we explore the use of a set of hierarchical models, which allow us to estimate increasingly more detailed sets of model parameters. These hierarchical models of increasing complexity are akin to scientific theories of increasing sophistication, with each new model/theory being a refinement of a previous one by either incorporating additional prior information or by introducing a new set of parameters to model an entirely new phenomenon. We apply these models to both a simulated and a real ellipsoidal PN to initially estimate the position, angular size, and orientation of the nebula as a two-dimensional object and use these estimates to later examine its three-dimensional properties. The efficiency/accuracy tradeoffs of the techniques are studied to determine the advantages and disadvantages of employing a set of hierarchical models over a single comprehensive model.

Knuth, Kevin H.↗

NASA NTRS2003

Neural Dynamics Underlying Event-Related Potentials

There are two opposing hypotheses about the brain mechanisms underlying sensory event-related potentials (ERPs). One holds that sensory ERPs are generated by phase resetting of ongoing electroencephalographic (EEG) activity, and the other that they result from signal averaging of stimulus-evoked neural responses. We tested several contrasting predictions of these hypotheses by direct intracortical analysis of neural activity in monkeys. Our findings clearly demonstrate evoked response contributions to the sensory ERP in the monkey, and they suggest the likelihood that a mixed (Evoked/Phase Resetting) model may account for the generation of scalp ERPs in humans.

Shah, Ankoor S.↗

NASA NTRS2003

Intelligent Machines in the 21st Century: Automating the Processes of Inference and Inquiry

The last century saw the application of Boolean algebra toward the construction of computing machines, which work by applying logical transformations to information contained in their memory. The development of information theory and the generalization of Boolean algebra to Bayesian inference have enabled these computing machines. in the last quarter of the twentieth century, to be endowed with the ability to learn by making inferences from data. This revolution is just beginning as new computational techniques continue to make difficult problems more accessible. However, modern intelligent machines work by inferring knowledge using only their pre-programmed prior knowledge and the data provided. They lack the ability to ask questions, or request data that would aid their inferences. Recent advances in understanding the foundations of probability theory have revealed implications for areas other than logic. Of relevance to intelligent machines, we identified the algebra of questions as the free distributive algebra, which now allows us to work with questions in a way analogous to that which Boolean algebra enables us to work with logical statements. In this paper we describe this logic of inference and inquiry using the mathematics of partially ordered sets and the scaffolding of lattice theory, discuss the far-reaching implications of the methodology, and demonstrate its application with current examples in machine learning. Automation of both inference and inquiry promises to allow robots to perform science in the far reaches of our solar system and in other star systems by enabling them to not only make inferences from data, but also decide which question to ask, experiment to perform, or measurement to take given what they have learned and what they are designed to understand.

Knuth, Kevin H.↗

NASA NTRS2003

Questions Revisited: A Close Examination of Calculus of Inference and Inquiry

In this paper I examine more closely the way in which probability theory, the calculus of inference, is derived from the Boolean lattice structure of logical assertions ordered by implication. I demonstrate how the duality between the logical conjunction and disjunction in Boolean algebra is lost when deriving the probability calculus. In addition, I look more closely at the other lattice identities to verify that they are satisfied by the probability calculus. Last, I look towards developing the calculus of inquiry demonstrating that there is a sum and product rule for the relevance measure as well as a Bayes theorem. Current difficulties in deriving the complete inquiry calculus will also be discussed.

Knuth, Kevin H.↗

NASA NTRS2002

What is a Question?

A given question can be defined in terms of the set of statements or assertions that answer it. Application of the logic of inference to this set of assertions allows one to derive the logic of inquiry among questions. There are interesting symmetries between the logics of inference and inquiry; where probability describes the degree to which a premise implies an assertion, there exists an analogous quantity that describes the bearing or relevance that a question has on an outstanding issue. These have been extended to suggest that the logic of inquiry results in functional relationships analogous to, although more general than, those found in information theory. Employing lattice theory, I examine in greater detail the structure of the space of assertions and questions demonstrating that the symmetries between the logical relations in each of the spaces derive directly from the lattice structure. Furthermore, I show that while symmetries between the spaces exist, the two lattices are not isomorphic. The lattice of assertions is described by a Boolean lattice 2(sup N) whereas the lattice of real questions is shown to be a sublattice of the free distributive lattice FD(N) = 2(sup 2(sup N)). Thus there does not exist a one-to-one mapping of assertions to questions, there is no reflection symmetry between the two spaces, and questions in general do not possess unique complements. Last, with these lattice structures in mind, I discuss the relationship between probability, relevance and entropy.

Knuth, Kevin H.↗