Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural encoding”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1991

A general weight matrix formulation using optimal control

Classical methods from optimal control theory are used in deriving general forms for neural network weights. The network learning or application task is encoded in a performance index of a general structure. Consequently, different instances of this performance index lead to special cases of weight rules, including some well-known forms. Comparisons are made with the outer product rule, spectral methods, and recurrent back-propagation. Simulation results and comparisons are presented.

Farotimi, Oluseyi↗
NASA NTRS1999

Electric organ discharges and electric images during electrolocation

Weakly electric fish use active electrolocation - the generation and detection of electric currents - to explore their surroundings. Although electrosensory systems include some of the most extensively understood circuits in the vertebrate central nervous system, relatively little is known quantitatively about how fish electrolocate objects. We believe a prerequisite to understanding electrolocation and its underlying neural substrates is to quantify and visualize the peripheral electrosensory information measured by the electroreceptors. We have therefore focused on reconstructing both the electric organ discharges (EODs) and the electric images resulting from nearby objects and the fish's exploratory behaviors. Here, we review results from a combination of techniques, including field measurements, numerical and semi-analytical simulations, and video imaging of behaviors. EOD maps are presented and interpreted for six gymnotiform species. They reveal diverse electric field patterns that have significant implications for both the electrosensory and electromotor systems. Our simulations generated predictions of the electric images from nearby objects as well as sequences of electric images during exploratory behaviors. These methods are leading to the identification of image features and computational algorithms that could reliably encode electrosensory information and may help guide electrophysiological experiments exploring the neural basis of electrolocation.

Review, Tutorial↗
NASA NTRS1999

Inspection of Space Station Cold Plate Using Visual and Automated Holographic Techniques

Real-time holography has been used to confirm the presence of non-uniformity in the construction of an International Space Station cold plate. Ultrasonic C-scans have previously shown suspected areas of cooling fin disbonds. But both neural-net processed and visual holography did not evidence any progressive permanent changes resulting from 3000 pressurization and relaxation cycles of a Dash 8 cold plate. Neural-net and visual inspections were performed of characteristic patterns generated from electronic time-average holograms of the vibrating cold plate. Normal modes of vibration were excited at very low amplitudes for this purpose, The neural nets were trained to flag very small changes in the mode shapes as encoded in the characteristic patterns. Both the whole cold plate and a zoomed region were inspected. The inspections were conducted before, after, and during pressurization and relaxation cycles of the cold plate. A water-filled cold plate was pressurized to 120 psig (827 kPa) and relaxed for each cycle. Each cycle required 5 seconds. Both the artificial neural networks and the inspectors were unable to detect changes in the mode shapes of the relaxed cold plate. The cold plate was also inspected visually using real-time holography and double-exposure holography. Regions of non-uniformity correlating with the C-scans were apparent, but the interference patterns did not change after 3000 pressurization and relaxation cycles. These tests constituted the first practical application of a neural-net inspection technique developed originally with support from the Director's Discretionary Fund at the Glenn Research Center at Lewis Field.

Decker, Arthur J.↗
NASA NTRS1991

A neural net based architecture for the segmentation of mixed gray-level and binary pictures

A neural-net-based architecture is proposed to perform segmentation in real time for mixed gray-level and binary pictures. In this approach, the composite picture is divided into 16 x 16 pixel blocks, which are identified as character blocks or image blocks on the basis of a dichotomy measure computed by an adaptive 16 x 16 neural net. For compression purposes, each image block is further divided into 4 x 4 subblocks; a one-bit nonparametric quantizer is used to encode 16 x 16 character and 4 x 4 image blocks; and the binary map and quantizer levels are obtained through a neural net segmentor over each block. The efficiency of the neural segmentation in terms of computational speed, data compression, and quality of the compressed picture is demonstrated. The effect of weight quantization is also discussed. VLSI implementations of such adaptive neural nets in CMOS technology are described and simulated in real time for a maximum block size of 256 pixels.

Tabatabai, Ali↗
NASA NTRS1991

An all digital implementation of a modified Hamming net for video compression with prediction and quantization circuits

The hardware and algorithms used to vector quantize (VQ) predicted pixel intensity differences for real-time video compression are described. The hardware is designed for rapid vector quantization performance, which entails the development of application-specific associative memory circuits. A modified DPCM algorithm is originally examined to determine how neural circuitry could enhance its operation. It was determined that quantization and encoding could be improved by consolidating these two functions into one, and by increasing the amount of information (i.e. number of pixels) quantized at a time. The result is a predictive scheme that vector quantizes differential values. Some of the disadvantages of VQ algorithms are solved using associative memories. The video compression algorithm and the associative memory design are described.

Kaul, Richard↗
NASA NTRS2002

Vestibular convergence patterns in vestibular nuclei neurons of alert primates

Sensory signal convergence is a fundamental and important aspect of brain function. Such convergence may often involve complex multidimensional interactions as those proposed for the processing of otolith and semicircular canal (SCC) information for the detection of translational head movements and the effective discrimination from physically congruent gravity signals. In the present study, we have examined the responses of primate rostral vestibular nuclei (VN) neurons that do not exhibit any eye movement-related activity using 0.5-Hz translational and three-dimensional (3D) rotational motion. Three distinct neural populations were identified. Approximately one-fourth of the cells exclusively encoded rotational movements (canal-only neurons) and were unresponsive to translation. The canal-only central neurons encoded head rotation in SCC coordinates, exhibited little orthogonal canal convergence, and were characterized with significantly higher sensitivities to rotation as compared to primary SCC afferents. Another fourth of the neurons modulated their firing rates during translation (otolith-only cells). During rotations, these neurons only responded when the axis of rotation was earth-horizontal and the head was changing orientation relative to gravity. The remaining one-half of VN neurons were sensitive to both rotations and translations (otolith + canal neurons). Unlike primary otolith afferents, however, central neurons often exhibited significant spatiotemporal (noncosine) tuning properties and a wide variety of response dynamics to translation. To characterize the pattern of SCC inputs to otolith + canal neurons, their rotational maximum sensitivity vectors were computed using exclusively responses during earth-vertical axis rotations (EVA). Maximum sensitivity vectors were distributed throughout the 3D space, suggesting strong convergence from multiple SCCs. These neurons were also tested with earth-horizontal axis rotations (EHA), which would activate both vertical canals and otolith organs. However, the recorded responses could not be predicted from a linear combination of EVA rotational and translational responses. In contrast, one-third of the neurons responded similarly during EVA and EHA rotations, although a significant response modulation was present during translation. Thus this subpopulation of otolith + canal cells, which included neurons with either high- or low-pass dynamics to translation, appear to selectively ignore the component of otolith-selective activation that is due to changes in the orientation of the head relative to gravity. Thus contrary to primary otolith afferents and otolith-only central neurons that respond equivalently to tilts relative to gravity and translational movements, approximately one-third of the otolith + canal cells seem to encode a true estimate of the translational component of the imposed passive head and body movement.

Non-NASA Center↗
NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or aerial wildfire operations reports to better understand the risks present. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement. These applications can benefit from the use of state-of-the-art natural language processing techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of adaptation of NLP tools to the aviation domain by fine-tuning transformer based models using aviation data sets. In 2018, a novel language model based on neural units (also called transformers) was created and became known as “Bidirectional Encoder Representations from Transformers” or BERT. This architecture combined with large amounts of English training data and innovative semi-supervised training tasks set the standard for what would later emerge as Large Language Models. The performance of these models was further improved by hyperparameter tuning and refinement of the semi-supervised training task and resulted in “Robustly Optimized BERT Pre-training Approach through hyperparameter tuning” or RoBERTa models. These pre-trained Large Language Models proved to be useful for a wide variety of natural language processing tasks such as text classification and question answering through a process called fine-tuning. The transformer architecture with pre-trained weights served as the basis with the last few layers replaced with layers fine-tuned to perform a new task e.g., a layer that provides a label for the entire input text. This process of fine-tuning can also be used to adapt the models to new domains; e.g., BioBERT started with the pre-trained BERT model and was completed by additional fine-tuning and training on biomedical documents. Transformer-based architectures can also be used to create rich representations of text called embeddings which can serve as the input to other machine learning models. This allows simpler algorithms such as logistic regression to use context-rich representations of the text while still remaining quick to train and evaluate. In the world of aviation, there is a growing demand for natural language processing and understanding but the domain presents unique challenges. Due to the technical content (and specialized language) of most aviation documents, fine-tuning pre-trained Large Language Models to specific tasks has not met the benchmark on natural language processing tasks set by simpler models trained from scratch on the data. To address this deficiency, this paper evaluates the improvements from fine-tuning a Large Language Model on a large set of aviation documents using the original semi-supervised training tasks before performing specific natural language tasks. In fine-tuning, a domain-specific dataset is used on the original training task but with the pre-trained Large Language Model instead of starting from a random initialization. This approach allows the model to be adapted to the specific domain language without discarding the information gained from training on general English data. This paper utilized two major dataset types to train and assess the RoBERTa fine-tuning performance. The first are 7,057 Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the semi-supervised fine-tuning. The second type is the 494 document classification labels to be used for evaluation. This down-stream evaluation aims to show the performance of the fine-tuned model, better understand how much data is needed for an effective fine-tuning, and how fine-tuning can be adapted for different applications in-the domain. After semi-supervised training, evaluation begins by encoding the documents for classification using the fine-tuned RoBERTa model. Then a logistic regression classifier is trained to label the document type and compared against our ground truth labels. This currently leads to a 82.8% accuracy on 10-fold cross validation showing improvement over baseline RoBERTa which achieved 81.0%. We plan to measure the improvements on additional tasks and it is expected that these improvements will lead to more robust models that can tackle the natural language processing challenges present in aviation datasets.

ATM↗
NASA NTRS2008

Spiking Neurons for Analysis of Patterns

Artificial neural networks comprising spiking neurons of a novel type have been conceived as improved pattern-analysis and pattern-recognition computational systems. These neurons are represented by a mathematical model denoted the state-variable model (SVM), which among other things, exploits a computational parallelism inherent in spiking-neuron geometry. Networks of SVM neurons offer advantages of speed and computational efficiency, relative to traditional artificial neural networks. The SVM also overcomes some of the limitations of prior spiking-neuron models. There are numerous potential pattern-recognition, tracking, and data-reduction (data preprocessing) applications for these SVM neural networks on Earth and in exploration of remote planets. Spiking neurons imitate biological neurons more closely than do the neurons of traditional artificial neural networks. A spiking neuron includes a central cell body (soma) surrounded by a tree-like interconnection network (dendrites). Spiking neurons are so named because they generate trains of output pulses (spikes) in response to inputs received from sensors or from other neurons. They gain their speed advantage over traditional neural networks by using the timing of individual spikes for computation, whereas traditional artificial neurons use averages of activity levels over time. Moreover, spiking neurons use the delays inherent in dendritic processing in order to efficiently encode the information content of incoming signals. Because traditional artificial neurons fail to capture this encoding, they have less processing capability, and so it is necessary to use more gates when implementing traditional artificial neurons in electronic circuitry. Such higher-order functions as dynamic tasking are effected by use of pools (collections) of spiking neurons interconnected by spike-transmitting fibers. The SVM includes adaptive thresholds and submodels of transport of ions (in imitation of such transport in biological neurons). These features enable the neurons to adapt their responses to high-rate inputs from sensors, and to adapt their firing thresholds to mitigate noise or effects of potential sensor failure. The mathematical derivation of the SVM starts from a prior model, known in the art as the point soma model, which captures all of the salient properties of neuronal response while keeping the computational cost low. The point-soma latency time is modified to be an exponentially decaying function of the strength of the applied potential. Choosing computational efficiency over biological fidelity, the dendrites surrounding a neuron are represented by simplified compartmental submodels and there are no dendritic spines. Updates to the dendritic potential, calcium-ion concentrations and conductances, and potassium-ion conductances are done by use of equations similar to those of the point soma. Diffusion processes in dendrites are modeled by averaging among nearest-neighbor compartments. Inputs to each of the dendritic compartments come from sensors. Alternatively or in addition, when an affected neuron is part of a pool, inputs can come from other spiking neurons. At present, SVM neural networks are implemented by computational simulation, using algorithms that encode the SVM and its submodels. However, it should be possible to implement these neural networks in hardware: The differential equations for the dendritic and cellular processes in the SVM model of spiking neurons map to equivalent circuits that can be implemented directly in analog very-large-scale integrated (VLSI) circuits.

Huntsberger, Terrance↗
NASA NTRS2020

Amino Acid Encoding for Deep Learning Applications

Background: The number of applications of deep learning algorithms in bioinformatics is increasing as they usually achieve superior performance over classical approaches, especially, when bigger training datasets are available. In deep learning applications, discrete data, e.g. words or n-grams in language, or amino acids or nucleotides in bioinformatics, are generally represented as a continuous vector through an embedding matrix. Recently, learning this embedding matrix directly from the data as part of the continuous iteration of the model to optimize the target prediction – a process called ‘end-to-end learning’ – has led to state-of-the-art results in many fields. Although usage of embeddings is well described in the bioinformatics literature, the potential of end-to-end learning for single amino acids, as compared to more classical manually-curated encoding strategies, has not been systematically addressed. To this end, we compared classical encoding matrices, namely one-hot, VHSE8 and BLOSUM62, to end-to-end learning of amino acid embeddings for two different prediction tasks using three widely used architectures, namely recurrent neural networks (RNN), convolutional neural networks (CNN), and the hybrid CNN-RNN. Results: By using different deep learning architectures, we show that end-to-end learning is on par with classical encodings for embeddings of the same dimension even when limited training data is available, and might allow for a reduction in the embedding dimension without performance loss, which is critical when deploying the models to devices with limited computational capacities. We found that the embedding dimension is a major factor in controlling the model performance. Surprisingly, we observed that deep learning models are capable of learning from random vectors of appropriate dimension. Conclusion: Our study shows that end-to-end learning is a flexible and powerful method for amino acid encoding. Further, due to the flexibility of deep learning systems, amino acid encoding schemes should be benchmarked against random vectors of the same dimension to disentangle the information content provided by the encoding scheme from the distinguishability effect provided by the scheme.

Deep-learning↗
NASA NTRS2020

Amino Acid Encoding for Deep Learning Applications

Background: The number of applications of deep learning algorithms in bioinformatics is increasing as they usually achieve superior performance over classical approaches, especially, when bigger training datasets are available. In deep learning applications, discrete data, e.g. words or n-grams in language, or amino acids or nucleotides in bioinformatics, are generally represented as a continuous vector through an embedding matrix. Recently, learning this embedding matrix directly from the data as part of the continuous iteration of the model to optimize the target prediction – a process called ‘end-to-end learning’ – has led to state-ofthe-art results in many fields. Although usage of embeddings is well described in the bioinformatics literature, the potential of end-to-end learning for single amino acids, as compared to more classical manually-curated encoding strategies, has not been systematically addressed. To this end, we compared classical encoding matrices, namely one-hot, VHSE8 and BLOSUM62, to end-to-end learning of amino acid embeddings for two different prediction tasks using three widely used architectures, namely recurrent neural networks (RNN), convolutional neural networks (CNN), and the hybrid CNN-RNN. Results: By using different deep learning architectures, we show that end-to-end learning is on par with classical encodings for embeddings of the same dimension even when limited training data is available, and might allow for a reduction in the embedding dimension without performance loss, which is critical when deploying the models to devices with limited computational capacities. We found that the embedding dimension is a major factor in controlling the model performance. Surprisingly, we observed that deep learning models are capable of learning from random vectors of appropriate dimension. Conclusion: Our study shows that end-to-end learning is a flexible and powerful method for amino acid encoding. Further, due to the flexibility of deep learning systems, amino acid encoding schemes should be benchmarked against random vectors of the same dimension to disentangle the information content provided by the encoding scheme from the distinguishability effect provided by the scheme.

Hesham ElAbd↗
NASA NTRS1991

Neural mechanisms of kinesthesia

Our primary interest is in where the signals for position and movement of the limbs originate and how the information is encoded. Humans can independently sense the positions and movements of their limbs in the absence of vision, and their senses are derived from mechanoreceptors located in the limbs. Receptors in muscles, skin and joints are potential sources of position and movement signals, but how (and whether) each of these groups contributes to kinesthesia remains uncertain. Recent evidence indicates that a sense of static-position derives from length receptors in muscle, whereas receptors in the skin, and possibly though not likely the joints, can signal movement of a limb but not its static position. At present, we are examining the consequences of not having a static-position sense at the fingers apart from an inability to detect very slow displacements. Using a more demanding test, we have found no substantial difference in the accuracy of reproducing target positions passively imposed on the MCP joint and the proximal interphalangeal (PIP) joint of the index finger. Our working hypothesis is that subjects normally use movement sense mechanisms to detect displacements, probably because they are faster, but how (or whether) subjects manage to obtain an absolute position reference with only a movement sense available remains unknown and is an issue under investigation.

Clark, Francis J.↗
NASA NTRS1992

Neural processing of gravity information

The goal of this project was to use the linear acceleration capabilities of the NASA Vestibular Research Facility (VRF) at Ames Research Center to directly examine encoding of linear accelerations in the vestibular system of the cat. Most previous studies, including my own, have utilized tilt stimuli, which at very low frequencies (e.g., 'static tilt') can be considered a reasonably pure linear acceleration (e.g., 'down'); however, higher frequencies of tilt, necessary for understanding the dynamic processing of linear acceleration information, necessarily involves rotations which can stimulate the semicircular canals. The VRF, particularly the Long Linear Sled, has promise to provide controlled pure linear accelerations at a variety of stimulus frequencies, with no confounding angular motion.

Schor, Robert H.↗
NASA NTRSDate not supplied

NEUROSPF: A Tool For the Symbolic Analysis of Neural Networks

This paper presents NEUROSPF, a tool for the symbolic analysis of neural networks. Given a trained neural network model, the tool extracts the architecture and model parameters and translates them into a Java representation that is amenable for analysis using the Symbolic PathFinder symbolic execution tool. Notably, NEUROSPF encodes specialized peer classes for parsing the model’s parameters, thereby enabling efficient analysis. With NEUROSPF the user has the flexibility to specify either the inputs or the network internal parameters as symbolic, promoting the application of program analysis and testing approaches from software engineering to the field of machine learning. For instance, NEUROSPF can be used for coverage-based testing and test generation, finding adversarial examples and also constraint-based repair of neural networks, thus improving the reliability of neural networks and of the applications that use them.

neural networks↗
NASA NTRS1990

Real-time optical correlator using computer-generated holographic filter on a liquid crystal light valve

Limitations associated with the binary phase-only filter often used in optical correlators are presently circumvented in the writing of complex-valued data on a gray-scale spatial light modulator through the use of a computer-generated hologram (CGH) algorithm. The CGH encodes complex-valued data into nonnegative real CGH data in such a way that it may be encoded in any of the available gray-scale spatial light modulators. A CdS liquid-crystal light valve is used for the complex-valued CGH encoding; computer simulations and experimental results are compared, and the use of such a CGH filter as the synapse hologram in a holographic optical neural net is discussed.

Chao, Tien-Hsin↗
NASA NTRS1992

High level cognitive information processing in neural networks

Two related research efforts were addressed: (1) high-level connectionist cognitive modeling; and (2) local neural circuit modeling. The goals of the first effort were to develop connectionist models of high-level cognitive processes such as problem solving or natural language understanding, and to understand the computational requirements of such models. The goals of the second effort were to develop biologically-realistic model of local neural circuits, and to understand the computational behavior of such models. In keeping with the nature of NASA's Innovative Research Program, all the work conducted under the grant was highly innovative. For instance, the following ideas, all summarized, are contributions to the study of connectionist/neural networks: (1) the temporal-winner-take-all, relative-position encoding, and pattern-similarity association techniques; (2) the importation of logical combinators into connection; (3) the use of analogy-based reasoning as a bridge across the gap between the traditional symbolic paradigm and the connectionist paradigm; and (4) the application of connectionism to the domain of belief representation/reasoning. The work on local neural circuit modeling also departs significantly from the work of related researchers. In particular, its concentration on low-level neural phenomena that could support high-level cognitive processing is unusual within the area of biological local circuit modeling, and also serves to expand the horizons of the artificial neural net field.

Barnden, John A.↗
NASA NTRS1993

Classifying multispectral data by neural networks

Several energy functions for synthesizing neural networks are tested on 2-D synthetic data and on Landsat-4 Thematic Mapper data. These new energy functions, designed specifically for minimizing misclassification error, in some cases yield significant improvements in classification accuracy over the standard least mean squares energy function. In addition to operating on networks with one output unit per class, a new energy function is tested for binary encoded outputs, which result in smaller network sizes. The Thematic Mapper data (four bands were used) is classified on a single pixel basis, to provide a starting benchmark against which further improvements will be measured. Improvements are underway to make use of both subpixel and superpixel (i.e. contextual or neighborhood) information in tile processing. For single pixel classification, the best neural network result is 78.7 percent, compared with 71.7 percent for a classical nearest neighbor classifier. The 78.7 percent result also improves on several earlier neural network results on this data.

Telfer, Brian A.↗
NASA NTRS1998

The adequate stimulus for avian short latency vestibular responses to linear translation

Transient linear acceleration stimuli have been shown to elicit eighth nerve vestibular compound action potentials in birds and mammals. The present study was undertaken to better define the nature of the adequate stimulus for neurons generating the response in the chicken (Gallus domesticus). In particular, the study evaluated the question of whether the neurons studied are most sensitive to the maximum level of linear acceleration achieved or to the rate of change in acceleration (da/dt, or jerk). To do this, vestibular response thresholds were measured as a function of stimulus onset slope. Traditional computer signal averaging was used to record responses to pulsed linear acceleration stimuli. Stimulus onset slope was systematically varied. Acceleration thresholds decreased with increasing stimulus onset slope (decreasing stimulus rise time). When stimuli were expressed in units of jerk (g/ms), thresholds were virtually constant for all stimulus rise times. Moreover, stimuli having identical jerk magnitudes but widely varying peak acceleration levels produced virtually identical responses. Vestibular response thresholds, latencies and amplitudes appear to be determined strictly by stimulus jerk magnitudes. Stimulus attributes such as peak acceleration or rise time alone do not provide sufficient information to predict response parameter quantities. Indeed, the major response parameters were shown to be virtually independent of peak acceleration levels or rise time when these stimulus features were isolated and considered separately. It is concluded that the neurons generating short latency vestibular evoked potentials do so as "jerk encoders" in the chicken. Primary afferents classified as "irregular", and which traditionally fall into the broad category of "dynamic" or "phasic" neurons, would seem to be the most likely candidates for the neural generators of short latency vestibular compound action potentials.

NASA Discipline Neuroscience↗