Search NASA⌕ Search

SEARCH · Search NASA

Results for “Back-Propagation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1991

Two papers on feed-forward networks

Connectionist feed-forward networks, trained with back-propagation, can be used both for nonlinear regression and for (discrete one-of-C) classification, depending on the form of training. This report contains two papers on feed-forward networks. The papers can be read independently. They are intended for the theoretically-aware practitioner or algorithm-designer; however, they also contain a review and comparison of several learning theories so they provide a perspective for the theoretician. The first paper works through Bayesian methods to complement back-propagation in the training of feed-forward networks. The second paper addresses a problem raised by the first: how to efficiently calculate second derivatives on feed-forward networks.

Buntine, Wray L.↗
NASA NTRS1992

Membership generation using multilayer neural network

There has been intensive research in neural network applications to pattern recognition problems. Particularly, the back-propagation network has attracted many researchers because of its outstanding performance in pattern recognition applications. In this section, we describe a new method to generate membership functions from training data using a multilayer neural network. The basic idea behind the approach is as follows. The output values of a sigmoid activation function of a neuron bear remarkable resemblance to membership values. Therefore, we can regard the sigmoid activation values as the membership values in fuzzy set theory. Thus, in order to generate class membership values, we first train a suitable multilayer network using a training algorithm such as the back-propagation algorithm. After the training procedure converges, the resulting network can be treated as a membership generation network, where the inputs are feature values and the outputs are membership values in the different classes. This method allows fairly complex membership functions to be generated because the network is highly nonlinear in general. Also, it is to be noted that the membership functions are generated from a classification point of view. For pattern recognition applications, this is highly desirable, although the membership values may not be indicative of the degree of typicality of a feature value in a particular class.

Kim, Jaeseok↗
NASA NTRS1987

The power of neural nets

Implementation of the Hopfield net which is used in the image processing type of applications where only partial information about the image may be available is discussed. The image classification type of algorithm of Hopfield and other learning algorithms, such as the Boltzmann machine and the back-propagation training algorithm, have many vital applications in space.

Ryan, J. P.↗
NASA NTRS1988

Supervised learning of probability distributions by neural networks

Supervised learning algorithms for feedforward neural networks are investigated analytically. The back-propagation algorithm described by Werbos (1974), Parker (1985), and Rumelhart et al. (1986) is generalized by redefining the values of the input and output neurons as probabilities. The synaptic weights are then varied to follow gradients in the logarithm of likelihood rather than in the error. This modification is shown to provide a more rigorous theoretical basis for the algorithm and to permit more accurate predictions. A typical application involving a medical-diagnosis expert system is discussed.

Baum, Eric B.↗
NASA NTRS1989

Classification of merged AVHRR and SMMR Arctic data with neural networks

A forward-feed back-propagation neural network is used to classify merged AVHRR and SMMR summer Arctic data. Four surface and eight cloud classes are identified. Partial memberships of each pixel to each class are examined for spectral ambiguities. Classification results are compared to manual interpretations and to those determined by a supervised maximum likelihood procedure. Results indicate that a neural network approach offers advantages in ease of use, interpretability, and utility for indistinct and time-variant spectral classes.

Key, J.↗
NASA NTRS1989

A comparison of two neural network schemes for navigation

Neural networks have been applied to tasks in several areas of artificial intelligence, including vision, speech, and language. Relatively little work has been done in the area of problem solving. Two approaches to path-finding are presented, both using neural network techniques. Both techniques require a training period. Training under the back propagation (BPL) method was accomplished by presenting representations of (current position, goal position) pairs as input and appropriate actions as output. The Hebbian/interactive activation (HIA) method uses the Hebbian rule to associate points that are nearby. A path to a goal is found by activating a representation of the goal in the network and processing until the current position is activated above some threshold level. BPL, using back-propagation learning, failed to learn, except in a very trivial fashion, that is equivalent to table lookup techniques. HIA, performed much better, and required storage of fewer weights. In drawing a comparison, it is important to note that back propagation techniques depend critically upon the forms of representation used, and can be sensitive to parameters in the simulations; hence the BPL technique may yet yield strong results.

Munro, Paul W.↗
NASA NTRS1990

A comparison of two neural network schemes for navigation

Neural networks have been applied to tasks in several areas of artificial intelligence, including vision, speech, and language. Relatively little work has been done in the area of problem solving. Two approaches to path-finding are presented, both using neural network techniques. Both techniques require a training period. Training under the back propagation (BPL) method was accomplished by presenting representations of current position, goal position pairs as input and appropriate actions as output. The Hebbian/interactive activation (HIA) method uses the Hebbian rule to associate points that are nearby. A path to a goal is found by activating a representation of the goal in the network and processing until the current position is activated above some threshold level. BPL, using back-propagation learning, failed to learn, except in a very trivial fashion, that is equivalent to table lookup techniques. HIA, performed much better, and required storage of fewer weights. In drawing a comparison, it is important to note that back propagation techniques depend critically upon the forms of representation used, and can be sensitive to parameters in the simulations; hence the BPL technique may yet yield strong results.

Munro, Paul↗
NASA NTRS1991

Simplified Learning Scheme For Analog Neural Network

Synaptic connections adjusted one at a time in small increments. Simplified gradient-descent learning scheme for electronic neural-network processor less efficient than better-known back-propagation scheme, but offers two advantages: easily implemented in circuitry because data-access circuitry separated from learning circuitry; and independence of data-access circuitry makes possible to implement feedforward as well as feedback networks, including those of multiple-attractor type. Important in such applications as recognition of patterns.

Eberhardt, Silvio P.↗
NASA NTRS1991

Modular, Multilayer Perceptron

Combination of proposed modular, multilayer perceptron and algorithm for its operation recognizes new objects after relatively brief retraining sessions. (Perceptron is multilayer, feedforward artificial neural network fully connected and trained via back-propagation learning algorithm.) Knowledge pertaining to each object to be recognized resides in subnetwork of full network, therefore not necessary to retrain full network to recognize each new object.

Cheng, Li-Jen↗
NASA NTRS1990

Receiving efficiency of monostatic pulsed coherent lidars. I - Theory. II - Applications

Pulsed coherent radars' receiving efficiency, eta, is presently investigated as a function of range z on the basis of a theory which relates eta(z) to both the transmitted laser intensity and the point-source receiving efficiency; this efficiency is calculated by a backward method employing the back-propagated local oscillator (BPLO) approach. The theory is applied to the ideal case, in order to study system optimization when both the transmitted and the BPLO fields at the antenna are Gaussian. In the second part of this work, eta(z) is calculated for various conditions of the NOAA/ERL Wave Propagation Laboratory CO2 Doppler lidar; the sensitivity of eta(z) to transmitted laser beam quality, telescope focal setting, telescope power, scanner astigmatism, and system misalignment.

Zhao, Yanzeng↗
NASA NTRS1990

Neural networks for function approximation in nonlinear control

Two neural network architectures are compared with a classical spline interpolation technique for the approximation of functions useful in a nonlinear control system. A standard back-propagation feedforward neural network and a cerebellar model articulation controller (CMAC) neural network are presented, and their results are compared with a B-spline interpolation procedure that is updated using recursive least-squares parameter identification. Each method is able to accurately represent a one-dimensional test function. Tradeoffs between size requirements, speed of operation, and speed of learning indicate that neural networks may be practical for identification and adaptation in a nonlinear control environment.

Linse, Dennis J.↗
NASA NTRS1990

Pattern recognition of clouds and ice in polar regions

The study is based on AVHRR imagery and results from Landsat high-spatial-resolution scenes. Among the textual features investigated are the gray level difference vector (GLDV), and sum and difference histogram (SADH) approaches as well as gray level run length, spatial-coherence, and spectral-histogram measures. The traditional stepwise discriminant analysis and neural-network analysis are used for the identification of 20 Arctic surface and cloud classes. A principal-component analysis and hybrid architecture employing a modularized competitive learning layer are utilized. It is pointed out that the cloud-classification accuracy comparable to that of back-propagation could be achieved with a training time two orders of magnitude faster.

Welch, R. M.↗
NASA NTRS1991

A general weight matrix formulation using optimal control

Classical methods from optimal control theory are used in deriving general forms for neural network weights. The network learning or application task is encoded in a performance index of a general structure. Consequently, different instances of this performance index lead to special cases of weight rules, including some well-known forms. Comparisons are made with the outer product rule, spectral methods, and recurrent back-propagation. Simulation results and comparisons are presented.

Farotimi, Oluseyi↗
NASA NTRS1992

Method Accelerates Training Of Some Neural Networks

Three-layer networks trained faster provided two conditions are satisfied: numbers of neurons in layers are such that majority of work done in synaptic connections between input and hidden layers, and number of neurons in input layer at least as great as number of training pairs of input and output vectors. Based on modified version of back-propagation method.

Shelton, Robert O.↗
NASA NTRS1992

Neural network classification of questionable EGRET events

High energy gamma rays (greater than 20 MeV) pair producing in the spark chamber of the Energetic Gamma Ray Telescope Experiment (EGRET) give rise to a characteristic but highly variable 3-D locus of spark sites, which must be processed to decide whether the event is to be included in the database. A significant fraction (about 15 percent or 10(exp 4) events/day) of the candidate events cannot be categorized (accept/reject) by an automated rule-based procedure; they are therefore tagged, and must be examined and classified manually by a team of expert analysts. We describe a feedforward, back-propagation neural network approach to the classification of the questionable events. The algorithm computes a set of coefficients using representative exemplars drawn from the preclassified set of questionable events. These coefficients map a given input event into a decision vector that, ideally, describes the correct disposition of the event. The net's accuracy is then tested using a different subset of preclassified events. Preliminary results demonstrate the net's ability to correctly classify a large proportion of the events for some categories of questionables. Current work includes the use of much larger training sets to improve the accuracy of the net.

Meetre, C. A.↗
NASA NTRS1992

Artificial neural network implementation of a near-ideal error prediction controller

A theory has been developed at the University of Virginia which explains the effects of including an ideal predictor in the forward loop of a linear error-sampled system. It has been shown that the presence of this ideal predictor tends to stabilize the class of systems considered. A prediction controller is merely a system which anticipates a signal or part of a signal before it actually occurs. It is understood that an exact prediction controller is physically unrealizable. However, in systems where the input tends to be repetitive or limited, (i.e., not random) near ideal prediction is possible. In order for the controller to act as a stability compensator, the predictor must be designed in a way that allows it to learn the expected error response of the system. In this way, an unstable system will become stable by including the predicted error in the system transfer function. Previous and current prediction controller include pattern recognition developments and fast-time simulation which are applicable to the analysis of linear sampled data type systems. The use of pattern recognition techniques, along with a template matching scheme, has been proposed as one realizable type of near-ideal prediction. Since many, if not most, systems are repeatedly subjected to similar inputs, it was proposed that an adaptive mechanism be used to 'learn' the correct predicted error response. Once the system has learned the response of all the expected inputs, it is necessary only to recognize the type of input with a template matching mechanism and then to use the correct predicted error to drive the system. Suggested here is an alternate approach to the realization of a near-ideal error prediction controller, one designed using Neural Networks. Neural Networks are good at recognizing patterns such as system responses, and the back-propagation architecture makes use of a template matching scheme. In using this type of error prediction, it is assumed that the system error responses be known for a particular input and modeled plant. These responses are used in the error prediction controller. An analysis was done on the general dynamic behavior that results from including a digital error predictor in a control loop and these were compared to those including the near-ideal Neural Network error predictor. This analysis was done for a second and third order system.

Mcvey, Eugene S.↗
NASA NTRS1992

Polar cloud and surface classification using AVHRR imagery - An intercomparison of methods

Six Advanced Very High-Resolution Radiometer local area coverage (AVHRR LAC) arctic scenes are classified into ten classes. Three different classifiers are examined: (1) the traditional stepwise discriminant analysis (SDA) method; (2) the feed-forward back-propagation (FFBP) neural network; and (3) the probabilistic neural network (PNN). More than 200 spectral and textural measures are computed. These are reduced to 20 features using sequential forward selection. Theoretical accuracy of the classifiers is determined using the bootstrap approach. Overall accuracy is 85.6 percent, 87.6 percent, and 87.0 percent for the SDA, FFBP, and PNN classifiers, respectively, with standard deviations of approximately 1 percent.

Welch, R. M.↗
NASA NTRS1992

Resource constrained design of artificial neural networks using comparator neural network

We present a systematic design method executed under resource constraints for automating the design of artificial neural networks using the back error propagation algorithm. Our system aims at finding the best possible configuration for solving the given application with proper tradeoff between the training time and the network complexity. The design of such a system is hampered by three related problems. First, there are infinitely many possible network configurations, each may take an exceedingly long time to train; hence, it is impossible to enumerate and train all of them to completion within fixed time, space, and resource constraints. Second, expert knowledge on predicting good network configurations is heuristic in nature and is application dependent, rendering it difficult to characterize fully in the design process. A learning procedure that refines this knowledge based on examples on training neural networks for various applications is, therefore, essential. Third, the objective of the network to be designed is ill-defined, as it is based on a subjective tradeoff between the training time and the network cost. A design process that proposes alternate configurations under different cost-performance tradeoff is important. We have developed a Design System which schedules the available time, divided into quanta, for testing alternative network configurations. Its goal is to select/generate and test alternative network configurations in each quantum, and find the best network when time is expended. Since time is limited, a dynamic schedule that determines the network configuration to be tested in each quantum is developed. The schedule is based on relative comparison of predicted training times of alternative network configurations using comparator network paradigm. The comparator network has been trained to compare training times for a large variety of traces of TSSE-versus-time collected during back-propagation learning of various applications.

Wah, Benjamin W.↗