Search NASA⌕ Search

SEARCH · Search NASA

Results for “Learning Rate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1990

Satellite servicing economic study

Previous studies have shown that satellite servicing is cost effective; however, all of these studies were of different formats, dollar year, learning rates, availability, etc. Threfore, it was difficult to correlate any useful trends from these studies. The reviewed study was initiated to correlate the economic data into a common data base, using a common set of assumptions. A selected set of existed funded programs was then analyzed to provide an independent analysis of the servicing options and potential economic benefits.

Source record↗
NASA NTRS1990

Learning reliable manipulation strategies without initial physical models

A description is given of a robot, possessing limited sensory and effectory capabilities but no initial model of the effects of its actions on the world, that acquires such a model through exploration, practice, and observation. By acquiring an increasingly correct model of its actions, it generates increasingly successful plans to achieve its goals. In an apparently nondeterministic world, achieving reliability requires the identification of reliable actions and a preference for using such actions. Furthermore, by selecting its training actions carefully, the robot can significantly improve its learning rate.

Christiansen, Alan D.↗
NASA NTRS1993

Learning characteristics of a space-time neural network as a tether skiprope observer

The Software Technology Laboratory at the Johnson Space Center is testing a Space Time Neural Network (STNN) for observing tether oscillations present during retrieval of a tethered satellite. Proper identification of tether oscillations, known as 'skiprope' motion, is vital to safe retrieval of the tethered satellite. Our studies indicate that STNN has certain learning characteristics that must be understood properly to utilize this type of neural network for the tethered satellite problem. We present our findings on the learning characteristics including a learning rate versus momentum performance table.

Lea, Robert N.↗
NASA NTRS1993

Two generalizations of Kohonen clustering

The relationship between the sequential hard c-means (SHCM), learning vector quantization (LVQ), and fuzzy c-means (FCM) clustering algorithms is discussed. LVQ and SHCM suffer from several major problems. For example, they depend heavily on initialization. If the initial values of the cluster centers are outside the convex hull of the input data, such algorithms, even if they terminate, may not produce meaningful results in terms of prototypes for cluster representation. This is due in part to the fact that they update only the winning prototype for every input vector. The impact and interaction of these two families with Kohonen's self-organizing feature mapping (SOFM), which is not a clustering method, but which often leads ideas to clustering algorithms is discussed. Then two generalizations of LVQ that are explicitly designed as clustering algorithms are presented; these algorithms are referred to as generalized LVQ = GLVQ; and fuzzy LVQ = FLVQ. Learning rules are derived to optimize an objective function whose goal is to produce 'good clusters'. GLVQ/FLVQ (may) update every node in the clustering net for each input vector. Neither GLVQ nor FLVQ depends upon a choice for the update neighborhood or learning rate distribution - these are taken care of automatically. Segmentation of a gray tone image is used as a typical application of these algorithms to illustrate the performance of GLVQ/FLVQ.

Bezdek, James C.↗
NASA NTRS1992

Image segmentation using fuzzy LVQ clustering networks

In this note we formulate image segmentation as a clustering problem. Feature vectors extracted from a raw image are clustered into subregions, thereby segmenting the image. A fuzzy generalization of a Kohonen learning vector quantization (LVQ) which integrates the Fuzzy c-Means (FCM) model with the learning rate and updating strategies of the LVQ is used for this task. This network, which segments images in an unsupervised manner, is thus related to the FCM optimization problem. Numerical examples on photographic and magnetic resonance images are given to illustrate this approach to image segmentation.

Tsao, Eric Chen-Kuo↗
NASA NTRS1992

Learning characteristics of a space-time neural network as a tether skiprope observer

The Software Technology Laboratory at JSC is testing a Space Time Neural Network (STNN) for observing tether oscillations present during retrieval of a tethered satellite. Proper identification of tether oscillations, known as 'skiprope' motion, is vital to safe retrieval of the tethered satellite. Our studies indicate that STNN has certain learning characteristics that must be understood properly to utilize this type of neural network for the tethered satellite problem. We present our findings on the learning characteristics including a learning rate versus momentum performance table.

Lea, Robert N.↗
NASA NTRS1992

Multilayer perceptron, fuzzy sets, and classification

A fuzzy neural network model based on the multilayer perceptron, using the back-propagation algorithm, and capable of fuzzy classification of patterns is described. The input vector consists of membership values to linguistic properties while the output vector is defined in terms of fuzzy class membership values. This allows efficient modeling of fuzzy or uncertain patterns with appropriate weights being assigned to the backpropagated errors depending upon the membership values at the corresponding outputs. During training, the learning rate is gradually decreased in discrete steps until the network converges to a minimum error solution. The effectiveness of the algorithm is demonstrated on a speech recognition problem. The results are compared with those of the conventional MLP, the Bayes classifier, and the other related models.

Pal, Sankar K.↗
NASA NTRS1993

A review and analysis of neural networks for classification of remotely sensed multispectral imagery

A literature survey and analysis of the use of neural networks for the classification of remotely sensed multispectral imagery is presented. As part of a brief mathematical review, the backpropagation algorithm, which is the most common method of training multi-layer networks, is discussed with an emphasis on its application to pattern recognition. The analysis is divided into five aspects of neural network classification: (1) input data preprocessing, structure, and encoding; (2) output encoding and extraction of classes; (3) network architecture, (4) training algorithms; and (5) comparisons to conventional classifiers. The advantages of the neural network method over traditional classifiers are its non-parametric nature, arbitrary decision boundary capabilities, easy adaptation to different types of data and input structures, fuzzy output values that can enhance classification, and good generalization for use with multiple images. The disadvantages of the method are slow training time, inconsistent results due to random initial weights, and the requirement of obscure initialization values (e.g., learning rate and hidden layer size). Possible techniques for ameliorating these problems are discussed. It is concluded that, although the neural network method has several unique capabilities, it will become a useful tool in remote sensing only if it is made faster, more predictable, and easier to use.

Paola, Justin D.↗
NASA NTRS1993

A stable second order method for training back propagation networks

A simple method for improving the learning rate of the back-propagation algorithm is described. The basis of the method is that approximate second order corrections can be incorporated in the output units. The extended method leads to significant improvements in the convergence rate.

Nachtsheim, Philip R.↗
NASA NTRS1996

Automated Decomposition of Model-based Learning Problems

A new generation of sensor rich, massively distributed autonomous systems is being developed that has the potential for unprecedented performance, such as smart buildings, reconfigurable factories, adaptive traffic systems and remote earth ecosystem monitoring. To achieve high performance these massive systems will need to accurately model themselves and their environment from sensor information. Accomplishing this on a grand scale requires automating the art of large-scale modeling. This paper presents a formalization of [\em decompositional model-based learning (DML)], a method developed by observing a modeler's expertise at decomposing large scale model estimation tasks. The method exploits a striking analogy between learning and consistency-based diagnosis. Moriarty, an implementation of DML, has been applied to thermal modeling of a smart building, demonstrating a significant improvement in learning rate.

Williams, Brian C.↗
NASA NTRS1997

Using APEX to Model Anticipated Human Error: Analysis of a GPS Navigational Aid

The interface development process can be dramatically improved by predicting design facilitated human error at an early stage in the design process. The approach we advocate is to SIMULATE the behavior of a human agent carrying out tasks with a well-specified user interface, ANALYZE the simulation for instances of human error, and then REFINE the interface or protocol to minimize predicted error. This approach, incorporated into the APEX modeling architecture, differs from past approaches to human simulation in Its emphasis on error rather than e.g. learning rate or speed of response. The APEX model consists of two major components: (1) a powerful action selection component capable of simulating behavior in complex, multiple-task environments; and (2) a resource architecture which constrains cognitive, perceptual, and motor capabilities to within empirically demonstrated limits. The model mimics human errors arising from interactions between limited human resources and elements of the computer interface whose design falls to anticipate those limits. We analyze the design of a hand-held Global Positioning System (GPS) device used for radical and navigational decisions in small yacht recalls. The analysis demonstrates how human system modeling can be an effective design aid, helping to accelerate the process of refining a product (or procedure).

VanSelst, Mark↗
NASA NTRS2002

Method and system for training dynamic nonlinear adaptive filters which have embedded memory

Described herein is a method and system for training nonlinear adaptive filters (or neural networks) which have embedded memory. Such memory can arise in a multi-layer finite impulse response (FIR) architecture, or an infinite impulse response (IIR) architecture. We focus on filter architectures with separate linear dynamic components and static nonlinear components. Such filters can be structured so as to restrict their degrees of computational freedom based on a priori knowledge about the dynamic operation to be emulated. The method is detailed for an FIR architecture which consists of linear FIR filters together with nonlinear generalized single layer subnets. For the IIR case, we extend the methodology to a general nonlinear architecture which uses feedback. For these dynamic architectures, we describe how one can apply optimization techniques which make updates closer to the Newton direction than those of a steepest descent method, such as backpropagation. We detail a novel adaptive modified Gauss-Newton optimization technique, which uses an adaptive learning rate to determine both the magnitude and direction of update steps. For a wide range of adaptive filtering applications, the new training algorithm converges faster and to a smaller value of cost than both steepest-descent methods such as backpropagation-through-time, and standard quasi-Newton methods. We apply the algorithm to modeling the inverse of a nonlinear dynamic tracking system 5, as well as a nonlinear amplifier 6.

Rabinowitz, Matthew↗
NASA NTRS2015

Effects of False Tilt Cues on the Training of Manual Roll Control Skills

This paper describes a transfer-of-training study performed in the NASA Ames Vertica lMotion Simulator. The purpose of the study was to investigate the effect of false tilt cues on training and transfer of training of manual roll control skills. Of specific interest were the skills needed to control unstable roll dynamics of a mid-size transport aircraft close to the stall point. Nineteen general aviation pilots trained on a roll control task with one of three motion conditions: no motion, roll motion only, or reduced coordinated roll motion. All pilots transferred to full coordinated roll motion in the transfer session. A novel multimodal pilot model identification technique was successfully applied to characterize how pilots' use of visual and motion cues changed over the course of training and after transfer. Pilots who trained with uncoordinated roll motion had significantly higher performance during training and after transfer, even though they experienced the false tilt cues. Furthermore, pilot control behavior significantly changed during the two sessions, as indicated by increasing visual and motion gains, and decreasing lead time constants. Pilots training without motion showed higher learning rates after transfer to the full coordinated roll motion case.

simulators↗
NASA NTRS2023

Using Random Hiveminds to Predict Solar Energetic Particles (SEPs)

The Problem: The use of conventional neural networks (CoNNs) to predict SEPs has become popular, but neural network models do not follow one-size-fits-all approaches and their chaotic natures can yield completely different results on identical data sets. Committees of neural networks identical in input features have been used to solve this problem by (Aminalragia et al., 2021), but they have the possibility of all agreeing together in lockstep and missing crucial information. The Solution: (O’Keefe et al., 2023) propose a solution consisting of neural network estimators in an ensemble, but with features randomly removed from them in a layout known as a random hivemind (RH). The decision weight, learning rate, and epoch count of each member in this ensemble are boosted in relation to how well its individual features perform in a chi-square test.

SMD↗
DOE OSTI2020

Energy Storage Analysis

This study presents a comprehensive techno-economic characterization of energy storage and exible low carbon power generation technologies that can shift energy across days, weeks, or months to balance daily, weekly, and seasonal disparities in supply and demand. Energy storage technologies evaluated here include pumped hydropower storage (PHS), adiabatic and diabatic compressed air energy storage (CAES), vanadium redox flow batteries (VRBs), pumped thermal energy storage (P-TES), and renewably produced hydrogen stored in either geologic formations or underground pipes with re-electrification via combustion turbines in combined cycles, stationary proton exchange membrane (PEM) fuel cells, or the novel use of PEM fuel cells designed for heavy-duty vehicles (HDVs) which are expected to have shorter lives but also lower capital costs. We also evaluate flexible low-carbon power generation systems, including ethanol combustion in gas turbines and natural gas combustion with carbon capture and sequestration (CCS). We estimate current costs with literature data, use learning rates to characterize future costs, and develop capacity factors calibrated to an 85% renewables grid to calculate the levelized cost of energy (LCOE) of each technology. Results illustrate that at the 12-hour storage duration, PHS and CAES have the lowest LCOE with current costs, and VRBs become competitive if future costs are achieved. At the 120-hour storage duration, hydrogen systems with geologic storage and natural gas with CCS achieve the lowest LCOE in both current and future capital cost scenarios. In particular, the new configuration of HDV-PEM fuel cells with hydrogen storage in geologic formations evaluated here could lower the LCOE by 22-27% compared to stationary fuel cell systems typically evaluated and might help enable very high (>80%) renewable energy electric power systems. P-TES and hydrogen stored in underground pipes are the least-cost options at the 120-hour storage duration rating that do not require some form of geologic storage. Sensitivity analysis and Monte Carlo analysis illustrate that the general trends seen in this study are valid for a wide range of capacity factors and future cost scenarios. The study also illustrates that coproducing and selling hydrogen to other markets could reduce the LCOE of hydrogen systems by up to 39%.

carbon capture and sequestration↗
DOE OSTI2020

A Nonlocal-Gradient Descent Method for Inverse Design in Nanophotonics

Local-gradient-based optimization approaches lack nonlocal exploration abilityrequired for escaping from local minima when searching non-convex landscapes.A directional Gaussian smoothing (DGS) approach was recently proposed in [29]and used to define a truly nonlocal gradient, referred to as the DGS gradient, inorder to enable nonlocal exploration in high-dimensional black-box optimization.Promising results show that replacing the traditional local gradient with the nonlocalDGS gradient can significantly improve the performance of gradient-based methodsin optimizing highly multi-modal loss functions. However, the current DGS methodis designed for unbounded and uncontrained optimization problems, making itinapplicable to real-world engineering optimization problems where the tuningparameters are often bounded and the loss function is usually constrained byphysical processes. In this work, we propose to extend to the DGS approachto the constrained inverse design framework in order to find better optima ofmulti-modal loss functions. A series of adaptive strategies for smoothing radiusand learning rate updating are developed to improve the computational efficiencyand robustness. Our methodology is demonstrated by an example of designing ananoscale wavelength demultiplexer, and shows superior performance compared tothe state-of-the-art approaches. By incorporating volume constraints, the optimizeddesign achieves an equivalently high performance but significantly reduces theamount of material usage.

Bi, Sirui↗
DOE OSTI2020

Do optimization methods in deep learning applications matter?

With advances in deep learning, exponential data growth and increasing model complexity, developing efficient optimization methods are attracting much research attention. Several implementations favor the use of Conjugate Gradient (CG) and Stochastic Gradient Descent (SGD) as being practical and elegant solutions to achieve quick convergence, however, these optimization processes also present many limitations in learning across deep learning applications. Recent research is exploring higher-order optimization functions as better approaches, but these present very complex computational challenges for practical use. Comparing first and higher-order optimization functions, in this paper, our experiments reveal that Levemberg-Marquardt (LM) significantly supersedes optimal convergence but suffers from very large processing time increasing the training complexity of both, classification and reinforcement learning problems. Our experiments compare off-the-shelf optimization functions(CG, SGD, LM and L-BFGS) in standard CIFAR, MNIST, CartPole and FlappyBird experiments.The paper presents arguments on which optimization functions to use and further, which functions would benefit from parallelization efforts to improve pretraining time and learning rate convergence.

Ozyildirim, Buse Melis↗
DOE OSTI2023

Adaptive Data-Driven Model Predictive Control for Heat Pipe Microreactors

To establish a technical basis for self-regulating microreactors, a model predictive control (MPC) system is investigated to proactively respond to anomalies and disturbances in anticipation of potential deviations from operating setpoints. Due to the difficulty of developing a physics-based surrogate model that can accurately match plant data in various operating conditions, machine learning algorithms are used in MPC, which allow for learning from both simulation and operation data, thus efficiently describing the targeted transient with arbitrary accuracy. However, one of the biggest concerns in applying ML algorithms like artificial neural networks (ANNs) is that the predictive capabilities of ANN are limited by training data. If there are gaps between the training and target domain, the accuracy of an ANN can degrade significantly when it is used to predict unseen data. To improve the predictive capability of ANN and enable a confident use of data-driven MPCs outside the training data, this study proposes an adaptive data-driven MPC framework. The system will monitor the discrepancy between plant responses and surrogate predictions, fine-tune the ANN-based surrogate when a large discrepancy is detected, and continue MPC operation with updated surrogates. The framework is demonstrated on a point kinetic model for microreactors. The hyperparameters of the update strategy, including layers to update, error thresholds, learning rate discount, and number of data points used for fine-tuning, are optimized so the simulated microreactor is able to follow changes in setpoint with the smallest of deviations.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗