Search NASA⌕ Search

SEARCH · Search NASA

Results for “gradient methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2011

Soft magnetism and microwave magnetic properties of Fe-Co-Hf films deposited by composition gradient sputtering

A composition gradient sputtering method was employed to deposit a series of (Fe{sub x}Co{sub 1-x}){sub 1-y}Hf{sub y} alloy films with different Fe/Co atomic ratios and Hf-doping. At Fe/Co = 70/30, the magnetically annealed (Fe{sub 0.7}Co{sub 0.3}){sub 1-y}Hf{sub y} films with y = 0.087-0.124 showed a very high uniaxial anisotropy and an ultrahigh ferromagnetic resonance frequency over 7 GHz after the films were annealed at 350 deg. C, while at Fe/Co 50/50, the as-deposited (Fe{sub 0.5}Co{sub 0.5}){sub 1-y}Hf{sub y} films with y 0.074-0.168 showed an excellent in-plane uniaxial anisotropy and high ferromagnetic resonance frequency over 3 GHz. These Fe-Co-Hf films deposited by the composition gradient sputtering method exhibited ahigh saturation magnetization of 1.8 - 2.2 T, a large uniaxial anisotropy field of 200 - 500 Oe, and a high ferromagnetic resonance frequency over 7 GHz, which provides great opportunities for integrated magnetic devices.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Optimization Methods for Large-Scale Machine Learning

Here this paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what makes them challenging. A major theme of our study is that large-scale machine learning represents a distinctive setting in which the stochastic gradient (SG) method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter. Based on this viewpoint, we present a comprehensive theory of a straightforward, yet versatile SG algorithm, discuss its practical behavior, and highlight opportunities for designing algorithms with improved performance. This leads to a discussion about the next generation of optimization methods for large-scale machine learning, including an investigation of two main streams of research on techniques that diminish noise in the stochastic directions and methods that make use of second-order derivative approximations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2008

Accuracy of Gradient Reconstruction on Grids with High Aspect Ratio

Gradient approximation methods commonly used in unstructured-grid finite-volume schemes intended for solutions of high Reynolds number flow equations are studied comprehensively. The accuracy of gradients within cells and within faces is evaluated systematically for both node-centered and cell-centered formulations. Computational and analytical evaluations are made on a series of high-aspect-ratio grids with different primal elements, including quadrilateral, triangular, and mixed element grids, with and without random perturbations to the mesh. Both rectangular and cylindrical geometries are considered; the latter serves to study the effects of geometric curvature. The study shows that the accuracy of gradient reconstruction on high-aspect-ratio grids is determined by a combination of the grid and the solution. The contributors to the error are identified and approaches to reduce errors are given, including the addition of higher-order terms in the direction of larger mesh spacing. A parameter GAMMA characterizing accuracy on curved high-aspect-ratio grids is discussed and an approximate-mapped-least-square method using a commonly-available distance function is presented; the method provides accurate gradient reconstruction on general grids. The study is intended to be a reference guide accompanying the construction of accurate and efficient methods for high Reynolds number applications

Thomas, James↗
DOE OSTI2020

Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks

Here we propose two approaches of locally adaptive activation functions namely, layer-wise and neuron-wise locally adaptive activation functions, which improve the performance of deep and physics-informed neural networks. The local adaptation of activation function is achieved by introducing a scalable parameter in each layer (layer-wise) and for every neuron (neuron-wise) separately, and then optimizing it using a variant of stochastic gradient descent algorithm. In order to further increase the training speed, an activation slope-based slope recovery term is added in the loss function, which further accelerates convergence, thereby reducing the training cost. On the theoretical side, we prove that in the proposed method, the gradient descent algorithms are not attracted to sub-optimal critical points or local minima under practical conditions on the initialization and learning rate, and that the gradient dynamics of the proposed method is not achievable by base methods with any (adaptive) learning rates. We further show that the adaptive activation methods accelerate the convergence by implicitly multiplying conditioning matrices to the gradient of the base method without any explicit computation of the conditioning matrix and the matrix–vector product. The different adaptive activation functions are shown to induce different implicit conditioning matrices. Furthermore, the proposed methods with the slope recovery are shown to accelerate the training process.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Electronic structure and thermoelectric properties of CoAsSb with post-DFT approaches

Here, we study the electronic structure and thermoelectric properties of recently synthesized CoAsSb. The calculated bandgap becomes more accurate for increasingly-complex electronic structure methods: generalized gradient approximation, hybrid functionals, selfconsistent linearized quasiparticle GW method (LQSGW), and LQSGW with simplified vertex corrections. By equating the bandgaps of each method from a rigid shift of the bands, we evaluate the contributions made to thermoelectric properties beyond the bandgap. In doing so, we evaluate the efficacy of a commonpractice: a rigid shift applied to less-costly electronic structure methods in order to gain some accuracy of the more-costly methods. We find that while the shift made the Seebeck coefficients much closer to one another than from the original electronic structures, there remain differences in the Goldsmid-Sharp (thermoelectric) bandgap between the methods and from the intended electronic bandgap. Additionally, some lasting differences in temperature dependence remain between the methods.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

Validating methods for modeling composition gradients in planar shock experiments

An interface is Rayleigh–Taylor (RT) unstable when acceleration pushes a less dense material into a more dense one, and the growth of the instability is governed partly by the Atwood number gradient. Double-shell inertial confinement fusion capsules have a foam spacer layer pushing on an inner capsule composed of a beryllium tamper and high-Z inner shell, and so have RT unstable interfaces that require benchmarking. To this end, the results of a planar shock experiment with beryllium/tungsten targets are presented. One target had the normal bilayer construction of beryllium and tungsten in two distinct layers; the second target had the beryllium grading into tungsten with a quasi-exponential profile, motivated by the potential for reduced RT growth with the gradient profile. Simulations mimic the shock profiles for both targets and match the shock velocity to within 5%. These results validate the ability of our simulations to model double-shell capsules with bilayer or graded layer Be/W inner shells, which are needed to design future experiments at the National Ignition Facility.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗
NASA NTRS1993

A new approach to mixed H2/H infinity controller synthesis using gradient-based parameter optimization methods

In the past few years, the mixed H(sub 2)/H-infinity control problem has been the object of much research interest since it allows the incorporation of robust stability into the LQG framework. The general mixed H(sub 2)/H-infinity design problem has yet to be solved analytically. Numerous schemes have considered upper bounds for the H(sub 2)-performance criterion and/or imposed restrictive constraints on the class of systems under investigation. Furthermore, many modern control applications rely on dynamic models obtained from finite-element analysis and thus involve high-order plant models. Hence the capability to design low-order (fixed-order) controllers is of great importance. In this research a new design method was developed that optimizes the exact H(sub 2)-norm of a certain subsystem subject to robust stability in terms of H-infinity constraints and a minimal number of system assumptions. The derived algorithm is based on a differentiable scalar time-domain penalty function to represent the H-infinity constraints in the overall optimization. The scheme is capable of handling multiple plant conditions and hence multiple performance criteria and H-infinity constraints and incorporates additional constraints such as fixed-order and/or fixed structure controllers. The defined penalty function is applicable to any constraint that is expressible in form of a real symmetric matrix-inequity.

Ly, Uy-Loi↗
NASA NTRS1991

A biconjugate gradient type algorithm on massively parallel architectures

The biconjugate gradient (BCG) method is the natural generalization of the classical conjugate gradient algorithm for Hermitian positive definite matrices to general non-Hermitian linear systems. Unfortunately, the original BCG algorithm is susceptible to possible breakdowns and numerical instabilities. Recently, Freund and Nachtigal have proposed a novel BCG type approach, the quasi-minimal residual method (QMR), which overcomes the problems of BCG. Here, an implementation is presented of QMR based on an s-step version of the nonsymmetric look-ahead Lanczos algorithm. The main feature of the s-step Lanczos algorithm is that, in general, all inner products, except for one, can be computed in parallel at the end of each block; this is unlike the other standard Lanczos process where inner products are generated sequentially. The resulting implementation of QMR is particularly attractive on massively parallel SIMD architectures, such as the Connection Machine.

Freund, Roland W.↗
DOE OSTI2017

An improved hyperbolic embedding algorithm

Because hyperbolic space has properties that make it amenable to graph representations, there is significant interest in scalable hyperbolic-space embedding methods. These embeddings enable constant-time approximation of shortest-path distances, and so are significantly more efficient than full shortest-path computations. In this article, we improve on existing landmark-based hyperbolic embedding algorithms for large-scale graphs. Whereas previous methods compute the embedding by using the derivative-free Nelder–Mead simplex optimization method, our approach uses the limited-memory BFGS (LBFGS) method, which is quasi-Newton optimization, with analytic gradients. Our method is not only significantly faster but also produces higher-quality embeddings. Moreover, we are able to include the hyperbolic curvature as a variable in the optimization. We compare our hyperbolic embedding method implementation in Python (called Hypy) against the best publicly available software, Rigel. Our method is an order of magnitude faster and shows significant improvements in the accuracy of the shortest-path distance calculations. Furthermore, tests are performed on a variety of real-world networks, and we show the scalability of our method by embedding a graph with 1.8 billion edges and 65 million nodes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2011

Software realization of real-time neutrons and {gamma}-rays pulse shape discrimination using CUDA platform

The two pulse shape discrimination methods were implemented in real-time. The pulse gradient analysis method was implemented programmatically on PC. The method based on artificial neural network was programmatically implemented using CUDA platform. It is shown that both implementations can provide up to 10{sup 6} pulses per second processing performance. The results for pulse shape discrimination using polycrystalline stilbene and LiF detectors were shown. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1992

Air data system optimization using a genetic algorithm

An optimization method for flush-orifice air data system design has been developed using the Genetic Algorithm approach. The optimization of the orifice array minimizes the effect of normally distributed random noise in the pressure readings on the calculation of air data parameters, namely, angle of attack, sideslip angle and freestream dynamic pressure. The optimization method is applied to the design of Pressure Distribution/Air Data System experiment (PD/ADS) proposed for inclusion in the Aeroassist Flight Experiment (AFE). Results obtained by the Genetic Algorithm method are compared to the results obtained by conventional gradient search method.

Deshpande, Samir M.↗