Search NASA⌕ Search

SEARCH · Search NASA

Results for “DESCENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Gradient-Informed Design Optimization of Select Nuclear Systems

In this work, we present a gradient-informed design optimization of nuclear reactor core components based on neutronics objectives with both continuous and discrete materials. The main argument in favor of using gradient-informed design optimization is that it scales well with increasing dimensionality of the design space. First, a challenge problem with 121 free parameters is solved with a gradient-informed method and then with a genetic algorithm. Then, a challenge problem to optimize the flux profile of a simplified assembly with eight axial zones is solved. Both challenge problems are solved using directly calculated derivatives from Tools for Sensitivity and Uncertainty Analysis Methodology Implementation (TSUNAMI) in the SCALE package. Furthermore, we demonstrate how a discrete optimization problem—selection of materials for 121 voxels—can be lifted into a continuous problem with mixed materials. In the continuous space, adjoint-based gradients are well-defined, and gradient descent is applicable. Then, a forcing function is introduced that with the selection of an appropriately sized hyperparameter can be used to guide the optimized continuous solution back into a discrete solution. This paper presents an account of the challenges that were faced when applying a gradient-informed optimization algorithm using a Monte Carlo calculation to estimate the gradient information and compares a gradient descent optimization method to a genetic algorithm optimization of the same geometry. Overall, this work demonstrates the potential use of adjoint-based gradient calculations in design optimization of nuclear systems.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

A backward SDE method for uncertainty quantification in deep learning

Here, we develop a backward stochastic differential equation based probabilistic machine learning method, which formulates a class of stochastic neural networks as a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced with the gradient computed through a backward stochastic differential equation. Convergence analysis for stochastic gradient descent optimization and numerical experiments for applications of stochastic neural networks are carried out to validate our methodology in both theory and performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Optimality of Gradient-MUSIC for Spectral Estimation

We introduce the Gradient-MUSIC algorithm for estimating the unknown frequencies and amplitudes of a nonharmonic signal from noisy time samples. While the classical MUSIC algorithm performs a computationally expensive search over a fine grid, Gradient-MUSIC is significantly more efficient and eliminates the need for discretization over a fine grid by using optimization techniques. It coarsely scans the 1D landscape to find initialization simultaneously for all frequencies followed by parallelizable local refinement via gradient descent. We also analyze its performance when the noise level is sufficiently small and the signal frequencies are separated by at least 8π/m, where π/m is the standard resolution of this problem. Even though the 1D landscape is nonconvex, we prove a global convergence result for Gradient-MUSIC: coarse scanning provably finds suitable initialization and gradient descent converges at a linear rate. In addition to convergence results, we also upper bound the error between the true signal frequencies and amplitudes with those found by Gradient-MUSIC. For example, if the noise has $\ell^\infty$ norm at most ϵ, then the frequencies and amplitudes are recovered up to error at most Cϵ/m and Cϵ respectively, which are minimax optimal in m and ϵ. Our theory can also handle stochastic noise with performance guarantees under nonstationary independent Gaussian noise. Our main approach is a comprehensive geometric analysis of the landscape, a perspective that has not been explored before.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Iron precipitation under controlled oxygen flow: Mineralogical implications for BIF precursors in the Archean ocean

Banded iron formation (BIF) deposition in the Archean ocean is thought to have been initiated by the oxidation of dissolved iron into ferric primary precursor phases that descended through the water column and were deposited on the seafloor. Effective interpretation of the trace element and isotopic composition of BIFs in the geologic record requires an understanding of the identity and longevity of those precursor phases. Temporal and spatial variation in oxidant concentration and ocean chemistry may have driven precursor mineralogy. Furthermore, precursor phases may undergo transitions during descent and burial. This experimental study follows the evolution of iron mineralogy, speciation, and redox state during the precipitation of iron under variable oxygen fluxes and in fluids containing different iron-complexing anions (chloride, sulfate, and phosphate). Additionally, suspensions collected after incomplete oxidation were anoxically aged to simulate changes to precursor mineralogy during descent below the seawater redoxcline. Results from X-ray diffraction, sequential dissolution, colorimetric determination of Fe redox state, and Fe K-edge X-ray absorption fine structure spectroscopy showed that under all experimental conditions, intermediate precipitates collected before complete iron uptake contained mineralized ferrous iron. Purely ferric mineral assemblages were not observed until iron was removed from solution. In low-oxygen experiments, intermediate precipitates contained phases with stoichiometric Fe 2+ , including magnetite, vivianite, and green rust (GR). Chloride green rust (GR1) had more ferric iron than sulfate green rust (GR2) and more readily transitioned to magnetite via disproportionation. GR2 was stable over a broader range of iron uptake; magnetite appeared mainly as an additional oxidation product in sulfate solution. In high oxygen experiments, mineralized ferrous iron occurred in amorphous phases or as a non-stoichiometric component in ferric oxides; final ferric assemblages were less crystalline than in low-oxygen equivalents. Low concentrations of phosphate increased the total iron oxidation rate, but also increased the ferrous content of intermediate precipitates. The incorporation of phosphate stabilized GR2 and facilitated GR2 precipitation over a larger range of iron uptake. In higher concentrations of phosphate, vivianite and poorly crystalline ferric hydroxides formed in place of GR. Here, these findings suggest that over a range of possible ocean chemistries, mixed-valence phases may have been short lived but relevant precursors to BIF. Additionally, anionic chemistry and oxidant concentration are shown to influence the crystallinity and chemical resistivity of final ferric assemblages, affecting their reactivity during later anoxic burial.

58 GEOSCIENCES↗
DOE OSTI2022

A deterministic gradient-based approach to avoid saddle points

Abstract Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning (ML) models efficiently. First-order methods such as gradient descent (GD) are usually the methods of choice for training ML models. However, these methods converge to saddle points for certain choices of initial guesses. In this paper, we propose a modification of the recently proposed Laplacian smoothing gradient descent (LSGD) [Osher et al., arXiv:1806.06317 ], called modified LSGD (mLSGD), and demonstrate its potential to avoid saddle points without sacrificing the convergence rate. Our analysis is based on the attraction region, formed by all starting points for which the considered numerical scheme converges to a saddle point. We investigate the attraction region’s dimension both analytically and numerically. For a canonical class of quadratic functions, we show that the dimension of the attraction region for mLSGD is $\lfloor (n-1)/2\rfloor$ , and hence it is significantly smaller than that of GD whose dimension is $n-1$ .

Mathematics↗
DOE OSTI2024

Towards provably efficient quantum algorithms for large-scale machine-learning models

Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, we show that fault-tolerant quantum computing could possibly provide provably efficient resolutions for generic (stochastic) gradient descent algorithms, scaling as $\mathcal{O}$(T 2 x polylog($n$)), where n is the size of the models and T is the number of iterations in the training, as long as the models are both sufficiently dissipative and sparse, with small learning rates. Based on earlier efficient quantum algorithms for dissipative differential equations, we find and prove that similar algorithms work for (stochastic) gradient descent, the primary algorithm for machine learning. In practice, we benchmark instances of large machine learning models from 7 million to 103 million parameters. We find that, in the context of sparse training, a quantum enhancement is possible at the early stage of learning after model pruning, motivating a sparse parameter download and re-upload scheme. Our work shows solidly that fault-tolerant quantum algorithms could potentially contribute to most state-of-the-art, large-scale machine-learning problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Seismic constraints from a Mars impact experiment using InSight and Perseverance

NASA’s InSight (Interior Exploration using Seismic Investigations, Geodesy and Heat Transport) mission has operated a sophisticated suite of seismology and geophysics instruments on the surface of Mars since its arrival in 2018. On 18 February 2021, we attempted to detect the seismic and acoustic waves produced by the entry, descent and landing of the Perseverance rover using the sensors onboard the InSight lander. Similar observations have been made on Earth using data from both crewed and uncrewed spacecraft, and on the Moon during the Apollo era, but never before on Mars or another planet. This was the only seismic event to occur on Mars since InSight began operations that had an a priori known and independently constrained timing and location. It therefore had the potential to be used as a calibration for other marsquakes recorded by InSight. Here we report that no signal from Perseverance’s entry, descent and landing is identifiable in the InSight data. Nonetheless, measurements made during the landing window enable us to place constraints on the distance–amplitude relationships used to predict the amplitude of seismic waves produced by planetary impacts and place in situ constraints on Martian impact seismic efficiency (the fraction of the impactor kinetic energy converted into seismic energy).

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Structure optimization with stochastic density functional theory

Linear-scaling techniques for Kohn–Sham density functional theory are essential to describe the ground state properties of extended systems. Still, these techniques often rely on the localization of the density matrix or accurate embedding approaches, limiting their applicability. In contrast, stochastic density functional theory (sDFT) achieves linear- and sub-linear scaling by statistically sampling the ground state density without relying on embedding or imposing localization. In return, ground state observables, such as the forces on the nuclei, fluctuate in sDFT, making optimizing the nuclear structure a highly non-trivial problem. In this work, we combine the most recent noise-reduction schemes for sDFT with stochastic optimization algorithms to perform structure optimization within sDFT. We compare the performance of the stochastic gradient descent approach and its variations (stochastic gradient descent with momentum) with stochastic optimization techniques that rely on the Hessian, such as the stochastic Broyden–Fletcher–Goldfarb–Shanno algorithm. In conclusion, we further provide a detailed assessment of the computational efficiency and its dependence on the optimization parameters of each method for determining the ground state structure of bulk silicon with varying supercell dimensions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Resource frugal optimizer for quantum machine learning

Quantum-enhanced data science, also known as quantum machine learning (QML), is of growing interest as an application of near-term quantum computers. Variational QML algorithms have the potential to solve practical problems on real hardware, particularly when involving quantum data. However, training these algorithms can be challenging and calls for tailored optimization procedures. Specifically, QML applications can require a large shot-count overhead due to the large datasets involved. In this work, we advocate for simultaneous random sampling over both the dataset as well as the measurement operators that define the loss function. We consider a highly general loss function that encompasses many QML applications, and we show how to construct an unbiased estimator of its gradient. This allows us to propose a shot-frugal gradient descent optimizer called Refoqus (REsource Frugal Optimizer for QUantum Stochastic gradient descent). Our numerics indicate that Refoqus can save several orders of magnitude in shot cost, even relative to optimizers that sample over measurement operators alone.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Infinite neural network quantum states: entanglement and training dynamics

We study infinite limits of neural network quantum states (∞-NNQS), which exhibit representation power through ensemble statistics, and also tractable gradient descent dynamics. Ensemble averages of entanglement entropies are expressed in terms of neural network correlators, and architectures that exhibit volume-law entanglement are presented. The analytic calculations of entanglement entropy bound are tractable because the ensemble statistics are simplified in the Gaussian process limit. A general framework is developed for studying the gradient descent dynamics of neural network quantum states (NNQS), using a quantum state neural tangent kernel (QS-NTK). For ∞-NNQS the training dynamics is simplified, since the QS-NTK becomes deterministic and constant. An analytic solution is derived for quantum state supervised learning, which allows an ∞-NNQS to recover any target wavefunction. Numerical experiments on finite and infinite NNQS in the transverse field Ising model and Fermi Hubbard model demonstrate excellent agreement with theory. ∞-NNQS opens up new opportunities for studying entanglement and training dynamics in other physics applications, such as in finding ground states.

quantum state supervised learning↗
DOE OSTI2020

Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks

Here we propose two approaches of locally adaptive activation functions namely, layer-wise and neuron-wise locally adaptive activation functions, which improve the performance of deep and physics-informed neural networks. The local adaptation of activation function is achieved by introducing a scalable parameter in each layer (layer-wise) and for every neuron (neuron-wise) separately, and then optimizing it using a variant of stochastic gradient descent algorithm. In order to further increase the training speed, an activation slope-based slope recovery term is added in the loss function, which further accelerates convergence, thereby reducing the training cost. On the theoretical side, we prove that in the proposed method, the gradient descent algorithms are not attracted to sub-optimal critical points or local minima under practical conditions on the initialization and learning rate, and that the gradient dynamics of the proposed method is not achievable by base methods with any (adaptive) learning rates. We further show that the adaptive activation methods accelerate the convergence by implicitly multiplying conditioning matrices to the gradient of the base method without any explicit computation of the conditioning matrix and the matrix–vector product. The different adaptive activation functions are shown to induce different implicit conditioning matrices. Furthermore, the proposed methods with the slope recovery are shown to accelerate the training process.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Stochastic noise can be helpful for variational quantum algorithms

Saddle points constitute a crucial challenge for first-order gradient descent algorithms. In notions of classical machine learning, they are avoided, for example, by means of stochastic gradient descent methods. In this work, we provide evidence that the saddle-points problem can be naturally avoided in variational quantum algorithms by exploiting the presence of stochasticity. We prove convergence guarantees and present practical examples in numerical simulations and on quantum hardware. We argue that the natural stochasticity of variational algorithms can be beneficial for avoiding strict saddle points, i.e., those saddle points with at least one negative Hessian eigenvalue. This insight that some levels of shot noise could help is expected to add a new perspective to notions of near-term variational quantum algorithms. Published by the American Physical Society 2025

Liu, Junyu↗
DOE OSTI2021

Elastic distributed training with fast convergence and efficient resource utilization

Distributed learning is now routinely conducted on cloud as well as dedicated clusters. Training with elastic resources brings new challenges and design choices. Prior studies focus on runtime performance and assume a static algorithmic behavior. In this work, by analyzing the impact of of resource scaling on convergence, we introduce schedules for synchronous stochastic gradient descent that proactively adapt the number of learners to reduce training time and improve convergence. Our approach no longer assumes a constant number of processors throughout training. In our experiment, distributed stochastic gradient descent with dynamic schedules and reduction momentum achieves better convergence and significant speedups over prior static ones. Numerous distributed training jobs running on cloud may benefit from our approach.

Cong, Guojing↗
DOE OSTI2023

Distributed-Memory Parallel JointNMF

Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.

Eswar, Srinivas↗
DOE OSTI2020

Analytical Investigation of the Role of Lateral Mixing in the Evolution of Nonprecipitating Cumulus. Part II: Dissolving Stage

A minimalistic analytical model allowing analysis of the dissolving stage of nonprecipitating convective clouds is proposed. The model takes into account two mechanisms: turbulent mixing with a dry environment and cloud volume settling. The temporal changes in the spatial structure of a cloud and in its immediate environment in the course of cloud dissolving are analyzed. The comparison of the effects of a temperature increase in the course of cloud descent and mixing with dry surrounding air shows that the descent is a dominating factor determining a decrease in the liquid water content (LWC), while mixing has a stronger effect on the cloud shape. Narrowing/broadening of clouds due to lateral mixing with dry air during cloud dissolving is determined by the potential evaporation parameter proportional to the ratio of the saturation deficit in the cloud environment to LWC. An equation for cloud dissolving time is obtained. After a cloud totally dissolves, it leaves behind an area with humidity exceeding that of the environment. The main parameter determining the dissolving time is the downdraft velocity. It should exceed 50 cm s -1 in order to provide reasonable dissolving time. The turbulent intensity, LWC, and humidity of the environment air also have an impact on dissolving time: the lower the LWC and the humidity of environment air, the faster cloud dissolving is. The simple solution presented in this paper can be useful for evaluation of cloud characteristics at the dissolving stage and can be included in procedures of parameterization of cloud cover formed by nonprecipitating or slightly precipitating cumulus clouds (Cu). Values of the environment humidity and temperature, LWC at cloud top, cloud width, vertical velocity of downdraft, and the turbulent coefficient should be parameters of this parameterization.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

New Empirical Formulation for the Sublimational Breakup of Graupel and Dendritic Snow

Abstract Ice fragments are generated by sublimation of ice particles in subsaturated conditions in natural clouds. Conceivably, such sublimational breakup would be expected to cause ice multiplication in natural clouds. Any fragment that survives will grow to become ice precipitation that may sublimate and fragment further. As a first step toward assessing this overlooked process, a formulation is proposed for the number of ice fragments from sublimation of ice particles for an atmospheric model. This is done by amalgamating laboratory observations from previously published studies. The concept of a “sublimated mass activity spectrum” for the breakup is applied to the dataset. The number of ice fragments is determined by the relative humidity over ice and the initial size of the parent ice particles. The new formulation applies to dendritic crystals and heavily rimed particles only. Finally, a thought experiment is performed for an idealized scenario of subsaturation with in-cloud descent. Scaling analysis yields an estimate of an ice enhancement ratio of about 5 (10) within a weak deep convective downdraft of about 2 m s −1 , for an initial monodisperse population of dendritic snow (graupel) particles of 3 L −1 and 2 mm. During descent, there is a dynamic equilibrium between continual emission of fragments and their depletion by sublimation. A simplified bin microphysics parcel model exhibits this dynamical quasi equilibrium, consistent with the thought experiment. The fragments have average lifetimes of around 90 and 70 s for dendrites and graupel, respectively. Sublimational breakup is predicted to cause significant secondary ice production.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Survival of Snow in the Melting Layer: Relative Humidity Influence

This study quantifies how far snow can fall into the melting layer (ML) before all snow has melted by examining a combination of in-situ observations from aircraft measurements in Lagrangian spiral descents from above through the ML and descents and ascents into the ML, as well as an extensive database of NOAA surface observer reports during the past 50 years. The airborne data contain information on the particle phase (solid, mixed, or liquid), population size distributions and shapes, along with temperature, relative humidity, and vertical velocity. A wide range of temperatures and ambient relative humidities are used for both the airborne and ground-based data. It is shown that an ice bulb temperature of 0°C, together with the air temperature and pressure (altitude), are good first order predictors of the highest temperature snowflakes can survive in the melting layer before completely melting. Particle size is also important, as is whether the particles are graupel or hail. If the relative humidity is too low, the particles will sublimate completely as they fall into the melting layer. Snow as warm as +7°C is observed from aircraft measurements and surface observations. Snow pellets survive to even warmer temperatures. Relationships are developed to represent the primary findings.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

End-to-End Differentiable Modeling and Management of the Environment

Focal Area: (2) Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization. We emphasize the importance of leveraging optimization techniques from AI/machine learning (ML) to solve challenging problems in Earth system modeling. Science Challenge: Automatic differentiation has had a transformative effect on ML by allowing the calculation of gradients of arbitrary functions in an incredibly large class of models. We can potentially realize similar improvements in parameter estimation and control for Earth system models (ESMs) by reimplementing them in computational frameworks from ML. Practitioners working with large (>10 7 parameters) models in ML and AI can obtain good predictive performance in a range of spatiotemporal tasks by making use of optimization via stochastic gradient descent and incorporating prior knowledge at multiple levels. We propose writing ESMs in open-source computational frameworks such as Torch, Tensorflow, and JAX to greatly expand the scope of environmental forecasting and management challenges, which can be addressed by leveraging automatic differentiation and gradient descent-like algorithms. We do not call for a wholesale replacement of physical models with data-driven surrogates, but rather advocate for interleaving physical and empirical equations in a manner that is most faithful to the extent of our scientific knowledge and observational data. Central to this topic is the merging of differentiable physical simulations with differentiable optimization layers, which are now both beginning to come to the forefront.

54 ENVIRONMENTAL SCIENCES↗