Search NASA⌕ Search

SEARCH · Search NASA

Results for “gradient methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,603 records · Page 89

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗

Implicit Monte Carlo with a linear discontinuous finite element material solution and piecewise non-constant opacity

Here, the non-linear thermal radiative-transfer equations can be solved in various ways. One popular way is the Fleck and Cummings Implicit Monte Carlo (IMC) method. The IMC method was originally formulated with piecewise-constant material properties. For domains with a coarse spatial grid and large temperature gradients, an error known as numerical teleportation may cause artificially non-causal energy propagation and consequently an inaccurate material temperature. Source tilting is a technique to reduce teleportation error by constructing sub-spatial-cell (or sub-cell) emission profiles from which IMC particles are sampled. Several source tilting schemes exist, but some allow teleportation error to persist. We examine the effect of source tilting in problems with a temperature-dependent opacity. Within each cell, the opacity is evaluated continuously from a temperature profile implied by the source tilt. For IMC, this is a new approach to modeling the opacity. We find that applying both source tilting along with a source tilt-dependent opacity can introduce another dominant error that overly inhibits thermal wavefronts. We show that we can mitigate both teleportation and under-propagation errors if we discretize the temperature equation with a linear discontinuous (LD) trial space. Our method is for opacities ~ 1/T 3 , but we formulate and test a slight extension for opacities ~ 1/T 3.5 , where T is temperature. We find our method avoids errors that can be incurred by IMC with continuous source tilt constructions and piecewise-constant material temperature updates.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Height-dependent Velocity Structure of Photospheric Convection in Granules and Intergranular Lanes with Hinode /SOT

The solar photosphere is the visible surface of the Sun, where many bright granules, surrounded by narrow dark intergranular lanes, are observed everywhere. The granular pattern is a manifestation of convective motion at the photospheric level, but its velocity structure in the height direction is poorly understood observationally. Applying bisector analysis to a photospheric spectral line recorded by the Hinode Solar Optical Telescope, we derived the velocity structure of the convective motion in granular regions and intergranular lanes separately. The amplitude of motion of the convective material decreases from 0.65 to 0.40 km s{sup −1} as the material rises in granules, whereas the amplitude of motion increases from 0.30 to 0.50 km s{sup −1} as it descends in intergranular lanes. These values are significantly larger than those obtained in previous studies using bisector analysis. The acceleration of descending materials with depth is not predicted from the convectively stable condition in a stratified atmosphere. Such convective instability can be developed more efficiently by radiative cooling and/or a gas pressure gradient, which can control the dynamical behavior of convective material in intergranular lanes. Our analysis demonstrated that bisector analysis is a useful method for investigating the long-term dynamic behavior of convective material when a large number of pixels is available. In addition, one example is the temporal evolution of granular fragmentation, in which downflowing material develops gradually from a higher layer downward.

79 ASTRONOMY AND ASTROPHYSICS↗

Source Estimation by Full Wave Form Inversion

Given time-dependent ground motion recordings at a number of receiver stations, we solve the inverse problem for estimating the parameters of the seismic source. The source is modeled as a point moment tensor source, characterized by its location, moment tensor components, the start time, and frequency parameter (rise time) of its source time function. In total, there are 11 unknown parameters. We use a non-linear conjugate gradient algorithm to minimize the full waveform misfit between observed and computed ground motions at the receiver stations. An important underlying assumption of the minimization problem is that the wave propagation is accurately described by the elastic wave equation in a heterogeneous isotropic material. We use a fourth order accurate finite difference method, developed in [12], to evolve the waves forwards in time. The adjoint wave equation corresponding to the discretized elastic wave equation is used to compute the gradient of the misfit, which is needed by the non-linear conjugated minimization algorithm. A new source point moment source discretization is derived that guarantees that the Hessian of the misfit is a continuous function of the source location. An efficient approach for calculating the Hessian is also presented. We show how the Hessian can be used to scale the problem to improve the convergence of the non-linear conjugated gradient algorithm. Numerical experiments are presented for estimating the source parameters from synthetic data in a layer over half-space problem (LOH.1), illustrating rapid convergence of the proposed approach.

97 MATHEMATICS AND COMPUTING↗

Improved Prediction of Cold-Air Pools in the Weather Research and Forecasting Model Using a Truly Horizontal Diffusion Scheme for Potential Temperature

The terrain-following vertical coordinate system used by many atmospheric models, including the Weather Research and Forecasting (WRF) Model, is prone to errors in regions of complex terrain. These errors stem, in part, from the calculation of horizontal gradients within the diffusion term of the momentum or scalar evolution equations. In WRF, such gradients can be calculated along coordinate surfaces, or using metric terms that help account for grid skewness. However, neither of these options ensures a truly horizontal gradient calculation, especially if a grid cell is skewed enough that the heights of the neighboring grid points used in the calculation fall outside the vertical range of the cell. In this work, an improved scheme that uses Taylor series approximations to vertically interpolate variables to the level necessary for a truly horizontal gradient calculation is implemented in WRF for the diffusion of potential temperature. The scheme is validated using an atmosphere-at-rest configuration, in which spurious flows develop only as a result of numerical errors and can thus be used as a proxy for model performance. Following validation, the method is applied to the simulation of cold-air pools (CAPs), which occur in regions of complex terrain and are characterized by strong near-surface temperature gradients. Using the truly horizontal scheme, idealized simulations demonstrate reduced numerical mixing in a quiescent CAP, and a realistic case study in the Columbia River basin shows a reduction in positive wind speed bias by up to roughly 20% compared to observations from the Second Wind Forecast Improvement Project.

54 ENVIRONMENTAL SCIENCES↗

Numerical approximations for the molecular beam epitaxial growth model based on the invariant energy quadratization method

The Molecular Beam Epitaxial model is derived from the variation of a free energy, that consists of either a fourth order Ginzburg–Landau double well potential or a nonlinear logarithmic potential in terms of the gradient of a height function. One challenge in solving the MBE model numerically is how to develop proper temporal discretization for the nonlinear terms in order to preserve energy stability at the time-discrete level. In this paper, we resolve this issue by developing a first and second order time-stepping scheme based on the “Invariant Energy Quadratization” (IEQ) method. The novelty is that all nonlinear terms are treated semi-explicitly, and the resulted semi-discrete equations form a linear system at each time step. Moreover, the linear operator is symmetric positive definite and thus can be solved efficiently. We then prove that all proposed schemes are unconditionally energy stable. The semi-discrete schemes are further discretized in space using finite difference methods and implemented on GPUs for high-performance computing. Various 2D and 3D numerical examples are presented to demonstrate stability and accuracy of the proposed schemes.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

A method for evaluating bias in global measurements of CO 2 total columns from space

We describe a method of evaluating systematic errors in measurements of total column dry-air mole fractions of CO{sub 2} (X{sub CO{sub 2}} ) from space, and we illustrate the method by applying it to the Atmospheric CO{sub 2} Observations from Space retrievals of the Greenhouse Gases Observing Satellite (ACOS-GOSAT) v2.8. The approach exploits the lack of large gradients in X{sub CO{sub 2}} south of 25{degree} S to identify large-scale offsets and other biases in the ACOS-GOSAT data with several retrieval parameters and errors in instrument calibration. We demonstrate the effectiveness of the method by comparing the ACOS-GOSAT data in the Northern Hemisphere with ground truth provided by the Total Carbon Column Observing Network (TCCON). We use the correlation between free-tropospheric temperature and X{sub CO{sub 2}} in the Northern Hemisphere to define a dynamically informed coincidence criterion between the ground-based TCCON measurements and the ACOS-GOSAT measurements. We illustrate that this approach provides larger sample sizes, hence giving a more robust comparison than one that simply uses time, latitude and longitude criteria. Our results show that the agreement with the TCCON data improves after accounting for the systematic errors. A preliminary evaluation of the improved v2.9 ACOS-GOSAT data is also discussed.

54 ENVIRONMENTAL SCIENCES↗

Medial axis and local thickness computation using the Fast Sweeping Method

This report describes an efficient and robust voxel-based methodology for computing the medial axis, local thickness, and distance-to-skeleton of arbitrary three-dimensional geometries. It is assumed that the object can be represented by an exact or approximate signed distance function on a discrete grid. The gradient of such function is used to formulate a hyperbolic partial differential equation (PDE) that models the collapse of the position vector in space. By exploiting the causality property of the PDE, the Fast Sweeping Method is able to obtain the solution in a finite number of sweeps independent of the mesh resolution. The intersection of characteristic lines leads to the formation of shocks and a discrete bisector function is used to identify the medial axis. The same PDE approach is used to compute the local thickness inside the object and obtain the distance-to-skeleton field. Multiple examples are given in two and three dimensions along with a resolution study. The methodology has optimal complexity and yields subsecond computational times for geometries with over a million zones on a single core. The methodology is also capable of parallelization across shared and distributed memory architectures.

97 MATHEMATICS AND COMPUTING↗

Reactive particle-tracking solutions to a benchmark problem on heavy metal cycling in lake sediments

Geochemical systems are known to exhibit highly variable spatiotemporal behavior. This may be observed both in non-smooth concentration curves in space for a single sampling time and also in variability between samples taken from the same location at different times. However, most models that are designed to simulate these systems provide only single-solution smooth curves and fail to capture the noise and variability seen in the data. We apply a recently developed reactive particle-tracking method to a system that displays highly complex geochemical behavior. When the method is made to most closely resemble a corresponding Eulerian method, in its unperturbed form, we see near-exact match between solutions of the two models. More importantly, we consider two approaches for perturbing the model and find that the spatially-perturbed condition is able to capture a greater degree of the variability present in the data. This method of perturbation is a task to which particle methods are uniquely suited and Eulerian models are not well-suited. Additionally, because of the nature of the algorithm, noisy spatial gradients can be highly resolved by a large number of mobile particles, and this incurs negligible computational cost, as compared to expensive chemistry calculations.

54 ENVIRONMENTAL SCIENCES↗

An inexact regularized Newton framework with a worst-case iteration complexity of $ {\mathscr O}(\varepsilon^{-3/2}) $ for nonconvex optimization

Abstract An algorithm for solving smooth nonconvex optimization problems is proposed that, in the worst-case, takes ${\mathscr O}(\varepsilon ^{-3/2})$ iterations to drive the norm of the gradient of the objective function below a prescribed positive real number $\varepsilon $ and can take ${\mathscr O}(\varepsilon ^{-3})$ iterations to drive the leftmost eigenvalue of the Hessian of the objective above $-\varepsilon $. The proposed algorithm is a general framework that covers a wide range of techniques including quadratically and cubically regularized Newton methods, such as the Adaptive Regularization using Cubics (arc) method and the recently proposed Trust-Region Algorithm with Contractions and Expansions (trace). The generality of our method is achieved through the introduction of generic conditions that each trial step is required to satisfy, which in particular allows for inexact regularized Newton steps to be used. These conditions center around a new subproblem that can be approximately solved to obtain trial steps that satisfy the conditions. A new instance of the framework, distinct from arc and trace, is described that may be viewed as a hybrid between quadratically and cubically regularized Newton methods. Numerical results demonstrate that our hybrid algorithm outperforms a cubically regularized Newton method.

Mathematics↗

Linear and Nonlinear Solvers for Simulating Multiphase Flow within Large-Scale Engineered Subsurface Systems

Simulation of multiphase flow in the subsurface is well-known to be computationally challenging. While there have been many studies that have explored approaches to overcoming these challenges, they often utilize relatively simple case studies. In this paper, we focus on the unique numerical challenges posed by modeling large-scale engineered subsurface systems, characterized by discrete features embedded in a heterogeneous natural subsurface setting. The man-made features such as shafts, tunnels, and barriers often cause multiple challenges in modeling the domain for multiphase porous media flow. This flow scenario can have a wide range of applications such as nuclear waste repositories, enhanced recovery of a petroleum reservoir, geothermal engineering, and carbon sequestration. An example of these severe numerical challenges is the case of performance assessment (PA) for Waste Isolation Pilot Plant (WIPP), the only operating deep geological repository in the US, which simulates extreme material properties of bedded salt rock formation and extreme contrast due to open excavation next to the formation. The models have extremes not only of permeability and porosity but also of the constitutive models needed for multiphase flow; additionally, they have process models like salt creep closure reducing porosity over time, fracturing in clay and anhydrite interbeds of the bedded salt, gas generation from the waste materials, and unintentional human borehole intrusions in some scenarios. Numerical simulations require the solution of coupled systems of nonlinear PDEs; in our work, we use the open-source simulator PFLOTRAN which is based on Finite Volume discretization. The solution of the nonlinear equations requires use of the Newton-Raphson iteration at each time step, which entails the solution of the linearized Jacobian system at each iteration. The effects of all the processes (i.e., large number of unknowns, highly nonlinear constitutive relations, large contrasts in material properties in short distances) lead to an ill-conditioned Jacobian matrix that severely challenges traditional linear solver, i.e., stabilized biconjugate gradient with block Jacobi incomplete LU preconditioner (BCGS-ILU) leading to non-convergence for traditional Newton-Raphson nonlinear solver causing unacceptably long computation time for each model. This paper presents linear solvers such as constrained pressure residual (CPR) two-stage preconditioner with alternate-block-factorization (ABF) and quasi- implicit pressure and explicit saturation (QIMPES) decouplers and flexible generalized residual solver (FGMRES). The new general-purpose nonlinear solver, Newton trust-region dogleg Cauchy (NTRDC), is also introduced to resolve extreme nonlinearities in the models. We demonstrate the effectiveness of each method relative to the default BCGS-Newton solver. The two best cases had nearly 50 times speed-up and achieved completion of a simulation in 14 hours that never completed due to non-convergence with the default solver. We also investigate the strong scalability of each method and discuss some of the deficiencies found for Block Jacobi preconditioner using parallel domain decomposition, and node packing effects of modern processor architecture.

Preconditioner, Nonlinear, Porous media, Multiphas↗

Monte Carlo calculations for reference dosimetry of electron beams with the PTW Roos and NE2571 ion chambers

Purpose: To investigate recommendations for reference dosimetry of electron beams and gradient effects for the NE2571 chamber and to provide beam quality conversion factors using Monte Carlo simulations of the PTW Roos and NE2571 ion chambers. Methods: The EGSnrc code system is used to calculate the absorbed dose-to-water and the dose to the gas in fully modeled ion chambers as a function of depth in water. Electron beams are modeled using realistic accelerator simulations as well as beams modeled as collimated point sources from realistic electron beam spectra or monoenergetic electrons. Beam quality conversion factors are calculated with ratios of the doses to water and to the air in the ion chamber in electron beams and a cobalt-60 reference field. The overall ion chamber correction factor is studied using calculations of water-to-air stopping power ratios. Results: The use of an effective point of measurement shift of 1.55 mm from the front face of the PTW Roos chamber, which places the point of measurement inside the chamber cavity, minimizes the difference betweenR{sub 50}, the beam quality specifier, calculated from chamber simulations compared to that obtained using depth-dose calculations in water. A similar shift minimizes the variation of the overall ion chamber correction factor with depth to the practical range and reduces the root-mean-square deviation of a fit to calculated beam quality conversion factors at the reference depth as a function of R{sub 50}. Similarly, an upstream shift of 0.34 r{sub cav} allows a more accurate determination of R{sub 50} from NE2571 chamber calculations and reduces the variation of the overall ion chamber correction factor with depth. The determination of the gradient correction using a shift of 0.22 r{sub cav} optimizes the root-mean-square deviation of a fit to calculated beam quality conversion factors if all beams investigated are considered. However, if only clinical beams are considered, a good fit to results for beam quality conversion factors is obtained without explicitly correcting for gradient effects. The inadequacy of R{sub 50} to uniquely specify beam quality for the accurate selection of k{sub Q} factors is discussed. Systematic uncertainties in beam quality conversion factors are analyzed for the NE2571 chamber and amount to between 0.4% and 1.2% depending on assumptions used. Conclusions: The calculated beam quality conversion factors for the PTW Roos chamber obtained here are in good agreement with literature data. These results characterize the use of an NE2571 ion chamber for reference dosimetry of electron beams even in low-energy beams.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Sequential formulation of all‐way coupled finite strain thermoporomechanics for largely deformable gas hydrate deposits

We develop a numerically stable sequential formulation of thermoporomechanics for largely deformable gas hydrate deposits, extended from the fixed stress split of infinitesimal transformation. Constitutive equations are based on the total Lagrangian approach for both flow and geomechanics, including dynamic full tensor permeability and thermal conductivity updated from the deformation gradient. For space discretization, we take the cell-centered finite volume and node-based finite element method for flow and geomechanics, respectively. Then, we propose a sequential implicit method for all-way coupled thermoporomechanics, where the nonisothermal multiphase flow problem of gas hydrates is solved implicitly first and then the geomechanics problem is solved implicitly at the next step. During solution of the flow problem, we fix the rate of first Pioal total stress for numerical stability as well as apply porosity correction and entropy correction to account for geomechanical effects. We test numerical examples where flow and geomechanics parameters are based on deep oceanic gas hydrate deposits. When applying depressurization, even though the results between the infinitesimal transformation and finite strain geomechanics are similar in the early stages due to small deformation, we find differences between them in the late times as deformation becomes large. Accordingly, permeability and thermal conductivity tensors become nonisotropic full tensors although they are initially isotropic. Furthermore, we identify numerical stability of the developed sequential method from the test cases that exhibit the highly complex coupled gas hydrate systems with large deformation. Thus, the proposed sequential formulation can be applied in largely deformable gas hydrate systems.

42 ENGINEERING↗

Multi-task deep reinforcement learning for intelligent multi-zone residential HVAC control

In this short communication, a data-driven deep reinforcement learning (deep RL) method is applied to minimize HVAC users’ energy consumption costs while maintaining users’ comfort. The applied deep RL method's efficiency is enhanced by conducting multi-task learning that can achieve an economic control strategy for a multi-zone residential HVAC system in both cooling and heating scenarios. The applied multi-task deep RL method is compared with a rule-based benchmark case and a single-task deep deterministic policy gradient algorithm to verify its effective and generalized application in optimizing HVAC operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Tumor Size on Abdominal MRI Versus Pathologic Specimen in Resected Pancreatic Adenocarcinoma: Implications for Radiation Treatment Planning

Purpose: We assessed the accuracy of abdominal magnetic resonance imaging (MRI) for determining tumor size by comparing the preoperative contrast-enhanced T1-weighted gradient echo (3-dimensional [3D] volumetric interpolated breath-hold [VIBE]) MRI tumor size with pathologic specimen size. Methods and Materials: The records of 92 patients who had both preoperative contrast-enhanced 3D VIBE MRI images and detailed pathologic specimen measurements were available for review. Primary tumor size from the MRI was independently measured by a single diagnostic radiologist (P.M.) who was blinded to the pathology reports. Pathologic tumor measurements from gross specimens were obtained from the pathology reports. The maximum dimensions of tumor measured in any plane on the MRI and the gross specimen were compared. The median difference between the pathology sample and the MRI measurements was calculated. A paired t test was conducted to test for differences between the MRI and pathology measurements. The Pearson correlation coefficient was used to measure the association of disparity between the MRI and pathology sizes with the pathology size. Disparities relative to pathology size were also examined and tested for significance using a 1-sample t test. Results: The median patient age was 64.5 years. The primary site was pancreatic head in 81 patients, body in 4, and tail in 7. Three patients were American Joint Commission on Cancer stage IA, 7 stage IB, 21 stage IIA, 58 stage IIB, and 3 stage III. The 3D VIBE MRI underestimated tumor size by a median difference of 4 mm (range, −34-22 mm). The median largest tumor dimensions on MRI and pathology specimen were 2.65 cm (range, 1.5-9.5 cm) and 3.2 cm (range, 1.3-10 cm), respectively. Conclusions: Contrast-enhanced 3D VIBE MRI underestimates tumor size by 4 mm when compared with pathologic specimen. Advanced abdominal MRI sequences warrant further investigation for radiation therapy planning in pancreatic adenocarcinoma before routine integration into the treatment planning process.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Force-Free Identification of Minimum-Energy Pathways and Transition States for Stochastic Electronic Structure Theories

Here, the accurate mapping of potential energy surfaces (PESs) is crucial to our understanding of the numerous physical and chemical processes mediated by atomic rearrangements, such as conformational changes and chemical reactions, and the thermodynamic and kinetic feasibility of these processes. Stochastic electronic structure theories, e.g., Quantum Monte Carlo (QMC) methods, enable highly accurate total energy calculations that in principle can be used to construct the PES. However, their stochastic nature poses a challenge to the computation and use of forces and Hessians, which are typically required in algorithms for minimum-energy pathway (MEP) and transition state (TS) identification, such as the nudged elastic band (NEB) algorithm and its climbing image formulation. Here, we present strategies that utilize the surrogate Hessian line-search method, previously developed for QMC structural optimization, to efficiently identify MEP and TS structures without requiring force calculations at the level of the stochastic electronic structure theory. By modifying the surrogate Hessian algorithm to operate in path-orthogonal subspaces and at saddle points, we show that it is possible to identify MEPs and TSs by using a force-free QMC approach. We demonstrate these strategies via two examples, the inversion of the ammonia (NH 3 ) molecule and the nucleophilic substitution (S N 2) reaction F – + CH 3 F → FCH 3 + F – . We validate our results using Density Functional Theory (DFT)- and Coupled Cluster (CCSD, CCSD(T))-based NEB calculations. We then introduce a hybrid DFT-QMC approach to compute thermodynamic and kinetic quantities, free energy differences, rate constants, and equilibrium constants that incorporates stochastically optimized structures and their energies, and show that this scheme improves upon DFT accuracy. Our methods generalize straightforwardly to other systems and other high-accuracy theories that similarly face challenges computing energy gradients, paving the way for highly accurate PES mapping, transition state determination, and thermodynamic and kinetic calculations at significantly reduced computational expense.

Iyer, Gopal R.↗

Summary report of working group 3: High gradient and laser-structure based acceleration

The charge for the working group on high gradient and laser-structure based acceleration was to assess the current challenges involved in developing an advanced accelerator based on electromagnetic structures, and survey state-of-the-art methods to address those challenges. The topics of more than 50 presentations in the working group covered a very broad range of issues, from ideas, theoretical models and simulations, to design and manufacturing of accelerating structures and, finally, experimental results on obtaining extremely high accelerating gradients in structures from conventional microwave frequency range up to THz and laser frequencies. Workshop discussion topics included advances in the understanding of the physics of breakdown and other phenomena, limiting high gradient performance of accelerating structures. New results presented in this workshop demonstrated significant progress in the fields of conventional vacuum structure-based acceleration, dielectric wakefield acceleration, and laser-structure acceleration.

43 PARTICLE ACCELERATORS↗