Search NASA⌕ Search

SEARCH · Search NASA

Results for “Loss function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Essential barrier height and a probabilistic approach in characterizing potential landscape

In this work we propose a probabilistic approach to investigate the shape of landscapes of multi-dimensional potential functions. Under a suitable coupling scheme, two copies of the overdamped Langevin dynamics associated with the potential function are coupled, and the coupling times are collected. Assuming a set of intuitive yet technically challenging conditions on the coupling scheme, it is shown that the tail distributions of the coupling times exhibit qualitatively different dependencies on the noise magnitude for single-well versus multi-well potential functions. More specifically, for convex single-well potentials, the negative tail exponent of the coupling time distribution is uniformly bounded away from zero by the convexity parameter and is independent of the noise magnitude. In contrast, for multi-well potentials, the negative tail exponent decreases exponentially as the noise vanishes, with the decay rate governed by the essential barrier height, a quantity introduced in this paper to characterize the non-convex nature of the potential function. Numerical investigations are conducted for a variety of examples, including the Rosenbrock function, interacting particle systems, and loss functions arising in artificial neural networks. These examples not only illustrate the theoretical results in various contexts but also provide crucial numerical validation of the conjectured assumptions, which are essential to the theoretical analysis yet lie beyond the reach of standard technical tools.

97 MATHEMATICS AND COMPUTING↗

First principles treatment of structural, optical, and thermoelectric properties of Li{sub 7}MnN{sub 4} as electrode for a Li secondary battery

The electronic structure, electronic charge density and linear optical properties of the metallic Li{sub 7}MnN{sub 4} compound, having cubic symmetry, are calculated using the full potential linearized augmented plane wave (FP-LAPW) method. The calculated band structure and density of states using the local density, generalized gradient and Engel–Vosko approximations, depict the metallic nature of the cubic Li{sub 7}MnN{sub 4} compound. The bands crossing the Fermi level in the calculated band structure are mainly from the Mn-d states with small support of N-p states. In addition, the Mn-d states at the Fermi level enhance the density of states, which is very useful for the electronic transport properties. The valence electronic charge density depicts strong covalent bond between Mn and two N atoms and polar covalent bond between Mn and Li atoms. The frequency dependent linear optical properties like real and imaginary part of the dielectric function, optical conductivity, reflectivity and energy loss function are calculated on the basis of the computed band structure. Both intra-band and inter-band transitions contribute to the calculated optical parameters. Using the BoltzTraP code, the thermoelectric properties like electrical and thermal conductivity, Seebeck coefficient, power coefficient and heat capacity of the Li{sub 7}MnN{sub 4} are also calculated as a function of temperature and studied.

36 MATERIALS SCIENCE↗

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗

Enabling Long-range Exploration in Minimization of Multimodal Functions

We consider the problem of minimizing multi-modal loss functions with a large number of local optima. Since the local gradient points to the direction of the steepest slope in an infinitesimal neighborhood, an optimizer guided by the local gradient is often trapped in a local minimum. To address this issue, we develop a novel nonlocal gradient to skip small local minima by capturing major structures of the loss’s landscape in black-box optimization. The nonlocal gradient is defined by a directional Gaussian smoothing (DGS) approach. The key idea of DGS is to conducts 1D long-range exploration with a large smoothing radius along d orthogonal directions in Rd, each of which defines a nonlocal directional derivative as a 1D integral. Such long-range exploration enables the nonlocal gradient to skip small local minima. The d directional derivatives are then assembled to form the nonlocal gradient. We use the Gauss-Hermite quadrature rule to approximate the d 1D integrals to obtain an accurate estimator. The superior performance of our method is demonstrated in three sets of examples, including benchmark functions for global optimization, and two real-world scientific problems.

Zhang, Jiaxin↗

Electronic, magnetic, and optical properties of bulk and (1 1 1)-surfaces of CoMnZnSi quaternary Heusler alloy

The structural stability, electronic structure, and magnetic and optical properties of CoMnZnSi quaternary Heusler alloy (QHA) in the bulk and (1 1 1)-slab forms have been investigated by performing density functional theory (DFT) calculations. Formation and cohesive energies, and elastic constants confirm that the bulk CoMnZnSi is chemically and mechanically stable at an equilibrium lattice parameter of 5.81 Å. The calculated elastic constants also indicate that this QHA has ductile and anisotropic features. We investigate and discuss the bonding behavior from charge density distribution and density of states. Furthermore, these calculations in the bulk phase show a perfect half-metallic behavior with an integer value of magnetic moment (4 µB) and a large spin-flip gap of 0.39 eV. On the other hand, the Co (1 1 1) and Si (1 1 1)-slabs exhibit semi-metallic nature at the Fermi level. These results indicate that CoMnZnSi qualifies for spintronic applications. In the bulk and Co (1 1 1), Zn (1 1 1), and Si (1 1 1)-surfaces, the magnetic moment of Co and Mn atoms are ferromagnetically aligned, while the ferrimagnetic alignment of Co with Mn has been found in Mn (1 1 1)-surface. We also investigate the optimal conductivity, dielectric functions, reflectivity, absorption coefficient, refractive index, and loss function to understand the underlying optical properties of this alloy.

36 MATERIALS SCIENCE↗

Automated RF Phase Adjustment for Beam Stabilization in the Fermilab Linac

The Fermilab Linac experiences longitudinal beam phase drift, leading to increased particle loss, conventionally corrected through labor-intensive manual RF adjustments. This project explores machine learning-based automation for drift correction, employing a prototype-based classification approach. Our model utilizes a 34-dimensional feature set (RF settings and BPM readings) and leverages a 7x27 response matrix for system modeling. To overcome limited real-world data, we generate synthetic data, enhancing model training and generalizability. Custom loss functions, including a surrogate energy-consistent loss and a temporal smoothness constraint, ensure physically plausible drift predictions. The goal is a robust system for autonomous phase adjustments, ensuring stable beam acceleration and reduced manual intervention.

Chichili, R. R. [Illinois U., Chicago]↗

Effect of Radiation on Chromospheric Magnetic Reconnection: Reactive and Collisional Multi-fluid Simulations

We study magnetic reconnection under chromospheric conditions in five different ionization levels from 0.5% to 50% using a self-consistent two-fluid (ions + neutrals) model that accounts for compressibility, collisional effects, chemical inequilibrium, and anisotropic heat conduction. Results with and without radiation are compared, using two models for the radiative losses: an optically thin radiation loss function, and an approximation of the radiative losses of a plasma with photospheric abundances. The results without radiation show that reconnection occurs faster for the weakly ionized cases as a result of the effect of ambipolar diffusion and fast recombination. The tearing mode instability appears earlier in the low ionized cases and grows rapidly. We find that radiative losses have a stronger effect than was found in previous results as the cooling changes the plasma pressure and the concentration of ions inside the current sheet. This affects the ambipolar diffusion and the chemical equilibrium, resulting in thin current sheets and enhanced reconnection. The results quantify this complex nonlinear interaction by showing that a strong cooling produces faster reconnections than have been found in models without radiation. The results accounting for radiation show timescales and outflows comparable to spicules and chromospheric jets.

79 ASTRONOMY AND ASTROPHYSICS↗

Multidimensional Distributional Neural Network Output Demonstrated in Super‐Resolution of Surface Wind Speed

Accurate quantification of uncertainty in neural network predictions remains a central challenge for scientific applications involving high-dimensional, correlated data. While existing methods capture either aleatoric or epistemic uncertainty, few offer closed-form, multidimensional distributions that preserve spatial correlation while remaining computationally tractable. In this work, we present a framework for training neural networks with a multidimensional Gaussian loss, generating a closed-form predictive distribution over outputs informed by non-identically distributed training data. Our approach captures aleatoric uncertainty by iteratively estimating the means and covariance matrices, and is demonstrated on a super-resolution example out-of-training-sample. We leverage a Fourier representation of the covariance matrix to stabilize network training and preserve spatial correlation. We introduce a novel regularization strategy—referred to as information sharing—that interpolates between image-specific and global covariance estimates, enabling convergence of the super-resolution downscaling network trained on image-specific distributional loss functions. This framework allows for efficient sampling, explicit correlation modeling, and extensions to more complex distribution families all without disrupting prediction performance. We demonstrate the method on a surface wind speed downscaling task and discuss its broader applicability to uncertainty-aware prediction in scientific models.

17 WIND ENERGY↗

Paw-Net: Stacking ensemble deep learning for segmenting scanning electron microscopy images of fine-grained shale samples

Segmentation of scanning electron microscopy (SEM) images is critical yet time-consuming for geological analyses, as it needs to differentiate the boundaries for different mineral objects to facilitate subsequent analyses, such as porosity calculation. Recently, various machine learning methods, especially convolutional neural networks (CNNs), have been explored to segment SEM images of fine-grained shale samples. However, we found that general CNNs do not yield optimal performance due to insufficient training data and imbalanced objects in SEM images. This work has revised the U-Net architecture, a popular approach for biomedical image analyses, by incorporating a loss function that addresses the imbalance issue. Furthermore, we used the ensemble learning method to train multiple models and combined the results to improve the overall performance of segmentation. We prepared 2162 sub-images from raw SEM images in our experiments and divided them into training, validation, and testing datasets. The overall results show that our method improves the average Intersection over Union (IOU) of mineral objects from 0.49 to 0.58, compared to the original U-Net model. Our method can clearly distinguish each object from others with boundaries, even in highly imbalanced images. Training our models takes less than three minutes using a single GPU, while manual labeling can take up to three hours for each image. Furthermore, the method helps geoscientists gain insights quickly and effectively by building neural network models from a small dataset of SEM images.

58 GEOSCIENCES↗

Deep learning symmetries and their Lie groups, algebras, and subalgebras from first principles

Abstract We design a deep-learning algorithm for the discovery and identification of the continuous group of symmetries present in a labeled dataset. We use fully connected neural networks to model the symmetry transformations and the corresponding generators. The constructed loss functions ensure that the applied transformations are symmetries and the corresponding set of generators forms a closed (sub)algebra. Our procedure is validated with several examples illustrating different types of conserved quantities preserved by symmetry. In the process of deriving the full set of symmetries, we analyze the complete subgroup structure of the rotation groups SO (2), SO (3), and SO (4), and of the Lorentz group S O ( 1 , 3 ) . Other examples include squeeze mapping, piecewise discontinuous labels, and SO (10), demonstrating that our method is completely general, with many possible applications in physics and data science. Our study also opens the door for using a machine learning approach in the mathematical study of Lie groups and their properties.

97 MATHEMATICS AND COMPUTING↗

Automated RF Phase Adjustment for Beam Stabilization in the Fermilab Linac

The Fermilab Linac experiences longitudinal beam phase drift, leading to increased particle loss, conventionally cor- rected through labor-intensive manual RF adjustments. This project explores machine learning-based automation for drift correction, employing a prototype-based classification approach. Our model utilizes a 34-dimensional feature set (RF settings and BPM readings) and leverages a 7x27 response matrix for system modeling. To overcome limited real-world data, we generate synthetic data, enhancing model training and generalizability. Custom loss functions, including a sur- rogate energy-consistent loss and a temporal smoothness constraint, ensure physically plausible drift predictions. The goal is a robust system for autonomous phase adjustments, ensuring stable beam acceleration and reduced manual intervention.

Chichili, R. R. [U. Illinois, Chicago]↗

First-Principles Study of Pressure Dependence of Optical Spectra of MnS

MnS is a semiconductor material that can be used in solar cell coatings as window/buffer material and in many other applications. The present contribution deals with the optical properties of MnS in the zinc-blende structure using ab initio calculations within the generalized gradient approximation. Features such as dielectric function, refractive index, reflectivity, optical absorption, conductivity, and electron loss function spectra have been reported and their pressure dependence has been examined and discussed. The information gathered from the present study can be useful for photovoltaic applications.

14 SOLAR ENERGY↗

Anomaly Detection in Liquid Sodium Cold Trap Operation with Multisensory Data Fusion Using Long Short-Term Memory Autoencoder

Sodium-cooled fast reactors (SFR), which use high temperature fluid near ambient pressure as coolant, are one of the most promising types of GEN IV reactors. One of the unique challenges of SFR operation is purification of high temperature liquid sodium with a cold trap to prevent corrosion and obstructing small orifices. We have developed a deep learning long short-term memory (LSTM) autoencoder for continuous monitoring of a cold trap and detection of operational anomaly. Transient data were obtained from the Mechanisms Engineering Test Loop (METL) liquid sodium facility at Argonne National Laboratory. The cold trap purification at METL is monitored with 31 variables, which are sensors measuring fluid temperatures, pressures and flow rates, and controller signals. Loss-of-coolant type anomaly in the cold trap operation was generated by temporarily choking one of the blowers, which resulted in temperature and flow rate spikes. The input layer of the autoencoder consisted of all the variables involved in monitoring the cold trap. The LSTM autoencoder was trained on the data corresponding to cold trap startup and normal operation regime, with the loss function calculated as the mean absolute error (MAE). The loss during training was determined to follow log-normal density distribution. During monitoring, we investigated a performance of the LSTM autoencoder for different loss threshold values, set at a progressively increasing number of standard deviations from the mean. The anomaly signal in the data was gradually attenuated, while preserving the noise of the original time series, so that the signal-to-noise ratio (SNR) averaged across all sensors decreased below unity. Results demonstrate detection of anomalies with sensor-averaged SNR < 1.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Endothelial sirtuin 1 inactivation enhances capillary rarefaction and fibrosis following kidney injury through Notch activation

Peritubular capillary (PTC) rarefaction along with tissue fibrosis is a hallmark of chronic kidney disease (CKD). However, molecular mechanisms of PTC loss have been poorly understood. Previous studies have demonstrated that functional loss of endothelial sirtuin 1 (SIRT1) impairs angiogenesis during development and tissue damage. Here, we found that endothelial SIRT1 dysfunction causes activation of endothelial Notch1 signaling, which leads to PTC rarefaction and fibrosis following kidney injury. In mice lacking functional SIRT1 in the endothelium (Sirt1 mutant), kidney injury enhanced apoptosis and senescence of PTC endothelial cells with impaired endothelial proliferation and expanded myofibroblast population and collagen deposition. Compared to wild-type kidneys, Sirt1 mutant kidneys up-regulated expression of Delta-like 4 (DLL4, a potent Notch1 ligand), Hey1 and Hes1 (Notch target genes), and Notch intracellular domain-1 (NICD1, active form of Notch1) in microvascular endothelial cells (MVECs) post-injury. Sirt1 mutant primary kidney MVECs reduced motility and vascular assembly and enhanced senescence compared to wild-type kidney MVECs. This difference in the phenotype was negated with Notch inhibition. Concurrent stimulation of DLL4 and transforming growth factor (TGF)-β1 increased trans-differentiation of primary kidney pericytes into myofibroblast more than TGF-β1 treatment alone. Collectively, these results indicate that endothelial SIRT1 counteracts PTC rarefaction by repression of Notch1 signaling and antagonizes fibrosis via suppression of endothelial DLL4 expression. - Highlights: • SIRT1 represses Notch1 signaling in capillary endothelial cells in the kidney. • Endothelial SIRT1 is depleted in the kidney following injury. • Activation of endothelial Notch impairs angiogenesis in the kidney. • Increased expression of endothelial DLL4 enhances renal fibrosis.

60 APPLIED LIFE SCIENCES↗

Exploiting the Local Parabolic Landscapes of Adversarial Losses to Accelerate Black-Box Adversarial Attack

Existing black-box adversarial attacks on image classifiers update the perturbation at each iteration from only a small number of queries of the loss function. Since the queries contain very limited information about the loss, black-box methods usually require much more queries than white-box methods. We propose to improve the query efficiency of black-box methods by exploiting the smoothness of the local loss landscape. However, many adversarial losses are not locally smooth with respect to pixel perturbations. To resolve this issue, our first contribution is to theoretically and experimentally justify that the adversarial losses of many standard and robust image classifiers behave like parabolas with respect to perturbations in the Fourier domain. Our second contribution is to exploit the parabolic landscape to build a quadratic approximation of the loss around the current state, and use this approximation to interpolate the loss value as well as update the perturbation without additional queries. Since the local region is already informed by the quadratic fitting, we use large perturbation steps to explore far areas. We demonstrate the efficiency of our method on MNIST, CIFAR-10 and ImageNet datasets for various standard and robust models, as well as on Google Cloud Vision. The experimental results show that exploiting the loss landscape can help significantly reduce the number of queries and increase the success rate. Our codes are available at https://github.com/HoangATran/BABIES.

Tran, Hoang↗

A reverse Monte Carlo method for deriving optical constants of solids from reflection electron energy-loss spectroscopy spectra

A reverse Monte Carlo (RMC) method is developed to obtain the energy loss function (ELF) and optical constants from a measured reflection electron energy-loss spectroscopy (REELS) spectrum by an iterative Monte Carlo (MC) simulation procedure. The method combines the simulated annealing method, i.e., a Markov chain Monte Carlo (MCMC) sampling of oscillator parameters, surface and bulk excitation weighting factors, and band gap energy, with a conventional MC simulation of electron interaction with solids, which acts as a single step of MCMC sampling in this RMC method. To examine the reliability of this method, we have verified that the output data of the dielectric function are essentially independent of the initial values of the trial parameters, which is a basic property of a MCMC method. The optical constants derived for SiO{sub 2} in the energy loss range of 8-90 eV are in good agreement with other available data, and relevant bulk ELFs are checked by oscillator strength-sum and perfect-screening-sum rules. Our results show that the dielectric function can be obtained by the RMC method even with a wide range of initial trial parameters. The RMC method is thus a general and effective method for determining the optical properties of solids from REELS measurements.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Data optimization for large batch distributed training of deep neural networks

Distributed training in deep learning (DL) is common practice as data and models grow. The current practice for distributed training of deep neural networks faces the challenges of communication bottlenecks when operating at scale, and model accuracy deterioration with an increase in global batch size. Present solutions focus on improving message exchange efficiency as well as implementing techniques to tweak batch sizes and models in the training process. The loss of training accuracy typically happens because the loss function gets trapped in a local minima. We observe that the loss landscape minimization is shaped by both the model and training data and propose a data optimization approach that utilizes machine learning to implicitly smooth out the loss landscape resulting in fewer local minima. Our approach filters out data points which are less important to feature learning, enabling us to speed up the training of models on larger batch sizes to improved accuracy.

Gahlot, Shubhankar↗

On the Stability Analysis of Astrophysical Cooling Functions

To model the temperature evolution of optically thin astrophysical environments at MHD scales, radiative and collisional cooling rates are typically either pretabulated or fit into a functional form and then input into MHD codes as a radiative loss function. Thermal balance requires estimates of the analogous heating rates, which are harder to calculate, and due to uncertainties in the underlying dissipative heating processes these rates are often simply parameterized. The resulting net cooling function defines an equilibrium curve that varies with density and temperature. Such cooling functions can make the gas prone to thermal instability (TI), which will cause departures from equilibrium. There has been no systematic study of thermally unstable parameter space for nonequilibrium states. Motivated by our recent finding that there is a related linear instability, catastrophic cooling instability, that can dominate over TI, here we carry out such a study. We show that Balbus instability criteria for TI can be used to define a critical cooling rate, Λ c , that permits a nonequilibrium analysis of cooling functions through the mapping of TI zones. We furthermore illustrate how thermal conduction modifies the shape of TI zones. Upon applying a Λ c -based stability analysis to coronal loop simulations, we find that loops undergoing periodic episodes of coronal rain formation are linearly unstable to catastrophic cooling instability, while TI is stabilized by thermal conduction.

79 ASTRONOMY AND ASTROPHYSICS↗