Search NASA⌕ Search

SEARCH · Search NASA

Results for “Loss function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Multilabel proportion prediction and out-of-distribution detection on gamma spectra of short-lived fission products

In the machine learning problem of multilabel classification, the objective is to determine for each test instance which classes the instance belongs to. In this work, we consider an extension of multilabel classification, called multilabel proportion prediction, in the context of radioisotope identification (RIID) using gamma spectra data. We aim to not only predict radioisotope proportions, but also identify out-of-distribution (OOD) spectra. We achieve this goal by viewing gamma spectra as discrete probability distributions, and based on this perspective, we develop a custom semi-supervised loss function that combines a traditional supervised loss with an unsupervised reconstruction error function. Our approach was motivated by its application to the analysis of short-lived fission products from spent nuclear fuel. In particular, we demonstrate that a neural network model trained with our loss function can successfully predict the relative proportions of 37 radioisotopes simultaneously. The model trained with synthetic data was then applied to measurements taken by Pacific Northwest National Laboratory (PNNL) to conduct analysis typically done by subject-matter experts. Here, we also extend our approach to successfully identify when measurements are OOD, and thus should not be trusted, whether due to the presence of a novel source or novel proportions.

Anomaly detection↗

HomPINNs: Homotopy physics-informed neural networks for learning multiple solutions of nonlinear elliptic differential equations

Physics-informed neural networks (PINNs) based machine learning is an emerging framework for solving nonlinear differential equations. However, due to the implicit regularity of neural network structure, PINNs can only find the flattest solution in most cases by minimizing the loss functions. In this paper, we combine PINNs with the homotopy continuation method, a classical numerical method to compute isolated roots of polynomial systems, and propose a new deep learning framework, named homotopy physics-informed neural networks (HomPINNs), for solving multiple solutions of nonlinear elliptic differential equations. The implementation of an HomPINN is a homotopy process that is composed of the training of a fully connected neural network, named the starting neural network, and training processes of several PINNs with different tracking parameters. The starting neural network is to approximate a starting function constructed by the trivial solutions, while other PINNs are to minimize the loss functions defined by boundary condition and homotopy functions, varying with different tracking parameters. These training processes are regraded as different steps of a homotopy process, and a PINN is initialized by the well-trained neural network of the previous step, while the first starting neural network is initialized using the default initialization method. Finally, several numerical examples are presented to show the efficiency of our proposed HomPINNs, including reaction-diffusion equations with a heart-shaped domain.

97 MATHEMATICS AND COMPUTING↗

Z-Target Radiography Postprocessing With A Deep Convolution Neural Network

Analyzing X-ray radiographs is crucial for understanding target behavior in Inertial Confinement Fusion (ICF) and High Energy Density (HED) platforms. However, the density of Magneto Raleigh Taylor (MRT) bands and limitations of target materials often obscure relevant spike growth and density information. To address this issue, machine learning postprocessing techniques can be applied to remove darkened regions in radiography images. In this study, a novel method is presented for removing MRT darkened regions from z-target radiographs using a convolutional neural network (CNN). The CNN, consisting of six layers, treats the darkened regions as noise and employs a mixed loss function and end-to-end frameworks to suppress them while preserving sharpness. The six-layer architecture is designed to effectively learn features when provided with a larger volume of learning space. Each layer is optimized using a mixed loss function that combines a standard loss pixel approach with a multi-scaled structural similarity index loss, which considers luminance, contrast, and structure in local neighborhoods. This approach is particularly beneficial for capturing the stochastic structure of MRT limbs. Due to the limited availability of experimental data, training is conducted using synthetic target radiography from 3D Alegra simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Physics constrained learning for data-driven inverse modeling from sparse observations

Deep neural networks (DNN) have been used to model nonlinear relations between physical quantities. Those DNNs are embedded in physical systems described by partial differential equations (PDE) and trained by minimizing a loss function that measures the discrepancy between predictions and observations in some chosen norm. This loss function often includes the PDE constraints as a penalty term when only sparse observations are available. As a result, the PDE is only satisfied approximately by the solution. However, the penalty term typically slows down the convergence of the optimizer for stiff problems. We present a new approach that trains the embedded DNNs while numerically satisfying the PDE constraints. We develop an algorithm that enables differentiating both explicit and implicit numerical solvers in reverse-mode automatic differentiation. This allows the gradients of the DNNs and the PDE solvers to be computed in a unified framework. We demonstrate that our approach enjoys faster convergence and better stability in relatively stiff problems compared to the penalty method. Furthermore, our approach allows for the potential to solve and accelerate a wide range of data-driven inverse modeling, where the physical constraints are described by PDEs and need to be satisfied accurately.

97 MATHEMATICS AND COMPUTING↗

Enforcing Self-Consistent Kinematic Constraints in Neutrino Energy Estimators

Machine learning algorithms have long been utilized across many experimental collaborations within the neutrino physics community in applications to ascertain the singular kinematic quantity of initial neutrino energy for use in neutrino oscillation analyses. However, most of these algorithms do not incorporate a coherent physical picture of initial neutrino kinematics, opting to introduce loss functions involving knowledge of only |pν |. Here, we argue for the introduction of composite loss functions utilizing the full kinematic description of the neutrino, pν ≡ (E, px, py , pz ), compiling all relevant energy and angle information consistently. The use of such a fully defined variable can be seen as a usage of Physics Informed Machine Learning.

Richi, R. R.↗

Consequences of Lineage-Specific Gene Loss on Functional Evolution of Surviving Paralogs: ALDH1A and Retinoic Acid Signaling in Vertebrate Genomes

Genome duplications increase genetic diversity and may facilitate the evolution of gene subfunctions. Little attention, however, has focused on the evolutionary impact of lineage-specific gene loss. Here, we show that identifying lineagespecific gene loss after genome duplication is important for understanding the evolution of gene subfunctions in surviving paralogs and for improving functional connectivity among human and model organism genomes. We examine the general principles of gene loss following duplication, coupled with expression analysis of the retinaldehyde dehydrogenase Aldh1a gene family during retinoic acid signaling in eye development as a case study. Humans have three ALDH1A genes, but teleosts have just one or two. We used comparative genomics and conserved syntenies to identify loss of ohnologs (paralogs derived from genome duplication) and to clarify uncertain phylogenies. Analysis showed that Aldh1a1 and Aldh1a2 form a clade that is sister to Aldh1a3-related genes. Genome comparisons showed secondarily loss of aldh1a1 in teleosts, revealing that Aldh1a1 is not a tetrapod innovation and that aldh1a3 was recently lost in medaka, making it the first known vertebrate with a single aldh1a gene. Interestingly, results revealed asymmetric distribution of surviving ohnologs between co-orthologous teleost chromosome segments, suggesting that local genome architecture can influence ohnolog survival. We propose a model that reconstructs the chromosomal history of the Aldh1a family in the ancestral vertebrate genome, coupled with the evolution of gene functions in surviving Aldh1a ohnologs after R1, R2, and R3 genome duplications. Results provide evidence for early subfunctionalization and late subfunction-partitioning and suggest a mechanistic model based on altered regulation leading to heterochronic gene expression to explain the acquisition or modification of subfunctions by surviving ohnologs that preserve unaltered ancestral developmental programs in the face of gene loss.

59 BASIC BIOLOGICAL SCIENCES↗

ICDARTS: Improving the Stability of Cyclic DARTS

Cyclic DARTS (CDARTS) is a Differentiable Architecture Search (DARTS)-based approach to neural architecture search (NAS) that uses a cyclic feedback mechanism to train search and evaluation networks concurrently. This training protocol aims to optimize the search process and evaluate the deep evaluation network comprised of discretized candidate operations. However, this approach introduces a loss function for the evaluation network dependent on the search network. The dissimilarity between the evaluation network’s loss function used during the search and retraining phases results in a search network that is a sub-optimal proxy for the final evaluation network accessed during retraining. We present a revised approach that removes the dependency of the evaluation network weights upon those of the search network. In addition, we introduce a modified process for relaxing the search network’s zero operations that allows these operations to be retained in the final evaluation networks.

Herron, Emily↗

Multi-Label Classification with Constraint-Based Learning for Hierarchical Consistency

We explore the limitations of traditional crossentropy loss in a hierarchical multi-label classification setting and introduce a novel loss function. This function is designed to integrate hierarchical constraints directly into the training process. By incorporating such constraints into the loss, our approach slightly improves the logical consistency of predictions in structured domains. We demonstrate the efficacy of our approach through experiments on primary site and histology classification by using electronic pathology reports. These results show that our proposed hierarchical loss function enhances the model's ability to produce predictions that are logically consistent with the natural data hierarchies, and it slightly improves predictive accuracy. Our framework may be extended to other hierarchical domains, however the performance gains are context specific.

Shivanna, Abhishek [ORNL] (ORCID:0009000665228593)↗

Mars Terrain Segmentation with Less Labels

Planetary rover systems need to perform terrain segmentation to identify drivable areas as well as identify specific types of soil for sample collection. The latest Martian terrain segmentation methods rely on supervised learning which is very data hungry and difficult to train where only a small number of labeled samples are available. Moreover, the semantic classes are defined differently for different applications (e.g., rover traversal vs. geological) and as a result the network has to be trained from scratch each time, which is an inefficient use of resources. This research proposes a semi-supervised learning framework for Mars terrain segmentation where a deep segmentation network trained in an unsupervised manner on unlabeled images is transferred to the task of terrain segmentation trained on few labeled images. The network incorporates a backbone module which is trained using a contrastive loss function and an output atrous convolution module which is trained using a pixel-wise cross-entropy loss function. Evaluation results using the metric of segmentation accuracy show that the proposed method with contrastive pre-training outperforms plain supervised learning by 2%-10%. Moreover, the proposed model is able to achieve a segmentation accuracy of 91.1% using only 161 training images (1% of the original dataset) compared to 81.9% with plain supervised learning.

Wilson, Brian D↗

Fourier Neural Networks as Function Approximators and Differential Equation Solvers

We present a Fourier neural network (FNN) that can be mapped directly to the Fourier decomposition. The choice of activation and loss function yields results that replicate a Fourier series expansion closely while preserving a straightforward architecture with a single hidden layer. The simplicity of this network architecture facilitates the integration with any other higher-complexity networks, at a data pre- or postprocessing stage. We validate this FNN on naturally periodic smooth functions and on piecewise continuous periodic functions. We showcase the use of this FNN for modeling or solving partial differential equations with periodic boundary conditions. The main advantages of the current approach are the validity of the solution outside the training region, interpretability of the trained model, and simplicity of use.

Fourier decomposition↗

Field Work Proposal ERKJ358: Black-box training for scientific machine learning models (Final Report)

The overarching goal of this project is to develop a scalable black-box training capability for scientific machine learning (SciML) problems that are non-trainable with existing automatic differentiation (AD)-based algorithms. AD assumes that a loss function can be decomposed into a sequence of elementary operations whose derivatives are known. This assumption is violated when the loss function includes a black-box physical model (e.g., a legacy simulator). The current strategy, converting a black-box simulator to an AD-enabled code via differential programming, is inflexible and time-, labor-consuming. Thus, black-box optimization is a main workhorse for training SciML models, e.g., in scientific reinforcement learning, hyper-parameter fine tuning, designing SciML models with adversarial robustness, etc.

97 MATHEMATICS AND COMPUTING↗

Vibrational and rotational cooling of electrons by water vapor

The cooling of electrons by vibrational and rotational excitation of water molecules plays an important role in the thermal balance of electrons in cometary ionospheres. The energy-loss function for rotational excitation and deexcitation of H2O by electron impact is calculated theoretically. The rotational cooling rate is calculated using this loss function for a wide range of electron and neutral temperatures. The vibrational cooling rate is calculated using measured values of electron-impact vibrational excitation cross sections. Analytical formulas are provided for some of the cooling rates. The interaction of ions with H2O molecules is also discussed, and a formula is suggested for the momentum-transfer collision frequency.

Cravens, T. E.↗

Stacking AsFMT overexpression with BdPMT loss of function enhances monolignol ferulate production in Brachypodium distachyon

To what degree can the lignin subunits in a monocot be derived from monolignol ferulate (ML-FA) conjugates? This simple question comes with a complex set of variables. Three potential requirements for optimizing ML-FA production are as follows: (1) The presence of an active FERULOYL-CoA MONOLIGNOL TRANSFERASE (FMT) enzyme throughout monolignol production; (2) Suppression or elimination of enzymatic pathways competing for monolignols and intermediates during lignin biosynthesis; and (3) Exclusion of alternative phenolic compounds that participate in lignification. A 16-fold increase in lignin-bound ML-FA incorporation was observed by introducing an AsFMT gene into Brachypodium distachyon. On its own, knocking out the native p-COUMAROYL-CoA MONOLIGNOL TRANSFERASE (BdPMT) pathway that competes for monolignols and the p-coumaroyl-CoA intermediate did not change ML-FA incorporation, nor did partial loss of CINNAMOYL-CoA REDUCTASE1 (CCR1) function, which reduced metabolic flux to monolignols. However, stacking AsFMT into the Bdpmt-1 mutant resulted in a 32-fold increase in ML-FA incorporation into lignin over the wild-type level.

59 BASIC BIOLOGICAL SCIENCES↗

Dynamic calibration of differential equations using machine learning, with application to turbulence models

We present a methodology for calibration of parametric ordinary and partial differential equation models, using off-the-shelf software for back-propagation in Neural Networks (NN). As a prototypical example, we consider calibration of a Reynolds-averaged Navier-Stokes (RANS) turbulence closure model, against ground truth data from direct numerical simulations (DNS) of two different turbulent flows. Numerical time integration is represented as a custom NN, where only the RANS model parameters are trainable. A loss function is defined to quantify the mismatch between the NN prediction and the ground truth over a predefined, finite time integration window. This loss function is then minimized using a gradient descent method utilizing the back-propagation algorithm. Furthermore, this dynamic approach to training is to be contrasted with a static approach, wherein a least square regression estimate for parameters is obtained in the limit of an infinitesimal time integration window. In a first test of static and dynamic approaches against ground truth data generated by the model, the former proves to be significantly faster and more accurate than the latter at recovering the parameters. When both calibration approaches are tested against DNS data, for which it is known that the model cannot achieve a perfect fit, the static approach yields a good prediction only for short times, while the dynamic approach results in physical and stable predictions over the entire integration window. After optimization of the dynamic approach for time step, spatial resolution, stability, and physics-based constraints, we obtain a 50% improvement of outcomes over those obtained from the existing, manually calibrated set of parameters, demonstrating the merits of this systematic and automated procedure.

97 MATHEMATICS AND COMPUTING↗

First-principles studies on linear and nonlinear optical effects in Ln{sub 4}GaSbS{sub 9} (Ln=Ce-Nd, Sm, Gd-Tm, Lu)

Density functional theory (DFT) calculations have been performed on electronic structures of 11 middle-infrared (mid-IR) nonlinear optical quaternary sulfides: Ln{sub 4}GaSbS{sub 9} (Ln=Ce-Nd, Sm, Gd-Tm, Lu). Our results show that Ln{sub 4}GaSbS{sub 9} are indirect gap semiconductors with a slight band gap increase from Ce to Lu. Their linear optical properties, including refractive index, absorption coefficient and energy loss function, as well as the nonlinear optical coefficients including static d{sub 31}, d{sub 32}, d{sub 33} and dynamic d{sub 32}, are calculated. More importantly, the strong SHG response of Ln{sub 4}GaSbS{sub 9} can be attributed to the electronic transitions from S 3p states in valence bands (VB) to Sb-S and Ln-S antibonding states (CB). - Graphical Abstract: Density functional theory (DFT) calculations on Ln{sub 4}GaSbS{sub 9} reveal their linear optical properties, including refractive index, adsorption coefficient and energy loss function, the nonlinear optical coefficients including static d{sub 31}, d{sub 32}, d{sub 33} and dynamic d{sub 32}, as well as the origins of the strong SHG response. Highlights: Black-Right-Pointing-Pointer First-principles studies on a newly discovered NLO active family Ln{sub 4}GaSbS{sub 9}. Black-Right-Pointing-Pointer Reports static SHG coefficients (d{sub 31}, d{sub 32}, d{sub 33}) and dynamic d{sub 32}. Black-Right-Pointing-Pointer Reveals the origin of the strong SHG response.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Correspondence between neuroevolution and gradient descent

Abstract We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian white noise. Averaged over independent realizations of the learning process, neuroevolution is equivalent to gradient descent on the loss function. We use numerical simulation to show that this correspondence can be observed for finite mutations, for shallow and deep neural networks. Our results provide a connection between two families of neural-network training methods that are usually considered to be fundamentally different.

97 MATHEMATICS AND COMPUTING↗

INSURE: An Information Theory iNspired diSentanglement and pURification modEl for Domain Generalization

Domain Generalization (DG) aims to learn a generalizable model on the unseen target domain by only training on the multiple observed source domains. Although a variety of DG methods have focused on extracting domain-invariant features, the domain-specific class-relevant features have attracted attention and been argued to benefit generalization to the unseen target domain. To take into account the class-relevant domain-specific information, in this paper we propose an Information theory iNspired diSentanglement and pURification modEl (INSURE) to explicitly disentangle the latent features to obtain sufficient and compact (necessary) class-relevant feature for generalization to the unseen domain. Specifically, we first propose an information theory inspired loss function to ensure the disentangled class-relevant features contain sufficient class label information and the other disentangled auxiliary feature has sufficient domain information. Additionally, we further propose a paired purification loss function to let the auxiliary feature discard all the class-relevant information and thus the class-relevant feature will contain sufficient and compact (necessary) class-relevant information. Moreover, instead of using multiple encoders, we propose to use a learnable binary mask as our disentangler to make the disentanglement more efficient and make the disentangled features complementary to each other. We conduct extensive experiments on five widely used DG benchmark datasets including PACS, VLCS, OfficeHome, TerraIncognita, and DomainNet. The proposed INSURE achieves state-of-the-art performance. We also empirically show that domain-specific class-relevant features are beneficial for domain generalization. The code is available at https://github.com/yuxi120407/INSURE .

97 MATHEMATICS AND COMPUTING↗

A robust approach to Gaussian process implementation

Abstract. Gaussian process (GP) regression is a flexible modeling technique used to predict outputs and to capture uncertainty in the predictions. However, the GP regression process becomes computationally intensive when the training spatial dataset has a large number of observations. To address this challenge, we introduce a scalable GP algorithm, termed MuyGPs, which incorporates nearest-neighbor and leave-one-out cross-validation during training. This approach enables the evaluation of large spatial datasets with state-of-the-art accuracy and speed in certain spatial problems. Despite these advantages, conventional quadratic loss functions used in the MuyGPs optimization, such as root mean squared error (RMSE), are highly influenced by outliers. We explore the behavior of MuyGPs in cases involving outlying observations and, subsequently, develop a robust approach to handle and mitigate their impact. Specifically, we introduce a novel leave-one-out loss function based on the pseudo-Huber function (LOOPH) that effectively accounts for outliers in large spatial datasets within the MuyGPs framework. Our simulation study shows that the LOOPH loss method maintains accuracy despite outlying observations, establishing MuyGPs as a powerful tool for mitigating unusual observation impacts in the large data regime. In the analysis of US ozone data, MuyGPs provides accurate predictions and uncertainty quantification, demonstrating its utility in managing data anomalies. Through these efforts, we advance the understanding of GP regression in spatial contexts.

Mukangango, Juliette↗