Search NASA⌕ Search

SEARCH · Search NASA

Results for “Loss function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

LossLens: Diagnostics for Machine Learning Through Loss Landscape Visual Analytics

Modern machine learning often relies on optimizing a neural network's parameters using a loss function to learn complex features. Beyond training, examining the loss function with respect to a network's parameters (i.e., as a loss landscape) can reveal insights into the architecture and learning process. While the local structure of the loss landscape surrounding an individual solution can be characterized using a variety of approaches, the global structure of a loss landscape, which includes potentially many local minima corresponding to different solutions, remains far more difficult to conceptualize and visualize. To address this difficulty, we introduce LossLens, a visual analytics framework that explores loss landscapes at multiple scales. LossLens integrates metrics from global and local scales into a comprehensive visual representation, enhancing model diagnostics. Here we demonstrate LossLens through two case studies: visualizing how residual connections influence a ResNet-20, and visualizing how physical parameters influence a physics-informed neural network (PINN) solving a simple convection problem.

97 MATHEMATICS AND COMPUTING↗

Star formation through thermal instability of radiative plasma with finite electron inertia and finite Larmor radius corrections

I have studied the effects of finite electron inertia, finite ion Larmor radius (FLR) corrections, and radiative heat-loss function on the thermal instability of an infinite homogeneous, viscous plasma incorporating the effect of thermal conductivity for star formation in interstellar medium (ISM). A general dispersion relation is derived using the normal mode analysis method with the help of relevant linearized perturbation equations of the problem. The wave propagation is discussed for longitudinal and transverse directions to the external magnetic field and the conditions of modified thermal instabilities and stabilities are discussed in different cases. We find that the thermal instability criterion is get modified into radiative instability criterion by inclusion of radiative heat-loss functions with thermal conductivity. The viscosity of medium removes the effect of FLR corrections from the condition of radiative instability. Numerical calculation shows stabilizing effect of heat-loss function, viscosity and FLR corrections, and destabilizing effect of finite electron inertia on the thermal instability. Results carried out in this paper shows that stars are formed in interstellar medium mainly due to thermal instability.

79 ASTRONOMY AND ASTROPHYSICS↗

PPPC 4 DM secondary: a Poor Particle Physicist Cookbook for secondary radiation from Dark Matter

We enlarge the set of recipes and ingredients at disposal of any poor particle physicist eager to cook up signatures from weak-scale Dark Matter models by computing two secondary emissions due to DM particles annihilating or decaying in the galactic halo, namely the radio signals from synchrotron emission and the gamma rays from bremsstrahlung. We consider several magnetic field configurations and propagation scenarios for electrons and positrons. We also provide an improved energy loss function for electrons and positrons in the Galaxy, including synchrotron losses in the different configurations, bremsstrahlung losses, ionization losses and Inverse Compton losses with an updated InterStellar Radiation Field.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Generalized Canonical Polyadic Tensor Decomposition

Tensor decomposition is a fundamental unsupervised machine learning method in data science, with applications including network analysis and sensor data processing. This work develops a generalized canonical polyadic (GCP) low-rank tensor decomposition that allows other loss functions besides squared error. For instance, we can use logistic loss or Kullback--Leibler divergence, enabling tensor decomposition for binary or count data. We present a variety of statistically motivated loss functions for various scenarios. We provide a generalized framework for computing gradients and handling missing data that enables the use of standard optimization methods for fitting the model. Furthermore, we demonstrate the flexibility of the GCP decomposition on several real-world examples including interactions in a social network, neural activity in a mouse, and monthly rainfall measurements in India.

97 MATHEMATICS AND COMPUTING↗

Error minimizing algorithms for nearest eighbor classifiers

Stack Filters define a large class of discrete nonlinear filter first introd uced in image and signal processing for noise removal. In recent years we have suggested their application to classification problems, and investigated their relationship to other types of discrete classifiers such as Decision Trees. In this paper we focus on a continuous domain version of Stack Filter Classifiers which we call Ordered Hypothesis Machines (OHM), and investigate their relationship to Nearest Neighbor classifiers. We show that OHM classifiers provide a novel framework in which to train Nearest Neighbor type classifiers by minimizing empirical error based loss functions. We use the framework to investigate a new cost sensitive loss function that allows us to train a Nearest Neighbor type classifier for low false alarm rate applications. We report results on both synthetic data and real-world image data.

97 MATHEMATICS AND COMPUTING↗

RNA-seq latent featurizer using center loss cost function

Predictive modeling of patient tumor growth response to drug treatment is severely limited by a lack of experimental data for training. Combining experimental data from several studies is an attractive approach, but presents two problems: batch effects and limited data for individual drugs and cell lines. Batch effects are caused by systematic procedural differences among studies, which causes systematic experimental outcome differences. Directly using these experimental results as features for machine learning commonly causes problems when training on one study and testing on another. This severely limits a model's ability to perform well on new experiments. Even after combining studies, predicting outcomes on new patient tumors remains an open challenge. We propose a semi-supervised, autoencoder-based, machine learning procedure, which learns a smaller set of gene expression features that are robust to batch effects using background information on a cell line or tissue's tumor type. We implemented this reduced feature representation and show that the new feature space clusters strongly according to tumor type. This experiment is carried out across multiple studies: CCLE, CTRP, gCSI, GDSC, NCI60, and patient derived tumors. We hypothesize that using a batch effect resistant feature set across studies will improve prediction performance. Genome Data Commons (GDC) gene expression profiles for publicly available human tissues and cell lines from NCI60 and CCLE were processed using the semi-supervised learning procedure. Our autoencoder repurposes the 'center loss' (CL) cost function of Wen et. al. to learn a more generalized set of features using the cell line or tissue's tumor type.

He, Stewart↗

Addressing bias in bagging and boosting regression models

As artificial intelligence (AI) becomes widespread, there is increasing attention on investigating bias in machine learning (ML) models. Previous research concentrated on classification problems, with little emphasis on regression models. This paper presents an easy-to-apply and effective methodology for mitigating bias in bagging and boosting regression models, that is also applicable to any model trained through minimizing a differentiable loss function. Our methodology measures bias rigorously and extends the ML model's loss function with a regularization term to penalize high correlations between model errors and protected attributes. We applied our approach to three popular tree-based ensemble models: a random forest model (RF), a gradient-boosted model (GBT), and an extreme gradient boosting model (XGBoost). We implemented our methodology on a case study for predicting road-level traffic volume, where RF, GBT, and XGBoost models were shown to have high accuracy. Despite high accuracy, the ML models were shown to perform poorly on roads in minority-populated areas. Our bias mitigation approach reduced minority-related bias by over 50%.

97 MATHEMATICS AND COMPUTING↗

Multilabel proportion prediction and out-of-distribution detection on gamma spectra of short-lived fission products

In the machine learning problem of multilabel classification, the objective is to determine for each test instance which classes the instance belongs to. In this work, we consider an extension of multilabel classification, called multilabel proportion prediction, in the context of radioisotope identification (RIID) using gamma spectra data. We aim to not only predict radioisotope proportions, but also identify out-of-distribution (OOD) spectra. We achieve this goal by viewing gamma spectra as discrete probability distributions, and based on this perspective, we develop a custom semi-supervised loss function that combines a traditional supervised loss with an unsupervised reconstruction error function. Our approach was motivated by its application to the analysis of short-lived fission products from spent nuclear fuel. In particular, we demonstrate that a neural network model trained with our loss function can successfully predict the relative proportions of 37 radioisotopes simultaneously. The model trained with synthetic data was then applied to measurements taken by Pacific Northwest National Laboratory (PNNL) to conduct analysis typically done by subject-matter experts. Here, we also extend our approach to successfully identify when measurements are OOD, and thus should not be trusted, whether due to the presence of a novel source or novel proportions.

Anomaly detection↗

HomPINNs: Homotopy physics-informed neural networks for learning multiple solutions of nonlinear elliptic differential equations

Physics-informed neural networks (PINNs) based machine learning is an emerging framework for solving nonlinear differential equations. However, due to the implicit regularity of neural network structure, PINNs can only find the flattest solution in most cases by minimizing the loss functions. In this paper, we combine PINNs with the homotopy continuation method, a classical numerical method to compute isolated roots of polynomial systems, and propose a new deep learning framework, named homotopy physics-informed neural networks (HomPINNs), for solving multiple solutions of nonlinear elliptic differential equations. The implementation of an HomPINN is a homotopy process that is composed of the training of a fully connected neural network, named the starting neural network, and training processes of several PINNs with different tracking parameters. The starting neural network is to approximate a starting function constructed by the trivial solutions, while other PINNs are to minimize the loss functions defined by boundary condition and homotopy functions, varying with different tracking parameters. These training processes are regraded as different steps of a homotopy process, and a PINN is initialized by the well-trained neural network of the previous step, while the first starting neural network is initialized using the default initialization method. Finally, several numerical examples are presented to show the efficiency of our proposed HomPINNs, including reaction-diffusion equations with a heart-shaped domain.

97 MATHEMATICS AND COMPUTING↗

Z-Target Radiography Postprocessing With A Deep Convolution Neural Network

Analyzing X-ray radiographs is crucial for understanding target behavior in Inertial Confinement Fusion (ICF) and High Energy Density (HED) platforms. However, the density of Magneto Raleigh Taylor (MRT) bands and limitations of target materials often obscure relevant spike growth and density information. To address this issue, machine learning postprocessing techniques can be applied to remove darkened regions in radiography images. In this study, a novel method is presented for removing MRT darkened regions from z-target radiographs using a convolutional neural network (CNN). The CNN, consisting of six layers, treats the darkened regions as noise and employs a mixed loss function and end-to-end frameworks to suppress them while preserving sharpness. The six-layer architecture is designed to effectively learn features when provided with a larger volume of learning space. Each layer is optimized using a mixed loss function that combines a standard loss pixel approach with a multi-scaled structural similarity index loss, which considers luminance, contrast, and structure in local neighborhoods. This approach is particularly beneficial for capturing the stochastic structure of MRT limbs. Due to the limited availability of experimental data, training is conducted using synthetic target radiography from 3D Alegra simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Physics constrained learning for data-driven inverse modeling from sparse observations

Deep neural networks (DNN) have been used to model nonlinear relations between physical quantities. Those DNNs are embedded in physical systems described by partial differential equations (PDE) and trained by minimizing a loss function that measures the discrepancy between predictions and observations in some chosen norm. This loss function often includes the PDE constraints as a penalty term when only sparse observations are available. As a result, the PDE is only satisfied approximately by the solution. However, the penalty term typically slows down the convergence of the optimizer for stiff problems. We present a new approach that trains the embedded DNNs while numerically satisfying the PDE constraints. We develop an algorithm that enables differentiating both explicit and implicit numerical solvers in reverse-mode automatic differentiation. This allows the gradients of the DNNs and the PDE solvers to be computed in a unified framework. We demonstrate that our approach enjoys faster convergence and better stability in relatively stiff problems compared to the penalty method. Furthermore, our approach allows for the potential to solve and accelerate a wide range of data-driven inverse modeling, where the physical constraints are described by PDEs and need to be satisfied accurately.

97 MATHEMATICS AND COMPUTING↗

Enforcing Self-Consistent Kinematic Constraints in Neutrino Energy Estimators

Machine learning algorithms have long been utilized across many experimental collaborations within the neutrino physics community in applications to ascertain the singular kinematic quantity of initial neutrino energy for use in neutrino oscillation analyses. However, most of these algorithms do not incorporate a coherent physical picture of initial neutrino kinematics, opting to introduce loss functions involving knowledge of only |pν |. Here, we argue for the introduction of composite loss functions utilizing the full kinematic description of the neutrino, pν ≡ (E, px, py , pz ), compiling all relevant energy and angle information consistently. The use of such a fully defined variable can be seen as a usage of Physics Informed Machine Learning.

Richi, R. R.↗

Consequences of Lineage-Specific Gene Loss on Functional Evolution of Surviving Paralogs: ALDH1A and Retinoic Acid Signaling in Vertebrate Genomes

Genome duplications increase genetic diversity and may facilitate the evolution of gene subfunctions. Little attention, however, has focused on the evolutionary impact of lineage-specific gene loss. Here, we show that identifying lineagespecific gene loss after genome duplication is important for understanding the evolution of gene subfunctions in surviving paralogs and for improving functional connectivity among human and model organism genomes. We examine the general principles of gene loss following duplication, coupled with expression analysis of the retinaldehyde dehydrogenase Aldh1a gene family during retinoic acid signaling in eye development as a case study. Humans have three ALDH1A genes, but teleosts have just one or two. We used comparative genomics and conserved syntenies to identify loss of ohnologs (paralogs derived from genome duplication) and to clarify uncertain phylogenies. Analysis showed that Aldh1a1 and Aldh1a2 form a clade that is sister to Aldh1a3-related genes. Genome comparisons showed secondarily loss of aldh1a1 in teleosts, revealing that Aldh1a1 is not a tetrapod innovation and that aldh1a3 was recently lost in medaka, making it the first known vertebrate with a single aldh1a gene. Interestingly, results revealed asymmetric distribution of surviving ohnologs between co-orthologous teleost chromosome segments, suggesting that local genome architecture can influence ohnolog survival. We propose a model that reconstructs the chromosomal history of the Aldh1a family in the ancestral vertebrate genome, coupled with the evolution of gene functions in surviving Aldh1a ohnologs after R1, R2, and R3 genome duplications. Results provide evidence for early subfunctionalization and late subfunction-partitioning and suggest a mechanistic model based on altered regulation leading to heterochronic gene expression to explain the acquisition or modification of subfunctions by surviving ohnologs that preserve unaltered ancestral developmental programs in the face of gene loss.

59 BASIC BIOLOGICAL SCIENCES↗

ICDARTS: Improving the Stability of Cyclic DARTS

Cyclic DARTS (CDARTS) is a Differentiable Architecture Search (DARTS)-based approach to neural architecture search (NAS) that uses a cyclic feedback mechanism to train search and evaluation networks concurrently. This training protocol aims to optimize the search process and evaluate the deep evaluation network comprised of discretized candidate operations. However, this approach introduces a loss function for the evaluation network dependent on the search network. The dissimilarity between the evaluation network’s loss function used during the search and retraining phases results in a search network that is a sub-optimal proxy for the final evaluation network accessed during retraining. We present a revised approach that removes the dependency of the evaluation network weights upon those of the search network. In addition, we introduce a modified process for relaxing the search network’s zero operations that allows these operations to be retained in the final evaluation networks.

Herron, Emily↗

Multi-Label Classification with Constraint-Based Learning for Hierarchical Consistency

We explore the limitations of traditional crossentropy loss in a hierarchical multi-label classification setting and introduce a novel loss function. This function is designed to integrate hierarchical constraints directly into the training process. By incorporating such constraints into the loss, our approach slightly improves the logical consistency of predictions in structured domains. We demonstrate the efficacy of our approach through experiments on primary site and histology classification by using electronic pathology reports. These results show that our proposed hierarchical loss function enhances the model's ability to produce predictions that are logically consistent with the natural data hierarchies, and it slightly improves predictive accuracy. Our framework may be extended to other hierarchical domains, however the performance gains are context specific.

Shivanna, Abhishek [ORNL] (ORCID:0009000665228593)↗

Fourier Neural Networks as Function Approximators and Differential Equation Solvers

We present a Fourier neural network (FNN) that can be mapped directly to the Fourier decomposition. The choice of activation and loss function yields results that replicate a Fourier series expansion closely while preserving a straightforward architecture with a single hidden layer. The simplicity of this network architecture facilitates the integration with any other higher-complexity networks, at a data pre- or postprocessing stage. We validate this FNN on naturally periodic smooth functions and on piecewise continuous periodic functions. We showcase the use of this FNN for modeling or solving partial differential equations with periodic boundary conditions. The main advantages of the current approach are the validity of the solution outside the training region, interpretability of the trained model, and simplicity of use.

Fourier decomposition↗

Field Work Proposal ERKJ358: Black-box training for scientific machine learning models (Final Report)

The overarching goal of this project is to develop a scalable black-box training capability for scientific machine learning (SciML) problems that are non-trainable with existing automatic differentiation (AD)-based algorithms. AD assumes that a loss function can be decomposed into a sequence of elementary operations whose derivatives are known. This assumption is violated when the loss function includes a black-box physical model (e.g., a legacy simulator). The current strategy, converting a black-box simulator to an AD-enabled code via differential programming, is inflexible and time-, labor-consuming. Thus, black-box optimization is a main workhorse for training SciML models, e.g., in scientific reinforcement learning, hyper-parameter fine tuning, designing SciML models with adversarial robustness, etc.

97 MATHEMATICS AND COMPUTING↗

Stacking AsFMT overexpression with BdPMT loss of function enhances monolignol ferulate production in Brachypodium distachyon

To what degree can the lignin subunits in a monocot be derived from monolignol ferulate (ML-FA) conjugates? This simple question comes with a complex set of variables. Three potential requirements for optimizing ML-FA production are as follows: (1) The presence of an active FERULOYL-CoA MONOLIGNOL TRANSFERASE (FMT) enzyme throughout monolignol production; (2) Suppression or elimination of enzymatic pathways competing for monolignols and intermediates during lignin biosynthesis; and (3) Exclusion of alternative phenolic compounds that participate in lignification. A 16-fold increase in lignin-bound ML-FA incorporation was observed by introducing an AsFMT gene into Brachypodium distachyon. On its own, knocking out the native p-COUMAROYL-CoA MONOLIGNOL TRANSFERASE (BdPMT) pathway that competes for monolignols and the p-coumaroyl-CoA intermediate did not change ML-FA incorporation, nor did partial loss of CINNAMOYL-CoA REDUCTASE1 (CCR1) function, which reduced metabolic flux to monolignols. However, stacking AsFMT into the Bdpmt-1 mutant resulted in a 32-fold increase in ML-FA incorporation into lignin over the wild-type level.

59 BASIC BIOLOGICAL SCIENCES↗