Search NASA⌕ Search

SEARCH · Search NASA

Results for “Loss function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

AIVT: Inference of turbulent thermal convection from measured 3D velocity data by physics-informed Kolmogorov-Arnold networks

We propose the artificial intelligence velocimetry-thermometry (AIVT) method to reconstruct a continuous and differentiable representation of the temperature and velocity in turbulent convection from measured three-dimensional (3D) velocity data. AIVT is based on physics-informed Kolmogorov-Arnold networks and trained by optimizing a loss function that minimizes residuals of the velocity data, boundary conditions, and governing equations. We apply AIVT to a set of simultaneously measured 3D temperature and velocity data of Rayleigh-Bénard convection, obtained by combining particle image thermometry and Lagrangian particle tracking. This enables us to directly compare machine learning results to true volumetric, simultaneous temperature and velocity measurements. We demonstrate that AIVT can reconstruct and infer continuous, instantaneous velocity and temperature fields and their gradients from sparse experimental data at a high resolution, providing an additional approach for understanding thermal turbulence.

Science & Technology - Other Topics↗

Plasmon–phonon coupling in the infrared reflectance spectra of Bi{sub 2}Se{sub 3} films

The results of studies of optical reflection in the far- and mid-infrared spectral regions are reported. The reflectance of five Bi{sub 2}Se{sub 3} topological insulator films grown by molecular-beam epitaxy on Si(111) substrates is measured. The characteristic parameters of phonons and plasmons are determined by means of dispersion analysis for multilayer structures. It is found that the plasma frequency in a layer close to the Si–film interface is noticeably higher than that in the film bulk. Calculations of the loss function show that plasmon–phonon coupling plays an important role in Bi{sub 2}Se{sub 3} films. The attenuated total internal reflection method is used to determine the frequency of the surface plasmon–phonon mode.

36 MATERIALS SCIENCE↗

Global Sensitivity Analysis for Statistical Model Parameters

Global sensitivity analysis (GSA) is typically used to analyze the influence of uncertain parameters in mathematical models and simulations. In principle, tools from GSA may be extended to analyze the influence of parameters in statistical models. Such analyses may enable reduced or parsimonious modeling and greater predictive capability. Yet, difficulties such as parameter correlation, model stochasticity, multivariate model output, and unknown parameter distributions prohibit a direct application of GSA tools to statistical models. By leveraging a loss function associated with the statistical model, we introduce a novel framework to address these difficulties and enable efficient GSA for statistical model parameters. Theoretical and computational properties are considered and hihglighted on a synthetic example. The framework is applied to a Gaussian process model from the literature, which depends on 95 parameters. Noninfluential parameters are discovered through GSA, and a reduced model with equal or stronger predictive capability is constructed by using only 79 parameters.

97 MATHEMATICS AND COMPUTING↗

fPINNs: Fractional Physics-Informed Neural Networks

Physics-informed neural networks (PINNs), introduced, are effective in solving integer-order partial differential equations (PDEs) based on scattered and noisy data. PINNs employ standard feedforward neural networks (NNs) with the PDEs explicitly encoded into the NN using automatic differentiation, while the sum of the mean-squared PDE residuals and the mean-squared error in initial-boundary conditions is minimized with respect to the NN parameters. Here we extend PINNs to fractional PINNs (fPINNs) to solve space-time fractional advection-diffusion equations (fractional ADEs), and we study systematically their convergence, hence explaining both fPINNs and PINNs for the first time. Specifically, we demonstrate their accuracy and effectiveness in solving multidimensional forward and inverse problems with forcing terms whose values are only known at randomly scattered spatio-temporal coordinates (black-box (BB) forcing terms). A novel element of the fPINNs is the hybrid approach that we introduce for constructing the residual in the loss function using both automatic differentiation for the integer-order operators and numerical discretization for the fractional operators. This approach bypasses the difficulties stemming from the fact that automatic differentiation is not applicable to fractional operators because the standard chain rule in integer calculus is not valid in fractional calculus. To discretize the fractional operators, we employ the Grünwald--Letnikov (GL) formula in one-dimensional fractional ADEs and the vector GL formula in conjunction with the directional fractional Laplacian in two- and three-dimensional fractional ADEs. We first consider the one-dimensional fractional Poisson equation and compare the convergence of the fPINNs against the finite difference method (FDM). We present the solution convergence using both the mean L 2 error as well as the standard deviation due to sensitivity to NN parameter initializations. Using different GL formulas we observe first-, second-, and third-order convergence rates for small size training sets but the error saturates for larger training sets. We explain these results by analyzing the four sources of numerical errors due to discretization, sampling, NN approximation, and optimization. The total error decays monotonically (below 10 –5 for a third-order GL formula) but it saturates beyond that point due to the optimization error. We also analyze the relative balance between discretization and sampling errors and observe that the sampling size and the number of discretization points (auxiliary points) should be comparable to achieve the highest accuracy. As we increase the depth of the NN up to certain value, the mean error decreases and the standard deviation increases, whereas the width has essentially no effect unless its value is either too small or too large. We next consider time-dependent fractional ADEs and compare white-box (WB) and BB forcing. We observe that for the WB forcing, our results are similar to the aforementioned cases; however, for the BB forcing fPINNs outperform FDM. Subsequently, we consider multidimensional time-, space-, and space-time-fractional ADEs using the directional fractional Laplacian and we observe relative errors of 10 –3 ~ 10 –4 . Lastly, we solve several inverse problems in one, two, and three dimensions to identify the fractional orders, diffusion coefficients, and transport velocities and obtain accurate results given proper initializations even in the presence of significant noise.

97 MATHEMATICS AND COMPUTING↗

Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks

One of the open problems in scientific computing is the long-time integration of nonlinear stochastic partial differential equations (SPDEs), especially with arbitrary initial data. We address this problem by taking advantage of recent advances in scientific machine learning and the spectral dynamically orthogonal (DO) and borthogonal (BO) methods for representing stochastic processes. The recently introduced DO/BO methods reduce the SPDE to solving a system of deterministic PDEs and a system of stochastic ordinary differential equations. Specifically, we propose two new physics-informed neural networks (PINNs) for solving time-dependent SPDEs, namely the neural network (NN)-DO/BO methods. The proposed methods incorporate the DO/BO constraints into the loss function (along with the modal decomposition of the SPDE) with an implicit form instead of generating explicit expressions for the temporal derivatives of the DO/BO modes. Hence, the NN-DO/BO methods can overcome some of the drawbacks of the original DO/BO methods. For example, we do not need the assumption that the covariance matrix of the random coefficients is invertible as in the original DO method, and we can remove the assumption of no eigenvalue crossing as in the original BO method. Moreover, the NN-DO/BO methods can be used to solve time-dependent stochastic inverse problems with the same formulation and same computational complexity as for forward problems. Furthermore, we demonstrate the capability of the proposed methods via several numerical examples, namely: (1) A linear stochastic advection equation with deterministic initial condition: we obtain good results with the proposed methods, while the original DO/BO methods cannot be applied directly in this case. (2) Long-time integration of the stochastic Burgers' equation: we show the good performance of NN-DO/BO methods, especially the effectiveness of the NN-BO approach for such problems with many eigenvalue crossings during the whole time evolution, while the original BO method fails. (3) Nonlinear reaction diffusion equation: we consider both the forward problem and the inverse problems, including very noisy initial point values, to investigate the flexibility of the NN-DO/BO methods in handling inverse and mixed type problems. Taken together, these simulation results demonstrate that the NN-DO/BO methods can be employed to effectively quantify uncertainty propagation in a wide range of physical problems, but future work should address the efficiency issue of PINNs for forward problems.

97 MATHEMATICS AND COMPUTING↗

Generative Ensemble Regression: Learning Particle Dynamics from Observations of Ensembles with Physics-informed Deep Generative Models

Here, we propose a new method for inferring the governing stochastic ordinary differential equations (SODEs) by observing particle ensembles at discrete and sparse time instants, i.e., multiple “snapshots.” Particle coordinates at a single time instant, possibly noisy or truncated, are recorded in each snapshot but are unpaired across the snapshots. By training a physics-informed generative model that generates “fake” sample paths, we aim to fit the observed particle ensemble distributions with a curve in the probability measure space, which is induced from the inferred particle dynamics. We employ different metrics to quantify the differences between distributions, e.g., the sliced Wasserstein distances and the adversarial losses in generative adversarial networks. We refer to this method as generative “ensemble-regression” (GER), in analogy to the classic “point-regression,” where we infer the dynamics by performing regression in the Euclidean space. We illustrate the GER by learning the drift and diffusion terms of particle ensembles governed by SODEs with Brownian motions and Lévy processes up to 100 dimensions. We also discuss how to treat cases with noisy or truncated observations. Apart from systems consisting of independent particles, we also tackle nonlocal interacting particle systems with unknown interaction potential parameters by constructing a physics-informed loss function. Finally, we investigate scenarios of paired observations and discuss how to reduce the dimensionality in such cases by proving a convergence theorem that provides theoretical support.

97 MATHEMATICS AND COMPUTING↗

Learning Optimal Multigrid Smoothers via Neural Networks

Multigrid methods are one of the most efficient techniques for solving large sparse linear systems arising from partial differential equations (PDEs) and graph Laplacians from machine learning applications. One of the key components of multigrid is smoothing, which aims at reducing high-frequency errors on each grid level. However, finding optimal smoothing algorithms is problem-dependent and can impose challenges for many problems. In this paper, we propose an efficient adaptive framework for learning optimized smoothers from operator stencils in the form of convolutional neural networks (CNNs). Here, the CNNs are trained on small-scale problems from a given type of PDEs based on a supervised loss function derived from multigrid convergence theories and can be applied to large-scale problems of the same class of PDEs. Numerical results on anisotropic rotated Laplacian problems and variable coefficient diffusion problems demonstrate improved convergence rates and solution time compared with classical hand-crafted relaxation methods.

97 MATHEMATICS AND COMPUTING↗

Designing observables for measurements with deep learning

Many analyses in particle and nuclear physics use simulations to infer fundamental, effective, or phenomenological parameters of the underlying physics models. When the inference is performed with unfolded cross sections, the observables are designed using physics intuition and heuristics. We propose to design targeted observables with machine learning. Unfolded, differential cross sections in a neural network output contain the most information about parameters of interest and can be well-measured by construction. The networks are trained using a custom loss function that rewards outputs that are sensitive to the parameter(s) of interest while simultaneously penalizing outputs that are different between particle-level and detector-level (to minimize detector distortions). We demonstrate this idea in simulation using two physics models for inclusive measurements in deep inelastic scattering. We find that the new approach is more sensitive than classical observables at distinguishing the two models and also has a reduced unfolding uncertainty due to the reduced detector distortions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Glass Refraction Distortion Object Detection via Abstract Features

Glass reflection and refraction lead to missing and distorted object feature data, affecting the accuracy of object detection. In order to solve the above problems, this paper proposed a glass refraction distortion object detection via abstract features. The number of parameters of the algorithm is reduced by introducing skip connections and expansion modules with different expansion rates. The abstract feature information of the object is extracted by binary cross-entropy loss. Meanwhile, the abstract feature distance between the object domain and source domain is reduced by a loss function, which improves the accuracy of object detection under glass interference. To verify the effectiveness of the algorithm in this paper, the GRI dataset is produced and made public on GitHub. The algorithm of this paper is compared with the current state-of-the-art Deep Face, VGG Face, TBE-CNN, DA-GAN, PEN-3D, LMZMPM, and the average detection accuracy of our algorithm is 92.57% at the highest, and the number of parameters is only 5.13 M.

Cai, Lei↗

PHYGNN (Physics Guided Neural Networks)

This implementation of a Physics Guided Neural Network (PHYGNN) augments a traditional neural network loss function with a generic term that can be used to guide the neural network to learn physical or theoretical constraints.

Buster, Grant↗

AADL: Anderson Accelerated Deep Learning

We propose a stable, distributed approach to perform AA that accelerates the convergence rate of stochastic first-order optimizers to train neural networks. Differently from previous works, we do not alter neither the scheme to perform AA nor the loss function minimized during the training. To improve robustness against stagnation, we customize general guidelines that suggest to relax the frequency of AA corrections by performing AA only at the end of an entire training epoch. To improve robustness of AA against the stochastic oscillations of first-order optimizers, we average the gradients computed on consecutive stochastic optimization updates. The improved regularity of the converging sequence and the reduced amplitude of stochastic oscillations across consecutive optimization steps allows AA to efficiently extrapolate an improved converging sequence, thereby overcoming limitations of existing approaches to perform AA on stochastic optimization.

Lupo Pasini, Massimiliano [Oak Ridge National Lab.↗

Characterizing possible failure modes in physics-informed neural networks (characterizing-pinns-failure-modes) v0.1.0

This software provides an implementation of the physics-informed neural network setup for different differential equations. It provides a way to solve differential equations using machine learning, via a physical constraint term in the loss function. Physics-informed neural networks are a popular approach, but here, the failures of such an approach are characterized and better solutions and training procedures are provided.

Krishnapriyan, Aditi↗

pnnl/mlxrd

A multi-task neural network for the simultaneous identification of multiple compounds from XRD data obtained in a synchrotron-based hydrothermal fluid system. To enhance the alignment between the training and test datasets, random noise is introduced into the synthetic training dataset, and minor labeled experimental data is incorporated. Additionally, we employ a weighted cross-entropy loss function to address data imbalances and ensure smoother model training.

Li, Ang↗

Fair Bagging Boosting Models [SWR-24-38]

Fair Bagging Boosting Models is a software implementation of a framework for building, measuring bias and correcting bias in 3 popular forest machine learning models: gradient boosted trees (GBT), random forest (RF), and XGBoost models, using the XGBoost library. The framework takes advantage of the flexibility in XGBoost library to represent gradient boosted tree and random forest models, as well as the ability to use custom loss function.

Ugirumurera, Juliette↗

PyTorch Implementation of Log-Additive Convolutional Neural Networks

This code is a collection of python code that defines, trains, and tests Log-Additive Convolutional Neural Networks. The model components and training routine are based on the PyTorch python library. The code implements the Log-Additive Convolutional Neural Networks as described in Pagendam et al. 2023. In addition to the Log-Additive Convolutional Neural Networks, this library also defines the Log-Normal Density loss function as described in Pagendam et al. 2023. Code from this paper is not publicly available, so the Pytorch implementation of this type of model is unique to this library.

Callis, Skylar↗

Moltensaltpropnet

MoltenSaltPropnet is a physics-informed machine learning framework that aims to predict the thermophysical properties of molten fluoride and chloride salt mixtures, which are crucial for the design and safety of Generation IV molten salt reactors. The code processes data from the Molten-Salt Thermal Properties Database (MSTDB-TP) and the Janz compendium, converting critically evaluated correlations into fast, differentiable surrogate models for density, viscosity, thermal conductivity, and heat capacity across 448 distinct salt systems. The implementation consists of several key components: 1. Data Curation: The code parses and cleans the raw data, normalizing elemental mole fractions and extracting relevant regression coefficients for various thermophysical properties. 2. Feature Engineering: It generates fixed-length numerical descriptors that encapsulate the composition and temperature, incorporating polynomial interaction terms and dimensionality-reduction techniques to optimize model performance. 3. Coefficient Learning: Four different machine learning architectures are employed: a deep residual network (ResNet), a Kolmogorov–Arnold network (KAN), a sparsity-inducing neural network (SNN), and classical regression models. Each model learns to predict coefficients that define the temperature-dependent correlations for the thermophysical properties. 4. Property Reconstruction: The predicted coefficients are used to compute temperature-dependent property values, ensuring positivity and monotonic trends through a composite loss function that enforces physical constraints. 5. User Interface: An open-source web application enables users to filter the database, train task-specific models, and visualize the results, allowing for rapid exploration of candidate salt mixtures. MoltenSaltPropnet bridges the gap between limited experimental data and high-fidelity reactor simulations, providing a powerful tool for researchers in the field of molten salt reactors and advanced nuclear energy systems.

Retamales, Mauricio Eduardo Tano [Idaho National L↗

TEMPEST

This repository solves the problem of driver identification through vehicular and biometric data. Through an embedding-based approach and a novel loss function, we're able to distinguish between different drivers' behaviors. This also provides preprocessing for reproducibility of results.The code preprocesses vehicular data, trains neural networks, and outputs predictions.This code introduces a novel embedding-based neural network with a 91% rank-1 accuracy, as well as all code to reproduce training and results.

Musgrove, Kyle↗

Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training

As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for processing sequential data like audio and text. However, most RSNNs suffer from two problems. 1. Due to the lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. 2. Training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNNs called Skip-Connected Self-Recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced in a stereotyped manner by adding self-recurrent connections to spiking neurons. The SNNs with self-recurrent connections can realize recurrent behaviors similar to those of more complex RSNNs while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. The network dynamics is enriched by skip connections between nonadjacent layers. Moreover, we propose a new backpropagation (BP) method called backpropagated intrinsic plasticity (BIP) to further boost the performance of ScSr-SNNs by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP method optimizes intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Here, based on challenging speech, neuromorphic speech, and neuromorphic image datasets, the proposed ScSr-SNNs can boost performance by up to 2.85% compared with other types of RSNNs trained by state-of-the-art BP methods.

97 MATHEMATICS AND COMPUTING↗