Search NASA⌕ Search

SEARCH · Search NASA

Results for “Loss function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Using support vector machines for anomalous change detonation

We cast anomalous change detection as a binary classification problem, and use a support vector machine (SVM) to build a detector that does not depend on assumptions about the underlying data distribution. To speed up the computation, our SVM is implemented, in part, on a graphical processing unit. Results on real and simulated anomalous changes are used to compare performance to algorithms which effectively assume a Gaussian distribution. In this paper, we investigate the use of support vector machines (SVMs) with radial basis kernels for finding anomalous changes. Compared to typical applications of SVMs, we are operating in a regime of very low false alarm rate. This means that even for relatively large training sets, the data are quite meager in the regime of operational interest. This drives us to use larger training sets, which in turn places more of a computational burden on the SVM. We initially considered three different approaches to to address the need to work in the very low false alarm rate regime. The first is a standard SVM which is trained at one threshold (where more reliable estimates of false alarm rates are possible) and then re-thresholded for the low false alarm rate regime. The second uses the same thresholding approach, but employs a so-called least squares SVM; here a quadratic (instead of a hinge-based) loss function is employed, and for this model, there are good theoretical arguments in favor of adjusting the threshold in a straightforward manner. The third approach employs a weighted support vector machine, where the weights for the two types of errors (false alarm and missed detection) are automatically adjusted to achieve the desired false alarm rate. We have found in previous experiments (not shown here) that the first two types can in some cases work well, while in other cases they do not. This renders both approaches unreliable for automated change detection. By contrast, the third approach reliably produces good results, but at the cost of larger computational requirements caused by the need to estimate very small false alarm rates. To address these computational requirements, we employ a recently developed in-house solver for SVMs that is significantly faster than freely available standard solvers. But these computational issues are secondary to the larger question: do kernelized solutions provide better performance, in terms of detection rates and false alarm rates, than more traditional methods for change detection that effectively assume Gaussian data distributions? To this end, we will compare ROC curves obtained from the SVM with those from chronochrome, covariance equalization, and hyperbolic anomalous change detection.

97 MATHEMATICS AND COMPUTING↗

Novel deep learning methods for track reconstruction

For the past year, the HEP.TrkX project has been investigating machine learning solutions to LHC particle track reconstruction problems. A variety of models were studied that drew inspiration from computer vision applications and operated on an image-like representation of tracking detector data. While these approaches have shown some promise, image-based methods face challenges in scaling up to realistic HL-LHC data due to high dimensionality and sparsity. In contrast, models that can operate on the spacepoint representation of track measurements ("hits") can exploit the structure of the data to solve tasks efficiently. In this paper we will show two sets of new deep learning models for reconstructing tracks using space-point data arranged as sequences or connected graphs. In the first set of models, Recurrent Neural Networks (RNNs) are used to extrapolate, build, and evaluate track candidates akin to Kalman Filter algorithms. Such models can express their own uncertainty when trained with an appropriate likelihood loss function. The second set of models use Graph Neural Networks (GNNs) for the tasks of hit classification and segment classification. These models read a graph of connected hits and compute features on the nodes and edges. They adaptively learn which hit connections are important and which are spurious. The models are scaleable with simple architecture and relatively few parameters. Results for all models will be presented on ACTS generic detector simulated data.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A Physics-Driven Deep-Learning Network for Solving Nonlinear Inverse Problems

Geosteering inversion, which can be viewed as a nonlinear inverse problem, is an important technique used by directional drilling. Traditional methods that rely on iterative procedures and regularization are sensitive to the selection of initial values and can be slow due to convergence issues. In industrial applications, a lookup table is used to produce fast predictions. However, performance is not guaranteed by this approach due to the limitation of the hardware. In this paper, we propose a novel physics-driven deep-learning framework for providing a fast, accurate surrogate to solve the inverse problem. Particularly, leveraged by the forward physical model and 1D convolutional neural network (1D-CNN), the proposed method provides more reliable solutions to the inverse problem with improved performance. In addition, a new physics-driven loss function is introduced to accommodate both the model misfit and the data misfit. In conclusion, our experiments demonstrate the effectiveness of our method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Anderson Acceleration for Distributed Training of Deep Learning Models

Anderson acceleration (AA) is an extrapolation technique that has recently gained interest in the deep learning (DL) community to speed-up the sequential training of DL models. However, when performed at large scale, the DL training is exposed to a higher risk of getting trapped into steep local minima of the training loss function, and standard AA does not provide sufficient acceleration to escape from these steep local minima. This results in poor generalizability and makes AA ineffective. To restore AA’s advantage to speed-up the training of DL models on large scale computing platforms, we combine AA with an adaptive moving average procedure that boosts the training to escape from steep local minima. By monitoring the relative standard deviation between consecutive iterations, we also introduce a criterion to automatically assess whether the moving average is needed. We applied the method to the following DL instantiations for image classification: (i) ResNet50 trained on the open-source CIFAR100 dataset and (ii) ResNet50 trained on the open-source ImageNet1k dataset. Numerical results obtained using up to 1,536 NVIDIA V100 GPUs on the OLCF supercomputer Summit showed the stabilizing effect of the moving average on AA for all the problems above.

Lupo Pasini, Massimiliano↗

Refining fast simulation using machine learning

At the CMS experiment, a growing reliance on the fast Monte Carlo application (FastSim) will accompany the high luminosity and detector granularity expected in Phase 2. The FastSim chain is roughly 10 times faster than the application based on the GEANT4 detector simulation and full reconstruction referred to as FullSim. However, this advantage comes at the price of decreased accuracy in some of the final analysis observables. In this contribution, a machine learning-based technique to refine those observables is presented. We employ a regression neural network trained with a sophisticated combination of multiple loss functions to provide post-hoc corrections to samples produced by the FastSim chain. The results show considerably improved agreement with the FullSim output and an improvement in correlations among output observables and external parameters. This technique is a promising replacement for existing correction factors, providing higher accuracy and thus contributing to the wider usage of FastSim.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The application of Plant Reliability Data Information System (PRINS) to CANDU reactor

As risk-informed applications (RIAs) are actively implanted in the nuclear industry, an issue associated with technical adequacy of Probabilistic Safety Assessment (PSA) arises in its modeling and data sourcing. In Korea, PSA for all Korean NPPs has been completed and KHNP(Korea Hydro and Nuclear Power Plant Company) developed the database called the Plant Reliability Data Information System (PRinS). It has several characteristics that distinguish it from other database system such as NPRDs (INPO,1994), PRIS (IAEA), and SRDF (EdF). This database has the function of systematic data management such as automatic data-gathering, periodic data deposition and updating, statistical analysis including Bayesian method, and trend analysis of failure rate or unavailability. In recent PSA for CANDU reactor, the component failure data of EPRI ALWR URD and Component Reliability Database were preferentially used as generic data set. The error factor for most component failure data was estimated by using the information NUREG/CR-4550 and NUREG/CR-4639. Also, annual trend analysis was performed for the functional losses of components using the statistical analysis and chart module of PRinS. Furthermore, the database has been updated regularly and maintained as a living program to reflect the current status. This paper presents the failure data analysis using PRinS which provides Bayesian analysis on main components in the CANDU reactor. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Exploring gauge-fixing conditions with gradient-based optimization

Lattice gauge fixing is required to compute gauge-variant quantities, for example those used in RI-MOM renormalization schemes or as objects of comparison for model calculations. Recently, gauge-variant quantities have also been found to be more amenable to signal-to-noise optimization using contour deformations. These applications motivate systematic parameterization and exploration of gauge-fixing schemes. This work introduces a differentiable parameterization of gauge fixing which is broad enough to cover Landau gauge, Coulomb gauge, and maximal tree gauges. The adjoint state method allows gradient-based optimization to select gauge-fixing schemes that minimize an arbitrary target loss function.

Detmold, William↗

Uncertainty-aware Continuous Implicit Neural Representations for Remote Sensing Object Counting

Many existing object counting methods rely on density map estimation (DME) of the discrete grid representation by decoding extracted image semantic features from designed convolutional neural networks (CNNs). Relying on discrete density maps not only leads to information loss dependent on the original image resolution, but also has a scalability issue when analyzing high-resolution images with cubically increasing memory complexity. Furthermore, none of the existing methods can offer reliable uncertainty quantification (UQ) for the derived count estimates. To overcome these limitations, we design UNcertainty-aware, hypernetwork-based Implicit neural representations for Counting (UNIC) to assign probabilities and the corresponding counting confidence over continuous spatial coordinates. We derive a sampling-based Bayesian counting loss function and develop the corresponding model training algorithm. UNIC outperforms existing methods on the Remote Sensing Object Counting (RSOC) dataset with reliable UQ and improved interpretability of the derived count estimates. Our code is available at https://github.com/SiyuanXu-tamu/UNIC.

97 MATHEMATICS AND COMPUTING↗

FedOSAA: Improving Federated Learning with One-Step Anderson Acceleration

Federated learning (FL) is a distributed machine learning approach that enables multiple local clients and a central server to collaboratively train a model while keeping the data on their own devices. First-order methods, particularly those incorporating variance reduction techniques, are the most widely used FL algorithms due to their simple implementation and stable performance. However, these methods tend to be slow and require a large number of communication rounds to reach the global minimizer. We propose FedOSAA, a novel approach that preserves the simplicity of first-order methods while achieving the rapid convergence typically associated with second-order methods. Our approach applies one Anderson acceleration (AA) step following classical local updates based on first-order methods with variance reduction, such as FedSVRG and SCAFFOLD, during local training. This AA step is able to leverage curvature information from the history points and gives a new update that approximates the Newton-GMRES direction, thereby significantly improving the convergence. We establish a local linear convergence rate to the global minimizer of FedOSAA for smooth and strongly convex loss functions. Numerical comparisons show that FedOSAA substantially improves the communication and computation efficiency of the original first-order methods, achieving performance comparable to second-order methods like GIANT.

Feng, Xue [University of California, Davis]↗

hippynn Python Package

hippynn is a python package for defining, training, and applying neural networks to atomistic systems. In particular, it focuses on Hierarchical Interacting Particle Neural Networks (HIP-NNs), a deep learning architecture for atomistic systems. HIP-NNs take input data describing the properties of atomistic systems (often generated using ab-initio quantum mechanics) to learn fast and accurate models. A trained HIP-NN can predict potential energy surfaces, atomic charges, and more. hippynn uses PyTorch for portable high-performance code, including both CPU and GPU support. hippynn allows for extensive customization, including user-defined models and loss functions, to facilitate future research into extensions of HIP-NN as well as other atomistic deep learning models.

Lubbers, Nicolas↗

LncRNA cancer susceptibility candidate 15 accelerates the breast cancer cells progression via miR-153–3p/KLF5 positive feedback loop

Highlights: • CASC15 expression level was aberrantly high-expressed in breast cancer tissue specimens and cells. • Loss-of-functional experiments showed that knockdown of CASC15 suppressed the malignant behaviors of breast cancer cells. • CASC15 functioned as ceRNA of miR-153–3p, besides, miR-153–3p targeted the 3‘-UTR of KLF5 mRNA. • KLF5 binds with the promoter region of CASC15 and activates the transcription. Emerging literature have illustrated the vital regulatory roles of long noncoding RNAs (lncRNAs) on the breast cancer tumorigenesis. Although series of researches have been proceeded on the pathogenesis, there are still much of unsolved mysteries worth investigating. This study uncovered that CASC15 expression level was aberrantly high-expressed in breast cancer tissue specimens and cells. Functionally, the loss-of-functional experiments showed that knockdown of CASC15 suppressed the malignant behaviors of breast cancer cells, such as proliferation, invasion and tumor growth in vitro and vivo. Mechanically, we confirmed that CASC15 functioned as a competing endogenous RNA (ceRNA) of miR-153–3p, besides, miR-153–3p targeted the 3‘-UTR of KLF5 mRNA utilizing the bioinformatics online tools, luciferase reporter assay and RNA immunoprecipitation. Interestingly, we confirmed that the transcription factor KLF5 binds with the promoter region of CASC15 and activates the transcription. In conclusion, we validated the positive feedback loop of KLF5/CASC15/miR-153–3p/KLF5 in the acceleration of breast cancer malignant behaviors and tumorigenesis, suggesting the important biologic roles of CASC15 on the breast cancer tumorigenesis.

60 APPLIED LIFE SCIENCES↗

Real-space Green's function approach for intrinsic losses in x-ray spectra

Intrinsic inelastic losses in x-ray spectra originate from excitations in an interacting electron system due to a suddenly created core-hole. These losses characterize the features observed in x-ray photoemission spectra (XPS), as well as many-body effects such as satellites and edge-singularities in x-ray absorption spectra (XAS). However, they are usually neglected in practical calculations. As shown by Langreth these losses can be treated within linear response in terms of a cumulant Green's function in momentum space. Here we present a complementary ab initio real-space Green's function generalization of the Langreth cumulant in terms of the dynamically screened core-hole interaction W c (ω) and the independent particle response function. Here we find that the cumulant kernel β⁡(ω) is analogous to XAS, but with the transition operator replaced by the core-hole potential with monopole selection rules. The behavior reflects the analytic structure of the loss function, with peaks near the zeros of the dielectric function, consistent with delocalized quasiboson excitations. The approach simplifies when W c (ω) is localized and spherically symmetric. In conclusion, illustrative results and comparisons are presented for the electron gas, sodium, and some early transition metal compounds.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Application of a deep learning semantic segmentation model to helium bubbles and voids in nuclear materials

Imaging nanoscale radiation-induced defects using the transmission electron microscope (TEM) is a key factor in the successful implementation of materials for nuclear energy structural applications. Analyzing each defect in a TEM micrograph is currently a manual task. To identify the defects in a single image can take anywhere from 15 min to an hour and a project can require the analysis of anywhere from tens to ≥ 100 images. Here, we use artificial intelligence (AI) models to automate this task. For simplification, we evaluated images with only a single type of defect; helium bubbles. Additionally, we performed semantic segmentation of these helium bubble defects in electron microscopy images of irradiated FeCrAl alloys using a deep learning DefectSegNet model. This model, which was previously used to classify crystal defects, is inspired by the classic DenseNet and U-Net image segmentation models. It claims high spatial resolution, but has poor performance at object boundaries. Our paper improves the DefectSegNet model’s application by adding two new features. First, the DefectSegNet model is applied not only to perform calculation pixel-wise but also object (or feature) wise. Because object-wise metrics are directly relevant to our final goal of detecting bubbles, whereas pixel-wise classification is only an intermediate step, it is an important part of our study. Second, a distance map loss (DML) function has been added to increase its performance at object boundaries. It is crucial to accurately represent defects boundaries, especially bubbles, in order to track the bubble-induced swelling caused by irradiation. The boundary-focused DML function is also compared to other loss functions like Cross-entropy, Weighted Binary Cross Entropy (WBCE), Dice and Intersection over Union (IOU). Finally, by incorporating new features, we found a marked improvement on segmentation quality and better shape preservation at the boundaries and areas of the bubbles.

42 ENGINEERING↗

Recovery of Spaceflight-induced Bone Loss: Bone Mineral Density after Long-Duration Missions as Fitted with an Exponential Function

The loss of bone mineral in NASA astronauts during spaceflight has been investigated throughout the more than 40 years of space travel. Consequently, it is a medical requirement at NASA Johnson Space Center (JSC) that changes in bone mass be monitored in crew members by measuring bone mineral density (BMD) with dual-energy x-ray absorptiometry (DXA) before and after flight on astronauts who serve on long-duration missions (4-6 months). We evaluated this repository of medical data to track whether there is recovery of bone mineral that was lost during spaceflight. Our analysis was supplemented by BMD data from cosmonauts ( by convention, a space traveler formally employed by the Russia Aviation and Space Agency or by the previous Soviet Union) who had also flown on long-duration missions. Data from a total of 45 individual crew members -- a small number of whom flew on more than one mission -- were used in this analysis. Changes in BMD (between 56 different sets of pre- and postflight measurements) were plotted as a function of time (days after landing). Plotted BMD changes were fitted to an exponential mathematical function that estimated: i) BMD change on landing day (day 0) and ii) the number of days after landing when 50% of the lost bone would be recovered ("50% recovery time") in the lumbar spine, trochanter, pelvis, femoral neck and calcaneus. In sum, averaged losses of bone mineral after long-duration spaceflight ranged between 2-9% across all sites with our recovery model predicting a 50% restoration of bone loss for all sites to be within 9 months.

Sibonga, J. D.↗

Plastidial transporters KEA1, -2, and -3 are essential for chloroplast osmoregulation, integrity, and pH regulation in Arabidopsis

Multiple K + transporters and channels and the corresponding mutants have been described and studied in the plasma membrane and organelle membranes of plant cells. However, knowledge about the molecular identity of chloroplast K + transporters is limited. Potassium transport and a well-balanced K + homeostasis were suggested to play important roles in chloroplast function. Because no loss-of-function mutants have been identified, the importance of K + transporters for chloroplast function and photosynthesis remains to be determined. In this paper, we report single and higher-order loss-of-function mutants in members of the cation/proton antiporters-2 antiporter superfamily KEA1, KEA2, and KEA3. KEA1 and KEA2 proteins are targeted to the inner envelope membrane of chloroplasts, whereas KEA3 is targeted to the thylakoid membrane. Higher-order but not single mutants showed increasingly impaired photosynthesis along with pale green leaves and severely stunted growth. The pH component of the proton motive force across the thylakoid membrane was significantly decreased in the kea1kea2 mutants, but increased in the kea3 mutant, indicating an altered chloroplast pH homeostasis. Electron microscopy of kea1kea2 leaf cells revealed dramatically swollen chloroplasts with disrupted envelope membranes and reduced thylakoid membrane density. Unexpectedly, exogenous NaCl application reversed the observed phenotypes. Furthermore, the kea1kea2 background enables genetic analyses of the functional significance of other chloroplast transporters as exemplified here in kea1kea2Na + /H + antiporter1 (nhd1) triple mutants. Taken together, the presented data demonstrate a fundamental role of inner envelope KEA1 and KEA2 and thylakoid KEA3 transporters in chloroplast osmoregulation, integrity, and ion and pH homeostasis.

59 BASIC BIOLOGICAL SCIENCES↗

Electronic structure of reduced CeO 2 (111) surfaces interacting with hydrogen as revealed through electron energy loss spectroscopy in comparison with theoretical investigations

Based on both, ab-initio cluster calculations as well as periodic density functional based loss function calculations, we have assigned the origins of the valence electron excitation regime in electron energy loss spectra of well-ordered ceria films in (111) orientation at various states of reduction as well as after exposing the reduced films to hydrogen from the gas phase (Li et al, Angew. Chem. Int. Ed., 58 (2019) 14686–14693). The explicit calculation of intensity distributions using the dipole approximation allow us to draw conclusions about the nature of oxygen vacancies, which occur upon reduction and how those interact with hydrogen to form hydride species. Herein, we also reports a brief discussion of vibrational excitations of hydrogen loaded ceria and corroborates previous interpretations, based on inelastic neutron scattering.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Down-regulation of Transducin-Like Enhancer of Split protein 4 in hepatocellular carcinoma promotes cell proliferation and epithelial-Mesenchymal-Transition

Background: Transducin-Like Enhancer of Split protein 4 (TLE4) has been reported to be involved in some subsets of acute myeloid leukemia and colorectal cancer. In the present study, we aimed to explore the role of TLE4 in tumorigenesis and cancer progression in hepatocellular carcinoma (HCC). Methods: The expression pattern of TLE4 in HCC was determined by Western-blot and qRT-PCR, gain-of-function and loss-of-function was used to explore the biological role of TLE4 in HCC cells. A xenograft model was established to confirm its effects on proliferation. Results: The protein expression levels of TLE4 were significantly down-regulated in HCC tissues compared to matched adjacent normal liver tissues. In vitro, down-regulation of TLE4 in Huh7 or SMMC-7721 promoted cell proliferation and ectopical expression of TLE4 in Hep3B or Bel-7404 suppressed cell proliferation. In addition, the cell colony formation ability was enhanced after down-regulation of TLE4 expression in Huh-7 but suppressed after over-expression in Hep3B. Furthermore, down-regulation of TLE4 increased the cell invasion ability, as well as increased the expression level of Vimentin and decreased that of E-cadherin, indicating a phenotype of epithelial-mesenchymal transition (EMT) in HCC cells. On the contrary, ectopical expression of TLE4 in HCC cells decreased the cell invasion ability and inhibited EMT. In vivo, compared to control group, xenograft tumor volumes were significantly decreased in TLE4 overexpression group. Conclusions: These results demonstrated that TLE4 might play important regulatory roles in cellular proliferation and EMT process in HCC. - Highlights: • TLE4 is significantly down-regulated in HCC samples. • Down regulated of TLE4 in HCC cells promotes cell proliferation. • Down regulated of TLE4 in HCC cells promotes epithelial-to-mesenchymal transition.

60 APPLIED LIFE SCIENCES↗