Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

A transfer learning approach for acoustic emission zonal localization on steel plate-like structure using numerical simulation and unsupervised domain adaptation

The detection and localization of damage in metallic structures using acoustic emission (AE) monitoring and artificial intelligence technology such as deep learning has been widely studied. However, a current challenge of this approach is the difficulty of obtaining sufficient labeled historical AE signals for the training process of deep learning models. This problem can be approached through the implementation of transfer learning. The innovation of this paper lies in the development of a transfer learning approach for AE source localization on a stainless-steel structure when no historical labeled AE signals are available for training. A finite element model is developed to generate numerical AE signals for the training. Unsupervised domain adaptation (UDA) technology is utilized to reduce the distribution difference between the numerical and the realistic AE signals and to derive the localization results of the unlabeled realistic AE signals. Finally, the results suggest that the proposed approach is capable of localizing AE signals with high accuracy in the absence of labeled training data.

42 ENGINEERING↗

INFERENCE OF HEATING PROPERTIES FROM “HOT” NON-FLARING PLASMAS IN ACTIVE REGION CORES. II. NANOFLARE TRAINS

Despite its prediction over two decades ago, the detection of faint, high-temperature (“hot”) emission due to nanoflare heating in non-flaring active region cores has proved challenging. Using an efficient two-fluid hydrodynamic model, this paper investigates the properties of the emission expected from repeating nanoflares (a nanoflare train) of varying frequency as well as the separate heating of electrons and ions. If the emission measure distribution (EM(T)) peaks at T = T{sub m} , we find that EM(T{sub m}) is independent of details of the nanoflare train, and EM(T) above and below T{sub m} reflects different aspects of the heating. Below T{sub m} , the main influence is the relationship of the waiting time between successive nanoflares to the nanoflare energy. Above T{sub m}, power-law nanoflare distributions lead to an extensive plasma population not present in a mono-energetic train. Furthermore, in some cases, characteristic features are present in EM(T). Such details may be detectable given adequate spectral resolution and a good knowledge of the relevant atomic physics. In the absence of such resolution we propose some metrics that can be used to infer the presence of “hot” plasma.

79 ASTRONOMY AND ASTROPHYSICS↗

Unsupervised domain adaptation for radioisotope identification in gamma spectroscopy

Training machine learning models for radioisotope identification using gamma spectroscopy remains an elusive challenge for many practical applications, largely stemming from the difficulty of acquiring and labeling large, diverse experimental datasets. Simulations can mitigate this challenge, but the accuracy of models trained on simulated data can deteriorate substantially when deployed to an out-of-distribution operational environment. In this study, we demonstrate that unsupervised domain adaptation (UDA) can improve the ability of a model trained on synthetic data to generalize to a new testing domain, provided unlabeled data from the target domain are available. Conventional supervised techniques are unable to utilize this data because the absence of isotope labels precludes defining a supervised classification loss. Instead, we first pretrain a spectral classifier using labeled synthetic data and subsequently leverage unlabeled target data to align the learned feature representations between the source and target domains. We compare a range of different UDA techniques, finding that minimizing the maximum mean discrepancy (MMD) between source and target feature vectors yields the most consistent improvement to testing scores. For instance, using a custom transformer-based neural network, we achieved a testing accuracy of $0.904 \pm 0.022$ on an experimental LaBr test set after performing unsupervised feature alignment via MMD minimization, compared to $0.754 \pm 0.014$ before alignment. Overall, our results highlight the potential of using UDA to adapt a radioisotope classifier trained on synthetic data for real-world deployment.

Lalor, Peter W.↗

Foundation models for atomistic simulation of chemistry and materials

Conventional computational methods for modeling chemical and materials systems are limited by system size and timescale, forcing a trade-off between quantum-mechanical accuracy and the sampling needed for realistic observables. Large language and vision foundation models — pre-trained on massive datasets using transformer architectures — have revolutionized many fields. It is thus interesting to ask whether a foundation model — subject to suitable data, parameter scaling and training — could enable learned simulations of chemistry and materials. Here, in this study, we review the field of machine-learned interatomic potentials (MLIPs) and posit that scaling up large and diverse chemical and materials datasets and highly expressive architectures using advanced training strategies should result in models that are: more efficient, transferable, robust to out-of-distribution scenarios, and easier to fine-tune to a variety of downstream physical observables than models trained from scratch on small datasets corresponding to specific, targeted atomistic simulation tasks. We provide specific criteria for creating such large-scale MLIP foundation models, coordinated strategies for their development, evaluation and deployment, and highlight potential emergent capabilities that could transform predictive simulations in chemistry and materials science and accelerate discovery across multiple technological domains.

Yuan, Eric C.-Y. [University of California, Berkel↗

Synthetic data generation for machine learning model training for energy theft scenarios using cosimulation

Abstract Technical and non‐technical losses in distribution circuits result in significant economic costs to power utilities. One type of non‐technical loss is energy theft by various means including illegal tapping of feeders, bypassing the meter, and billing fraud. These losses are usually hard to detect, and can remain undetected for long periods of time. Machine learning models have been proven effective in detecting these conditions, but rely on the availability of large, good‐quality training data sets. The problem is exacerbated by the imbalanced nature of data related to these conditions—energy theft, though costly, is very rare. The available data sets generally have very few samples of theft with most of the data pertaining to normal operation. Such data sets are generally not suitable to train machine learning models. In this paper, an overview of energy theft detection techniques, the challenges with their data needs, and the limitations of current techniques to bridge such data limitations is presented. A co‐simulation framework is proposed to generate reliable training data for machine learning algorithms for theft detection. An example scenario is presented and a machine learning model is built to detect certain kinds of energy theft.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Development of an Interactive Valuation Model for Distributed Energy Resource Value

The Oklahoma Secretary of Commerce & Tourism (OSCT) seeks to address perceived challenges and barriers to the deployment of distributed energy generation resources (DERs), including solar and other technologies, in the state of Oklahoma. The Scope of Work (SOW) outlines follow-on tasks related to the previous CRADA # CRD-18-762. The first task will finalize the Interactive Valuation Model Requirements Document, the final draft of which was provided in June 2018. The second task includes the building, validation, and finalization of the interactive valuation model for expressing distributed energy resource value based on user inputs to be completed by June 30, 2019. Training for how to prepare the data inputs and use the model will also be provided. To address perceived challenges and barriers to the deployment of distributed energy generation resources (DERs), including solar and other technologies, in the state of Oklahoma, the work under this CRADA will: 1) finalize the draft Interactive Valuation Model Requirements Document delivered in June 2018; 2) finish building and validating the interactive valuation model for expressing DER value based on user inputs, and 3) provide training for use of this model.

14 SOLAR ENERGY↗

Computed Tomography Radiomics Predicts HPV Status and Local Tumor Control After Definitive Radiochemotherapy in Head and Neck Squamous Cell Carcinoma

Purpose: This study aimed to predict local tumor control (LC) after radiochemotherapy of head and neck squamous cell carcinoma (HNSCC) and human papillomavirus (HPV) status using computed tomography (CT) radiomics. Methods and Materials: HNSCC patients treated with definitive radiochemotherapy were included in the retrospective study approved by the local ethical commission (93 and 56 patients in the training and validation cohorts, respectively). Three hundred seventeen CT radiomic features, including those based on shape, intensity, texture, and wavelet transform, were calculated in the primary tumor region. Cox and logistic regression models were built to predict LC and HPV status, respectively. The best-performing features in the univariable analysis were included in the multivariable analysis after the exclusion of redundant features. The quality of the models was assessed using the concordance index (CI) for modeling of LC and receiver operating characteristics area under the curve (AUC) for HPV status prediction. The radiomics LC model was compared to a model incorporating clinical parameters (tumor stage, volume, and HPV status) and a mixed model. Results: A radiomic signature comprising 3 features was significantly associated with LC (CI{sub training} = 0.75 and CI{sub validation} = 0.78), showing that tumors with a more heterogeneous CT density distribution are at risk for decreased LC. The addition of clinical parameters to the radiomics model slightly improved the model in the training cohort but not in the validation cohort. Another radiomic signature showed good performance in HPV status prediction (AUC{sub training} = 0.85 and AUC{sub validation} = 0.78) and indicated that HPV-positive tumors have a more homogenous CT density distribution. Conclusions: Heterogeneity of HNSCC tumor density, quantified by CT radiomics, is associated with LC after radiochemotherapy and HPV status.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

SU-E-J-244: Development and Validation of a Knowledge Based Planning Model for External Beam Radiation Therapy of Locally Advanced Non-Small Cell Lung Cancer

Purpose: The study aims to develop and validate a knowledge based planning (KBP) model for external beam radiation therapy of locally advanced non-small cell lung cancer (LA-NSCLC). Methods: RapidPlan™ technology was used to develop a lung KBP model. Plans from 65 patients with LA-NSCLC were used to train the model. 25 patients were treated with VMAT, and the other patients were treated with IMRT. Organs-at-risk (OARs) included right lung, left lung, heart, esophagus, and spinal cord. DVH and geometric distribution DVH were extracted from the treated plans. The model was trained using principal component analysis and step-wise multiple regression. Box plot and regression plot tools were used to identify geometric outliers and dosimetry outliers and help fine-tune the model. The validation was performed by (a) comparing predicted DVH boundaries to actual DVHs of 63 patients and (b) using an independent set of treatment planning data. Results: 63 out of 65 plans were included in the final KBP model with PTV volume ranging from 102.5cc to 1450.2cc. Total treatment dose prescription varied from 50Gy to 70Gy based on institutional guidelines. One patient was excluded due to geometric outlier where 2.18cc of spinal cord was included in PTV. The other patient was excluded due to dosimetric outlier where the dose sparing to spinal cord was heavily enforced in the clinical plan. Target volume, OAR volume, OAR overlap volume percentage to target, and OAR out-of-field volume were included in the trained model. Lungs and heart had two principal component scores of GEDVH, whereas spinal cord and esophagus had three in the final model. Predicted DVH band (mean ±1 standard deviation) represented 66.2±3.6% of all DVHs. Conclusion: A KBP model was developed and validated for radiotherapy of LA-NSCLC in a commercial treatment planning system. The clinical implementation may improve the consistency of IMRT/VMAT planning.

60 APPLIED LIFE SCIENCES↗

Extracting the gamma-ray source-count distribution below the Fermi-LAT detection limit with deep learning

We reconstruct the extra-galactic gamma-ray source-count distribution, or dN/dS, of resolved and unresolved sources by adopting machine learning techniques. Specifically, we train a convolutional neural network on synthetic 2-dimensional sky-maps, which are built by varying parameters of underlying source-counts models and incorporate the Fermi-LAT instrumental response functions. The trained neural network is then applied to the Fermi-LAT data, from which we estimate the source count distribution down to flux levels a factor of 50 below the Fermi-LAT threshold. We perform our analysis using 14 years of data collected in the (1,10) GeV energy range. The results we obtain show a source count distribution which, in the resolved regime, is in excellent agreement with the one derived from cataloged sources, and then extends as dN/dS ~ S -2 in the unresolved regime, down to fluxes of 5 · 10 -12 cm -2 s -1 . The neural network architecture and the devised methodology have the flexibility to enable future analyses to study the energy dependence of the source-count distribution.

79 ASTRONOMY AND ASTROPHYSICS↗

Effect of spatial sampling from European flux towers for estimating carbon and water fluxes with artificial neural networks

Abstract Empirical modeling approaches are frequently used to upscale local eddy covariance observations of carbon, water, and energy fluxes to regional and global scales. The predictive capacity of such models largely depends on the data used for parameterization and identification of input‐output relationships, while prediction for conditions outside the training domain is generally uncertain. In this work, artificial neural networks (ANNs) were used for the prediction of gross primary production (GPP) and latent heat flux (LE) on local and European scales with the aim to assess the portion of uncertainties in extrapolation due to sample selection. ANNs were found to be a useful tool for GPP and LE prediction, in particular for extrapolation in time (mean absolute error MAE for GPP between 0.53 and 1.56 gC m −2 d −1 ). Extrapolation in space in similar climatic and vegetation conditions also gave good results (GPP MAE 0.7–1.41 gC m −2 d −1 ), while extrapolation in areas with different seasonal cycles and controlling factors (e.g., the tropical regions) showed noticeably higher errors (GPP MAE 0.8–2.09 gC m −2 d −1 ). The distribution and the number of sites used for ANN training had a remarkable effect on prediction uncertainty in both, regional GPP and LE budgets and their interannual variability. Results obtained show that for ANN upscaling for continents with relatively small networks of sites, the error due to the sampling can be large and needs to be considered and quantified. The analysis of the spatial variability of the uncertainty helped to identify the meteorological drivers driving the uncertainty.

Papale, Dario↗

Trainability of ReLU networks and Data-dependent Initialization

Here, in this paper, we study the trainability of rectified linear unit (ReLU) networks at initialization. A ReLU neuron is said to be dead if it only outputs a constant for any input. Two death states of neurons are introduced–tentative and permanent death. A network is then said to be trainable if the number of permanently dead neurons is sufficiently small for a learning task. We refer to the probability of a randomly initialized network being trainable as trainability. We show that a network being trainable is a necessary condition for successful training, and the trainability serves as an upper bound of training success rates. In order to quantify the trainability, we study the probability distribution of the number of active neurons at initialization. In many applications, overspecified or overparameterized neural networks are successfully employed and shown to be trained effectively. With the notion of trainability, we show that overparameterization is both a necessary and a sufficient condition for achieving a zero training loss. Furthermore, we propose a data-dependent initialization method in an overparameterized setting. Numerical examples are provided to demonstrate the effectiveness of the method and our theoretical findings.

97 MATHEMATICS AND COMPUTING↗

Improving Solar and Solar+Storage Screening Techniques to Reduce Utility Interconnection Time and Costs (Final Technical Report)

Residential PV installations have increased rapidly over the last decade, and the increased application volume has caused permitting delays and lower overall adoption rates. In this project, we developed and evaluated whether data-driven secondary modeling and screening techniques can help utilities assess customer applications more accurately than traditional screening shortcuts. Secondary topologies are predicted using decision trees and commonly available information, such as service transformer, customer, and street locations. Conductors were predicted using a logistic regression method based on real world object (RWO) types, service transformer ratings, conductor length, and distance to transformer. After developing the combined primary and secondary distribution network model, hosting capacity results were used to train a random forest model to predict the pass/fail likelihood of a customer application. Powerflow based models with predicted secondaries and data-driven methods both increased the screening success rate, relative to common utility heuristics, by as much as 55 percentage points. Data-driven screening techniques were described by one utility as a "right-sized" approach for residential customers given the low-risk of small errors and the high-cost of accurate modeling.

14 SOLAR ENERGY↗

Generative adversarial networks for scintillation signal simulation in EXO-200

Generative Adversarial Networks trained on samples of simulated or actual events have been proposed as a way of generating large simulated datasets at a reduced computational cost. In this work, a novel approach to perform the simulation of photodetector signals from the time projection chamber of the EXO-200 experiment is demonstrated. The method is based on a Wasserstein Generative Adversarial Network — a deep learning technique allowing for implicit non-parametric estimation of the population distribution for a given set of objects. Our network is trained on real calibration data using raw scintillation waveforms as input. We find that it is able to produce high-quality simulated waveforms an order of magnitude faster than the traditional simulation approach and, importantly, generalize from the training sample and discern salient high-level features of the data. In particular, the network correctly deduces position dependency of scintillation light response in the detector and correctly recognizes dead photodetector channels. Furthermore, the network output is then integrated into the EXO-200 analysis framework to show that the standard EXO-200 reconstruction routine processes the simulated waveforms to produce energy distributions comparable to that of real waveforms. Finally, the remaining discrepancies and potential ways to improve the approach further are highlighted.

47 OTHER INSTRUMENTATION↗

Solving inverse problems in stochastic models using deep neural networks and adversarial training

Inverse problems associated with stochastic models constitute a significant portion of scientific and engineering applications. In such cases the unknown quantities are distributions. The applicability of traditional methods is limited because of their demanding assumptions or prohibitive computational consumption; for example, maximum likelihood methods require closed-form density functions, and Markov Chain Monte Carlo needs a large number of simulations. We propose a new method that estimates the unknown distribution by matching the statistical properties between observed and simulated random processes. We leverage the expressive power of neural networks to approximate the unknown distribution and use a discriminative neural network for computing the statistical discrepancies between the observed and simulated random processes. Here we demonstrated numerically that the proposed methods can estimate both the model parameters and learn complicated unknown distributions.

42 ENGINEERING↗

What does fault tolerant Deep Learning need from MPI?

Deep Learning (DL) algorithms have become the {\em de facto} Machine Learning (ML) algorithm for large scale data analysis. DL algorithms are computationally expensive -- even distributed DL implementations which use MPI require days of training (model learning) time on commonly studied datasets. Long running DL applications become susceptible to faults -- requiring development of a fault tolerant system infrastructure, in addition to fault tolerant DL algorithms. This raises an important question: {\em What is needed from MPI for designing fault tolerant DL implementations?} In this paper, we address this problem for permanent faults. We motivate the need for a fault tolerant MPI specification by an in-depth consideration of recent innovations in DL algorithms and their properties, which drive the need for specific fault tolerance features. We present an in-depth discussion on the suitability of different parallelism types (model, data and hybrid); a need (or lack thereof) for check-pointing of any critical data structures; and most importantly, consideration for several fault tolerance proposals (user-level fault mitigation (ULFM), Reinit) in MPI and their applicability to fault tolerant DL implementations. We leverage a distributed memory implementation of Caffe, currently available under the Machine Learning Toolkit for Extreme Scale (MaTEx). We implement our approaches by extending MaTEx-Caffe for using ULFM-based implementation. Our evaluation using the ImageNet dataset and AlexNet neural network topology demonstrates the effectiveness of the proposed fault tolerant DL implementation using OpenMPI based ULFM.

Amatya, Vinay C.↗

Voltage Calculations in Secondary Distribution Networks via Physics-Inspired Neural Network Using Smart Meter Data

The increasing penetration of distributed energy resources (DERs) leads to voltage issues across distribution networks, necessitating voltage calculations by utilities. Electric model-free voltage calculation offers an enticing solution. However, most researches mainly focus on primary distribution networks ignoring secondary distribution networks and commonly overlook extreme voltage case calculations, which require the model’s extrapolation abilities. Here, in addressing the gaps, this paper presents a customized physics-inspired neural network (PINN) model, the structure of which is inspired by the derived coupled power flow model of primary-secondary distribution networks. To ensure precision and rapid convergence, a crafted training framework for the PINN model is proposed. The PINN’s “structure-mimetic” design enables superior extrapolation for unseen scenarios and enhances physical information awareness. We demonstrate this through two applications: hosting capacity analysis and customer-transformer connectivity. The effectiveness and advantages of the proposed PINN model are validated on two public testing systems and one utility distribution feeder model.

Distribution network↗

Reinforcement Learning for Volt- Var Control: A Novel Two-stage Progressive Training Strategy

This paper develops a reinforcement learning (RL) approach to solve a cooperative, multi-agent Volt-Var Control (VVC) problem for high solar penetration distribution systems. The ingenuity of our RL method lies in a novel two-stage progressive training strategy that can effectively improve training speed and convergence of the machine learning algorithm. In Stage 1 (individual training), while holding all the other agents inactive, we separately train each agent to obtain its own optimal VVC actions in the action space: fconsume, generate, do-nothingg. In Stage 2 (cooperative training), all agents are trained again coordinatively to share VVC responsibility. Rewards and costs in our RL scheme include (i) a system-level reward (for taking an action), (ii) an agent-level reward (for doing-nothing), and (iii) an agent-level action cost function. This new framework allows rewards to be dynamically allocated to each agent based on their contribution while accounting for the trade-off between control effectiveness and action cost. The proposed methodology is tested and validated in a modified IEEE 123-bus system using realistic PV and load profiles. Simulation results confirm that the proposed approach is robust and computationally efficient; and it achieves desirable volt-var control performance under a wide range of operation conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Decentralized Voltage Control of Large-Scale Distribution System with PVs Based on MADRL

This paper proposes a model-free decentralized control framework for the voltage regulation of large-scale distribution systems through the coordinated control of PV inverters. This is achieved by developing a novel interaction mechanism between the surrogate model and the centralized training and decentralized execution multiagent deep reinforcement learning framework. Specifically, the sparse Gaussian processes regression method is first utilized to develop the surrogate model of the original distribution system for reward calculation during the training stage, where each agent represents a sub-region in the centralized fashion for coordination strategy learning. After that, the learned control rules are used to inform controllers within each sub-region for real-time decisions with only local measurements. Comparative tests among various methods on the EPRI Ckt5 test system demonstrate the effectiveness of the proposed method.

distribution system↗