Results for “TRAINING - UNIVERSITIES” — Page 6

DOE OSTI2021

Plateau Phenomenon in Gradient Descent Training of RELU Networks: Explanation, Quantification, and Avoidance

The ability of neural networks to provide ‘best in class’ approximation across a wide range of applications is well-documented. Nevertheless, the powerful expressivity of neural networks comes to naught if one is unable to effectively train (choose) the parameters defining the network. In general, neural networks are trained by gradient descent type optimization methods,a stochastic variant thereof. In practice, such methods result in the loss function decreases rapidly at the beginning of training but then, after a relatively small number of steps, significantly slow down. The loss may even appear to stagnate over the period of a large number of epochs, only to then suddenly start to decrease fast again for no apparent reason. This so-called plateau phenomenon manifests itself in many learning tasks. The present work aims to identify and quantify the root causes of plateau phenomenon.analysis is carried out in the setting of univariate ReLU networks. No assumptions are made on the number of neurons relative to the number of training data, and our results hold for both the lazy and adaptive regimes. Here, the main findings are: plateaux correspond to periods during which activation patterns remain constant, where activation pattern refers to the number of data points that activate a given neuron; quantification of convergence of the gradient flow dynamics; and, characterization stationary points in terms solutions of local least squares regression lines over subsets of the training data. Based on these conclusions, we propose a new iterative training method, the Active Neuron Least Squares (ANLS), characterised by the explicit adjustment of the activation pattern at each step, which is designed to enable a quick exit from a plateau. Illustrative numerical examples are included throughout.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Using GANs with adaptive training data to search for new molecules

The process of drug discovery involves a search over the space of all possible chemical compounds. Generative Adversarial Networks (GANs) provide a valuable tool towards exploring chemical space and optimizing known compounds for a desired functionality. Standard approaches to training GANs, however, can result in mode collapse, in which the generator primarily produces samples closely related to a small subset of the training data. In contrast, the search for novel compounds necessitates exploration beyond the original data. In this work, we present an approach to training GANs that promotes incremental exploration and limits the impacts of mode collapse using concepts from Genetic Algorithms. In our approach, valid samples from the generator are used to replace samples from the training data. We consider both random and guided selection along with recombination during replacement. By tracking the number of novel compounds produced during training, we show that updates to the training data drastically outperform the traditional approach, increasing potential applications for GANs in drug discovery.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

Abstract Graph Convolutional Neural Network (GCNN) is a popular class of deep learning (DL) models in material science to predict material properties from the graph representation of molecular structures. Training an accurate and comprehensive GCNN surrogate for molecular design requires large-scale graph datasets and is usually a time-consuming process. Recent advances in GPUs and distributed computing open a path to reduce the computational cost for GCNN training effectively. However, efficient utilization of high performance computing (HPC) resources for training requires simultaneously optimizing large-scale data management and scalable stochastic batched optimization techniques. In this work, we focus on building GCNN models on HPC systems to predict material properties of millions of molecules. We use HydraGNN, our in-house library for large-scale GCNN training, leveraging distributed data parallelism in PyTorch. We use ADIOS, a high-performance data management framework for efficient storage and reading of large molecular graph data. We perform parallel training on two open-source large-scale graph datasets to build a GCNN predictor for an important quantum property known as the HOMO-LUMO gap. We measure the scalability, accuracy, and convergence of our approach on two DOE supercomputers: the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF) and the Perlmutter system at the National Energy Research Scientific Computing Center (NERSC). We present our experimental results with HydraGNN showing (i) reduction of data loading time up to 4.2 times compared with a conventional method and (ii) linear scaling performance for training up to 1024 GPUs on both Summit and Perlmutter.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

Training dataset and results for geothermal exploration artificial intelligence, applied to Brady Hot Springs and Desert Peak

The submission includes the labeled datasets, as ESRI Grid files (.gri, .grd) used for training and classification results for our machine leaning model: - brady_som_output.gri, brady_som_output.grd, brady_som_output.* - desert_som_output.gri, desert_som_output.grd, desert_som_output.* The data corresponds to two sites: Brady Hot Springs and Desert Peak, both located near Fallon, NV. Input layers include: - Geothermal: Labeled data (0: Non-geothermal; 1: Geothermal) - Minerals: Hydrothermal mineral alterations, as a result of spectral analysis using Chalcedony, Kaolinite, Gypsum, Hematite and Epsomite - Temperature: Land surface temperature (% of times a pixel was classified as "Hot" by K-Means) - Faults: Fault density with a 300mradius - Subsidence: PSInSAR results showing subsidence displacement of more than 5mm - Uplift: PSInSAR results showing subsidence displacement of more than 5mm Also, the results of the classification using Brady and Desert Peak to build 2 Convolutional Neural Networks. These were applied to the training site as well as the other site, the results are in GeoTiff format. - brady_classification: Results of classification of the Brady-trained model - desert_classification: Results of classification of the Desert Peak-trained model - b2d_classification: Results of classification of Desert Peak using the Brady-trained model - d2b_classification: Results of classification of Brady using the Desert Peak-trained model

15 GEOTHERMAL ENERGY↗

DOE OSTI2025

BOPTest As a Platform for Building Controls and Grid-Interactive Buildings Workforce Training

Building automation and controls are becoming increasingly complex with the emergence of Grid Integrated Efficient Buildings (GEBs) as well as new highly efficient sequences of operation and data-driven control schemes. However, there remains a significant gap in hands-on training opportunities for building operators and technicians to gain practical experience with advanced control systems in a low-risk environment. This paper presents BOPTEST (Building Optimization Performance Test) as a suitable platform for workforce training in building controls and GEB technologies. BOPTEST provides a suite of standardized building simulation test cases with a REST API, real-time control interfaces through BACnet, semantic models connecting users to building data, and built-in calculation of control metrics and performance indicators. The platform enables trainees to interact with virtual buildings using industry-standard protocols while learning how to implement and innovate control strategies. The training platform is designed to offer a structured and interactive learning experience for building engineers, helping them effectively develop, learn, and retain skills in fault identification, troubleshooting, and correction. The workflow is divided into three main phases: 1) Setup, 2) Exercise, and 3) Review, each comprising specific activities performed by either the instructor or the student. Initial pilot training sessions have yielded positive feedback from instructors and participants and demonstrates that BOPTEST effectively fills an industry need for a low-risk training resource via simulation of real building control systems, allowing trainees to gain practical experience before working in the field. The platform's ability to provide immediate performance feedback while maintaining familiar industry interfaces makes it particularly suitable for workforce development programs. This work provides a replicable model for leveraging building simulation in control education and training.

Paul, Lazlo↗

DOE OSTI2021

In situ Parallel Training of Analog Neural Network Using Electrochemical Random-Access Memory

In-memory computing based on non-volatile resistive memory can significantly improve the energy efficiency of artificial neural networks. However, accurate in situ training has been challenging due to the nonlinear and stochastic switching of the resistive memory elements. One promising analog memory is the electrochemical random-access memory (ECRAM), also known as the redox transistor. Its low write currents and linear switching properties across hundreds of analog states enable accurate and massively parallel updates of a full crossbar array, which yield rapid and energy-efficient training. While simulations predict that ECRAM based neural networks achieve high training accuracy at significantly higher energy efficiency than digital implementations, these predictions have not been experimentally achieved. In this work, we train a 3 × 3 array of ECRAM devices that learns to discriminate several elementary logic gates (AND, OR, NAND). We record the evolution of the network’s synaptic weights during parallel in situ (on-line) training, with outer product updates. Due to linear and reproducible device switching characteristics, our crossbar simulations not only accurately simulate the epochs to convergence, but also quantitatively capture the evolution of weights in individual devices. The implementation of the first in situ parallel training together with strong agreement with simulation results provides a significant advance toward developing ECRAM into larger crossbar arrays for artificial neural network accelerators, which could enable orders of magnitude improvements in energy efficiency of deep neural networks.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks

The Deep Operator Network (DeepONet) framework is a different class of neural network architecture that one trains to learn nonlinear operators, i.e., mappings between infinite-dimensional spaces. Traditionally, DeepONets are trained using a centralized strategy that requires transferring the training data to a centralized location. Such a strategy, however, limits our ability to secure data privacy or use high-performance distributed/parallel computing platforms. To alleviate such limitations, in this paper, we study the federated training of DeepONets for the first time. That is, we develop a framework, which we refer to as Fed-DeepONet, that allows multiple clients to train DeepONets collaboratively under the coordination of a centralized server. To achieve Fed-DeepONets, we propose an efficient stochastic gradient-based algorithm that enables the distributed optimization of the DeepONet parameters by averaging first-order estimates of the DeepONet loss gradient. Then, to accelerate the training convergence of Fed-DeepONets, we propose a moment-enhanced (i.e., adaptive) stochastic gradient-based strategy. Finally, we verify the performance of Fed-DeepONet by learning, for different configurations of the number of clients and fractions of available clients, (i) the solution operator of a gravity pendulum and (ii) the dynamic response of a parametric library of pendulums.

Moya, Christian↗

DOE OSTI2022

Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer

The High Throughput Truthing project aims to develop a dataset for validating artificial intelligence and machine learning models (AI/ML) fit for regulatory purposes. The context of this AI/ML validation dataset is the reporting of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer biopsy specimens. After completing the pilot study, we found notable variability in the sTILs estimates as well as inconsistencies and gaps in the provided training to pathologists. Using the pilot study data and an expert panel, we created custom training materials to improve pathologist annotation quality for the pivotal study. We categorized regions of interest (ROIs) based on their mean sTILs density and selected ROIs with the highest and lowest sTILs variability. In a series of eight one-hour sessions, the expert panel reviewed each ROI and provided verbal density estimates and comments on features that confounded the sTILs evaluation. We aggregated and shaped the comments to identify pitfalls and instructions to improve our training materials. From these selected ROIs, we created a training set and proficiency test set to improve pathologist training with the goal to improve data collection for the pivotal study. We are not exploring AI/ML performance in this paper. Instead, we are creating materials that will train crowd-sourced pathologists to be the reference standard in a pivotal study to create an AI/ML model validation dataset. The issues discussed here are also important for clinicians to understand about the evaluation of sTILs in clinical practice and can provide insight to developers of AI/ML models.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2025

BOPTEST as a Platform for Building Controls and Grid-Interactive Buildings Workforce Training

Building automation and controls are becoming increasingly complex with the emergence of Grid Integrated Efficient Buildings (GEBs) as well as new highly efficient sequences of operation and data-driven control schemes. However, there remains a significant gap in hands-on training opportunities for building operators and technicians to gain practical experience with advanced control systems in a low-risk environment. This paper presents BOPTEST (Building Optimization Performance Test) as a suitable platform for workforce training in building controls and GEB technologies. BOPTEST provides a suite of standardized building simulation test cases with a REST API, real-time control interfaces through BACnet, semantic models connecting users to building data, and built-in calculation of control metrics and performance indicators. The platform enables trainees to interact with virtual buildings using industry-standard protocols while learning how to implement and innovate control strategies. The training platform is designed to offer a structured and interactive learning experience for building engineers, helping them effectively develop, learn, and retain skills in fault identification, troubleshooting, and correction. The workflow is divided into three main phases: 1) Setup, 2) Exercise, and 3) Review, each comprising specific activities performed by either the instructor or the student. Initial pilot training sessions have yielded positive feedback from instructors and participants and demonstrates that BOPTEST effectively fills an industry need for a low-risk training resource via simulation of real building control systems, allowing trainees to gain practical experience before working in the field. The platform's ability to provide immediate performance feedback while maintaining familiar industry interfaces makes it particularly suitable for workforce development programs. This work provides a replicable model for leveraging building simulation in control education and training.

Paul, Lazlo↗

DOE OSTI2023

Selection of Training Sets for 235U Source Detection Classifiers Using Gamma Signatures

The machine learning methods for classifiers to detect low level radiation sources are of interest when suitable training data sets are available. Their application and performance assessment, however, involves the aspects of over-fitting and training data selection that are somewhat uncommon in other existing methods for this task.We study U-235 gamma signatures using data sets collected by 21 NaI detectors under controlled conditions. The gamma spectra are collected by the detectors located at different distances from the source, and we study their choice as training sets for classifiers to detect a source. The detectors form the near, middle and outer groups based on the distance to source. The classifiers based on the outer group are susceptible to over-fitting, that is, they achieve low training error but incur much higher testing error in independent tests. The other two groups achieve lower training error and comparable testing error, and the near group achieves the overall lowest error. In detecting a source at an unknown distance, the farther detectors in the middle group achieve the overall lowest testing error with limited over-fitting, thereby indicating the complex dependencies between the training and classifier performance.

Rao, Nageswara↗

DOE OSTI2025

GSplit: Scaling Graph Neural Network Training on Large Graphs via Split-Parallelism

Graph neural networks (GNNs), an emerging class of machine learning models for graphs, have gained popularity for their superior performance in various graph analytical tasks. Mini-batch training is commonly used to train GNNs on large graphs, and data parallelism is the standard approach to scale mini-batch training across multiple GPUs. Data parallel approaches contain redundant work as subgraphs sampled by different GPUs contain significant overlap. To address this issue, we introduce a hybrid parallel mini-batch training paradigm called Split parallelism. Split parallelism avoids redundant work by splitting the sampling, loading, and training of each mini-batch across multiple GPUs. Split parallelism, however, introduces communication overheads that can be more than the savings from removing redundant work. We further present a lightweight partitioning algorithm that probabilistically minimizes these overheads. We implement spllit parllelism in GSplit and show that it outperforms state-of-the-art mini-batch training systems like DGL, Quiver, and P3.

Lim, Seung-Hwan [ORNL] (ORCID:0000000194616866)↗

DOE OSTI2022

A Subcritical Assembly for Training and Education Use at the Oak Ridge National Laboratory [Abstract]

A near-final design has been completed for a new subcritical assembly at Oak Ridge National Laboratory (ORNL). This subcritical assembly will support the Nuclear Criticality Safety Program (NCSP) training and education program to augment current subcritical assembly capabilities at Sandia National Laboratories (SNL) and at the National Criticality Experiments Research Center (NCERC). Other collaborations have been proposed with the Sunrise Consortium and others. This proposed subcritical assembly uses legacy AGN-201M research reactor fuel plates that are available from the Y-12 National Security Complex. This subcritical assembly contains approximately 620 grams of 235 U as UO 2 particulates distributed homogeneously in radiation-stabilized polyethylene. The fuel plates will have a graphite neutron reflector to obtain a core multiplication, M, from 10 to 50, corresponding to a k eff of 0.9 to 0.95, respectively. The subcritical assembly will be able to support at least four experiments for the training courses: (1) the addition of fissile material to increase neutron multiplication (mass), (2) a core separation experiment (interaction), (3) the effect of adding moderators to the core (moderation), and (4) the effect of adding neutron absorbers to the core (poison/absorption). The proposed ORNL subcritical assembly will provide additional student bandwidth to support NCSP training courses and to provide for some backup capacity if nuclear facility operations are disrupted at SNL or NCERC due to some unforeseen issue. The subcritical assembly will be designed to be an inherently safe subcritical assembly—subcritical under all normal and abnormal conditions—and it will provide the capability to conduct hands-on training to support NCSP and general nuclear criticality safety (NCS) staff training and qualification goals.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2023

In Silico Versus In Situ: The Challenging Landscape of Nuclear Criticality Safety Training

Nuclear criticality safety grew out of the ranks of experimentalists studying the physics of chain-reacting systems at critical experiment facilities. Consequently, critical experiment facilities provide the best forum for conducting training in nuclear criticality safety. As the nuclear renaissance gains traction, there is an increased demand to train personnel in nuclear criticality safety (NCS). Strides have been made in both on-line and virtual reality based NCS training. While these are perhaps a necessary component of NCS training, hands-on training at critical experiments facilities remains the most effective means of developing competency for fissionable material handlers, managers of fissionable material operations, criticality safety analysts, and experimentalists. This paper explores the challenges of developing and maintaining workforce competency in nuclear criticality safety to support safe and efficient fissionable material operations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2024

Investigation of candidates for reactor produced radioactive materials in support of radiological training exercises

Bromine-82, Potassium-42 and Copper-64 have been successfully adopted as radioactive surrogates for outdoor large area contamination training. The goal of this project was to discover new materials that could supplement potassium bromide (KBr) and copper pellets in radiological dispersal device (RDD) training events to reduce the down time of the training fields and to broaden the toolbox of the RDD surrogate training event program at Idaho National Laboratory. Of the ten different materials investigated, sodium nitrite, gallium metal, and gallium oxide presented the greatest promise as potential materials to replace potassium bromide in RDD training events.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

DOE OSTI2021

Cybersecurity Training Curriculum Analysis through the application of Machine Learning Text Classification and Natural Language Processing

In the domain of cybersecurity, there are a wide variety of education and training courses offered by a variety of training providers (commercial vendors, governmental, and academic institutions). Unfortunately, a comprehensive cross-provider mapping of the body of course offerings does not currently exist. Furthermore, although individual providers have made efforts, an accepted cross-provider mapping of education and training courses (or categories) to an accepted framework of work-roles by cybersecurity work-role and competency level has not yet been developed. The inability to compare training courses by topic or cybersecurity work-role and the level of difficulty (competency level) affects all organizations in identifying and selecting potential training opportunities for their personnel performing cybersecurity duties. In this work, we report on our application of Machine Learning (ML) Text Classification and Natural Language Processing (NLP) methodologies to begin to create a process to organize an accurate and thorough catalog of course offerings by work-role and competency level. Through the analysis of text-based course attributes (description, learning objectives, prerequisites, etc.) along with the text-based attributes for identified cybersecurity work-roles, our ML efforts examined methods to determine the work-role "best-fit" for each course as well as the competency level through action-verb text comparison.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2021

3D Virtual Simulation for Radiation Safety and Survey Training

3D virtual technologies have been widely used in remote training. Training integrated with 3D visualization technologies can enhance students’ engagement and reduce cost. The Applied Visualization Lab collaborates with College of Eastern Idaho on creating a 3D desktop application that simulate a pipe environment for radiation safety and survey training. Students can learn to perform radiation and contamination surveys remotely on their desktop. This simulation provides random scenarios, guided instructions, user interactions, and visual and sound feedback. It will promote utilizing virtual training for education outreach and minimize radiation and contamination exposure during the training.

3D↗

DOE OSTI2022

A Subcritical Assembly for Training and Education Use at the Oak Ridge National Laboratory

A near-final design has been completed for a new subcritical assembly at Oak Ridge National Laboratory (ORNL). This subcritical assembly will support the Nuclear Criticality Safety Program (NCSP) training and education program to augment current subcritical assembly capabilities at Sandia National Laboratories (SNL) and at the National Criticality Experiments Research Center (NCERC). Other collaborations have been proposed with the Sunrise Consortium and others. This proposed subcritical assembly uses legacy AGN-201M research reactor fuel plates that are available from the Y-12 National Security Complex. This subcritical assembly contains approximately 620 grams of 235 U as UO 2 particulates distributed homogeneously in radiation-stabilized polyethylene. The fuel plates will have a graphite neutron reflector to obtain a core multiplication, M, from 10 to 50, corresponding to a k eff of 0.9 to 0.95, respectively. The subcritical assembly will be able to support at least four experiments for the training courses: (1) the addition of fissile material to increase neutron multiplication (mass), (2) a core separation experiment (interaction), (3) the effect of adding moderators to the core (moderation), and (4) the effect of adding neutron absorbers to the core (poison/absorption). The proposed ORNL subcritical assembly will provide additional student bandwidth to support NCSP training courses and to provide for some backup capacity if nuclear facility operations are disrupted at SNL or NCERC due to some unforeseen issue. The subcritical assembly will be designed to be an inherently safe subcritical assembly—subcritical under all normal and abnormal conditions—and it will provide the capability to conduct hands-on training to support NCSP and general nuclear criticality safety (NCS) staff training and qualification goals.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Offshore Wind Workforce Safety Standards and Training Resource

The standardization of safety training requirements for people working at sea to build and operate offshore wind energy projects, adoption of those standards, and alignment with offshore safety training curriculum development was identified as a high-priority workforce gap in the U.S. Offshore Wind Workforce Assessment (Stefek, 2022). Safety is a top priority for the offshore wind industry; however, there has been uncertainty surrounding what unique safety standards and training will be needed and how various offshore wind safety stakeholders will be involved. The scope of safety training for this resource includes workers who build and operate offshore wind energy projects at sea and defines the stakeholders in the complex ecosystem. This fact sheet aims to help reduce these uncertainties by: 1) Delineating jurisdictional lines, while clarifying roles and responsibilities amongst stakeholders; 2) Identifying key actions or partnerships that can encourage further success with safety training and workforce development.

offshore wind↗