Search NASA⌕ Search

SEARCH · Search NASA

Results for “batch active learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

23 records · Page 2

Feature learning and generalization in deep networks with orthogonal weights

Fully-connected deep neural networks with weights initialized from independent Gaussian distributions can be tuned to criticality, which prevents the exponential growth or decay of signals propagating through the network. However, such networks still exhibit fluctuations that grow linearly with the depth of the network, which may impair the training of networks with width comparable to depth. We show analytically that rectangular networks with tanh activations and weights initialized from the ensemble of orthogonal matrices have corresponding preactivation fluctuations which are independent of depth, to leading order in inverse width. Moreover, we demonstrate numerically that, at initialization, all correlators involving the neural tangent kernel (NTK) and its descendants at leading order in inverse width—which govern the evolution of observables during training—saturate at a depth of ~20, rather than growing without bound as in the case of Gaussian initializations. We speculate that this structure preserves finite-width feature learning while reducing overall noise, thus improving both generalization and training speed in deep networks with depth comparable to width. We provide some experimental justification by relating empirical measurements of the NTK to the superior performance of deep non-linear orthogonal networks trained under full-batch gradient descent on the MNIST and CIFAR-10 classification tasks.

97 MATHEMATICS AND COMPUTING↗

Cholesky-based experimental design for Gaussian process and kernel-based emulation and calibration.

Gaussian processes and other kernel-based methods are used extensively to construct approximations of multivariate data sets. The accuracy of these approximations is dependent on the data used. This paper presents a computationally efficient algorithm to greedily select training samples that minimize the weighted L p error of kernel-based approximations for a given number of data. The method successively generates nested samples, with the goal of minimizing the error in high probability regions of densities specified by users. The algorithm presented is extremely simple and can be implemented using existing pivoted Cholesky factorization methods. Training samples are generated in batches which allows training data to be evaluated (labeled) in parallel. For smooth kernels, the algorithm performs comparably with the greedy integrated variance design but has significantly lower complexity. Numerical experiments demonstrate the efficacy of the approach for bounded, unbounded, multi-modal and non-tensor product densities. We also show how to use the proposed algorithm to efficiently generate surrogates for inferring unknown model parameters from data using Bayesian inference.

97 MATHEMATICS AND COMPUTING↗

Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units

Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs such as social networks. We demonstrate a novel hardware-software co-design approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations including a planner and vectorization for the gather-scatter operations targeted for Graphcore’s Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed co-design approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes and sparsity. Here, we demonstrate that such a co-design approach can reduce the training time of atomistic GNNs and can improve the performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Aluminum Metal Dissolution for Mark-18A Target Processing

The Mark-18A Program was established to preserve the unique materials found in Mark-18A targets for future research and development activity within the Department of Energy or other U.S. government agencies. The Savannah River Site has 65 Mark-18A targets to be provided for the recovery of high valued isotopes. The majority of the recovery work will be performed in the Savannah River National Laboratory (SRNL) Shielded Cells due to dose concerns. The recovered product will be transported to the Oak Ridge National Laboratory (ORNL) for future research and development tasks. The Mark-18A target material will be removed from their current configuration, dissolved, chemically separated, and calcined to a stable oxide. The baseline flowsheet is patterned after the flowsheet ORNL has previously used to process both Mark-18A and Mark-42 targets. However, it is necessary for SRNL to adapt its facility and the ORNL flowsheet to identify process improvements to successfully recover the actinide materials within the constraints of the SRNL Shielded Cells. These studies evaluated and demonstrated a chemical method for cladding removal using a mixture of sodium nitrate (NaNO 3 ) and sodium hydroxide (NaOH). Experiments provided better insights into dissolution of the Mark-18A aluminum cladding. In particular, these studies measured the effects of temperature on dissolution rate, flammable gas generation, and nitrite formation. Using the lessons learned from batch dissolution testing at SRNL, a continuous dissolution flowsheet was successfully demonstrated using prototypic-shaped materials and chemically-conservative materials. The SRNL target dissolution flowsheet departs from the baseline ORNL flowsheet in small but significant ways as discussed in the following paragraphs. These flowsheet changes facilitate faster initial reaction rates at a given temperature, enable cladding removal at a lower process temperature, and yield a more uniform dissolution rate throughout cladding removal. Multiple tests indicate that the total time at the dissolution temperatures will be about six hours for one fourth of a target. The shorter dissolution time provides the benefit of a single-shift dissolver operation. Off gas sampling determined that the quantities of flammable hydrogen (H 2 ) and ammonia (NH 3 ) expected in the process off gas will meet allowable flammable gas generation rates for the Shielded Cells Facility. The SRNL approach enables NaOH to be added at a constant rate, which simplifies operations in the shielded cells. In all six continuous dissolution tests, the process was easy to control with minimal operator interface. Analyses of process solutions throughout the experiments provide a better understanding of system behavior. The data show consistent behavior across several similar but different test conditions. Between the batch-test and continuous-test data, if future changes to the flowsheet are required, sufficient data exist to adequately predict how the system will respond to those changes. Future work will focus on design, fabrication, and evaluation of a system for dissolution of one-fourth of a Mark-18A target, which will be full-scale operations for SRNL. Based on these results, the process parameters are adequately understood and the system can be operated safely.

36 MATERIALS SCIENCE↗

Importance of Depth and Artificial Structure as Predictors of Female Red Snapper Reproductive Parameters

Abstract The Red Snapper Lutjanus campechanus is a structure‐associated species occurring across a wide depth range in the northern Gulf of Mexico. We used the random forest machine learning algorithm to understand which habitat and individual fish characteristics could predict reproductive parameters of female Red Snapper. We evaluated fish captured from 2016 to 2018 on three artificial structure types with various structure heights at depths of 100 m or less. Overall, we found that depth and month were important predictors for most reproductive parameters, but the type of structure (artificial reefs, oil platforms, and rigs‐to‐reefs structures) was not important. Maturity was correctly classified in 88.9% of the cases when using the random forest ensemble model, with important predictors including FL, depth, structure height, and month of collection. Spawning seasonality (measured as gonadosomatic index [GSI]) was correctly classified in 59.5% of the cases when using histology reproductive phase, FL, month, and depth variables. Reproductively active or inactive females were correctly classified in 89.3% of the cases using GSI, month, FL, and depth, while females in the developing versus spawning capable phases were correctly classified in 82.2% of the cases using GSI, FL, month, and depth. Histological indicators that show potential spawning within a 36‐h period were correctly classified 61.5% of the time, with the best predictors being depth, FL, GSI, and month. Stepwise regression indicated that month was the only factor that significantly predicted contrasts in relative batch fecundity, with significantly greater values in August compared to all other months. Our findings suggest that female Red Snapper reproductive effort is not consistently or well predicted by artificial structure type or height but that a combination of fish FL, month, and depth can predict reproductive characteristics of female Red Snapper.

Brown‐Peterson, Nancy J.↗