Search NASA⌕ Search

SEARCH · Search NASA

Results for “Hyper-parameter Optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

22 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

167-PFlops Deep Learning for Electron Microscopy: From Learning Physics to Atomic Manipulation

An artificial intelligence system called MENNDL, which used 25,200 Nvidia Volta GPUs on Oak Ridge National Laboratory’s Summit machine, automatically designed an optimal deep learning network in order to extract structural information from raw atomic-resolution microscopy data. In a few hours, MENNDL creates and evaluates millions of networks using a scalable, parallel, asynchronous genetic algorithm augmented with a support vector machine to automatically find a superior deep learning network topology and hyper-parameter set than a human expert can find in months. For the application of electron microscopy, the system furthers the goal of improving our understanding of the electron-beam-matter interactions and real-time image-based feedback, which enables a huge step beyond human capacity toward nanofabricating materials automatically. Finally, MENNDL has been scaled to the 4,200 available nodes of Summit achieving a measured 152.5 PFlops, with an estimated sustained performance of 167 PFlops when the entire machine is available.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

167-PFlops deep learning for electron microscopy: from learning physics to atomic manipulation

An artificial intelligence system called MENNDL, which used 25,200 NVIDIA Volta GPUs on Oak Ridge National Laboratory's Summit machine, automatically designed an optimal deep learning network in order to extract structural information from raw atomic-resolution microscopy data. In a few hours, MENNDL creates and evaluates millions of networks using a scalable, parallel, asynchronous genetic algorithm augmented with a support vector machine to automatically find a superior deep learning network topology and hyper-parameter set than a human expert can find in months. For the application of electron microscopy, the system furthers the goal of improving our understanding of the electron-beam-matter interactions and real-time image-based feedback, which enables a huge step beyond human capacity towards nanofabricating materials automatically. MENNDL has been scaled to the 4,200 available nodes of Summit achieving a measured 152.5 PFlops, with an estimated sustained performance of 167 PFlops when the entire machine is available.

Patton, Robert↗
DOE OSTI2020

FLEET: Flexible Efficient Ensemble Training for Heterogeneous Deep Neural Networks

Parallel training of an ensemble of Deep Neural Networks (DNN) on a cluster of nodes is an effective approach to shorten the process of neural network architecture search and hyper-parameter tuning for a given learning task. Prior efforts have shown that data sharing, where the common preprocessing operation is shared across the DNN training pipelines, saves computational resources and improves pipeline efficiency. Data sharing strategy, however, performs poorly for a heterogeneous set of DNNs where each DNN has varying computational needs and thus different training rate and convergence speed. This paper proposes FLEET, a flexible ensemble DNN training framework for efficiently training a heterogeneous set of DNNs. We build FLEET via several technical innovations. We theoretically prove that an optimal resource allocation is NP-hard and propose a greedy algorithm to efficiently allocate resources for training each DNN with data sharing. We integrate data-parallel DNN training into ensemble training to mitigate the differences in training rates and introduce checkpointing into this context to address the issue of different convergence speeds. Experiments show that FLEET significantly improves the training efficiency of DNN ensembles without compromising the quality of the result.

Guan, Hui↗
DOE OSTI2020

Optimizing Simulation Parameters for Weak Lensing Analyses Involving Non-Gaussian Observables

We performed a series of numerical experiments to quantify the sensitivity of the predictions for weak lensing statistics obtained in ray-tracing dark matter (DM)-only simulations, to two hyper-parameters that influence the accuracy as well as the computational cost of the predictions: the thickness of the lens planes used to build past light cones and the mass resolution of the underlying DM simulation. The statistics considered are the power spectrum (PS) and a series of non-Gaussian observables, including the one-point probability density function, lensing peaks, and Minkowski functionals. Counterintuitively, we find that using thin lens planes (< 60 h {sup −1} Mpc on a 240 h {sup −1} Mpc simulation box) suppresses the PS over a broad range of scales beyond what would be acceptable for a survey comparable to the Large Synoptic Survey Telescope (LSST). A mass resolution of 7.2 × 10{sup 11} h {sup −1} M {sub ⊙} per DM particle (or 256{sup 3} particles in a (240 h {sup −1} Mpc){sup 3} box) is sufficient to extract information using the PS and non-Gaussian statistics from weak lensing data at angular scales down to 1′ with LSST-like levels of shape noise.

79 ASTRONOMY AND ASTROPHYSICS↗