Search NASA⌕ Search

SEARCH · Search NASA

Results for “layer-parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

16 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration

TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.

Layer-parallel↗
DOE OSTI2020

Layer-Parallel Training of Deep Residual Neural Networks

Residual neural networks (ResNets) are a promising class of deep neural networks that have shown excellent performance for a number of learning tasks, e.g., image classification and recognition. Mathematically, ResNet architectures can be interpreted as forward Euler discretizations of a nonlinear initial value problem whose time-dependent control variables represent the weights of the neural network. Hence, training a ResNet can be cast as an optimal control problem of the associated dynamical system. For similar time-dependent optimal control problems arising in engineering applications, parallel-in-time methods have shown notable improvements in scalability. This paper demonstrates the use of those techniques for efficient and effective training of ResNets. The proposed algorithms replace the classical (sequential) forward and backward propagation through the network layers with a parallel nonlinear multigrid iteration applied to the layer domain. This adds a new dimension of parallelism across layers that is attractive when training very deep networks. From this basic idea, we derive multiple layer-parallel methods. The most efficient version employs a simultaneous optimization approach where updates to the network parameters are based on inexact gradient information in order to speed up the training process. Finally, using numerical examples from supervised classification, we demonstrate that the new approach achieves a training performance similar to that of traditional methods, but enables layer-parallelism and thus provides speedup over layer-serial methods through greater concurrency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Kink mechanism in Cu/Nb nanolaminates explored by $\mathcal{in}$ $\mathcal{situ}$ pillar compression

We report Nano metallic laminates (NMLs) exhibit different failure modes depending on the loading conditions due to their mechanical anisotropies. Kinking is a typical failure mode in many NMLs compressed along a layer-parallel direction. However, a detailed description of the microstructure evolution during kink band (KB) formation and an in-depth understanding of the formation mechanisms are lacking. In this work, the KB process is investigated in Cu/Nb NMLs by in situ micro pillar compression in the scanning electron microscope (SEM) along a layer-parallel direction. Post-mortem S/TEM and transmission Kikuchi diffraction (TKD) analyses show that kink banding leads to significant microstructure changes characterized by an accumulation of geometrically necessary dislocations (GNDs) and of tilt geometrically necessary boundaries (GNBs) near KB boundaries (KBBs). The distinct microstructure evolution implies that KB formation is facilitated by the inhomogeneous microstructures resulting in constrained deformation modes. Specifically, dislocations active on slip planes nearly parallel to the interfaces make a major contribution to kink evolution after the onset of kinking. Once layer-parallel slip systems are activated, preexisting lattice dislocations and dislocations nucleating from interfaces will accumulate as GNDs near KBBs via the stochastic storage of lattice dislocations that have certain Burgers vectors. GNDs can further transform into GNBs via cross-slip and climb driven processes near the KBB. Furthermore, GNBs near KBBs can grow by incorporating more GNDs or by coalescence to accommodate the KB evolution. We further hypothesize that microstructural perturbations and their ensuing stresses can initiate KB formation in Cu/Nb NMLs.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Non-local large-strain FFT-based formulation and its application to interface-dominated plasticity of nano-metallic laminates

Here, this paper presents a novel formulation and its robust numerical implementation of strain-gradient (SG) crystal plasticity within a large-strain (LS) elasto-viscoplastic (EVP) fast Fourier transform (FFT)-based micromechanical model. The resulting non-local SG-LS-EVPFFT formulation is used to model and understand the process of kink band formation during layer-parallel compression of nano-metallic laminates (NMLs). NMLs are layered composites with nanoscale thicknesses, thus requiring consideration of the interaction between dislocations and interfaces within the micromechanical model. The length-scale parameter of the SG model is calibrated by simulating a double pile-up and comparing predictions to analytical solution. This required new expressions for the defect energy, resulting in more accurate double pile-up predictions. The calibrated SG-LS-EVPFFT model is then used to simulate layer-parallel compression of copper-niobium NML. Formation of kink bands is predicted, and the model is used to rationalize the microscopic mechanisms enabling the formation process. It is found that accumulation of dislocations at interfaces leads to activation of layer-parallel slip, which in turn leads to kink band formation.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Unraveling kinking: A plasticity enhancing failure mode in high strength nano metallic laminates

Kinking is an important and plasticity-enhancing deformation/failure mode in numerous mechanically anisotropic materials including high-strength nano metallic laminates (NMLs). However, our current limited understanding of the mechanics of kinking and its dependence on microstructural attributes is insufficient for thoroughly comprehending and eventually being able to control failure behaviors of materials. In this study, we investigate kinking dependencies on microstructural attributes in NMLs via in situ micropillar compression, multiscale microstructure characterization, dislocation dynamic simulations, and crystal plasticity modeling. Additionally, by examining several NML systems (Cu/Fe, Ag/Fe, Al-4Mg/Fe), we demonstrate that the development of internal stresses during loading activates local layer-parallel glide triggering kinking in NMLs. Furthermore, this work reveals the effect of key microstructural features including layer thickness, layer waviness, interface barrier strength, and work hardening capacity on kink band formation in NMLs. More broadly, our efforts represent a generically applicable approach for probing large-strain deformation behavior of complex materials via synergetic modeling and experimental efforts.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Generalized grain boundary constitutive description implemented in a strain-gradient large-strain FFT-based formulation: Application to nano-metallic laminates

Herein this paper presents a general treatment of grain boundary constitutive behavior in the context of strain-gradient (SG) plasticity, and its numerical implementation in a large-strain (LS) elasto-viscoplastic (EVP) fast Fourier transform (FFT)-based micromechanical model. Two novel grain boundary constitutive equations are proposed, allowing for more accurate description of the Burgers vector flow at the grain boundary. The capabilities of the generalized SG-LS-EVPFFT formulation are illustrated for the case of kink-band formation during layer-parallel compression of nano-metallic laminates (NMLs), requiring consideration of the interaction between dislocations and interfaces.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Comparison of Biotite Elastic Properties Recovered by Spherical Nanoindentations and Atomistic Simulations — Influence of Nano-Scale Defects in Phyllosilicates

Phyllosilicate minerals, due to their sheets structure and morphology, are known to cause anisotropy in bulk rock properties and make the bulk rock more compliant. Accurately characterizing the micromechanical behavior of phyllosilicate minerals from laboratory observations, which eventually translates to the bulk rock behavior, is still challenging due to their fine-grained nature. Recent advances in atomistic simulations open the possibility of theoretically investigating such mineral mechanical behavior. We compare the elastic properties of biotites recovered by spherical nanoindentation with those predicted from density functional theory (DFT) simulations to investigate to what extent theoretical predictions reproduce actual phyllosilicate properties. Spherical nanoindentation was conducted using schist rocks from Poorman Formation, South Dakota, USA, to recover continuous indentation stress-strain curves. Loading in the layer-normal orientation shows an average indentation modulus (M) of about 35 GPa, while loading in the layer-parallel orientation gives a higher average of about 95 GPa. To facilitate comparison, the elastic stiffness constants (cij) determined from DFT were converted to indentation modulus (M) using solutions proposed in this study. The majority of the nanoindentation modulus results are below the values inferred from the simulation results representing ideal defect-free minerals. We suggest that crystal defects present at the nano-scale, potentially ripplocations, are the dominant cause of the lower indentation modulus recovered from nanoindentation compared to those inferred from DFT simulations. Results highlight the importance of acknowledging the defects that exist down to the nano-scale as it modifies the mechanical properties of phyllosilicates compared to its pure defect-free form.

58 GEOSCIENCES↗
DOE OSTI2020

TorchBraid

Torchbraid is a python package designed for interaction with the PyTorch library. It enables training of neural networks in a layer-parallel fashion.SAND2020-12228 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Cyr, Eric↗
NASA NTRS1993

Chocolate tablet aspects of cytherean Meshkenet Tessera

Meshkenet Tessera structures were mapped from Magellan data and several resemblances to chocolate tablet boudinage were found. The complex fault sets display polyphase tectonic sequences of a few main deformation phases. Shear and tension have contributed to the areal deformation. Main faults cut the 1600-km long Meshkenet Tessera highland into bar-like blocks which have ridge and groove pattern oriented along or at high angles to the faults. The first approach to the surface block deformation is an assumption of initial parallel shear faulting followed by a chocolate tablet boudinage. Major faults which cut Meshkenet Tessera into rectangular blocks have been active repetitively while two progressive or superposed boudinage set formations have taken place at high angles during the relaxational or flattening type deformation of the area. Chocolate tablet boudinage is caused by a layer-parallel two-dimensional extension resulting in fracturing of the competent layer. Such structures, defined by two sets of boudin neck lines at right angles to each other, have been described by a number of authors. They develop in a flattening type of bulk deformation or during superposed deformation where the rock is elongated in two dimensions parallel to the surface. This is an attempt to describe and understand the formation and development of structures of Meshkenet Tessera which has complicated fault structures.

Raitala, J.↗