Search NASA⌕ Search

SEARCH · Search NASA

Results for “Memory Optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

MDLoader: A Hybrid Model-Driven Data Loader for Distributed Graph Neural Network Training

Scalable data management is essential for processing large scientific dataset on HPC platforms for distributed deep learning. In-memory distributed storage is preferred for its speed, enabling rapid, random, and frequent data access required by stochastic optimizers. Processes use one-sided or collective communication to fetch remote data, with optimal performance depending on (i) dataset characteristics, (ii) training scale, and (iii) interconnection network. Empirical analysis shows collective communication excels with larger mini-batch sizes and/or fewer processes, whereas one-sided communication outperforms at larger scales. We propose MDLoader, a hybrid in-memory data loader for distributed graph neural network training. MDLoader features a model-driven performance estimator that dynamically selects between one-sided and collective communication at the beginning of training using Tree of Parzen Estimators (TPE). Evaluations on NERSC Perlmutter and OLCF Summit show MDLoader outperforms single-backend loaders by up to 2.83 × and predicts the suitable communication method with 96.3% (Perlmutter) and 94.3% (Summit) success rate.

Bae, Jonghyun↗
DOE OSTI2015

Parallel implementation and performance optimization of the configuration-interaction method

The configuration-interaction (CI) method, long a popular approach to describe quantum many-body systems, is cast as a very large sparse matrix eigenpair problem with matrices whose dimension can exceed one billion. Such formulations place high demands on memory capacity and memory bandwidth - - two quantities at a premium today. In this paper, we describe an efficient, scalable implementation, BIGSTICK, which, by factorizing both the basis and the interaction into two levels, can reconstruct the nonzero matrix elements on the fly, reduce the memory requirements by one or two orders of magnitude, and enable researchers to trade reduced resources for increased computational time. We optimize BIGSTICK on two leading HPC platforms - - the Cray XC30 and the IBM Blue Gene/Q. Specifically, we not only develop an empirically-driven load balancing strategy that can evenly distribute the matrix-vector multiplication across 256K threads, we also developed techniques that improve the performance of the Lanczos reorthogonalization. Combined, these optimizations improved performance by 1.3-8× depending on platform and configuration.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Blackcomb2: Hardware-Software Co-design for Nonvolatile Memory in Exascale Systems

The impending shift from DRAM and Flash to fast nonvolatile memory (NVM) provides a game-changing opportunity to rethink traditional system architectures; to address their energy inefficiencies; to leverage the new devices for greater performance; and to exploit the new capability of nonvolatility to enhance system resilience, even in the face of larger system scale and degraded reliability of components. Building on the accomplishments of the Blackcomb Project, funded in 2010, we propose to identify, evaluate, and optimize the most promising NVM hardware and software technologies, which are essential to provide the necessary memory capacity, performance, resilience, and energy efficiency in exascale systems. Capacity and energy are the key drivers. Today's pressing application requirements mandate exascale computing; the datasets and concurrency of these applications demand memory capacity that cannot be met by traditional memory technologies. Although charge-based DRAM will endure, experts foresee only modest DRAM capacity gains. And, as moving data between levels of the hierarchy, and storing and accessing data on disk consumes significant energy, a more energy efficient solution is required. We, therefore, posit that exascale systems will need high density, energy-efficient storage technologies, such as nonvolatile memory for access, transformation, and management of exascale data. To address these issues, in this renewal request, we propose continue the work of our vertically- integrated, focussed team, where our co-design is informed by proxies and interactions with DOE co-design centers. In software, we start by assuming that nonvolatility requires new abstractions and new infrastructure, allowing persistent objects, and delivering performance gains for _le systems and robustness through fast checkpointing. In architecture, we evaluate the tradeoffs of performance and energy by employing a scalable and accurate simulation system to analyze realistic proxy applications. Through this co-design process, we rethink the whole memory architecture, from the cell to the array to the device, with ECC, wearout management, and interfaces. Finally, we provide open-source software models of these NVM-based memory systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Large-scale quasi-Newton trust-region methods with low-dimensional linear equality constraints

We propose two limited-memory BFGS (L-BFGS) trust-region methods for large-scale optimization with linear equality constraints. Here, the methods are intended for problems where the number of equality constraints is small. By exploiting the structure of the quasi-Newton compact representation, both proposed methods solve the trust-region subproblems nearly exactly, even for large problems. We derive theoretical global convergence results of the proposed algorithms, and compare their numerical effectiveness and performance on a variety of large-scale problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Joint perpendicular anisotropy and strong interlayer exchange coupling in systems with thin vanadium spacers

We study the influence of the insertion of a vanadium spacer layer between an FeCoB layer and a [Co/Ni] multilayer in an MgO substrate-based system mimicking the reference system of a perpendicular anisotropy magnetic tunnel junction. The anisotropy of the [Co/Ni] multilayer gradually improves with the vanadium thicknesses t, up to an optimized state for t = 8 Å, with little influence of the thermal annealing. The interlayer exchange coupling is ferromagnetic and very strong for t≤6 Å. It can be adjusted by thermal treatment at t = 8 Å from no coupling in the as-grown state to more than 2 mJ/m{sup 2} after 250 °C annealing. For this spacer thickness, the magnetic properties are consistent with the occurrence of a bcc (001) to an fcc (111) crystalline structure transition at the vanadium spacer. The remaining interlayer exchange coupling at t = 8 Å is still substantially higher than the one formerly obtained with a Tantalum spacer, which holds promise for further optimization of the reference layers of tunnel junctions meant for magnetic random access memories.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Sparse Deep Neural Network Inference using different Programming Models

Sparse deep neural networks have gained increasing attention recently in achieving speedups on inference with reduced memory footprints. However, the real-world applications are little shown with specialized optimizations, yet a wide variety of DNN tasks remain dense without exploiting the advantages of sparsity in networks. Recent work presented by MIT/IEEE/Amazon GraphChallenge has demonstrated significant speedups and various techniques. Still, we find that there is limited investigation of the impact of various Python and C\slash C++ based programming models to explore new opportunities in general cases. In this work, we provide performance evaluation through different programming models using CuPy, cuSPARSE, and OpenMP to discuss the advantages and disadvantages of our sparse implementations on single-GPU and multiple GPUs of NVIDIA DGX-A100 40GB/80GB platforms.

machine learning, HPC↗
DOE OSTI2016

Low-temperature SiON films deposited by plasma-enhanced atomic layer deposition method using activated silicon precursor

It has not been an easy task to deposit SiN at low temperature by conventional plasma-enhanced atomic layer deposition (PE-ALD) since Si organic precursors generally have high activation energy for adsorption of the Si atoms on the Si-N networks. In this work, in order to achieve successful deposition of SiN film at low temperature, the plasma processing steps in the PE-ALD have been modified for easier activation of Si precursors. In this modification, the efficiency of chemisorption of Si precursor has been improved by additional plasma steps after purging of the Si precursor. As the result, the SiN films prepared by the modified PE-ALD processes demonstrated higher purity of Si and N atoms with unwanted impurities such as C and O having below 10 at. % and Si-rich films could be formed consequently. Also, a very high step coverage ratio of 97% was obtained. Furthermore, the process-optimized SiN film showed a permissible charge-trapping capability with a wide memory window of 3.1 V when a capacitor structure was fabricated and measured with an insertion of the SiN film as the charge-trap layer. The modified PE-ALD process using the activated Si precursor would be one of the most practical and promising solutions for SiN deposition with lower thermal budget and higher cost-effectiveness.

36 MATERIALS SCIENCE↗
DOE OSTI2016

Extending the Binomial Checkpointing Technique for Resilience

In terms of computing time, adjoint methods offer a very attractive alternative to compute gradient information, re- quired, e.g., for optimization purposes. However, together with this very favorable temporal complexity result comes a memory requirement that is in essence proportional with the operation count of the underlying function, e.g., if algo- rithmic differentiation is used to provide the adjoints. For this reason, checkpointing approaches in many variants have become popular. This paper analyzes an extension of the so-called binomial approach to cover also possible failures of the computing systems. Such a measure of precaution is of special interest for massive parallel simulations and adjoint calculations where the mean time between failure of the large scale computing system is smaller than the time needed to complete the calculation of the adjoint information. We de- scribe the extensions of standard checkpointing approaches required for such resilience, provide a corresponding imple- mentation and discuss numerical results.

Walther, Andrea↗
DOE OSTI2016

Covering Resilience: A Recent Development for Binomial Checkpointing

In terms of computing time, adjoint methods offer a very attractive alternative to compute gradient information, required, e.g., for optimization purposes. However, together with this very favorable temporal complexity result comes a memory requirement that is in essence proportional with the operation count of the underlying function, e.g., if algorithmic differentiation is used to provide the adjoints. For this reason, checkpointing approaches in many variants have become popular. This paper analyzes an extension of the so-called binomial approach to cover also possible failures of the computing systems. Such a measure of precaution is of special interest for massive parallel simulations and adjoint calculations where the mean time between failure of the large scale computing system is smaller than the time needed to complete the calculation of the adjoint information. We describe the extensions of standard checkpointing approaches required for such resilience, provide a corresponding implementation and discuss first numerical results.

Walther, Andrea↗
DOE OSTI2016

Rewritable three-dimensional holographic data storage via optical forces

The development of nanostructures that can be reversibly arranged and assembled into 3D patterns may enable optical tunability. However, current dynamic recording materials such as photorefractive polymers cannot be used to store information permanently while also retaining configurability. Here, we describe the synthesis and optimization of a silver nanoparticle doped poly(2-hydroxyethyl methacrylate-co-methacrylic acid) recording medium for reversibly recording 3D holograms. We theoretically and experimentally demonstrate organizing nanoparticles into 3D assemblies in the recording medium using optical forces produced by the gradients of standing waves. The nanoparticles in the recording medium are organized by multiple nanosecond laser pulses to produce reconfigurable slanted multilayer structures. We demonstrate the capability of producing rewritable optical elements such as multilayer Bragg diffraction gratings, 1D photonic crystals, and 3D multiplexed optical gratings. We also show that 3D virtual holograms can be reversibly recorded. This recording strategy may have applications in reconfigurable optical elements, data storage devices, and dynamic holographic displays.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Communication Lower Bounds and Optimal Algorithms for Symmetric Matrix Computations

In this article, we focus on the communication costs of three symmetric matrix computations: (i) multiplying a matrix with its transpose, known as a symmetric rank-k update (SYRK) (ii) adding the result of the multiplication of a matrix with the transpose of another matrix and the transpose of that result, known as a symmetric rank-2k update (SYR2K) (iii) performing matrix multiplication with a symmetric input matrix (SYMM). All three computations appear in the Level 3 Basic Linear Algebra Subroutines (BLAS) and have wide use in applications involving symmetric matrices. We establish communication lower bounds for these kernels using sequential and distributed-memory parallel computational models, and we show that our bounds are tight by presenting communication-optimal algorithms for each setting. Our lower bound proofs rely on applying a geometric inequality for symmetric computations and analytically solving constrained nonlinear optimization problems. As a result, the symmetric matrix and its corresponding computations are accessed and performed according to a triangular block partitioning scheme in the optimal algorithms.

Al Daas, Hussam [Rutherford Appleton Laboratory, D↗
DOE OSTI2019

Systems and methods for convex relaxations and linear approximations for optimal power flow in multiphase radial networks

Centralized node controllers in accordance with embodiments of the invention enable linear approximation of optimal power flow. One embodiment includes a centralized node controller including: a network interface, a processor, and a memory containing: a centralized power control application a network topology, where the network is multiphase unbalanced and comprises a plurality of connected nodes; wherein the processor is configured by the centralized controller application to: request node operating parameters from the plurality of connected nodes; calculate network operating parameters using a linear approximation of optimal power flow and the node operating parameters from the plurality of connected nodes; send network operating parameters to the plurality of connected nodes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Enabling Efficient Sparse Computations using Linear Algebra Aware Compilers

This project developed the LAPIS compiler framework, built on the Multilevel Intermediate Representation (MLIR), to optimize sparse linear algebra operations and support performance portability across diverse architectures. The main innovation of LAPIS is the Kokkos dialect, which allows for lowering codes from a high productivity language to different architectures in an elegant way. The dialect also allows the conversion of lower-level MLIR code to C++ Kokkos code, facilitating the integration of scientific machine learning (SciML) models into applications. To extend LAPIS for distributed memory architectures, a new partition dialect was created to manage the distribution of sparse tensors and express communication patterns for sparse linear algebra operations. This dialect also supports the distributed execution of operators and includes algorithmic optimizations to minimize communication to improve performance. The project also demonstrates that MLIR can enable effective linear algebra-level optimizations, improving performance on different GPUs for both sparse and dense linear algebra kernels. Key applications of LAPIS include sparse linear algebra and graph kernels, TenSQL, a relational database management solution built on GraphBLAS, and the development of subgraph isomorphism and monomorphism kernels, showcasing performance portability. In summary, the LAPIS framework supports productivity, performance, portability, and distributed memory execution, while also enabling linear algebra-level optimizations that are challenging in traditional programming languages, with successful applications ranging from simple sparse linear algebra to complex graph kernels.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Equation‐Free Surrogate Modeling of Geophysical Flows at the Intersection of Machine Learning and Data Assimilation

Abstract There is a growing interest in developing data‐driven reduced‐order models for atmospheric and oceanic flows that are trained on data obtained either from high‐resolution simulations or satellite observations. The data‐driven models are non‐intrusive in nature and offer significant computational savings compared to large‐scale numerical models. These low‐dimensional models can be utilized to reduce the computational burden of generating forecasts and estimating model uncertainty without losing the key information needed for data assimilation (DA) to produce accurate state estimates. This paper aims at exploring an equation‐free surrogate modeling approach at the intersection of machine learning and DA in Earth system modeling. With this objective, we introduce an end‐to‐end non‐intrusive reduced‐order modeling (NIROM) framework equipped with contributions in modal decomposition, time series prediction, optimal sensor placement, and sequential DA. Specifically, we use proper orthogonal decomposition (POD) to identify the dominant structures of the flow, and a long short‐term memory network to model the dynamics of the POD modes. The NIROM is integrated within the deterministic ensemble Kalman filter (DEnKF) to incorporate sparse and noisy observations at optimal sensor locations obtained through QR pivoting. The feasibility and the benefit of the proposed framework are demonstrated for the NOAA Optimum Interpolation Sea Surface Temperature (SST) V2 data set. Our results indicate that the NIROM is stable for long‐term forecasting and can model dynamics of SST with a reasonable level of accuracy. Furthermore, the prediction accuracy of the NIROM gets improved by almost one order of magnitude by the DEnKF algorithm.

Pawar, Suraj↗
DOE OSTI2019

Scalable optimal Bayesian classification of single-cell trajectories under regulatory model uncertainty

Single-cell gene expression measurements offer opportunities in deriving mechanistic understanding of complex diseases, including cancer. However, due to the complex regulatory machinery of the cell, gene regulatory network (GRN) model inference based on such data still manifests significant uncertainty. The goal of this paper is to develop optimal classification of single-cell trajectories accounting for potential model uncertainty. Partially-observed Boolean dynamical systems (POBDS) are used for modeling gene regulatory networks observed through noisy gene-expression data. We derive the exact optimal Bayesian classifier (OBC) for binary classification of single-cell trajectories. The application of the OBC becomes impractical for large GRNs, due to computational and memory requirements. To address this, we introduce a particle-based single-cell classification method that is highly scalable for large GRNs with much lower complexity than the optimal solution. The performance of the proposed particle-based method is demonstrated through numerical experiments using a POBDS model of the well-known T-cell large granular lymphocyte (T-LGL) leukemia network with noisy time-series gene-expression data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

Optimizing the Four-Index Integral Transform Using Data Movement Lower Bounds Analysis

The four-index integral transform is a fundamental and computationally demanding calculation used in many computational chemistry suites such as NWChem. It transforms a four-dimensional tensor from an atomic basis to a molecular basis. This transformation is most efficiently implemented as a sequence of four tensor contractions that each contract a four-dimensional tensor with a two-dimensional transformation matrix. Differing degrees of permutation symmetry in the intermediate and final tensors in the sequence of contractions cause intermediate tensors to be much larger than the final tensor and limit the number of electronic states in the modeled systems. Loop fusion, in conjunction with tiling, can be very effective in reducing the total space requirement, as well as data movement. However, the large number of possible choices for loop fusion and tiling, and data/computation distribution across a parallel system, make it challenging to develop an optimized parallel implementation for the four-index integral transform. We develop a novel approach to address this problem, using lower bounds modeling of data movement complexity. We establish relationships between available aggregate physical memory in a parallel computer system and ineffective fusion configurations, enabling their pruning and consequent identification of effective choices and a characterization of optimality criteria. This work has resulted in the development of a significantly improved implementation of the four-index transform that enables higher performance and the ability to model larger electronic systems than the current implementation in the NWChem quantum chemistry software suite.

Rajbhandari, Samyam↗
DOE OSTI2015

Quasi-Optimal Elimination Trees for 2D Grids with Singularities

We construct quasi-optimal elimination trees for 2D finite element meshes with singularities. These trees minimize the complexity of the solution of the discrete system. The computational cost estimates of the elimination process model the execution of the multifrontal algorithms in serial and in parallel shared-memory executions. Since the meshes considered are a subspace of all possible mesh partitions, we call these minimizers quasi-optimal. We minimize the cost functionals using dynamic programming. Finding these minimizers is more computationally expensive than solving the original algebraic system. Nevertheless, from the insights provided by the analysis of the dynamic programming minima, we propose a heuristic construction of the elimination trees that has cost O N e log ⁡ N e , where N e is the number of elements in the mesh. We show that this heuristic ordering has similar computational cost to the quasi-optimal elimination trees found with dynamic programming and outperforms state-of-the-art alternatives in our numerical experiments.

Paszyńska, A.↗
DOE OSTI2018

Parallelizable adjoint stencil computations using transposed forward-mode algorithmic differentiation

Algorithmic differentiation (AD) is a tool for generating discrete adjoint solvers, which efficiently compute gradients of functions with many inputs, for example for use in gradient-based optimization. AD is often applied to large computations such as stencil operators, which are an important part of most structured-mesh PDE solvers. Stencil computations are often parallelized, for example by using OpenMP, and optimized by using techniques such as cache-blocking and tiling to fully utilize multicore CPUs and many-core accelerators and GPUs. Differentiating these codes with conventional reverse-mode AD results in adjoint codes that cannot be expressed as stencil operations and may not be easily parallelizable. They thus leave most of the compute power of modern architectures unused. We present a method that combines forward-mode AD and loop transformation to generate adjoint solvers that use the same memory access pattern as the original computation that they are derived from and can benefit from the same optimization techniques. The effectiveness of this method is demonstrated by generating a scalable adjoint CFD solver for multicore CPUs and Xeon Phi accelerators.

algorithmic differentiation↗