Search NASA⌕ Search

SEARCH · Search NASA

Results for “compressed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Efficient I/O for Neural Network Training with Compressed Data

FanStore is a shared object store that enables efficient and scalable neural network training on supercomputers. By providing a global cache layer on node-local burst buffers using a compressed representation, it significantly enhances the processing capability of deep learning (DL) applications on existing hardware. In addition, FanStore allows POSIX-compliant file access to the compressed data in user space. We investigate the tradeoff between runtime overhead and data compression ratio using real-world datasets and applications, and propose a compressor selection algorithm to maximize storage capacity given performance constraints. We consider both asynchronous (i.e., with prefetching) and synchronous I/O strategies, and propose mechanisms for selecting compressors for both approaches. Using FanStore, the same storage hardware can host 2-13x more data for example applications without significant runtime overhead. Empirically, our experiments show that FanStore scales to 512 compute nodes with near linear performance scalability.

Zhang, Zhao↗
DOE OSTI2020

Optimization of energy requirements for CO 2 post-combustion capture process through advanced thermal integration

The energy optimization modeling work described here was performed to determine efficiency improvements that could be achieved for existing coal-fired power plants to retrofit a partial CO 2 capture from the post-combustion flue gas for carbon sequestration through thermal integration. The work presented includes optimization of the mono-ethanol amine (MEA)-based post-combustion CO 2 capture to reduce energy requirements that could be achieved at existing power plants by thermal integration of the steam turbine cycle, boiler, CO 2 compression train and post-combustion CO 2 capture process to offset efficiency and capacity losses that would be incurred by retrofit or implementation of post-combustion CO 2 capture. Furthermore, partial CO 2 capture, involving treatment of less than 100% of the flue gas leaving the plant and modular design of the CO 2 scrubbing system, was also investigated. Thermal integration of the steam turbine cycle with boiler and CO 2 compression train improved cycle and plant performance and offset, in part, the negative effects of post-combustion CO 2 capture. The best-analyzed integration options improved gross power output by 5% and net unit efficiency by 1.57%, relative to the conventional MEA process. Operating with 40% CO 2 capture increased gross power output by 11.6–14% (depending on the MEA thermal integration option), relative to the conventional MEA integration and 90% CO 2 capture. The improvement in net unit performance is larger compared to the improvement in turbine cycle performance because of the CO 2 compression work, which is also reduced by partial CO 2 capture.

42 ENGINEERING↗
DOE OSTI2022

Emissions mitigation technology for advanced water-lean solvent-based CO 2 capture processes

This technical final report submitted to DOE/NETL presents all the research activities performed during the entirety of DE-FE0031660 project-Emissions Mitigation Technology for Advanced Water-Lean Solvent-Based CO 2 Capture Processes which spans from October 2018 through March 2022. RTI International has been conducting studies from fundamental and operational aspects to reduce the overall amine emissions from the advanced Water-Lean Solvent (WLS) systems, specifically RTI’s Non-Aqueous Solvent (NAS). This technical final report will highlight the key findings from project which align closely to the project objectives which are: Identify the contribution of vapor loss, entrainment, and aerosols to the overall emissions of water-lean systems; Determine the significance of CO 2 capture system operating parameters to the amine emissions; Develop an emissions model based on critical operating parameters; Evaluate the effectiveness of emissions mitigation devices to reduce the amine emissions to <1 ppm under flue coal-fired flue gas; and, Determine the contribution of the ECTs to the overall CO 2 capture cost. The following are the key findings based on numerous tests using both lab-scale setups and parametric testing performed at RTI’s Bench-scale Gas Absorption System (BsGAS). During the BP1, the aerosol generation system and monitoring equipment were installed at BsGAS to produce and determine the aerosol characteristics during the NAS CO 2 capture process. The aerosol produced by this setup produced aerosols with the peak diameter and concentration of 50 micron and 1.2E10 7 cm -3 , respectively. These particle sizes and concentrations are matched to those observed in the actual coal-fired power plant flue gases and expected to be found at the absorber inlet of the CO 2 capture system. Over 1,300 hours of parametric testing have been conducted to evaluate the impact of the aerosols and operating conditions during the CO 2 capture with NAS on the overall amine emissions in the treated flue gas. At the worse condition tested, the presence of the aerosols in the flue gas could increase the overall emissions by 10X compared to the baseline emissions from NAS’s vapor pressure. CO 2 capture rate was found to be a main factor impacting the overall emissions as well as aerosol size and concentrations in the absorber off-gas. The higher CO 2 capture rate, the higher amine emissions in the treated gas. The temperature difference between the temperature bulge seen in the absorber and the water wash temperature also impacts the particle growth where the larger the temperature difference, the more amine emissions from aerosols in the treated gas. The majority of the aerosols did not grow substantially in the system, and the particle concentrations remained nearly constant between the absorber inlet and wash outlet. Only a small portion of the particles were found to grow significantly. The high efficiency demister with mesh size of 5-10 micron can be installed to remove a portion of the aerosols from the gas stream leaving the water wash. Overall, these results from parametric testing have established the emission baseline and validate our assumption on the need of emission control technologies (ECT) in order to minimize the emissions from the baseline NAS CO 2 capture process. Over 2,000 of BsGAS operating hours was used to investigate a handful of process improvements which led to a selection of the vital few changes that effectively control the amine emissions. These process improvements are lime-coated-filters for absorber gas inlet, advanced demister at the top of the absorber, a second water wash with amine recovery unit were designed, installed, and tested at BsGAS at the end of BP1. The result showed that the NAS CO 2 capture process with these additional emission control devices could lower the amine emission in the treated gas to about 1 ppm using a simulated coal-fire flue gas stream. The main contributor in lowering the amine emission came from the second water wash with amine recovery unit where the amine concentration in the scrubbing water was kept below 2 wt% through a continuous amine removal via an adsorbent bed, resulting in a low amine vapor pressure. The adsorbent bed was regenerated via a direct steam regeneration and the recover amine was returned to the absorber to minimize wastewater and makeup amine. A flue gas generation system was designed and installed during the first half of BP2 to support the emission testing using a real coal-derived flue gas. The system is capable of generating both coal- and natural gas- derived flue gases with the composition of the gaseous species highly resemble to that of the power plant flue gases. The particulates detected in the coal-derived flue gas showed the mean diameter of 1 micron. The CO 2 capture operating was then proceed using the real coal-derived flue gas where the amine emission was controlled to be about 0-3 ppm for the total run time of about 200 hours. Similar testing was conducted with natural gas-derived flue gas and the result showed a highly amine emission of 30 ppm under the total run time of 200 hours. The Principal Component Analysis (PCA) and the Partial Least Squares Projection to Latent Structures (PLS) techniques were applied to the parametric testing data to derive a multivariate statistical model. The model was validated and trained with half of the data collected, and the predictive ability of the model was evaluated using the remaining half of the data. The resulting empirical model was capable of predicting the overall emissions from the NAS process without the ECTs with ±15% accuracy (average absolute deviation, AAD) in BP1. As more emission data were obtained under the real coal-flue gas in the BP2, the model incorporated these new set of data to reflect the final process configuration, operating parameters, and amine emission. This results in the updated empirical model predicting the amine emission from the NAS CO 2 capture process with 84% goodness-of-fit (R 2 ), 85% predictability (Q 2 ), and 15% AAD. The study evaluates the use of RTI’s Non-Aqueous Solvent technology for 90% CO 2 capture from a net 650 MWe pulverized coal power plant, downstream of the flue-gas desulfurization unit. The captured CO 2 has a purity of > 95% CO 2 , and is dried, compressed to 15.3 MPa (2,215 psia), ready for sequestration. The analysis uses Case B12B from the DOE Baseline study on Bituminous Coal, Revision 4 where the Cansolv CO 2 capture plant is replaced by the RTI CO 2 Capture plant. The CO 2 capture plant has been sized to capture >90% CO 2 from flue gas derived from a net 650 MWe supercritical pulverized coal power plant. The CO 2 capture plant is equipped with emission control technologies that limits the amine emissions to < 1 ppm. Two different cases were evaluated for the technoeconomic study. The key difference between the two cases is the regenerator pressure. In Case 1, the regenerator operates at 0.195 MPa (28.3 psia), whereas in Case 2, the regenerator pressure is 0.44 MPa (64 psia) thus removing the need for the first stage of compression of the eight-stage compression train. Results from the TEA are compared against the DOE reference cases for SCPC plant with and without CO 2 Capture (Case B12A and Case B12B of the DOE Baseline study, respectively). Case 2 with CO 2 regeneration at higher pressure results in the lower cost of CO 2 capture. The total capital cost of the capture process has been estimated using 2018 dollars in Aspen Process Economic Analyzer and was estimated to be $579 MM. The capture plant operation leads to a total parasitic power loss rate of 96 MWe, resulting in a decrease in pulverized coal power plant efficiency of 7.8% points. The resulting cost of electric power increases from 64.4 mills/kWh, for no capture, to 97.5 mills/kWh, with 90% capture, an increase of 51% in the COE. The cost of capturing 90% CO 2 was estimated to be $38.2/tonne-CO 2 , and meets the DOE target of $40/t-CO 2 . Emission control technologies (ECT) investigated in this project includes a second water wash with use of activated carbon beds for removal of amine from the wash water prior to recirculation in the water wash. These ECT allow operation of the CO 2 capture plant with < 1 ppm amine emissions with the treated flue gas and contributes to $2.4/t-CO 2 captured. Amine emissions derived from thermal and oxidative degradations were investigated under this project along with the emissions derived from aerosols for the NAS system. The thermally degraded of the lean NAS showed less than 4% decreased of the original total amine content in the NAS at 150 °C while the result obtained at 120 °C showed no drop in total amine content, suggesting that thermal degradation of the NAS is minimal. These results also suggested that the thermally degraded species are not likely formed and contributed to the emissions due to the low regeneration temperature of the NAS at 90-105 °C. The oxidative degradation, on the other hand, could become problematic as some of these oxidative degraded species were observed during the NAS-5 testing at National Carbon Capture Center (NCCC) and SINTEF in our previous project. The rapid screening of selected inhibitors suggested that oxidative degradation of NAS can be suppressed using thiol containing compounds in amounts of at least 1 mol%. The detailed mechanistic degradation pathway was conceived for a specific amine used in NAS formulation during BP2. he reduction of the nitrosamines caused by the NO x present in the flue gas was also examined. The study suggested that the thermo-chemical treatment of the NAS solvent would be a more effective and economically viable compared to removing NO x at the DCC.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2022

Front End Engineering Design of Linde-BASF Advanced Post-Combustion CO 2 Capture Technology at a Southern Company Natural Gas-Fired Power Plant (Final Scientific/Technical Report)

This document details the execution of Cooperative Agreement DE-FE0031847, “Front End Engineering Design of Linde-BASF Advanced Post-Combustion Carbon Dioxide (CO 2 ) Capture Technology at a Southern Company Natural Gas-Fired Power Plant” during the period of 10/1/2019 to 6/30/2022. The project was funded by the U.S. Department of Energy’s Office of Fossil Energy and Carbon Management (FECM) and managed by the National Energy Technology Laboratory (NETL). Southern Company Services, Inc. (SCS) was the prime recipient and led the project team. Other members of the project team included Linde, Inc. (Linde), Linde Engineering – Dresden (LED), and BASF. The overall goal of the project was to complete a front-end engineering design (FEED) study for installing the Linde-BASF post-combustion capture (PCC) technology at an existing domestic natural gas-fired combined cycle (NGCC) power plant within Southern Company’s portfolio of assets. The CO 2 capture plant was to be of commercial scale (at least 375 MWe) and include process units for pre-conditioning of the flue gas system, the CO 2 capture plant island, storage vessels, the CO 2 compression train, and any necessary components for integration into the NGCC plant. Mississippi Power’s Plant Daniel Unit #4 was chosen as the host site for the FEED with the target of capturing 90% of CO 2 emissions from the existing combustion turbines. The information produced by the FEED was used to develop a cost estimate of +/- 15% accuracy. Capital costs, excluding financing, are estimated at approximately $\$752$ million dollars (2021). The execution of a project based on this FEED study has an estimated duration of almost five years.

03 NATURAL GAS↗
DOE OSTI2024

Geometry-aware training of factorized layers in tensor Tucker format

Reducing parameter redundancies in neural network architectures is crucial for achieving feasible computational and memory requirements during train and inference of large networks. Given its easy implementation and flexibility, one promising approach is layer factorization, which reshapes weight tensors into a matrix format and parameterizes it as the product of two rank-r matrices. However, this family of approaches often requires an initial full-model warm-up phase, prior knowledge of a feasible rank, and it is sensitive to parameter initialization.In this work, we introduce a novel approach to train the factors of a Tucker decomposition of the weight tensors. Our training proposal proves to be optimal in locally approximating the original unfactorized dynamics and stable for the initialization. Furthermore, the rank of each mode is dynamically updated during training.We provide a theoretical analysis of the algorithm, showing convergence, approximation and local descent guarantees. The method's performance is further illustrated through a variety of experiments, showing remarkable training compression rates and comparable or even better performance than the full baseline and alternative layer factorization strategies.

Zangrando, Emanuele [Gran Sasso Science Institute ↗
DOE OSTI2022

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗
DOE OSTI2025

Learning-Based Quantum Compilation: Translating QASM to QIR with CodeBERT

We propose a learning-based approach to quantum compilation by translating OpenQASM to Quantum Intermediate Representation (QIR) using a fine-tuned CodeBERT model. Trained on 10,000 synthetic QASM-QIR pairs, the model captures code semantics while addressing QIR verbosity and the 512-token limit via a custom token compression scheme. Finetuning was performed on the Frontier supercomputer, with results showing syntactic correctness and stable validation loss reduction. Our method moves toward enabling flexible, language-modeldriven quantum software tools. It also introduces syntax error handling and the possibility of incorporating classical control constructs, addressing limitations in existing rule-based compilers like qBraid-QIR. While the current model has been validated on quantum-only circuits, we propose future evaluations on hybrid quantum-classical examples. This poster will provide architecture insights, compression examples, training loss plots, and QIR outputs. Our work highlights the potential for scalable, adaptable compilation in future quantum toolchains.

Afrose, Sharmin [ORNL]↗
DOE OSTI2020

Estimating Lossy Compressibility of Scientific Data Using Deep Neural Networks

Simulation based scientific applications generate increasingly large amounts of data on high-performance computing (HPC) systems. To allow data to be stored and analyzed efficiently, data compression is often utilized to reduce the volume and velocity of data. However, a question often raised by domain scientists is the level of compression that can be expected so that they can make more informed decisions, balancing between accuracy and performance. In this letter, we propose a deep neural network based approach for estimating the compressibility of scientific data. To train the neural network, we build both general features as well as compressor-specific features so that the characteristics of both data and lossy compressors are captured in training. Our approach is demonstrated to outperform a prior analytical model as well as a sampling based approach in the case of a biased estimation, i.e., for SZ. However, for the unbiased estimation (i.e., ZFP), the sampling based approach yields the best accuracy, despite the high overhead involved in sampling the target dataset.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Dynamic Low-Rank Training with Spectral Regularization: Achieving Robustness in Compressed Representations

Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing clean accuracy. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94\% compression while recovering or improving adversarial accuracy relative to uncompressed baselines.

Schotthoefer, Steffen [ORNL] (ORCID:00000002156965↗
DOE OSTI2021

Analyzing acoustic emission data to identify cracking modes in cement paste using an artificial neural network

This research is focused on the identification of cracking mechanisms for cement paste using acoustic emission data, recorded from compression and notched four-point bending tests. A procedure is developed for analyzing the data by employing an agglomerative hierarchical clustering method, an artificial neural network, and a ray-tracing source location algorithm. An agglomerative hierarchical clustering method is utilized to cluster the AE data from a compression test using frequency-dependent features. A neural network is trained using the compression test data and applied to the AE data emitted during the four-point bending test. The clustered data from the four-point bending test is localized using a ray-tracing algorithm. Based on the occurrence and locations of the clustered events and signal feature analyses, potential cracking mechanisms are identified and assigned.

36 MATERIALS SCIENCE↗
DOE OSTI2024

An Incremental Tensor Train Decomposition Algorithm

We present a new algorithm for incrementally updating the tensor train decomposition of a stream of tensor data. This new algorithm, called the tensor train incremental core expansion (TT-ICE) improves upon the current state-of-the-art algorithms for compressing in tensor train format by developing a new adaptive approach that incurs significantly slower rank growth and guarantees compression accuracy. This capability is achieved by limiting the number of new vectors appended to the TT-cores of an existing accumulation tensor after each data increment. These vectors represent directions orthogonal to the span of existing cores and are limited to those needed to represent a newly arrived tensor to a target accuracy. We provide two versions of the algorithm: TT-ICE and TT-ICE accelerated with heuristics (TT-ICE*). Here, we provide a proof of correctness for TT-ICE and empirically demonstrate the performance of the algorithms in compressing large-scale video and scientific simulation datasets. Compared to existing approaches that also use rank adaptation, TT-ICE* achieves 57× higher compression and up to 95% reduction in computational time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Bayesian tensorized neural networks with automatic rank selection

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In order to address this challenge, this paper proposes a low-rank Bayesian tensorized neural network. Our Bayesian method performs automatic model compression via an adaptive tensor rank determination. We also present approaches for posterior density calculation and maximum a posteriori (MAP) estimation for the end-to-end training of our tensorized neural network. Here, we provide experimental validation on a two-layer fully connected neural network, a 6-layer CNN and a 110-layer residual neural network where our work produces 7.4x to 137x more compact neural networks directly from the training while achieving high prediction accuracy.

Low-rank tensor↗
DOE OSTI2023

Black-box statistical prediction of lossy compression ratios for scientific data

Lossy compressors are increasingly adopted in scientific research, tackling volumes of data from experiments or parallel numerical simulations and facilitating data storage and movement. In contrast with the notion of entropy in lossless compression, no theoretical or data-based quantification of lossy compressibility exists for scientific data. Users rely on trial and error to assess lossy compression performance. As a strong data-driven effort toward quantifying lossy compressibility of scientific datasets, we provide a statistical framework to predict compression ratios of lossy compressors. Our method is a two-step framework where (i) compressor-agnostic predictors are computed and (ii) statistical prediction models relying on these predictors are trained on observed compression ratios. Proposed predictors exploit spatial correlations and notions of entropy and lossyness via the quantized entropy. We study 8+ compressors on 6 scientific datasets and achieve a median percentage prediction error less than 12%, which is substantially smaller than that of other methods while achieving at least a 8.8× speedup for searching for a specific compression ratio and 7.8× speedup for determining the best compressor out of a collection.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Simultaneous contrast improvement and temporal compression using divided-pulse nonlinear compression

We experimentally demonstrate how divided-pulse nonlinear compression can be used to improve the temporal contrast of a laser pulse train while simultaneously temporally compressing the pulses. We measure a contrast improvement of almost four orders of magnitude on a nanosecond time scale and temporally compress the pulses from 1.2 ps to 187 fs. The efficiency of our method is also competitive with other contrast improvement methods, with 72% efficiency measured for the main pulse. We expect the method will be useful in the continuing development of high-power, Yb regenerative amplifiers, which suffer from both significant prepulses and relatively long pulse durations.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Georgia Tech Accelerated, Compressed, and Regularized Compute of Kinetic-based PDEs (Final Report)

This report summarizes the collaborative effort between Lawrence Livermore National Laboratory and Georgia Tech to enhance the BoBa library for tensor train computation in PDE solvers, with a target on kinetic equations and their continuum limits. We aimed to reduce computational cost and memory usage by replacing traditional array-based computations with tensor trains. We examined the compressibility of time-evolving solutions to the Euler equations with discontinuities. We also explored using the first invsicid and linear regularization of the compressible flow equations via the information geometric regularization (IGR). We explored this in a tensor train formulation. To identify that inverse terms in the IGR equations pose problems for tensor train formulations and investigate efficient methods for batched inversion of tensor trains.

97 MATHEMATICS AND COMPUTING↗