SEARCH · Search NASA

Results for “data reduction techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

All sources NASA NTRS DOE OSTI

DOE OSTI2012

Improving I/O Throughput with PRIMACY: Preconditioning ID-Mapper for Compressing Incompressibility

The ability to efficiently handle massive amounts of data is necessary for the continuing development towards exascale scientific data-mining applications and database systems. Unfortunately, recent years have shown a growing gap between the size and complexity of data produced from scientific applications and the limited I/O bandwidth available on modern high-performance computing systems. Utilizing data compression in order to lower the degree of I/O activity offers a promising means to addressing this problem. However, the standard compression algorithms previously explored for such use offer limited gains on both the end-to-end throughput and storage fronts. In this paper, we introduce an in-situ compression scheme aimed at improving end-to-end I/O throughput as well as reduction of dataset size. Our technique, PRIMACY (Preconditioning Id-MApper for Compressing incompressibility), acts as a preconditioner for standard compression libraries by modifying representation of original floating-point scientific data to increase byte-level repeatability, allowing standard loss less compressors to take advantage of their entropy-based byte-level encoding schemes. We additionally present a theoretical model for compression efficiency in high-performance computing environments and evaluate the efficiency of our approach via comparative analysis. Based on our evaluations on 20 real-world scientific datasets, PRIMACY achieved up to 38% and 22% improvements upon standard end-to-end write and read throughputs respectively in addition to a 25% increase in compression ratios paired with 3-to-4-fold improvement in both compression and decompression throughput over general purpose compressors.

Computer Science↗

DOE OSTI2017

Integrated Nondestructive Assay Systems to Estimate Plutonium in Spent Fuel Assemblies

An integrated nondestructive assay (NDA) system combining active (neutron generator) and passive neutron detection and passive gamma (PG) detection is being analyzed in order to estimate the amount of plutonium, verify initial enrichment, burnup, and cooling time, and detect partial defects in a spent fuel assembly (SFA). Active signals are measured using the differential die-away (DDA), delayed neutron (DN), and delayed gamma (DG) techniques. Passive signals are measured using total neutron (TN) counts and both gross and spectral resolved gamma counts. To quantify how a system of several NDA techniques is expected to perform, all of the relevant NDA techniques listed above were simulated as a function of various reactor conditions such as initial enrichment, burnup, cooling time, assembly shuffling pattern, reactor operating conditions (including temperature, pressure, and the presence of burnable poisons) by simulating the NDA response for five sets of light water reactor assemblies. In this study, we compare the performance of several exploratory model-fitting options (including neural networks, adaptive regression with splines, iterative bias reduction smoothing, projection pursuit regression, and regression with quadratic terms and interaction terms) to relate data simulated with measurement and model error effects from various subsets of the NDA techniques to the total Pu mass. Isotope masses for SFAs and expected detector responses (DRs) for several NDA techniques are simulated using MCNP, and the DRs become inputs to the fitting process. Such responses include eight signals from DDA, one from DN, one from TN, and up to seven from PG; the DG signal will be examined separately. Results are summarized using the root-mean-squared estimation error for plutonium mass in held-out subsets of the data for a range of model and measurement error variances. Different simulation assumptions lead to different spent fuel libraries relating DRs to Pu mass. Some results for training with one library and testing with another library are also given.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2015

Ecosystem biogeochemistry model parameterization: Do more flux data result in a better model in predicting carbon flux?

Reliability of terrestrial ecosystem models highly depends on the quantity and quality of thedata that have been used to calibrate the models. Nowadays, in situ observations of carbon fluxes areabundant. However, the knowledge of how much data (data length) and which subset of the time seriesdata (data period) should be used to effectively calibrate the model is still lacking. This study uses theAmeriFlux carbon flux data to parameterize the Terrestrial Ecosystem Model (TEM) with an adjoint-baseddata assimilation technique for various ecosystem types. Parameterization experiments are thus conductedto explore the impact of both data length and data period on the uncertainty reduction of the posteriormodel parameters and the quantification of site and regional carbon dynamics. We find that: the modelis better constrained when it uses two-year data comparing to using one-year data. Further, two-year datais sufficient in calibrating TEM’s carbon dynamics, since using three-year data could only marginallyimprove the model performance at our study sites; the model is better constrained with the data thathave a higher‘‘climate variability’’than that having a lower one. The climate variability is used to measurethe overall possibility of the ecosystem to experience all climatic conditions including drought and extremeair temperatures and radiation; the U.S. regional simulations indicate that the effect of calibration datalength on carbon dynamics is amplified at regional and temporal scales, leading to large discrepanciesamong different parameterization experiments, especially in July and August. Our findings areconditioned on the specific model we used and the calibration sites we selected. The optimal calibrationdata length may not be suitable for other models. However, this study demonstrates that there may exist athreshold for calibration data length and simply using more data would not guarantee a better modelparameterization and prediction. More importantly, climate variability might be an effective indicator ofinformation within the data, which could help data selection for model parameterization. As a result, we believe ourfindings will benefit the ecosystem modeling community in using multiple-year data to improve modelpredictability.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2020

An investigation of Newton-Sketch and subsampled Newton methods

Sketching, a dimensionality reduction technique, has received much attention in the statistics community. In this paper, we study sketching in the context of Newton's method for solving finite-sum optimization problems in which the number of variables and data points are both large. In this work, we study two forms of sketching that perform dimensionality reduction in data space: Hessian subsampling and randomized Hadamard transformations. Each has its own advantages, and their relative tradeoffs have not been investigated in the optimization literature. Additionally, our study focuses on practical versions of the two methods in which the resulting linear systems of equations are solved approximately, at every iteration, using an iterative solver. The advantages of using the conjugate gradient method vs. a stochastic gradient iteration are revealed through a set of numerical experiments, and a complexity analysis of the Hessian subsampling method is presented.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

SU-E-I-73: Clinical Evaluation of CT Image Reconstructed Using Interior Tomography

Purpose: Radiation dose reduction has been a long standing challenge in CT imaging of obese patients. Recent advances in interior tomography (reconstruction of an interior region of interest (ROI) from line integrals associated with only paths through the ROI) promise to achieve significant radiation dose reduction without compromising image quality. This study is to investigate the application of this technique in CT imaging through evaluating imaging quality reconstructed from patient data. Methods: Projection data were directly obtained from patients who had CT examinations in a Dual Source CT scanner (DSCT). Two detectors in a DSCT acquired projection data simultaneously. One detector provided projection data for full field of view (FOV, 50 cm) while another detectors provided truncated projection data for a FOV of 26 cm. Full FOV CT images were reconstructed using both filtered back projection and iterative algorithm; while interior tomography algorithm was implemented to reconstruct ROI images. For comparison reason, FBP was also used to reconstruct ROI images. Reconstructed CT images were evaluated by radiologists and compared with images from CT scanner. Results: The results show that the reconstructed ROI image was in excellent agreement with the truth inside the ROI, obtained from images from CT scanner, and the detailed features in the ROI were quantitatively accurate. Radiologists evaluation shows that CT images reconstructed with interior tomography met diagnosis requirements. Radiation dose may be reduced up to 50% using interior tomography, depending on patient size. Conclusion: This study shows that interior tomography can be readily employed in CT imaging for radiation dose reduction. It may be especially useful in imaging obese patients, whose subcutaneous tissue is less clinically relevant but may significantly increase radiation dose.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2020

Scalable line and plane relaxation in a parallel structured multigrid solver

The efficient solution of sparse, linear systems that arise through the discretization of partial differential equations remains a key challenge for a range of high performance scientific simulations. One approach for reducing data movement and improving performance is by exposing and exploiting structure in a problem through the use of robust structured multilevel solvers. By choosing coarsening that preserves the structure of the problem, these methods maintain efficient structured computation and communication throughout the multigrid hierarchy. However, when coarsening is not permitted to be dependent on the operator, anisotropy must be addressed by the smoother — producing error compatible for coarse-grid correction with structured coarsening. Here, the components required in a scalable parallel structured solver are described with a focus on memory and communication efficiency of robust smoothers. While the implementation of communication and memory reduction techniques in smoothers integrated in a complete 3D solver present a significant engineering challenge, a novel approach is proposed that addresses these challenges systematically through a change to the solver’s execution model. Enabled by user-level threading paired with a set of data and communication abstractions, this approach permits seamless aggregation of communication in plane smoothers — directly reusing code for a 2D distributed multilevel cycle. Results show an effective reduction in communication costs for coarse-grid problems, and result in a speedup of 8.7x in smoothing routines shown in Fig. 12 using this approach. This produces a significant improvement to strong scalability while maintaining favorable weak scaling behavior. Finally, a parallel scaling study using a series of refined meshes is included that demonstrates the effectiveness of this approach in an application of interest.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

SU-E-T-285: Revisiting the Nomogram for Intra-Operative Planning Based Pd-103 Brachytherapy

Purpose: The seed implant technique at our institution involves using a published nomogram for seed ordering based on CT based volume studies of the prostate. Ultrasound volume studies are subsequently used in the operating room for planning a modified peripheral loading with urethra sparing seed implant. The purpose of this study is to determine the appropriate modality for prostate volume measurement and creating an updated nomogram for intra-operative planning specific to our technique for pd-103 brachytherapy for efficient seed ordering. Methods: Prostate volumes based on pre-implant CT (Pre-CT), intra-operative ultrasound (TRUS), and post-implant CT (post-CT) studies as well as the total airkerma strength (AKS) of the implants were analyzed for 135 seed implant cases (69 monotherapy, 66 boost). Regression analysis was performed to derive the relationship between the total AKS and pre-CT and TRUS volumes. The correlation between TRUS and pre-CT volumes and TRUS and post-CT volumes were also studied. Results: Ultrasound based prostate volume exhibited a stronger correlation with total AKS than the pre-implant CT volume (R{sup 2} = 0.97 vs 0.88 for monotherapy and 0.96 vs 0.89 for boost). In general the pre-CT overestimated the prostate volume leading to ordering of a larger number of seeds and thus leading to higher number of unused/wasted seeds. Newly derived TRUS based nomogram was better suited for our technique than the published data. The post-implant CT volume closely followed the ultrasound volume (R{sup 2} = 0.88) as compared to pre-implant CT volumes (R{sup 2} = 0.57). Conclusion: In an era of costconscious health care where waste reduction is of utmost importance, an updated technique-specific nomogram is useful for ordering optimal number of seeds resulting in significant cost savings. In addition, our study shows that ultrasound based prostate volume is a better predictor for seed ordering for intra-operative planning than pre-implant CT.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DOE OSTI2022

Error-Bounded Learned Scientific Data Compression with Preservation of Derived Quantities

Scientific applications continue to grow and produce extremely large amounts of data, which require efficient compression algorithms for long-term storage. Compression errors in scientific applications can have a deleterious impact on downstream processing. Thus, it is crucial to preserve all the “known” Quantities of Interest (QoI) during compression. To address this issue, most existing approaches guarantee the reconstruction error of the original data or primary data (PD), but cannot directly control the problem of preserving the QoI. In this work, we propose a physics-informed compression technique that is composed of two parts: (i) reduction of the PD with bounded errors and (ii) preservation of the QoI. In the first step, we combine tensor decompositions, autoencoders, product quantizers, and error-bounded lossy compressors to bound the reconstruction error at high levels of compression. In the second step, we use constraint satisfaction post-processing followed by quantization to preserve the QoI. To illustrate the challenges of reducing the reconstruction errors of the PD and QoI, we focus on simulation data generated by a large-scale fusion code, XGC, which can produce tens of petabytes in a single day. The results show that our approach can achieve a high compression amount while accurately preserving the QoI within scientifically acceptable bounds.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Machine learning-assisted identification of potential sources of bias in measurements of prompt-fission neutron spectra

Unrecognized sources of uncertainty (USU) can bias the reported mean and/or covariance of experimental nuclear data. These biases, in turn, can propagate through evaluated nuclear data to application simulations or may poorly inform nuclear theory that is fitted to the experimental data. Such unknown sources of bias must be tied to the inherent physical constituents of the measurements such as the characteristics of a detector response or a background reduction technique. Here, in this article, a sparse Bayesian learning model is used to support experts in their efforts to identify and characterize USU in experimental prompt fission neutron spectra (PFNS) for spontaneous fissioning of 252 Cf by linking observed biases to features of the measurement system. Three different bias components were found. The first acts as a verification case for the algorithm as it identifies a bias coming from a well-known source related to the use of 6 Li in the neutron detection system. The second two cases demonstrate how this method can benefit the evaluation of experimental nuclear data by identifying, quantifying, and relating unknown biases to potential causes.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2024

Combining lattice QCD and phenomenological inputs on generalised parton distributions at moderate skewness

Abstract We present a systematic study demonstrating the impact of lattice QCD data on the extraction of generalised parton distributions (GPDs). For this purpose, we use a previously developed modelling of GPDs based on machine learning techniques fulfilling the theoretical requirements of polynomiality, a form of positivity constraint and known reduction limits. A special care is given to estimate the uncertainty stemming from the ill-posed character of the connection between GPDs and the experimental processes usually considered to constrain them, like deeply virtual Compton scattering (DVCS). Moke lattice QCD data inputs are included in a Bayesian framework to a prior model based on an Artificial Neural Network. This prior model is fitted to reproduce the most experimentally accessible information of a phenomenological extraction by Goloskokov and Kroll. We highlight the impact of the precision, correlation and kinematic coverage of lattice data on GPD extraction at moderate $$\xi $$ ξ which has only been brushed in the literature so far, paving the way for a joint extraction of GPDs.

Physics↗

DOE OSTI2026

Ensemble Kalman filter for data assimilation coupled with low-resolution computations techniques applied in fluid dynamics

This paper presents an innovative Reduced-order model (ROM) for merging experimental and simulation data using data assimilation (DA) to estimate the "True" state of a fluid dynamics system, leading to more accurate predictions. Our methodology introduces a novel approach by implementing the ensemble Kalman filter (EnKF) within a reduced-dimensional framework, grounded in a robust theoretical foundation and applied to fluid dynamics. To address the substantial computational demands of DA, the proposed ROM employs low-resolution (LR) techniques to drastically reduce computational costs. This innovative approach involves downsampling datasets for DA computations, followed by an advanced reconstruction technique based on low-cost singular value decomposition (lcSVD). The lcSVD method, a key innovation in this paper, has never been applied to DA before and offers a highly efficient way to enhance resolution with minimal computational resources. Our results demonstrate significant reductions in both computation time and RAM usage through these LR techniques without compromising the accuracy of the estimations. For instance, in a turbulent test case, for a data compression rate of 15.9, the LR approach can achieve a speed-up of 13.7 and a RAM compression of 90.9% while maintaining a low relative root mean square error (RRMSE) of 2.6%, compared to 0.8% in the high-resolution (HR) reference. Furthermore, we highlight the effectiveness of the EnKF in estimating and predicting the state of fluid flow systems based on limited observations and given low-fidelity numerical data. This paper highlights the potential of the proposed DA method in fluid dynamics applications, particularly for improving computational efficiency in CFD and related fields. Its ability to balance accuracy with low computational and memory costs makes it especially suitable for large-scale and real-time applications, such as environmental monitoring or engineering design. This method will be incorporated into ModelFLOWs-app.

Data Assimilation↗

DOE OSTI2025

Final report- UFL - RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2013

Multi-fault Tolerance for Cartesian Data Distributions

Faults are expected to play an increasingly important role in how algorithms and applications are designed to run on future extreme-scale sys- tems. Algorithm-based fault tolerance (ABFT) is a promising approach that involves modications to the algorithm to recover from faults with lower over- heads than replicated storage and a signicant reduction in lost work compared to checkpoint-restart techniques. Fault-tolerant linear algebra (FTLA) algo- rithms employ additional processors that store parities along the dimensions of a matrix to tolerate multiple, simultaneous faults. Existing approaches as- sume regular data distributions (blocked or block-cyclic) with the failures of each data block being independent. To match the characteristics of failures on parallel computers, we extend these approaches to mapping parity blocks in several important ways. First, we handle parity computation for generalized Cartesian data distributions with each processor holding arbitrary subsets of blocks in a Cartesian-distributed array. Second, techniques to handle corre- lated failures, i.e., multiple processors that can be expected to fail together, are presented. Third, we handle the colocation of parity blocks with the data blocks and do not require them to be on additional processors. Several al- ternative approaches, based on graph matching, are presented that attempt to balance the memory overhead on processors while guaranteeing the same fault tolerance properties as existing approaches that assume independent fail- ures on regular blocked data distributions. The evaluation of these algorithms demonstrates that the additional desirable properties are provided by the pro- posed approach with minimal overhead.

Ali, Nawab↗

DOE OSTI2016

Measurement of θ 13 in Double Chooz using neutron captures on hydrogen with novel background rejection techniques

We observed a measurement of the Double Chooz collaboration and the neutrino mixing angle θ 13 using reactor $\bar{v}$ e via the inverse beta decay reaction in which the neutron is captured on hydrogen. Our measurement is based on 462.72 live days data, approximately twice as much data as in the previous such analysis, collected with a detector positioned at an average distance of 1050 m from two reactor cores. Several novel techniques have been developed to achieve significant reductions of the backgrounds and systematic uncertainties. Accidental coincidences, the dominant background in this analysis, are suppressed by more than an order of magnitude with respect to our previous publication by a multi-variate analysis. Furthermore, these improvements demonstrate the capability of precise measurement of reactor $\bar{v}$ e without gadolinium loading. Spectral distortions from the $\bar{v}$ e reactor flux predictions previously reported with the neutron capture on gadolinium events are confirmed in the independent data sample presented here. A value of sin 2 2θ 13 = 0.095 0.039 +0.038 (stat+syst) is obtained from a fit to the observed event rate as a function of the reactor power, a method insensitive to the energy spectrum shape. A simultaneous fit of the hydrogen capture events and of the gadolinium capture events yields a measurement of sin 2 2θ 13 = 0.088 ± 0.033(stat+syst).

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2023

Canonical and noncanonical Hamiltonian operator inference

Here, a method for the nonintrusive and structure-preserving model reduction of canonical and noncanonical Hamiltonian systems is presented. Based on the idea of operator inference, this technique is provably convergent and reduces to a straightforward linear solve given snapshot data and gray-box knowledge of the system Hamiltonian. Examples involving several hyperbolic partial differential equations show that the proposed method yields reduced models which, in addition to being accurate and stable with respect to the addition of basis modes, preserve conserved quantities well outside the range of their training data.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Metabolic kinetic modeling provides insight into complex biological questions, but hurdles remain

Metabolic models containing kinetic information can answer unique questions about cellular metabolism that are useful to metabolic engineering. Several kinetic modeling frameworks have recently been developed or improved. In addition, techniques for systematic identification of model structure, including regulatory interactions, have been reported. Each framework has advantages and limitations, which can make it difficult to choose the most appropriate framework. Common limitations are data availability and computational time, especially in large-scale modeling efforts. However, recently developed experimental techniques, parameter identification algorithms, as well as model reduction techniques help alleviate these computational bottlenecks. Opportunities for additional improvements may come from the rich literature in catalysis and chemical networks. In all, kinetic models are positioned to make significant impact in cellular engineering.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2016

Iterative Monte Carlo analysis of spin-dependent parton distributions

We present a comprehensive new global QCD analysis of polarized inclusive deep-inelastic scattering, including the latest high-precision data on longitudinal and transverse polarization asymmetries from Jefferson Lab and elsewhere. The analysis is performed using a new iterative Monte Carlo fitting technique which generates stable fits to polarized parton distribution functions (PDFs) with statistically rigorous uncertainties. Inclusion of the Jefferson Lab data leads to a reduction in the PDF errors for the valence and sea quarks, as well as in the gluon polarization uncertainty at x ≳ 0.1. Furthermore, the study also provides the first determination of the flavor-separated twist-3 PDFs and the d 2 moment of the nucleon within a global PDF analysis.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2014

Poster — Thur Eve — 06: Dose assessment of cone beam CT imaging protocols as part of SPECT/CT examinations

Purpose: To assess radiation dose from the cone beam CT (CBCT) component of SPECT/CT studies and to compare with other CT examinations performed in our institution. Methods: We used an anthropomorphic chest phantom and the 6 cc ion chamber to measure entrance breast dose for several CBCT and diagnostic CT acquisition protocols. The CBCT effective dose was calculated with ImPACT software; the CT effective dose was evaluated from the DLP value and conversion factor, dependent on the anatomic region. The RADAR medical procedure radiation dose calculator was used to assess the nuclear medicine component of exam dose. Results: The entrance dose to the breast measured with the anthropomorphic phantom was 0.48 mGy and 9.41 mGy for cardiac and chest CBCT scans; and 4.59 mGy for diagnostic thoracic CT. The effective doses were 0.2 mSv, 3.2 mSv and 2.8 mSv respectively. For a small patient represented by the anthropomorphic phantom, the dose from the diagnostic CT was lower than from the CBCT scan, as a result of the exposure reduction options available on modern CT scanners. The CBCT protocols used the same fixed scanning techniques. The diagnostic CT dose based on the patient data was 35% higher than the phantom dose. For most SPECT/CT studies the dose from the CBCT component was comparable with the dose from the radiopharmaceutical. Conclusions: The patient radiation dose from the cone beam CT scan can be higher than that from a diagnostic CT and should be taken into consideration in evaluating total SPECT/CT patient dose.

60 APPLIED LIFE SCIENCES↗