Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Deep multiscale model learning

The objective of this article is to design novel multi-layer neural networks for multiscale simulations of flows taking into account the observed fine data and physical modeling concepts. Our approaches use deep learning techniques combined with local multiscale model reduction methodologies to predict flow dynamics. Using reduced-order model concepts is important for constructing robust deep learning architectures since the reduced-order models provide fewer degrees of freedom. We consider flow dynamics in porous media as multi-layer networks in this work. More precisely, the solution (e.g., pressures and saturation) at the time instant n+1 depends on the solution at the time instant n and input parameters, such as permeability fields, forcing terms, and initial conditions. One can regard the solution as a multi-layer network, where each layer, in general, is a nonlinear forward map and the number of layers relates to the internal time steps. We will rely on rigorous model reduction concepts to define unknowns and connections between layers. It is critical to use reduced-order models for this purpose, which will identify the regions of influence and the appropriate number of variables. Furthermore, due to the lack of available observed fine data, the reduced-order model can provide us sufficient inexpensive data as needed. The designed deep neural network will be trained using both coarse simulation data which is obtained from the reduced-order model and observed fine data. We will present the main ingredients of our approach and numerical examples. Numerical results show that using deep learning with data generated from multiscale models as well as available observed fine data, we can obtain an improved forward map which can better approximate the fine scale model.

97 MATHEMATICS AND COMPUTING↗

MO-DE-207A-10: One-Step CT Reconstruction for Metal Artifact Reduction by a Modification of Penalized Weighted Least-Squares (PWLS)

Purpose: Metal objects create severe artifacts in kilo-voltage (kV) CT image reconstructions due to the high attenuation coefficients of high atomic number objects. Most of the techniques devised to reduce this artifact utilize a two-step approach, which do not reliably yield the qualified reconstructed images. Thus, for accuracy and simplicity, this work presents a one-step reconstruction method based on a modified penalized weighted least-squares (PWLS) technique. Methods: Existing techniques for metal artifact reduction mostly adopt a two-step approach, which conduct additional reconstruction with the modified projection data from the initial reconstruction. This procedure does not consistently perform well due to the uncertainties in manipulating the metal-contaminated projection data by thresholding and linear interpolation. This study proposes a one-step reconstruction process using a new PWLS operation with total-variation (TV) minimization, while not manipulating the projection. The PWLS for CT reconstruction has been investigated using a pre-defined weight, based on the variance of the projection datum at each detector bin. It works well when reconstructing CT images from metal-free projection data, which does not appropriately penalize metal-contaminated projection data. The proposed work defines the weight at each projection element under the assumption of a Poisson random variable. This small modification using element-wise penalization has a large impact in reducing metal artifacts. For evaluation, the proposed technique was assessed with two noisy, metal-contaminated digital phantoms, against the existing PWLS with TV minimization and the two-step approach. Result: The proposed PWLS with TV minimization greatly improved the metal artifact reduction, relative to the other techniques, by watching the results. Numerically, the new approach lowered the normalized root-mean-square error about 30 and 60% for the two cases, respectively, compared to the two-step method. Conclusion: A new PWLS operation shows promise for improving metal artifact reduction in CT imaging, as well as simplifying the reconstructing procedure.

60 APPLIED LIFE SCIENCES↗

Parallel performance optimizations on unstructured mesh-based simulations

This paper addresses two key parallelization challenges the unstructured mesh-based ocean modeling code, MPAS-Ocean, which uses a mesh based on Voronoi tessellations: (1) load imbalance across processes, and (2) unstructured data access patterns, that inhibit intra- and inter-node performance. Our work analyzes the load imbalance due to naive partitioning of the mesh, and develops methods to generate mesh partitioning with better load balance and reduced communication. Furthermore, we present methods that minimize both inter- and intranode data movement and maximize data reuse. Our techniques include predictive ordering of data elements for higher cache efficiency, as well as communication reduction approaches. We present detailed performance data when running on thousands of cores using the Cray XC30 supercomputer and show that our optimization strategies can exceed the original performance by over 2×. Additionally, many of these solutions can be broadly applied to a wide variety of unstructured grid-based computations.

97 MATHEMATICS AND COMPUTING↗

Multivariate Pointwise Information-Driven Data Sampling and Visualization

With increasing computing capabilities of modern supercomputers, the size of the data generated from the scientific simulations is growing rapidly. As a result, application scientists need effective data summarization techniques that can reduce large-scale multivariate spatiotemporal data sets while preserving the important data properties so that the reduced data can answer domain-specific queries involving multiple variables with sufficient accuracy. While analyzing complex scientific events, domain experts often analyze and visualize two or more variables together to obtain a better understanding of the characteristics of the data features. Therefore, data summarization techniques are required to analyze multi-variable relationships in detail and then perform data reduction such that the important features involving multiple variables are preserved in the reduced data. To achieve this, in this work, we propose a data sub-sampling algorithm for performing statistical data summarization that leverages pointwise information theoretic measures to quantify the statistical association of data points considering multiple variables and generates a sub-sampled data that preserves the statistical association among multi-variables. Using such reduced sampled data, we show that multivariate feature query and analysis can be done effectively. The efficacy of the proposed multivariate association driven sampling algorithm is presented by applying it on several scientific data sets.

97 MATHEMATICS AND COMPUTING↗

Xi-cam : a versatile interface for data visualization and analysis

Xi-cam is an extensible platform for data management, analysis and visualization. Xi-cam aims to provide a flexible and extensible approach to synchrotron data treatment as a solution to rising demands for high-volume/high-throughput processing pipelines. The core of Xi-cam is an extensible plugin-based graphical user interface platform which provides users with an interactive interface to processing algorithms. Plugins are available for SAXS/WAXS/GISAXS/GIWAXS, tomography and NEXAFS data. With Xi-cam 's `advanced' mode, data processing steps are designed as a graph-based workflow, which can be executed live, locally or remotely. Remote execution utilizes high-performance computing or de-localized resources, allowing for the effective reduction of high-throughput data. Xi-cam 's plugin-based architecture targets cross-facility and cross-technique collaborative development, in support of multi-modal analysis. Xi-cam is open-source and cross-platform, and available for download on GitHub.

Pandolfi, Ronald J. (ORCID:0000000308248548)↗

THE ARCONS PIPELINE: DATA REDUCTION FOR MKID ARRAYS

The Array Camera for Optical to Near-IR Spectrophotometry, or ARCONS, is a camera based on Microwave Kinetic Inductance Detectors (MKIDs), a new technology that has the potential for broad application in astronomy. Using an array of MKIDs, the instrument is able to produce time-resolved imaging and low-resolution spectroscopy constructed from detections of individual photons. The arrival time and energy of each photon are recorded in a manner similar to X-ray calorimetry, but at higher photon fluxes. The technique works over a very large wavelength range, is free from fundamental read noise and dark-current limitations, and provides microsecond-level timing resolution. Since the instrument reads out all pixels continuously while exposing, there is no loss of active exposure time to readout. The technology requires a different approach to data reduction compared to conventional CCDs. Here we outline here the prototype data reduction pipeline developed for ARCONS, though many of the principles are also more broadly applicable to energy-resolved photon counting arrays (e.g., transition edge sensors, superconducting tunnel junctions). We describe the pipeline's current status, and the algorithms and techniques employed in taking data from the arrival of photons at the MKID array to the production of images, spectra, and time-resolved light curves.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Integrated top-down process and voxel-based microstructure modeling for Ti-6Al-4V in laser wire direct energy deposition process

Laser-wire metal additive manufacturing (AM) is one of the ideal direct energy deposition (DED) processes for creating large-scale parts with a medium level of complexity. However, the DED process involves complex thermal signatures and wide length scales making the fabrication of realistic AM components and part qualification often reliant on experimental trial-and-error optimization. While experimental measurements over the full volume of a part are valuable and necessary, measuring the entire area of a part is significantly laborious and practically infeasible, particularly for large parts in terms of cost and rapid qualification. Therefore, in this work, we developed an effective thermal and microstructure modeling framework based on the Johnson–Mehl-Avrami-Kolmogorov (JMAK) and Koistinen & Marburger (KM) models through a top-down approach that considers plate distortion-affected thermal profiles. A voxel-by-voxel simulation method is used to predict individual phase fractions of Ti-6Al-4 V. The predicted results were validated through detailed metallurgical measurements. A combined voxel-by-voxel approach with a sparse data reconstruction technique produced a near-perfect reconstruction of the original data. This approach anticipates a significant reduction in data points and computation time and resources. Lastly, we conclude with potential extensions of this work to other modeling efforts.

36 MATERIALS SCIENCE↗

NEAR-INFRARED THERMAL EMISSION DETECTIONS OF A NUMBER OF HOT JUPITERS AND THE SYSTEMATICS OF GROUND-BASED NEAR-INFRARED PHOTOMETRY

We present detections of the near-infrared thermal emission of three hot Jupiters and one brown dwarf using the Wide-field Infrared Camera (WIRCam) on the Canada-France-Hawaii Telescope (CFHT). These include Ks-band secondary eclipse detections of the hot Jupiters WASP-3b and Qatar-1b and the brown dwarf KELT-1b. We also report Y-band, K {sub CONT}-band, and two new and one reanalyzed Ks-band detections of the thermal emission of the hot Jupiter WASP-12b. We present a new reduction pipeline for CFHT/WIRCam data, which is optimized for high precision photometry. We also describe novel techniques for constraining systematic errors in ground-based near-infrared photometry, so as to return reliable secondary eclipse depths and uncertainties. We discuss the noise properties of our ground-based photometry for wavelengths spanning the near-infrared (the YJHK bands), for faint and bright stars, and for the same object on several occasions. For the hot Jupiters WASP-3b and WASP-12b we demonstrate the repeatability of our eclipse depth measurements in the Ks band; we therefore place stringent limits on the systematics of ground-based, near-infrared photometry, and also rule out violent weather changes in the deep, high pressure atmospheres of these two hot Jupiters at the epochs of our observations.

79 ASTRONOMY AND ASTROPHYSICS↗

acdc – Automated Contamination Detection and Confidence estimation for single-cell genome data

Abstract Background A major obstacle in single-cell sequencing is sample contamination with foreign DNA. To guarantee clean genome assemblies and to prevent the introduction of contamination into public databases, considerable quality control efforts are put into post-sequencing analysis. Contamination screening generally relies on reference-based methods such as database alignment or marker gene search, which limits the set of detectable contaminants to organisms with closely related reference species. As genomic coverage in the tree of life is highly fragmented, there is an urgent need for a reference-free methodology for contaminant identification in sequence data. Results We present acdc, a tool specifically developed to aid the quality control process of genomic sequence data. By combining supervised and unsupervised methods, it reliably detects both known and de novo contaminants. First, 16S rRNA gene prediction and the inclusion of ultrafast exact alignment techniques allow sequence classification using existing knowledge from databases. Second, reference-free inspection is enabled by the use of state-of-the-art machine learning techniques that include fast, non-linear dimensionality reduction of oligonucleotide signatures and subsequent clustering algorithms that automatically estimate the number of clusters. The latter also enables the removal of any contaminant, yielding a clean sample. Furthermore, given the data complexity and the ill-posedness of clustering, acdc employs bootstrapping techniques to provide statistically profound confidence values. Tested on a large number of samples from diverse sequencing projects, our software is able to quickly and accurately identify contamination. Results are displayed in an interactive user interface. Acdc can be run from the web as well as a dedicated command line application, which allows easy integration into large sequencing project analysis workflows. Conclusions Acdc can reliably detect contamination in single-cell genome data. In addition to database-driven detection, it complements existing tools by its unsupervised techniques, which allow for the detection of de novo contaminants. Our contribution has the potential to drastically reduce the amount of resources put into these processes, particularly in the context of limited availability of reference species. As single-cell genome data continues to grow rapidly, acdc adds to the toolkit of crucial quality assurance tools.

59 BASIC BIOLOGICAL SCIENCES↗

Speed and Memory Improvements to MCNP6 Delayed-Gamma Line Treatment

The delayed-gamma (DG) line-emission (line) sampling was first introduced into MCNPX in 2005. At the time, storage demands for problems with large numbers of DG lines was a major concern. In order to accommodate large amounts of line data while keeping the memory footprint of such a problem from exceeding memory limits, complicated logic was included to sift through various storage options. In mid-2015 concerns arose over the adequacy of the MCNP6 DG source option (inherited from MCNPX) with line sampling for modeling high-fidelity problems. One indication that there was a problem came when studying the MCNP6 produced DG spectra of U-238 and its progeny using high-fidelity spectroscopic tallies when compared to DG line amplitudes directly from ENDF/B-VII.1 emission data. Despite producing a correct integral number of gammas, the comparison to ENDF data showed a broadening in the energy lines and a reduction in the amplitude of more prominent lines. It was later found that the cause of this broadening was one of the memory-saving techniques used to store DG line data when problems required many DG lines. Specifically, the DG lines were integrated into fine energy bins for sampling. When a DG line was to be sampled, a random number would be used to choose which energy bin to sample based on emission probability. Once chosen, a second random number was used to perform nearest-line sampling in that bin. The integration of lines led to an oversampling of low-probability lines, especially when in close proximity to high-probability lines. This treatment also caused a similar issue with the time-dependent behavior of DG line emission which occurred when low-probability lines were sampled near high probability lines with the former decay constant being used. The effect on time-dependent gamma production was demonstrated in analytic benchmarks performed by Weldon et al.. To address these issues a new gamma line storage and sampling treatment was introduced in 2015. This new treatment, stores gamma emission data in a line-by-line CDF which is sampled directly, thereby resolving both the line-broadening issue, and the time-dependence issue as was demonstrated by Tutt el al.. In addition, the new DG sampling treatment was written to use dynamically dimensioned storage arrays as opposed to large fixed-dimensioned arrays, as in the former treatment, and also makes more efficient use of line data that is stored by reducing or eliminating repeated data. As a result, storage arrays are generated only for DG data relevant to the problem being executed. The result should be a reduction in both memory requirements and runtime. However, these reductions could not be realized until the logic for the old treatment could be removed. This paper provides an update on recent work which has focused on the removal of the old DG sampling logic and attempts to quantify the memory reduction and speed up of selected problems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Deep Learning-based Parameterization of Complex 3D CO2 Saturation Data in Large-scale Geological Carbon Storage

In deep learning (DL), dimension reduction plays a pivotal role in improving training efficiency and minimizing overfitting, especially when working with complex datasets like three-dimensional (3D) saturation data. In the context of geological carbon storage (GCS), 3D saturation data introduces unique challenges due to its sparse nature and sharp transitions at plume boundaries, known as shock fronts. To tackle these challenges, we developed a novel DL framework that combines dimension reduction with advanced 3D reconstruction techniques. Our approach utilizes latent variables derived from 2D average saturation fields to efficiently capture the essential features of high-dimensional data while reducing the number of variables. This enhances both the robustness and accuracy of DL models, making the framework more practical for real-world applications. By offering a tailored solution for modeling complex 3D saturation dynamics, this framework holds significant potential for environmental monitoring, energy storage, and other geological applications.

Wang, Hongsheng [University of Texas at Austin]↗

Rapid detection of rare events from in situ X-ray diffraction data using machine learning

High-energy X-ray diffraction methods can non-destructively map the 3D microstructure and associated attributes of metallic polycrystalline engineering materials in their bulk form. These methods are often combined with external stimuli such as thermo-mechanical loading to take snapshots of the evolving microstructure and attributes over time. However, the extreme data volumes and the high costs of traditional data acquisition and reduction approaches pose a barrier to quickly extracting actionable insights and improving the temporal resolution of these snapshots. This article presents a fully automated technique capable of rapidly detecting the onset of plasticity in high-energy X-ray microscopy data. The technique is computationally faster by at least 50 times than the traditional approaches and works for data sets that are up to nine times sparser than a full data set. This new technique leverages self-supervised image representation learning and clustering to transform massive data sets into compact, semantic-rich representations of visually salient characteristics ( e.g. peak shapes). These characteristics can rapidly indicate anomalous events, such as changes in diffraction peak shapes. It is anticipated that this technique will provide just-in-time actionable information to drive smarter experiments that effectively deploy multi-modal X-ray diffraction methods spanning many decades of length scales.

Zheng, Weijian↗

AIACHNE's contribution for Nuclear Energy Agency Working Party on International Nuclear Data Evaluation Co-operation Subgroup 50

The AIACHNE (AI/ML Informed cAlifornium CHi Nuclear data Experiment) project aims at designing an experiment for the 252 Cf Prompt Fission Neutron Spectrum (PFNS) that explores systematic biases in an experimental database retrieved from the EXFOR databases. To that end, machine learning (ML) methods were applied to pint-point measurement features likely related to bias. From that information, we selected a feature that should be explored by the AIACHNE experiment. Measurement features are metadata encapsulating all pertinent information about the physical measurement and analysis techniques. Examples are, for instance, what neutron and fission detectors were used for the physical metadata, and what background reduction techniques were employed for analysis techniques. Such metadata were retrieved both from EXFOR entries as well as the literature of data sets described in detail in Reference 2 (at the end of the article).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

AIACHNE's contribution for Nuclear Energy Agency Working Party on International Nuclear Data Evaluation Co-operation Subgroup 50

The AIACHNE (AI/ML Informed cAlifornium CHi Nuclear data Experiment) project aims at designing an experiment for the 252 Cf Prompt Fission Neutron Spectrum (PFNS) that explores systematic biases in an experimental database retrieved from the EXFOR databases. To that end, machine learning (ML) methods were applied to pint-point measurement features likely related to bia. From that information, we selected a feature that should be explored by the AIACHNE experiment. Measurement features are metadata encapsulating all pertinent information about the physical measurement and analysis techniques. Examples are, for instance, what neutron and fission detectors were used for the physical metadata, and what background reduction techniques were employed for analysis techniques. Such metadata were retrieved both from EXFOR entries as well as the literature of data sets described in detail in Ref. [2]. The prerequisite for applying machine learning techniques is casting the metadata into a format that can be parsed by the algorithm. This step might seem trivial but requires to find a unique language where metadata that carry the same physics meaning across several experiments must have the same identifier. One example is, for instance, the neutron detector. As seen in Figure 1, the machine learning code identified the use of 6 Li detectors as being related to bias in some datasets of the AIACHNE 252 Cf PFNS experimental database. In fact, here are several experiments that used neutron detectors containing 6Li in the database, for instance for the example below. EXFOR format has a unique keywords describing detectors such as “SCIN” or “GLASD”. One may think that these keywords are already sufficient descriptors for ML to uniquely find an issue. However, “SCIN” (used for [3, 4]) and “GLASD” (used for [5]) fail to inform the algorithm what is the active material in the detector. And, the key common issue leading to bias in 252 Cf related to neutron detectors is not whether it is a glass detector or a scintillator. No, the issue is that 6 Li was within both detector types and that even small mistakes in the detector response functions around approximately 200 keV are amplified by the 6 Li(n,α) resonance there leading to bias in data as highlighted in Fig. 1 and Ref. [1]. Hence, the features describing the neutron detector must call out the active material in the detector, rather than the existing EXFOR detector keyword, that the ML algorithm can find physically meaningful features related to bias. The AIACHNE team used a precursor of the WPEC (Working Party on International Nuclear Data Evaluation Co-operation) SG(Subgroup)-50 format to store the metadata for the ML analysis.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

On single-crystal total scattering data reduction and correction protocols for analysis in direct space

Data reduction and correction steps and processed data reproducibility in the emerging single-crystal total-scattering-based technique of three-dimensional differential atomic pair distribution function (3D-ΔPDF) analysis are explored. All steps from sample measurement to data processing are outlined using a crystal of CuIr 2 S 4 as an example, studied in a setup equipped with a high-energy X-ray beam and a flat-panel area detector. Computational overhead as pertains to data sampling and the associated data-processing steps is also discussed. Various aspects of the final 3D-ΔPDF reproducibility are explicitly tested by varying the data-processing order and included steps, and by carrying out a crystal-to-crystal data comparison. Situations in which the 3D-ΔPDF is robust are identified, and caution against a few particular cases which can lead to inconsistent 3D-ΔPDFs is noted. Although not all the approaches applied herein will be valid across all systems, and a more in-depth analysis of some of the effects of the data-processing steps may still needed, the methods collected herein represent the start of a more systematic discussion about data processing and corrections in this field.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Fault Detection and Diagnosis In Hall–Héroult Cells Based on Individual Anode Current Measurements Using Dynamic Kernel PCA

Individual anode current signals in aluminum reduction cells provide localized cell conditions in the vicinity of each anode, which contain more information than the conventionally measured cell voltage and line current. One common use of this measurement is to identify process faults that can cause significant changes in the anode current signals. While this method is simple and direct, it ignores the interactions between anode currents and other important process variables. This paper presents an approach that applies multivariate statistical analysis techniques to individual anode currents and other process operating data, for the detection and diagnosis of local process abnormalities in aluminum reduction cells. Specifically, since the Hall–Héroult process is time-varying with its process variables dynamically and nonlinearly correlated, dynamic kernel principal component analysis with moving windows is used. The cell is discretized into a number of subsystems, with each subsystem representing one anode and cell conditions in its vicinity. The fault associated with each subsystem is identified based on multivariate statistical control charts. The results show that the proposed approach is able to not only effectively pinpoint the problematic areas in the cell, but also assess the effect of the fault on different parts of the cell.

36 MATERIALS SCIENCE↗

Sky Residual Correction

The near-infrared is populated by numerous emission lines radiated from the Earth’s atmosphere, and these emission lines are often several orders of magnitude brighter in intensity than the typical astrophysical science target. The subtraction of these emission lines, sky subtraction, can create large systematic errors in ground-based astronomical spectra effectively limiting the number of usable resolution elements. A more effective sky subtraction, and the reduction of the systematic errors due to the sky subtraction process, is a major hurdle which ground-based astronomy must overcome to increase the amount of usable spectrum, and to be able to observe fainter scientific targets. Large high-quality data sets such as the Sloan Digital Sky Survey (SDSS) in themselves present opportunities for a reduction of the systematic sky subtraction errors through self-calibration. The sky residual correction method of Wild and Hewett is one such self-calibration technique which uses principal component analysis to reduce the systematic sky residual errors present in SDSS spectra. We apply sky residual corrections to the SDSS, Baryon Oscillation Spectroscopic Survey, and Apache Point Observatory Galactic Evolution Experiment data sets to optimize the number of subtraction components, and to quantify the reduction of the systematic errors in the science spectra. Finally as a proof of concept we use the sky-residual-corrected SDSS luminous red galaxy spectra to search for gravitationally lensed emission line Galaxies.

79 ASTRONOMY AND ASTROPHYSICS↗

Review of non destructive assay techniques utilised for decommissioning of plutonium facilities at AWE - 14418

Over the past fifteen years AWE has decommissioned and size reduced a large number of glove boxes that contained a wide spectrum of equipment that ranged from highly radiation attenuating structures, such as furnaces and lathes, to other less radiation attenuating apparatus, such as small discrete tools or simple work benches. In order to generate meaningful glove box residual fissile material inventory data, which can constructively inform the selection of measures needed to underwrite the criticality safety of size reduction and consignment to waste activities, several different types of Non Destructive Assay (NDA) measurement techniques are available at AWE. These are deployed for the estimate of the residual fissile material inventory prior to decommissioning activities and for the final NDA of waste containers, post decommissioning. The techniques range from the Decommissioning In-Situ Plutonium Inventory Monitor (DISPIM), through other Neutron Coincidence Counting (NCC) techniques (Neutron Panel monitoring) and gamma measurements to consideration of Thermo Luminescent Detector (TLD) data. This paper will present a comparison of the fissile material identified in the waste generated post decommissioning operations to the NDA estimate of the residual fissile material inventory for the glove boxes, as well as provide a direct comparison between the different NDA techniques available at AWE. The findings to date indicate that the glove box residual fissile material inventory used for criticality safety purposes bounds the decommissioning process, subject to the suitable deployment of the NDA technique and the inclusion of appropriate uncertainties. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗