Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data reduction methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Data reduction methods, systems, and devices

Systems, methods, and apparatus are disclosed herein for data pattern detection and data reduction. Devices include an input port configured to receive data values that include a plurality of data units. The devices may also include a data analyzer configured to determine a test statistic based on at least some of the plurality of data units, the test statistic indicating a degree of difference between a first data unit and at least a second data unit, the second data unit being received at the input port before the first data unit. The data analyzer includes the first data unit in a first data block responsive to a determination that the test statistic indicates a low degree of difference, the determination being based on a comparison with a designated difference threshold, the first data block being a same data block as a second data block that includes the second data unit.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

HPDR: High-Performance Portable Scientific Data Reduction Framework

The rapid growth in scientific data generation is outpacing advancements in computing systems necessary for efficient storage, transfer, and analysis, particularly in the context of exascale computing. With the deployment of first-generation exascale computing systems and next-generation experimental facilities, this gap is widening and necessitates effective data reduction techniques to manage enormous data volumes. Over the past decade, various data reduction methods, including lossless compression, error-controlled lossy compression, and data refactoring, have been developed to accelerate I/O in scientific workflows. Despite significant reductions in data volume, these methods introduce considerable computational overhead, which can become the new bottleneck in data processing. To mitigate this, GPU-accelerated data reduction algorithms have been introduced. However, challenges remain in their integration into exascale workflows, including limited portability across different GPU architectures, substantial memory transfer overhead, and reduced scalability on dense multi-GPU systems. To address these challenges, we propose HPDR, a high-performance and portable data reduction framework. HPDR is designed to enable the execution of state-of-the-art reduction algorithms across diverse processor architectures while reducing memory transfer overhead to 2.3 % of the original, resulting in up to 3.5× faster throughput compared to existing solutions. It also achieves up to 96% of the theoretical speedup in multi-GPU settings. In addition, evaluations on accelerating I/O operations at scale up to 1,024 nodes of the Frontier supercomputer demonstrate that HPDR can achieve up to 103 TB/s reduction throughput, providing up to 4× acceleration in parallel I/O performance compared to existing data reduction routines. This work highlights the potential of HPDR to significantly enhance data reduction efficiency in exascale computing environments.

Chen, Jieyang [University of Oregon]↗

Publishing unbinned differential cross section results

Machine learning tools have empowered a qualitatively new way to perform differential cross section measurements whereby the data are unbinned, possibly in many dimensions. Unbinned measurements can enable, improve, or at least simplify comparisons between experiments and with theoretical predictions. Furthermore, many-dimensional measurements can be used to define observables after the measurement instead of before. There is currently no community standard for publishing unbinned data. While there are also essentially no measurements of this type public, unbinned measurements are expected in the near future given recent methodological advances. The purpose of this paper is to propose a scheme for presenting and using unbinned results, which can hopefully form the basis for a community standard to allow for integration into analysis workflows. This is foreseen to be the start of an evolving community dialogue, in order to accommodate future developments in this field that is rapidly evolving.

47 OTHER INSTRUMENTATION↗

POLARIMETRY WITH THE GEMINI PLANET IMAGER: METHODS, PERFORMANCE AT FIRST LIGHT, AND THE CIRCUMSTELLAR RING AROUND HR 4796A

We present the first results from the polarimetry mode of the Gemini Planet Imager (GPI), which uses a new integral field polarimetry architecture to provide high contrast linear polarimetry with minimal systematic biases between the orthogonal polarizations. We describe the design, data reduction methods, and performance of polarimetry with GPI. Point-spread function (PSF) subtraction via differential polarimetry suppresses unpolarized starlight by a factor of over 100, and provides sensitivity to circumstellar dust reaching the photon noise limit for these observations. In the case of the circumstellar disk around HR 4796A, GPI's advanced adaptive optics system reveals the disk clearly even prior to PSF subtraction. In polarized light, the disk is seen all the way in to its semi-minor axis for the first time. The disk exhibits surprisingly strong asymmetry in polarized intensity, with the west side ≳ 9 times brighter than the east side despite the fact that the east side is slightly brighter in total intensity. Based on a synthesis of the total and polarized intensities, we now believe that the west side is closer to us, contrary to most prior interpretations. Forward scattering by relatively large silicate dust particles leads to the strong polarized intensity on the west side, and the ring must be slightly optically thick in order to explain the lower brightness in total intensity there. These findings suggest that the ring is geometrically narrow and dynamically cold, perhaps shepherded by larger bodies in the same manner as Saturn's F ring.

79 ASTRONOMY AND ASTROPHYSICS↗

SPITZER OBSERVATIONS OF ABELL 1763. I. INFRARED AND OPTICAL PHOTOMETRY

We present a photometric analysis of the galaxy cluster Abell 1763 at visible and infrared wavelengths. Included are fully reduced images in r', J, H, and K{sub s} obtained using the Palomar 200in telescope, as well as the IRAC and MIPS images from Spitzer. The cluster is covered out to approximately 3 virial radii with deep 24 {mu}m imaging (a 5{sigma} depth of 0.2 mJy). This same field of {approx}40' x 40' is covered in all four IRAC bands as well as the longer wavelength MIPS bands (70 and 160 {mu}m). The r' imaging covers {approx}0.8 deg{sup 2} down to 25.5 mag, and overlaps with most of the MIPS field of view. The J, H, and K{sub s} images cover the cluster core and roughly half of the filament galaxies, which extend toward the neighboring cluster, Abell 1770. This first, in a series of papers on Abell 1763, discusses the data reduction methods and source extraction techniques used for each data set. We present catalogs of infrared sources (with 24 and/or 70 {mu}m emission) and their corresponding emission in the optical (u', g', r', i', z'), and near- to far-IR (J, H, K{sub s} , IRAC, and MIPS 160 {mu}m). We provide the catalogs and reduced images to the community through the NASA/IPAC Infrared Science Archive.

79 ASTRONOMY AND ASTROPHYSICS↗

Automated detection and analysis of particle beams in laser-plasma accelerator simulations

Numerical simulations of laser-plasma wakefield (particle) accelerators model the acceleration of electrons trapped in plasma oscillations (wakes) left behind when an intense laser pulse propagates through the plasma. The goal of these simulations is to better understand the process involved in plasma wake generation and how electrons are trapped and accelerated by the wake. Understanding of such accelerators, and their development, offer high accelerating gradients, potentially reducing size and cost of new accelerators. One operating regime of interest is where a trapped subset of electrons loads the wake and forms an isolated group of accelerated particles with low spread in momentum and position, desirable characteristics for many applications. The electrons trapped in the wake may be accelerated to high energies, the plasma gradient in the wake reaching up to a gigaelectronvolt per centimeter. High-energy electron accelerators power intense X-ray radiation to terahertz sources, and are used in many applications including medical radiotherapy and imaging. To extract information from the simulation about the quality of the beam, a typical approach is to examine plots of the entire dataset, visually determining the adequate parameters necessary to select a subset of particles, which is then further analyzed. This procedure requires laborious examination of massive data sets over many time steps using several plots, a routine that is unfeasible for large data collections. Demand for automated analysis is growing along with the volume and size of simulations. Current 2D LWFA simulation datasets are typically between 1GB and 100GB in size, but simulations in 3D are of the order of TBs. The increase in the number of datasets and dataset sizes leads to a need for automatic routines to recognize particle patterns as particle bunches (beam of electrons) for subsequent analysis. Because of the growth in dataset size, the application of machine learning techniques for scientific data mining is increasingly considered. In plasma simulations, Bagherjeiran et al. presented a comprehensive report on applying graph-based techniques for orbit classification. They used the KAM classifier to label points and components in single and multiple orbits. Love et al. conducted an image space analysis of coherent structures in plasma simulations. They used a number of segmentation and region-growing techniques to isolate regions of interest in orbit plots. Both approaches analyzed particle accelerator data, targeting the system dynamics in terms of particle orbits. However, they did not address particle dynamics as a function of time or inspected the behavior of bunches of particles. Ruebel et al. addressed the visual analysis of massive laser wakefield acceleration (LWFA) simulation data using interactive procedures to query the data. Sophisticated visualization tools were provided to inspect the data manually. Ruebel et al. have integrated these tools to the visualization and analysis system VisIt, in addition to utilizing efficient data management based on HDF5, H5Part, and the index/query tool FastBit. In Ruebel et al. proposed automatic beam path analysis using a suite of methods to classify particles in simulation data and to analyze their temporal evolution. To enable researchers to accurately define particle beams, the method computes a set of measures based on the path of particles relative to the distance of the particles to a beam. To achieve good performance, this framework uses an analysis pipeline designed to quickly reduce the amount of data that needs to be considered in the actual path distance computation. As part of this process, region-growing methods are utilized to detect particle bunches at single time steps. Efficient data reduction is essential to enable automated analysis of large data sets as described in the next section, where data reduction methods are steered to the particular requirements of our clustering analysis. Previously, we have described the application of a set of algorithms to automate the data analysis and classification of particle beams in the LWFA simulation data, identifying locations with high density of high energy particles. These algorithms detected high density locations (nodes) in each time step, i.e. maximum points on the particle distribution for only one spatial variable. Each node was correlated to a node in previous or later time steps by linking these nodes according to a pruned minimum spanning tree (PMST). We call the PMST representation 'a lifetime diagram', which is a graphical tool to show temporal information of high dense groups of particles in the longitudinal direction for the time series. Electron bunch compactness was described by another step of the processing, designed to partition each time step, using fuzzy clustering, into a fixed number of clusters.

43 PARTICLE ACCELERATORS↗

Fast Semi-automated Filtration Method for Non-targeted LC-QTOF Data of Aged Nitroplasticizer Samples

A full dataset of aged nitroplasticizer (NP) is composed of more than 2000 unique mass-to-charges (m/z) when combining the non-targeted data obtained from both positive and negative electrospray ionization modes in time-of-flight mass spectrometry. Therefore, manual processing of these data often takes days, weeks, or even months to scrutinize for mechanistic insights. To effectively extract meaningful signals that represent vital degradation intermediates in the early NP degradation mechanism, a semi-automated postprocessing workflow for data filtering, tailored to the aging experiment of NP, has been developed. The automated portion of this workflow is written in a Python code (using pandas, numpy, and matplotlib libraries), which removes more than 65% of potential false signals within seconds via four threshold-based adjustable filters: signal sensitivity, coefficient of variation, number of measurements, and retention time variability. As for the manual portion, a pattern-based inspection method is employed to reduce another 23% or more false positives, which greatly simplifies data visualization and results in less than 3% of potential candidate m/z needing in-depth data interpretation. As a positive control, known compounds are verified. Using this semi-automated data reduction method, the amount of time required is reduced to a matter of hours for data filtering in the non-targeted datasets of aged NP, which saves more time and effort for compound identification.

36 MATERIALS SCIENCE↗

Experimental methods for laboratory measurements of helium spectral line broadening in white dwarf photospheres

White Dwarf (WD) stars are the most common stellar remnant in the universe. WDs usually have a hydrogen or helium atmosphere, and helium WD (called DB) spectra can be used to solve outstanding problems in stellar and galactic evolution. DB origins, which are still a mystery, must be known to solve these problems. DB masses are crucial for discriminating between different proposed DB evolutionary hypotheses. Current DB mass determination methods deliver conflicting results. The spectroscopic mass determination method relies on line broadening models that have not been validated at DB atmosphere conditions. We performed helium benchmark experiments using the White Dwarf Photosphere Experiment (WDPE) platform at Sandia National Laboratories' Z-machine that aims to study He line broadening at DB conditions. Using hydrogen/helium mixture plasmas allows investigating the importance of He Stark and van der Waals broadening simultaneously. Accurate experimental data reduction methods are essential to test these line-broadening theories. In this paper, we present data calibration methods for these benchmark He line shape experiments. We give a detailed account of data processing, spectral power calibrations, and instrument broadening measurements. Uncertainties for each data calibration step are also derived. We demonstrate that our experiments meet all benchmark experiment accuracy requirements: WDPE wavelength uncertainties are <1 Å, spectral powers can be determined to within 15%, densities are accurate at the 20% level, and instrumental broadening can be measured with 20% accuracy. Fulfilling these stringent requirements enables WDPE experimental data to provide physically meaningful conclusions about line broadening at DB conditions.

79 ASTRONOMY AND ASTROPHYSICS↗

Expanding Lorentz and spectrum corrections to large volumes of reciprocal space for single-crystal time-of-flight neutron diffraction

Evidence is mounting that potentially exploitable properties of technologically and chemically interesting crystalline materials are often attributable to local structure effects, which can be observed as modulated diffuse scattering (mDS) next to Bragg diffraction (BD). BD forms a regular sparse grid of intense discrete points in reciprocal space. Traditionally, the intensity of each Bragg peak is extracted by integration of each individual reflection first, followed by application of the required corrections. In contrast, mDS is weak and covers expansive volumes of reciprocal space close to, or between, Bragg reflections. For a representative measurement of the diffuse scattering, multiple sample orientations are generally required, where many points in reciprocal space are measured multiple times and the resulting data are combined. The common post-integration data reduction method is not optimal with regard to counting statistics. A general and inclusive data processing method is needed. In this contribution, a comprehensive data analysis approach is introduced to correct and merge the full volume of scattering data in a single step, while correctly accounting for the statistical weight of the individual measurements. Development of this new approach required the exploration of a data treatment and correction protocol that includes the entire collected reciprocal space volume, using neutron time-of-flight or wavelength-resolved data collected at TOPAZ at the Spallation Neutron Source at Oak Ridge National Laboratory.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Data reduction considerations for the burning velocity of spherical constant volume flames of R32 (CH 2 F 2 ) with air

Here, the present work explores data reduction techniques for the measurement of the laminar burning velocities of R32(CH 2 F 2 )-air mixtures using a constant volume combustion device, in which the pressure-time history is the only measured parameter. To allow clear assessment of the accuracy of the data reduction methods, the pressure-time histories used for analysis are synthetically generated via a detailed numerical simulation employing full kinetics and with and without an optically-thin radiation model. Various data reduction models are employed, including a two-zone model and two multi-zone models, and these are compared with the results from the burning velocity obtained from the output of the numerical simulation. The data reduction schemes are shown to be accurate if the same radiation model is employed in the data reduction as was used in the flame simulation to generate the pressure trace used for post-processing. If the incorrect radiation model is employed, however, the errors can be quite large. The effects of stretch, radiation, and different data post-processing methodologies are explored and the errors quantified. Stretch is shown to be important for the early stages and the selected data range that is used for extrapolation has a significant effect on the extrapolated burning velocity. However, with an appropriate choice of data considered for extrapolation, the prediction of the unstretched burning velocity can be quite accurate.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Efficient Data Query for Gaussian Process Compressed Data through Value Range Estimation [Slides]

When the resolution of the data increases, data reduction methods are applied to simulation output, including Gaussian process, neural representation and compression algorithms. Lots of data analysis/visualization techniques requires data query, but data query from reduced representation is still challenging. This report will provide examples and provide possible answers to why data query from reduced representation is still challenging.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Film condensation with high heat fluxes and scaled experiments using pure steam for reactor containment cooling

Condensation tests were performed using a newly developed test facility for scaling the passive containment cooling system (PCCS) to a small modular reactor (SMR). The PCCS of the SMR plays a pivotal role in ensuring greater safety, reliability, and compactness than what is afforded by traditional reactors. Therefore, a well-designed PCCS is essential to SMRs. However, previous studies and test data were unsuitable for scaling, due to high variation in the test geometry and operating conditions. This study intends to close this research gap by using a novel designed scaled test facility consisting of vertical condensing test sections featuring 1-, 2-, and 4-inch-diameter condensing tubes with annular water cooling, and by applying superheated and saturated steam with different steam mass flow ranges of 5–25 g/s. Further, the primary test data, including axial temperatures, mass flow rates, and pressures, were used in conjunction with a standard data reduction method to estimate critical parameters such as heat fluxes, heat transfer coefficients, and condensation rates. These scaled test data would support improving empirical correlations and validating condensation models to identify scaling distortion for SMR PCCSs.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

THE BOLOCAM GALACTIC PLANE SURVEY: SURVEY DESCRIPTION AND DATA REDUCTION

We present the Bolocam Galactic Plane Survey (BGPS), a 1.1 mm continuum survey at 33'' effective resolution of 170 deg{sup 2} of the Galactic Plane visible from the northern hemisphere. The BGPS is one of the first large area, systematic surveys of the Galactic Plane in the millimeter continuum without pre-selected targets. The survey is contiguous over the range -10.5 {<=} l {<=} 90.5, |b| {<=} 0.5. Toward the Cygnus X spiral arm, the coverage was flared to |b| {<=} 1.5 for 75.5 {<=} l {<=} 87.5. In addition, cross-cuts to |b| {<=} 1.5 were made at l= 3, 15, 30, and 31. The total area of this section is 133 deg{sup 2}. With the exception of the increase in latitude, no pre-selection criteria were applied to the coverage in this region. In addition to the contiguous region, four targeted regions in the outer Galaxy were observed: IC1396 (9 deg{sup 2}, 97.5 {<=} l {<=} 100.5, 2.25 {<=} b {<=} 5.25), a region toward the Perseus Arm (4 deg{sup 2} centered on l = 111, b = 0 near NGC 7538), W3/4/5 (18 deg{sup 2}, 132.5 {<=} l {<=} 138.5), and Gem OB1 (6 deg{sup 2}, 187.5 {<=} l {<=} 193.5). The survey has detected approximately 8400 clumps over the entire area to a limiting non-uniform 1{sigma} noise level in the range 11-53 mJy beam{sup -1} in the inner Galaxy. The BGPS source catalog is presented in a previously published companion paper. This paper details the survey observations and data reduction methods for the images. We discuss in detail the determination of astrometric and flux density calibration uncertainties and compare our results to the literature. Data processing algorithms that separate astronomical signals from time-variable atmospheric fluctuations in the data timestream are presented. These algorithms reproduce the structure of the astronomical sky over a limited range of angular scales and produce artifacts in the vicinity of bright sources. Based on simulations, we find that extended emission on scales larger than about 5.'9 is nearly completely attenuated (>90%) and the linear scale at which the attenuation reaches 50% is 3.'8. Comparison with other millimeter-wave data sets implies a possible systematic offset in flux calibration, for which no cause has been discovered. This presentation serves as a companion and guide to the public data release (http://irsa.ipac.caltech.edu/Missions/bolocam.html) through NASA's Infrared Processing and Analysis Center (IPAC) Infrared Science Archive (IRSA). New data releases will be provided through IPAC-IRSA with any future improvements in the reduction. The BGPS provides a complementary long-wavelength spectral band for the ongoing ATLASGAL and Herschel-SPIRE surveys, and an important database and context for imminent observations with SCUBA-2 and ALMA.

79 ASTRONOMY AND ASTROPHYSICS↗

Multilevel Techniques for Compression and Reduction of Scientific Data---The Multivariate Case

We develop a technique for multigrid adaptive reduction of data (MGARD). Special attention is given to the case of tensor product grids, where our approach permits the use of nonuniformly spaced grids in each direction, which can prove problematic for many types of data reduction methods. An important feature of our approach is the provision of guaranteed, computable bounds on the loss incurred by the reduction of the data. Many users are leery of lossy algorithms and will only consider using them provided that numerical bounds on the pointwise difference between the original and the reduced datasets are given. Accordingly, we develop techniques for bounding the loss measured in the $L^{\infty}(\Omega)$ norm, and we show that these bounds are realistic in the sense that they do not significantly overestimate the actual loss. Here, the resulting loss indicators are used to guide the adaptive reduction of the data so that the reduced dataset meets a user-prescribed tolerance or memory constraint. Illustrative numerical examples, including the reduction of data arising from the simulation of a nonlinear reaction-diffusion problem, a turbulent channel flow, and a climate simulation, are provided.

97 MATHEMATICS AND COMPUTING↗

INITIAL DATA RELEASE OF THE KEPLER-INT SURVEY

This paper describes the first data release of the Kepler-INT Survey (KIS) that covers a 116 deg{sup 2} region of the Cygnus and Lyra constellations. The Kepler field is the target of the most intensive search for transiting planets to date. Despite the fact that the Kepler mission provides superior time-series photometry, with an enormous impact on all areas of stellar variability, its field lacks optical photometry complete to the confusion limit of the Kepler instrument necessary for selecting various classes of targets. For this reason, we follow the observing strategy and data reduction method used in the IPHAS and UVEX galactic plane surveys in order to produce a deep optical survey of the Kepler field. This initial release concerns data taken between 2011 May and August, using the Isaac Newton Telescope on the island of La Palma. Four broadband filters were used, U, g, r, i, as well as one narrowband one, H{alpha}, reaching down to a 10{sigma} limit of {approx}20th mag in the Vega system. Observations covering {approx}50 deg{sup 2}, thus about half of the field, passed our quality control thresholds and constitute this first data release. We derive a global photometric calibration by placing the KIS magnitudes as close as possible to the Kepler Input Catalog (KIC) photometry. The initial data release catalog containing around 6 million sources from all the good photometric fields is available for download from the KIS Web site (www.astro.warwick.ac.uk/research/kis/) as well as via MAST (KIS magnitudes can be retrieved using the MAST enhanced target search page http://archive.stsci.edu/kepler/kepler{sub f}ov/search.php and also via Casjobs at MAST Web site http://mastweb.stsci.edu/kplrcasjobs/).

79 ASTRONOMY AND ASTROPHYSICS↗