Search NASA⌕ Search

SEARCH · Search NASA

Results for “reduction data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

The Spectroscopic Classification of Astronomical Transients (SCAT) Survey: Overview, Pipeline Description, Initial Results, and Future Plans

Here, we present the Spectroscopic Classification of Astronomical Transients (SCAT) survey, which is dedicated to spectrophotometric observations of transient objects such as supernovae and tidal disruption events. SCAT uses the SuperNova Integral-Field Spectrograph (SNIFS) on the University of Hawai’i 2.2 m (UH2.2m) telescope. SNIFS was designed specifically for accurate transient spectrophotometry, including absolute flux calibration and host-galaxy removal. We describe the data reduction and calibration pipeline including spectral extraction, telluric correction, atmospheric characterization, nightly photometricity, and spectrophotometric precision. We achieve ≲5% spectrophotometry across the full optical wavelength range (3500–9000 Å) under photometric conditions. The inclusion of photometry from the SNIFS multi-filter mosaic imager allows for decent spectrophotometric calibration (10%–20%) even under unfavorable weather/atmospheric conditions. SCAT obtained ≈640 spectra of transients over the first 3 yr of operations, including supernovae of all types, active galactic nuclei, cataclysmic variables, and rare transients such as superluminous supernovae and tidal disruption events. These observations will provide the community with benchmark spectrophotometry to constrain the next generation of hydrodynamic and radiative transfer models.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Keck Infrared Transient Survey. I. Survey Description and Data Release 1

Abstract We present the Keck Infrared Transient Survey, a NASA Key Strategic Mission Support program to obtain near-infrared (NIR) spectra of astrophysical transients of all types, and its first data release, consisting of 105 NIR spectra of 50 transients. Such a data set is essential as we enter a new era of IR astronomy with the James Webb Space Telescope (JWST) and the upcoming Nancy Grace Roman Space Telescope (Roman). NIR spectral templates will be essential to search JWST images for stellar explosions of the first stars and to plan an effective Roman SN Ia cosmology survey, both key science objectives for mission success. Between 2022 February and 2023 July, we systematically obtained 274 NIR spectra of 146 astronomical transients, representing a significant increase in the number of available NIR spectra in the literature. Here, we describe the first release of data from the 2022A semester. We systematically observed three samples: a flux-limited sample that includes all transients <17 mag in a red optical band (usually ZTF r or ATLAS o bands); a volume-limited sample including all transients within redshift z < 0.01 ( D ≈ 50 Mpc); and an SN Ia sample targeting objects at phases and light-curve parameters that had scant existing NIR data in the literature. The flux-limited sample is 39% complete (60% excluding SNe Ia), while the volume-limited sample is 54% complete and is 79% complete to z = 0.005. Transient classes observed include common Type Ia and core-collapse supernovae, tidal disruption events, luminous red novae, and the newly categorized hydrogen-free/helium-poor interacting Type Icn supernovae. We describe our observing procedures and data reduction using PypeIt , which requires minimal human interaction to ensure reproducibility.

Tinyanont, S. (ORCID:0000000214814676)↗
DOE OSTI2024

Smart pixel sensors: towards on-sensor filtering of pixel clusters with deep learning

Highly granular pixel detectors allow for increasingly precise measurements of charged particle tracks. Next-generation detectors require that pixel sizes will be further reduced, leading to unprecedented data rates exceeding those foreseen at the High- Luminosity Large Hadron Collider. Signal processing that handles data incoming at a rate of $\mathcal{O}$(40 MHz) and intelligently reduces the data within the pixelated region of the detector at rate will enhance physics performance at high luminosity and enable physics analyses that are not currently possible. Using the shape of charge clusters deposited in an array of small pixels, the physical properties of the traversing particle can be extracted with locally customized neural networks. In this first demonstration, we present a neural network that can be embedded into the on-sensor readout and filter out hits from low momentum tracks, reducing the detector's data volume by 57.1%–75.7%. The network is designed and simulated as a custom readout integrated circuit with 28 nm CMOS technology and is expected to operate at less than 300 μW with an area of less than 0.2 mm 2 . The temporal development of charge clusters is investigated to demonstrate possible future performance gains, and there is also a discussion of future algorithmic and technological improvements that could enhance efficiency, data reduction, and power per area.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

SHαDE: survey description and mass–kinematics scaling relations for dwarf galaxies

ABSTRACT The Study of H α from Dwarf Emissions (SHαDE) is a high spectral resolution (R = 13 500) H α integral field survey of 69 dwarf galaxies with stellar masses 106 < M⋆ < 109 M⊙. The survey used FLAMES on the ESO Very Large Telescope. SHαDE is designed to study the kinematics and stellar populations of dwarf galaxies using consistent methods applied to massive galaxies and at matching level of detail, connecting these mass ranges in an unbiased way. In this paper, we set out the science goals of SHαDE, describe the sample properties, outline the data reduction, and analysis processes. We investigate the log M⋆–log S0.5 mass–kinematics scaling relation, which has previously shown potential for combining galaxies of all morphologies in a single scaling relation. We extend the scaling relation from massive galaxies to dwarf galaxies, demonstrating this relation is linear down to a stellar mass of M⋆ ∼ 108.6 M⊙. Below this limit, the kinematics of galaxies inside one effective radius appears to be dominated by the internal velocity dispersion limit of the H α-emitting gas, giving a bend in the log M⋆–log S0.5 relation. Replacing stellar mass with total baryonic mass using gas mass estimate reduces the severity but does not remove the linearity limit of the scaling relation. An extrapolation to estimate the galaxies’ dark matter halo masses, yields a log Mh–log S0.5 scaling relation that is free of any bend, has reduced curvature over the whole mass range, and brings galaxies of all masses and morphologies on to the virial relation.

Barat, Dilyar↗
DOE OSTI2024

Chemical evolution of local post-starburst galaxies: implications for the mass–metallicity relation

ABSTRACT We use the stellar fossil record to constrain the stellar metallicity evolution and star-formation histories of the post-starburst (PSB) regions within 45 local PSB galaxies from the MaNGA survey. The direct measurement of the regions’ stellar metallicity evolution is achieved by a new two-step metallicity model that allows for stellar metallicity to change at the peak of the starburst. We also employ a Gaussian process noise model that accounts for correlated errors introduced by the observational data reduction or inaccuracies in the models. We find that a majority of PSB regions (69 per cent at >1σ significance) increased in stellar metallicity during the recent starburst, with an average increase of 0.8 dex and a standard deviation of 0.4 dex. A much smaller fraction of PSBs are found to have remained constant (22 per cent) or declined in metallicity (9 per cent, average decrease 0.4 dex, standard deviation 0.3 dex). The pre-burst metallicities of the PSB galaxies are in good agreement with the mass–metallicity (MZ) relation of local star-forming galaxies. These results are consistent with hydrodynamic simulations, which suggest that mergers between gas-rich galaxies are the primary formation mechanism of local PSBs, and rapid metal recycling during the starburst outweighs the impact of dilution by any gas inflows. The final mass-weighted metallicities of the PSB galaxies are consistent with the MZ relation of local passive galaxies. Our results suggest that rapid quenching following a merger-driven starburst is entirely consistent with the observed gap between the stellar mass–metallicity relations of local star-forming and passive galaxies.

Leung, Ho-Hin (ORCID:0000000304865178)↗
DOE OSTI2021

Quantitative texture analysis using the NOMAD time-of-flight neutron diffractometer

Strategies for efficient and reliable texture measurements have been explored using the Nanoscale Ordered Materials Diffractometer (NOMAD) at the Spallation Neutron Source located at Oak Ridge National Laboratory (ORNL). To test these strategies, the texture of an Al alloy was also investigated using another neutron diffraction instrument, a constant-wavelength neutron diffractometer (NRSF2) located at the High Flux Isotope Reactor, also at ORNL. Reasonable agreement was found across the two experimental methods, but differences in overall texture strength and the symmetry of some components were noted, depending on the data reduction and analysis method selected. Finally, on the basis of these results, potential improvements are identified which would enhance the texture measurement capability on NOMAD.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Towards scanning nanostructure X-ray microscopy

This article demonstrates spatial mapping of the local and nanoscale structure of thin film objects using spatially resolved pair distribution function (PDF) analysis of synchrotron X-ray diffraction data. This is exemplified in a lab-on-chip combinatorial array of sample spots containing catalytically interesting nanoparticles deposited from liquid precursors using an ink-jet liquid-handling system. A software implementation is presented of the whole protocol, including an approach for automated data acquisition and analysis using the atomic PDF method. The protocol software can handle semi-automated data reduction, normalization and modeling, with user-defined recipes generating a comprehensive collection of metadata and analysis results. By slicing the collection using included functions, it is possible to build images of different contrast features chosen by the user, giving insights into different aspects of the local structure.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2026

Correlation-aware binning for small-angle neutron scattering via Gaussian-process inference

Binning in small-angle neutron scattering (SANS) is typically performed empirically, with fixed parameters chosen for convenience rather than statistical optimality. Such practices often fail to balance statistical precision and spatial resolution, leading to inconsistencies across instruments and datasets. Here we establish a correlation-aware framework that determines the optimal bin width from first principles by extending the classical Freedman–Diaconis (FD) rule to account for inter-bin correlations with a Gaussian process. In this formulation, the scattering intensity is treated as a smooth stochastic field whose statistical coherence is described by a covariance matrix. Analytical expressions of errors derived from this model yield closed-form criteria that separate the total deviation into contributions from counting noise, aliasing distortion and curvature-dependent correlation effects. Expressed in reduced variables, the resulting dimensionless error surface reveals a continuous transition from the uncorrelated FD regime to the correlation-dominated limit, providing a unified description of noise suppression and resolution control. Because the formulation depends only on the profile characteristics of scattering intensity I(Q), specifically its average intensity and first- and second-order derivatives, it applies generally to any SANS measurement regardless of sample, instrument or geometry. Experimental validation using small- and ultra-small-angle neutron scattering data confirms the predicted scaling behavior, demonstrating that correlation-aware inference systematically reduces mean-squared error and enables information-efficient reproducible data reduction across materials and instruments.

Tung, Chi-Huan [ORNL] (ORCID:0000000221972074)↗
DOE OSTI2023

Lorentz factor for time-of-flight neutron Bragg and total scattering

We report the three fundamental origins of the Lorentz factor for neutron time-of-flight powder diffraction are revisited. A detailed derivation of the Lorentz factor is presented in the context of diffuse scattering modelling in reciprocal space when perfect periodicity is assumed, and the total scattering pattern is constructed in its discrete form – the factor in this case becomes 1/Q 2 (or d 2 ). Discussion is also presented with respect to practical data reduction where a vanadium measurement is usually taken as the normalization factor (to account for various factors such as detector efficiency), and it is shown that the existence of the Lorentz factor is independent of such a normalization process.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Towards the spatial resolution of metalloprotein charge states by detailed modeling of XFEL crystallographic diffraction

Oxidation states of individual metal atoms within a metalloprotein can be assigned by examining X-ray absorption edges, which shift to higher energy for progressively more positive valence numbers. Indeed, X-ray crystallography is well suited for such a measurement, owing to its ability to spatially resolve the scattering contributions of individual metal atoms that have distinct electronic environments contributing to protein function. However, as the magnitude of the shift is quite small, about +2 eV per valence state for iron, it has only been possible to measure the effect when performed with monochromated X-ray sources at synchrotron facilities with energy resolutions in the range 2–3 × 10 −4 (Δ E / E ). This paper tests whether X-ray free-electron laser (XFEL) pulses, which have a broader bandpass (Δ E / E = 3 × 10 −3 ) when used without a monochromator, might also be useful for such studies. The program nanoBragg is used to simulate serial femtosecond crystallography (SFX) diffraction images with sufficient granularity to model the XFEL spectrum, the crystal mosaicity and the wavelength-dependent anomalous scattering factors contributed by two differently charged iron centers in the 110-amino-acid protein, ferredoxin. Bayesian methods are then used to deduce, from the simulated data, the most likely X-ray absorption curves for each metal atom in the protein, which agree well with the curves chosen for the simulation. The data analysis relies critically on the ability to measure the incident spectrum for each pulse, and also on the nanoBragg simulator to predict the size, shape and intensity profile of Bragg spots based on an underlying physical model that includes the absorption curves, which are then modified to produce the best agreement with the simulated data. This inference methodology potentially enables the use of SFX diffraction for the study of metalloenzyme mechanisms and, in general, offers a more detailed approach to Bragg spot data reduction.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors

Today's scientific simulations require a significant reduction of data volume because of extremely large amounts of data they produce and the limited I/O bandwidth and storage space. Here, error-bounded lossy compression has been considered one of the most effective solutions to the above problem. In practice, however, the best-fit compression method often needs to be customized or optimized in particular because of diverse characteristics in different datasets and various user requirements on the compression quality and performance. In this paper, we address this issue with a novel modular, composable compression framework named SZ3. Our contributions are four-folds. (1) We develop SZ3 which features an innovative modular abstraction for the prediction-based compression framework, such that compression modules can be plugged in easily to create new compressors based on characteristics of data and user requirements. (2) We create a new compression pipeline by SZ3 for GAMESS data, which significantly improves the compression ratios over state-of-the-art compressors. (3) We develop an adaptive compression pipeline by SZ3 for APS data with minimal efforts, which leads to the best rate-distortion among all existing error-bounded lossy compressors for any bit-rate. (4) We compare the sustainability of SZ3 with leading error-bounded prediction-based compressors, and then demonstrate the necessity of diverse pipelines by integrating and evaluating several compression pipelines on diverse scientific datasets from multiple disciplines. Experiments show that SZ3 incurs very limited overhead in compressor integration and our customized compression pipelines lead to up to 20% improvement in compression ratios under the same data distortion, when compared with the best existing approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Measurement of Convective Heat Transfer Coefficients With Supercritical CO 2 Using the Wilson-Plot Technique

This paper describes the measurement of convective heat transfer coefficients and friction factors for sCO 2 flowing in a smooth tube and compares the results with published correlations for validation. The paper also describes the Heat Exchange and Experimental Testing (HEET) rig recently designed and built at the U.S. Department of Energy’s (DoE’s) National Energy Technology Laboratory (NETL) in Morgantown, WV. Here, the Wilson-plot technique used for measuring the heat transfer coefficients is described along with the data reduction process. The Wilson-plot technique was chosen as the basis for the design of NETL’s HEET rig. Advantages of the Wilson-plot technique include the (1) ability to measure high convective heat transfer coefficients accurately, (2) ability to measure average heat transfer coefficient for complicated heat exchange geometries like those produced using additive manufacturing, (3) ability to measure heat transfer coefficients on both sides of a heat exchanger independently, and (4) simplicity of experimental setup. Capabilities of the HEET rig include pressure to 24 MPa (3500 psig), temperature to 538 °C (1000 °F), mass flow rate to 1.5 kg/s (3 lb/s), and Re to 500,000. The rig is designed to operate with pure CO 2 or a mixture of CO 2 and up to 10% N2 by volume to study the impact of gas mixtures typical of direct-fired sCO 2 power cycles on the convective heat transfer and pressure drop. Preliminary tests in the HEET rig were performed with smooth stainless-steel tube and pure CO 2 , and the results were compared with published correlations for Nusselt number (Nu) and friction factor. Over a Reynolds number (Re) range from 58,000 to 228,000, measured Nu was compared to predictions using the Dittus and Boelter equation (Kreith and Bohn, 1993, “Principles of Heat Transfer, West Publishing Company”) within 5% and measured friction factors were compared to predictions using the McAdams correlation (“McAdams, 1954, “Heat Transmission,” 3rd ed., McGraw Hill, New York)” for smooth tube to be within 5%.

42 ENGINEERING↗
DOE OSTI2020

Cross-Cutting Software Solutions in Support of Experimental Analysis Challenges at National Scattering Facilities

Rapid improvements in instrumentation at neutron and light source combined with new capabilities in computing, data analytics, and machine learning offer unprecedented scientific capabilities to address fundamental and applied science questions in a wide area of subjects from quantum materials to biological systems. In order to make these capabilities accessible to the broader scientific community, significant developments are needed related to workflow spanning from experimental control to advanced materials simulations as well as the underpinning scientific software. In this chapter, we explore these needs and recent developments at the Spallation Neutron Source (SNS) at Oak Ridge National Laboratory. A central part of the workflow at the SNS is the data reduction framework Mantid which is developed as a partnership between European and US neutron-scattering facilities.

Proffen, Thomas↗
DOE OSTI2025

DEDUPKV: A Space-Efficient and High-Performance Key-Value Store via Fine-Grained Deduplication

Log-Structured Merge Tree (LSM-tree) based key-value stores excel in write-intensive environments but suffer from data duplication, consuming up to 49% of storage space in LSM-tree-based key-value store deployments. Traditional solutions like compression and coarse-grained file system-level deduplication introduce overhead or have limited effectiveness. In this study, we propose DedupKV, a fine-grained deduplication framework tailored for LSM-tree, maximizing data reduction efficiency while minimizing write stalls and read overheads. DedupKV features three key innovations: (1) FLUSH-integrated inline deduplication, which removes duplicates during memory-to-storage writes; (2) WAL file-based offline deduplication, repurposing write-ahead logs to avoid double writes; and (3) elastic execution, dynamically balancing inline and offline deduplication based on memory pressure and workload intensity. Additionally, dynamic granularity management reduces deduplication metadata overhead. We implemented these four ideas in RocksDB for the first time and conducted experiments in a Linux environment. Our evaluation shows that WAL file-based offline deduplication and DedupKV outperform BlobDB by 33% and 23%, respectively, in write-heavy workloads, while reducing write amplification by 1.2 ×, 2 ×, and 1.6 × for real KV datasets.

Jamil, Safdar [Sogang University]↗
DOE OSTI2025

Enabling Command-and-Control in Advanced In Situ Workflows

Scientific discovery is progressing towards autonomous science with the combination of scientific instruments, high-performance computing, and artificial intelligence in complex workflows. This evolution introduces new requirements for managing scientific workflows, including feedback loops, near real-time constraints, and the ability to dynamically control workflow execution. In situ workflows that analyze and visualize data as it is generated are well-suited to satisfy stringent time constraints and their iterative nature offers greater opportunities for command-and-control. However, only a few of the many workflow management systems available have been specifically designed to manage in situ workflows and often lack support for automated feedback loops that allow analysis and visualization components to interact with the main scientific data producer. To address this need, we present in this paper how to add command-and-control capabilities to a workflow management system. We identify the functional design requirements of such a command-and-control system, detail its architecture, interface, and core mechanisms, and illustrate how advanced in situ workflows can leverage command-and-control in three use cases: graceful termination with checkpoint, dynamic and adaptive data reduction, and event-triggered analysis.

Mehta, Kshitij [ORNL] (ORCID:0000000297149981)↗
DOE OSTI2020

ADREM

Analytical Data Reduction Excel Macro

Hobbs, Ian↗
DOE OSTI2026

pySimpleMask

SF-26-118 pySimpleMask is a tool for creating masks and Q-partition maps for X-ray scattering patterns, supporting SAXS, WAXS, and XPCS data reduction. It ships both a desktop GUI and a headless Python API that can drive the full pipeline from scripts.

Chu, Miaoqi [Argonne National Laboratory (ANL), Ar↗
DOE OSTI2024

ZFP: A compressed array representation for numerical computations

HPC trends favor algorithms and implementations that reduce data motion relative to FLOPS. We investigate the use of lossy compressed data arrays in place of traditional IEEE floating point arrays to store the primary data of calculations. Simulation is fundamentally an exercise in controlled approximation, and error introduced by finite-precision arithmetic (or lossy compression) is just one of several sources of error that need to be managed to ensure sufficient accuracy in a computed result. We describe ZFP, a compressed numerical format designed for in-memory storage of multidimensional arrays, and summarize theoretical results that demonstrate that the error of repeated lossy compression can be bounded and controlled. Furthermore, we establish a relationship between grid resolution and compression-induced errors and show that, contrary to conventional floating point, ZFP reduces finite-difference errors with finer grids. We present example calculations that demonstrate data reduction by 4x or more with negligible impact on solution accuracy. Our results further demonstrate several orders-of-magnitude increase in accuracy using ZFP over IEEE floating point and Posits for the same storage budget.

Lindstrom, Peter↗