Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Reduction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Striped Data Analysis Framework

Traditionally, High Energy data analysis is based on the model where data are stored in files and analyzed by running multiple analysis processes, each reading one or more of the data files. This process involves repeated data reduction step, which produces smaller files, which is time consuming and leads to data duplication. We propose an alternative approach to data storage and analysis, based on the Big Data technologies. The idea is to store each element of data once and only once in a distributed scalable database and analyze data by reading only "interesting" pieces of data from the database. To make this approach possible, we developed columnar Striped Data Representation Format as the basis of the framework. Traditional columnar approach allows for efficient analysis of complex data structures. While keeping all the benefits of columnar data representation, striped mechanism goes further by enabling efficient parallelization of computations and flexible dist ribution of data analysis. The framework includes scalable and elastic data storage, compute and user analysis backend components. The framework uses off-the shelf web services and data caching technologies as the compute/data co-location mechanism. Flexible architecture allows the framework run in the cloud using container technologies. The framework offers Python/Jupyter as the user analysis backend platform, but can also run in command line or batch mode. In the article, we will present the results of the FNAL-LDRD-2016-032 FNAL LDRD project, the design, implementation, features and performance characteristics of the Striped Data Analysis Framework.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

CONTINUUM HALOS IN NEARBY GALAXIES: AN EVLA SURVEY (CHANG-ES). II. FIRST RESULTS ON NGC 4631

We present the first results from the Continuum Halos in Nearby Galaxies-an EVLA Survey (CHANG-ES), a new survey of 35 edge-on galaxies to search for both in-disk and extraplanar radio continuum emission. CHANG-ES is exploiting the new wide-band, multi-channel capabilities of the Karl G. Jansky Very Large Array (i.e., the Expanded Very Large Array or EVLA) with observations in two bands centered at 1.5 and 6 GHz in a variety of array configurations with full polarization. The motivation and science case for the survey are presented in a companion paper (Paper I). These first results are based on C-array test observations in both observing bands of the well-known radio halo galaxy, NGC 4631. In this paper, we outline the observations and the data reduction steps that are required for wide-band calibration and mapping of EVLA data, including polarization. With modest on-source observing times (30 minutes at 1.5 GHz and 75 minutes at 6 GHz for the test data), we have achieved best rms noise levels of 22 and 3.5 {mu}Jy beam{sup -1} at 1.5 GHz and 6 GHz, respectively. New disk-halo features have been detected, among them two at 1.5 GHz that appear as loops in projection. We present the first 1.5 GHz spectral index map of NGC 4631 to be formed from a single wide-band observation in a single array configuration. This map represents tangent slopes to the intensities within the band centered at 1.5 GHz, rather than fits across widely separated frequencies as has been done in the past and is also the highest spatial resolution spectral index map yet presented for this galaxy. The average spectral index in the disk is {alpha}-bar{sub 1.5GHz} = -0.84 {+-} 0.05 indicating that the emission is largely non-thermal, but a small global thermal contribution is sufficient to explain a positive curvature term in the spectral index over the band. Two specific star-forming regions have spectral indices that are consistent with thermal emission. Polarization results (uncorrected for internal Faraday rotation) are consistent with previous observations and also reveal some new features. On broad scales, we find strong support for the notion that magnetic fields constrain the X-ray-emitting hot gas.

79 ASTRONOMY AND ASTROPHYSICS↗

HIGH-RESOLUTION XMM-NEWTON SPECTROSCOPY OF THE COOLING FLOW CLUSTER A3112

We examine high signal-to-noise XMM-Newton European Photon Imaging Camera (EPIC) and Reflection Grating Spectrometer (RGS) observations to determine the physical characteristics of the gas in the cool core and outskirts of the nearby rich cluster A3112. The XMM-Newton Extended Source Analysis Software data reduction and background modeling methods were used to analyze the XMM-Newton EPIC data. From the EPIC data, we find that the iron and silicon abundance gradients show significant increase toward the center of the cluster while the oxygen abundance profile is centrally peaked but has a shallower distribution than that of iron. The X-ray mass modeling is based on the temperature and deprojected density distributions of the intracluster medium determined from EPIC observations. The total mass of A3112 obeys the M-T scaling relations found using XMM-Newton and Chandra observations of massive clusters at r{sub 500}. The gas mass fraction f{sub gas} = 0.149{sup +0.036}{sub -0.032} at r{sub 500} is consistent with the seven-year Wilkinson Microwave Anisotropy Probe results. The comparisons of line fluxes and flux limits on the Fe XVII and Fe XVIII lines obtained from high-resolution RGS spectra indicate that there is no spectral evidence for cooler gas associated with the cluster with temperature below 1.0 keV in the central <38'' ({approx}52 kpc) region of A3112. High-resolution RGS spectra also yield an upper limit to the turbulent motions in the compact core of A3112 (206 km s{sup -1}). We find that the contribution of turbulence to total energy is less than 6%. This upper limit is consistent with the energy contribution measured in recent high-resolution simulations of relaxed galaxy clusters.

79 ASTRONOMY AND ASTROPHYSICS↗

Expanding Lorentz and spectrum corrections to large volumes of reciprocal space for single-crystal time-of-flight neutron diffraction

Evidence is mounting that potentially exploitable properties of technologically and chemically interesting crystalline materials are often attributable to local structure effects, which can be observed as modulated diffuse scattering (mDS) next to Bragg diffraction (BD). BD forms a regular sparse grid of intense discrete points in reciprocal space. Traditionally, the intensity of each Bragg peak is extracted by integration of each individual reflection first, followed by application of the required corrections. In contrast, mDS is weak and covers expansive volumes of reciprocal space close to, or between, Bragg reflections. For a representative measurement of the diffuse scattering, multiple sample orientations are generally required, where many points in reciprocal space are measured multiple times and the resulting data are combined. The common post-integration data reduction method is not optimal with regard to counting statistics. A general and inclusive data processing method is needed. In this contribution, a comprehensive data analysis approach is introduced to correct and merge the full volume of scattering data in a single step, while correctly accounting for the statistical weight of the individual measurements. Development of this new approach required the exploration of a data treatment and correction protocol that includes the entire collected reciprocal space volume, using neutron time-of-flight or wavelength-resolved data collected at TOPAZ at the Spallation Neutron Source at Oak Ridge National Laboratory.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

High-Throughput Method for Strontium Isotope Analysis by Multi-Collector-Inductively Coupled Plasma-Mass Spectrometer

This technical report presents the details of the Sr column configuration and the high-throughput Sr separation protocol. Data showing the performance of the method as well as the best practices for optimizing Sr isotope analysis by MC-ICP-MS is presented. Lastly, this report offers tools for data handling and data reduction of Sr isotope results from the Thermo Scientific Neptune software to assist in data quality assurance, which help avoid issues of data glut associated with high sample throughput rapid analysis.

47 OTHER INSTRUMENTATION↗

High-Throughput Method for Strontium Isotope Analysis by Multi-Collector-Inductively Coupled Plasma-Mass Spectrometer

This technical report presents the details of the Sr column configuration and the high-throughput Sr separation protocol. Data showing the performance of the method as well as the best practices for optimizing Sr isotope analysis by MC-ICP-MS is presented. Lastly, this report offers tools for data handling and data reduction of Sr isotope results from the Thermo Scientific Neptune software to assist in data quality assurance, which help avoid issues of data glut associated with high sample throughput rapid analysis.

47 OTHER INSTRUMENTATION↗

Design and Testing of the Endcap Concentrator ASICs for the CMS High-Granularity Calorimeter Upgrade

A major upgrade of the High-Granularity Calorimeter (HGCAL) in the CMS detector is planned for Long-Shutdown 3 (LS3), currently expected to start in 2026. This upgrade will contain over 6 million channels and the electronics will be required to be low power and to withstand a radiation environment with a High-Energy Hadron flux of 3x10**6 particles per square centimeter per second. The solution to this significant challenge is the two Endcap Concentrator (ECON) ASICs, ECON-T and ECON-D, working in tandem with the HGCROC front-end ASIC. The two ECON ASICs provide critical on-detector data reduction for both the 40 MHz trigger path (ECON-T) and 750 kHz data acquisition path (ECON-D) of the HGCAL. The ASICs are fabricated in 65nm CMOS. They are rad-tolerant to 600 Mrad with low power consumption (<2.5 mW/channel). This presentation will be a comprehensive description of each ECON design, including the infrastructure that they share as well as the elements that make each unique. The presentation will also include functionality and radiation tests for both ASICs, and the first high statistics characterization results from the full production of 75k ECON-D and ECON-T ASICs.

Hoff, James R. [Fermilab] (ORCID:0000000163514592)↗

ISABELA for effective in situ compression of scientific data: ISABELA FOR EFFECTIVE IN-SITU REDUCTION OF SPATIO-TEMPORAL DATA

Exploding dataset sizes from extreme-scale scientific simulations necessitates efficient data management and reduction schemes to mitigate I/O costs. With the discrepancy between I/O bandwidth and computational power, scientists are forced to capture data infrequently, thereby making data collection an inherently lossy process. Although data compression can be an effective solution, the random nature of real-valued scientific datasets renders lossless compression routines ineffective. These techniques also impose significant overhead during decompression, making them unsuitable for data analysis and visualization, which require repeated data access. To address this problem, we propose an effective method for In situ Sort-And-B-spline Error-bounded Lossy Abatement (ISABELA) of scientific data that is widely regarded as effectively incompressible. With ISABELA, we apply a pre-conditioner to seemingly random and noisy data along spatial resolution to achieve an accurate fitting model that guarantees a ≥0.99 correlation with the original data. We further take advantage of temporal patterns in scientific data to compress data b ≈ 85%, while introducing only a negligible overhead on simulations in terms of runtime. ISABELA significantly outperforms existing lossy compression methods, such as wavelet compression, in terms of data reduction and accuracy. We extend upon our previous paper by additionally building a communication-free, scalable parallel storage framework on top of ISABELA-compressed data that is ideally suited for extreme-scale analytical processing. The basis for our storage framework is an inherently local decompression method (it need not decode the entire data), which allows for random access decompression and low-overhead task division that can be exploited over heterogeneous architectures. Furthermore, analytical operations such as correlation and query processing run quickly and accurately over data in the compressed space. Copyright © 2012 John Wiley & Sons, Ltd.

Computer Science↗

The LSST DESC data challenge 1: generation and analysis of synthetic images for next-generation surveys

Data Challenge 1 (DC1) is the first synthetic data set produced by the Rubin Observatory Legacy Survey of Space and Time (LSST) Dark Energy Science Collaboration (DESC). DC1 is designed to develop and validate data reduction and analysis and to study the impact of systematic effects that will affect the LSST data set. DC1 is comprised of r -band observations of 40 deg 2 to 10 yr LSST depth. In this paper, we present each stage of the simulation and analysis process: (a) generation, by synthesizing sources from cosmological N -body simulations in individual sensor-visit images with different observing conditions; (b) reduction using a development version of the LSST Science Pipelines; and (c) matching to the input cosmological catalogue for validation and testing. We verify that testable LSST requirements pass within the fidelity of DC1. We establish a selection procedure that produces a sufficiently clean extragalactic sample for clustering analyses and we discuss residual sample contamination, including contributions from inefficiency in star–galaxy separation and imperfect deblending. We compute the galaxy power spectrum on the simulated field and conclude that: (i) survey properties have an impact of 50 per cent of the statistical uncertainty for the scales and models used in DC1; (ii) a selection to eliminate artefacts in the catalogues is necessary to avoid biases in the measured clustering; and (iii) the presence of bright objects has a significant impact (2σ–6σ) in the estimated power spectra at small scales (ℓ > 1200), highlighting the impact of blending in studies at small angular scales in LSST.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A machine-learning-driven data labeling pipeline for scientific analysis in MLExchange

This study introduces a novel labeling pipeline to accelerate the labeling process of scientific data sets by using artificial intelligence (AI)-guided tagging techniques. This pipeline includes a set of interconnected web-based graphical user interfaces (GUIs), where Data Clinic and MLCoach enable the preparation of machine learning (ML) models for data reduction and classification, respectively, while Label Maker is used for label assignment. Throughout this pipeline, data can be accessed through a direct connection to a file system or through Tiled for access through Hypertext Transfer Protocol (HTTP). Our experimental results present three use cases where this labeling pipeline has been instrumental for the study of large X-ray scattering data sets in the area of pattern recognition, the remote analysis of resonant soft X-ray scattering data and the fine-tuning process of foundation models. These use cases highlight the labeling capabilities of this pipeline, including the ability to label large data sets in a short period of time, to perform remote data analysis while minimizing data movement and to enhance the fine-tuning process of complex ML models with human involvement.

Chavez, Tanny (ORCID:0000000193172896)↗

Fast Semi-automated Filtration Method for Non-targeted LC-QTOF Data of Aged Nitroplasticizer Samples

A full dataset of aged nitroplasticizer (NP) is composed of more than 2000 unique mass-to-charges (m/z) when combining the non-targeted data obtained from both positive and negative electrospray ionization modes in time-of-flight mass spectrometry. Therefore, manual processing of these data often takes days, weeks, or even months to scrutinize for mechanistic insights. To effectively extract meaningful signals that represent vital degradation intermediates in the early NP degradation mechanism, a semi-automated postprocessing workflow for data filtering, tailored to the aging experiment of NP, has been developed. The automated portion of this workflow is written in a Python code (using pandas, numpy, and matplotlib libraries), which removes more than 65% of potential false signals within seconds via four threshold-based adjustable filters: signal sensitivity, coefficient of variation, number of measurements, and retention time variability. As for the manual portion, a pattern-based inspection method is employed to reduce another 23% or more false positives, which greatly simplifies data visualization and results in less than 3% of potential candidate m/z needing in-depth data interpretation. As a positive control, known compounds are verified. Using this semi-automated data reduction method, the amount of time required is reduced to a matter of hours for data filtering in the non-targeted datasets of aged NP, which saves more time and effort for compound identification.

36 MATERIALS SCIENCE↗

POWERFUL ACTIVITY IN THE BRIGHT AGES. I. A VISIBLE/IR SURVEY OF HIGH REDSHIFT 3C RADIO GALAXIES AND QUASARS

We present new rest-frame UV and visible observations of 22 high- z (1 < z < 2.5) 3C radio galaxies and QSOs obtained with the Hubble Space Telescope ’s Wide Field Camera 3 instrument. Using a custom data reduction strategy in order to assure the removal of cosmic rays, persistence signal, and other data artifacts, we have produced high-quality science-ready images of the targets and their local environments. We observe targets with regions of UV emission suggestive of active star formation. In addition, several targets exhibit highly distorted host galaxy morphologies in the rest frame visible images. Photometric analyses reveal that brighter QSOs generally tend to be redder than their dimmer counterparts. Using emission line fluxes from the literature, we estimate that emission line contamination is relatively small in the rest frame UV images for the QSOs. Using archival VLA data, we have also created radio map overlays for each of our targets, allowing for analysis of the optical and radio axes alignment.

79 ASTRONOMY AND ASTROPHYSICS↗

What's New in GSAS-II

The General Structure and Analysis Software II (GSAS-II) package is an all-new crystallographic analysis package written to replace and extend the capabilities of the universal and widely used GSAS and EXPGUI packages. GSAS-II was described in a 2013 article, but considerable work has been completed since then. This paper describes the advances, which include: rigid body fitting and structure solution modules; improved treatment for parametric refinements and equation of state fitting; and small-angle scattering data reduction and analysis. GSAS-II offers versatile and extensible modules for import and export of data and results. Capabilities are provided for users to select any version of the code. Code documentation has reached 150 pages and 17 web-tutorials are offered. © 2014 International Centre for Diffraction Data.

Toby, Brian H.↗

Experimental methods for laboratory measurements of helium spectral line broadening in white dwarf photospheres

White Dwarf (WD) stars are the most common stellar remnant in the universe. WDs usually have a hydrogen or helium atmosphere, and helium WD (called DB) spectra can be used to solve outstanding problems in stellar and galactic evolution. DB origins, which are still a mystery, must be known to solve these problems. DB masses are crucial for discriminating between different proposed DB evolutionary hypotheses. Current DB mass determination methods deliver conflicting results. The spectroscopic mass determination method relies on line broadening models that have not been validated at DB atmosphere conditions. We performed helium benchmark experiments using the White Dwarf Photosphere Experiment (WDPE) platform at Sandia National Laboratories' Z-machine that aims to study He line broadening at DB conditions. Using hydrogen/helium mixture plasmas allows investigating the importance of He Stark and van der Waals broadening simultaneously. Accurate experimental data reduction methods are essential to test these line-broadening theories. In this paper, we present data calibration methods for these benchmark He line shape experiments. We give a detailed account of data processing, spectral power calibrations, and instrument broadening measurements. Uncertainties for each data calibration step are also derived. We demonstrate that our experiments meet all benchmark experiment accuracy requirements: WDPE wavelength uncertainties are <1 Å, spectral powers can be determined to within 15%, densities are accurate at the 20% level, and instrumental broadening can be measured with 20% accuracy. Fulfilling these stringent requirements enables WDPE experimental data to provide physically meaningful conclusions about line broadening at DB conditions.

79 ASTRONOMY AND ASTROPHYSICS↗

THE ATACAMA COSMOLOGY TELESCOPE: DATA CHARACTERIZATION AND MAPMAKING

We present a description of the data reduction and mapmaking pipeline used for the 2008 observing season of the Atacama Cosmology Telescope (ACT). The data presented here at 148 GHz represent 12% of the 90 TB collected by ACT from 2007 to 2010. In 2008 we observed for 136 days, producing a total of 1423 hr of data (11 TB for the 148 GHz band only), with a daily average of 10.5 hr of observation. From these, 1085 hr were devoted to an 850 deg{sup 2} stripe (11.2 hr by 9. Degree-Sign 1) centered on a declination of -52. Degree-Sign 7, while 175 hr were devoted to a 280 deg{sup 2} stripe (4.5 hr by 4. Degree-Sign 8) centered at the celestial equator. The remaining 163 hr correspond to calibration runs. We discuss sources of statistical and systematic noise, calibration, telescope pointing, and data selection. For the 148 GHz band, out of 1260 survey hours and 1024 detectors in the array, 816 hr and 593 effective detectors remain after data selection, yielding a 38% survey efficiency. The total sensitivity in 2008, determined from the noise level between 5 Hz and 20 Hz in the time-ordered data stream (TOD), is 32 {mu}K{radical}s in cosmic microwave background units. Atmospheric brightness fluctuations constitute the main contaminant in the data and dominate the detector noise covariance at low frequencies in the TOD. The maps were made by solving the least-squares problem using the Preconditioned Conjugate Gradient method, incorporating the details of the detector and noise correlations. Simulations, as well as cross-correlations with Wilkinson Microwave Anisotropy Probe sky maps on large angular scales, reveal that our maps are unbiased at multipoles l > 300. This paper accompanies the public release of the 148 GHz southern stripe maps from 2008. The techniques described here will be applied to future maps and data releases.

79 ASTRONOMY AND ASTROPHYSICS↗

THE COS-HALOS SURVEY: KECK LRIS AND MAGELLAN MagE OPTICAL SPECTROSCOPY

We present high signal-to-noise optical spectra for 67 low-redshift (0.1 < z < 0.4) galaxies that lie within close projected distances (5 kpc < {rho} < 150 kpc) of 38 background UV-bright QSOs. The Keck LRIS and Magellan MagE data presented here are part of a survey that aims to construct a statistically sampled map of the physical state and metallicity of gaseous galaxy halos using the Cosmic Origins Spectrograph (COS) on the Hubble Space Telescope. We provide a detailed description of the optical data reduction and subsequent spectral analysis that allow us to derive the physical properties of this uniquely data-rich sample of galaxies. The galaxy sample is divided into 38 pre-selected L {approx} L*, z {approx} 0.2 'target' galaxies and 29 'bonus' galaxies that lie in close proximity to the QSO sightlines. We report galaxy spectroscopic redshifts accurate to {+-}30 km s{sup -1}, impact parameters, rest-frame colors, stellar masses, total star formation rates (SFRs), and gas-phase interstellar medium oxygen abundances. When we compare the distribution of these galaxy characteristics to those of the general low-redshift population, we find good agreement. The L {approx} L* galaxies in this sample span a diverse range of color (1.0 < u - r < 3.0), stellar mass (10{sup 9.5} < M/M{sub Sun} < 10{sup 11.5}), and SFRs (0.01-20 M{sub Sun} yr{sup -1}). These optical data, along with the COS UV spectroscopy, comprise the backbone of our efforts to understand how halo gas properties may correlate with their host galaxy properties, and ultimately to uncover the processes that drive gas outflow and/or are influenced by gas inflow.

79 ASTRONOMY AND ASTROPHYSICS↗

Efficient Data Query for Gaussian Process Compressed Data through Value Range Estimation [Slides]

When the resolution of the data increases, data reduction methods are applied to simulation output, including Gaussian process, neural representation and compression algorithms. Lots of data analysis/visualization techniques requires data query, but data query from reduced representation is still challenging. This report will provide examples and provide possible answers to why data query from reduced representation is still challenging.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗