Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Colonie FUSRAP Site - Radioactive Dust Sampling of Residential and Commercial Properties - 16384

The Colonie Formerly Utilized Sites Remedial Action Program (FUSRAP) Site is located in the Town of Colonie, Albany County, New York. The U.S. Army Corps of Engineers (USACE) is currently addressing environmental contamination associated with the Site. The area surrounding the Site consists of residential and commercial properties, known as vicinity properties (VPs). Cleanup activities have been completed on many of these VPs under the FUSRAP to address depleted uranium contamination. Soil remediation activities have been substantially completed at the Colonie FUSRAP Site and its VPs under the FUSRAP. Recent studies performed by an independent party and under FUSRAP identified uranium within indoor dust in residencies and businesses in the immediate vicinity of the Site. The studies were limited to non-living areas such as basement window sills, garages, and attics. This paper discusses the indoor dust sampling and analysis, in support of the Remedial Investigation (RI) for the VPs, and presents sampling results. The indoor dust sample media presented many technical challenges regarding sampling design, data reduction, and data interpretation. In addition, because the RI is focused on numerous resident-owned, rental, and commercial properties, challenges where presented regarding right-of-entry to sample. Isotopic uranium ratios observed in the majority of the samples are indicative of the depleted uranium contaminant, even at trace levels. While the sampling and analysis methods were effective at identifying depleted uranium in the majority of the samples, preliminary property-specific risk assessment results do not indicate the presence of depleted uranium in sufficient concentrations to pose unacceptable risk to VP inhabitants. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Machine-learning-assisted automation of single-crystal neutron diffraction

Neutron scattering is a powerful but expensive technique to study materials and discover new matter. Advanced detector technology has significantly improved the efficiency of neutron experiments, increasing the complexity of neutron data reduction and analysis. Machine learning (ML) brings new directions for neutron diffraction data reduction and experiment operation. Here, this work presents an ML-assisted data reduction and analysis method for precise recognition of Bragg peaks and the corresponding regions of interest; it can then automatically screen and align a measured crystal using the recognized peaks, and subsequently plan and optimize the data collection with user-provided information and uncertainty quantification values of detected peaks. This method shows robust performance in different complex sample environments and enables automated single-crystal neutron diffraction.

47 OTHER INSTRUMENTATION↗

Robustness of the smartpixels classifier for different simulated sensor geometries and non-ideal detector conditions

Pixel tracking detectors at upcoming collider experiments will see unprecedented charged-particle densities. Real-time data reduction on the detector will enable higher granularity and faster readout, possibly enabling the use of the pixel detector in high-rate online event selection, such as the ATLAS or CMS first-level trigger systems. This data reduction can be accomplished with a neural network (NN) in the readout chip bonded with the sensor that recognizes and rejects tracks with low transverse momentum (p T ) based on the geometrical shape of the charge deposition (“cluster”). To design viable detectors for deployment, the dependence of the NN as a function of the sensor geometry, external magnetic field, irradiation, and noise must be understood. In this paper, we present first studies of the efficiency and data reduction for planar pixel sensors exploring these parameters. For the CMS HL-LHC sensor geometry, we obtain a signal efficiency of (91.9 ± 0.7)% and a data reduction of (29.7 ± 1.0)%. A smaller sensor pitch in the bending direction improves the p T discrimination, but a larger pitch can be partially compensated with detector thickness. Any accumulated radiation damage also changes the cluster shape, reducing the signal efficiency compared to the baseline by approximately 30–60% in absolute terms, but nearly all of the performance can be recovered through retraining of the network and updating the weights. Finally, the impact of noise was investigated, and retraining the network on noise-injected datasets was found to maintain performance within 6% of the baseline network trained and evaluated on noiseless data. •ASIC-compatible track-momentum classifier is robust in realistic detector conditions.•About 90% signal efficiency and 30% data reduction per layer for CMS HL-LHC geometry.•Single-layer signal efficiency increases for smaller pixel pitch or thicker sensors.•Performance with noise or after radiation damage mostly recovered by retraining.

Shekar, Danush [Illinois U., Chicago] (ORCID:00000↗

Tools for Visualization and Analysis of Small-Angle Neutron Scattering Data: Descriptions and Examples

A great deal of progress has been made in improving the data reduction experience for the SANS instruments at the SNS and HFIR at ORNL. The existing data reduction toolset, drtsans, makes it possible to integrate data analysis and visualization tools into the data reduction scripts, thereby providing new opportunities for more automated data processing for users of the SNS and HFIR. Here, the first set of tools developed is described with usage examples.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

ECON-T and ECON-D: Endcap Concentrator ASICs for the CMS HGCAL

With over 6 million channels, the High Granularity Calorimeter (HGCAL) for the CMS HL-LHC Upgrade presents a unique data challenge. The ECON ASICs provide critical on-detector data reduction for the 40 MHz trigger path (ECON-T) and 750 kHz data acquisition path (ECON-D) of the HGCAL. The ASICs, fabricated in 65 nm CMOS, are rad-tolerant (600 Mrad) with low power consumption (<2.5 mW/channel). This presentation is the first comprehensive description of the ECON designs, first functionality and radiation tests for the ECON-T ASIC, and first results from the full production of 75k ECON-D and ECON-T ASICs.

Bergamin, G. [CERN] (ORCID:0000000285758704)↗

Multilevel Techniques for Compression and Reduction of Scientific Data---The Multivariate Case

We develop a technique for multigrid adaptive reduction of data (MGARD). Special attention is given to the case of tensor product grids, where our approach permits the use of nonuniformly spaced grids in each direction, which can prove problematic for many types of data reduction methods. An important feature of our approach is the provision of guaranteed, computable bounds on the loss incurred by the reduction of the data. Many users are leery of lossy algorithms and will only consider using them provided that numerical bounds on the pointwise difference between the original and the reduced datasets are given. Accordingly, we develop techniques for bounding the loss measured in the $L^{\infty}(\Omega)$ norm, and we show that these bounds are realistic in the sense that they do not significantly overestimate the actual loss. Here, the resulting loss indicators are used to guide the adaptive reduction of the data so that the reduced dataset meets a user-prescribed tolerance or memory constraint. Illustrative numerical examples, including the reduction of data arising from the simulation of a nonlinear reaction-diffusion problem, a turbulent channel flow, and a climate simulation, are provided.

97 MATHEMATICS AND COMPUTING↗

Information and Perception of Meaningful Patterns

The visual system needs to extract the most important elements of the external world from a large flux of information in a short time for survival purposes. It is widely believed that in performing this task, it operates a strong data reduction at an early stage, by creating a compact summary of relevant information that can be handled by further levels of processing. In this work we formulate a model of early vision based on a pattern-filtering architecture, partly inspired by high-speed digital data reduction in experimental high-energy physics (HEP). This allows a much stronger data reduction than models based just on redundancy reduction. We show that optimizing this model for best information preservation under tight constraints on computational resources yields surprisingly specific a-priori predictions for the shape of biologically plausible features, and for experimental observations on fast extraction of salient visual features by human observers. Interestingly, applying the same optimized model to HEP data acquisition systems based on pattern-filtering architectures leads to specific a-priori predictions for the relevant data patterns that these devices extract from their inputs. These results suggest that the limitedness of computing resources can play an important role in shaping the nature of perception, by determining what is perceived as ‘‘meaningful features’’ in the input data.

59 BASIC BIOLOGICAL SCIENCES↗

Online data analysis and reduction: An important co-design motif for extreme-scale computers

A growing disparity between supercomputer computation speeds and I/O rates means that it is rapidly becoming infeasible to analyze supercomputer application output only after that output has been written to a file system. Instead, data-generating applications must run concurrently with data reduction and/or analysis operations, with which they exchange information via high-speed methods such as interprocess communications. The resulting parallel computing motif, online data analysis and reduction (ODAR), has important implications for both application and HPC systems design. Here we introduce the ODAR motif and its co-design concerns, describe a co-design process for identifying and addressing those concerns, present tools that assist in the co-design process, and present case studies to illustrate the use of the process and tools in practical settings.

Data Analysis↗

Serial millisecond crystallography of membrane and soluble protein microcrystals using synchrotron radiation

Crystal structure determination of biological macromolecules using the novel technique of serial femtosecond crystallography (SFX) is severely limited by the scarcity of X-ray free-electron laser (XFEL) sources. However, recent and future upgrades render microfocus beamlines at synchrotron-radiation sources suitable for room-temperature serial crystallography data collection also. Owing to the longer exposure times that are needed at synchrotrons, serial data collection is termed serial millisecond crystallography (SMX). As a result, the number of SMX experiments is growing rapidly, with a dozen experiments reported so far. Here, the first high-viscosity injector-based SMX experiments carried out at a US synchrotron source, the Advanced Photon Source (APS), are reported. Microcrystals (5–20 µm) of a wide variety of proteins, including lysozyme, thaumatin, phycocyanin, the human A 2A adenosine receptor (A 2A AR), the soluble fragment of the membrane lipoprotein Flpp3 and proteinase K, were screened. Crystals suspended in lipidic cubic phase (LCP) or a high-molecular-weight poly(ethylene oxide) (PEO; molecular weight 8 000 000) were delivered to the beam using a high-viscosity injector. In-house data-reduction (hit-finding) software developed at APS as well as the SFX data-reduction and analysis software suites Cheetah and CrystFEL enabled efficient on-site SMX data monitoring, reduction and processing. Complete data sets were collected for A 2A AR, phycocyanin, Flpp3, proteinase K and lysozyme, and the structures of A 2A AR, phycocyanin, proteinase K and lysozyme were determined at 3.2, 3.1, 2.65 and 2.05 Å resolution, respectively. The data demonstrate the feasibility of serial millisecond crystallography from 5–20 µm crystals using a high-viscosity injector at APS. The resolution of the crystal structures obtained in this study was dictated by the current flux density and crystal size, but upcoming developments in beamline optics and the planned APS-U upgrade will increase the intensity by two orders of magnitude. Furthermore, these developments will enable structure determination from smaller and/or weakly diffracting microcrystals.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Developing ML/AI Methods for High-Throughput Characterization of Multiple-Sensor Streams of Tokamak Dynamics for High-Speed Control (Final Report)

This project evaluated and developed new mathematical and algorithmic techniques capable of handling (in real-time) the growing amounts of data generated by modern fusion research. While existing numerical linear algebra (NLA) methods provide the backbone to classical data analysis and algorithms, these methods fundamentally do not port to distributed architectures nor do they allow low-latency data reduction for control. Motivated by the needs for modern fusion reactors, this project explored and implemented new numerical methods to characterize plasma dynamics, respond in real-time to discharge evolution, and to process massive-scale data accurately and rapidly more fully. This project links expertise in multiple-sensor diagnostics of tokamak plasma dynamics from Columbia University’s Plasma Physics Laboratory with expertise in massive-scale data reduction and extreme data control algorithms at Columbia University’s Data Science Institute. This interdisciplinary project (i) applied machine learning methods, (ii) implemented a properly-trained neural-network for very fast processing of high-speed plasma videography, and (ii) developed the applied mathematical methods, based on randomized-NLA (rNLA) routines, for data analysis, reduction, and real-time control. The Columbia University High Beta Tokamak-Extended Pulse (HBT-EP) facility provided data to test new algorithms and partnership with Columbia University's Data Sciences Institute evaluated the broader use of new algorithms for many challenging control applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

User Guide for Sample Reduction at GP-SANS

This manual is intended as a quick guide for data reduction of GP-SANS data. It includes all necessary steps to do the data reduction based on absolute calibration using the open beam method and how to transfer the reduced data to the personal computer system. If any errors are coming up so that the reduction script is not functioning as intended, please contact the instrument scientist.

97 MATHEMATICS AND COMPUTING↗

CANDELS: THE COSMIC ASSEMBLY NEAR-INFRARED DEEP EXTRAGALACTIC LEGACY SURVEY-THE HUBBLE SPACE TELESCOPE OBSERVATIONS, IMAGING DATA PRODUCTS, AND MOSAICS

This paper describes the Hubble Space Telescope imaging data products and data reduction procedures for the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS). This survey is designed to document the evolution of galaxies and black holes at z Almost-Equal-To 1.5-8, and to study Type Ia supernovae at z > 1.5. Five premier multi-wavelength sky regions are selected, each with extensive multi-wavelength observations. The primary CANDELS data consist of imaging obtained in the Wide Field Camera 3 infrared channel (WFC3/IR) and the WFC3 ultraviolet/optical channel, along with the Advanced Camera for Surveys (ACS). The CANDELS/Deep survey covers {approx}125 arcmin{sup 2} within GOODS-N and GOODS-S, while the remainder consists of the CANDELS/Wide survey, achieving a total of {approx}800 arcmin{sup 2} across GOODS and three additional fields (Extended Groth Strip, COSMOS, and Ultra-Deep Survey). We summarize the observational aspects of the survey as motivated by the scientific goals and present a detailed description of the data reduction procedures and products from the survey. Our data reduction methods utilize the most up-to-date calibration files and image combination procedures. We have paid special attention to correcting a range of instrumental effects, including charge transfer efficiency degradation for ACS, removal of electronic bias-striping present in ACS data after Servicing Mission 4, and persistence effects and other artifacts in WFC3/IR. For each field, we release mosaics for individual epochs and eventual mosaics containing data from all epochs combined, to facilitate photometric variability studies and the deepest possible photometry. A more detailed overview of the science goals and observational design of the survey are presented in a companion paper.

79 ASTRONOMY AND ASTROPHYSICS↗

Another Set of Python Tools for Visualizing and Manipulating Small-Angle Neutron Scattering Data: Descriptions and Examples

The GP-SANS, Bio-SANS and EQ-SANS instruments at ORNL utilize drtsans for data reduction. drtsans is built on Python, and it can be run using python scripts and Jupyter notebooks. The flexibility afforded by Python makes it possible to incorporate additional actions into the scripts used for data reduction, such as analysis and visualization. Here, a new set of tools for visualizing and manipulating SANS data that can be incorporated into the data reduction scripts for the ORNL SANS instruments, or employed during post–processing, is presented that expands the capabilities of the two previously-released tool sets.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

VLA-ANGST: A HIGH-RESOLUTION H I SURVEY OF NEARBY DWARF GALAXIES

We present the 'Very Large Array survey of Advanced Camera for Surveys Nearby Galaxy Survey Treasury galaxies (VLA-ANGST)'. VLA-ANGST is a National Radio Astronomy Observatory Large Program consisting of high spectral (0.6-2.6 km s{sup -1}) and spatial ({approx}6'') resolution observations of neutral, atomic hydrogen (H I) emission toward 35 nearby dwarf galaxies from the ANGST survey. ANGST is a systematic Hubble Space Telescope survey to establish a legacy of uniform multi-color photometry of resolved stars for a volume-limited sample of nearby galaxies (D {approx}< 4 Mpc). VLA-ANGST provides VLA H I observations of the sub-sample of ANGST galaxies with recent star formation that are observable from the northern hemisphere and that were not observed in the 'The H I Nearby Galaxy Survey' (THINGS). The overarching scientific goal of VLA-ANGST is to investigate fundamental characteristics of the neutral interstellar medium (ISM) of dwarf galaxies. Here we describe the VLA observations, the data reduction, and the final VLA-ANGST data products. We present an atlas of the integrated H I maps, the intensity-weighted velocity fields, the second moment maps as a measure for the velocity dispersion of the H I, individual channel maps, and integrated H I spectra for each VLA-ANGST galaxy. We closely follow the observational setup and data reduction of THINGS to achieve comparable sensitivity and angular resolution. A major difference between VLA-ANGST and THINGS, however, is the high velocity resolution of the VLA-ANGST observations (0.65 and 1.3 km s{sup -1} for the majority of the galaxies). The VLA-ANGST data products are made publicly available through a dedicated Web site (https://science.nrao.edu/science/surveys/vla-angst). With available star formation histories from resolved stellar populations and lower resolution ancillary observations from the far-infrared to the ultraviolet, VLA-ANGST will enable detailed studies of the relationship between the ISM and star formation in dwarf galaxies on a {approx}100 pc scale.

79 ASTRONOMY AND ASTROPHYSICS↗

Sensor Co-design for $\textit{smartpixels}$

Pixel tracking detectors at upcoming collider experiments will see unprecedented charged-particle densities. Real-time data reduction on the detector will enable higher granularity and faster readout, possibly enabling the use of the pixel detector in the first level of the trigger for a hadron collider. This data reduction can be accomplished with a neural network (NN) in the readout chip bonded with the sensor that recognizes and rejects tracks with low transverse momentum (p$_T$) based on the geometrical shape of the charge deposition (``cluster''). To design a viable detector for deployment at an experiment, the dependence of the NN as a function of the sensor geometry, external magnetic field, and irradiation must be understood. In this paper, we present first studies of the efficiency and data reduction for planar pixel sensors exploring these parameters. A smaller sensor pitch in the bending direction improves the p$_T$ discrimination, but a larger pitch can be partially compensated with detector depth. An external magnetic field parallel to the sensor plane induces Lorentz drift of the electron-hole pairs produced by the charged particle, broadening the cluster and improving the network performance. The absence of the external field diminishes the background rejection compared to the baseline by $\mathcal{O}$(10%). Any accumulated radiation damage also changes the cluster shape, reducing the signal efficiency compared to the baseline by $\sim$ 30 - 60%, but nearly all of the performance can be recovered through retraining of the network and updating the weights. Finally, the impact of noise was investigated, and retraining the network on noise-injected datasets was found to maintain performance within 6% of the baseline network trained and evaluated on noiseless data.

Shekar, Danush [Illinois U., Chicago]↗

SciPhon : a data analysis software for nuclear resonant inelastic X-ray scattering with applications to Fe, Kr, Sn, Eu and Dy

We present the synchrotron radiation technique of nuclear resonant inelastic X-ray scattering (NRIXS), also known as nuclear resonance vibrational spectroscopy or nuclear inelastic scattering, provides a wealth of information on the vibrational properties of solids. It has found applications in studies of lattice dynamics and elasticity, superconductivity, heme biochemistry, seismology, isotope geochemistry and many other fields. It involves probing the vibrational modes of solids by using the nuclear resonance of Mössbauer isotopes such as 57 Fe, 83 Kr, 119 Sn, 151 Eu and 161 Dy. After data reduction, it provides the partial phonon density of states of the Mössbauer isotope that is investigated, as well as many other derived quantities such as the mean force constant of the chemical bonds and the Debye velocity. The data reduction is, however, not straightforward and involves removal of the elastic peak, normalization and Fourier–Log transformation. Furthermore, some of the quantities derived are highly sensitive to details in the baseline correction. A software package and several novel procedures to streamline and hopefully improve the reduction of the NRIXS data generated at sector 3ID of the Advanced Photon Source have been developed. The graphical user interface software is named SciPhon and runs as a Mathematica package. It is easily portable to other platforms and can be easily adapted for reducing data generated at other beamlines. Several tests and comparisons are presented that demonstrate the usefulness of this software, whose results have already been used in several publications. Here, the SciPhon software is used to reduce Kr, Sn, Eu and Dy NRIXS data, and potential implications for interpreting natural isotopic variations in those systems are discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

PROTEUS: Machine Learning Driven Resilience for Extreme-scale Systems

The objective of this project is to design, develop, and evaluate scalable software to enhance resilience, data checkpointing, program restart, and analysis. The proposed tasks are to 1) develop scalable machine learning techniques to learn temporal change patterns in a scalable and in-situ manner, and to minimize data movement and maximize learning locally closest to data; 2) design a concise data representation and indexing mechanism to capture the distribution of changes in data that can guarantee point-wise user-defined tolerable errors while reducing the data storage requirements by an order of magnitude or more; 3) develop data reduction techniques as library modules; 4) exploit local SSD for minimizing data movement in storage hierarchy; 5) develop anomaly detection algorithms that can predict corruptions based on learning of emerging patterns; 6) develop software libraries to be incorporated within widely used data formats and APIs; and 7) evaluate the proposed software using DOE scientific applications. The outcomes of the proposed work are to satisfy many synergistic data reduction and resilience requirements for large-scale data intensive applications executed on extreme-scale computing systems. The developed mechanism for error-bound data approximation is directly applicable to existing scientific applications. Through machine learning from historical events and change distribution, this work will enable anomaly detection for DOE computer facility.

97 MATHEMATICS AND COMPUTING↗

Automated detection and analysis of particle beams in laser-plasma accelerator simulations

Numerical simulations of laser-plasma wakefield (particle) accelerators model the acceleration of electrons trapped in plasma oscillations (wakes) left behind when an intense laser pulse propagates through the plasma. The goal of these simulations is to better understand the process involved in plasma wake generation and how electrons are trapped and accelerated by the wake. Understanding of such accelerators, and their development, offer high accelerating gradients, potentially reducing size and cost of new accelerators. One operating regime of interest is where a trapped subset of electrons loads the wake and forms an isolated group of accelerated particles with low spread in momentum and position, desirable characteristics for many applications. The electrons trapped in the wake may be accelerated to high energies, the plasma gradient in the wake reaching up to a gigaelectronvolt per centimeter. High-energy electron accelerators power intense X-ray radiation to terahertz sources, and are used in many applications including medical radiotherapy and imaging. To extract information from the simulation about the quality of the beam, a typical approach is to examine plots of the entire dataset, visually determining the adequate parameters necessary to select a subset of particles, which is then further analyzed. This procedure requires laborious examination of massive data sets over many time steps using several plots, a routine that is unfeasible for large data collections. Demand for automated analysis is growing along with the volume and size of simulations. Current 2D LWFA simulation datasets are typically between 1GB and 100GB in size, but simulations in 3D are of the order of TBs. The increase in the number of datasets and dataset sizes leads to a need for automatic routines to recognize particle patterns as particle bunches (beam of electrons) for subsequent analysis. Because of the growth in dataset size, the application of machine learning techniques for scientific data mining is increasingly considered. In plasma simulations, Bagherjeiran et al. presented a comprehensive report on applying graph-based techniques for orbit classification. They used the KAM classifier to label points and components in single and multiple orbits. Love et al. conducted an image space analysis of coherent structures in plasma simulations. They used a number of segmentation and region-growing techniques to isolate regions of interest in orbit plots. Both approaches analyzed particle accelerator data, targeting the system dynamics in terms of particle orbits. However, they did not address particle dynamics as a function of time or inspected the behavior of bunches of particles. Ruebel et al. addressed the visual analysis of massive laser wakefield acceleration (LWFA) simulation data using interactive procedures to query the data. Sophisticated visualization tools were provided to inspect the data manually. Ruebel et al. have integrated these tools to the visualization and analysis system VisIt, in addition to utilizing efficient data management based on HDF5, H5Part, and the index/query tool FastBit. In Ruebel et al. proposed automatic beam path analysis using a suite of methods to classify particles in simulation data and to analyze their temporal evolution. To enable researchers to accurately define particle beams, the method computes a set of measures based on the path of particles relative to the distance of the particles to a beam. To achieve good performance, this framework uses an analysis pipeline designed to quickly reduce the amount of data that needs to be considered in the actual path distance computation. As part of this process, region-growing methods are utilized to detect particle bunches at single time steps. Efficient data reduction is essential to enable automated analysis of large data sets as described in the next section, where data reduction methods are steered to the particular requirements of our clustering analysis. Previously, we have described the application of a set of algorithms to automate the data analysis and classification of particle beams in the LWFA simulation data, identifying locations with high density of high energy particles. These algorithms detected high density locations (nodes) in each time step, i.e. maximum points on the particle distribution for only one spatial variable. Each node was correlated to a node in previous or later time steps by linking these nodes according to a pruned minimum spanning tree (PMST). We call the PMST representation 'a lifetime diagram', which is a graphical tool to show temporal information of high dense groups of particles in the longitudinal direction for the time series. Electron bunch compactness was described by another step of the processing, designed to partition each time step, using fuzzy clustering, into a fixed number of clusters.

43 PARTICLE ACCELERATORS↗