Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

On single-crystal total scattering data reduction and correction protocols for analysis in direct space

Data reduction and correction steps and processed data reproducibility in the emerging single-crystal total-scattering-based technique of three-dimensional differential atomic pair distribution function (3D-ΔPDF) analysis are explored. All steps from sample measurement to data processing are outlined using a crystal of CuIr 2 S 4 as an example, studied in a setup equipped with a high-energy X-ray beam and a flat-panel area detector. Computational overhead as pertains to data sampling and the associated data-processing steps is also discussed. Various aspects of the final 3D-ΔPDF reproducibility are explicitly tested by varying the data-processing order and included steps, and by carrying out a crystal-to-crystal data comparison. Situations in which the 3D-ΔPDF is robust are identified, and caution against a few particular cases which can lead to inconsistent 3D-ΔPDFs is noted. Although not all the approaches applied herein will be valid across all systems, and a more in-depth analysis of some of the effects of the data-processing steps may still needed, the methods collected herein represent the start of a more systematic discussion about data processing and corrections in this field.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2013

Instrumental Backgrounds to $t\bar{t}$ and Single Top Production at Hadron Colliders

A number of backgrounds to top-quark production at the LHC and Tevatron experiments are characterized as instrumental: beam related-backgrounds, cosmic ray backgrounds, and backgrounds from non-prompt (or fake) leptons. A number of strategies have been deployed in the experiments to minimize these backgrounds and to estimate their rates. The strategies are applied at the level of the accelerator, the detector apparatus, the first few cuts which are common to many physics analyses, and more refined kinematic cuts. Remaining fake lepton background is estimated using sophisticated data-driven techniques. We present here a survey of the state-of-the-art in the reduction of instrumental backgrounds and also their estimation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2018

Fault Detection and Diagnosis In Hall–Héroult Cells Based on Individual Anode Current Measurements Using Dynamic Kernel PCA

Individual anode current signals in aluminum reduction cells provide localized cell conditions in the vicinity of each anode, which contain more information than the conventionally measured cell voltage and line current. One common use of this measurement is to identify process faults that can cause significant changes in the anode current signals. While this method is simple and direct, it ignores the interactions between anode currents and other important process variables. This paper presents an approach that applies multivariate statistical analysis techniques to individual anode currents and other process operating data, for the detection and diagnosis of local process abnormalities in aluminum reduction cells. Specifically, since the Hall–Héroult process is time-varying with its process variables dynamically and nonlinearly correlated, dynamic kernel principal component analysis with moving windows is used. The cell is discretized into a number of subsystems, with each subsystem representing one anode and cell conditions in its vicinity. The fault associated with each subsystem is identified based on multivariate statistical control charts. The results show that the proposed approach is able to not only effectively pinpoint the problematic areas in the cell, but also assess the effect of the fault on different parts of the cell.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Sky Residual Correction

The near-infrared is populated by numerous emission lines radiated from the Earth’s atmosphere, and these emission lines are often several orders of magnitude brighter in intensity than the typical astrophysical science target. The subtraction of these emission lines, sky subtraction, can create large systematic errors in ground-based astronomical spectra effectively limiting the number of usable resolution elements. A more effective sky subtraction, and the reduction of the systematic errors due to the sky subtraction process, is a major hurdle which ground-based astronomy must overcome to increase the amount of usable spectrum, and to be able to observe fainter scientific targets. Large high-quality data sets such as the Sloan Digital Sky Survey (SDSS) in themselves present opportunities for a reduction of the systematic sky subtraction errors through self-calibration. The sky residual correction method of Wild and Hewett is one such self-calibration technique which uses principal component analysis to reduce the systematic sky residual errors present in SDSS spectra. We apply sky residual corrections to the SDSS, Baryon Oscillation Spectroscopic Survey, and Apache Point Observatory Galactic Evolution Experiment data sets to optimize the number of subtraction components, and to quantify the reduction of the systematic errors in the science spectra. Finally as a proof of concept we use the sky-residual-corrected SDSS luminous red galaxy spectra to search for gravitationally lensed emission line Galaxies.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

Review of non destructive assay techniques utilised for decommissioning of plutonium facilities at AWE - 14418

Over the past fifteen years AWE has decommissioned and size reduced a large number of glove boxes that contained a wide spectrum of equipment that ranged from highly radiation attenuating structures, such as furnaces and lathes, to other less radiation attenuating apparatus, such as small discrete tools or simple work benches. In order to generate meaningful glove box residual fissile material inventory data, which can constructively inform the selection of measures needed to underwrite the criticality safety of size reduction and consignment to waste activities, several different types of Non Destructive Assay (NDA) measurement techniques are available at AWE. These are deployed for the estimate of the residual fissile material inventory prior to decommissioning activities and for the final NDA of waste containers, post decommissioning. The techniques range from the Decommissioning In-Situ Plutonium Inventory Monitor (DISPIM), through other Neutron Coincidence Counting (NCC) techniques (Neutron Panel monitoring) and gamma measurements to consideration of Thermo Luminescent Detector (TLD) data. This paper will present a comparison of the fissile material identified in the waste generated post decommissioning operations to the NDA estimate of the residual fissile material inventory for the glove boxes, as well as provide a direct comparison between the different NDA techniques available at AWE. The findings to date indicate that the glove box residual fissile material inventory used for criticality safety purposes bounds the decommissioning process, subject to the suitable deployment of the NDA technique and the inclusion of appropriate uncertainties. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2014

Investigating nearby star-forming galaxies in the ultraviolet with HST/COS spectroscopy. I. Spectral analysis and interstellar abundance determinations

This is the first in a series of three papers describing a project with the Cosmic Origins Spectrograph on the Hubble Space Telescope to measure abundances of the neutral interstellar medium (ISM) in a sample of nine nearby star-forming galaxies. The goal is to assess the (in)homogeneities of the multiphase ISM in galaxies where the bulk of metals can be hidden in the neutral phase, yet the metallicity is inferred from the ionized gas in the H II regions. The sample, spanning a wide range in physical properties, is to date the best suited to investigate the metallicity behavior of the neutral gas at redshift z = 0. ISM absorption lines were detected against the far-ultraviolet spectra of the brightest star-forming region(s) within each galaxy. Here we report on the observations, data reduction, and analysis of these spectra. Column densities were measured by a multicomponent line-profile fitting technique, and neutral-gas abundances were obtained for a wide range of elements. Several caveats were considered, including line saturation, ionization corrections, and dust depletion. Ionization effects were quantified with ad hoc CLOUDY models reproducing the complex photoionization structure of the ionized and neutral gas surrounding the UV-bright sources. An 'average spectrum of a redshift z = 0 star-forming galaxy' was obtained from the average column densities of unsaturated profiles of neutral-gas species. This template can be used as a powerful tool for studies of the neutral ISM at both low and high redshift.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Surrogate models for plasma displacement and current in 3D perturbed magnetohydrodynamic equilibria in tokamaks

Abstract A numerical database of over one thousand perturbed three-dimensional (3D) equilibria has been generated, constructed based on the MARS-F (Liu et al 2000 Phys. Plasmas 7 3681) computed plasma response to the externally applied 3D field sources in multiple tokamak devices. Perturbed 3D equilibria with the n = 1–4 ( n is the toroidal mode number) toroidal periodicity are computed. Surrogate models are created for the computed perturbed 3D equilibrium utilizing model order reduction (MOR) techniques. In particular, retaining the first few eigenstates from the singular value decomposition (SVD) of the data is found to produce reasonably accurate MOR-representations for the key perturbed quantities, such as the perturbed parallel plasma current density and the plasma radial displacement. SVD also helps to reveal the core versus edge plasma response to the applied 3D field. For the database covering the conventional aspect ratio devices, about 95% of data can be represented by the truncated SVD-series with inclusion of only the first five eigenstates, achieving a relative error (RE) below 20%. The MOR-data is further utilized to train neural networks (NNs) to enable fast reconstruction of perturbed 3D equilibria, based on the two-dimensional equilibrium input and the 3D source field. The best NN-training is achieved for the MOR-data obtained with a global SVD approach, where the full set of samples used for NN training and testing are stretched and form a large matrix which is then subject to SVD. The fully connected multi-layer perceptron, with one or two hidden layers, can be trained to predict the MOR-data with less than 10% RE. As a key insight, a better strategy is to train separate NNs for the plasma response fields with different toroidal mode numbers. It is also better to apply MOR and to subsequently train NNs separately for conventional and low aspect ratio devices, due to enhanced toroidal coupling of Fourier spectra in the plasma response in the latter case.

3D equilibrium↗
DOE OSTI2014

The dark side of galaxy colour: evidence from new SDSS measurements of galaxy clustering and lensing

The age matching model has recently been shown to predict correctly the luminosity L and g-r color of galaxies residing within dark matter halos. The central tenet of the model is intuitive: older halos tend to host galaxies with older stellar populations. In this paper, we demonstrate that age matching also correctly predicts the g-r color trends exhibited in a wide variety of statistics of the galaxy distribution for stellar mass M* threshold samples. In particular, we present new measurements of the galaxy two-point correlation function and the galaxy-galaxy lensing signal as a function of M* and g-r color from the Sloan Digital Sky Survey, and show that age matching exhibits remarkable agreement with these and other statistics of low-redshift galaxies. In so doing, we also demonstrate good agreement between the galaxy-galaxy lensing observed by SDSS and the signal predicted by abundance matching, a new success of this model. We describe how age matching is a specific example of a larger class of Conditional Abundance Matching models (CAM), a theoretical framework we introduce here for the first time. CAM provides a general formalism to study correlations at fixed mass between any galaxy property and any halo property. The striking success of our simple implementation of CAM provides compelling evidence that this technique has the potential to describe the same set of data as alternative models, but with a dramatic reduction in the required number of parameters. CAM achieves this reduction by exploiting the capability of contemporary N-body simulations to determine dark matter halo properties other than mass alone, which distinguishes our model from conventional approaches to the galaxy-halo connection.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2014

Band Excitation in Scanning Probe Microscopy: Recognition and Functional Imaging

Field confinement at the junction between a biased scanning probe microscope s (SPM) tip and solid surface enables local probing of various bias-induced transformations such as polarization switching, ionic motion, or electrochemical reactions to name a few. The nanoscale size of the biased region is smaller or comparable to features like grain boundaries and dislocations, potentially allows for the study of kinetics and thermodynamics at the level of a single defect. In contrast to classical statistically averaged approaches, this allows one to link structure to functionality and deterministically decipher associated mesoscopic and atomistic mechanisms. Furthermore, this type of information can serve as a fingerprint of local material functionality, allowing for local recognition imaging. Here, current progress in multidimensional SPM techniques based on band-excitation time and voltage spectroscopies is illustrated, including discussions on data acquisition, dimensionality reduction, and visualization along with future challenges and opportunities for the field.

Jesse, Stephen↗
DOE OSTI2021

First M87 Event Horizon Telescope Results. VII. Polarization of the Ring

In 2017 April, the Event Horizon Telescope (EHT) observed the near-horizon region around the supermassive black hole at the core of the M87 galaxy. These 1.3 mm wavelength observations revealed a compact asymmetric ring-like source morphology. This structure originates from synchrotron emission produced by relativistic plasma located in the immediate vicinity of the black hole. Here we present the corresponding linear-polarimetric EHT images of the center of M87. We find that only a part of the ring is significantly polarized. The resolved fractional linear polarization has a maximum located in the southwest part of the ring, where it rises to the level of ~15%. The polarization position angles are arranged in a nearly azimuthal pattern. We perform quantitative measurements of relevant polarimetric properties of the compact emission and find evidence for the temporal evolution of the polarized source structure over one week of EHT observations. The details of the polarimetric data reduction and calibration methodology are provided. We carry out the data analysis using multiple independent imaging and modeling techniques, each of which is validated against a suite of synthetic data sets. The gross polarimetric structure and its apparent evolution with time are insensitive to the method used to reconstruct the image. These polarimetric images carry information about the structure of the magnetic fields responsible for the synchrotron emission. Their physical interpretation is discussed in an accompanying publication.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Gridded daily weather data for North America with comprehensive uncertainty quantification

Abstract Access to daily high-resolution gridded surface weather data based on direct observations and over long time periods is essential for many studies and applications including vegetation, wildlife, soil health, hydrological modelling, and as driver data in Earth system models. We present Daymet V4, a 40-year daily meteorological dataset on a 1 km grid for North America, Hawaii, and Puerto Rico, providing temperature, precipitation, shortwave radiation, vapor pressure, snow water equivalent, and day length. The dataset includes an objective quantification of uncertainty based on strict cross-validation analysis for temperature and precipitation results. The dataset represents several improvements from a previous version, and this data descriptor provides complete documentation for updated methods. Improvements include: reductions in the timing bias of input reporting weather station measurements; improvement to the three-dimensional regression model techniques in the core algorithm; and a novel approach to handling high elevation temperature measurement biases. We show cross-validation analyses with the underlying weather station data to demonstrate the technical validity of new dataset generation methods, and to quantify improved accuracy.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2013

LINEBACkER: Bio-inspired Data Reduction Toward Real Time Network Traffic Analysis

Abstract—One essential component of resilient cyber applications is the ability to detect adversaries and protect systems with the same flexibility adversaries will use to achieve their goals. Current detection techniques do not enable this degree of flexibility because most existing applications are built using exact or regular-expression matching to libraries of rule sets. Further, network traffic defies traditional cyber security approaches that focus on limiting access based on the use of passwords and examination of lists of installed or downloaded programs. These approaches do not readily apply to network traffic occurring beyond the access control point, and when the data in question are combined control and payload data of ever increasing speed and volume. Manual analysis of network traffic is not normally possible because of the magnitude of the data that is being exchanged and the length of time that this analysis takes. At the same time, using an exact matching scheme to identify malicious traffic in real time often fails because the lists against which such searches must operate grow too large. In this work, we introduce an alternative method for cyber network detection based on similarity-measuring algorithms for gene sequence analysis. These methods are ideal because they were designed to identify similar but nonidentical sequences. We demonstrate that our method is generally applicable to the problem of network traffic analysis by illustrating its use in two different areas both based on different attributes of network traffic. Our approach provides a logical framework for organizing large collections of network data, prioritizing traffic of interest to human analysts, and makes it possible to discover traffic signatures without the bias introduced by expert-directed signature generation. Pattern recognition on reduced representations of network traffic offers a fast, efficient, and more robust way to detect anomalies.

Teuton, Jeremy R.↗
DOE OSTI2024

Machine learning in materials research: Developments over the last decade and challenges for the future

The number of studies that apply machine learning (ML) to materials science has been growing at a rate of approximately 1.67 times per year over the past decade. In this review, I examine this growth in various contexts. First, I present an analysis of the most commonly used tools (software, databases, materials science methods, and ML methods) used within papers that apply ML to materials science. The analysis demonstrates that despite the growth of deep learning techniques, the use of classical machine learning is still dominant as a whole. It also demonstrates how new research can effectively build upon past research, particular in the domain of ML models trained on density functional theory calculation data. Next, I present the progression of best scores as a function of time on the matbench materials science benchmark for formation enthalpy prediction. In particular, a dramatic improvement of 7 times reduction in error is obtained when progressing from feature-based methods that use conventional ML (random forest, support vector regression, etc.) to the use of graph neural network techniques. Finally, I provide views on future challenges and opportunities, focusing on data size and complexity, extrapolation, interpretation, access, and relevance.

36 MATERIALS SCIENCE↗
DOE OSTI2015

Variable length adjacent partitioning for PTS based PAPR reduction of OFDM signal

Peak-to-Average power ratio (PAPR) is a major drawback in OFDM communication. It leads the power amplifier into nonlinear region operation resulting into loss of data integrity. As such, there is a strong motivation to find techniques to reduce PAPR. Partial Transmit Sequence (PTS) is an attractive scheme for this purpose. Judicious partitioning the OFDM data frame into disjoint subsets is a pivotal component of any PTS scheme. Out of the existing partitioning techniques, adjacent partitioning is characterized by an attractive trade-off between cost and performance. With an aim of determining effects of length variability of adjacent partitions, we performed an investigation into the performances of a variable length adjacent partitioning (VL-AP) and fixed length adjacent partitioning in comparison with other partitioning schemes such as pseudorandom partitioning. Simulation results with different modulation and partitioning scenarios showed that fixed length adjacent partition had better performance compared to variable length adjacent partitioning. As expected, simulation results showed a slightly better performance of pseudorandom partitioning technique compared to fixed and variable adjacent partitioning schemes. However, as the pseudorandom technique incurs high computational complexities, adjacent partitioning schemes were still seen as favorable candidates for PAPR reduction.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2013

Multi-GPU Implementation of a 3D Finite Difference Time Domain Earthquake Code on Heterogeneous Supercomputers

We have developed a highly scalable 3D Finite Difference GPU code for use in earthquake engineering and disaster management through regional petascale earthquake simulations. This MPI-CUDA code is based on a widely-used wave propagation code called AWP-ODC and restructured for high throughput and efficiency on a heterogeneous computing architecture. We present an effective communication reduction technique for leveraging GPUs with minimal PCI-e overhead, and a novel overlapping method to fully hide data communication latency between GPUs. The optimization concept used in this work can be extended to general stencil computing on a structured grid. The benchmarks demonstrated sustained 100 TFlops in single precision for 49 billion mesh points using 952 GPUs on the NCCS Titan Phase 5 system, which is a 77-fold speedup compared to the CPU version of the code. This multi-GPU implementation has been validated and used for a large-scale verification wave propagation simulation of Mw5.4 Chino Hills earthquake using 128 GPUs.

58 GEOSCIENCES↗
DOE OSTI2022

Efficient Data Query for Gaussian Process Compressed Data through Value Range Estimation [Slides]

When the resolution of the data increases, data reduction methods are applied to simulation output, including Gaussian process, neural representation and compression algorithms. Lots of data analysis/visualization techniques requires data query, but data query from reduced representation is still challenging. This report will provide examples and provide possible answers to why data query from reduced representation is still challenging.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2015

Systems-based approach to understanding Technetium immobilization in the subsurface - 15326

The purpose of this work is to identify the chemical and biological reactions that occur and potentially influence the mobility of Technetium (Tc-99) in the subsurface. Tc-99 is a by-product of nuclear fission processes and has a relatively long half-life (t{sub 1/2}=212,000 years); additionally, it is highly soluble and mobile in groundwater and is most often remediated through pump and treat systems. The oxidized form of Tc-99 [Pertechnetate, Tc(VII)O{sub 4}-] is highly soluble and very mobile, while the reduced form of Tc-99 [Technetium-oxide, Tc(IV)O{sub 2}·nH{sub 2}O] is insoluble. Historically, complex interactions within the subsurface have made clean-up efforts for Tc-99 challenging. It is thought that environmental bacteria play a role in reducing contaminants to less soluble forms but it is difficult to predict or estimate the interactions and impacts of the complex, mixed microbial communities found in the environment. We have set up controlled batch experiments to examine a selection of organisms (Anaeromyxobacter dehalogenans strain CP-C, Geobacter sulfurreducens strain PAC, Geobacter daltonii strain FRC-32, and Shewanella oneidensis strain MR1) that have known metal reducing capabilities, for their ability to reduce Tc-99 to less soluble forms. An additional organism, Cellulomonas strain ES6 was chosen based on its origin of isolation from the Hanford site. Organisms were evaluated under anaerobic conditions both as isolates and in mixed consortia with and without the presence of iron-bearing mineral substrate. Experiments were monitored using spectroscopy, qPCR, volatile organic acid (VOA) analysis, and inductively coupled plasma-optical emission spectroscopy (ICP-OES) and mass spectrometer techniques (ICP-MS). Initial data collected has shown that the presence and type of substrate may increase the reduction of Tc-99 to less mobile forms. Data collected from these experiments will allow for (a) a better understanding of the mobility of Tc-99 in the subsurface, (b) better predictions of contaminant fate and transport over time, and (c) the development of long term remediation strategies for Tc-99. Additionally, information collected will be used to develop a systems-based metabolic model with predictive capabilities. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2019

Hyperspectral vegetation identification at a legacy underground nuclear explosion test site

The detection, location, and identification of suspected underground nuclear explosions (UNEs) are global security priorities that rely on integrated analysis of multiple data modalities for uncertainty reduction in event analysis. Vegetation disturbances may provide complementary signatures that can confirm or build on the observables produced by prompt sensing techniques such as seismic or radionuclide monitoring networks. For instance, the emergence of non-native species in an area may be indicative of anthropogenic activity or changes in vegetation health may reflect changes in the site conditions resulting from an underground explosion. Previously, we collected high spatial resolution (10 cm) hyperspectral data from an unmanned aerial system at a legacy underground nuclear explosion test site and its surrounds. These data consist of visible and near-infrared wavebands over 4.3 km 2 of high desert terrain along with high spatial resolution (2.5 cm) RGB context imagery. In this work, we employ various spectral detection and classification algorithms to identify and map vegetation species in an area of interest containing the legacy test site. Here, we employed a frequentist framework for fusing multiple spectral detections across various reference spectra captured at different times and sampled from multiple locations. The spatial distribution of vegetation species is compared to the location of the underground nuclear explosion. We discover a difference in species abundance within a 130 m radius of the center of the test site.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗