Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Nuclear Data Dimension Reduction

Our nuclear data consist of 30 energy group cross sections as well as nubar and PFNS. They are sampled from a Gaussian distribution with correlations shown in Figure 1. A few of these samples are shown in Figure 2. The purpose of this section is to describe how we can reduce the dimension of the nuclear data for purposes of uncertainty quantification. Sections 2 and 3 describe why dimension reduction is useful, as well as our approach to dimension reduction. When using these nuclear data as inputs to the PARTISN code, some of the resulting $k_{eff}$ values are unrealistic. For instance, when PARTISN is simulating the Jezebel critical assembly, some of the nuclear data yield $k_{eff}$ far from one. We would like to weight the nuclear data combinations that yield more realistic output higher, while downweighting those that result in unrealistic output. Our approach to this is described in Section 4. Throughout the other sections, we will merely refer to a prior distribution of nuclear data, which is not constrained by the Jezebel experiment, and a posterior distribution of nuclear data, which is conditioned on satisfactory performance for the Jezebel experiment. For instance, the correlation matrix for the reweighted (posterior) nuclear data is shown in Figure 3, along with the difference from the prior correlation matrix. Finally, we will describe dimension reduction results in Section 5.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Multilevel Techniques for Compression and Reduction of Scientific Data-Quantitative Control of Accuracy in Derived Quantities

Although many compression algorithms are focused on preserving pointwise values of the data, application scientists are generally more concerned with derived quantities. Equally well, the user may even be willing to accept a high level of lossiness in the compression provided that the compressed data respect certain invariants, such as mass conservation. In the current work, we develop a mathematical framework and techniques that enable data to be adaptively compressed while maintaining a specified tolerance on a class of user-prescribed quantities. Here, the algorithm is used to augment the functionality of the data reduction package MGARD developed in previous work and the functionality is illustrated by a range of application including data from computational simulation of autocatalytic reaction simulation, turbulent combustion simulation, experimental data obtained from magnetic confinement fusion experiment, and simulation of turbulent flow along a rectangular channel. In each case, we consider one or more relevant quantities of interest and reduce the data so as to preserve these quantities.

97 MATHEMATICS AND COMPUTING↗

Towards Resilient Near Real-Time Analysis Workflows in Fusion Energy Science

Nuclear fusion holds the promise of an endless source of energy. Several research experiments across the world and joint modeling and simulation efforts between the nuclear physics and high performance computing communities are actively preparing the operation of the International Thermonuclear Experimental Reactor (ITER). Both experimental reactors and their simulated counterparts generate data that must be analyzed quickly and in a resilient way to support decision making for the configuration of subsequent runs or prevent a catastrophic failure. However, the cost if the traditional techniques used to improve the resilience of analysis workflows, i.e., replicating datasets and computational tasks, becomes prohibitive with explosion of the volume of data produced by modern instruments and simulations. Therefore, we advocate in this paper for an alternate approach based on data reduction and data streaming. The rationale is that by allowing for a reasonable, controlled, and guaranteed loss of accuracy it becomes possible to transfer smaller amounts of data, shorten the execution time of analysis workflows, and lower the cost of replication to increase resilience. We develop our research and development roadmap towards resilient near real-time analysis workflows in fusion energy science and present early results showing that data streaming and data reduction is a promising way to speed up the execution and improve the resilience of analysis workflows.

Suter, Fred↗

Accelerating Data Acquisition, Reduction, and Analysis at the Spallation Neutron Source

ORNL operates the world's brightest neutron source, the Spallation Neutron Source (SNS). Funded by the US DOE Office of Basic Energy Science, this national user facility hosts hundreds of scientists from around the world, providing a platform to enable break-through research in materials science, sustainable energy, and basic science. While the SNS provides scientists with advanced experimental instruments, the deluge of data generated from these instruments represents both a big data challenge and a big data opportunity. For example, instruments at the SNS can now generate multiple millions of neutron events per second providing unprecedented experiment fidelity but leaving the user with a dataset that cannot be processed and analyzed in a timely fashion using legacy techniques. To address this big data challenge, ORNL has developed a near real-time streaming data reduction and analysis infrastructure. The Accelerating Data Acquisition, Reduction, and Analysis (ADARA) system provides a live streaming data infrastructure based on a high-performance publish subscribe system, in situ data reduction, visualization, and analysis tools, and integration with a high-performance computing and data storage infrastructure. ADARA allows users of the SNS instruments to analyze their experiment as it is run and make changes to the experiment in real-time and visualize the results of these changes. In this paper we describe ADARA, provide a high-level architectural overview of the system, and present a set of use-cases and real-world demonstrations of the technology.

Campbell, Stuart I↗

A Codesign Framework for Online Data Analysis and Reduction

In this paper we discuss our design of a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for performing parametric studies on coupled applications. Cheetah facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage vs. compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next-generation simulations that make use of online processing techniques. We illustrate the design choices of Cheetah by using a reaction-diffusion simulation (Gray-Scott) paired with an analysis application to demonstrate initial results of fine-grained process placement on Summit, a pre-exascale supercomputer at Oak Ridge National Laboratory.

Mehta, Kshitij↗

MetaStrider: Architectures for Scalable Memory-centric Reduction of Sparse Data Streams

Reduction is an operation performed on the values of two or more key-value pairs that share the same key. Reduction of sparse data streams finds application in a wide variety of domains such as data and graph analytics, cybersecurity, machine learning, and HPC applications. However, these applications exhibit low locality of reference, rendering traditional architectures and data representations inefficient. This article presents MetaStrider, a significant algorithmic and architectural enhancement to the state-of-the-art, SuperStrider. Furthermore, these enhancements enable a variety of parallel, memory-centric architectures that we propose, resulting in demonstrated performance that scales near-linearly with available memory-level parallelism.

97 MATHEMATICS AND COMPUTING↗

Automated single-ion peak fitting as an efficient approach for analyzing complex chromatographic data

Chromatography provides important detail on the composition of environmental samples and their chemical processing. However, the complexity of these samples and their tendency to contain many structurally and chemically similar compounds frequently results in convoluted or poorly resolved data. Data reduction from raw chromatograms of complex environmental data into integrated peak areas consequently often requires substantial operator interaction. This difficulty has led to a bottleneck in analysis that increases analysis time, decreases data quality, and will worsen as advances in field-based instrumentation multiply the quantity and informational density of data produced. In this work, we develop and validate an automated approach to fitting chromatographic data within a target retention time window with a combination of multiple idealized peaks (Gaussian peaks either with or without an exponential decay component). Herein we compare this single-ion peak fitting approach to drawn baseline integration methods of more than 70,000 peaks collected by field-based chromatographs spanning across a wide range of volatilities and functionalities. Accuracy of peak fitting under real-world conditions is found to be within 10%. The quantitative parameters describing the fit (e.g. coefficients, fit residuals, etc.) are found to provide valuable information to increase the efficiency of quality control and provide constraints to accurately integrate peaks that are significantly convoluted with neighboring peaks. Implementation of the peak fitting method is shown to yield accurate integration of peaks otherwise too poorly resolved to separate into individual compounds and improved quantitative metrics to determine the fidelity of the data reduction process, while substantially decreasing the time spent by operators on data reduction.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Multimodal X-ray nano-spectromicroscopy analysis of chemically heterogeneous systems

Abstract Understanding the nanoscale chemical speciation of heterogeneous systems in their native environment is critical for several disciplines such as life and environmental sciences, biogeochemistry, and materials science. Synchrotron-based X-ray spectromicroscopy tools are widely used to understand the chemistry and morphology of complex material systems owing to their high penetration depth and sensitivity. The multidimensional (4D+) structure of spectromicroscopy data poses visualization and data-reduction challenges. This paper reports the strategies for the visualization and analysis of spectromicroscopy data. We created a new graphical user interface and data analysis platform named XMIDAS (X-ray multimodal image data analysis software) to visualize spectromicroscopy data from both image and spectrum representations. The interactive data analysis toolkit combined conventional analysis methods with well-established machine learning classification algorithms (e.g. nonnegative matrix factorization) for data reduction. The data visualization and analysis methodologies were then defined and optimized using a model particle aggregate with known chemical composition. Nanoprobe-based X-ray fluorescence (nano-XRF) and X-ray absorption near edge structure (nano-XANES) spectromicroscopy techniques were used to probe elemental and chemical state information of the aggregate sample. We illustrated the complete chemical speciation methodology of the model particle by using XMIDAS. Next, we demonstrated the application of this approach in detecting and characterizing nanoparticles associated with alveolar macrophages. Our multimodal approach combining nano-XRF, nano-XANES, and differential phase-contrast imaging efficiently visualizes the chemistry of localized nanostructure with the morphology. We believe that the optimized data-reduction strategies and tool development will facilitate the analysis of complex biological and environmental samples using X-ray spectromicroscopy techniques.

36 MATERIALS SCIENCE↗

Automated data processing architecture for the Gemini Planet Imager Exoplanet Survey

The Gemini Planet Imager Exoplanet Survey (GPIES) is a multiyear direct imaging survey of 600 stars to discover and characterize young Jovian exoplanets and their environments. We have developed an automated data architecture to process and index all data related to the survey uniformly. An automated and flexible data processing framework, which we term the Data Cruncher, combines multiple data reduction pipelines (DRPs) together to process all spectroscopic, polarimetric, and calibration data taken with GPIES. With no human intervention, fully reduced and calibrated data products are available less than an hour after the data are taken to expedite follow up on potential objects of interest. The Data Cruncher can run on a supercomputer to reprocess all GPIES data in a single day as improvements are made to our DRPs. A backend MySQL database indexes all files, which are synced to the cloud, and a front-end web server allows for easy browsing of all files associated with GPIES. To help observers, quicklook displays show reduced data as they are processed in real time, and chatbots on Slack post observing information as well as reduced data products. Together, the GPIES automated data processing architecture reduces our workload, provides real-time data reduction, optimizes our observing strategy, and maintains a homogeneously reduced dataset to study planet occurrence and instrument performance.

79 ASTRONOMY AND ASTROPHYSICS↗

AMM: Adaptive Multilinear Meshes

Adaptive representations are increasingly indispensable for reducing the in-memory and on-disk footprints of large-scale data. Usual solutions are designed broadly along two themes: reducing data precision, e.g., through compression, or adapting data resolution, e.g., using spatial hierarchies. Additionally, recent research suggests that combining the two approaches, i.e., adapting both resolution and precision simultaneously, can offer significant gains over using them individually. However, there currently exist no practical solutions to creating and evaluating such representations at scale. In this work, we present a new resolution-precision-adaptive representation to support hybrid data reduction schemes and offer an interface to existing tools and algorithms. Through novelties in spatial hierarchy, our representation, Adaptive Multilinear Meshes (AMM), provides considerable reduction in the mesh size. AMM creates a piecewise multilinear representation of uniformly sampled scalar data and can selectively relax or enforce constraints on conformity, continuity, and coverage, delivering a flexible adaptive representation. AMM also supports representing the function using mixed-precision values to further the achievable gains in data reduction. We describe a practical approach to creating AMM incrementally using arbitrary orderings of data and demonstrate AMM on six types of resolution and precision datastreams. By interfacing with state-of-the-art rendering tools through VTK, we demonstrate the practical and computational advantages of our representation for visualization techniques. With an open-source release of our tool to create AMM, we make such evaluation of data reduction accessible to the community, which we hope will foster new opportunities and future data reduction schemes.

97 MATHEMATICS AND COMPUTING↗

Preparing for In Situ Processing on Upcoming Leading-edge Supercomputers

High performance computing applications are producing increasingly large amounts of data and placing enormous stress on current capabilities for traditional post-hoc visualization techniques. Because of the growing compute and I/O imbalance, data reductions, including in situ visualization, are required. These reduced data are used for analysis and visualization in a variety of different ways. Many of he visualization and analysis requirements are known a priori, but when they are not, scientists are dependent on the reduced data to accurately represent the simulation in post hoc analysis. The contributions of this paper is a description of the directions we are pursuing to assist a large scale fusion simulation code succeed on the next generation of supercomputers. Finally, these directions include the role of in situ processing for performing data reductions, as well as the tradeoffs between data size and data integrity within the context of complex operations in a typical scientific workflow.

97 MATHEMATICS AND COMPUTING↗

THE HUBBLE SPACE TELESCOPE TREASURY PROGRAM ON THE ORION NEBULA CLUSTER {sup ,}

The Hubble Space Telescope (HST) Treasury Program on the Orion Nebula Cluster (ONC) has used 104 orbits of HST time to image the Great Orion Nebula region with the Advanced Camera for Surveys (ACS), the Wide-Field/Planetary Camera 2 (WFPC2), and the Near-Infrared Camera and Multi-Object Spectrograph (NICMOS) instrument in 11 filters ranging from the U band to the H band equivalent of HST. The program has been intended to perform the definitive study of the stellar component of the ONC at visible wavelengths, addressing key questions like the cluster initial mass function, age spread, mass accretion, binarity, and cirumstellar disk evolution. The scanning pattern allowed us to cover a contiguous field of approximately 600 arcmin{sup 2} with both ACS and WFPC2, with a typical exposure time of approximately 11 minutes per ACS filter, corresponding to a point source depth AB(F435W) = 25.8 and AB(F775W) = 25.2 with 0.2 mag of photometric error. We describe the observations, data reduction, and data products, including images, source catalogs, and tools for quick look preview. In particular, we provide ACS photometry for 3399 stars, most of them detected at multiple epochs; WFPC2 photometry for 1643 stars, 1021 of them detected in the U band; and NICMOS JH photometry for 2116 stars. We summarize the early science results that have been presented in a number of papers. The final set of images and the photometric catalogs are publicly available through the archive as High Level Science Products at the STScI Multimission Archive hosted by the Space Telescope Science Institute.

79 ASTRONOMY AND ASTROPHYSICS↗

archi : pipeline for light curve extraction of CHEOPS background stars

ABSTRACT High precision time-series photometry from space is being used for a number of scientific cases. In this context, the recently launched CHaracterizing ExOPlanet Satellite (CHEOPS) (ESA) mission promises to bring 20 ppm precision over an exposure time of 6 h, when targeting nearby bright stars, having in mind the detailed characterization of exoplanetary systems through transit measurements. However, the official CHEOPS (ESA) mission pipeline only provides photometry for the main target (the central star in the field). In order to explore the potential of CHEOPS photometry for all stars in the field, in this paper, we present archi, an additional open-source pipeline module1 to analyse the background stars present in the image. As archi uses the official data reduction pipeline data as input, it is not meant to be used as an independent tool to process raw CHEOPS data but, instead, to be used as an add-on to the official pipeline. We test archi using CHEOPS simulated images, and show that photometry of background stars in CHEOPS images is only slightly degraded (by a factor of 2–3) with respect to the main target. This opens a potential for the use of CHEOPS to produce photometric time-series of several close-by targets at once, as well as to use different stars in the image to calibrate systematic errors. We also show one clear scientific application where the study of the companion light curve can be important for the understanding of the contamination on the main target.

Silva, André M.↗

Ultra-small-angle x-ray scattering at the Advanced Photon Source.

The design and operation of a versatile ultra-small-angle X-ray scattering (USAXS) instrument at the Advanced Photon Source (APS) at Argonne National Laboratory are presented. The instrument is optimized for the high brilliance and low emittance of an APS undulator source. It has angular and energy resolutions of the order of 10{sup -4}, accurate and repeatable X-ray energy tunability over its operational energy range from 8 to 18 keV, and a dynamic intensity range of 10{sup 8} to 10{sup 9}, depending on the configuration. It further offers quantitative primary calibration of X-ray scattering cross sections, a scattering vector range from 0.0001 to 1 {angstrom}{sup -1}, and stability and reliability over extended running periods. Its operational configurations include one-dimensional collimated (slit-smeared) USAXS, two-dimensional collimated USAXS and USAXS imaging. A robust data reduction and data analysis package, which was developed in parallel with the instrument, is available and supported at the APS.

43 PARTICLE ACCELERATORS↗

LANL CSSE L2: Case Study of In Situ Data Analysis in ASC Integrated Codes [Slides]

The massive, complex datasets generated by scientific simulations on next-generation computer architectures at extreme scale pose new challenges for data management and scientific understanding. A critical element in meeting these new challenges is in situ data analysis where information is reduced and analyzed as close to the point of generation as possible, for example, at run time or as the data stream to longer term storage. This milestone will demonstrate in situ data analysis in xRAGE, an ASC Eulerian Radiation Hydrodynamics code. We will work with the Eulerian Applications Project to identify target problems that will exercise physics of interest to the program, and we will work closely with users to maximize the achieved usability and productivity improvements. We will investigate enhancements to the in situ process that enable and improve techniques of data reduction and data triage in addition to improving in situ performance by using portable acceleration technology. We will report on their viability and performance in xRAGE or another ASC code. These products will provide ASC IC developers with reference codes and tools that will benefit DSW by advancing the technology for predictive science at extreme scales.

97 MATHEMATICS AND COMPUTING↗

Demonstration and Feasibility Analyses of using the Dual-Purpose Canister Drain Pipe to Support the Filling Process

This report documents experiments that are being used to determine the feasibility of filling dual-purpose canisters (DPCs) using the existing drain pipe. Mercury was identified as a surrogate filler for the initial experiments. Oak Ridge National Laboratory’s Target Test Facility (TTF), which is a full-scale Spallation Neutron Source (SNS) prototypical mercury loop, is being used for the planned pipe experiments using mercury. The pipe experiments were divided into two phases to provide a separate effects approach to the problem and to allow for development of system scalability. The first phase included taking strain measurements in a short length pipe, as well as measurements of flow-induced vibration. The second phase consists of the design and construction of a separated mercury flow loop capable of hosting exchangeable test sections and more localized instrumentation. Data acquisition of Phase I was performed during this reporting time period, and subsequent data reduction and data analysis will be performed in the future.

36 MATERIALS SCIENCE↗

LOW X-RAY LUMINOSITY GALAXY CLUSTERS: MAIN GOALS, SAMPLE SELECTION, PHOTOMETRIC AND SPECTROSCOPIC OBSERVATIONS

We present our study of 19 low X-ray luminosity galaxy clusters (L{sub X} ∼ 0.5–45 × 10{sup 43} erg s{sup −1}), selected from the ROSAT Position Sensitive Proportional Counters Pointed Observations and the revised version of Mullis et al. in the redshift range of 0.16–0.7. This is the introductory paper of a series presenting the sample selection, photometric and spectroscopic observations, and data reduction. Photometric data in different passbands were taken for eight galaxy clusters at Las Campanas Observatory; three clusters at Cerro Tololo Interamerican Observatory; and eight clusters at the Gemini Observatory. Spectroscopic data were collected for only four galaxy clusters using Gemini telescopes. Using the photometry, the galaxies were defined based on the star-galaxy separation taking into account photometric parameters. For each galaxy cluster, the catalogs contain the point-spread function and aperture magnitudes of galaxies within the 90% completeness limit. They are used together with structural parameters to study the galaxy morphology and to estimate photometric redshifts. With the spectroscopy, the derived galaxy velocity dispersion of our clusters ranged from 507 km s{sup −1} for [VMF98]022 to 775 km s{sup −1} for [VMF98]097 with signs of substructure. Cluster membership has been extensively discussed taking into account spectroscopic and photometric redshift estimates. In this sense, members are the galaxies within a projected radius of 0.75 Mpc from the X-ray emission peak and with clustercentric velocities smaller than the cluster velocity dispersion or 6000 km s{sup −1}, respectively. These results will be used in forthcoming papers to study, among the main topics, the red cluster sequence, blue cloud and green populations, the galaxy luminosity function, and cluster dynamics.

79 ASTRONOMY AND ASTROPHYSICS↗

Indoor Dust Remedial Investigation Sampling at the Colonie FUSRAP Site - 15578

The Colonie Formerly Utilized Sites Remedial Action Program (FUSRAP) Site is located in the Town of Colonie, Albany County, New York. The U.S. Army Corps of Engineers (USACE) is currently addressing environmental contamination associated with the Site. The area surrounding the Site consists of residential and commercial properties, known as vicinity properties. Cleanup activities have been completed on many of these VPs under the FUSRAP to address depleted uranium contamination. Soil remediation activities have been substantially completed at the Colonie FUSRAP Site and its VPs under the FUSRAP. Recent studies performed by an independent party and under FUSRAP identified uranium within indoor dust in residencies and businesses in the immediate vicinity of the Site. The studies were limited to non-living areas such as basement window sills, garages, and attics. This paper discusses the indoor dust Remedial Investigation (RI) for the VPs at the Colonie FUSRAP Site. The indoor dust sample media presented many technical challenges regarding sampling design, data reduction, and data interpretation. In addition, because the RI is focused on numerous resident-owned, rental, and commercial properties, challenges where presented regarding right-of-entry to sample and concerns were raised regarding confidentiality of the results. In response to this, a community outreach communication and education program is being developed to mitigate property-owner concerns. The paper also identifies specific action levels used to determine future actions. Ultimately, property-specific risk assessments, performed at properties exceeding the action levels, will be used to determine whether a response action is appropriate under FUSRAP. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗