Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Determination of hydrogen cluster velocities and comparison with numerical calculations

The use of powerful hydrogen cluster jet targets in storage ring experiments led to the need of precise data on the mean cluster velocity as function of the stagnation temperature and pressure for the determination of the volume density of the target beams. For this purpose a large data set of hydrogen cluster velocity distributions and mean velocities was measured at a high density hydrogen cluster jet target using a trumpet shaped nozzle. The measurements have been performed at pressures above and below the critical pressure and for a broad range of temperatures relevant for target operation, e.g., at storage ring experiments. The used experimental method is described which allows for the velocity measurement of single clusters using a time-of-flight technique. Since this method is rather time-consuming and these measurements are typically interfering negatively with storage ring experiments, a method for a precise calculation of these mean velocities was needed. For this, the determined mean cluster velocities are compared with model calculations based on an isentropic one-dimensional van der Waals gas. Based on the obtained data and the presented numerical calculations, a new method has been developed which allows to predict the mean cluster velocities with an accuracy of about 5%. For this two cut-off parameters defining positions inside the nozzle are introduced, which can be determined for a given nozzle by only two velocity measurements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Population Genomic Analysis of 1,777 Extended-Spectrum Beta-Lactamase-Producing Klebsiella pneumoniae Isolates, Houston, Texas: Unexpected Abundance of Clonal Group 307

Klebsiella pneumoniae is a major human pathogen responsible for high morbidity and mortality rates. The emergence and spread of strains resistant to multiple antimicrobial agents and documented large nosocomial outbreaks are especially concerning. To develop new therapeutic strategies for K. pneumoniae, it is imperative to understand the population genomic structure of strains causing human infections. To address this knowledge gap, we sequenced the genomes of 1,777 extended-spectrum beta-lactamase-producing K. pneumoniae strains cultured from patients in the 2,000-bed Houston Methodist Hospital system between September 2011 and May 2015, representing a comprehensive, population-based strain sample. Strains of largely uncharacterized clonal group 307 (CG307) caused more infections than those of well-studied epidemic CG258. Strains varied markedly in gene content and had an extensive array of small and very large plasmids, often containing antimicrobial resistance genes. Some patients with multiple strains cultured over time were infected with genetically distinct clones. We identified 15 strains expressing the New Delhi metallo-beta-lactamase 1 (NDM-1) enzyme that confers broad resistance to nearly all beta-lactam antibiotics. Transcriptome sequencing analysis of 10 phylogenetically diverse strains showed that the global transcriptome of each strain was unique and highly variable. Experimental mouse infection provided new information about immunological parameters of host-pathogen interaction. We exploited the large data set to develop whole-genome sequence-based classifiers that accurately predict clinical antimicrobial resistance for 12 of the 16 antibiotics tested. We conclude that analysis of large, comprehensive, population-based strain samples can assist understanding of the molecular diversity of these organisms and contribute to enhanced translational research.

59 BASIC BIOLOGICAL SCIENCES↗

Analyzing Ever Growing Datasets in PHENIX

After 10 years of running, the PHENIX experiment has by now accumulated more than 700 TB of reconstructed data which are directly used for analysis. Analyzing these amounts of data efficiently requires a coordinated approach. Beginning in 2005 we started to develop a system for the RHIC Atlas Computing Facility (RACF) which allows the efficient analysis of these large data sets. The Analysis Taxi is now the tool which allows any collaborator to process any data set taken since 2003 in weekly passes with turnaround times of typically three to four days.

43 PARTICLE ACCELERATORS↗

Use of the Tool to Support Renewable Energy Auctions Processes (RE Data Explorer)

Renewable energy auctions are now a common competitive approach to procure low-cost renewable power around the world. Ensuring a successful auction process increasingly depends on the capabilities of auction designers and participants to identify actionable and defensible insights from large data sets (on renewable energy resources and complementary data) to both attract potential investors and address stakeholder concerns. The Renewable Energy (RE) Data Explorer is a user-friendly geospatial analysis tool for analyzing renewable energy potential and informing decisions. Developed by the National Renewable Energy Laboratory (NREL) and supported by the U.S. Agency for International Development (USAID), RE Data Explorer performs visualization and analysis of renewable energy potential that can be customized for different scenarios. RE Data Explorer can support prospecting, integrated planning, policymaking, and other decision-making activities to accelerate renewable energy deployment. The broader RE Explorer website provides guidance and information to link the RE Data Explorer geospatial analysis tool to key decision areas. This document provides information on how the RE Data Explorer can be used to support renewable energy auction processes.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Clusters of galaxies up to z = 1.5 identified from photometric data of the Dark Energy Survey and unWISE

ABSTRACT Using photometric data from the Dark Energy Survey and the Wide-field Infrared Survey Explorer, we estimate photometric redshifts for 105 million galaxies using the nearest-neighbour algorithm. From such a large data base, 151 244 clusters of galaxies are identified in the redshift range of 0.1 < z ≲ 1.5 based on the overdensity of the total stellar mass of galaxies within a given photometric redshift slice, among which 76 826 clusters are newly identified and 30 477 clusters have a redshift z > 1. We cross-match these clusters with those in the catalogues identified from the X-ray surveys and the Sunyaev–Zel’dovich (SZ) effect by the Planck, South Pole Telescope and Atacama Cosmology Telescope surveys, and get the redshifts for 45 X-ray clusters and 56 SZ clusters. More than 95 per cent SZ clusters in the sky region have counterparts in our catalogue. We find multiple optical clusters in the line of sight towards about 15 per cent of SZ clusters.

79 ASTRONOMY AND ASTROPHYSICS↗

Quantum chaotic scattering in microwave resonators

In a frequency range where a microwave resonator simulates a chaotic quantum billiard, we have measured moduli and phases of reflection and transmission amplitudes in the regimes of both isolated and of weakly overlapping resonances and for resonators with and without time-reversal invariance. Statistical measures for S-matrix fluctuations were determined from the data and compared with extant and/or newly derived theoretical results obtained from the random-matrix approach to quantum chaotic scattering. The latter contained a small number of fit parameters. The large data sets taken made it possible to test the theoretical expressions with unprecedented accuracy. The theory is confirmed by both a goodness-of-fit-test and the agreement of predicted values for those statistical measures that were not used for the fits, with the data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

TEACHING AN OLD ACCELERATOR NEW TRICKS

The Argonne Tandem Linac Accelerator System (ATLAS) has been a National User Facility since 1985. In that time, many of the systems that help operators retrieve, modify, and store beamline parameters have not kept pace with the advancement of technology. Development of a new method of storing and retrieving beamline parameters resulted in the testing and installation of a time-series database as a potential replacement for the traditional relational database. InfluxDB was selected due to its self-hosted Open-Source version availability as well as the simplicity of installation and setup. A program was written to periodically gather all accelerator parameters in the control system and store them in the time-series database. This resulted in over 13,000 distinct data points, captured at 5-minute intervals. A second test captured 35 channels on a 1-minute cadence. Graphing of the captured data is being done on Grafana, an Open-Source version is available that co-exists well with InfluxDB as the back-end. Grafana made visualizing the data simple and flexible. The testing has allowed for the use of modern graphing tools to generate new insights into operating the accelerator, as well as opened the door to building large data sets suitable for Artificial Intelligence and Machine Learning applications.

Novak, D.↗

Methods and apparatus of analyzing electrical power grid data

Apparatus and methods of processing large-scale data regarding an electrical power grid are described. According to one aspect, a method of processing large-scale data regarding an electrical power grid includes accessing a large-scale data set comprising information regarding an electrical power grid; processing data of the large-scale data set to identify a filter which is configured to remove erroneous data from the large-scale data set; using the filter, removing erroneous data from the large-scale data set; and after the removing, processing data of the large-scale data set to identify an event detector which is configured to identify events of interest in the large-scale data set.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Mahanaxar: quality of service guarantees in high-bandwidth, real-time streaming data storage

Large radio telescopes, cyber-security systems monitoring real-time network traffic, and others have specialized data storage needs: guaranteed capture of an ultra-high-bandwidth data stream, retention of the data long enough to determine what is 'interesting,' retention of interesting data indefinitely, and concurrent read/write access to determine what data is interesting, without interrupting the ongoing capture of incoming data. Mahanaxar addresses this problem. Mahanaxar guarantees streaming real-time data capture at (nearly) the full rate of the raw device, allows concurrent read and write access to the device on a best-effort basis without interrupting the data capture, and retains data as long as possible given the available storage. It has built in mechanisms for reliability and indexing, can scale to meet arbitrary bandwidth requirements, and handles both small and large data elements equally well. Results from our prototype implementation shows that Mahanaxar provides both better guarantees and better performance than traditional file systems.

97 MATHEMATICS AND COMPUTING↗

Multi-GPU implementation of a VMAT treatment plan optimization algorithm

Purpose: Volumetric modulated arc therapy (VMAT) optimization is a computationally challenging problem due to its large data size, high degrees of freedom, and many hardware constraints. High-performance graphics processing units (GPUs) have been used to speed up the computations. However, GPU’s relatively small memory size cannot handle cases with a large dose-deposition coefficient (DDC) matrix in cases of, e.g., those with a large target size, multiple targets, multiple arcs, and/or small beamlet size. The main purpose of this paper is to report an implementation of a column-generation-based VMAT algorithm, previously developed in the authors’ group, on a multi-GPU platform to solve the memory limitation problem. While the column-generation-based VMAT algorithm has been previously developed, the GPU implementation details have not been reported. Hence, another purpose is to present detailed techniques employed for GPU implementation. The authors also would like to utilize this particular problem as an example problem to study the feasibility of using a multi-GPU platform to solve large-scale problems in medical physics. Methods: The column-generation approach generates VMAT apertures sequentially by solving a pricing problem (PP) and a master problem (MP) iteratively. In the authors’ method, the sparse DDC matrix is first stored on a CPU in coordinate list format (COO). On the GPU side, this matrix is split into four submatrices according to beam angles, which are stored on four GPUs in compressed sparse row format. Computation of beamlet price, the first step in PP, is accomplished using multi-GPUs. A fast inter-GPU data transfer scheme is accomplished using peer-to-peer access. The remaining steps of PP and MP problems are implemented on CPU or a single GPU due to their modest problem scale and computational loads. Barzilai and Borwein algorithm with a subspace step scheme is adopted here to solve the MP problem. A head and neck (H and N) cancer case is then used to validate the authors’ method. The authors also compare their multi-GPU implementation with three different single GPU implementation strategies, i.e., truncating DDC matrix (S1), repeatedly transferring DDC matrix between CPU and GPU (S2), and porting computations involving DDC matrix to CPU (S3), in terms of both plan quality and computational efficiency. Two more H and N patient cases and three prostate cases are used to demonstrate the advantages of the authors’ method. Results: The authors’ multi-GPU implementation can finish the optimization process within ∼1 min for the H and N patient case. S1 leads to an inferior plan quality although its total time was 10 s shorter than the multi-GPU implementation due to the reduced matrix size. S2 and S3 yield the same plan quality as the multi-GPU implementation but take ∼4 and ∼6 min, respectively. High computational efficiency was consistently achieved for the other five patient cases tested, with VMAT plans of clinically acceptable quality obtained within 23–46 s. Conversely, to obtain clinically comparable or acceptable plans for all six of these VMAT cases that the authors have tested in this paper, the optimization time needed in a commercial TPS system on CPU was found to be in an order of several minutes. Conclusions: The results demonstrate that the multi-GPU implementation of the authors’ column-generation-based VMAT optimization can handle the large-scale VMAT optimization problem efficiently without sacrificing plan quality. The authors’ study may serve as an example to shed some light on other large-scale medical physics problems that require multi-GPU techniques.

60 APPLIED LIFE SCIENCES↗

Data and Analysis Preservation in the PHENIX Experiment at RHIC

The PHENIX experiment (the Pioneering High Energy Nuclear Interaction eXperiment) is the largest of the four experiments that have operated at the Relativistic Heavy Ion Collider (RHIC), taking data in 2000-2016. PHENIX has made fundamental contributions to the discovery and study of the Quark-Gluon Plasma, advancement of spin physics and other areas. Currently, the PHENIX Collaboration is analyzing large data samples previously collected, while facing challenges in the area of Data and Analysis Preservation. We describe the strategy and practices employed by the PHENIX Collaboration to meet these challenges by leveraging state-of-the-art platforms and tools created and maintained by the High Energy and Nuclear Physics communities, including Zenodo, HEPData, OpenData, REANA and others.

97 MATHEMATICS AND COMPUTING↗

Cheetah : software for high-throughput reduction and analysis of serial femtosecond X-ray diffraction data

The emerging technique of serial X-ray diffraction, in which diffraction data are collected from samples flowing across a pulsed X-ray source at repetition rates of 100 Hz or higher, has necessitated the development of new software in order to handle the large data volumes produced. Sorting of data according to different criteria and rapid filtering of events to retain only diffraction patterns of interest results in significant reductions in data volume, thereby simplifying subsequent data analysis and management tasks. Meanwhile the generation of reduced data in the form of virtual powder patterns, radial stacks, histograms and other meta data creates data set summaries for analysis and overall experiment evaluation. Rapid data reduction early in the analysis pipeline is proving to be an essential first step in serial imaging experiments, prompting the authors to make the tool described in this article available to the general community. Originally developed for experiments at X-ray free-electron lasers, the software is based on a modular facility-independent library to promote portability between different experiments and is available under version 3 or later of the GNU General Public License.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

An Approach to Enhance pnetCDF Performance in Environmental Modeling Applications

I/O has been considered as a bottleneck in parallel applications. The software package, pnetCDF which works with parallel file systems, was developed to address this issue and provide parallel I/O capability. This study examines the performance of a novel approach which performs data aggregation along either row or column dimension of the spatial domain, and then applies the pnetCDF parallel I/O paradigm. The test was done with three different domain sizes which represents small, moderately large and large data domains, using a small scale Community Multi-scale Air Quality model (CMAQ) mocked up code. The examination includes comparing I/O performance with traditional serial I/O technique, straight application of pnetCDF, and the data aggregation along row and column dimension before applying pnetCDF. After the comparison, “optimal” I/O configurations for this new novel approach were quantified. Data aggregation along the row dimension (pnetCDFcr) works better than along the column dimension (pnetCDFcc) although it may perform slightly worse than straight the pnetCDF method with a small number of processors. When the number of processors becomes larger, pnetCDFcr out performs pnetCDF significantly. If the number of processors keeps increasing, pnetCDF reaches a point that the performance is even worse than the serial I/O technique. This new approach has also been tested on a real application where it performs two times better than the straight pnetCDF paradigm.

I/O↗

Managing high-bandwidth real-time data storage

There exist certain systems which generate real-time data at high bandwidth, but do not necessarily require the long-term retention of that data in normal conditions. In some cases, the data may not actually be useful, and in others, there may be too much data to permanently retain in long-term storage whether it is useful or not. However, certain portions of the data may be identified as being vitally important from time to time, and must therefore be retained for further analysis or permanent storage without interrupting the ongoing collection of new data. We have developed a system, Mahanaxar, intended to address this problem. It provides quality of service guarantees for incoming real-time data streams and simultaneous access to already-recorded data on a best-effort basis utilizing any spare bandwidth. It has built in mechanisms for reliability and indexing, can scale upwards to meet increasing bandwidth requirements, and handles both small and large data elements equally well. We will show that a prototype version of this system provides better performance than a flat file (traditional filesystem) based version, particularly with regard to quality of service guarantees and hard real-time requirements.

99 GENERAL AND MISCELLANEOUS↗

Input/Output of ab-initio nuclear structure calculations for improved performance and portability

Many modern scientific applications rely on highly computation intensive calculations. However, most applications do not concentrate as much on the role that input/output operations can play for improved performance and portability. Parallelizing input/output operations of large files can significantly improve the performance of parallel applications where sequential I/O is a bottleneck. A proper choice of I/O library also offers a scope for making input/output operations portable across different architectures. Thus, use of parallel I/O libraries for organizing I/O of large data files offers great scope in improving performance and portability of applications. In particular, sequential I/O has been identified as a bottleneck for the highly scalable MFDn (Many Fermion Dynamics for nuclear structure) code performing ab-initio nuclear structure calculations. We develop interfaces and parallel I/O procedures to use a well-known parallel I/O library in MFDn. As a result, we gain efficient I/O of large datasets along with their portability and ease of use in the down-stream processing. Even situations where the amount of data to be written is not huge, proper use of input/output operations can boost the performance of scientific applications. Application checkpointing offers enormous performance improvement and flexibility by doing a negligible amount of I/O to disk. Checkpointing saves and resumes application state in such a manner that in most cases the application is unaware that there has been an interruption to its execution. This helps in saving large amount of work that has been previously done and continue application execution. This small amount of I/O provides substantial time saving by offering restart/resume capability to applications. The need for checkpointing in optimization code NEWUOA has been identified and checkpoint/restart capability has been implemented in NEWUOA by using simple file I/O.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

ADIOS Visualization Schema: A First Step Towards Improving Interdisciplinary Collaboration in High Performance Computing

Scientific communities have benefitted from a significant increase of available computing and storage resources in the last few decades. For science projects that have access to leadership scale computing resources, the capacity to produce data has been growing exponentially. Teams working on such projects must now include, in addition to the traditional application scientists, experts in various disciplines including applied mathematicians for development of algorithms, visualization specialists for large data, and I/O specialists. Sharing of knowledge and data is becoming a requirement for scientific discovery, providing useful mechanisms to facilitate this sharing is a key challenge for e-Science. Our hypothesis is that in order to decrease the time to solution for application scientists we need to lower the barrier of entry into related computing fields. We aim at improving users' experience when interacting with a vast software ecosystem and/or huge amount of data, while maintaining focus on their primary research field. In this context we present our approach to bridge the gap between the application scientists and the visualization experts through a visualization schema as a first step and proof of concept for a new way to look at interdisciplinary collaboration among scientists dealing with big data. The key to our approach is recognizing that our users are scientists who mostly work as islands. They tend to work in very specialized environment but occasionally have to collaborate with other researchers in order to take full advantage of computing innovations and get insight from big data. We present an example of identifying the connecting elements between one of such relationships and offer a liaison schema to facilitate their collaboration.

Computer Science↗

Modeling adsorption properties of structurally deformed metal–organic frameworks using structure–property map

Structural deformation and collapse in metal-organic frameworks (MOFs) can lead to loss of long-range order, making it a challenge to model these amorphous materials using conventional computational methods. In this work, we show that a structure–property map consisting of simulated data for crystalline MOFs can be used to indirectly obtain adsorption properties of structurally deformed MOFs. The structure–property map (with dimensions such as Henry coefficient, heat of adsorption, and pore volume) was constructed using a large data set of over 12000 crystalline MOFs from molecular simulations. By mapping the experimental data points of deformed SNU-200, MOF-5, and Ni-MOF-74 onto this structure–property map, we show that the experimentally deformed MOFs share similar adsorption properties with their nearest neighbor crystalline structures. Furthermore, once the nearest neighbor crystalline MOFs for a deformed MOF are selected from a structure–property map at a specific condition, then the adsorption properties of these MOFs can be successfully transformed onto the degraded MOFs, leading to a new way to obtain properties of materials whose structural information is lost.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A View from ORNL: Scientific Data Research Opportunities in the Big Data Age

One of the core issues across computer and computational science today is adapting to, managing, and learning from the influx of "Big Data". In the commercial space, this problem has led to a huge investment in new technologies and capabilities that are well adapted to dealing with the sorts of human-generated logs, videos, texts, and other large-data artifacts that are processed and resulted in an explosion of useful platforms and languages (Hadoop, Spark, Pandas, etc.). However, translating this work from the enterprise space to the computational science and HPC community has proven somewhat difficult, in part because of some of the fundamental differences in type and scale of data and timescales surrounding its generation and use. We describe a forward-looking research and development plan which centers around the concept of making Input/Output (I/O) intelligent for users in the scientific community, whether they are accessing scalable storage or performing in situ workflow tasks. Much of our work is based on our experience with the Adaptable I/O System (ADIOS 1.X), and our next generation version of the software ADIOS 2.X [1].

Klasky, Scott↗