Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Scalable Data Management of the Uintah Simulation Framework for Next-Generation Engineering Problems with Radiation

The need to scale next-generation industrial engineering problems to the largest computational platforms presents unique challenges. This paper focuses on data management related problems faced by the Uintah simulation framework at a production scale of 260K processes. Uintah provides a highly scalable asynchronous many-task runtime system, which in this work is used for the modeling of a 1000 megawatt electric (MWe) ultra-supercritical (USC) coal boiler. At 260K processes, we faced both parallel I/O and visualization related challenges, e.g., the default file-per-process I/O approach of Uintah did not scale on Mira. In this paper we present a simple to implement, restructuring based parallel I/O technique. We impose a restructuring step that alters the distribution of data among processes. The goal is to distribute the dataset such that each process holds a larger chunk of data, which is then written to a file independently. This approach finds a middle ground between two of the most common parallel I/O schemes–file per process I/O and shared file I/O–in terms of both the total number of generated files, and the extent of communication involved during the data aggregation phase. To address scalability issues when visualizing the simulation data, we developed a lightweight renderer using OSPRay, which allows scientists to visualize the data interactively at high quality and make production movies. Finally, this work presents a highly efficient and scalable radiation model based on the sweeping method, which significantly outperforms previous approaches in Uintah, like discrete ordinates. The integrated approach allowed the USC boiler problem to run on 260K CPU cores on Mira.

97 MATHEMATICS AND COMPUTING↗

IDC Reengineering Phase 2 Project Scope

The Comprehensive Nuclear-Test-Ban Treaty Organization (CTBTO) operates the International Data Centre (IDC) to support treaty verification activities by Member States. The IDC collects, stores, processes, and distributes data from seismic, hydroacoustic, infrasound, and radionuclide stations. A significant component of the IDC is the seismic, hydroacoustic, and infrasound (SHI) processing system. The IDC has recognized the need to reengineer the SHI processing system and has defined a three-phase reengineering effort. IDC Reengineering Phase 1 (RP1) started in 2011 with the goal to significantly modernize SHI automatic and interactive data processing software, focusing on enhancements to individual components of the system. Reengineering Phase 2 (RP2), started in 2014, addresses the task of specifying a unified architecture for all SHI software, across processing stages, to pave the way for further software development and sustainment in the future. A key objective of RP2 is an architecture sufficient to provide a basis for a cost estimate for the development or enhancement of the software components and subsystems. This report provides a brief description of the scope of the IDC Reengineering Phase 2 project. It describes the goals and objectives of reengineering, the system definition, and the technical scope of the system.This report expands upon the “Scope for IDC Reengineering Phase 2”, July 2014, ECS-DIS-WGB-43-PTS-MATERIAL-12-ADD-1, produced by the IDC.

58 GEOSCIENCES↗

Fast analysis of molecular dynamics trajectories with graphics processing units-Radial distribution function histogramming

The calculation of radial distribution functions (RDFs) from molecular dynamics trajectory data is a common and computationally expensive analysis task. The rate limiting step in the calculation of the RDF is building a histogram of the distance between atom pairs in each trajectory frame. Here we present an implementation of this histogramming scheme for multiple graphics processing units (GPUs). The algorithm features a tiling scheme to maximize the reuse of data at the fastest levels of the GPU's memory hierarchy and dynamic load balancing to allow high performance on heterogeneous configurations of GPUs. Several versions of the RDF algorithm are presented, utilizing the specific hardware features found on different generations of GPUs. We take advantage of larger shared memory and atomic memory operations available on state-of-the-art GPUs to accelerate the code significantly. The use of atomic memory operations allows the fast, limited-capacity on-chip memory to be used much more efficiently, resulting in a fivefold increase in performance compared to the version of the algorithm without atomic operations. The ultimate version of the algorithm running in parallel on four NVIDIA GeForce GTX 480 (Fermi) GPUs was found to be 92 times faster than a multithreaded implementation running on an Intel Xeon 5550 CPU. On this multi-GPU hardware, the RDF between two selections of 1,000,000 atoms each can be calculated in 26.9 s per frame. The multi-GPU RDF algorithms described here are implemented in VMD, a widely used and freely available software package for molecular dynamics visualization and analysis.

97 MATHEMATICS AND COMPUTING↗

Extracting Resilience Metrics From Distribution Utility Data Using Outage and Restore Process Statistics

Resilience curves track the accumulation and restoration of outages during an event on an electric distribution grid. We show that a resilience curve generated from utility data can always be decomposed into an outage process and a restore process and that these processes generally overlap in time. We use many events in real utility data to characterize the statistics of these processes, and derive formulas based on these statistics for resilience metrics such as restore duration, customer hours not served, and outage and restore rates. The formulas express the mean value of these metrics as a function of the number of outages in the event. We also give a formula for the variability of restore duration, which allows us to predict a maximum restore duration with 95% confidence. Overall, we give a simple and general way to decompose resilience curves into outage and restore processes and then show how to use these processes to extract resilience metrics from standard distribution system data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Methods for Processing ENDF/B-VII with NJOY

The NJOY Nuclear Data Processing System is widely used to convert evaluations in the Evaluated Nuclear Data Files (ENDF) format into forms useful for practical applications such as fission and fusion reactor analysis, stockpile stewardship calculations, criticality safety, radiation shielding, nuclear waste management, nuclear medicine procedures, and more. This paper provides a description of the system's capabilities, summary descriptions of the methods used, and information on how to use the code to process the modern evaluated nuclear data files from ENDF/B-VII. It begins with the generation of pointwise libraries, including reaction and resonance reconstruction, Doppler broadening, radiation heating and damage, thermal scattering data, unresolved resonance data, and gas production. It then reviews the production of libraries for the continuous-energy Monte Carlo code MCNP, multigroup neutron, photon, and particle cross sections and matrices, and photon interaction data. The generation of uncertainty information for ENDF data is discussed, including new capabilities for calculating covariances of resonance data, angular distributions, energy distributions, and radioactive nuclide production. NJOY's ability to prepare thermal scattering data evaluations for bound moderators (which was used during the preparation of the ENDF/B-VII library) is described. The strong plotting capabilities of NJOY are summarized. Many examples of black and white and color Postscript plots are included throughout the paper. The capabilities of NJOY to output multigroup data in several different formats to suit various applications is reviewed. Finally, a section is included that summarizes the history of the development of the NJOY system over the last 37 years.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Near Real-Time Two and Three-Dimensional Electrical Geophysical Monitoring of Natural and Engineered Processes Associated with Subsurface Remediation - 16052

Legacy soil and groundwater contamination associated with nuclear weapons production is projected to be one of the largest cleanup liabilities to the U.S. Department of Energy for the foreseeable future. Remediation of subsurface contaminants is complicated by the combined effects of geochemical heterogeneity, geological heterogeneity, and the difficulty and expense of adequate borehole access. Each of these complications contributes to uncertainty in the distribution of contaminants and the performance of in situ treatments, which increases risk to human health and the environment and cost to closure in general. Electrical resistivity tomography (ERT) is a method of remotely imaging the electrical properties of the subsurface, which are governed by both geochemical and geological structure, thereby providing useful proxies for understanding contaminant distribution and the behavior in situ remediation processes. Recent advancements in both data collection and data processing capabilities are enabling ERT monitoring to be executed and provided to site operators in near real-time. To summarize the process, ERT surveys are rapidly and continuously collected during a time-sensitive operation such as an amendment injection. At the completion of each survey, data are transferred by wireless internet to offsite supercomputing resources for parallel tomographic inversion. Inversion results are then transferred back to onsite operators and/or other locations for visualization. In this paper, we demonstrate the utility of time-lapse ERT for monitoring fluid transport (both gas and liquid phase). We then demonstrate two real-time imaging applications for monitoring both natural and engineered processes at the Hanford Site within the vadose and the saturated zones. We expect this new capability for autonomous real-time imaging to enable ERT to be used as a cost-effective, short-term rapid feedback mechanism for guiding subsurface remediation injections, in addition to longer term monitoring of post-injection performance and environmental impact, all using the same electrode array and instrumentation. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

The US-CMS Tier1 center network evolving toward 100Gbps

Fermilab hosts the US Tier-1 Center for the LHC's Compact Muon Collider (CMS) experiment. The Tier-1s are the central points for the processing and movement of LHC data. They sink raw data from the Tier-0 at CERN, process and store it locally, and then distribute the processed data to Tier-2s for simulation studies and analysis. The Fermilab Tier-1 Center is the largest of the CMS Tier-1s, accounting for roughly 35% of the experiment's Tier-1 computing and storage capacity. Providing capacious, resilient network services, both in terms of local network infrastructure and off-site data movement capabilities, presents significant challenges. This article will describe the current architecture, status, and near term plans for network support of the US-CMS Tier-1 facility.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

EJFAT Scientific Perspective

Presented new computing model to the test by deploying the EJFAT system alongside a data-stream processing framework running the production-level CLAS12 event reconstruction application. In this experiment, a continuous stream of CLAS12 Level-1 identified events was processed in real-time using the EJFAT load balancer, distributing the workload across 90 computing nodes located across the U.S. This marks the first-ever large-scale, real-time distributed data stream processing experiment, demonstrating that scientific data-streaming pipelines can efficiently scale across four dimensions, thanks to EJFAT’s advanced hardware and software capabilities.

Gyurjyan, Vardan [Thomas Jefferson National Accele↗

A versatile scalable PET processing system

Positron Emission Tomography (PET) historically has major clinical and preclinical applications in cancerous oncology, neurology, and cardiovascular diseases. Recently, in a new direction, an application specific PET system is being developed at Thomas Jefferson National Accelerator Facility (Jefferson Lab) in collaboration with Duke University, University of Maryland at Baltimore (UMAB), and West Virginia University (WVU) targeted for plant eco-physiology research. The new plant imaging PET system is versatile and scalable such that it could adapt to several plant imaging needs - imaging many important plant organs including leaves, roots, and stems. The mechanical arrangement of the detectors is designed to accommodate the unpredictable and random distribution in space of the plant organs without requiring the plant be disturbed. Prototyping such a system requires a new data acquisition system (DAQ) and data processing system which are adaptable to the requirements of these unique and versatile detectors.

43 PARTICLE ACCELERATORS↗

Direct measurement of the Kirkwood-Rihaczek distribution for the spatial properties of a coherent light beam

We present a direct measurement of the Kirkwood-Rihaczek (KR) distribution for spatial properties of a coherent light beam in terms of position and momentum (angle) coordinates. We employ a two-local oscillator (LO) balanced heterodyne detection (BHD) to simultaneously extract the distribution of the transverse position and momentum of a light beam. The two-LO BHD can measure the KR distribution for any complex wave field (including quantum mechanical wave function) without applying tomography methods (inverse Radon transformation). The transformation of the KR distribution to the Wigner, Glauber-Sudarshan P, and Husimi or Q distributions in spatial coordinates are illustrated through experimental data. The KR distribution can provide the local information of a wave field, which is suitable for studying particle properties of a quantum system. Meanwhile, the Wigner function is suitable for studying wave properties such as interference, and hence provides nonlocal information of the wave field. The method developed here can be used for exploring the spatial quantum state for information processing and optical phase-space imaging for biomedical applications.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Dynamic Distribution of High-Rate Data Processing from CERN to Remote HPC Data Centers

The prompt reconstruction of the data recorded from the Large Hadron Collider (LHC) detectors has always been addressed by dedicated resources at the CERN Tier-0. Such workloads come in spikes due to the nature of the operation of the accelerator and in special high load occasions experiments have commissioned methods to distribute (spill-over) a fraction of the load to sites outside CERN. The present work demonstrates a new way of supporting the Tier-0 environment by provisioning resources elastically for such spilled-over workflows onto the Piz Daint Supercomputer at CSCS. Furthermore, this is implemented using containers, tuning the existing batch scheduler and reinforcing the scratch file system, while still using standard Grid middleware. ATLAS, CMS and CSCS have jointly run selected prompt data reconstruction on up to several thousand cores on Piz Daint into a shared environment, thereby probing the viability of the CSCS high performance computer site as on demand extension of the CERN Tier-0, which could play a role in addressing the future LHC computing challenges for the high luminosity LHC.

97 MATHEMATICS AND COMPUTING↗

Using spatial statistical methods to determine an effective minimum detectable activity - 15681

Instrument minimum detectable activity (MDA) is generally established by collecting multiple measurements over time of a single representative background location to estimate a probability distribution of instrument readings. A reading that is considered to be greater than would be expected from the background probability distribution (i.e., a low-probability event) is deemed to be sufficiently unlikely that it must be attributable to added radioactivity and is therefore used as the instrument MDA. An effective MDA can be determined from spatially distributed data that are much lower than the instrument MDA. This has been accomplished using a large number of spatially distributed data points (from a radiation survey) where patterns of elevated survey readings can be distinguished from background at levels. Given the variation of the instrument response and the relatively low regulatory action levels, it is vital to establish MDA thresholds that are well below corrective action levels. Processing of radiological survey data with the use of histogram binning and geo-statistics in enabling pattern recognition can be used to establish effective field MDAs that are much lower than instrument MDAs. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A fast and memory-sparing probabilistic selection algorithm for the GPU

A fast and memory-sparing probabilistic top-N selection algorithm is implemented on the GPU. This probabilistic algorithm gives a deterministic result and always terminates. The use of randomization reduces the amount of data that needs heavy processing, and so reduces both the memory requirements and the average time required for the algorithm. This algorithm is well-suited to more general parallel processors with multiple layers of memory hierarchy. Probabilistic Las Vegas algorithms of this kind are a form of stochastic optimization and can be especially useful for processors having a limited amount of fast memory available.

99 GENERAL AND MISCELLANEOUS↗

Fourth moments reveal the negativity of the Wigner function

The presence of unique quantum correlations is the core of quantum-information processing and general quantum theory. We address the fundamental question of how quantum correlations of a generic quantum system can be probed using correlation functions defined for quasiprobability distributions. In particular, we discuss the possibility of probing the negativity of a quasiprobability by comparing moments of the Wigner function. We show that one must take at least the fourth moments to find the negativity in general and the eighth moments for states with a rotationally invariant Wigner function.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Support Vector Machine-Based Endmember Extraction

Introduced in this paper is the utilization of Support Vector Machines (SVMs) to automatically perform endmember extraction from hyperspectral data. The strengths of SVM are exploited to provide a fast and accurate calculated representation of high-dimensional data sets that may consist of multiple distributions. Once this representation is computed, the number of distributions can be determined without prior knowledge. For each distribution, an optimal transform can be determined that preserves informational content while reducing the data dimensionality, and hence, the computational cost. Finally, endmember extraction for the whole data set is accomplished. Results indicate that this Support Vector Machine-Based Endmember Extraction (SVM-BEE) algorithm has the capability of autonomously determining endmembers from multiple clusters with computational speed and accuracy, while maintaining a robust tolerance to noise.

97 MATHEMATICS AND COMPUTING↗

Large-Scale Multi-Dimensional Document Clustering on GPU Clusters

Document clustering plays an important role in data mining systems. Recently, a flocking-based document clustering algorithm has been proposed to solve the problem through simulation resembling the flocking behavior of birds in nature. This method is superior to other clustering algorithms, including k-means, in the sense that the outcome is not sensitive to the initial state. One limitation of this approach is that the algorithmic complexity is inherently quadratic in the number of documents. As a result, execution time becomes a bottleneck with large number of documents. In this paper, we assess the benefits of exploiting the computational power of Beowulf-like clusters equipped with contemporary Graphics Processing Units (GPUs) as a means to significantly reduce the runtime of flocking-based document clustering. Our framework scales up to over one million documents processed simultaneously in a sixteennode GPU cluster. Results are also compared to a four-node cluster with higher-end GPUs. On these clusters, we observe 30X-50X speedups, which demonstrates the potential of GPU clusters to efficiently solve massive data mining problems. Such speedups combined with the scalability potential and accelerator-based parallelization are unique in the domain of document-based data mining, to the best of our knowledge.

99 GENERAL AND MISCELLANEOUS↗