Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58

New parton distributions from large-x and low-Q 2 data

We report results of a new global next-to-leading order fit of parton distribution functions in which cuts on W and Q are relaxed, thereby including more data at high values of x. Effects of target mass corrections (TMCs), higher twist contributions, and nuclear corrections for deuterium data are significant in the large-x region. The leading twist parton distributions are found to be stable to TMC model variations as long as higher twist contributions are also included. Furthermore, the behavior of the d quark as x → 1 is particularly sensitive to the deuterium corrections, and using realistic nuclear smearing models the d-quark distribution at large x is found to be softer than in previous fits performed with more restrictive cuts.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Scalable and accurate multi-GPU-based image reconstruction of large-scale ptychography data

Abstract While the advances in synchrotron light sources, together with the development of focusing optics and detectors, allow nanoscale ptychographic imaging of materials and biological specimens, the corresponding experiments can yield terabyte-scale volumes of data that can impose a heavy burden on the computing platform. Although graphics processing units (GPUs) provide high performance for such large-scale ptychography datasets, a single GPU is typically insufficient for analysis and reconstruction. Several works have considered leveraging multiple GPUs to accelerate the ptychographic reconstruction. However, most of these works utilize only the Message Passing Interface to handle the communications between GPUs. This approach poses inefficiency for a hardware configuration that has multiple GPUs in a single node, especially while reconstructing a single large projection, since it provides no optimizations to handle the heterogeneous GPU interconnections containing both low-speed (e.g., PCIe) and high-speed links (e.g., NVLink). In this paper, we provide an optimized intranode multi-GPU implementation that can efficiently solve large-scale ptychographic reconstruction problems. We focus on the maximum likelihood reconstruction problem using a conjugate gradient (CG) method for the solution and propose a novel hybrid parallelization model to address the performance bottlenecks in the CG solver. Accordingly, we have developed a tool, called PtyGer ( Pty chographic G PU(multipl e )-based r econstruction), implementing our hybrid parallelization model design. A comprehensive evaluation verifies that PtyGer can fully preserve the original algorithm’s accuracy while achieving outstanding intranode GPU scalability.

97 MATHEMATICS AND COMPUTING↗

Evaluation of a multi-atlas based method for segmentation of cardiac CTA data: a large-scale, multicenter, and multivendor study

Purpose: Computed tomography angiography (CTA) is increasingly used for the diagnosis of coronary artery disease (CAD). However, CTA is not commonly used for the assessment of ventricular and atrial function, although functional information extracted from CTA data is expected to improve the diagnostic value of the examination. In clinical practice, the extraction of ventricular and atrial functional information, such as stroke volume and ejection fraction, requires accurate delineation of cardiac chambers. In this paper, we investigated the accuracy and robustness of cardiac chamber delineation using a multiatlas based segmentation method on multicenter and multivendor CTA data. Methods: A fully automatic multiatlas based method for segmenting the whole heart (i.e., the outer surface of the pericardium) and cardiac chambers from CTA data is presented and evaluated. In the segmentation approach, eight atlas images are registered to a new patient's CTA scan. The eight corresponding manually labeled images are then propagated and combined using a per voxel majority voting procedure, to obtain a cardiac segmentation. Results: The method was evaluated on a multicenter/multivendor database, consisting of (1) a set of 1380 Siemens scans from 795 patients and (2) a set of 60 multivendor scans (Siemens, Philips, and GE) from different patients, acquired in six different institutions worldwide. A leave-one-out 3D quantitative validation was carried out on the eight atlas images; we obtained a mean surface-to-surface error of 0.94{+-}1.12 mm and an average Dice coefficient of 0.93 was achieved. A 2D quantitative evaluation was performed on the 60 multivendor data sets. Here, we observed a mean surface-to-surface error of 1.26{+-}1.25 mm and an average Dice coefficient of 0.91 was achieved. In addition to this quantitative evaluation, a large-scale 2D and 3D qualitative evaluation was performed on 1380 and 140 images, respectively. Experts evaluated that 49% of the 1380 images were very accurately segmented (below 1 mm error) and that 29% were accurately segmented (error between 1 and 3 mm), which demonstrates the robustness of the presented method. Conclusions: A fully automatic method for whole heart and cardiac chamber segmentation was presented and evaluated using multicenter/multivendor CTA data. The accuracy and robustness of the method were demonstrated by successfully applying the method to 1420 multicenter/multivendor data sets.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems, In: SC '14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

The Oak Ridge Leadership Computing Facility (OLCF) has deployed multiple large-scale parallel file systems (PFS) to support its operations. During this process, OLCF acquired significant expertise in large-scale storage system design, file system software development, technology evaluation, benchmarking, procurement, deployment, and operational practices. Based on the lessons learned from each new PFS deployment, OLCF improved its operating procedures, and strategies. This paper provides an account of our experience and lessons learned in acquiring, deploying, and operating large-scale parallel file systems. We believe that these lessons will be useful to the wider HPC community.

Computer Science↗

Using cross-layer adaptations for dynamic data management in large scale coupled scientific workflows, In: SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

As system scales and application complexity grow, managing and processing simulation data has become a significant challenge. While recent approaches based on data staging and in-situ/in-transit data processing are promising, dynamic data volumes and distributions, such as those occurring in AMR-based simulations, make the efficient use of these techniques challenging. In this paper we propose cross-layer adaptations that address these challenges and respond at runtime to dynamic data management requirements. Specifically we explore (1) adaptations of the spatial resolution at which the data is processed, (2) dynamic placement and scheduling of data processing kernels, and (3) dynamic allocation of in-transit resources. We also exploit coordinated approaches that dynamically combine these adaptations at the different layers. We evaluate the performance of our adaptive cross-layer management approach on the Intrepid IBM-BlueGene/P and Titan Cray-XK7 systems using Chombo-based AMR applications, and demonstrate its effectiveness in improving overall time-to-solution and increasing resource efficiency.

Computer Science↗

Deep Learning-based Parameterization of Complex 3D CO2 Saturation Data in Large-scale Geological Carbon Storage

In deep learning (DL), dimension reduction plays a pivotal role in improving training efficiency and minimizing overfitting, especially when working with complex datasets like three-dimensional (3D) saturation data. In the context of geological carbon storage (GCS), 3D saturation data introduces unique challenges due to its sparse nature and sharp transitions at plume boundaries, known as shock fronts. To tackle these challenges, we developed a novel DL framework that combines dimension reduction with advanced 3D reconstruction techniques. Our approach utilizes latent variables derived from 2D average saturation fields to efficiently capture the essential features of high-dimensional data while reducing the number of variables. This enhances both the robustness and accuracy of DL models, making the framework more practical for real-world applications. By offering a tailored solution for modeling complex 3D saturation dynamics, this framework holds significant potential for environmental monitoring, energy storage, and other geological applications.

Wang, Hongsheng [University of Texas at Austin]↗

Two Weeks In The Life of Skybridge

This report documents a new large public data set for researchers studying the behavior of large commodity high-performance computing systems. Such large data sets are typically confined within institutions and access to them is limited to institutional partners. We provide it to promote HPC research more widely. The data set provides a two week time series of performance data collected once per minute using the Lightweight Distributed Metric Service from the system Skybridge at Sandia National Laboratories and the corresponding job-level accounting information. General system log information is not provided.

97 MATHEMATICS AND COMPUTING↗

Compactly‐Supported Nonstationary Kernels for Computing Exact Gaussian Processes on Big Data

The Gaussian process (GP) is a widely used method for analyzing large-scale data sets, including spatio-temporal measurements of nonlinear processes that are now commonplace in the environmental sciences. Traditional implementations of GPs involve stationary kernels (also termed covariance functions) that limit their flexibility, and exact methods for inference that prevent application to data sets with more than about 10,000 points. Modern approaches to address stationarity assumptions generally fail to accommodate large data sets, while all attempts to address scalability focus on approximating the Gaussian likelihood, which can involve subjectivity and lead to inaccuracies. In this work, we explicitly derive an alternative kernel that can discover and encode both sparsity and nonstationarity. We embed the kernel within a fully Bayesian GP model and leverage high-performance computing resources to enable the analysis of massive data sets. We demonstrate the favorable performance of our novel kernel relative to existing exact and approximate GP methods across a variety of synthetic data examples. Furthermore, we conduct space–time prediction based on more than 1 million measurements of daily maximum temperature and verify that our results outperform state-of-the-art methods in the Earth sciences. More broadly, having access to exact GPs that use ultra-scalable, sparsity-discovering, nonstationary kernels allows GP methods to truly compete with a wide variety of machine learning methods.

Gaussian processes↗

Fast Spatio-Temporal Data Mining from Large Geophysical Datasets

Use of the UCLA CONQUEST (CONtent-based Querying in Space and Time) is reviewed for performance of automatic cyclone extraction and detection of spatio-temporal blocking conditions on MPP. CONQUEST is a data analysis environment for knowledge and data mining to aid in high-resolution modeling of climate modeling.

knowledge discovery data mining climate modeling c↗

Assessing the Accuracy of a Borehole-Controlled Velocity Model of Yucca Flat, Nevada, Using Large-N Seismic Data

The Source Physics Experiment (SPE) aims to improve nuclear test monitoring capabilities through a series of chemical explosions in Yucca Flat within the Nevada National Security Site (NNSS) (Snelson et al., 2013). With geologic data from over 950 boreholes, Yucca Flat has excellent borehole control on stratigraphy. These data were used to create a Geologic Framework Model (GFM) of the basin to aid in the study of seismic wave propagation through the Earth. However, the seismic velocities of the geologic units in the GFM are less well-defined. With the acquisition of five new active source Large-N datasets, we can now quantitatively assess the accuracy of the seismic model previously controlled by borehole data alone.

58 GEOSCIENCES↗

Large Scale Flutter Data for Design of Rotating Blades Using Navier-Stokes Equations

A procedure to compute flutter boundaries of rotating blades is presented; a) Navier-Stokes equations. b) Frequency domain method compatible with industry practice. Procedure is initially validated: a) Unsteady loads with flapping wing experiment. b) Flutter boundary with fixed wing experiment. Large scale flutter computation is demonstrated for rotating blade: a) Single job submission script. b) Flutter boundary in 24 hour wall clock time with 100 cores. c) Linearly scalable with number of cores. Tested with 1000 cores that produced data in 25 hrs for 10 flutter boundaries. Further wall-clock speed-up is possible by performing parallel computations within each case.

Guruswamy, Guru P.↗

Interactive Exploration and Analysis of Large-Scale Simulations Using Topology-Based Data Segmentation

Large-scale simulations are increasingly being used to study complex scientific and engineering phenomena. As a result, advanced visualization and data analysis are also becoming an integral part of the scientific process. Often, a key step in extracting insight from these large simulations involves the definition, extraction, and evaluation of features in the space and time coordinates of the solution. However, in many applications, these features involve a range of parameters and decisions that will affect the quality and direction of the analysis. Examples include particular level sets of a specific scalar field, or local inequalities between derived quantities. A critical step in the analysis is to understand how these arbitrary parameters/decisions impact the statistical properties of the features, since such a characterization will help to evaluate the conclusions of the analysis as a whole. We present a new topological framework that in a single-pass extracts and encodes entire families of possible features definitions as well as their statistical properties. For each time step we construct a hierarchical merge tree a highly compact, yet flexible feature representation. While this data structure is more than two orders of magnitude smaller than the raw simulation data it allows us to extract a set of features for any given parameter selection in a postprocessing step. Furthermore, we augment the trees with additional attributes making it possible to gather a large number of useful global, local, as well as conditional statistic that would otherwise be extremely difficult to compile. We also use this representation to create tracking graphs that describe the temporal evolution of the features over time. Our system provides a linked-view interface to explore the time-evolution of the graph interactively alongside the segmentation, thus making it possible to perform extensive data analysis in a very efficient manner. We demonstrate our framework by extracting and analyzing burning cells from a large-scale turbulent combustion simulation. In particular, we show how the statistical analysis enabled by our techniques provides new insight into the combustion process.

97 MATHEMATICS AND COMPUTING↗

Beyond Music Sharing: An Evaluation of Peer-to-Peer Data Dissemination Techniques in Large Scientific Collaborations

The avalanche of data from scientific instruments and the ensuing interest from geographically distributed users to analyze and interpret it accentuates the need for efficient data dissemination. A suitable data distribution scheme will find the delicate balance between conflicting requirements of minimizing transfer times, minimizing the impact on the network, and uniformly distributing load among participants. We identify several data distribution techniques, some successfully employed by today's peer-to-peer networks: staging, data partitioning, orthogonal bandwidth exploitation, and combinations of the above. We use simulations to explore the performance of these techniques in contexts similar to those used by today's data-centric scientific collaborations and derive several recommendations for efficient data dissemination. Our experimental results show that the peer-to-peer solutions that offer load balancing and good fault tolerance properties and have embedded participation incentives lead to unjustified costs in today's scientific data collaborations deployed on over-provisioned network cores. However, as user communities grow and these deployments scale, peer-to-peer data delivery mechanisms will likely outperform other techniques.

97 MATHEMATICS AND COMPUTING↗

Nontargeted virus sequence discovery pipeline and virus clustering for metagenomic data

The analysis of large microbiome data sets holds great promise for the delineation of the biological and metabolic functioning of living organisms and their role in the environment. In the midst of this genomic puzzle, viruses, especially those that infect microbial communities, represent a major reservoir of genetic diversity with great impact on biogeochemical cycles and organismal health. Overcoming the limitations associated with virus detection directly from microbiomes can provide key insights into how ecosystem dynamics are modulated. Here, we present a computational protocol for accurate detection and grouping of viral sequences from microbiome samples. Our approach relies on an expanded and curated set of viral protein families used as bait to identify viral sequences directly from metagenomic assemblies. This protocol describes how to use the viral protein families catalog (~7 h) and recommended filters for the detection of viral contigs in metagenomic samples (~6 h), and it describes the specific parameters for a nucleotide-sequence-identity-based method of organizing the viral sequences into quasi-species taxonomic-level groups (~10 min).

Paez-Espino, David↗

Pitch Oscillation Data and Analysis for a Large HSCT Semispan Wing

Data from wind-tunnel tests of the Rigid Semispan Model are presented. The primary focus is on data obtained from testing on the Oscillating Turntable (OTT), The OTT is capable of oscillating models in pitch at various amplitudes and frequencies about mean angles of attack. Steady and unsteady pressure data is presented and compared to data from previous tests on a load balance and on a Pitch and Plunge Apparatus (PAPA).

Scott, Robert C.↗

Parallel Rendering of Large Time-Varying Volume Data

Interactive visualization of large time-varying 3D volume datasets has been and still is a great challenge to the modem computational world. It stretches the limits of the memory capacity, the disk space, the network bandwidth and the CPU speed of a conventional computer. In this SURF project, we propose to develop a parallel volume rendering program on SGI's Prism, a cluster computer equipped with state-of-the-art graphic hardware. The proposed program combines both parallel computing and hardware rendering in order to achieve an interactive rendering rate. We use 3D texture mapping and a hardware shader to implement 3D volume rendering on each workstation. We use SGI's VisServer to enable remote rendering using Prism's graphic hardware. And last, we will integrate this new program with ParVox, a parallel distributed visualization system developed at JPL. At the end of the project, we Will demonstrate remote interactive visualization using this new hardware volume renderer on JPL's Prism System using a time-varying dataset from selected JPL applications.

Garbutt, Alexander E.↗

The use of TOMS-modified VAS data for large-scale NWP

Collocation statistics obtained by comparing VISSR Atmospheric Sounder (VAS) temperature soundings with those from nearby rawinsondes indicate good agreement. However, the VAS soundings exhibited a substantial cold bias in the middle and upper troposphere. The error makes promising the use of the Total Ozone Mapping Spectrometer (TOMS) to obtain an independent estimate of tropopause pressure for use in the retrieval program. Good agreement is found between TOMS data and tropopause pressure. A quantitative assessment for the correlation of tropopause pressure, obtained from TOMS by regression and from rawinsondes over Europe, is reported.

Baker, Wayman E.↗