Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,459 records · Page 81

Chemical Blast Standard (1 kg)

Chemical explosions create blast waves with large overpressure disturbances. It is important to develop a standard blast model based on data to accurately predict acoustic blast-wave amplitudes near detonations and invert for explosion energy from distant observations of blast-wave signals. However, open data from large, controlled chemical explosions with reliable ground truth can be challenging to find. The lack of access to such data could limit the number of contributions to related research and potentially stifle the rate of discoveries or validation of existing models. Here, to address these data scarcity problem, we have curated and compiled a standardized set of 817 blast-wave waveforms from 19 distinct high-explosive events. The blast-wave waveforms are standardized to a 1 kg trinitrotoluene explosion using scaling laws and corrections for location effects. A brief overview of the dataset is presented along with explosion feature models as well as recommendations for extracting explosion features. The resulting dataset is distributed to an open repository in both Seismic Analysis Code and pandas DataFrame formats containing the waveforms, the scaled distances, and the sample rates.

58 GEOSCIENCES↗

Markov state models from short non-equilibrium simulations—Analysis and correction of estimation bias

To begin, many state-of-the-art methods for the thermodynamic and kinetic characterization of large and complex biomolecular systems by simulation rely on ensemble approaches, where data from large numbers of relatively short trajectories are integrated. In this context, Markov state models (MSMs) are extremely popular because they can be used to compute stationary quantities and long-time kinetics from ensembles of short simulations, provided that these short simulations are in “local equilibrium” within the MSM states. However, over the last 15 years since the inception of MSMs, it has been controversially discussed and not yet been answered how deviations from local equilibrium can be detected, whether these deviations induce a practical bias in MSM estimation, and how to correct for them. In this paper, we address these issues: We systematically analyze the estimation of MSMs from short non-equilibrium simulations, and we provide an expression for the error between unbiased transition probabilities and the expected estimate from many short simulations. We show that the unbiased MSM estimate can be obtained even from relatively short non-equilibrium simulations in the limit of long lag times and good discretization. Further, we exploit observable operator model (OOM) theory to derive an unbiased estimator for the MSM transition matrix that corrects for the effect of starting out of equilibrium, even when short lag times are used. Finally, we show how the OOM framework can be used to estimate the exact eigenvalues or relaxation time scales of the system without estimating an MSM transition matrix, which allows us to practically assess the discretization quality of the MSM. Applications to model systems and molecular dynamics simulation data of alanine dipeptide are included for illustration. The improved MSM estimator is implemented in PyEMMA of version 2.3.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Streaming data analytics via message passing with application to graph algorithms

The need to process streaming data, which arrives continuously at high-volume in real-time, arises in a variety of contexts including data produced by experiments, collections of environmental or network sensors, and running simulations. Streaming data can also be formulated as queries or transactions which operate on a large dynamic data store, e.g. a distributed database. We describe a lightweight, portable framework named PHISH which enables a set of independent processes to compute on a stream of data in a distributed-memory parallel manner. Datums are routed between processes in patterns defined by the application. PHISH can run on top of either message-passing via MPI or sockets via ZMQ. The former means streaming computations can be run on any parallel machine which supports MPI; the latter allows them to run on a heterogeneous, geographically dispersed network of machines. We illustrate how PHISH can support streaming MapReduce operations, and describe streaming versions of three algorithms for large, sparse graph analytics: triangle enumeration, subgraph isomorphism matching, and connected component finding. Lastly, we also provide benchmark timings for MPI versus socket performance of several kernel operations useful in streaming algorithms.

97 MATHEMATICS AND COMPUTING↗

An energy-limited model of algal biofuel production: Toward the next generation of advanced biofuels

Algal biofuels are increasingly important as a source of renewable energy. The absence of reliable thermodynamic and other property data, and the large amount of kinetic data that would normally be required have created a major barrier to simulation. Additionally, the absence of a generally accepted flowsheet for biofuel production means that detailed simulation of the wrong approach is a real possibility. This model of algal biofuel production estimates the necessary data and places it into a heuristic model using a commercial simulator that back-calculates the process structure required. Furthermore, complex kinetics can be obviated for now by putting the simulator into energy limitation and forcing it to solve for the missing design variables, such as bioreactor surface area, productivity, and oil content. The model does not attempt to prescribe a particular approach, but provides a guide towards a sound engineering approach to this challenging and important problem.

09 BIOMASS FUELS↗

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Summary There are many data sources that produce large volumes of data. The Big Data nature requires new distributed processing approaches to extract the valuable information. Real‐time sentiment analysis is one of the most demanding research areas that requires powerful Big Data analytics tools such as Spark. Prior literature survey work has shown that, though there are many conventional sentiment analysis researches, there are only few works realizing sentiment analysis in real time. One major point that affects the quality of real‐time sentiment analysis is the confidence of the generated data. In more clear terms, it is a valuable research question to determine whether the owner that generates sentiment is genuine or not. Since data generated by fake personalities may decrease accuracy of the outcome, a smart/intelligent service that can identify the source of data is one of the key points in the analysis. In this context, we include a fake account detection service to the proposed framework. Both sentiment analysis and fake account detection systems are trained and tested using Naïve Bayes model from Apache Spark's machine learning library. The developed system consists of four integrated software components, ie, (i) machine learning and streaming service for sentiment prediction, (ii) a Twitter streaming service to retrieve tweets, (iii) a Twitter fake account detection service to assess the owner of the retrieved tweet, and (iv) a real‐time reporting and dashboard component to visualize the results of sentiment analysis. The sentiment classification performances of the system for offline and real‐time modes are 86.77% and 80.93%, respectively.

Kılınç, Deniz↗

Finding Hidden Patterns in High Resolution Wind Flow Model Simulations

Wind flow data is critical in terms of investment decisions and policy making. High resolution data from wind flow model simulations serve as a supplement to the limited resource of original wind flow data collection. Given the large size of data, finding hidden patterns in wind flow model simulations are critical for reducing the dimensionality of the analysis. In this work, we first perform dimension reduction with two autoencoder models: the CNN-based autoencoder (CNN-AE) [1], and hierarchical autoencoder (HIER-AE) [2], and compare their performance with the Principal Component Analysis (PCA). We then investigate the super-resolution of the wind flow data. By training a Generative Adversarial Network (GAN) with 300 epochs, we obtained a trained model with 2× resolution enhancement. We compare the results of GAN with Convolutional Neural Network (CNN), and GAN results show finer structure as expected in the data field images. Also, the kinetic energy spectra comparisons show that GAN outperforms CNN in terms of reproducing the physical properties for high wavenumbers and is critical for analysis where high-wavenumber kinetics play an important role.

97 MATHEMATICS AND COMPUTING↗

Parallel streaming between heterogeneous HPC resources for real-time analysis

Performing analysis or generating visualizations concurrently with high performance simulations can yield great benefits compared to post-processing data. Writing and reading large volumes of data can be reduced or eliminated, thereby producing an I/O cost savings. One such method for concurrent simulation and analysis is in transit - streaming data from the resource running the simulation to a separate resource running the analysis. In transit analysis can be beneficial since computational resources may not have certain resources needed for visualization and analysis (e.g. GPUs) and to reduce the impact of performing analysis tasks to the run time of the simulation. When sending and receiving data in transit, data redistribution mechanisms are needed in order to support heterogeneous data layouts that may be required by the simulation and analysis applications. The work described in this paper compares two mechanisms for on-the-fly data redistribution when streaming data in parallel between two distributed memory applications. Finally, our results show that it is often more advantageous to stream data in the same layout as the sender and redistribute data amongst processes on the receiving end than to stream data in the final layout needed by the receiver.

97 MATHEMATICS AND COMPUTING↗

Data Set Analysis to Reduce Uncertainty in Formula Assignments of Ultrahigh Resolution Mass Spectra

Environmental samples contain a vast array of organic compounds with diverse elemental compositions and heteroatom content. Molecular formula assignments of ultrahigh resolution mass spectra (HRMS) hold promise for elucidating the molecular composition of these compounds. However, the need to account for an assortment of heteroatoms increases the uncertainty associated with individual assignments – and ultimately the ecological, biological, and biogeochemical insights gleaned from the assignments. To address this challenge, we introduce a formula assignment strategy that leverages HRMS data sets to improve assignment confidence, filter false assignments, and mitigate bias in assignment routines. The strategy, implemented using CoreMS, first identifies the highest confidence assignment for a recurring ion in a data set by assessing the mass accuracy and isotopologue similarity of all assignments to the ion across the data set. The second component of the strategy examines the consistency of mass errors for an assigned ion throughout a data set and flags formulas with statistically unlikely deviations in mass error. Here, we illustrate the application and utility of the strategy by comparing its results against documented misassignment patterns within a set of oceanographic samples that were measured with 21 T Fourier Transform Ion Cyclotron Resonance Mass Spectrometry. Because the efficacy of our strategy improves with data set size, it is particularly useful for enhancing assignment confidence in large HRMS data sets common in studies of environmental systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Directly imaging the cooling flow in the Phoenix cluster

In the centres of many galaxy clusters, the hot (approximately 10 7 kelvin) intracluster medium can become dense enough that it should cool on short timescales. However, the low measured star formation rates in massive central galaxies and the absence of soft X-ray lines from the cooling gas suggest that most of this gas never cools. This is known as the cooling flow problem. The latest observations suggest that black hole jets are maintaining the vast majority of gas at high temperatures. A cooling flow has yet to be fully mapped through all the gas phases in any galaxy cluster. Here we present observations of the Phoenix cluster using the James Webb Space Telescope to map the [Ne vi] λ 7.652-μm emission line, enabling us to probe the gas at 10 5.5 kelvin on large scales. These data show extended [Ne vi] emission that is cospatial with the cooling peak in the intracluster medium, the coolest gas phases and the sites of active star formation. Taken together, these imply a recent episode of rapid cooling, causing a short-lived spike in the cooling rate, which we estimate to be 5,000–23,000 solar masses per year. As a result, these data provide a large-scale map of gas at temperatures between 10 5 kelvin and 10 6 kelvin in a cluster core, and highlight the critical role that black hole feedback has in not only regulating cooling but also promoting it.

79 ASTRONOMY AND ASTROPHYSICS↗

Challenges in Developing Clinical Workstation

Over the years, medical imaging has become very common and data intensive. New technology is needed to help visualize and analyze these large, complex data sets, especially in an acute care situation where time is of the essence. Also it is very important to present the data in an efficient and simple manner to aid the clinical decision making processes. There is a need for a clinical workstation that handles data from different modalities and performs the necessary post- processing operations on the data in order to enhance the image quality and improve the reliability of diagnosis. This paper briefly explains clinical workstation, emphasizing the requirements and challenges in design and architecture for the development of such systems.

99 GENERAL AND MISCELLANEOUS↗

STRUCTURE AND DYNAMICS OF THE 2009 JULY 22 ECLIPSE WHITE-LIGHT CORONA

The white-light corona (WLC) during the total solar eclipse of 2009 July 22 was observed by several teams in the Moon's shadow stretching from India and China across the Pacific Ocean with its many isolated islands. We present a comparison of the WLC as observed by eclipse teams located in China (Shanghai region) and on the Enewetak Atoll in the Marshall Islands, with observations taken 112 minutes apart, combined with near-simultaneous space observations. The eclipse was observed at the beginning of solar cycle 24, during a deep solar minimum (officially estimated as 2008 December according to the smoothed sunspot number, but very extended). The solar corona shows several different types of features (coronal holes, polar rays, helmet streamers, faint loops, voids, etc.), though it was extremely sparse in streamers as shown from Large-Angle Spectroscopic Coronagraph data. No large-scale dynamical phenomena were seen when comparing the observations from the two sites, confirming that the corona was quiescent. We measure a Ludendorff flattening coefficient of 0.238, typical of solar minimum.

79 ASTRONOMY AND ASTROPHYSICS↗

Data handling with SAM and art at the NO v A experiment

During operations, NOvA produces between 5,000 and 7,000 raw files per day with peaks in excess of 12,000. These files must be processed in several stages to produce fully calibrated and reconstructed analysis files. In addition, many simulated neutrino interactions must be produced and processed through the same stages as data. To accommodate the large volume of data and Monte Carlo, production must be possible both on the Fermilab grid and on off-site farms, such as the ones accessible through the Open Science Grid. To handle the challenge of cataloging these files and to facilitate their off-line processing, we have adopted the SAM system developed at Fermilab. SAM indexes files according to metadata, keeps track of each file's physical locations, provides dataset management facilities, and facilitates data transfer to off-site grids. To integrate SAM with Fermilab's art software framework and the NOvA production workflow, we have developed methods to embed metadata into our configuration files, art files, and standalone ROOT files. A module in the art framework propagates the embedded information from configuration files into art files, and from input art files to output art files, allowing us to maintain a complete processing history within our files. Embedding metadata in configuration files also allows configuration files indexed in SAM to be used as inputs to Monte Carlo production jobs. Further, SAM keeps track of the input files used to create each output file. Parentage information enables the construction of self-draining datasets which have become the primary production paradigm used at NOvA. In this study we will present an overview of SAM at NOvA and how it has transformed the file production framework used by the experiment.

97 MATHEMATICS AND COMPUTING↗

Use of computational modeling combined with advanced visualization to develop strategies for the design of crop ideotypes to address food security

Sustainable crop production is a contributing factor to current and future food security. Innovative technologies are needed to design strategies that will achieve higher crop yields on less land and with fewer resources. Computational modeling coupled with advanced scientific visualization enables researchers to explore and interact with complex agriculture, nutrition, and climate data to predict how crops will respond to untested environments. These virtual observations and predictions can direct the development of crop ideotypes designed to meet future yield and nutritional demands. This review surveys modeling strategies for the development of crop ideotypes and scientific visualization technologies that have led to discoveries in “big data” analysis. Combined modeling and visualization approaches have been used to realistically simulate crops and to guide selection that immediately enhances crop quantity and quality under challenging environmental conditions. Lastly, this survey of current and developing technologies indicates that integrative modeling and advanced scientific visualization may help overcome challenges in agriculture and nutrition data as large-scale and multidimensional data become available in these fields.

3D crop modeling↗

Network bandwidth utilization forecast model on high bandwidth networks

With the increasing number of geographically distributed scientific collaborations and the scale of the data size growth, it has become more challenging for users to achieve the best possible network performance on a shared network. We have developed a forecast model to predict expected bandwidth utilization for high-bandwidth wide area network. The forecast model can improve the efficiency of resource utilization and scheduling data movements on high-bandwidth network to accommodate ever increasing data volume for large-scale scientific data applications. Univariate model is developed with STL and ARIMA on SNMP path utilization data. Compared with traditional approach such as Box-Jenkins methodology, our forecast model reduces computation time by 83.2%. It also shows resilience against abrupt network usage change. The accuracy of the forecast model is within the standard deviation of the monitored measurements.

97 MATHEMATICS AND COMPUTING↗

A nonvoxel-based dose convolution/superposition algorithm optimized for scalable GPU architectures

Purpose: Real-time adaptive planning and treatment has been infeasible due in part to its high computational complexity. There have been many recent efforts to utilize graphics processing units (GPUs) to accelerate the computational performance and dose accuracy in radiation therapy. Data structure and memory access patterns are the key GPU factors that determine the computational performance and accuracy. In this paper, the authors present a nonvoxel-based (NVB) approach to maximize computational and memory access efficiency and throughput on the GPU. Methods: The proposed algorithm employs a ray-tracing mechanism to restructure the 3D data sets computed from the CT anatomy into a nonvoxel-based framework. In a process that takes only a few milliseconds of computing time, the algorithm restructured the data sets by ray-tracing through precalculated CT volumes to realign the coordinate system along the convolution direction, as defined by zenithal and azimuthal angles. During the ray-tracing step, the data were resampled according to radial sampling and parallel ray-spacing parameters making the algorithm independent of the original CT resolution. The nonvoxel-based algorithm presented in this paper also demonstrated a trade-off in computational performance and dose accuracy for different coordinate system configurations. In order to find the best balance between the computed speedup and the accuracy, the authors employed an exhaustive parameter search on all sampling parameters that defined the coordinate system configuration: zenithal, azimuthal, and radial sampling of the convolution algorithm, as well as the parallel ray spacing during ray tracing. The angular sampling parameters were varied between 4 and 48 discrete angles, while both radial sampling and parallel ray spacing were varied from 0.5 to 10 mm. The gamma distribution analysis method (γ) was used to compare the dose distributions using 2% and 2 mm dose difference and distance-to-agreement criteria, respectively. Accuracy was investigated using three distinct phantoms with varied geometries and heterogeneities and on a series of 14 segmented lung CT data sets. Performance gains were calculated using three 256 mm cube homogenous water phantoms, with isotropic voxel dimensions of 1, 2, and 4 mm. Results: The nonvoxel-based GPU algorithm was independent of the data size and provided significant computational gains over the CPU algorithm for large CT data sizes. The parameter search analysis also showed that the ray combination of 8 zenithal and 8 azimuthal angles along with 1 mm radial sampling and 2 mm parallel ray spacing maintained dose accuracy with greater than 99% of voxels passing the γ test. Combining the acceleration obtained from GPU parallelization with the sampling optimization, the authors achieved a total performance improvement factor of >175 000 when compared to our voxel-based ground truth CPU benchmark and a factor of 20 compared with a voxel-based GPU dose convolution method. Conclusions: The nonvoxel-based convolution method yielded substantial performance improvements over a generic GPU implementation, while maintaining accuracy as compared to a CPU computed ground truth dose distribution. Such an algorithm can be a key contribution toward developing tools for adaptive radiation therapy systems.

60 APPLIED LIFE SCIENCES↗

Parton showers and matching uncertainties in top quark pair production with Herwig 7

Abstract We evaluate the theoretical uncertainties in next-to-leading order plus parton shower predictions for top quark pair production and decay in hadronic collisions. Our work is carried out using the Herwig 7 event generator and presents an in-depth study of variations in matching schemes with two systematically different shower algorithms, the traditional angular-ordered and alternative dipole shower. We also present all of the required extensions of the Herwig dipole shower algorithm to properly take into account quark mass effects, as well as its ability to perform top quark decays. The predictions are compared at parton level as well as to Large Hadron Collider data, including in the boosted regime. We find that the regions where predictions with a non-top-quark-specific tune differ drastically from data are plagued by large uncertainties which are consistent between our two shower and matching algorithms.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Observational constraints on early dark energy

In this paper, we review and update constraints on the Early Dark Energy (EDE) model from cosmological data sets, in particular Planck PR3 and PR4 cosmic microwave background (CMB) data and large-scale structure (LSS) data sets including galaxy clustering and weak lensing data from the Dark Energy Survey, Subaru Hyper Suprime-Cam and KiDS+VIKING-450, as well as BOSS/eBOSS galaxy clustering and Lyman-[Formula: see text] forest data. We detail the fit to CMB data, and perform the first analyses of EDE using the CAMSPEC and Hillipop likelihoods for Planck CMB data, rather than Plik, both of which yield a tighter upper bound on the allowed EDE fraction than that found with Plik. We then supplement CMB data with LSS data in a series of new analyses. All these analyses are concordant in their Bayesian preference for [Formula: see text]CDM over EDE, as indicated by marginalized posterior distributions. We perform a series of tests of the impact of priors in these results, and compare with frequentist analyses based on the profile likelihood, finding qualitative agreement with the Bayesian results. All these tests suggest prior volume effects are not a determining factor in analyses of EDE. This work provides both a review of existing constraints and several new analyses.

Astronomy & Astrophysics↗

Constructing a Network Fundamental Diagram: Synthetic Origin–Destination Approach

The Network Fundamental Diagram (NFD) is significantly used in traffic monitoring and control. One obstacle to a wider application of NFDs in network control is the difficulty of obtaining data from all vehicles traveling in the network to construct an accurate NFD. One solution is to estimate the NFD using data from only a fraction of vehicles (i.e., probe vehicles), where the probe vehicle market penetration rate (MPR) needs to be estimated. A previous study conducted by the authors demonstrated that a distance or time weighted harmonic mean was needed to estimate the flow- and density-based MPRs, respectively using a pairing k-mean clustering approach. This paper introduces another approach that utilizes probe vehicle and observed link volume data to estimate the MPR. A heuristic model is proposed to identify the optimum locations from which to collect link traffic volume data for use in the MPR estimation. The estimated MPR can then be used to construct the NFD. Results show that these models can accurately estimate the NFD with limited probe vehicle and link traffic volume data. Accordingly, the models can be used in the field to estimate the NFD using readily available loop detector and probe vehicle data. The ideal locations for traffic volume data collection can also be proactively chosen to generate optimum estimation results. Since the models proposed here show no significant gains with an increased magnitude of collected data after a certain threshold, they will be helpful, particularly when large-scale data collection is not affordable or realistic.

42 ENGINEERING↗