Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Evolution of HEP Processing Frameworks

HEP data-processing software must support the disparate physics needs of many experiments. For both collider and neutrino environments, HEP experiments typically use data-processing frameworks to manage the computational complexities of their large-scale data processing needs. Data-processing frameworks are being faced with new challenges this decade. The computing landscape has changed from the past three decades of homogeneous single-core x86 batch jobs running on grid sites. Frameworks must now work on a heterogeneous mixture of different platforms: multi-core machines, different CPU architectures, and computational accelerators; and different computing sites: grid, cloud, and high-performance computing. We describe these challenges in more detail and how frameworks may confront them. Given their historic success, frameworks will continue to be critical software systems that enable HEP experiments to meet their computing needs. Frameworks have weathered computing revolutions in the past; they will do so again with support from the HEP community

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

A case study on parallel HDF5 dataset concatenation for high energy physics data analysis

In High Energy Physics (HEP), experimentalists generate large volumes of data that, when analyzed, helps us better understand the fundamental particles and their interactions. This data is often captured in many files of small size, creating a data management challenge for scientists. In order to better facilitate data management, transfer, and analysis on large scale platforms, it is advantageous to aggregate data further into a smaller number of larger files. However, this translation process can consume significant time and resources, and if performed incorrectly the resulting aggregated files can be inefficient for highly parallel access during analysis on large scale platforms. In this paper, we present our case study on parallel I/O strategies and HDF5 features for reducing data aggregation time, making effective use of compression, and ensuring efficient access to the resulting data during analysis at scale. We focus on NOvA detector data in this case study, a large-scale HEP experiment generating many terabytes of data. Here, the lessons learned from our case study inform the handling of similar datasets, thus expanding community knowledge related to this common data management task.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Improving the LandScan USA Non-Obligate Population Estimate (NOPE)

Where do people go when they have nowhere to be? Nonobligate activities are a significant part of our social and cultural lives, but there are no existing large scale data which characterize spatial variability in population allocation for these activities. As large scale population estimates have ever-finer resolutions, gaps in our ability to estimate this population segment have an increasingly large impact on high resolution population estimates. In this paper, we demonstrate an improved method for estimating the spatial allocation of the non-obligate population - people who are not at work, school, or in another residential institution. This method builds upon on anonymized and aggregate data on visits to public places, allocating the non-obligate population proportionally to worker population while accounting for the estimated ratio of visitors to workers in public places.

Brelsford, Christa↗
DOE OSTI2020

Policy Considerations When Federating Facilities for Experimental and Observational Data Analysis

Today’s computational, experimental, and observational facilities afford us tremendous opportunities to couple theory and experiment at increasingly large scales. Empirical sensing capabilities are growing dramatically with beam line and detector improvements, and with advances in our ability to deploy large-scale data gathering observations of the natural world. The coupling of computational simulations and analysis to process the data from experimental and observational facilities is giving rise to cross-facility workflows. Such federations of facilities are in fact becoming an explicit requirement for large-scale scientific discovery. As we scale up these pipelines of scientific discovery, each participating facility needs to establish and align policies so that the federation can work seamlessly in an end-to-end manner. This chapter outlines specific policy considerations in enabling the federation of facilities for data analysis. Design choices and vital policy decisions cover the areas of data acquisition and storage, data transfer, computational resource allocation and co-scheduling, seamless federated user access, and cross-cutting governance. By highlighting the explicit and implicit interdependencies between facilities, we aim to provide facility designers and policymakers the information on policy issues to address early in a facility’s operations, thus enabling successful cross-facility federation and improved experimental and observational data analysis outcomes.

Shankar, Mallikarjun (Arjun)↗
DOE OSTI2023

Chemical Blast Standard (1 kg)

Chemical explosions create blast waves with large overpressure disturbances. It is important to develop a standard blast model based on data to accurately predict acoustic blast-wave amplitudes near detonations and invert for explosion energy from distant observations of blast-wave signals. However, open data from large, controlled chemical explosions with reliable ground truth can be challenging to find. The lack of access to such data could limit the number of contributions to related research and potentially stifle the rate of discoveries or validation of existing models. Here, to address these data scarcity problem, we have curated and compiled a standardized set of 817 blast-wave waveforms from 19 distinct high-explosive events. The blast-wave waveforms are standardized to a 1 kg trinitrotoluene explosion using scaling laws and corrections for location effects. A brief overview of the dataset is presented along with explosion feature models as well as recommendations for extracting explosion features. The resulting dataset is distributed to an open repository in both Seismic Analysis Code and pandas DataFrame formats containing the waveforms, the scaled distances, and the sample rates.

58 GEOSCIENCES↗
DOE OSTI2022

Finding Hidden Patterns in High Resolution Wind Flow Model Simulations

Wind flow data is critical in terms of investment decisions and policy making. High resolution data from wind flow model simulations serve as a supplement to the limited resource of original wind flow data collection. Given the large size of data, finding hidden patterns in wind flow model simulations are critical for reducing the dimensionality of the analysis. In this work, we first perform dimension reduction with two autoencoder models: the CNN-based autoencoder (CNN-AE) [1], and hierarchical autoencoder (HIER-AE) [2], and compare their performance with the Principal Component Analysis (PCA). We then investigate the super-resolution of the wind flow data. By training a Generative Adversarial Network (GAN) with 300 epochs, we obtained a trained model with 2× resolution enhancement. We compare the results of GAN with Convolutional Neural Network (CNN), and GAN results show finer structure as expected in the data field images. Also, the kinetic energy spectra comparisons show that GAN outperforms CNN in terms of reproducing the physical properties for high wavenumbers and is critical for analysis where high-wavenumber kinetics play an important role.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Data Set Analysis to Reduce Uncertainty in Formula Assignments of Ultrahigh Resolution Mass Spectra

Environmental samples contain a vast array of organic compounds with diverse elemental compositions and heteroatom content. Molecular formula assignments of ultrahigh resolution mass spectra (HRMS) hold promise for elucidating the molecular composition of these compounds. However, the need to account for an assortment of heteroatoms increases the uncertainty associated with individual assignments – and ultimately the ecological, biological, and biogeochemical insights gleaned from the assignments. To address this challenge, we introduce a formula assignment strategy that leverages HRMS data sets to improve assignment confidence, filter false assignments, and mitigate bias in assignment routines. The strategy, implemented using CoreMS, first identifies the highest confidence assignment for a recurring ion in a data set by assessing the mass accuracy and isotopologue similarity of all assignments to the ion across the data set. The second component of the strategy examines the consistency of mass errors for an assigned ion throughout a data set and flags formulas with statistically unlikely deviations in mass error. Here, we illustrate the application and utility of the strategy by comparing its results against documented misassignment patterns within a set of oceanographic samples that were measured with 21 T Fourier Transform Ion Cyclotron Resonance Mass Spectrometry. Because the efficacy of our strategy improves with data set size, it is particularly useful for enhancing assignment confidence in large HRMS data sets common in studies of environmental systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Directly imaging the cooling flow in the Phoenix cluster

In the centres of many galaxy clusters, the hot (approximately 10 7 kelvin) intracluster medium can become dense enough that it should cool on short timescales. However, the low measured star formation rates in massive central galaxies and the absence of soft X-ray lines from the cooling gas suggest that most of this gas never cools. This is known as the cooling flow problem. The latest observations suggest that black hole jets are maintaining the vast majority of gas at high temperatures. A cooling flow has yet to be fully mapped through all the gas phases in any galaxy cluster. Here we present observations of the Phoenix cluster using the James Webb Space Telescope to map the [Ne vi] λ 7.652-μm emission line, enabling us to probe the gas at 10 5.5 kelvin on large scales. These data show extended [Ne vi] emission that is cospatial with the cooling peak in the intracluster medium, the coolest gas phases and the sites of active star formation. Taken together, these imply a recent episode of rapid cooling, causing a short-lived spike in the cooling rate, which we estimate to be 5,000–23,000 solar masses per year. As a result, these data provide a large-scale map of gas at temperatures between 10 5 kelvin and 10 6 kelvin in a cluster core, and highlight the critical role that black hole feedback has in not only regulating cooling but also promoting it.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Observational constraints on early dark energy

In this paper, we review and update constraints on the Early Dark Energy (EDE) model from cosmological data sets, in particular Planck PR3 and PR4 cosmic microwave background (CMB) data and large-scale structure (LSS) data sets including galaxy clustering and weak lensing data from the Dark Energy Survey, Subaru Hyper Suprime-Cam and KiDS+VIKING-450, as well as BOSS/eBOSS galaxy clustering and Lyman-[Formula: see text] forest data. We detail the fit to CMB data, and perform the first analyses of EDE using the CAMSPEC and Hillipop likelihoods for Planck CMB data, rather than Plik, both of which yield a tighter upper bound on the allowed EDE fraction than that found with Plik. We then supplement CMB data with LSS data in a series of new analyses. All these analyses are concordant in their Bayesian preference for [Formula: see text]CDM over EDE, as indicated by marginalized posterior distributions. We perform a series of tests of the impact of priors in these results, and compare with frequentist analyses based on the profile likelihood, finding qualitative agreement with the Bayesian results. All these tests suggest prior volume effects are not a determining factor in analyses of EDE. This work provides both a review of existing constraints and several new analyses.

Astronomy & Astrophysics↗
DOE OSTI2021

Wind Plant Performance Prediction Benchmark Phase 1 (Technical Report)

Financial risk resulting from the uncertainty associated with developing, owning, and operating wind power plants remains a barrier to reducing the levelized cost of energy (LCOE). On average, modern wind power plants in the U.S. underperform their expected annual energy output by 3.5-4.5% , with many underperforming by over 10%. To compensate for this uncertainty, investors require a larger return on investment (ROI) and apply "knock-down" factors that mask much of the underlying sources of uncertainty. Wind energy projects thus have reduced access to low-cost capital. Furthermore, operating wind plants often take a simple approach to estimating operations & maintenance (O&M) costs (e.g. straight-line estimates based on similar plants), which can eat into profits. To overcome these issues, the wind industry must improve the models they use for estimating wind plant performance and operations. An industry consortium (IC) requested that the National Renewable Energy Laboratory (NREL) lead a Department of Energy (DOE) working group to benchmark the accuracy of wind power plant energy predictions against real operational data. The IC was also motivated by DOE and NREL's potential to characterize systematic energy underperformance, identify sources of uncertainty, and explore root causes. The Wind Plant Performance Prediction (WP3) project was created out of this request, and this report represents the successful completion of Phase 1 of the WP3 project. During the project, wind plant owners provided both pre-construction and operational data to NREL. The pre-construction data was provided to wind resource assessment (WRA) consultants so they could conduct energy yield assessments (EYA). NREL took all of the completed EYAs, along with the operational data, and conducted an operational assessment to benchmark the EYA results against actual operational data. Given the large amounts of sensitive data required for this effort, as well as historical opposition to sharing data within industry, successful completion of Phase 1 represents an unprecedented milestone for industry data sharing. To improve the accuracy and confidence of pre-construction EYAs, wind plant owners and investors need better, more certain, energy yield predictions. The WP3 Benchmark Project is an industry-driven response to this reality. For the first time, industry has taken the important step of working together at scale, sharing valuable operational data with DOE and NREL in order to investigate the sources of bias and uncertainty in these energy estimates. This IC provides wind plant preconstruction and operational data to NREL in an organized and documented fashion and provides guidance and feedback as needed. The IC also provides introspection of the design of experiment, key metrics of success, data challenges, analysis best practices, and quality of results.

17 WIND ENERGY↗
DOE OSTI2025

The Gravity Collective: A Comprehensive Analysis of the Electromagnetic Search for the Binary Neutron Star Merger GW190425

We present an ultraviolet to infrared search for the electromagnetic (EM) counterpart to GW190425, the second ever binary neutron star merger discovered by the LIGO-Virgo-KAGRA Collaboration. GW190425 was more distant and had a larger localization area than GW170817, so we use a new tool, Teglon, to redistribute the GW190425 localization probability in the context of galaxy catalogs within the final localization volume. We derive a 90th percentile area of 6688 deg 2 , a ∼1.5× improvement relative to the LIGO/Virgo map, and show how Teglon provides an order-of-magnitude boost to the search efficiency of small (≤1 deg 2 ) field-of-view instruments. We combine our data with a large, publicly reported imaging data set, covering 9078.59 deg 2 of unique area and 48.13% of the LIGO/Virgo-assigned localization probability, to calculate the most comprehensive kilonova (KN), short gamma-ray burst (sGRB) afterglow, and model-independent constraints on the EM emission from a hypothetical counterpart to GW190425 to date under the assumption that no counterpart was found in these data. If the counterpart were similar to AT 2017gfo, there would be a 28.4% chance of it being detected in the combined data set. We are relatively insensitive to an on-axis sGRB, and rule out a generic transient with a similar peak luminosity and decline rate as AT 2017gfo to 30% confidence. Finally, across our new imaging and publicly reported data, we find 28 candidate optical counterparts that we cannot rule out as being associated with GW190425, finding that four such counterparts discovered within the localization volume and within 5 days of merger exhibit luminosities consistent with a KN.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Analysis of Real-World Preignition Data Using Neural Networks

Increasing adoption of downsized, boosted, spark-ignition engines has improved vehicle fuel economy, and continued improvement is desirable to reduce carbon emissions in the near-term. However, this strategy is limited by damaging preignition events which can cause hardware failure. Research to date has shed light on various contributing factors related to fuel and lubricant properties as well as calibration strategies, but the causal factors behind an individual preignition cycle remain elusive. If actionable precursors could be identified, mitigation through active control strategies would be possible. This paper uses artificial neural networks to search for identifiable precursors in the cylinder pressure data from a large real-world data set containing many preignition cycles. It is found that while follow-up preignition cycles in clusters can be readily predicted, the initial preignition cycle is not predictable based on features of the cylinder pressure. Further, this indicates that the alternating pattern of preignition cycles within clusters is influenced by the thermodynamic state as reflected in the pressure, but that the trigger for the initial preignition cycle is not thermodynamic in nature, but more likely tied to a critical threshold in the chemistry of the fuel/lubricant mixture in the upper crevice or other factors related to the presence of an ignition source.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Spatio-Temporal Denoising Graph Autoencoders with Data Augmentation for Missing Photovoltaic Data Imputation

The integration of the global Photovoltaic (PV) market with real time data-loggers has enabled large scale PV data analytical pipelines for power forecasting and long-term reliability assessment of PV fleets. Nevertheless, the performance of PV data analysis heavily depends on the quality of PV timeseries data. This paper proposes a novel Spatio-Temporal Denoising Graph Autoencoder (STD-GAE) framework to impute missing PV Power Data. STDGAE exploits temporal correlation, spatial coherence, and value dependencies from domain knowledge to recover missing data. It is empowered by two modules. (1) To cope with sparse yet various scenarios of missing data, STD-GAE incorporates a domain-knowledge aware data augmentation module that creates plausible variations of missing data patterns. This generalizes STD-GAE to robust imputation over different seasons and environment. (2) STD-GAE nontrivially integrates spatiotemporal graph convolution layers (to recover local missing data by observed “neighboring” PV plants) and denoising autoencoder (to recover corrupted data from augmented counterpart) to improve the accuracy of imputation accuracy at PV fleet level. We have evaluated our proposed model on two realworld PV datasets. Experimental results show that STD-GAE can achieve a gain of 43.14% in imputation accuracy and remains less sensitive to missing rate, different seasons, and missing scenarios, compared with state-of-the-art data imputation methods such as MIDA and LRTC-TNN.

Fan, Yangxin↗
DOE OSTI2023

A geospatial risk analysis graphical user interface for identifying hazardous chemical emission sources

Background: Performing back trajectory and forward trajectory using the Hybrid Single-Particle Lagrangian Integrated Trajectory Model (HYSPLIT) is a reliable approach for assessing particle transport after release among mid-field atmospheric models. HYSPLIT has an externally facing online interface that allows non-expert users to run the model trajectories without requiring extensive training or programming. However, the existing HYSPLIT interface is limited if simulations have a large amount of meteorological data and timesteps that are not coincident. The objective of this study is to design and develop a more robust tool to rapidly evaluate hazard transport conditions and to perform risk analysis, while still maintaining an intuitive and user-friendly interface. Methods: HYSPLIT calculates forward and backward trajectories of particles based on wind speed, wind direction, and the corresponding location, timestamp, and Pasquill stability classes of the regions of the atmosphere in terms of the wind speed, the amount of solar radiation, and the fractional cloud cover. The computed particle transport trajectories, combined with the online Proton Transfer Reaction-Mass Spectrometry (PTR-MS) data (https://figshare.com/articles/dataset/ARL_Data_from_PROS_station_at_Hanford_site/19993964), can be used to identify and quantify the sources and affected area of the hazardous chemicals’ emission using the potential source distribution function (PSDF). PSDF is an improved statistical function based on the well-known potential source contribution function (PSCF) in establishing the air pollutant source and receptor relationship. Performing this analysis requires a range of meteorological and pollutant concentration measurements to be statistically meaningful. The existing HYSPLIT graphical user interface (GUI) does not easily permit computations of trajectories of a dataset of meteorological data in high temporal frequency. To improve the performance of HYSPLIT computations from a large dataset and enhance risk analysis of the accidental release of material at risk, a geospatial risk analysis tool (GRAT-GUI) is created to allow large data sets to be processed instantaneously and to provide ease of visualization. Results: The GRAT-GUI is a native desktop-based application and can be run in any Windows 10 system without any internet access requirements, thus providing a secure way to process large meteorological datasets even on a standalone computer. GRAT-GUI has features to import, integrate, and convert meteorological data with various formats for hazardous chemical emission source identification and risk analysis as a self-explanatory user interface. The tool is available at https://figshare.com/articles/software/GRAT/19426742.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

High-throughput solubility determination for data-driven materials design and discovery in redox flow battery research

Solubility is crucial for redox flow batteries because it affects their energy density. A data-driven approach based on artificial intelligence/machine learning models can accelerate the development of highly soluble redox-active materials, but the lack of relevant, large-quantity data makes accurate solubility prediction difficult. To overcome this deficiency, we developed a high-throughput experimentation process that combines a robotically controlled platform with high-throughput methodology to collect large-scale and high-quality solubility data. We demonstrate the potential utility and applicability of this high-throughput process by measuring the aqueous and non-aqueous solubilities of redox-active materials and studying the effect of additives on their solubilities for both aqueous and non-aqueous redox flow battery applications. A redox flow battery based on our optimized negative electrolyte formulation and a ferrocyanide-positive electrolyte offers highly stable performance over 18 days (>100 cycles) with consistent capacity and a 24% boost in energy density.

25 ENERGY STORAGE↗
DOE OSTI2020

Improved Access and Analysis of Data Provides New Opportunities for Smart Cities

Making smart, informed, data-driven energy decisions for cities requires large amounts of high-value data and analysis. Cities’ energy data has traditionally been difficult to access and costly to store and manage for city administrators. When data is available, many cities lack the expertise to properly analyze and model large data sets. NREL’s transparency and knowledge about data availability, integration, and interpretation is critical to decision-making for cities.

cities' data↗
DOE OSTI2023

Efficiency and Demand Flexibility in Large Office Buildings

Data is associated with Report "Efficiency and Demand Flexibility in Large Office Buildings" by Joyce McLaren, Thomas Bowen, and Chioke Harris ( https://doi.org/10.2172/1989231 ). Results are created from repos GEB_ECM_Impact_Estimator ( https://github.nlr.gov/tbowen/GEB_ECM_Impact_Estimator ). Results outline the changes in building load in large office buildings based on measures from NREL's Scout, and the impacts those changes in load have on grid-induced emissions, grid operating costs, and customer bills.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗