Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82

Network Bandwidth Utilization Forecast Model on High Bandwidth Network

With the increasing number of geographically distributed scientific collaborations and the scale of the data size growth, it has become more challenging for users to achieve the best possible network performance on a shared network. We have developed a forecast model to predict expected bandwidth utilization for high-bandwidth wide area network. The forecast model can improve the efficiency of resource utilization and scheduling data movements on high-bandwidth network to accommodate ever increasing data volume for large-scale scientific data applications. Univariate model is developed with STL and ARIMA on SNMP path utilization data. Compared with traditional approach such as Box-Jenkins methodology, our forecast model reduces computation time by 83.2percent. It also shows resilience against abrupt network usage change. The accuracy of the forecast model is within the standard deviation of the monitored measurements.

97 MATHEMATICS AND COMPUTING↗

Wind Plant Performance Prediction Benchmark Phase 1 (Technical Report)

Financial risk resulting from the uncertainty associated with developing, owning, and operating wind power plants remains a barrier to reducing the levelized cost of energy (LCOE). On average, modern wind power plants in the U.S. underperform their expected annual energy output by 3.5-4.5% , with many underperforming by over 10%. To compensate for this uncertainty, investors require a larger return on investment (ROI) and apply "knock-down" factors that mask much of the underlying sources of uncertainty. Wind energy projects thus have reduced access to low-cost capital. Furthermore, operating wind plants often take a simple approach to estimating operations & maintenance (O&M) costs (e.g. straight-line estimates based on similar plants), which can eat into profits. To overcome these issues, the wind industry must improve the models they use for estimating wind plant performance and operations. An industry consortium (IC) requested that the National Renewable Energy Laboratory (NREL) lead a Department of Energy (DOE) working group to benchmark the accuracy of wind power plant energy predictions against real operational data. The IC was also motivated by DOE and NREL's potential to characterize systematic energy underperformance, identify sources of uncertainty, and explore root causes. The Wind Plant Performance Prediction (WP3) project was created out of this request, and this report represents the successful completion of Phase 1 of the WP3 project. During the project, wind plant owners provided both pre-construction and operational data to NREL. The pre-construction data was provided to wind resource assessment (WRA) consultants so they could conduct energy yield assessments (EYA). NREL took all of the completed EYAs, along with the operational data, and conducted an operational assessment to benchmark the EYA results against actual operational data. Given the large amounts of sensitive data required for this effort, as well as historical opposition to sharing data within industry, successful completion of Phase 1 represents an unprecedented milestone for industry data sharing. To improve the accuracy and confidence of pre-construction EYAs, wind plant owners and investors need better, more certain, energy yield predictions. The WP3 Benchmark Project is an industry-driven response to this reality. For the first time, industry has taken the important step of working together at scale, sharing valuable operational data with DOE and NREL in order to investigate the sources of bias and uncertainty in these energy estimates. This IC provides wind plant preconstruction and operational data to NREL in an organized and documented fashion and provides guidance and feedback as needed. The IC also provides introspection of the design of experiment, key metrics of success, data challenges, analysis best practices, and quality of results.

17 WIND ENERGY↗

The Gravity Collective: A Comprehensive Analysis of the Electromagnetic Search for the Binary Neutron Star Merger GW190425

We present an ultraviolet to infrared search for the electromagnetic (EM) counterpart to GW190425, the second ever binary neutron star merger discovered by the LIGO-Virgo-KAGRA Collaboration. GW190425 was more distant and had a larger localization area than GW170817, so we use a new tool, Teglon, to redistribute the GW190425 localization probability in the context of galaxy catalogs within the final localization volume. We derive a 90th percentile area of 6688 deg 2 , a ∼1.5× improvement relative to the LIGO/Virgo map, and show how Teglon provides an order-of-magnitude boost to the search efficiency of small (≤1 deg 2 ) field-of-view instruments. We combine our data with a large, publicly reported imaging data set, covering 9078.59 deg 2 of unique area and 48.13% of the LIGO/Virgo-assigned localization probability, to calculate the most comprehensive kilonova (KN), short gamma-ray burst (sGRB) afterglow, and model-independent constraints on the EM emission from a hypothetical counterpart to GW190425 to date under the assumption that no counterpart was found in these data. If the counterpart were similar to AT 2017gfo, there would be a 28.4% chance of it being detected in the combined data set. We are relatively insensitive to an on-axis sGRB, and rule out a generic transient with a similar peak luminosity and decline rate as AT 2017gfo to 30% confidence. Finally, across our new imaging and publicly reported data, we find 28 candidate optical counterparts that we cannot rule out as being associated with GW190425, finding that four such counterparts discovered within the localization volume and within 5 days of merger exhibit luminosities consistent with a KN.

79 ASTRONOMY AND ASTROPHYSICS↗

Analysis of Real-World Preignition Data Using Neural Networks

Increasing adoption of downsized, boosted, spark-ignition engines has improved vehicle fuel economy, and continued improvement is desirable to reduce carbon emissions in the near-term. However, this strategy is limited by damaging preignition events which can cause hardware failure. Research to date has shed light on various contributing factors related to fuel and lubricant properties as well as calibration strategies, but the causal factors behind an individual preignition cycle remain elusive. If actionable precursors could be identified, mitigation through active control strategies would be possible. This paper uses artificial neural networks to search for identifiable precursors in the cylinder pressure data from a large real-world data set containing many preignition cycles. It is found that while follow-up preignition cycles in clusters can be readily predicted, the initial preignition cycle is not predictable based on features of the cylinder pressure. Further, this indicates that the alternating pattern of preignition cycles within clusters is influenced by the thermodynamic state as reflected in the pressure, but that the trigger for the initial preignition cycle is not thermodynamic in nature, but more likely tied to a critical threshold in the chemistry of the fuel/lubricant mixture in the upper crevice or other factors related to the presence of an ignition source.

33 ADVANCED PROPULSION SYSTEMS↗

Assimilating compact phase space retrievals of atmospheric composition with WRF-Chem/DART: a regional chemical transport/ensemble Kalman filter data assimilation system

This study introduces the Weather Research and Forecasting Model with chemistry/Data Assimilation Research Testbed (WRF-Chem/DART) chemical transport forecasting/data assimilation system together with the assimilation of ''compact phase space retrievals'' of satellite-derived atmospheric composition products. WRF-Chem is a state-of-the-art chemical transport model. DART is a flexible software environment for researching ensemble data assimilation with different assimilation and forecast model options. DART's primary assimilation tool is the ensemble adjustment Kalman filter. WRF-Chem/DART is applied to the assimilation of Terra/Measurement of Pollution in the Troposphere (MOPITT) carbon monoxide (CO) trace gas retrieval profiles. Those CO observations are first assimilated as quasi-optimal retrievals (QORs). Our results show that assimilation of the CO retrievals (i) reduced WRF-Chem's CO bias in retrieval and state space, and (ii) improved the CO forecast skill by reducing the Root Mean Square Error (RMSE) and increasing the Coefficient of Determination ($R^2$). Those CO forecast improvements were significant at the 95 % level. Trace gas retrieval data sets contain (i) large amounts of data with limited information content per observation, (ii) error covariance cross-correlations, and (iii) contributions from the retrieval prior profile that should be removed before assimilation. Those characteristics present challenges to the assimilation of retrievals. This paper addresses those challenges by introducing the assimilation of compact phase space retrievals (CPSRs). CPSRs are obtained by preprocessing retrieval data sets with an algorithm that (i) compresses the retrieval data, (ii) diagonalizes the error covariance, and (iii) removes the retrieval prior profile contribution. Most modern ensemble assimilation algorithms can efficiently assimilate CPSRs. Our results show that assimilation of MOPITT CO CPSRs reduced the number of observations (and assimilation computation costs) by ~ 35 %, while providing CO forecast improvements comparable to or better than with the assimilation of MOPITT CO QORs.

58 GEOSCIENCES↗

Conceptual design of a data reduction system

A telemetry data processing system was defined of the Data Reduction. Data reduction activities in support of the developmental flights of the Space Shuttle were used as references against which requirements are assessed in general terms. A conceptual system design believed to offer significant throughput for the anticipated types of data reduction activities is presented. The design identifies the use of a large, intermediate data store as a key element in a complex of high speed, single purpose processors, each of which performs predesignated, repetitive operations on either raw or partially processed data. The recommended approach to implement the design concept is to adopt an established interface standard and rely heavily on mature or promising technologies which are considered main stream of the integrated circuit industry. The design system concept, is believed to be implementable without reliance on exotic devices and/or operational procedures. Numerical methods were employed to examine the feasibility of digital discrimination of FDM composite signals, and of eliminating line frequency noises in data measurements.

Source record↗

Buckling characteristics of hypersonic aircraft wing tubular panels

The buckling characteristics of Rene 41 tubular panels installed as wing panels on a hypersonic wing test structure (HWTS) were determined nondestructively through use of a force/stiffness technique. The nondestructive buckling tests were carried out under different combined load conditions and different temperature environments. Two panels were subsequently tested to buckling failure in a universal tension compression testing machine. In spite of some data scattering because of large extrapolations of data points resulting from termination of the test at a somewhat low applied load, the overall test data correlated fairly well with theoretically predicted buckling interaction curves. The structural efficiency of the tubular panels was slightly higher than that of the beaded panels which they replaced.

Ko, William L.↗

Subsetting and Formatting Landsat-7 LOR ETM+ and Data Products

The Landsat-7 Processing System (LPS) processes Landsat-7 Enhanced Thematic Mapper (ETM+) instrument data into large, contiguous segments called "subintervals" and stores them in Level OR (LOR) data files. The LPS processed subinterval products must be subsetted and reformatted before the Level I processing systems can ingest them. The initial full subintervals produced by the LPS are stored mainly in HDF Earth Observing System (HDF-EOS) format which is an extension to the Hierarchical Data Format (HDF). The final LOR products are stored in native HDF format. Primarily the EOS Core System (ECS) and alternately the DAAC Emergency System (DES) subset the subinterval data for the operational Landsat-7 data processing systems. The HDF and HDF-EOS application programming interfaces (APIs) can be used for extensive data subsetting and data reorganization. A stand-alone subsetter tool has been developed which is based on some of the DES code. This tool makes use of the HDF and HDFEOS APIs to perform Landsat-7 LOR product subsetting and demonstrates how HDF and HDFEOS can be used for creating various configurations of full LOR products. How these APIs can be used to efficiently subset, format, and organize Landsat-7 LOR data as demonstrated by the subsetter tool and the DES is discussed.

Reid, Michael R.↗

KDD Services at the Goddard Earth Sciences Distributed Active Archive Center

NASA's Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) processes, stores and distributes earth science data from a variety of remote sensing satellites. End users of the data range from instrument scientists to global change and climate researchers to federal agencies and foreign governments. Many of these users apply data mining techniques to large volumes of data (up to 1 TB) received from the GES DAAC. However, rapid advances in processing power are enabling increases in data processing that are outpacing tape drive performance and network capacity. As a result, the proportion of data that can be distributed to users continues to decrease. As mitigation, we are migrating more data mining and mining preparation activities into the data center in order to reduce the data volume that needs to be distributed and to offer the users a more useful and manageable product. This migration of activities faces a number of technical and human-factor challenges. As data reduction and mining algorithms are normally quite specific to the user's research needs, the user's algorithm must be integrated virtually unchanged into the archive environment. Also, the archive itself is busy with everyday data archive and distribution activities and cannot be dedicated to, or even impacted by, the mining activities. Therefore, we schedule KDD 'campaigns' (similar to reprocessing campaigns), during which we schedule a wholesale retrieval of specific data products, offering users the opportunity to extract information from the data being retrieved during the campaign.

Lynnes, Christopher↗

Features of Point Clouds Synthesized from Multi-View ALOS/PRISM Data and Comparisons with LiDAR Data in Forested Areas

LiDAR waveform data from airborne LiDAR scanners (ALS) e.g. the Land Vegetation and Ice Sensor (LVIS) havebeen successfully used for estimation of forest height and biomass at local scales and have become the preferredremote sensing dataset. However, regional and global applications are limited by the cost of the airborne LiDARdata acquisition and there are no available spaceborne LiDAR systems. Some researchers have demonstrated thepotential for mapping forest height using aerial or spaceborne stereo imagery with very high spatial resolutions.For stereo imageswith global coverage but coarse resolution newanalysis methods need to be used. Unlike mostresearch based on digital surface models, this study concentrated on analyzing the features of point cloud datagenerated from stereo imagery. The synthesizing of point cloud data from multi-view stereo imagery increasedthe point density of the data. The point cloud data over forested areas were analyzed and compared to small footprintLiDAR data and large-footprint LiDAR waveform data. The results showed that the synthesized point clouddata from ALOSPRISM triplets produce vertical distributions similar to LiDAR data and detected the verticalstructure of sparse and non-closed forests at 30mresolution. For dense forest canopies, the canopy could be capturedbut the ground surface could not be seen, so surface elevations from other sourceswould be needed to calculatethe height of the canopy. A canopy height map with 30 m pixels was produced by subtracting nationalelevation dataset (NED) fromthe averaged elevation of synthesized point clouds,which exhibited spatial featuresof roads, forest edges and patches. The linear regression showed that the canopy height map had a good correlationwith RH50 of LVIS data with a slope of 1.04 and R2 of 0.74 indicating that the canopy height derived fromPRISM triplets can be used to estimate forest biomass at 30 m resolution.

LiDARD↗

Battery Health Quantification for TDRS Spacecraft by Using Signature Discriminability Measurement

The NASA/GSFC Space Network Project Office (SN) currently operates a constellation of ten geosynchronous TDRS spacecraft launched over the past 30 years. The SN project collects up to 16.5 Gigabytes of telemetry every month. Generally, the spacecraft health and functionality are obtained by the use of real-time telemetry data for the multiple spacecraft subsystems, which are transmitted to the main ground station at the White Sands Complex in Las Cruces, NM. Recently, the SN has instituted a program of Big Data to analyze the large amounts of data using a variety of tools including Machine Learning, Artificial Intelligence, development of training sets, and a variety of mathematical modeling tools. The goal is to improve spacecraft management and obtain a more accurate prediction of the spacecraft end of life. The combination of these efforts with those of the Aerospace Corporation, which has a contract with the SN to produce yearly reliability estimates for the TDRS fleet, will be performed. This paper presents a new concept called telemetry quality quantification (TQQ) and discusses the progress that has been made in battery performance estimation for the second-generation TDRS spacecraft using a signature discriminability measures (SDM) algorithm combined with the Aerospace Corp. battery life estimation models. This activity is important because many of the TDRS fleet of spacecraft have exceeded their on-orbit design lifetime and, therefore, NASA must carefully manage the spacecraft to continue operations while avoiding an end-of-mission scenario that leaves a non-functioning spacecraft in geosynchronous orbit.

Ma, Kenneth Y.↗

Potential impact of aerosols on convective clouds revealed by Himawari-8 observations over different terrain types in Eastern China

Convective clouds are common and play a major role in Earth’s water cycle and energy balance; they may even develop into storms and cause severe rainfall events. To understand the convective cloud development process, this study investigates the impact of aerosols on convective clouds by considering the influence of both topography and diurnal variation of radiation. By combining texture analysis, clustering and thresholding methods, we identify all convective clouds in two warm seasons (May-September, 2016-2017) in eastern China based on Himawari-8 Level 1 data. Having a large diurnally resolved cloud data together with surface meteorological and environmental measurements, we investigate convective cloud properties and their variation, stratified by elevation and diurnal change. We then analyze the potential impact of aerosol on convective clouds under different meteorological conditions and topographies. In general, convective clouds tend to occur preferentially under polluted conditions in the morning, which reverses in the afternoon. Convective cloud fraction first increases then decreases with aerosol loading, which may contribute to this phenomenon. Topography and diurnal meteorological variations may affect the strength of aerosol microphysical and radiative effects. Updraft is always stronger along the windward slopes of mountains and plateaus, especially in northern China. The prevailing southerly wind near the foothills of mountains and plateaus is likely to contribute to this windward strengthening of updraft and to bring more pollutant into the mountains, thereby strengthening the microphysical effect, invigorating convective clouds. By comparison, over plain, aerosols decrease surface heating and suppresses convection by blocking solar radiation reaching the surface.

Tianmeng Chen↗

CONSTRUCTING A NETWORK FUNDAMENTAL DIAGRAM: A SYNTHETIC ORIGIN-DESTINATION APPROACH

The Network Fundamental Diagram (NFD) is increasingly used in traffic monitoring and control. One obstacle to a wider application of NFDs in network control is the difficulty of obtaining data from all vehicles traveling in the network to construct an accurate NFD. One solution is to estimate the NFD using data from only a fraction of vehicles (i.e., probe vehicles), where the probe vehicle market penetration rate (MPR) needs to be estimated. A previous study conducted by the authors demonstrated that a distance or time weighted harmonic mean was needed to estimate the flow- and density-based MPRs, respectively using a pairing k-mean clustering approach. This paper proposes another approach that utilizes probe vehicle and observed link volume data to estimate the MPR. A heuristic model is proposed to identify the optimum locations from which to collect link traffic volume data for use in the MPR estimation. The estimated MPR can then be used to construct the NFD. Results show that these models can accurately estimate the NFD with limited probe vehicle and link traffic volume data. Accordingly, the models can be used in the field to estimate the NFD using readily available loop detector and probe vehicle data. The ideal locations for traffic volume data collection can also be proactively chosen to generate optimum estimation results. Since the models proposed here show no significant gains with an increased magnitude of collected data after a certain threshold, they will be helpful, particularly when large-scale data collection is not affordable or realistic.

Spatial Analysis, Market Penetration Rate, NFD, Sy↗

Creating a Face from a DNA Sample

The idea of creating a recognizable face from a sample of DNA is moving closer to reality as Oak Ridge National Laboratory (ORNL) researchers employ a novel statistical representation of the face and a genome-wide association study (GWAS) to relate facial characteristics to genetic mutations. The reduction in cost of DNA sequencing and genotyping technology is leading to the availability of massive datasets of human genome sequences and human genotypes representing mutational variation among people. These large-scale data provide for the first time the potential to correlate an individual’s specific genetic code with their physical appearance. Using a specialized statistical method for representing the entire 3D shape of faces, ORNL researchers performed a GWAS to discover associations between genetic mutations and aspects of face shape. Data from 3,556 subjects resulted in the discovery of a very large set of statistically significant associations, which is strong evidence for the viability of predicting faces from DNA.

59 BASIC BIOLOGICAL SCIENCES↗

Spatio-Temporal Denoising Graph Autoencoders with Data Augmentation for Missing Photovoltaic Data Imputation

The integration of the global Photovoltaic (PV) market with real time data-loggers has enabled large scale PV data analytical pipelines for power forecasting and long-term reliability assessment of PV fleets. Nevertheless, the performance of PV data analysis heavily depends on the quality of PV timeseries data. This paper proposes a novel Spatio-Temporal Denoising Graph Autoencoder (STD-GAE) framework to impute missing PV Power Data. STDGAE exploits temporal correlation, spatial coherence, and value dependencies from domain knowledge to recover missing data. It is empowered by two modules. (1) To cope with sparse yet various scenarios of missing data, STD-GAE incorporates a domain-knowledge aware data augmentation module that creates plausible variations of missing data patterns. This generalizes STD-GAE to robust imputation over different seasons and environment. (2) STD-GAE nontrivially integrates spatiotemporal graph convolution layers (to recover local missing data by observed “neighboring” PV plants) and denoising autoencoder (to recover corrupted data from augmented counterpart) to improve the accuracy of imputation accuracy at PV fleet level. We have evaluated our proposed model on two realworld PV datasets. Experimental results show that STD-GAE can achieve a gain of 43.14% in imputation accuracy and remains less sensitive to missing rate, different seasons, and missing scenarios, compared with state-of-the-art data imputation methods such as MIDA and LRTC-TNN.

Fan, Yangxin↗

The Evolution of Research and Education Networks and their Essential Role in Modern Science

ESnet - the Energy Sciences Network - has the mission of enabling the aspects of the US Department of Energy's Office of Science programs and facilities that depend on large collaborations and large-scale data sharing to accomplish their science. The Office of Science supports a large fraction of all U.S. physical science research and operates many large science instruments and supercomputers that are used by both DOE and University researchers. The network requirements of this community have been explored in some detail by ESnet and a long-term plan has been developed in order to ensure adequate networking to support the science. In this paper we describe the planning process (which has been in place for several years and was the basis of a new network that is just now being completed and a new set of network services) and examine the effectiveness and adequacy of the planning process in the light of evolving science requirements.

42 ENGINEERING↗

Characterization and classification of vegetation canopy structure and distribution within the Great Smoky Mountains National Park using LiDAR

Vegetation canopy structure is a critically important habit characteristic for many threatened and endangered birds and other animal species, and it is key information needed by forest and wildlife managers for monitoring and managing forest resources, conservation planning and fostering biodiversity. Advances in Light Detection and Ranging (LiDAR) technologies have enabled remote sensing-based studies of vegetation canopies by capturing three-dimensional structures, yielding information not available in two-dimensional images of the landscape pro- vided by traditional multi-spectral remote sensing platforms. However, the large volume data sets produced by airborne LiDAR instruments pose a significant computational challenge, requiring algorithms to identify and analyze patterns of interest buried within LiDAR point clouds in a computationally efficient manner, utilizing state-of-art computing infrastructure. We developed and applied a computationally efficient approach to analyze a large volume of LiDAR data and to characterize and map the vegetation canopy structures for 139,859 hectares (540 sq. miles) in the Great Smoky Mountains National Park. This study helps improve our understanding of the distribution of vegetation and animal habitats in this extremely diverse ecosystem.

Clustering↗