Search NASA⌕ Search

SEARCH · Search NASA

Results for “data sources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

A Metadata Action Language

The data management problem comprises data processing and data tracking. Data processing is the creation of new data based on existing data sources. Data tracking consists of storing metadata descriptions of available data. This paper addresses the data management problem by casting it as an AI planning problem. Actions are data-processing commands, plans are dataflow programs and goals are metadata descriptions of desired data products. Data manipulation is simply plan generation and execution, and a key component of data tracking is inferring the effects of an observed plan. We introduce a new action language for data management domains, called ADILM. We discuss the connection between data processing and information integration and show how a language for the latter must be modified to support the former. The paper also discusses information gathering within a data-processing framework, and show how ADILM metadata expressions are a generalization of Local Completeness.

Golden, Keith↗

All Source Solution Decision Support Products Created for Stennis Space Center in Response to Hurricane Katrina

In the aftermath of Hurricane Katrina and in response to the needs of SSC (Stennis Space Center), NASA required the generation of decision support products with a broad range of geospatial inputs. Applying a systems engineering approach, the NASA ARTPO (Applied Research and Technology Project Office) at SSC evaluated the Center's requirements and source data quality. ARTPO identified data and information products that had the potential to meet decision-making requirements; included were remotely sensed data ranging from high-spatial-resolution aerial images through high-temporal-resolution MODIS (Moderate Resolution Imaging Spectroradiometer) products. Geospatial products, such as FEMA's (Federal Emergency Management Agency's) Advisory Base Flood Elevations, were also relevant. Where possible, ARTPO applied SSC calibration/validation expertise to both clarify the quality of various data source options and to validate that the inputs that were finally chosen met SSC requirements. ARTPO integrated various information sources into multiple decision support products, including two maps: Hurricane Katrina Inundation Effects at Stennis Space Center (highlighting surge risk posture) and Vegetation Change In and Around Stennis Space Center: Katrina and Beyond (highlighting fire risk posture).

Ross, Kenton W.↗

Systems for the Intermodal Routing of Spent Nuclear Fuel

The safe and secure movement of spent nuclear fuel from shutdown and active reactor facilities to intermediate or long term storage sites may, in some instances, require the use of several modes of transportation to accomplish the move. To that end, a fully operable multi-modal routing system is being developed within Oak Ridge National Laboratory s (ORNL) WebTRAGIS (Transportation Routing Analysis Geographic Information System). This study aims to provide an overview of multi-modal routing, the existing state of the TRAGIS networks, the source data needs, and the requirements for developing structural relationships between various modes to create a suitable system for modeling the transport of spent nuclear fuel via a multimodal network. Modern transportation systems are comprised of interconnected, yet separate, modal networks. Efficient transportation networks rely upon the smooth transfer of cargoes at junction points that serve as connectors between modes. A key logistical impediment to the shipment of spent nuclear fuel is the absence of identified or designated transfer locations between transport modes. Understanding the potential network impacts on intermodal transportation of spent nuclear fuel is vital for planning transportation routes from origin to destination. By identifying key locations where modes intersect, routing decisions can be made to prioritize cost savings, optimize transport times and minimize potential risks to the population and environment. In order to facilitate such a process, ORNL began the development of a base intermodal network and associated routing code. The network was developed using previous intermodal networks and information from publicly available data sources to construct a database of potential intermodal transfer locations with likely capability to handle spent nuclear fuel casks. The coding development focused on modifying the existing WebTRAGIS routing code to accommodate intermodal transfers and the selection of prioritization constraints and modifiers to determine route selection. The limitations of the current model and future directions for development are discussed, including the current state of information on possible intermodal transfer locations for spent fuel.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

EDGE COMPUTING AND CONTEXTUAL INFORMATION FOR THE INTERNET OF THINGS SENSORS

Interpreting sensor data require knowledge about sensor placement and the surrounding environment. For a single sensor measurement, it is easy to document the context by visual observation, however for millions of sensors reporting data back to a server, the contextual information needs to be automatically extracted from either data analysis or leveraging complimentary data sources. Data layers that overlap spatially or temporally with sensor locations, can be used to extract the context and to validate the measurement. To minimize the amount of data transmitted through the internet, while preserving signal information content, two methods are explored; computation at the edge and compressed sensing. We validate the above methods on wind and chemical sensor data (1) eliminate redundant measurement from wind sensors and (2) extract peak value of a chemical sensor measuring a methane plume. We present a general cloud based framework to validate sensor data based on statistical and physical modeling and contextual data extracted from geospatial data.

03 NATURAL GAS↗

EDGE COMPUTING AND CONTEXTUAL INFORMATION FOR THE INTERNET OF THINGS SENSORS

Interpreting sensor data require knowledge about sensor placement and the surrounding environment. For a single sensor measurement, it is easy to document the context by visual observation, however for millions of sensors reporting data back to a server, the contextual information needs to be automatically extracted from either data analysis or leveraging complimentary data sources. Data layers that overlap spatially or temporally with sensor locations, can be used to extract the context and to validate the measurement. To minimize the amount of data transmitted through the internet, while preserving signal information content, two methods are explored; computation at the edge and compressed sensing. We validate the above methods on wind and chemical sensor data (1) eliminate redundant measurement from wind sensors and (2) extract peak value of a chemical sensor measuring a methane plume. We present a general cloud based framework to validate sensor data based on statistical and physical modeling and contextual data extracted from geospatial data.

Klein, Levente↗

A Systematic Review of Local to Regional Yield Forecasting Approaches and Frequently Used Data Resources

Forecasting crop yields, or providing an expectation of ex-ante harvest amounts, is highly relevant to the whole agricultural production chain. Farmers can adapt their management, traders or insurers their pricing schemes, suppliers their stocks, logistic companies their routes, national authorities their food balance sheets to guide import or export and, finally, international aid organizations can mobilize reliefs. Evidence has grown in the literature that such forecasts with a meaningful lead time are possible on various geographic scales and for a broad range of crops. Here, we present a systematic review of the methods applied in end-of-season yield forecasting and three frequently used data sources: weather data, satellite data and crop masks. Our literature database comprises 362 studies (2004–2019) which were evaluated regarding methods, crops, regions, data sources, lead time and performance. Moreover, we present 24 sources of real-time and predictive weather data, 21 sources of remote sensing data and 16 crop masks. Yield forecasting in our literature sample has been performed for 44 crops in 71 countries, also including many non-staple crops, but with an apparent bias in regions and crops. Forecasting performance depends on various factors, including crop, region, method, lead time to harvest and input diversity. Our systematic review supports a broader application of locally successful approaches at larger scales by providing a comprehensive, accessible compendium of necessary information for yield forecasting. We discuss improvement potentials with respect to methodological approaches and available data sources. We additionally suggest standardization procedures for future forecasting studies and encourage studying additional crops and geographic regions. Implications of forecasts for different target groups on different scales and the adaptation towards climate change are also discussed.

Seasonal crop yield forecasting↗

Stout: Cloudy’s atomic and molecular database

We describe a new atomic and molecular database we developed for use in the spectral synthesis code Cloudy. The design of Stout is driven by the data needs of Cloudy, which simulates molecular, atomic, and ionized gas with kinetic temperatures 2.8 K<T<10{sup 10} K and densities spanning the low-to high-density limits. The radiation field between photon energies 10{sup −8} Ry and 100 MeV is considered, along with all atoms and ions of the lightest 30 elements, and ∼10{sup 2} molecules. For ease of maintenance, the data are stored in a format as close as possible to the original data sources. Few data sources include the full range of data we need. We describe how we fill in the gaps in the data or extrapolate rates beyond their tabulated range. We tabulate data sources both for the atomic spectroscopic parameters and for collision data for the next release of Cloudy. This is not intended as a review of the current status of atomic data, but rather a description of the features of the database which we will build upon.

79 ASTRONOMY AND ASTROPHYSICS↗

Urban-scale Energy Modeling: Scaling Beyond Tax Assessor Data

In an attempt to attain building-specific characteristics for urban-scale building energy models, county-specific tax assessors’ data is often an initial data source. This data source can contain valuable information such as year built, area, height, HVAC type, and roof/wall descriptions.We will show examples of 2,000 fields from Hamilton County in Tennessee with examples of many fields which are not relevant to urban-scale building energy modeling, are incorrect compared to other data sources, and highlight some lessons learned working with such a data source.There are currently 3,142 counties in the United States, each with their own data format, field definitions, and data access policy. As urban-scale involves city-scale analysis potentially covering multiple counties and matures toward state- or nation-scale analysis, county-by-county approaches are not scalable. While there are efforts to unify these datasets, there is an increasing proliferation of data and algorithms that can cover wider areas and provide more accurate inputs for urban-scale models. This paper summarizes computer vision of imagery, cartographic layers, building type assessment, and model generation used to achieve scalable detection and analysis of buildings.

New, Joshua↗

Linking resource availability to pantropical forest canopy resistance and resilience to cyclone disturbance

Statement of purpose: Tropical cyclones are intensifying and occurring at higher latitudes in recent decades, but the mechanisms underpinning the resistance (ability to withstand disturbance-induced change) and resilience (pace of return to pre-disturbance reference values) of tropical forests to cyclones remains largely unexplored at the pantropical scale. We conducted a meta-analysis to investigate the role of soil resource availability (i.e., total soil phosphorus concentration) in mediating site-level forest canopy resistance and resilience to cyclones pan-tropically. We evaluated cyclone-induced and post-cyclone litterfall mass (g/m2/day), phosphorus (P) and nitrogen (N) fluxes (mg/m2/day), as well as concentrations (mg/g) across 73 case studies in Australia, Guadeloupe, Hawaii, Mexico, Puerto Rico, and Taiwan. The dataset zip file includes three data and two metadata files: - The compiled Litterfall Mass Flux data from tropical forests across the globe prior to and after varying tropical cyclone disturbances are provided in Litterfall_Mass.csv. This data file also includes site location, geographical characteristics, elevation, soil phosphorus concentration, geology, and several variables related to each tropical cyclone disturbance. - The compiled Litterfall Nitrogen and Phosphorus Flux data from tropical forests across the globe prior to and after varying tropical cyclone disturbances are provided in Litterfall_Nutrients.csv. This data file also includes site location, geographical characteristics, elevation, soil phosphorus concentration, geology, and several variables related to each tropical cyclone disturbance. - Tropical cyclone track data compiled from HURDAT2 and IBTrACS databases and used as input in the HURRECON model (https://github.com/hurrecon-model/HurreconR) to generate wind data is provided in hurdat2-1851-2019-052520.txt. - The metadata file (Metadata_Meta-analysis_Litterfall-Mass.pdf) has the complete information on each variable included in the Litterfall_Mass.csv dataset, the data sources, and data processing information. - The metadata file (Metadata_Meta-analysis_Litterfall-Nutrients.pdf) has the complete information on each variable included in the Litterfall_Nutrients.csv dataset, the data sources, and data processing information.

54 ENVIRONMENTAL SCIENCES↗

'Big Data' Collaboration: Exploring, Recording and Sharing Enterprise Knowledge

As data sources and data size proliferate, knowledge discovery from "Big Data" is starting to pose several challenges. In this paper, we address a specific challenge in the practice of enterprise knowledge management while extracting actionable nuggets from diverse data sources of seemingly-related information. In particular, we address the challenge of archiving knowledge gained through collaboration, dissemination and visualization as part of the data analysis, inference and decision-making lifecycle. We motivate the implementation of an enterprise data-discovery and knowledge recorder tool, called SEEKER based on real world case-study. We demonstrate SEEKER capturing schema and data-element relationships, tracking the data elements of value based on the queries and the analytical artifacts that are being created by analysts as they use the data. We show how the tool serves as digital record of institutional domain knowledge and a documentation for the evolution of data elements, queries and schemas over time. As a knowledge management service, a tool like SEEKER saves enterprise resources and time by avoiding analytic silos, expediting the process of multi-source data integration and intelligently documenting discoveries from fellow analysts.

Big Data↗

Data Management at the Advanced Photon Source

Data are crucial to the scientific discoveries enabled by experiments performed at the APS. Presently the APS generates approximately 10 PB of raw experimental data per year from its sixty-six operating beamlines that house over 100 unique instruments. This data is generated as a part of over 6,000 annual experiments performed by over 5,500 facility users each year. Similar to other synchrotrons, the amount of data generated at the APS continues to quickly increase due to beamline advances, such as new measurement techniques, technological advances in detectors, high-throughput instrumentation, multi-modal instruments that can acquire several measurements in a single experiment, and advanced data processing algorithms. This trend is expected to continue in the future. The revolutionized high-energy synchrotron facility proposed in the APS-U project will increase brightness and coherence, leading to further increases in data rates. Successful management of data is of vital importance to the current and future scientific productivity of the APS.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Scalable Metadata Management for a Large Multi-Source Seismic Data Repository

In this work, we implemented the key metadata management components of a scalable seismic data ingestion framework to address limitations in our existing system, and to position it for anticipated growth in volume and complexity. We began the effort with an assessment of open source data flow tools from the Hadoop ecosystem. We then began the construction of a layered architecture that is specifically designed to address many of the scalability and data quality issues we experience with our current pipeline. This included implementing basic functionality in each of the layers, such as establishing a data lake, designing a unified metadata schema, tracking provenance, and calculating data quality metrics. Our original intent was to test and validate the new ingestion framework with data from a large-scale field deployment in a temporary network. This delivered somewhat unsatisfying results, since the new system immediately identified fatal flaws in the data relatively early in the pipeline. Although this is a correct result it did not allow us to sufficiently exercise the whole framework. We then widened our scope to process all available metadata from over a dozen online seismic data sources to further test the implementation and validate the design. This experiment also uncovered a higher than expected frequency of certain types of metadata issues that challenged us to further tune our data management strategy to handle them. Our result from this project is a greatly improved understanding of real world data issues, a validated design, and prototype implementations of major components of an eventual production framework. This successfully forms the basis of future development for the Geophysical Monitoring Program data pipeline, which is a critical asset supporting multiple programs. It also positions us very well to deliver valuable metadata management expertise to our sponsors, and has already resulted in an NNSA Office of Defense Nuclear Nonproliferation commitment to a multi-year project for follow-on work.

58 GEOSCIENCES↗

Scenario driven data modelling: a method for integrating diverse sources of data and data streams

A system and method of integrating diverse sources of data and data streams is presented. The method can include selecting a scenario based on a topic, creating a multi-relational directed graph based on the scenario, identifying and converting resources in accordance with the scenario and updating the multi-directed graph based on the resources, identifying data feeds in accordance with the scenario and updating the multi-directed graph based on the data feeds, identifying analytical routines in accordance with the scenario and updating the multi-directed graph using the analytical routines and identifying data outputs in accordance with the scenario and defining queries to produce the data outputs from the multi-directed graph.

97 MATHEMATICS AND COMPUTING↗

Multi-Formalism Modeling for Disaster Resilience, Forecasting, and Response

Disaster response preparedness, risk mitigation, and effective post-event response are critical for national security, public health, and economic integrity. For areas impacted by extreme events, timely and accurate impact assessments of lifeline infrastructures are critical. Traditional disaster management and situational awareness platforms are not capable of handling the massive influx of available information from traditional and non-traditional sources. Next-generation analytic assessment approaches, where use of dynamically sourced, heterogeneous data with near real-time availability are integrated into a continually-updated analytics framework. The objective of this research was to develop a foundational multi-formalism modeling platform to integrate novel data sources for use in improving situational awareness after a disaster event. Specifically, to 1) provide an architecture for machine-learning based data fusion various multi-sensor datasets; 2) classify, geolocate, and temporally bin novel data sources for use in spatially-enabled machine-learning based data fusion; and 3) derive confidence-based observation data with machine-learning based prediction to fill spatial and temporal observation gaps. This project developed a reusable cloud-based pipeline architecture for ingesting, storing, and analyzing relevant data, executing a range of deep-learning models, and visualize the information geographically. The pipeline development was informed by previous PNNL investments (StreamKit) and built upon leading-edge technologies consisting of: 1) a raw datastore; 2) a data enrichment pipeline with custom and third-party analytics; 3) enriched data storage; and 4) a custom UI to display the enriched data. Machine learning/deep learning models were selected and tested to classify social media images for relevancy and presence of flood waters. Our custom trained relevance models were run on a validation dataset and the overall accuracy of the relevance classifiers ranged from 65% to 90%. We found that convolutional neural network (CNN) models were both faster and more accurate than more traditional machine learning models. We applied the relevance classifier to remove non-relevant images and then undertook a limited training of CNN models to detect flood on ~1,800 labeled images. The testing accuracy of the flood detection models were ~88%. To improve the geographic location of both social media and oblique aerial images we tested a multi-step workflow built on a pre-trained deep-learning model. To identify similarity features from test and reference scenes, we applied the Xception and Faiss models to a set of test images with inaccurate spatial locations, as well and precisely-located reference scenes from the general area. Matching reference scenes to the test scenes with the closest similarity index had limited success in improving the precision of test scene location and additional methods have been identified for testing. Data fusion between authoritative geographic data (satellite imaging) and non-authoritative crowdsourced data is an underlying theme throughout this research and methods are tested bring these data to a common geographic form through feature matching and transformations. The automated extraction of supporting information for disaster response from oblique aerial images or ground-level social media images is accomplished through semantic image segmentation and transformation. The non-authoritative data are used to validate the authoritative data collections and processing and optionally, t+0 flood models. In addition, the common form geographic data are combined with other static landscape features to develop spatially-explicit training vectors used in deep generative statistical inference models to develop complex non-linear relationships in static and dynamic data and ultimately derive a spatially continuous and probabilistic assessment of flood damage with the event domain.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Riverine Plastic Pollution: Sampling and Analysis Methods

Riverine plastic pollution has been found in all major U.S. rivers, but the exact amount of plastic being released to the oceans has not been quantified. Field studies conducted in U.S. rivers have used a range of sampling and analysis techniques and rarely measured the mass of the plastic collected. Measurements of riverine plastic pollution are needed to calibrate and validate models used to estimate the U.S. riverine plastic emissions to the oceans. This report surveys measurement methods used to quantify riverine pollution and current estimates of U.S. riverine plastic pollution from measurements and models. Measurement methods include field sampling and laboratory analysis. Field sampling methods are described for large (macro) and small (micro) plastic particles. Laboratory analysis methods are described for macro and microplastic with an emphasis on the detailed characterization processes of microplastics. Waterborne leachate analysis is also briefly described. Three models are described that estimate plastic pollution based on mismanaged plastic waste in the river catchment basins. The models were validated and calibrated with global data sources. The data sources were predominantly outside of the U.S., where the magnitude and composition of plastic pollution is different than what is found in U.S. rivers. Comprehensive measurements of riverine plastics are needed not only to characterize the riverine plastic pollution, but also parameterize and validate models of plastic fate and transport. This report also describes five key U.S. rivers that span a range of sizes and environmental conditions that could be sampled to obtain data to support characterization and model development of plastic pollution from rivers to oceans. Sampling and analysis protocol recommendations are made to ensure the highest quality of data are collected in the five rivers.

54 ENVIRONMENTAL SCIENCES↗

MASKING VERSUS REMOVING POINT SOURCES IN CMB DATA: THE SOURCE-CORRECTED WMAP POWER SPECTRUM FROM NEW EXTENDED CATALOG

In Scodeller et al., a new and extended point source catalog obtained from the Wilkinson Microwave Anisotropy Probe (WMAP) seven-year data was presented. It includes most of the sources included in the standard WMAP seven-year point source catalogs as well as a large number of new detections. Here, we study the effects on the estimated CMB power spectrum when taking the newly detected point sources into consideration. We create point source masks for all the 2102 sources that we detected as well as a smaller one for the 665 sources detected in the Q, V, and W bands. We also create WMAP7 maps with point sources subtracted in order to compare with the spectrum obtained with source masks. The extended point source masks and point source cleaned WMAP7 maps are made publicly available. Using the proper residual correction, we find that the CMB power spectrum obtained from the point source cleaned map without any source mask is fully consistent with the spectrum obtained from the masked map. We further find that the spectrum obtained masking all 2102 sources is consistent with the results obtained using the standard WMAP seven-year point source mask (KQ85y7). We also verify that the removal of point sources does not introduce any skewness.

79 ASTRONOMY AND ASTROPHYSICS↗

Open Source Scalable Data Services and Data Fusion for Biological and Environmental Sciences (SBIR Phase I Final Scientific/ Technical Report)

The overarching goal of the project is to develop an integrated open-source scientific data management system (Apache V2 license), ResonantEco, that meets the need of biological and environmental researchers and developers for data management, curation, and data processing for analyses with a wide range of scale and complexity. ResonantEco will provide web enabled data services with features such as unified data interfaces and federated views of data and metadata for heterogeneous data sources with an interactive web client for data exploration. Our use of the term fusion is taken from geospatial (GIS) domain where data fusion is often synonymous with data integration. In particular, data integration in ResonantEco involves combining data residing in different sources and providing users with a unified view of them.

99 GENERAL AND MISCELLANEOUS↗