Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integration.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,747 records · Page 97
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Continuous Emulation and Multiscale Visualization of Traffic Flow Using Stationary Roadside Sensor Data

With the advent of the next-generation traffic monitoring systems, there has been a significant increase in the spatial-temporal resolution of vehicle mobility data in many cities. Effective analysis and visualization of such data can provide transportation planners with data-driven insights, which can facilitate the understanding of multiscale traffic dynamics. In this paper, we present a web-based traffic emulator for emulating and visualizing near-real-time and historical traffic flows on highways using data from road-side sensors. To construct a continuous traffic flow, the emulator adopts an analytical pipeline that can (a) integrate traffic data collected from discrete road-side radar detection sensors, (b) interpolate traffic conditions (vehicle speed and volume) on unmeasured road segments based on traffic flow theory, and (c) generate lane-specific vehicle trajectories and movements using a mathematically optimized representation of the road network. Our app also provides an integrated visual workflow that allows users to explore the interconnected traffic dynamics using an appropriate traffic flow visualization selected based on the level of detail. We devise two innovative geo-visualization techniques that utilize an animated strips-network representation and a lane usage matrix to visualize lane performances. To ensure a smooth emulation of large-scale traffic flow in an easy-to-access web environment, we implement the emulator using client-side GPU-accelerated techniques. Lastly, we close with a case study that visualizes traffic dynamics of two scenarios - an afternoon peak hour and a traffic accident - in Chattanooga, Tennessee. Our app visualizes the responses of traffic dynamics during different traffic conditions, and to the presence of the traffic accident at different spatial scales.

42 ENGINEERING↗
DOE OSTI2016

The Medical Science DMZ

Objective We describe use cases and an institutional reference architecture for maintaining high-capacity, data-intensive network flows (e.g., 10, 40, 100 Gbpsþ) in a scientific, medical context while still adhering to security and privacy laws and regulations. Materials and Methods High-end networking, packet filter firewalls, network intrusion detection systems. Results We describe a “Medical Science DMZ” concept as an option for secure, high-volume transport of large, sensitive data sets between research institutions over national research networks. Discussion The exponentially increasing amounts of “omics” data, the rapid increase of high-quality imaging, and other rapidly growing clinical data sets have resulted in the rise of biomedical research “big data.” The storage, analysis, and network resources required to process these data and integrate them into patient diagnoses and treatments have grown to scales that strain the capabilities of academic health centers. Some data are not generated locally and cannot be sustained locally, and shared data repositories such as those provided by the National Library of Medicine, the National Cancer Institute, and international partners such as the European Bioinformatics Institute are rapidly growing. The ability to store and compute using these data must therefore be addressed by a combination of local, national, and industry resources that exchange large data sets. Maintaining data-intensive flows that comply with HIPAA and other regulations presents a new challenge for biomedical research. Recognizing this, we describe a strategy that marries performance and security by borrowing from and redefining the concept of a “Science DMZ”—a framework that is used in physical sciences and engineering research to manage high-capacity data flows. Conclusion By implementing a Medical Science DMZ architecture, biomedical researchers can leverage the scale provided by high-performance computer and cloud storage facilities and national high-speed research networks while preserving privacy and meeting regulatory requirements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

DataFed: Towards Reproducible Research via Federated Data Management

The increasingly collaborative, globalized nature of scientific research combined with the need to share data and the explosion in data volumes present an urgent need for a scientific data management system (SDMS). An SDMS presents a logical and holistic view of data that greatly simplifies and empowers data organization, curation, searching, sharing, dissemination, etc. We present DataFed - a lightweight, distributed SDMS that spans a federation of storage systems within a loosely-coupled network of scientific facilities. Unlike existing SDMS offerings, DataFed uses high-performance and scalable user management and data transfer technologies that simplify deployment, maintenance, and expansion of DataFed. DataFed provides web-based and command-line interfaces to manage data and integrate with complex scientific workflows. DataFed represents a step towards reproducible scientific research by enabling reliable staging of the correct data at the desired environment.

Stansberry, Dale↗
DOE OSTI2013

Advanced Seismic Data Analysis Program (The Hot Pot Project), DOE Award: DE-EE0002839 (Phase 1 Report)

A five-line (23 mile) reflection- seismic survey was conducted at the Hot Pot geothermal prospect area in north-central Nevada under the USDOE (United States Department of Energy) Geothermal Technologies Program. The project objective was to utilize innovative seismic data processing, integrated with existing geological, geophysical and geochemical information, to identify high-potential drilling targets and to reduce drilling risk. Data acquisition and interpretation took place between October 2010 and April 2011. The first round of data processing resulted in large areas of relatively poor data, and obvious reflectors known from existing subsurface information either did not appear on the seismic profiles or appeared at the wrong depth. To resolve these issues, the velocity model was adjusted to include geologic input, and the lines were reprocessed. The resulting products were significantly improved, and additional detail was recovered within the high-velocity and in part acoustically isotropic basement. Features visible on the improved seismic images include interpreted low angle thrust faults within the Paleozoic Valmy Formation, which potentially are reactivated in the current stress field. Intermediate-depth wells are currently targeted to test these features. The seismic images also suggest the existence of Paleogene sedimentary and volcanic rocks which potentially may function as a near-surface reservoir, charged by deeper structures in Paleozoic rocks.

15 GEOTHERMAL ENERGY↗
DOE OSTI2017

Potential Futures for Information.

Information is one of the most powerful tools available today. All advances in technology may be used, as David Sarnoff said, for the benefit or harm of society. Information can be used to shape the future by free people, or used to control people by less than benevolent governments, as has been demonstrated since the mid - 1930s, and with growing frequency over the past 50 years. What promised to once set people free and fuel an industrial revolution that might improve the standard of living over most of the world, has also been used to manipulate and enslave entire populations. The future of information is tied to the future of technologies that support the collection of data, processing those data into information and knowledge, and distribution. Technologies supporting the future of information must include technologies that help protect the integrity of data and information, and help to guarantee its discoverability and appropriate availability -- often to the whole of society. This Page Intentionally Left Blank

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2019

Exploring the Use of FHWA Truck Traffic Volume and Weight Data to Support National Truck Freight Mobility Study

The goal of this exploratory study was to investigate alternative data sources and develop methodologies to estimate trends and adjustment factors that could be applied to calibrate statistics generated from the outdated 2002 VIUS. The main focus of this study was on utilizing the FHWA’s traffic data (i.e., VTRIS and TMAS), along with other supplemental freight data (i.e., FAF and CFS), to examine changes in truck volumes and truck weight distributions over time. Ultimately, this effort seeks to develop certain adjustments/factors that could be used to realign perspective VIUS data. Unfortunately, as pointed out previously in this technical memorandum, there are no readily available data sources that could be used to explicitly validate the estimated results from this study. This study compared the estimated distributions from the 2002 VTRIS to the 2002 VIUS data to establish linkages between the two datasets. With distributions and patterns generated based on VTRIS and TMAS data series for additional years, trend factors were then estimated and applied to “update” VIUS-based information. This study resulted in three key highlights: (1) Updated estimates on truck volume and truck VMT can be utilized to better estimate on energy uses by truck. (2) Vehicle class distribution, percent/share of empty trucks, and average truck payload could be updated by utilizing more current annual data series such as VTRIS and TMAS. (3) Additional information could be estimated (e.g., distance distribution by operational type) by integrating other data sources, such as CFS and FAF, instead of relying on the 2002 VIUS data.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

Semi-automated Design of Artificial Intelligence Earth Science Models

Prediction and observation of water cycles involve not only patterns isolated in space and time, but rather modeling complex spatio-temporal relationships across multiple sources of data and domains. For instance, Evapotranspiration (ET) and Leaf Area Indexes (LAI) are two critical components in DOE’s Energy Exascale Earth System Model (E3SM). Accurate assessments of ET and LAI are critical for understanding hydrological processes, deforestation, crop yield, and irrigation impacts. However, current ET estimates for global simulations are available at very coarse spatial resolution. They are usually derived from satellite data based on broad plant functional types (PFT), which fail to capture the fine-scale variations due to change in vegetation type across the globe. Within this context and in light of the data-model integration challenges highlighted in the EESSD Strategic Plan, the new era of AI model development for geosciences calls for data-driven methods that provide domain scientists with estimations of parameters such as PFT and LAI in an efficient, interpretable, and easy-to-operate manner.

Ambrozio Dias, Philipe↗
DOE OSTI2018

Linking big models to big data: efficient ecosystem model calibration through Bayesian model emulation

Data-model integration plays a critical role in assessing and improving our capacity to predict ecosystem dynamics. Similarly, the ability to attach quantitative statements of uncertainty around model forecasts is crucial for model assessment and interpretation and for setting field research priorities. Bayesian methods provide a rigorous data assimilation framework for these applications, especially for problems with multiple data constraints. However, the Markov chain Monte Carlo (MCMC) techniques underlying most Bayesian calibration can be prohibitive for computationally demanding models and large datasets. We employ an alternative method, Bayesian model emulation of sufficient statistics, that can approximate the full joint posterior density, is more amenable to parallelization, and provides an estimate of parameter sensitivity. Analysis involved informative priors constructed from a meta-analysis of the primary literature and specification of both model and data uncertainties, and it introduced novel approaches to autocorrelation corrections on multiple data streams and emulating the sufficient statistics surface. We report the integration of this method within an ecological workflow management software, Predictive Ecosystem Analyzer (PEcAn), and its application and validation with two process-based terrestrial ecosystem models: SIPNET and ED2. In a test against a synthetic dataset, the emulator was able to retrieve the true parameter values. A comparison of the emulator approach to standard brute-force MCMC involving multiple data constraints showed that the emulator method was able to constrain the faster and simpler SIPNET model's parameters with comparable performance to the brute-force approach but reduced computation time by more than 2 orders of magnitude. The emulator was then applied to calibration of the ED2 model, whose complexity precludes standard (brute-force) Bayesian data assimilation techniques. Both models are constrained after assimilation of the observational data with the emulator method, reducing the uncertainty around their predictions. Performance metrics showed increased agreement between model predictions and data. Our study furthers efforts toward reducing model uncertainties, showing that the emulator method makes it possible to efficiently calibrate complex models.

58 GEOSCIENCES↗
DOE OSTI2025

RE-INTEGRATE EMT Simulation Tool: Input Data Processing Layer for Bulk Power System

This paper introduces an advanced input data processing layer for EMT simulations of large-scale bulk power systems. The paper proposes two versions of the RE-INTEGRATE EMT simulation tool, RE-INTEGRATE Gen-0 and RE-INTEGRATE Gen-1, which are developed to enhance simulation generalizability, scalability, and accuracy. The framework leverages a generic class design for components to incorporate linear equations, which are generated by discretizing the Differential-Algebraic Equations (DAEs) that represent the dynamics of the components. In addition, the framework employs a parsing algorithm that parses a power system’s raw and dyr files to generate a connectivity graph which is then traversed to form the overall system’s dynamics. The proposed input data processing layer is used to simulate the IEEE 39-bus test system. The obtained results demonstrate the framework’s capability to achieve simulation scalability and accuracy. Further, the results indicate that EMT simulations performed using the proposed automations can effectively handle complex grid configurations.

Mishra, Rahul [ORNL] (ORCID:0000000328205932)↗
NASA NTRS2011

Expedition Earth and Beyond: Engaging Classrooms in Student-Led Research Using NASA Data, Access to Scientists, and Integrated Educational Strategies

Classroom teachers are challenged with engaging and preparing today s students for the future. Activities are driven by state required skills, education standards, and high-stakes testing. Providing educators with standards-aligned, inquiry-based activities that will help them engage their students in student-led research in the classroom will help them teach required standards, essential skills, and help inspire their students to become motivated learners. The Astromaterials Research and Exploration Science (ARES) Education Program, classroom educators, and ARES scientists at the NASA Johnson Space Center created the Expedition Earth and Beyond education program to help teachers promote student-led research in their classrooms (grades 5-14) by using NASA data, providing access to scientists, and using integrated educational strategies.

Graff, P. V.↗
DOE OSTI2020

Community-Driven Metadata Standards for Agricultural Microbiome Research

Accelerating the pace of microbiome science to enhance crop productivity and agroecosystem health will require transdisciplinary studies, comparisons among datasets, and synthetic analyses of research from diverse crop management contexts. However, despite the widespread availability of crop-associated microbiome data, variation in field sampling and laboratory processing methodologies, as well as metadata collection and reporting, significantly constrains the potential for integrative and comparative analyses. Here we discuss the need for agriculture-specific metadata standards for microbiome research, and propose a list of “required” and “desirable” metadata categories and ontologies essential to be included in a future minimum information metadata standards checklist for describing agricultural microbiome studies. We begin by briefly reviewing existing metadata standards relevant to agricultural microbiome research, and describe ongoing efforts to enhance the potential for integration of data across research studies. Our goal is not to delineate a fixed list of metadata requirements. Instead, we hope to advance the field by providing a starting point for discussion, and inspire researchers to adopt standardized procedures for collecting and reporting consistent and well-annotated metadata for agricultural microbiome research.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

Dry Storage Cask Inventory Assessment

The report, Commercial Spent Nuclear Fuel and High-Level Radioactive Waste Inventory Report [FCRDNFST-2013-000263, Vinson/Metzger/Carter 2017], provides information on the inventory of commercial spent nuclear fuel (SNF) as well as Government-owned SNF and High Level Waste (HLW) in the U.S. Department of Energy (DOE) complex. Inventory forecasts for commercial SNF were made for a few selected scenarios of future commercial nuclear power generation involving the existing reactor fleet including one scenario involving reactors under construction. This Dry Storage Cask Inventory Assessment report uses the data contained in FCRD-NFST-2013- 000263 to define the existing inventory of SNF in dry storage. This information is integrated with data on dry storage canisters and casks from the report, Storage and Transport Cask Data for Used Commercial Nuclear Fuel, 2013 U. S. Edition [ATI-TR-13047 2013] in a Microsoft Access database (hereinafter referred to as the Dry Storage Cask/Inventory Database). The Dry Storage Cask/Inventory Database is used to produce queries for assessing the various systems used for the existing inventory of dry storage casks. The database and this report will be revised in the future as the inventory of fuel in dry storage changes and as additional information becomes available. Both bare fuel and canistered dry storage casks are currently used to dry store SNF. Only 4 utilities currently use bare fuel storage systems for dry storage at 5 reactor sites. Seven unique bare fuel storage cask systems are used to store a total of 9,694 spent fuel assemblies in 212 total casks.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS1986

The Tasseled Cap de-mystified

The fundamental concepts on which the Tasseled Cap transformations of MSS and TM data are based - particularly the identification of inherent data structures - are explained and discussed. Emphasis on the structures present in data from any given sensor, which are themselves the expression of physical characteristics of scene classes, provides a number of advantages, including (a) reduction in data volume with minimal information loss; (b) spectral features which can be applied, without re-definition or adjustment, to any data set for a given sensor; (c) spectral features which can be directly associated with important physical parameters; and (d) easier integration of data from multiple sensors.

Crist, E. P.↗
NASA NTRS1986

A specification-based approach to concurrent structure verification in multiprocessor systems

A recently initiated research project concerned with the concurrent detection of software errors and errors due to physical failures in the hardware of multiprocessor systems is described in this paper. An approach to error detection is described, which is specification based and relies on the structural verification of program control flow and data structure integrity. The techniques discussed utilize the hardware redundancy inherent in parallel processing systems to provide verification of both program structure and data concurrently with program execution.

Fuchs, W. Kent↗
NASA NTRS2014

New Approach to Monitor Transboundary Particulate Pollution over Northeast Asia

A new approach to more accurately monitor and evaluate transboundary particulate matter (PM) pollution is introduced based on aerosol optical products from Korea's Geostationary Ocean Color Imager (GOCI). The area studied is Northeast Asia (including eastern parts of China, the Korean peninsula and Japan), where GOCI has been monitoring since June 2010. The hourly multi-spectral aerosol optical data that were retrieved from GOCI sensor onboard geostationary satellite COMS (Communication, Ocean, and Meteorology Satellite) through the Yonsei aerosol retrieval algorithm were first presented and used in this study. The GOCI-retrieved aerosol optical data are integrated with estimated aerosol distributions from US EPA Models-3/CMAQ (Community Multi-scale Air Quality) v4.5.1 model simulations via data assimilation technique, thereby making the aerosol data spatially continuous and available even for cloud contamination cells. The assimilated aerosol optical data are utilized to provide quantitative estimates of transboundary PM pollution from China to the Korean peninsula and Japan. For the period of 1 April to 31 May, 2011 this analysis yields estimates that AOD as a proxy for PM2.5 or PM10 during long-range transport events increased by 117-265% compared to background average AOD (aerosol optical depth) at the four AERONET sites in Korea, and average AOD increases of 121% were found when averaged over the entire Korean peninsula. This paper demonstrates that the use of multi-spectral AOD retrievals from geostationary satellites can improve estimates of transboundary PM pollution. Such data will become more widely available later this decade when new sensors such as the GEMS (Geostationary Environment Monitoring Spectrometer) and GOCI-2 are scheduled to be launched.

Data Assimilation↗
NASA NTRS2020

Global earth mineral inventory: A data legacy

Minerals contain important clues to understanding the complex geologic history of Earth and other planetary bodies. Therefore, geologists have been collecting mineral samples and compiling data about these samples for centuries. These data have been used to better understand the movement of continental plates, the oxidation of Earth's atmosphere and the water regime of ancient martian landscapes. Datasets found at ‘RRUFF.info/Evolution’ and ‘mindat.org’ have documented a wealth of mineral occurrences around the world. One of the main goals in geoinformatics has been to facilitate discovery by creating and merging datasets from various scientific fields and using statistical methods and visualization tools to inspire and test hypotheses applicable to modelling Earth's past environments. To help achieve this goal, we have compiled physical, chemical and geological properties of minerals and linked them to the above-mentioned mineral occurrence datasets. As a part of the Deep Time Data Infrastructure, funded by the W.M. Keck Foundation, with significant support from the Deep Carbon Observatory (DCO) and the A.P. Sloan Foundation, GEMI (‘Global Earth Mineral Inventory’) was developed from the need of researchers to have all of the required mineral data visible in a single portal, connected by a robust, yet easy to understand schema. Our data legacy integrates these resources into a digestible format for exploration and analysis and has allowed researchers to gain valuable insights from mineralogical data. GEMI can be considered a network, with every node representing some feature of the datasets, for example, a node can represent geological parameters like colour, hardness or lustre. Exploring subnetworks gives the researcher a specific view of the data required for the task at hand. GEMI is accessible through the DCO Data Portal (https://dx.deepcarbon.net/11121/6200-6954-6634-8243-CC). We describe our efforts in compiling GEMI, the Data Policies for usage and sharing, and the evaluation metrics for this data legacy.

data legacy↗
DOE OSTI2020

Improved Access and Analysis of Data Provides New Opportunities for Smart Cities

Making smart, informed, data-driven energy decisions for cities requires large amounts of high-value data and analysis. Cities’ energy data has traditionally been difficult to access and costly to store and manage for city administrators. When data is available, many cities lack the expertise to properly analyze and model large data sets. NREL’s transparency and knowledge about data availability, integration, and interpretation is critical to decision-making for cities.

cities' data↗
DOE OSTI2024

Geospatial Data Platform for All

Spatiotemporal data has evolved in scale due to augmented use in cross-domain applications. Simultaneously, there is substantial growth in the availability of Geographic Information Systems (GIS) data provided by the United States Geological Survey (USGS) along with other federal, state, county, or local agencies through open-data portals and public access APIs. However, data availability does not equate with accessibility. Large-scale analyses and applications require robust, performant data management with co-location of data storage and computing. The insufficiency of data management infrastructure compels researchers to adopt ad hoc project- specific GIS data storage solutions (e.g., copying data to High-Performance computer file systems). As an ad hoc storage strategy does not scale, it hampers cross-domain analyses causing difficulty in data reuse and utilizing existing code bases. Furthermore, GIS data is complex and requires expertise to analyze and manipulate due to its intricate data structures and data-specific projection transformations. Despite the challenges, we recognize that derived GIS data products, e.g., satellite or LIDAR-based images, can be used in downstream applications such as AI by domain, but non-GIS experts. To address the data needs and overcome the challenges, we are working towards a GIS Data Platform focused on efficient data storage, data discovery and access, and an API to enable common workflows. We propose a knowledge-graph (KG) approach for data discovery, whereby datasets are semantically linked to higher- level constructs such as projects and research areas. The semantic data links enable researchers to explore datasets in a top-down approach by specifying relevant and meaningful terms (assists in finding hidden data). An advantage is that the nodes and edges in a knowledge graph create built-in semantic documentation. Deeper spatiotemporal connections between data sources can be encoded via Graph Neural Networks (GNN) (Zhang et al., 2021). The KG approach can be extended to integrate the data itself in a Virtual KG (VKG). Our work will derive inspiration from large-scale VKG efforts that have been undertaken or are currently underway as part of the OpenStreetMap project (Ding et al., 2021). For DOE Data Days, we share the proposed geospatial data platform hybrid (cloud/on-prem) architecture, our work-to-date on storing, retrieving, and transforming LiDAR and raster data relevant to two important NREL use-cases, including the Renewable Energy Potential (reV) Model, and present our proposal for a KG based data discovery engine.

data platform↗