Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integration.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Recap of DIP Workshop Series: #1 DIP Architecture and Data Integrations Services

This workshop will cover DIP architecture and data integration services to obtain feedback from America for Airlines (A4A) Air Traffic Management Council (ATMC). Participants will get a look at how the DIP architecture is set-up as well as how data integration services are planned to be hosted on the platform. The DIP architecture review is intended to cover how DIP was envisioned and how DIP is being developed to address data needs across the industry. Participants will have a chance to provide feedback on the DIP architecture and gain insight into how one might interface with the DIP to send or receive data. The data integration services portion is intended to cover DIP’s technical approach to data integration. As an example implementation, there will be a first look at possible data fusion on the platform, including utilizing NASA’s Fuser, and tailoring for industry data consumers. Descriptions, at a high-level, of input to and output of the Fuser will also be discussed.

ATM-X↗
DOE OSTI2014

ARM Data Integrator

The Atmospheric Radiation Measurement (ARM) Data Integrator (ADI) streamlines the development of scientific algorithms and analysis of time-series NetCDF data, and improves the content and consistency of the output data products produced by these algorithms. The framework automates the process of retrieving and preparing data for analysis, and allows users to design output data products through a graphical interface. It also provides a modular, flexible software development architecture that scientists can use to generate C, Python, and IDL source code templates that embed the pre and post processing logic allowing the scientist to focus on only their science. The input data, preprocessing, and output data specifications of algorithms are defined through a graphical interface and stored in a database. ADI implements workflow for data integration and supports user access to data through a library of software modules. Data preprocess capabilities supported include automated retrieval of data from input files, merging the retrieved data into appropriately sized chunks, and transformation of the data onto a common coordinate system grid. Through the graphical interface, users can view the details of both their data products and those in the ARM catalog and allows developers to use existing data product to build new data products. Views of the output data products include an overlay of how the design meets ARM archive's data standards providing the user with a visual cue indicating where their output violates an archive standard. The ADI libraries access the information provided through the GUI via a Postgres database. The ADI framework and its supporting components can significantly decrease the time and cost of implementing scientific algorithms while improving the ability of scientists to disseminate their results.

Source record↗
DOE OSTI2014

ARM Data Integrator

The Atmospheric Radiation Measurement (ARM) Data Integrator (ADI) streamlines the development of scientific algorithms and analysis of time-series NetCDF data, and improves the content and consistency of the output data products produced by these algorithms. The framework automates the process of retrieving and preparing data for analysis, and allows users to design output data products through a graphical interface. It also provides a modular, flexible software development architecture that scientists can use to generate C, Python, and IDL source code templates that embed the pre and post processing logic allowing the scientist to focus on only their science. The input data, preprocessing, and output data specifications of algorithms are defined through a graphical interface and stored in a database. ADI implements workflow for data integration and supports user access to data through a library of software modules. Data preprocess capabilities supported include automated retrieval of data from input files, merging the retrieved data into appropriately sized chunks, and transformation of the data onto a common coordinate system grid. Through the graphical interface, users can view the details of both their data products and those in the ARM catalog and allows developers to use existing data product to build new data products. Views of the output data products include an overlay of how the design meets ARM archive's data standards providing the user with a visual cue indicating where their output violates an archive standard. The ADI libraries access the information provided through the GUI via a Postgres database. The ADI framework and its supporting components can significantly decrease the time and cost of implementing scientific algorithms while improving the ability of scientists to disseminate their results.

Source record↗
DOE OSTI2019

Data Integration in Poplar: ‘Omics Layers and Integration Strategies

Populus trichocarpa is an important biofuel feedstock that has been the target of extensive research and is emerging as a model organism for plants, especially woody perennials. This research has generated several large ‘omics datasets. However, only few studies in Populus have attempted to integrate various data types. This review will summarize various ‘omics data layers, focusing on their application in Populus species. Subsequently, network and signal processing techniques for the integration and analysis of these data types will be discussed, with particular reference to examples in Populus.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1978

A conceptual design for an integrated data base management system for remote sensing data

The requirements of potential users were considered in the design of an integrated data base management system, developed to be independent of any specific computer or operating system, and to be used to support investigations in weather and climate. Ultimately, the system would expand to include data from the agriculture, hydrology, and related Earth resources disciplines. An overview of the system and its capabilities is presented. Aspects discussed cover the proposed interactive command language; the application program command language; storage and tabular data maintained by the regional data base management system; the handling of data files and the use of system standard formats; various control structures required to support the internal architecture of the system; and the actual system architecture with the various modules needed to implement the system. The concepts on which the relational data model is based; data integrity, consistency, and quality; and provisions for supporting concurrent access to data within the system are covered in the appendices.

Maresca, P. A.↗
DOE OSTI2019

Method and application of data integration at a nuclear power plant. Light Water Reactor Sustainability Program report

The data of a nuclear power plant (NPP) is stored in silos for the various plant tools. A previous study conducted by the LWRS program identified the different data sources that can benefit online monitoring in an NPP. These data have different structures and tools and are therefore used independently. Integrating data from various sources can result in multiple benefits to online monitoring by enhancing the ability to compare data, reduce statistical uncertainty due to a larger data set, increase data heterogeneity in space and time to reveal confounded information, develop intelligent methods, and share data. Data integration can result in direct cost saving by reducing the amount of effort needed to find needed information. It can also enable establishing a data warehouse, the backbone repository of data for an NPP, and is necessary to evolve NPPs to a holistic online monitoring approach and automate labor-intensive activities. Whether within an NPP or for a fleet of NPPs, deploying a data warehouse in a plant or data broker for a fleet of plants requires overcoming multiple challenges, including the lack of a standard form of storing and using data and the lack of integration methods to compare and map the different data sources. This effort demonstrates two methods and applications for data integration through an application integration pilot and a data fusion pilot. The application integration is targeted towards the software interfacing of data and data management. This approach is commonly used by data integrators. The data fusion pilot is targeted towards intelligently mapping or coupling data for optimized data use. The pilots were performed in collaboration with two NPPs.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2013

Integrated Data Collection Analysis (IDCA) Program - RDX Standard Data Set 2

The Integrated Data Collection Analysis (IDCA) program is conducting a proficiency study for Small- Scale Safety and Thermal (SSST) testing of homemade explosives (HMEs). Described here are the results for impact, friction, electrostatic discharge, and differential scanning calorimetry analysis of the RDX Type II Class 5 standard, from testing the second time in the Proficiency Test. This RDX testing (Set 2) compared to the first (Set 1) was found to have about the same impact sensitivity, have more BAM friction sensitivity, less ABL friction sensitivity, similar ESD sensitivity, and same DSC sensitivity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1998

An emerging network storage management standard: Media error monitoring and reporting information (MEMRI) - to determine optical tape data integrity

Sophisticated network storage management applications are rapidly evolving to satisfy a market demand for highly reliable data storage systems with large data storage capacities and performance requirements. To preserve a high degree of data integrity, these applications must rely on intelligent data storage devices that can provide reliable indicators of data degradation. Error correction activity generally occurs within storage devices without notification to the host. Early indicators of degradation and media error monitoring 333 and reporting (MEMR) techniques implemented in data storage devices allow network storage management applications to notify system administrators of these events and to take appropriate corrective actions before catastrophic errors occur. Although MEMR techniques have been implemented in data storage devices for many years, until 1996 no MEMR standards existed. In 1996 the American National Standards Institute (ANSI) approved the only known (world-wide) industry standard specifying MEMR techniques to verify stored data on optical disks. This industry standard was developed under the auspices of the Association for Information and Image Management (AIIM). A recently formed AIIM Optical Tape Subcommittee initiated the development of another data integrity standard specifying a set of media error monitoring tools and media error monitoring information (MEMRI) to verify stored data on optical tape media. This paper discusses the need for intelligent storage devices that can provide data integrity metadata, the content of the existing data integrity standard for optical disks, and the content of the MEMRI standard being developed by the AIIM Optical Tape Subcommittee.

Podio, Fernando↗
DOE OSTI2016

Creating an Energy Intelligent Campus: Data Integration Challenges and Solutions at a Large Research Campus

Rich, well-organized building performance and energy consumption data enable a host of analytic capabilities for building owners and operators, from basic energy benchmarking to detailed fault detection and system optimization. Unfortunately, data integration for building control systems is challenging and costly in any setting. Large portfolios of buildings--campuses, cities, and corporate portfolios--experience these integration challenges most acutely. These large portfolios often have a wide array of control systems, including multiple vendors and nonstandard communication protocols. They typically have complex information technology (IT) networks and cybersecurity requirements and may integrate distributed energy resources into their infrastructure. Although the challenges are significant, the integration of control system data has the potential to provide proportionally greater value for these organizations through portfolio-scale analytics, comprehensive demand management, and asset performance visibility. As a large research campus, the National Renewable Energy Laboratory (NREL) experiences significant data integration challenges. To meet them, NREL has developed an architecture for effective data collection, integration, and analysis, providing a comprehensive view of data integration based on functional layers. The architecture is being evaluated on the NREL campus through deployment of three pilot implementations.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1998

Local Data Integration in East Central Florida Using the ARPS Data Analysis System

This paper describes the Applied Meteorology Unit's (AMU) efforts to configure, implement, and test a version of the Advanced Regional Prediction System (ARPS) Data Analysis System (ADAS; Brewster 1996) that assimilates all available data within 250 km of the Kennedy Space Center (KSC) and the Eastern Range at Cape Canaveral Air Station (CCAS). The objective for running a Local Data Integration System (LDIS) such as ADAS is to generate products which may enhance weather nowcasts and short-range (less than 6 h) forecasts issued in support of ground and aerospace operations at KSC/CCAS. A LDIS such as ADAS has the potential to provide added value because it combines observational data to produce gridded analyses of temperature, wind, and moisture (including clouds) and diagnostic quantities such as vorticity, divergence, etc. at specified temporal and spatial resolutions. In this regard, a LDTS along with suitable visualization tools may provide users with a ignore complete and comprehensive understanding of evolving weather than could be developed by individually examining the disparate data sets over the same area and time. The AMU implemented a working prototype of the ADAS which does not run in real-time. Instead, the AMU is evaluating ADAS through post-analyses of weather events for a warm and cool season case. The case studies were chosen to investigate the capabilities and limitations of a LDIS such as ADAS including the impact of non-incorporation of specific data sources on the utility of the subsequent analyses.

Case, Jonathan↗
NASA NTRS1985

Multisensor Data Integration Techniques

The availability of data from sensors operating in several different wavelength regions had led to the development of new techniques and strategies for both data management and image analysis. Work is ongoing to develop computer techniques for analysis of integrated data sets. These techniques include coregistration of multisensor images, rectification of radar images in areas of topographic relief to ensure pixel to pixel registration with planimetric data sets, calibration of data so that signatures can be applied to remote areas, normalization of data acquired with disparate sensors and determination of extended spectral signatures of surface units. In addition, software is being developed to analyze coregistrated digital terrain and image data so that automated stratigraphic and structural analyses can be performed. These software procedures include: strike and dip determination, terrain profile generation, stratigraphic column generation, stratigraphic thickness measurements, structural cross-section generation, and creation of 3-D block diagrams. These techniques were applied to coregistered LANDSAT 4 Thematic Mapper (TM), Thermal Infrared Multispectral Scanner (TIMS), and multipolarization synthetic aperture radar (SAR) data of the Wind River Basin in Wyoming.

Evans, D. L.↗
DOE OSTI2026

REDI – Readiness Engine for Data Integration

The Readiness Engine for Data Integration (REDI) is an open-source framework for automating, standardizing, and assessing the process of preparing scientific data for AI training. REDI implements a five-stage pipeline (ingest, preprocess, transform, structure, output) with per-stage provenance instrumentation via Flowcept, domain-aware transformation logic (PII anonymization, regridding, graph encoding, and more), and built-in readiness assessment and validation modes. REDI has been evaluated across climate, proteomics, materials science, and nuclear fusion datasets, demonstrating near-ideal parallel scaling to 100 nodes on OLCF's Frontier system. REDI is deployable as an agent-callable skill in coding environments such as Claude Code and OpenAI Codex, and is complemented by SetGo for FAIR compliance and catalog publication.

Brewer, Wesley [Oak Ridge National Laboratory (ORN↗
NASA NTRSDate not supplied

Data Integrity Challenges in NASA Giovanni

The Geospatial Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni) is an online tool developed by the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC), one of 12 NASA Science Mission Directorate Data Centers (DAACs) to analyze and visualize NASA remote sensing and model data without downloading data and software. As of this writing, over 2000 Earth satellite and model variables are available in Giovanni, including several well-known NASA satellite missions (e.g., TRMM, GPM) and projects (e.g., MERRA-2, GPCP). There are twenty-two plots provided by Giovanni that can be used to analyze, compare, and explore Earth data across disciplines. Results can be shared with colleagues and downloaded for further analysis. Giovanni has helped publish over 3000 referral papers over the years. As open science policies roll in, data integrity has become a major challenge for Giovanni and other tools. For integrity, both data and workflows must be transparent. FAIR-compliant data, including input, intermediate, and result products, as well as their associated statistics, metadata, and information, are needed. The NASA Data Product Development Guide for Data Producers provides a key resource on how to develop FAIR-compliant data products. Data quality information is also needed from data producers and analysis services like Giovanni. The workflow part is quite challenging and requires workflow management improvements, such as recording workflows and making them available to users. In this presentation, we will discuss the data integrity challenges in Giovanni.

data analysis, visualization↗
DOE OSTI2012

Metrics for Assessment of Smart Grid Data Integrity Attacks

There is an emerging consensus that the nation’s electricity grid is vulnerable to cyber attacks. This vulnerability arises from the increasing reliance on using remote measurements, transmitting them over legacy data networks to system operators who make critical decisions based on available data. Data integrity attacks are a class of cyber attacks that involve a compromise of information that is processed by the grid operator. This information can include meter readings of injected power at remote generators, power flows on transmission lines, and relay states. These data integrity attacks have consequences only when the system operator responds to compromised data by redispatching generation under normal or contingency protocols. These consequences include (a) financial losses from sub-optimal economic dispatch to service loads, (b) robustness/resiliency losses from placing the grid at operating points that are at greater risk from contingencies, and (c) systemic losses resulting from cascading failures induced by poor operational choices. This paper is focused on understanding the connections between grid operational procedures and cyber attacks. We first offer two examples to illustrate how data integrity attacks can cause economic and physical damage by misleading operators into taking inappropriate decisions. We then focus on unobservable data integrity attacks involving power meter data. These are coordinated attacks where the compromised data are consistent with the physics of power flow, and are therefore passed by any bad data detection algorithm. We develop metrics to assess the economic impact of these attacks under re-dispatch decisions using optimal power flow methods. These metrics can be use to prioritize the adoption of appropriate countermeasures including PMU placement, encryption, hardware upgrades, and advance attack detection algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2019

An Integrated Data Analytics Platform

An Integrated Science Data Analytics Platform is an environment that enables the confluence of resources for scientific investigation. It harmonizes data, tools and computational resources which subsequently enable the research community to focus on the investigation rather than spending time on security, data preparation, management, etc. OceanWorks is a NASA technology integration project to establish a cloud-based Integrated Ocean Science Data Analytics Platform at NASA’s Physical Oceanography Distributed Active Archive Center (PO.DAAC) for big ocean science. It focuses on advancement and maturity by bringing together several NASA open-source, big data projects for parallel analytics, anomaly detection, in-situ to satellite data matchup, quality-screened data subsetting, search relevancy, and data discovery. Our communities are relying on data distributed through data centers such as the PO.DAAC, COAPS, NCAR, and many others to conduct their research. In typical investigations, scientists would engage in: search for data, evaluate the relevance of that data, download it, and then apply algorithms to identify trends. Such workflow cannot scale if the research involves a massive amount of data or multi-variate measurements. NASA’s Surface Water and Ocean Topography (SWOT) mission is expected to produce massive amount of observational data during its 3-year nominal mission. Collections like SWOT challenges all existing Earth Science data archival, distribution and analysis paradigms. In this paper, we will discuss how OceanWorks enhances the analysis of physical ocean data where the computation is done on an elastic cloud platform next to the archive to deliver fast, web-accessible services for working with oceanographic measurements.

Yang, Chaowei↗
NASA NTRS1982

Oceanographic satellite remote sensing: Registration, rectification, and data integration requirements

The problem of data integration in oceanography is discussed. Recommendations are made for technique development and evaluation, understanding requirements, and packaging techniques for speed, efficiency and ease of use. The primary satellite sensors of interest to oceanography are summarized. It is concluded that imaging type sensors make image processing an important tool for oceanographic studies.

Nichols, D. A.↗
NASA NTRS2007

Community Coordinated Modeling Center Support of Science Needs for Integrated Data Environment

Space science models are essential component of integrated data environment. Space science models are indispensable tools to facilitate effective use of wide variety of distributed scientific sources and to place multi-point local measurements into global context. The Community Coordinated Modeling Center (CCMC) hosts a set of state-of-the- art space science models ranging from the solar atmosphere to the Earth's upper atmosphere. The majority of models residing at CCMC are comprehensive computationally intensive physics-based models. To allow the models to be driven by data relevant to particular events, the CCMC developed an online data file generation tool that automatically downloads data from data providers and transforms them to required format. CCMC provides a tailored web-based visualization interface for the model output, as well as the capability to download simulations output in portable standard format with comprehensive metadata and user-friendly model output analysis library of routines that can be called from any C supporting language. CCMC is developing data interpolation tools that enable to present model output in the same format as observations. CCMC invite community comments and suggestions to better address science needs for the integrated data environment.

Kuznetsova, M. M.↗