Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integration.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

High-performance End-to-End Integrity Verification on Big Data Transfer

The scale of scientific data generated by experimental facilities and simulations in high-performance computing facilities has been proliferating with the emergence of IoT-based big data. In many cases, this data must be transmitted rapidly and reliably to remote facilities for storage, analysis, or sharing, for the Internet of Things (IoT) applications. Simultaneously, IoT data can be verified using a checksum after the data has been written to the disk at the destination to ensure its integrity. However, this end-to-end integrity verification inevitably creates overheads (extra disk I/O and more computation). Thus, the overall data transfer time increases. In this article, we evaluate strategies to maximize the overlap between data transfer and checksum computation for astronomical observation data. Specifically, we examine file-level and block-level (with various block sizes) pipelining to overlap data transfer and checksum computation. We analyze these pipelining approaches in the context of GridFTP, a widely used protocol for scientific data transfers. Theoretical analysis and experiments are conducted to evaluate our methods. The results show that block-level pipelining is effective in maximizing the overlap mentioned above, and can improve the overall data transfer time with end-to-end integrity verification by up to 70% compared to the sequential execution of transfer and checksum, and by up to 60% compared to file-level pipelining.

data integrity↗
DOE OSTI2026

Integrated Energy-Water Data for Cross-Sector Resilience

This white paper focuses on the “energy-for-water” domain, addressing the urgent need for integrated, empirical data to support regional management, benchmarking, and research on improving efficiency and developing technologies for water and wastewater management systems. The costs and energy required for the supply, treatment, and distribution of water and wastewater lack a standard data collection mechanism and centralized database or storage infrastructure, limiting data-driven decision-making across interdependent infrastructure systems.

42 ENGINEERING↗
NASA NTRS1979

Integrated use of Landsat data for state resource management

The study deals with the integration of a practical information resource - data from the earth orbiting satellite Landsat - with alternate forms of data to structure a state resource data base. State use of Landsat data within a comprehensive resource management scheme is examined. The efforts of NASA in the transfer of this space technology to state resource management applications are outlined. The role two nongovernmental sectors, universities and private industry, play in assisting states to develop Landsat data analysis capability is described.

Schneider, W. G., Jr.↗
NASA NTRS2021

Integrating Inland and Coastal Water Quality Data for Actionable Knowledge

Water quality measures for inland and coastal waters are available as discrete samples from professional and volunteer water quality monitoring programs and higher-frequency, near-continuous data from automated in situ sensors. Water quality parameters also are estimated from model outputs and remote sensing. The integration of these data, via data assimilation, can result in a more holistic characterization of these highly dynamic ecosystems, and consequently improve water resource management. It is becoming common to see combinations of these data applied to answer relevant scientific questions. Yet, methods for scaling water quality data across regions and beyond, to provide actionable knowledge for stakeholders, have emerged only recently, particularly with the availability of satellite data now providing global coverage at high spatial resolution. In this paper, data sources and existing data integration frameworks are reviewed to give an overview of the present status and identify the gaps in existing frameworks. We propose an integration framework to provide information to user communities through the the Group on Earth Observations (GEO) AquaWatch Initiative. This aims to develop and build the global capacity and utility of water quality data, products, and information to support equitable and inclusive access for water resource management, policy and decision making.

Ghada Y.H. El Serafy↗
DOE OSTI2021

Integration of AI/ML with Data Assimilation for Earth System Prediction

Focal Area(s): Development of dynamically consistent AI/ML methods that can integrate with the data assimilation cycle to improve the efficiency and effectiveness of state estimation and prediction at short to medium range forecast horizons. Science Challenge: This white paper addresses a first step toward complete integration of AI/ML and DA. Here, we address model emulation and its application to simple coupled atmosphere-ocean dynamics for use in data assimilation applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Open Science principles for accelerating trait-based science across the Tree of Life

Synthesizing trait observations and knowledge across the Tree of Life remains a grand challenge for biodiversity science. Species traits are widely used in ecological and evolutionary science, and new data and methods have proliferated rapidly. Yet accessing and integrating disparate data sources remains a considerable challenge, slowing progress toward a global synthesis to integrate trait data across organisms. Trait science needs a vision for achieving global integration across all organisms. In this perspective, we outline how the adoption of key Open Science principles—open data, open source and open methods—is transforming trait science, increasing transparency, democratizing access and accelerating global synthesis. To enhance widespread adoption of these principles, we introduce the Open Traits Network (OTN), a global, decentralized community welcoming all researchers and institutions pursuing the collaborative goal of standardizing and integrating trait data across organisms. We demonstrate how adherence to Open Science principles is key to the OTN community and outline five activities that can accelerate the synthesis of trait data across the Tree of Life, thereby facilitating rapid advances to address scientific inquiries and environmental issues. Lessons learned along the path to a global synthesis of trait data will provide a framework for addressing similarly complex data science and informatics challenges.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2014

Methods for radiation detection and characterization using a multiple detector probe

Apparatuses, methods, and systems relating to radiological characterization of environments are disclosed. Multi-detector probes with a plurality of detectors in a common housing may be used to substantially concurrently detect a plurality of different radiation activities and types. Multiple multi-detector probes may be used in a down-hole environment to substantially concurrently detect radioactive activity and contents of a buried waste container. Software may process, analyze, and integrate the data from the different multi-detector probes and the different detector types therein to provide source location and integrated analysis as to the source types and activity in the measured environment. Further, the integrated data may be used to compensate for differential density effects and the effects of radiation shielding materials within the volume being measured.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

An Integrated Platform for Collaborative Data Analytics

While collaboration among data scientists is a key to organizational productivity, data analysts face significant barriers to achieving this end, including data sharing, accessing and configuring the required computational environment, and a unified method of sharing knowledge. Each of these barriers to collaboration is related to the fundamental question of knowledge management “how can organizations use knowledge more effectively?”. In this paper, we consider the problem of knowledge management in collaborative data analytics and present ShareAL, an integrated knowledge management platform, as a solution to that problem. The ShareAL platform consists of three core components: a full stack web application, a dashboard for analyzing streaming data and a High Performance Computing (HPC) cluster for performing real time analysis. Prior research has not applied knowledge management to collaborative analytics or developed a platform with the same capabilities as ShareAL. ShareAL overcomes the barriers data scientists face to collaboration by providing intuitive sharing of data and analytics via the web application, a shared computing environment via the HPC cluster and knowledge sharing and collaboration via a real time messaging application.

Oesch, T↗
DOE OSTI2015

Integrated IO Services for Trilinos Data Structures

The Trilinos Project is an effort to develop algorithms and enabling technologies within an object- oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems. A unique design feature of Trilinos is its focus on packages. While the abstractions make it easy to incorporate advanced processing and data manipulation tools, it is not always obvious how to take advantage of these features. The trios package incorporated two years ago offers general data management services, but has yet to offer integrated support for core Trilinos data structures, such as those offered in the Tpetra package. An initial attempt to incorporate native Trilinos data structure support into trios services revealed the complexity, from a non-mathematician's perspective, of using Trilinos. This project sought to understand the complexities and potential barriers not just for non-mathematicians that want to contribute to or use Trilinos, but potentially for new mathematically-inclined users as well that may want to offer services to support users. This report documents the challenges for trios to offer some simple data manipulation required as a precursor to any direct data services integration and makes recommendations for clarifying the performance implications and general approach to use.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Missing data in multi-omics integration: Recent advances through artificial intelligence

Biological systems function through complex interactions between various ‘omics (biomolecules), and a more complete understanding of these systems is only possible through an integrated, multi-omic perspective. This has presented the need for the development of integration approaches that are able to capture the complex, often non-linear, interactions that define these biological systems and are adapted to the challenges of combining the heterogenous data across ‘omic views. A principal challenge to multi-omic integration is missing data because all biomolecules are not measured in all samples. Due to either cost, instrument sensitivity, or other experimental factors, data for a biological sample may be missing for one or more ‘omic techologies. Recent methodological developments in artificial intelligence and statistical learning have greatly facilitated the analyses of multi-omics data, however many of these techniques assume access to completely observed data. A subset of these methods incorporate mechanisms for handling partially observed samples, and these methods are the focus of this review. We describe recently developed approaches, noting their primary use cases and highlighting each method's approach to handling missing data. We additionally provide an overview of the more traditional missing data workflows and their limitations; and we discuss potential avenues for further developments as well as how the missing data issue and its current solutions may generalize beyond the multi-omics context.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Big data analytics in smart grids: state-of-the-art, challenges, opportunities, and future directions

Big data has a potential to unlock novel groundbreaking opportunities in the power grid sector that enhances a multitude of technical, social, and economic gains. The currently untapped potential of applying the science of big data for better planning and operation of the power grid is a very challenging task and needs significant efforts all-around. As power grid technologies evolve in conjunction with measurement and communication technologies, this results in unprecedented amount of heterogeneous big data sets from diverse sources. In particular, computational complexity, data security, and operational integration of big data into utility decision frameworks are the key challenges to transform the heterogeneous large dataset into actionable outcomes. Moreover, due to the complex nature of power grids along with the need to balance power in real time, seamless integration of big data into utility operations is very critical. In this context, big data analytics combined with grid visualization can lead to better situational awareness and predictive decisions. This paper presents a comprehensive state-of-the-art review of big data analytics and its applications in power grids, and also identifies challenges and opportunities from utility, industry, and research perspectives. The paper analyzes research gaps and presents insights on future research directions to integrate big data analytics into electric utility decision framework. Detailed information for utilities looking to apply big data analytics and details insights on how utilities can enhance revenue streams and bring disruptive innovation in the industry are discussed. More importantly, general guidelines for utilities to make the right investment in the adoption of big data analytics by unveiling interdependencies among critical infrastructures and operations are also provided.

13 HYDRO ENERGY↗
DOE OSTI2026

A Feasibility Study on the Integration of Human Performance Data From Diverse Sources Based on the Complexity of a Proceduralized Task

Securing the safety of socio-technical systems including nuclear facilities is the upmost goal to ensure their sustainability because historical records demonstrate that the performance degradation of human operators (e.g., human errors) is one of the crucial contributors to the occurrence of unexpected events resulting in extensive casualties and financial losses. This implies that the collection of human performance data in diverse conditions with which they could be faced during the operation of nuclear facilities. As this collection requires significant resources, it is necessary to resolve how to accomplish it with limited resources. To address this challenge, as suggested in the SHEEP framework, it is indispensable to extract valuable insights after integrating various kinds of human performance data obtained from different sources. However, a practical method to soundly integrate them seems to be still incomplete. Accordingly, the applicability of TACOM (Task Complexity) measure is investigated as a tool to identify useful information based on the integration of human performance data observed from different simulation conditions. As a result, it is expected that the TACOM measure would play an important role in addressing the technical challenge in securing human performance data.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS2003

ISIS Processing Tools for Thermal Emission Spectrometer Data

The Integrated Software for Imagers and Spectrometers (ISIS) package is a widely used planetary data processing and cartography software system produced by the Astrogeology program of the USGS in Flagstaff, AZ. Recent additions to the ISIS system provide useful tools for extracting and projecting Thermal Emission Spectrometer (TES) data for use with other ISIS programs to process, analyze, and visualize these data, particularly in comparison with other Mars data sets. A general overview of various programs and tools used for extracting and processing TES data are presented.

K Becker↗
DOE OSTI2023

The Monarch Initiative in 2024: an analytic platform integrating phenotypes, genes and diseases across species

Abstract Bridging the gap between genetic variations, environmental determinants, and phenotypic outcomes is critical for supporting clinical diagnosis and understanding mechanisms of diseases. It requires integrating open data at a global scale. The Monarch Initiative advances these goals by developing open ontologies, semantic data models, and knowledge graphs for translational research. The Monarch App is an integrated platform combining data about genes, phenotypes, and diseases across species. Monarch's APIs enable access to carefully curated datasets and advanced analysis tools that support the understanding and diagnosis of disease for diverse applications such as variant prioritization, deep phenotyping, and patient profile-matching. We have migrated our system into a scalable, cloud-based infrastructure; simplified Monarch's data ingestion and knowledge graph integration systems; enhanced data mapping and integration standards; and developed a new user interface with novel search and graph navigation features. Furthermore, we advanced Monarch's analytic tools by developing a customized plugin for OpenAI’s ChatGPT to increase the reliability of its responses about phenotypic data, allowing us to interrogate the knowledge in the Monarch graph using state-of-the-art Large Language Models. The resources of the Monarch Initiative can be found at monarchinitiative.org and its corresponding code repository at github.com/monarch-initiative/monarch-app.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1984

Application of MAGSAT to Lithospheric Modeling in South America. Part 2: Synthesis of Geologic and Seismic Data for Development of Integrated Crustal Models

Research activities performed on MAGSAT scalar data over South America, Central America, and the adjacent marine areas are summarized. The geologic utility of magnetic anomalies detected by satellite is demonstrated by focusing on the spherical-Earth interpretation of scalar MAGSAT data in combination with ancillary geological and geophysical data to obtain lithospheric models for these regions related to their contemporary crustal dynamics processes, geologic history, current volcanism seismicity and natural resources.

Keller, G. R.↗
NASA NTRS2012

Monitoring Strategies for REDD+: Integrating Field, Airborne, and Satellite Observations of Amazon Forests

Large-scale tropical forest monitoring efforts in support of REDD+ (Reducing Emissions from Deforestation and forest Degradation plus enhancing forest carbon stocks) confront a range of challenges. REDD+ activities typically have short reporting time scales, diverse data needs, and low tolerance for uncertainties. Meeting these challenges will require innovative use of remote sensing data, including integrating data at different spatial and temporal resolutions. The global scientific community is engaged in developing, evaluating, and applying new methods for regional to global scale forest monitoring. Pilot REDD+ activities are underway across the tropics with support from a range of national and international groups, including SilvaCarbon, an interagency effort to coordinate US expertise on forest monitoring and resource management. Early actions on REDD+ have exposed some of the inherent tradeoffs that arise from the use of incomplete or inaccurate data to quantify forest area changes and related carbon emissions. Here, we summarize recent advances in forest monitoring to identify and target the main sources of uncertainty in estimates of forest area changes, aboveground carbon stocks, and Amazon forest carbon emissions.

Morton, Douglas↗