Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integration.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Data And Informatics Working Group On Virtual Data Integration Workshop Report

This report is the outcome of a workshop that was commissioned by the Department of Energy’s Climate and Environmental Sciences Division (CESD) to examine current and future data infrastructure requirements that would be foundational to achieving CESD’s scientific mission goals. Over the past several years, data volumes in CESD disciplines have risen sharply to unprecedented levels (tens of petabytes). So too has the complexity and diversity of the research data (simulation, observation, and reanalysis) needing to be captured, stored, verified, analyzed, and integrated. With the trends of increased data volume (in the hundreds of petabytes), more complex analysis processes, and growing crossdisciplinary collaborations, it is timely to investigate whether the CESD community has the right computational and data support to realize the full scientific potential from its data collections. In recognition of the challenges, a partnership is forming across CESD and with national and international agencies to investigate the viability of creating an integrated, collaborative data infrastructure: a virtual laboratory. The overarching goal of this report is to identify the community’s key data technology requirements and high-priority development needs for sustaining and growing their scientific discovery potential. The report also aims to map these requirements to existing solutions and to identify gaps in current services, tools, and infrastructure that will need to be addressed in the short, medium, and long term so as not to impede scientific progress

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Integrating Data Centers and Grid Technologies at Scale

This presentation focuses on the challenge of integrating AI-driven data centers with the power grid at scale. It examines the AI data center capacity challenge and the role of new Medium Voltage Direct Current (MVDC) and other grid-enhancing technologies in enabling efficient and reliable power delivery. The session will highlight the National Laboratory of the Rockies' ARIES capabilities and planning tools, along with collaborative examples involving Verrus, Compass, and Schneider through the Agora test bed for grid-friendly data center evaluations, and ON. Energy for UPS evaluation. It will showcase the NLR Stable Grid Platform for studying oscillations caused by large-scale data centers, along with planning tools to assess grid security and reliability. Additionally, the presentation covers reconductoring strategies to increase grid capacity and explores innovative data center architectures, including the Advanced DC Architectures with Power-electronic Transformers (ADAPT) platform, which enables testing of complete DC architectures for data centers.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

A Data Exchange Interface for a Standards Based Data Integration Platform

The modern electrical grid is a complex and a data rich system, requiring several sub-systems to provide the capability and support needed to perform functions such as distributed energy resource management, outage management, and fault location and isolation, to name a few. There is a need to develop software solutions and platforms that provide an integrated infrastructure to allow interoperability between these subsystems, while leveraging the abundant data available from today’s grid. In this paper, we present the development and deployment of an application service to integrate GridAPPS-D which is an open source, standards-based software platform with SurvalentONE, a commonly used DNP3 based ADMS platform that allows operation, monitoring, analysis, restoration, and optimization of network operations. We present details of the integration architecture, including its implementation, and provide results from functional testing of the architecture on a 13 bus IEEE system. The results validate that the integration is successful and is able to provide a two-way exchange of communication and translation between different communication protocols.

Singh, Alka↗
DOE OSTI2012

Integrating Data Management and Collaborative Sharing with Computational Science Processes

Structured Scientific Data Management - the management of storage, access, usage, lifecycle, content and meaning for scientific data - is not as commonly employed in computational science as it is in other fields of scientific endeavor. However, where it has been co-developed and integrated with the computational science research it has had a transformational influence on the scientific work. These infrastructures enabled not only new research previously impossible, but also helped to speed up the research process and improved the quality of the research output. Good data management systems are capable of facilitating effective scientific collaborations on a group, institutional, national or international level, through the easy sharing of resources and results. Today as computational science is becoming more data rich and collaborative, integrated scientific data management is becoming an essential tool for every computational science research and production environment. This chapter will describe the fundamental principles and components of a good data management system, provide real world examples of successful implementations and provides an outlook on future developments. We conclude with a short section on how to get started for those whose interest has been peaked by this chapter

42 ENGINEERING↗
DOE OSTI2019

GenomeGraphR: A user-friendly open-source web application for foodborne pathogen whole genome sequencing data integration, analysis, and visualization

Food safety risk assessments and large-scale epidemiological investigations have the potential to provide better and new types of information when whole genome sequence (WGS) data are effectively integrated. Today, the NCBI Pathogen Detection database WGS collections have grown significantly through improvements in technology, coordination, and collaboration, such as the GenomeTrakr and PulseNet networks. However, high-quality genomic data is not often coupled with high-quality epidemiological or food chain metadata. We have created a set of tools for cleaning, curation, integration, analysis and visualization of microbial genome sequencing data. It has been tested using Salmonella enterica and Listeria monocytogenes data sets provided by NCBI Pathogen Detection (160,000 sequenced isolates in 2018). GenomeGraphR presents foodborne pathogen WGS data and associated curated metadata in a user-friendly interface that allows a user to query a variety of research questions such as, transmission sources and dynamics, global reach, and persistence of genotypes associated with contamination in the food supply and foodborne illness across time or space. The application is freely available (https://fda-riskmodels.foodrisk.org/ genomegraphr/).

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

Learning-Based Inversion-Free Model-Data Integration to Advance Ecosystem Model Prediction

Ecosystem model prediction is important for understanding ecosystem responses to climate change and for management support. Model prediction and quantification of predictive uncertainty of ecosystems have long been investigated. The traditional workflow, which calibrates models to match observations and then uses the calibrated models for predictions, relies heavily on inverse modeling to constrain uncertain parameters in complex forward models. This inversion-based prediction approach is infeasible for complex models with heterogeneous parameter uncertainties and incapable of rapid integration of streaming and multiple sources of data because of the difficulty and computational cost in the model inversion, which is typically ill-posed and can require hundreds of thousands of expensive forward simulations to be performed iteratively. We propose to circumvent inverse modeling by precomputing an ensemble of unconstrained forward simulations and then using machine learning (ML) methods to learn the statistical relationship between simulated observation and prediction quantities. Once the ML model has learned the relationship, it can be used to make predictions of future system behavior with uncertainty quantification based on observations. The proposed learningbased inversion-free model prediction (LIMP) framework is computationally efficient which only requires a few thousands of fully parallelizable forward simulations. Additionally, LIMP can continually update predictions based on streaming observations from multiple locations and sources without necessarily requiring extra model simulations. In this study, we apply LIMP to a regional terrestrial ecosystem model with 47 parameters for testing, refining, and evaluating the approach. We demonstrate that LIMP can be used for efficient model prediction, rapid data assimilation, and cost-effective experimental design for improving robust predictive understanding of ecosystems.

Lu, Dan↗
DOE OSTI2013

Reservoir Modeling by Data Integration via Intermediate Spaces and Artificial Intelligence Tools in MPS Simulation Frameworks

Conditioning stochastic simulations are very important in many geostatistical applications that call for the introduction of nonlinear and multiple-point data in reservoir modeling. Here, a new methodology is proposed for the incorporation of different data types into multiple-point statistics (MPS) simulation frameworks. Unlike the previous techniques that call for an approximate forward model (filter) for integration of secondary data into geologically constructed models, the proposed approach develops an intermediate space where all the primary and secondary data are easily mapped onto. Definition of the intermediate space, as may be achieved via application of artificial intelligence tools like neural networks and fuzzy inference systems, eliminates the need for using filters as in previous techniques. The applicability of the proposed approach in conditioning MPS simulations to static and geologic data is verified by modeling a real example of discrete fracture networks using conventional well-log data. The training patterns are well reproduced in the realizations, while the model is also consistent with the map of secondary data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

Energy Systems Integration: Data Call -- Become a Data Partner

This project aims to advance the understanding of costs associated with integrating PV onto the electric power distribution system while maintaining reliable grid operations. We have developed a bottom-up framework for calculating these costs as a function of PV penetration levels on specific feeders. This framework will used to inform and improve utility planning decisions, increase the transparency and speed associated with the interconnection process, and provide policymakers with more information on the total cost of energy from PV.

14 SOLAR ENERGY↗
DOE OSTI2017

Tandem Swift and INTEGRAL Data to Revisit the Orbital and Superorbital Periods of 1E 1740.7–2942

The black hole candidate 1E 1740.7−2942 is one of the strongest hard X-ray sources in the Galactic Center region. No counterparts in longer wavelengths have been identified for this object yet. The presence of characteristic timing signatures in the flux history of X-ray sources has been shown to be an important diagnostic tool for the properties of these systems. Using simultaneous data from NASA’s Swift and ESA’s INTEGRAL missions, we have found two periodic signatures at 12.61 ± 0.06 days and 171.1 ± 3.0 days in long-term hard X-ray light curves of 1E 1740.7−2942. We interpret those as the orbital and superorbital periods of the object, respectively. The reported orbital period is in good agreement with previous studies of 1E 1740.7−2942 using NASA’s RXTE data. We present here the first firm evidence of a superorbital period for 1E 1740.7−2942, which has important implications for the nature of the binary system.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1993

MAC Europe 1991 campaign: AIRSAR/AVIRIS data integration for agricultural test site classification

During summer 1991, multi-sensor data were acquired over the Italian test site 'Otrepo Pavese', an agricultural flat area in Northern Italy. This area has been the Telespazio pilot test site for experimental activities related to agriculture applications. The aim of the investigation described in the following paper is to assess the amount of information contained in the AIRSAR (Airborne Synthetic Aperture Radar) and AVIRIS (Airborne Visible/Infrared Imaging Spectrometer) data, and to evaluate classification results obtained from each sensor data separately and from the combined dataset. All classifications are examined by means of the resulting confusion matrices and Khat coefficients. Improvements of the classification results obtained by using the integrated dataset are finally evaluated.

Sangiovanni, S.↗
DOE OSTI2018

IMG/M v.5.0: an integrated data management and comparative analysis system for microbial genomes and microbiomes

The Integrated Microbial Genomes & Microbiomes system v.5.0 (IMG/M: https://img.jgi.doe.gov/m/) contains annotated datasets categorized into: archaea, bacteria, eukarya, plasmids, viruses, genome fragments, metagenomes, cell enrichments, single particle sorts, and metatranscriptomes. Source datasets include those generated by the DOE's Joint Genome Institute (JGI), submitted by external scientists, or collected from public sequence data archives such as NCBI. All submissions are typically processed through the IMG annotation pipeline and then loaded into the IMG data warehouse. IMG's web user interface provides a variety of analytical and visualization tools for comparative analysis of isolate genomes and metagenomes in IMG. IMG/M allows open access to all public genomes in the IMG data warehouse, while its expert review (ER) system (IMG/MER: https://img.jgi.doe.gov/mer/) enabled registered users to access their private genomes and to store their private datasets in workspace for sharing and for further analysis. IMG/M data content has increased by 60% since the last report published in the 2017 NAR Database Issue. IMG/M v.5.0 has a new and more powerful genome search feature, new statistical tools, and supports metagenome binning.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

Streamline-Based Time-Lapse-Seismic-Data Integration Incorporating Pressure and Saturation Effects

In this paper, we present an efficient history-matching technique that simultaneously integrates 4D repeat seismic surveys with well-production data. This approach is particularly well-suited for the calibration of the reservoir properties of high-resolution geologic models because the seismic data are a really dense but sparse in time, whereas the production data are finely sampled in time but spatially averaged. The joint history matching is performed by use of streamline-based sensitivities derived from either finite-difference or streamline-based flow simulation. For the most part, earlier approaches have focused on the role of saturation changes, but the effects of pressure have largely been ignored. Here, we present a streamline-based semianalytic approach for computing model-parameter sensitivities, accounting for both pressure and saturation effects. The novelty of the method lies in the semianalytic sensitivity computations, making it computationally efficient for high-resolution geologic models. The approach is implemented by use of a finite-difference simulator incorporating the detailed physics. Its efficacy is demonstrated by use of both synthetic and field applications. Lastly, for both the synthetic and the field cases, the advantages of incorporating the time-lapse variations are clear, seen through the improved estimation of the permeability distribution, the pressure profile, the evolution of the fluid saturation, and the swept volumes.

58 GEOSCIENCES↗
DOE OSTI2024

A Framework to Demonstrate a DNP3 Interface With a CIM-Based Data Integration Platform: Preprint

The contemporary electrical grid is characterized by its complexity and abundance of data. A control-rich environment supported by information and communication technologies within an Advanced Distribution Management System (ADMS) presents a viable and cost-effective option for utility companies aiming to implement advanced real-time analytical schemes for monitoring and remotely controlling distribution feeders. Modular platform-based approaches to distribution operations require a structured framework for acquiring field device measurements, performing analytics, converting the setpoint to the correct protocol, and sending it on the appropriate communications network to the field devices. We present the development and deployment of an application service to integrate an open-source standardsbased platform with an ADMS test bed with field devices using the Distributed Network Protocol (DNP3) for data exchange. The step-by-step procedure for establishing the DNP3-Master service on an open-source distribution platform is outlined, comprehensively explaining the Master setup process. Moreover, sample use case results highlight the capabilities of the DNP3- Master service setup. Results demonstrate the scalability and configurability of the DNP3-Master service, making it adaptable for integration with other relevant applications, thus providing potential opportunities for real-world field trials and real-time assessments.

ADMS↗
DOE OSTI2021

Abridged spectral matrix inversion: parametric fitting of X-ray fluorescence spectra following integrative data reduction

Recent improvements in both X-ray detectors and readout speeds have led to a substantial increase in the volume of X-ray fluorescence data being produced at synchrotron facilities. This in turn results in increased challenges associated with processing and fitting such data, both temporally and computationally. Herein an abridging approach is described that both reduces and partially integrates X-ray fluorescence (XRF) data sets to obtain a fivefold total improvement in processing time with negligible decrease in quality of fitting. The approach is demonstrated using linear least-squares matrix inversion on XRF data with strongly overlapping fluorescent peaks. This approach is applicable to any type of linear algebra based fitting algorithm to fit spectra containing overlapping signals wherein the spectra also contain unimportant (non-characteristic) regions which add little (or no) weight to fitted values, e.g. energy regions in XRF spectra that contain little or no peak information.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2016

The Monarch Initiative: an integrative data and analytic platform connecting phenotypes to genotypes across species

The correlation of phenotypic outcomes with genetic variation and environmental factors is a core pursuit in biology and biomedicine. Numerous challenges impede our progress: patient phenotypes may not match known diseases, candidate variants may be in genes that have not been characterized, model organisms may not recapitulate human or veterinary diseases, filling evolutionary gaps is difficult, and many resources must be queried to find potentially significant genotype-phenotype associations. Nonhuman organisms have proven instrumental in revealing biological mechanisms. Advanced informatics tools can identify phenotypically relevant disease models in research and diagnostic contexts. Large-scale integration of model organism and clinical research data can provide a breadth of knowledge not available from individual sources and can provide contextualization of data back to these sources. The Monarch Initiative (monarchinitiative.org) is a collaborative, open science effort that aims to semantically integrate genotype-phenotype data from many species and sources in order to support precision medicine, disease modeling, and mechanistic exploration. Our integrated knowledge graph, analytic tools, and web services enable diverse users to explore relationships between phenotypes and genotypes across species.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2016

NASA/GEWEX Surface Radiation Budget: Integrated Data Product With Reprocessed Radiance, Cloud, and Meteorology Inputs, and New Surface Albedo Treatment

The NASA/GEWEX Surface Radiation Budget (SRB) project produces shortwave and longwave surface and top of atmosphere radiative fluxes for the 1983-near present time period. Spatial resolution is 1 degree. The current release 3.0 (available at gewex-srb.larc.nasa.gov) uses the International Satellite Cloud Climatology Project (ISCCP) DX product for pixel level radiance and cloud information. This product is subsampled to 30 km. ISCCP is currently recalibrating and recomputing their entire data series, to be released as the H product, at 10km resolution. The ninefold increase in pixel number will allow SRB a higher resolution gridded product (e.g. 0.5 degree), as well as the production of pixel-level fluxes. In addition to the input data improvements, several important algorithm improvements have been made. Most notable has been the adaptation of Angular Distribution Models (ADMs) from CERES to improve the initial calculation of shortwave TOA fluxes, from which the surface flux calculations follow. Other key input improvements include a detailed aerosol history using the Max Planck Institut Aerosol Climatology (MAC), temperature and moisture profiles from HIRS, and new topography, surface type, and snow/ice. Here we present results for the improved GEWEX Shortwave and Longwave algorithm (GSW and GLW) with new ISCCP data, the various other improved input data sets and the incorporation of many additional internal SRB model improvements. As of the time of abstract submission, results from 2007 have been produced with ISCCP H availability the limiting factor. More SRB data will be produced as ISCCP reprocessing continues. The SRB data produced will be released as part of the Release 4.0 Integrated Product, recognizing the interdependence of the radiative fluxes with other GEWEX products providing estimates of the Earth's global water and energy cycle (I.e., ISCCP, SeaFlux, LandFlux, NVAP, etc.).

Cox, Stephen J.↗