Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Integration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

sciCAN: single-cell chromatin accessibility and gene expression data integration via cycle-consistent adversarial network

The boom in single-cell technologies has brought a surge of high dimensional data that come from different sources and represent cellular systems from different views. With advances in these single-cell technologies, integrating single-cell data across modalities arises as a new computational challenge. Here, we present an adversarial approach, sciCAN, to integrate single-cell chromatin accessibility and gene expression data in an unsupervised manner. We benchmarked sciCAN with 5 existing methods in 5 scATAC-seq/scRNA-seq datasets, and we demonstrated that our method dealt with data integration with consistent performance across datasets and better balance of mutual transferring between modalities than the other 5 existing methods. We further applied sciCAN to 10X Multiome data and confirmed that the integrated representation preserves biological relationships within the hematopoietic hierarchy. Finally, we investigated CRISPR-perturbed single-cell K562 ATAC-seq and RNA-seq data to identify cells with related responses to different perturbations in these different modalities.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Integrating data types to estimate spatial patterns of avian migration across the Western Hemisphere

For many avian species, spatial migration patterns remain largely undescribed, especially across hemispheric extents. Recent advancements in tracking technologies and high-resolution species distribution models (i.e., eBird Status and Trends products) provide new insights into migratory bird movements and offer a promising opportunity for integrating independent data sources to describe avian migration. Here, we present a three-stage modeling framework for estimating spatial patterns of avian migration. First, we integrate tracking and band re-encounter data to quantify migratory connectivity, defined as the relative proportions of individuals migrating between breeding and nonbreeding regions. Next, we use estimated connectivity proportions along with eBird occurrence probabilities to produce probabilistic least-cost path (LCP) indices. In a final step, we use generalized additive mixed models (GAMMs) both to evaluate the ability of LCP indices to accurately predict (i.e., as a covariate) observed locations derived from tracking and band re-encounter data sets versus pseudo-absence locations during migratory periods and to create a fully integrated (i.e., eBird occurrence, LCP, and tracking/band re-encounter data) spatial prediction index for mapping species-specific seasonal migrations. To illustrate this approach, we apply this framework to describe seasonal migrations of 12 bird species across the Western Hemisphere during pre- and postbreeding migratory periods (i.e., spring and fall, respectively). We found that including LCP indices with eBird occurrence in GAMMs generally improved the ability to accurately predict observed migratory locations compared to models with eBird occurrence alone. Using three performance metrics, the eBird + LCP model demonstrated equivalent or superior fit relative to the eBird-only model for 22 of 24 species–season GAMMs. In particular, the integrated index filled in spatial gaps for species with over-water movements and those that migrated over land where there were few eBird sightings and, thus, low predictive ability of eBird occurrence probabilities (e.g., Amazonian rainforest in South America). This methodology of combining individual-based seasonal movement data with temporally dynamic species distribution models provides a comprehensive approach to integrating multiple data types to describe broad-scale spatial patterns of animal movement. Further development and customization of this approach will continue to advance knowledge about the full annual cycle and conservation of migratory birds.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Integrating solvation shell structure in experimentally driven molecular dynamics using x-ray solution scattering data

In the past few decades, prediction of macromolecular structures beyond the native conformation has been aided by the development of molecular dynamics (MD) protocols aimed at exploration of the energetic landscape of proteins. Yet, the computed structures do not always agree with experimental observables, calling for further development of the MD strategies to bring the computations and experiments closer together. Here, we report a scalable, efficient MD simulation approach that incorporates an x-ray solution scattering signal as a driving force for the conformational search of stable structural configurations outside of the native basin. We further demonstrate the importance of inclusion of the hydration layer effect for a precise description of the processes involving large changes in the solvent exposed area, such as unfolding. Utilization of the graphics processing unit allows for an efficient all-atom calculation of scattering patterns on-the-fly, even for large biomolecules, resulting in a speed-up of the calculation of the associated driving force. The utility of the methodology is demonstrated on two model protein systems, the structural transition of lysine-, arginine-, ornithine-binding protein and the folding of deca-alanine. We discuss how the present approach will aid in the interpretation of dynamical scattering experiments on protein folding and association.

Hsu, Darren J.↗
DOE OSTI2016

Working Group on Virtual Data Integration

This report is the outcome of a workshop commissioned by the U.S. Department of Energy’s (DOE) Climate and Environmental Sciences Division (CESD) to examine current and future data infrastructure requirements foundational for achieving CESD scientific mission goals in advancing a robust, predictive understanding of Earth’s climate and environmental systems. Over the past several years, data volumes in CESD disciplines have risen sharply to unprecedented levels (tens of petabytes). Moreover, the complexity and diversity of this research data— including simulations, observations, and reanalysis— have grown significantly, posing new challenges for data capture, storage, verification, analysis, and integration. With the trends of increased data volume (in the hundreds of petabytes), more complex analysis processes, and growing cross-disciplinary collaborations, it is timely to investigate whether the CESD community has the computational and data support needed to fully realize the scientific potential of its data collections. In recognition of the challenges, a partnership is forming across CESD and among national and international agencies to examine the viability of creating an integrated, collaborative data infrastructure: a Virtual Laboratory. The overarching goal of this report is to identify the community’s key data technology requirements and high-priority development needs for sustaining and growing its scientific discovery potential. The report also aims to map these requirements to existing solutions and to identify gaps in current services, tools, and infrastructure that will need to be addressed in the short, medium, and long term to advance scientific progress.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2015

Data And Informatics Working Group On Virtual Data Integration Workshop Report

This report is the outcome of a workshop that was commissioned by the Department of Energy’s Climate and Environmental Sciences Division (CESD) to examine current and future data infrastructure requirements that would be foundational to achieving CESD’s scientific mission goals. Over the past several years, data volumes in CESD disciplines have risen sharply to unprecedented levels (tens of petabytes). So too has the complexity and diversity of the research data (simulation, observation, and reanalysis) needing to be captured, stored, verified, analyzed, and integrated. With the trends of increased data volume (in the hundreds of petabytes), more complex analysis processes, and growing crossdisciplinary collaborations, it is timely to investigate whether the CESD community has the right computational and data support to realize the full scientific potential from its data collections. In recognition of the challenges, a partnership is forming across CESD and with national and international agencies to investigate the viability of creating an integrated, collaborative data infrastructure: a virtual laboratory. The overarching goal of this report is to identify the community’s key data technology requirements and high-priority development needs for sustaining and growing their scientific discovery potential. The report also aims to map these requirements to existing solutions and to identify gaps in current services, tools, and infrastructure that will need to be addressed in the short, medium, and long term so as not to impede scientific progress

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Integrating Data Centers and Grid Technologies at Scale

This presentation focuses on the challenge of integrating AI-driven data centers with the power grid at scale. It examines the AI data center capacity challenge and the role of new Medium Voltage Direct Current (MVDC) and other grid-enhancing technologies in enabling efficient and reliable power delivery. The session will highlight the National Laboratory of the Rockies' ARIES capabilities and planning tools, along with collaborative examples involving Verrus, Compass, and Schneider through the Agora test bed for grid-friendly data center evaluations, and ON. Energy for UPS evaluation. It will showcase the NLR Stable Grid Platform for studying oscillations caused by large-scale data centers, along with planning tools to assess grid security and reliability. Additionally, the presentation covers reconductoring strategies to increase grid capacity and explores innovative data center architectures, including the Advanced DC Architectures with Power-electronic Transformers (ADAPT) platform, which enables testing of complete DC architectures for data centers.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

A Data Exchange Interface for a Standards Based Data Integration Platform

The modern electrical grid is a complex and a data rich system, requiring several sub-systems to provide the capability and support needed to perform functions such as distributed energy resource management, outage management, and fault location and isolation, to name a few. There is a need to develop software solutions and platforms that provide an integrated infrastructure to allow interoperability between these subsystems, while leveraging the abundant data available from today’s grid. In this paper, we present the development and deployment of an application service to integrate GridAPPS-D which is an open source, standards-based software platform with SurvalentONE, a commonly used DNP3 based ADMS platform that allows operation, monitoring, analysis, restoration, and optimization of network operations. We present details of the integration architecture, including its implementation, and provide results from functional testing of the architecture on a 13 bus IEEE system. The results validate that the integration is successful and is able to provide a two-way exchange of communication and translation between different communication protocols.

Singh, Alka↗
DOE OSTI2019

GenomeGraphR: A user-friendly open-source web application for foodborne pathogen whole genome sequencing data integration, analysis, and visualization

Food safety risk assessments and large-scale epidemiological investigations have the potential to provide better and new types of information when whole genome sequence (WGS) data are effectively integrated. Today, the NCBI Pathogen Detection database WGS collections have grown significantly through improvements in technology, coordination, and collaboration, such as the GenomeTrakr and PulseNet networks. However, high-quality genomic data is not often coupled with high-quality epidemiological or food chain metadata. We have created a set of tools for cleaning, curation, integration, analysis and visualization of microbial genome sequencing data. It has been tested using Salmonella enterica and Listeria monocytogenes data sets provided by NCBI Pathogen Detection (160,000 sequenced isolates in 2018). GenomeGraphR presents foodborne pathogen WGS data and associated curated metadata in a user-friendly interface that allows a user to query a variety of research questions such as, transmission sources and dynamics, global reach, and persistence of genotypes associated with contamination in the food supply and foodborne illness across time or space. The application is freely available (https://fda-riskmodels.foodrisk.org/ genomegraphr/).

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

Learning-Based Inversion-Free Model-Data Integration to Advance Ecosystem Model Prediction

Ecosystem model prediction is important for understanding ecosystem responses to climate change and for management support. Model prediction and quantification of predictive uncertainty of ecosystems have long been investigated. The traditional workflow, which calibrates models to match observations and then uses the calibrated models for predictions, relies heavily on inverse modeling to constrain uncertain parameters in complex forward models. This inversion-based prediction approach is infeasible for complex models with heterogeneous parameter uncertainties and incapable of rapid integration of streaming and multiple sources of data because of the difficulty and computational cost in the model inversion, which is typically ill-posed and can require hundreds of thousands of expensive forward simulations to be performed iteratively. We propose to circumvent inverse modeling by precomputing an ensemble of unconstrained forward simulations and then using machine learning (ML) methods to learn the statistical relationship between simulated observation and prediction quantities. Once the ML model has learned the relationship, it can be used to make predictions of future system behavior with uncertainty quantification based on observations. The proposed learningbased inversion-free model prediction (LIMP) framework is computationally efficient which only requires a few thousands of fully parallelizable forward simulations. Additionally, LIMP can continually update predictions based on streaming observations from multiple locations and sources without necessarily requiring extra model simulations. In this study, we apply LIMP to a regional terrestrial ecosystem model with 47 parameters for testing, refining, and evaluating the approach. We demonstrate that LIMP can be used for efficient model prediction, rapid data assimilation, and cost-effective experimental design for improving robust predictive understanding of ecosystems.

Lu, Dan↗
DOE OSTI2017

Energy Systems Integration: Data Call -- Become a Data Partner

This project aims to advance the understanding of costs associated with integrating PV onto the electric power distribution system while maintaining reliable grid operations. We have developed a bottom-up framework for calculating these costs as a function of PV penetration levels on specific feeders. This framework will used to inform and improve utility planning decisions, increase the transparency and speed associated with the interconnection process, and provide policymakers with more information on the total cost of energy from PV.

14 SOLAR ENERGY↗
DOE OSTI2017

Tandem Swift and INTEGRAL Data to Revisit the Orbital and Superorbital Periods of 1E 1740.7–2942

The black hole candidate 1E 1740.7−2942 is one of the strongest hard X-ray sources in the Galactic Center region. No counterparts in longer wavelengths have been identified for this object yet. The presence of characteristic timing signatures in the flux history of X-ray sources has been shown to be an important diagnostic tool for the properties of these systems. Using simultaneous data from NASA’s Swift and ESA’s INTEGRAL missions, we have found two periodic signatures at 12.61 ± 0.06 days and 171.1 ± 3.0 days in long-term hard X-ray light curves of 1E 1740.7−2942. We interpret those as the orbital and superorbital periods of the object, respectively. The reported orbital period is in good agreement with previous studies of 1E 1740.7−2942 using NASA’s RXTE data. We present here the first firm evidence of a superorbital period for 1E 1740.7−2942, which has important implications for the nature of the binary system.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

IMG/M v.5.0: an integrated data management and comparative analysis system for microbial genomes and microbiomes

The Integrated Microbial Genomes & Microbiomes system v.5.0 (IMG/M: https://img.jgi.doe.gov/m/) contains annotated datasets categorized into: archaea, bacteria, eukarya, plasmids, viruses, genome fragments, metagenomes, cell enrichments, single particle sorts, and metatranscriptomes. Source datasets include those generated by the DOE's Joint Genome Institute (JGI), submitted by external scientists, or collected from public sequence data archives such as NCBI. All submissions are typically processed through the IMG annotation pipeline and then loaded into the IMG data warehouse. IMG's web user interface provides a variety of analytical and visualization tools for comparative analysis of isolate genomes and metagenomes in IMG. IMG/M allows open access to all public genomes in the IMG data warehouse, while its expert review (ER) system (IMG/MER: https://img.jgi.doe.gov/mer/) enabled registered users to access their private genomes and to store their private datasets in workspace for sharing and for further analysis. IMG/M data content has increased by 60% since the last report published in the 2017 NAR Database Issue. IMG/M v.5.0 has a new and more powerful genome search feature, new statistical tools, and supports metagenome binning.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

Streamline-Based Time-Lapse-Seismic-Data Integration Incorporating Pressure and Saturation Effects

In this paper, we present an efficient history-matching technique that simultaneously integrates 4D repeat seismic surveys with well-production data. This approach is particularly well-suited for the calibration of the reservoir properties of high-resolution geologic models because the seismic data are a really dense but sparse in time, whereas the production data are finely sampled in time but spatially averaged. The joint history matching is performed by use of streamline-based sensitivities derived from either finite-difference or streamline-based flow simulation. For the most part, earlier approaches have focused on the role of saturation changes, but the effects of pressure have largely been ignored. Here, we present a streamline-based semianalytic approach for computing model-parameter sensitivities, accounting for both pressure and saturation effects. The novelty of the method lies in the semianalytic sensitivity computations, making it computationally efficient for high-resolution geologic models. The approach is implemented by use of a finite-difference simulator incorporating the detailed physics. Its efficacy is demonstrated by use of both synthetic and field applications. Lastly, for both the synthetic and the field cases, the advantages of incorporating the time-lapse variations are clear, seen through the improved estimation of the permeability distribution, the pressure profile, the evolution of the fluid saturation, and the swept volumes.

58 GEOSCIENCES↗
DOE OSTI2024

A Framework to Demonstrate a DNP3 Interface With a CIM-Based Data Integration Platform: Preprint

The contemporary electrical grid is characterized by its complexity and abundance of data. A control-rich environment supported by information and communication technologies within an Advanced Distribution Management System (ADMS) presents a viable and cost-effective option for utility companies aiming to implement advanced real-time analytical schemes for monitoring and remotely controlling distribution feeders. Modular platform-based approaches to distribution operations require a structured framework for acquiring field device measurements, performing analytics, converting the setpoint to the correct protocol, and sending it on the appropriate communications network to the field devices. We present the development and deployment of an application service to integrate an open-source standardsbased platform with an ADMS test bed with field devices using the Distributed Network Protocol (DNP3) for data exchange. The step-by-step procedure for establishing the DNP3-Master service on an open-source distribution platform is outlined, comprehensively explaining the Master setup process. Moreover, sample use case results highlight the capabilities of the DNP3- Master service setup. Results demonstrate the scalability and configurability of the DNP3-Master service, making it adaptable for integration with other relevant applications, thus providing potential opportunities for real-world field trials and real-time assessments.

ADMS↗
DOE OSTI2021

Abridged spectral matrix inversion: parametric fitting of X-ray fluorescence spectra following integrative data reduction

Recent improvements in both X-ray detectors and readout speeds have led to a substantial increase in the volume of X-ray fluorescence data being produced at synchrotron facilities. This in turn results in increased challenges associated with processing and fitting such data, both temporally and computationally. Herein an abridging approach is described that both reduces and partially integrates X-ray fluorescence (XRF) data sets to obtain a fivefold total improvement in processing time with negligible decrease in quality of fitting. The approach is demonstrated using linear least-squares matrix inversion on XRF data with strongly overlapping fluorescent peaks. This approach is applicable to any type of linear algebra based fitting algorithm to fit spectra containing overlapping signals wherein the spectra also contain unimportant (non-characteristic) regions which add little (or no) weight to fitted values, e.g. energy regions in XRF spectra that contain little or no peak information.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2016

The Monarch Initiative: an integrative data and analytic platform connecting phenotypes to genotypes across species

The correlation of phenotypic outcomes with genetic variation and environmental factors is a core pursuit in biology and biomedicine. Numerous challenges impede our progress: patient phenotypes may not match known diseases, candidate variants may be in genes that have not been characterized, model organisms may not recapitulate human or veterinary diseases, filling evolutionary gaps is difficult, and many resources must be queried to find potentially significant genotype-phenotype associations. Nonhuman organisms have proven instrumental in revealing biological mechanisms. Advanced informatics tools can identify phenotypically relevant disease models in research and diagnostic contexts. Large-scale integration of model organism and clinical research data can provide a breadth of knowledge not available from individual sources and can provide contextualization of data back to these sources. The Monarch Initiative (monarchinitiative.org) is a collaborative, open science effort that aims to semantically integrate genotype-phenotype data from many species and sources in order to support precision medicine, disease modeling, and mechanistic exploration. Our integrated knowledge graph, analytic tools, and web services enable diverse users to explore relationships between phenotypes and genotypes across species.

59 BASIC BIOLOGICAL SCIENCES↗