Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integration.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,423 records · Page 79
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

BraggNet: integrating Bragg peaks using neural networks

Neutron crystallography offers enormous potential to complement structures from X-ray crystallography by clarifying the positions of low-Z elements, namely hydrogen. Macromolecular neutron crystallography, however, remains limited, in part owing to the challenge of integrating peak shapes from pulsed-source experiments. To advance existing software, this article demonstrates the use of machine learning to refine peak locations, predict peak shapes and yield more accurate integrated intensities when applied to whole data sets from a protein crystal. The artificial neural network, based on the U-Net architecture commonly used for image segmentation, is trained using about 100 000 simulated training peaks derived from strong peaks. After 100 training epochs (a round of training over the whole data set broken into smaller batches), training converges and achieves a Dice coefficient of around 65%, in contrast to just 15% for negative control data sets. Integrating whole peak sets using the neural network yields improved intensity statistics compared with other integration methods, including k-nearest neighbours. These results demonstrate, for the first time, that neural networks can learn peak shapes and be used to integrate Bragg peaks. In conclusion, it is expected that integration using neural networks can be further developed to increase the quality of neutron, electron and X-ray crystallography data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Modeling Information Multiplexing in the Hippocampus

Modern computers are constantly faced with the challenge of processing ever-growing quantities of data that span a wide range of modalities. Of particular relevance to national security interests is the ability to integrate multimodal data for the purpose of fast decision making. The brain is a biological system that is specialized for high performance at this task, suggesting that understanding the mechanisms by which neural circuits integrate multimodal data may lead to improved man-made detection systems. This research focused on understanding these neural algorithms, and specifically tested the hypothesis that hippocampal neurons multiplex information from two different input streams. Specifically, we compare the spiking behavior of a computational model of hippocampal circuitry with neurophysiological data recorded from rodent hippocampus.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Measurement of integrated luminosities at BESIII for data samples at center-of-mass energies between 4.0 and 4.6 GeV*

Abstract The integrated luminosities of data samples collected in the BESIII experiment in 2016–2017 at center-of-mass energies between 4.19 and 4.28 GeV are measured with a precision better than 1% by analyzing large-angle Bhabha scattering events. The integrated luminosities of old datasets collected in 2010–2014 are updated by considering corrections related to detector performance, offsetting the effect of newly discovered readout errors in the electromagnetic calorimeter, which can haphazardly occur.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2013

LANL CSSE L2: Case Study of In Situ Data Analysis in ASC Integrated Codes [Slides]

The massive, complex datasets generated by scientific simulations on next-generation computer architectures at extreme scale pose new challenges for data management and scientific understanding. A critical element in meeting these new challenges is in situ data analysis where information is reduced and analyzed as close to the point of generation as possible, for example, at run time or as the data stream to longer term storage. This milestone will demonstrate in situ data analysis in xRAGE, an ASC Eulerian Radiation Hydrodynamics code. We will work with the Eulerian Applications Project to identify target problems that will exercise physics of interest to the program, and we will work closely with users to maximize the achieved usability and productivity improvements. We will investigate enhancements to the in situ process that enable and improve techniques of data reduction and data triage in addition to improving in situ performance by using portable acceleration technology. We will report on their viability and performance in xRAGE or another ASC code. These products will provide ASC IC developers with reference codes and tools that will benefit DSW by advancing the technology for predictive science at extreme scales.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2014

Recent Advances in Registration, Integration and Fusion of Remotely Sensed Data: Redundant Representations and Frames

In recent years, sophisticated mathematical techniques have been successfully applied to the field of remote sensing to produce significant advances in applications such as registration, integration and fusion of remotely sensed data. Registration, integration and fusion of multiple source imagery are the most important issues when dealing with Earth Science remote sensing data where information from multiple sensors, exhibiting various resolutions, must be integrated. Issues ranging from different sensor geometries, different spectral responses, differing illumination conditions, different seasons, and various amounts of noise need to be dealt with when designing an image registration, integration or fusion method. This tutorial will first define the problems and challenges associated with these applications and then will review some mathematical techniques that have been successfully utilized to solve them. In particular, we will cover topics on geometric multiscale representations, redundant representations and fusion frames, graph operators, diffusion wavelets, as well as spatial-spectral and operator-based data fusion. All the algorithms will be illustrated using remotely sensed data, with an emphasis on current and operational instruments.

Fusion↗
DOE OSTI2025

miss-SNF: a multimodal patient similarity network integration approach to handle completely missing data sources

Abstract Motivation Precision medicine leverages patient-specific multimodal data to improve prevention, diagnosis, prognosis, and treatment of diseases. Advancing precision medicine requires the non-trivial integration of complex, heterogeneous, and potentially high-dimensional data sources, such as multi-omics and clinical data. In the literature, several approaches have been proposed to manage missing data, but are usually limited to the recovery of subsets of features for a subset of patients. A largely overlooked problem is the integration of multiple sources of data when one or more of them are completely missing for a subset of patients, a relatively common condition in clinical practice. Results We propose miss-Similarity Network Fusion (miss-SNF), a novel general-purpose data integration approach designed to manage completely missing data in the context of patient similarity networks. miss-SNF integrates incomplete unimodal patient similarity networks by leveraging a non-linear message-passing strategy borrowed from the SNF algorithm. miss-SNF is able to recover missing patient similarities and is “task agnostic”, in the sense that can integrate partial data for both unsupervised and supervised prediction tasks. Experimental analyses on nine cancer datasets from The Cancer Genome Atlas (TCGA) demonstrate that miss-SNF achieves state-of-the-art results in recovering similarities and in identifying patients subgroups enriched in clinically relevant variables and having differential survival. Moreover, amputation experiments show that miss-SNF supervised prediction of cancer clinical outcomes and Alzheimer’s disease diagnosis with completely missing data achieves results comparable to those obtained when all the data are available. Availability and implementation miss-SNF code, implemented in R, is available at https://github.com/AnacletoLAB/missSNF.

Biochemistry & Molecular Biology↗
DOE OSTI2022

Perspectives on the integration between first-principles and data-driven modeling

Efficiently embedding and/or integrating mechanistic information with data-driven models is essential if it is desired to simultaneously take advantage of both engineering principles and data-science. Further the opportunity for hybridization occurs in many scenarios, such as the development of a faster model of an accurate high-fidelity computer model; the correction of a mechanistic model that does not fully-capture the physical phenomena of the system; or the integration of a data-driven component approximating an unknown correlation within a mechanistic model. At the same time, different techniques have been proposed and applied in different literatures to achieve this hybridization, such as hybrid modeling, physics-informed Machine Learning (ML) and model calibration. In this paper we review the methods, challenges, applications and algorithms of these three research areas and discuss them in the context of the different hybridization scenarios. Moreover, we provide a comprehensive comparison of the hybridization techniques with respect to their differences and similarities, as well as advantages and limitations and future perspectives. Finally, we apply and illustrate hybrid modeling, physics-informed ML and model calibration via a chemical reactor case study.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2014

Modeling the Energy Use of a Connected and Automated Transportation System (Poster)

Early research points to large potential impacts of connected and automated vehicles (CAVs) on transportation energy use - dramatic savings, increased use, or anything in between. Due to a lack of suitable data and integrated modeling tools to explore these complex future systems, analyses to date have relied on simple combinations of isolated effects. This poster proposes a framework for modeling the potential energy implications from increasing penetration of CAV technologies and for assessing technology and policy options to steer them toward favorable energy outcomes. Current CAV modeling challenges include estimating behavior change, understanding potential vehicle-to-vehicle interactions, and assessing traffic flow and vehicle use under different automation scenarios. To bridge these gaps and develop a picture of potential future automated systems, NREL is integrating existing modeling capabilities with additional tools and data inputs to create a more fully integrated CAV assessment toolkit.

30 DIRECT ENERGY CONVERSION↗
NASA NTRS1999

Forced and Moment Measurements with Pressure-Sensitive Paint

The potential of pressure-sensitive paint (PSP) to provide aerodynamic loads measurements has been a driving force behind the development of this measurement technique. To demonstrate the suitability of PSP for this purpose, it is necessary to show that PSP-derived pressures can be accurately integrated over the model surface. This cannot be done simply by demonstrating the accuracy of PSP as compared to pressure taps. PSP errors due to misregistration or temperature sensitivity may be high near model edges, where they will have a strong effect on moment measurements, but where pressure taps are rarely installed. A more suitable technique is to compare integrated PSP data over the entire model surface with balance and/or CFD results. This paper presents results from three experiments in which integrated PSP data is compared with balance and/or CFD data. This allows the usefulness of PSP for force and moment measurements, and by implication for loads measurements, to be assessed.

Bell, James H.↗
DOE OSTI2012

Visual Data Analysis as an Integral Part of Environmental Management

The U.S. Department of Energy's (DOE) Office of Environmental Management (DOE/EM) currently supports an effort to understand and predict the fate of nuclear contaminants and their transport in natural and engineered systems. Geologists, hydrologists, physicists and computer scientists are working together to create models of existing nuclear waste sites, to simulate their behavior and to extrapolate it into the future. We use visualization as an integral part in each step of this process. In the first step, visualization is used to verify model setup and to estimate critical parameters. High-performance computing simulations of contaminant transport produces massive amounts of data, which is then analyzed using visualization software specifically designed for parallel processing of large amounts of structured and unstructured data. Finally, simulation results are validated by comparing simulation results to measured current and historical field data. We describe in this article how visual analysis is used as an integral part of the decision-making process in the planning of ongoing and future treatment options for the contaminated nuclear waste sites. Lessons learned from visually analyzing our large-scale simulation runs will also have an impact on deciding on treatment measures for other contaminated sites.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

New Architecture to Support Integration and Processing of Seismic Data from Heterogeneous Sources

The Geophysical Monitoring Program (GMP) at Lawrence Livermore National Lab (LLNL) maintains a database and supporting infrastructure for geophysical data used in support of the Nuclear Detonation Detection mission. This database includes data from multiple sources, many of which do not distribute data to the public or for which there is no automated means of access. For example, Figure 1 shows (left) the distribution of waveform data in our database by source. The Incorporated Research Institutions for Seismology Data Management Center (IRISDMC) is our major source of waveform data and those data may be retrieved at will using the Federated Digital Seismograph Networks FDSN web Application Programming Interface (API). However, the next 6 most important sources of waveform data have no or only limited automated access to waveforms. As Figure 1 (right) shows, it is very common for waveform records associate with an event in our database to come from two or more sources, and in some cases data come from 10 sources. This diversity of data sources drives our need for efficient and correct integration of metadata, parametric data, and waveform data.

58 GEOSCIENCES↗
DOE OSTI2013

Additions to the ICSBEP and IRPhEP Handbooks since NCSD 2009

High-quality integral benchmark experiments have always been a priority for criticality safety. However, interest in integral benchmark data is increasing as efforts to quantify and reduce calculational uncertainties accelerate to meet the demands of future criticality safety needs to support next generation reactor and advanced fuel cycle concepts. The importance of drawing upon existing benchmark data is becoming more apparent because of dwindling availability of critical facilities worldwide and the high cost of performing new experiments. Integral benchmark data from the International Handbook of Evaluated Criticality Safety Benchmark Experiments and the International Handbook of Reactor Physics Benchmark Experiments are widely used. Significant Benchmark data have been added to those two handbooks since the last Nuclear Criticality Safety Division Topical Meeting in Richland, Washington (September 2009). This paper highlights those additions.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Seismicity-constrained fault detection and characterization with a multitask machine learning model

Geological fault detection and characterization are crucial for understanding subsurface dynamics across scales. While methods for fault delineation based on either seismicity location analysis or seismic image reflector discontinuity are well-established, a systematic approach that integrates both data types remains absent. We develop a novel machine learning model that unifies seismic reflector images and seismicity location information to automatically identify geological faults and characterize their geometrical properties. The model encodes a seismic image and a seismicity location image separately, and fuses the encoded features with a spatial-channel attention fusion module to improve the learning of important features in both inputs. We design an automated strategy to generate high-quality synthetic training data and labels. To improve the realism of the seismicity location image, we include random seismicity noise and missing seismicity location associated with some of the faults. We validate the model’s efficacy and accuracy using synthetic data examples and two field data examples. Moreover, we show that fine-tuning the trained model with a small, domain-specific dataset enhances its fidelity for field data applications. The results demonstrate that integrating seismicity location and seismic images into a unified framework allows the end-to-end neural network to achieve higher fidelity and accuracy in delineating subsurface faults and their geometrical properties compared with image-only fault detection methods. Our approach offers an adaptive data-driven tool for geological fault characterization and seismic hazard mitigation, bridging the gap between seismicity location and image-based fault detection methods.

58 GEOSCIENCES↗
DOE OSTI2023

Uncovering Where Compensating Errors Could Hide in ENDF/B-VIII.0

Unconstrained physics spaces between two or more nuclear data observables in a library occur when their values can be simultaneously adjusted without violating the uncertainties in either differential information or simulations of relevant integral experiments. Differential data are often too imprecise to fully bound all nuclear data observables of interest for application simulations. Integral data are simulated with combinations of nuclear data so that an error in one observable may be hidden by a counterbalancing error in another. In this manner compensating errors may lurk within nuclear data libraries and these errors have the potential to undermine the predictive power of neutron transport simulations, particularly in situations where there is no conclusive validation experiment that resembles the application of interest. The EUCLID project (Experiments Underpinned by Computational Learning for Improvements in Nuclear Data) developed a preliminary workflow to identify these unconstrained physics spaces by bringing together results from a large collection of integral experiments with their simulated counter-parts as well as differential information that have a one-to-one correspondence to nuclear data. This wealth of information is processed by machine learning tools for subsequent refinement by human experts. Here, we show how the EUCLID work-flow is executed by applying it first to 239 Pu and then to 9 Be nuclear data in ENDF/B-VIII.0.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2018

A Surveillance Infrastructure for Malaria Analytics: Provisioning Data Access and Preservation of Interoperability

Background: According to the World Health Organization, malaria surveillance is weakest in countries and regions with the highest malaria burden. A core obstacle is that the data required to perform malaria surveillance are fragmented in multiple data silos distributed across geographic regions. Furthermore, consistent integrated malaria data sources are few, and a low degree of interoperability exists between them. As a result, it is difficult to identify disease trends and to plan for effective interventions. Objective: We propose the Semantics, Interoperability, and Evolution for Malaria Analytics (SIEMA) platform for use in malaria surveillance based on semantic data federation. Using this approach, it is possible to access distributed data, extend and preserve interoperability between multiple dynamic distributed malaria sources, and facilitate detection of system changes that can interrupt mission-critical global surveillance activities. Methods: We used Semantic Automated Discovery and Integration (SADI) Semantic Web Services to enable data access and improve interoperability, and the graphical user interface-enabled semantic query engine HYDRA to implement the target queries typical of malaria programs. We implemented a custom algorithm to detect changes to community-developed terminologies, data sources, and services that are core to SIEMA. This algorithm reports to a dashboard. Valet SADI is used to mitigate the impact of changes by rebuilding affected services. Results: We developed a prototype surveillance and change management platform from a combination of third-party tools, community-developed terminologies, and custom algorithms. We illustrated a methodology and core infrastructure to facilitate interoperable access to distributed data sources using SADI Semantic Web services. This degree of access makes it possible to implement complex queries needed by our user community with minimal technical skill. We implemented a dashboard that reports on terminology changes that can render the services inactive, jeopardizing system interoperability. Using this information, end users can control and reactively rebuild services to preserve interoperability and minimize service downtime. Conclusions: We introduce a framework suitable for use in malaria surveillance that supports the creation of flexible surveillance queries across distributed data resources. The platform provides interoperable access to target data sources, is domain agnostic, and with updates to core terminological resources is readily transferable to other surveillance activities. A dashboard enables users to review changes to the infrastructure and invoke system updates. The platform significantly extends the range of functionalities offered by malaria information systems, beyond the state-of-the-art.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Bayesian optimization of generalized data

Direct application of Bayes' theorem to generalized data yields a posterior probability distribution function (PDF) that is a product of a prior PDF of generalized data and a likelihood function, where generalized data consists of model parameters, measured data, and model defect data. The prior PDF of generalized data is defined by prior expectation values and a prior covariance matrix of generalized data that naturally includes covariance between any two components of generalized data. A set of constraints imposed on the posterior expectation values and covariances of generalized data via a given model is formally solved by the method of Lagrange multipliers. Posterior expectation values of the constraints and their covariance matrix are conventionally set to zero, leading to a likelihood function that is a Dirac delta function of the constraining equation. It is shown that setting constraints to values other than zero is analogous to introducing a model defect. Since posterior expectation values of any function of generalized data are integrals of that function over all generalized data weighted by the posterior PDF, all elements of generalized data may be viewed as nuisance parameters marginalized by this integration. One simple form of posterior PDF is obtained when the prior PDF and the likelihood function are normal PDFs. For linear models without a defect this PDF becomes equivalent to constrained least squares (CLS) method, that is, the χ2 minimization method.

Arbanas, Goran↗
DOE OSTI2023

In-depth analysis on parallel processing patterns for high-performance Dataframes

The Data Science domain has expanded monumentally in both research and industry communities during the past decade, predominantly owing to the Big Data revolution. Artificial Intelligence (AI) and Machine Learning (ML) are bringing more complexities to data engineering applications, which are now integrated into data processing pipelines to process terabytes of data. Typically, a significant amount of time is spent on data preprocessing in these pipelines, and hence improving its efficiency directly impacts the overall pipeline performance. The community has recently embraced the concept of Dataframes as the de-facto data structure for data representation and manipulation. However, the most widely used serial Dataframes today (R, pandas) experience performance limitations while working on even moderately large data sets. We believe that there is plenty of room for improvement by taking a look at this problem from a high-performance computing point of view. In a prior publication, we presented a set of parallel processing patterns for distributed dataframe operators and the reference runtime implementation, Cylon. In this paper, we are expanding on the initial concept by introducing a cost model for evaluating the said patterns. Furthermore, we evaluate the performance of Cylon on the ORNL Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗