Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integration.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Interannual variability of ecosystem carbon exchange: From observation to prediction

Here, terrestrial ecosystems have sequestered, on average, the equivalent of 30% of anthropogenic carbon (C) emissions during the past decades, but annual sequestration varies from year to year. For effective C management, it is imperative to develop a predictive understanding of the interannual variability (IAV) of terrestrial net ecosystem C exchange (NEE). Location: Global terrestrial ecosystems. Furthermore, we conducted a comprehensive review to examine the IAV of NEE at global, regional and ecosystem scales. Then we outlined a conceptual framework for understanding how anomalies in climate factors impact ecological processes of C cycling and thus influence the IAV of NEE through biogeochemical regulation. As a result, the phenomenon of IAV in land NEE has been ubiquitously observed at global, regional and ecosystem scales. Global IAV is often attributable to either tropical or semi-arid regions, or to some combination thereof, which is still under debate. Previous studies focus on identifying climate factors as driving forces of IAV, whereas biological mechanisms underlying the IAV of ecosystem NEE are less clear. We found that climate anomalies affect the IAV of NEE primarily through their differential impacts on ecosystem C uptake and respiration. Moreover, recent studies suggest that the carbon uptake period makes less contribution than the carbon uptake amplitude to IAV in NEE. Although land models incorporate most processes underlying IAV, their efficacy to predict the IAV in NEE remains low. In conclusion, to improve our ability to predict future IAV of the terrestrial C cycle, we have to understand biological mechanisms through which anomalies in climate factors cause the IAV of NEE. Future research needs to pay more attention not only to the differential effects of climate anomalies on photosynthesis and respiration but also to the relative importance of the C uptake period and amplitude in causing the IAV of NEE. Ultimately, we need multiple independent approaches, such as benchmark analysis, data assimilation and time-series statistics, to integrate data, modelling frameworks and theory to improve our ability to predict future IAV in the terrestrial C cycle.

54 ENVIRONMENTAL SCIENCES↗

Tools for Visualization and Analysis of Small-Angle Neutron Scattering Data: Descriptions and Examples

A great deal of progress has been made in improving the data reduction experience for the SANS instruments at the SNS and HFIR at ORNL. The existing data reduction toolset, drtsans, makes it possible to integrate data analysis and visualization tools into the data reduction scripts, thereby providing new opportunities for more automated data processing for users of the SNS and HFIR. Here, the first set of tools developed is described with usage examples.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Integrating Multi-Source Data for Bi-Level Traffic Simulator Calibration: A Literature Review and Highway Case Study

Traffic simulation serves as a powerful tool for pre-evaluating policies and technologies. In this context, simulation-based Dynamic traffic assignment (DTA) models are capable of capturing traffic dynamics. They are well-known as critical tools in controlling and predicting traffic situations. The reliability of simulation results heavily depends on the calibration process. Most studies in the literature formulate and calibrate simulators based on a single source of collected data or multiple data sets with the same spatiotemporal characteristics. However, in practice, traffic data is collected by various tools with usually different spatial and temporal resolutions. This study introduces a novel approach to taking into account diverse input data from a variety of sources. An iterative bi-level solution is proposed. to equally treat traffic flow and speed data. The upper level solves flow calibration with the exact solution method, and the lower level calibrates the speed with the simultaneous perturbation stochastic approximation (SPSA) algorithm. Subsequently, the effectiveness of the proposed model is investigated using data from a six-mile section of Nashville's I-24 highway in Tennessee. The results demonstrate that our proposed model creates an effective feedback loop between the optimizer and the simulator for calibrating flow and speed to reduce the error between simulated and real data.

42 ENGINEERING↗

Integration of a Self-Coherence Algorithm into DISAT for Forced Oscillation Detection

With the increasing number of phasor measurement units on the power system, behaviors typically not observable on the power system are becoming more apparent. Oscillatory behavior on the power system, notably forced oscillations, are one such behavior. However, the large amounts of data coming from the PMUs makes manually detecting and locating these oscillations difficult. To automate portions of the process, an oscillation detection routine was coded into the Data Integrity and Situational Awareness Tool (DISAT) framework. Integration into the DISAT framework allows forced oscillations to be detected and information about the event provided to operational engineers. The oscillation detection algorithm integrates with the data handling and atypical data detecting capabilities of DISAT, building off of a standard library of functions. This report details that integration with information on the algorithm, some implementation issues, and some sample results from the western United States’ power grid.

Forced oscillations↗

Online and Offline Data Quality Monitoring for the Mu2e Calorimeter

This thesis presents the design, implementation, and validation of a calorimeter Data Quality Monitoring (DQM) toolchain for the Mu2e experiment at Fermilab. Mu2e searches for charged lepton flavor violation via coherent muon-to-electron conversion in the field of an aluminum nucleus, $\mu^- Al \rightarrow e^-Al$, a process whose observation would constitute clear evidence of physics beyond the Standard Model. Achieving target sensitivity requires stringent control of detector performance and data integrity during acquisition, as subtle issues in readout configuration, data formatting, or electronics behavior can compromise reconstruction and bias downstream analyzes. To address these challenges, this work develops a multi-layer DQM approach spanning both raw data validation and reconstructed digi-level diagnostics. At the low level, a fragment analysis component performs word- and bit-field decoding of calorimeter readout blocks, enabling sanity checks of the expected structure and producing detailed error and integrity statistics useful for commissioning and troubleshooting. At the digi level, the CaloDigiDQM analyzer is implemented within the art framework and transforms each CaloDigiCollection into a structured hierarchy of ROOT histograms designed for fast drill-down diagnostics. The module generates coherent monitoring views at global, disk, board, and channel granularity, including occupancy, waveform-derived features (baseline, RMS, peak amplitude and position), and left-right sensor consistency metrics. Detector-aware channel-to-electronics mapping is performed through the conditions system (CaloDAQMap), ensuring that diagnostics remain aligned with hardware identifiers used in operations. For end-to-end testing without reliance on live DAQ data, a synthetic CaloDigi producer is developed to generate realistic waveforms with controlled noise and pulse shapes. The resulting system supports both offline ROOT-file production and online operation, including optional histogram streaming through otsdaq via ots::HistoSender. This toolchain provides a practical and scalable foundation for calorimeter commissioning and stable data collection, enabling early detection of anomalies and reducing operational risk for Mu2e.

Vakulenko, Mark [Drew U.] (ORCID:0009000276197818)↗

Managing Data on Tiered Storage for Scientific Workflows (MaDaTS) v1.1.2

MaDaTS provides an integrated data management and workflow execution framework on multi-tiered storage systems. Users of MaDaTS can execute a workflow by either specifying the workflow stages in a YAML description file, or use the API to manage the workflow through a Virtual Data Space (VDS) abstraction.

Ramakrishnan, Lavanya↗

Spectral state dependence of the 0.4–2 MeV polarized emission in Cygnus X-1 seen with INTEGRAL/IBIS, and links with the AMI radio data

Polarization of the ≳400 keV hard tail of the microquasar Cygnus X-1 has been independently reported by INTEGRAL/Imager on Board the INTEGRAL Satellite (IBIS), and INTEGRAL/SPectrometer on INTEGRAL and interpreted as emission from a compact jet. These conclusions were, however, based on the accumulation of all INTEGRAL data regardless of the spectral state. We utilize additional INTEGRAL exposure accumulated until 2012 December, and include the AMI/Ryle (15 GHz) radio data in our study. We separate the observations into hard, soft, and intermediate/transitional states and detect radio emission from a compact jet in hard and intermediate states (IS), but not in the soft. The 10–400 keV INTEGRAL (JEM-X and IBIS) state resolved spectra are well modeled with thermal Comptonization and reflection components. We detect a hard tail in the 0.4–2 MeV range for the hard state only. We extract the state dependent polarigrams of Cyg X-1, which are all compatible with no or an undetectable level of polarization except in the 400–2000 keV range in the hard state where the polarization fraction is 75% ± 32% and the polarization angle 40.°0 ± 14.°3. An upper limit on the 0.4–2 MeV soft state polarization fraction is 70%. Due to the short exposure, we obtain no meaningful constraint for the IS. The likely detection of a >400 keV polarized tail in the hard state, together with the simultaneous presence of a radio jet, reinforce the notion of a compact jet origin of the >400 keV emission.

79 ASTRONOMY AND ASTROPHYSICS↗

The MG-RAST Metagenomics Database and Portal in 2015

MG-RAST (http://metagenomics.anl.gov) is an opensubmission data portal for processing, analyzing, sharing and disseminating metagenomic datasets. Currently, the system hosts over 200 000 datasets and is continuously updated. The volume of submissions has increased 4-fold over the past 24 months, now averaging 4 terabasepairs per month. In addition to several new features, we report changes to the analysis workflow and the technologies used to scale the pipeline up to the required throughput levels. Lastly, to show possible uses for the data from MG-RAST, we present several examples integrating data and analyses from MG-RAST into popular third-party analysis tools or sequence alignment tools.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Automated pipeline processing X-ray diffraction data from dynamic compression experiments on the Extreme Conditions Beamline of PETRA III

Presented and discussed here is the implementation of a software solution that provides prompt X-ray diffraction data analysis during fast dynamic compression experiments conducted within the dynamic diamond anvil cell technique. It includes efficient data collection, streaming of data and metadata to a high-performance cluster (HPC), fast azimuthal data integration on the cluster, and tools for controlling the data processing steps and visualizing the data using the DIOPTAS software package. This data processing pipeline is invaluable for a great number of studies. The potential of the pipeline is illustrated with two examples of data collected on ammonia–water mixtures and multiphase mineral assemblies under high pressure. The pipeline is designed to be generic in nature and could be readily adapted to provide rapid feedback for many other X-ray diffraction techniques, e.g. large-volume press studies, in situ stress/strain studies, phase transformation studies, chemical reactions studied with high-resolution diffraction etc.

97 MATHEMATICS AND COMPUTING↗

A Data Quality Filter for PMU Measurements: Description, Experience, and Examples

Networks of phasor measurement units (PMUs) continue to grow, and along with them, the amount of data available for analysis. With so much data, it is impractical to identify and remove poor quality data manually. The data quality filter described in this paper was developed for use with the Data Integrity and Situation Awareness Tool (DISAT), which analyzes PMU data to identify anomalous system behavior. The filter operates based only on the information included in the data files, without supervisory control and data acquisition (SCADA) data, state estimator values, or system topology information. Measurements are compared to preselected thresholds to determine if they are reliable. Along with the filter's description, examples of data quality issues from application of the filter to nine months of archived PMU data are provided. The paper is intended to aid the reader in recognizing and properly addressing data quality issues in PMU data.

Phasor Measurement Unit↗

Implementing efficient data compression and encryption in a persistent key-value store for HPC

Recently, persistent data structures, like key-value stores (KVSs), which are stored in a high-performance computing (HPC) system’s nonvolatile memory, provide an attractive solution for a number of emerging challenges like limited I/O performance. Data compression and encryption are two well-known techniques for improving several properties of such data-oriented systems. This article investigates how to efficiently integrate data compression and encryption into persistent KVSs for HPC with the ultimate goal of hiding their costs and complexity in terms of performance and ease of use. Our compression technique exploits deep memory hierarchy in an HPC system to achieve both storage reduction and performance improvement. Our encryption technique provides a practical level of security and enables sharing of sensitive data securely in complex scientific workflows with nearly imperceptible cost. We implement the proposed techniques on top of a distributed embedded KVS to evaluate the benefits and costs of incorporating these capabilities along different points in the dataflow path, illustrating differences in effective bandwidth, latency, and additional computational expense on Swiss National Supercomputing Centre’s Grand Tavé and National Energy Research Scientific Computing Center’s Cori.

97 MATHEMATICS AND COMPUTING↗

Rock Physics-Based Data Assimilation of Integrated Continuous Active-Source Seismic and Pressure Monitoring Data during Geological Carbon Storage

Summary There has been substantial controversy concerning the role of geological carbon storage (GCS) in sequestering anthropogenic carbon emissions to mitigate climate change and global warming. Arguments center on the inability to monitor a geological storage site precisely and continuously, especially highlighting the associated costs and spatiotemporal trade-offs when using conventional subsurface monitoring techniques (well logs, core samples, chemical tracers, and 4D seismics). Active surveillance of GCS sites is essential for managing and mitigating potential leaks but is also required by regulation. With the goal of enhancing the monitoring capability at GCS sites, we present a rock physics-based joint data assimilation model to study a popular GCS site at Cranfield, Mississippi, USA. Synthetic continuous active-source seismic monitoring (CASSM) data (in the form of Vp and Qp measurements) and wellbore pressure monitoring data are assimilated with an ensemble of reservoir realizations to monitor gas saturation and reservoir pressure changes over a period of 100 years. Synthetic seismic attributes are generated using rock physics models (RPMs) and wellbore pressure monitoring data are extracted from the ground truth. Two assimilation methods, ensemble Kalman filter (EnKF) and ensemble Kalman smoother (EnKS), are tested in an observation system simulation experiment (OSSE) environment to assess the prediction accuracy of the individual and composite observation systems. The joint monitoring system achieves more accurate estimates of gas saturation and pressure, across the time span from start of injection to end of forecast, as compared to a single type of monitoring tool and irrespective of data assimilation algorithm choice. These results indicate that jointly assimilated data from two types of sensors (in this case, crosswell seismic and downhole pressure) may lead to a more risk-reducing monitoring design. One would expect that more data, vis-à-vis inclusion of a new sensor type, will improve the accuracy of any GCS monitoring system. However, from a practical standpoint, one important question is whether such a gain in accuracy is worth the additional cost associated with the new sensor. This paper focuses on quantifying the gain in accuracy, such that a practitioner can answer this question.

Engineering↗

Establishing MICHCARB, a geological carbon sequestration research and education center for Michigan, implemented through the Michigan Geological Repository for Research and Education, part of the Department of Geosciences at Western Michigan University

The Michigan Geological Repository for Research and Education (MGRRE), part of the Department of Geosciences at Western Michigan University (WMU) at Kalamazoo, Michigan, established MichCarb—a geological carbon sequestration resource center by: • Archiving and maintaining a current reference collection of carbon sequestration published literature • Developing statewide and site-specific digital research databases for Michigan’s deep geological formations relevant to CO2 storage, containment and potential for enhanced oil recovery • Producing maps and tables of physical properties as components of these databases • Compiling all information into a digital atlas • Conducting geologic and fluid flow modeling to address specific predictive uses of CO2 storage and enhanced oil recovery, including compiling data for geological and fluid flow models, formulating models, integrating data, and running the models; applying models to specific predictive uses of CO2 storage and enhanced oil recovery • Conducting technical research on CO2 sequestration and enhanced oil recovery through basic and applied research of characterizing Michigan oil and gas and saline reservoirs for CO2 storage potential volume, injectivity and containment. Based on our research, we have concluded that the Michigan Basin has excellent saline aquifer (residual entrapment) and CO2/Enhanced oil recovery related (CO2/EOR; buoyant entrapment) geological carbon sequestration potential with substantial, associated incremental oil production potential. These storage reservoirs possess at least satisfactory injectivity and reliable, permanent containment resulting from associated, thick, low permeability confining layers. Saline aquifer storage resource estimates in the two major residual entrapment, reservoir target zones (Lower Paleozoic Sandstone and Middle Paleozoic carbonate and sandstone reservoirs) are in excess of 70-80 Gmt (at an overall 10% storage efficiency factor; an approximately P50 probability range for all formations using DOE-NETL, 2010, storage resource estimation methodology). Incremental oil production resulting from successful implementation of CO2/EOR for the highest potential Middle Paleozoic reef reservoirs (Silurian, Northern Niagaran Reef trend) in Michigan is estimated at 130 to over 200 MMBO (22-33 Mm3). In addition, between 200 and 400 Mmt of CO2 could be sequestered in the course of successful deployment of CO2/EOR in the northern reef trend’s largest depleted (primary production) oil fields (those that have produced in excess of 500,000 BO; 80,000 m3of oil). • Effecting technology transfer to members of industry and governmental agencies by establishing an Internet Website at which all data, reports and results are accessible; publishing results in relevant journals; conducting technology transfer workshops as part of our role as the Michigan Center of the Petroleum Technology Transfer Council or any successor organization.

58 GEOSCIENCES↗

A Nuclear Security Enterprise Study of High-Reliability Systems, Collaboration, and Data

It may seem simple and trivial, but defining the difference between data and information is contested and has implications that may affect the security of United States interests and even cost lives. For security, data are raw facts or figures without context, while information is the compilation or articulation of data that forms context. Security depends on clarity in the differences between data and information and controlling them. Control is necessary to ensure that data and information are not inadvertently released to foreign governments, the public, or those without Need-to-Know. A primary concern in the practice of security is the control of data to avoid the inadvertent conversion to sensitive information. The complexity of this concern is further augmented when institutions are part of tightly coupled networks that informally share data and information. Additionally, those that share data as a function of legislative action—and/or formally integrate data and information system infrastructures—may be a higher security risk. This paper will present a case study that utilizes elements of literature from Knowledge Management and networks to tell a story of an issue in security—specifically, controlling the conversion of data to information.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Comparison of Integral Parameter Estimated Uncertainties in Fast Neutron Systems with COMMARA-2.1 and COMMARA-2.0 Covariance Data

This paper shows the main differences between the COMMARA-2.0 and COMMARA-2.1 evaluated covariance data in the uncertainty estimation of integral parameters of interest for a large number of typical innovative fast neutron systems. The study provides some guidelines to the nuclear data evaluators on which materials and which reaction types need to be evaluated most accurately to achieve reliable results in computational analyses of similar systems. The assessment of the quality and reliability of the covariance data is outside the scope of the study.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

CrossMP: Enabling Cross-Modality Translation between Single-Cell RNA-Seq and Single-Cell ATAC-Seq through Web-Based Portal

In recent years, there has been a growing interest in profiling multiomic modalities within individual cells simultaneously. One such example is integrating combined single-cell RNA sequencing (scRNA-seq) data and single-cell transposase-accessible chromatin sequencing (scATAC-seq) data. Integrated analysis of diverse modalities has helped researchers make more accurate predictions and gain a more comprehensive understanding than with single-modality analysis. However, generating such multimodal data is technically challenging and expensive, leading to limited availability of single-cell co-assay data. Here, we propose a model for cross-modal prediction between the transcriptome and chromatin profiles in single cells. Our model is based on a deep neural network architecture that learns the latent representations from the source modality and then predicts the target modality. It demonstrates reliable performance in accurately translating between these modalities across multiple paired human scATAC-seq and scRNA-seq datasets. Additionally, we developed CrossMP, a web-based portal allowing researchers to upload their single-cell modality data through an interactive web interface and predict the other type of modality data, using high-performance computing resources plugged at the backend.

59 BASIC BIOLOGICAL SCIENCES↗