Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integration.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Flexible Data Fusion for Air Quality Estimation and Forecasting in Google Earth Engine to support Global Health Management Needs

The assessment and forecasting of air quality around the world at high spatial and temporal resolution can be enhanced by integrating data from multiple sources including models, satellites, regulatory monitors, and low-cost sensors. Such integration is subject to numerous technical challenges, however, including heterogeneous data resolution and formatting, different levels of data availability and reliability, and computational and capacity challenges to developing data fusion tools and platforms. This presentation will provide an overview of a NASA-funded effort to develop a data fusion system within the Google Earth Engine platform which integrates these air quality data sources to produce comprehensive assessments and forecasts of key air pollutants at sub-daily and sub-city scales. The system is being developed in collaboration with city- and regional-level air quality managers and will provide them with information to the assess and anticipate the health impacts of poor air quality, track local changes in air quality due to ongoing transportation and land use changes, and identify potential gaps in their current air quality monitoring strategies. The presentation will report advances achieved through the project, including bringing local air quality monitoring data into Google Earth Engine, quantifying uncertainties in air quality estimates and forecasts, and tailored communications tools providing integration into end-user processes to meet their needs.

Nathan R. Pavlovic↗
DOE OSTI2025

Online and Offline Data Quality Monitoring for the Mu2e Calorimeter

This thesis presents the design, implementation, and validation of a calorimeter Data Quality Monitoring (DQM) toolchain for the Mu2e experiment at Fermilab. Mu2e searches for charged lepton flavor violation via coherent muon-to-electron conversion in the field of an aluminum nucleus, $\mu^- Al \rightarrow e^-Al$, a process whose observation would constitute clear evidence of physics beyond the Standard Model. Achieving target sensitivity requires stringent control of detector performance and data integrity during acquisition, as subtle issues in readout configuration, data formatting, or electronics behavior can compromise reconstruction and bias downstream analyzes. To address these challenges, this work develops a multi-layer DQM approach spanning both raw data validation and reconstructed digi-level diagnostics. At the low level, a fragment analysis component performs word- and bit-field decoding of calorimeter readout blocks, enabling sanity checks of the expected structure and producing detailed error and integrity statistics useful for commissioning and troubleshooting. At the digi level, the CaloDigiDQM analyzer is implemented within the art framework and transforms each CaloDigiCollection into a structured hierarchy of ROOT histograms designed for fast drill-down diagnostics. The module generates coherent monitoring views at global, disk, board, and channel granularity, including occupancy, waveform-derived features (baseline, RMS, peak amplitude and position), and left-right sensor consistency metrics. Detector-aware channel-to-electronics mapping is performed through the conditions system (CaloDAQMap), ensuring that diagnostics remain aligned with hardware identifiers used in operations. For end-to-end testing without reliance on live DAQ data, a synthetic CaloDigi producer is developed to generate realistic waveforms with controlled noise and pulse shapes. The resulting system supports both offline ROOT-file production and online operation, including optional histogram streaming through otsdaq via ots::HistoSender. This toolchain provides a practical and scalable foundation for calorimeter commissioning and stable data collection, enabling early detection of anomalies and reducing operational risk for Mu2e.

Vakulenko, Mark [Drew U.] (ORCID:0009000276197818)↗
NASA NTRS2006

Sensor Management for Applied Research Technologies (SMART)-On Demand Modeling (ODM) Project

NASA requires timely on-demand data and analysis capabilities to enable practical benefits of Earth science observations. However, a significant challenge exists in accessing and integrating data from multiple sensors or platforms to address Earth science problems because of the large data volumes, varying sensor scan characteristics, unique orbital coverage, and the steep learning curve associated with each sensor and data type. The development of sensor web capabilities to autonomously process these data streams (whether real-time or archived) provides an opportunity to overcome these obstacles and facilitate the integration and synthesis of Earth science data and weather model output. A three year project, entitled Sensor Management for Applied Research Technologies (SMART) - On Demand Modeling (ODM), will develop and demonstrate the readiness of Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) capabilities that integrate both Earth observations and forecast model output into new data acquisition and assimilation strategies. The advancement of SWE-enabled systems (i.e., use of SensorML, sensor planning services - SPS, sensor observation services - SOS, sensor alert services - SAS and common observation model protocols) will have practical and efficient uses in the Earth science community for enhanced data set generation, real-time data assimilation with operational applications, and for autonomous sensor tasking for unique data collection.

Goodman, M.↗
DOE OSTI2018

Managing Data on Tiered Storage for Scientific Workflows (MaDaTS) v1.1.2

MaDaTS provides an integrated data management and workflow execution framework on multi-tiered storage systems. Users of MaDaTS can execute a workflow by either specifying the workflow stages in a YAML description file, or use the API to manage the workflow through a Virtual Data Space (VDS) abstraction.

Ramakrishnan, Lavanya↗
DOE OSTI2015

Spectral state dependence of the 0.4–2 MeV polarized emission in Cygnus X-1 seen with INTEGRAL/IBIS, and links with the AMI radio data

Polarization of the ≳400 keV hard tail of the microquasar Cygnus X-1 has been independently reported by INTEGRAL/Imager on Board the INTEGRAL Satellite (IBIS), and INTEGRAL/SPectrometer on INTEGRAL and interpreted as emission from a compact jet. These conclusions were, however, based on the accumulation of all INTEGRAL data regardless of the spectral state. We utilize additional INTEGRAL exposure accumulated until 2012 December, and include the AMI/Ryle (15 GHz) radio data in our study. We separate the observations into hard, soft, and intermediate/transitional states and detect radio emission from a compact jet in hard and intermediate states (IS), but not in the soft. The 10–400 keV INTEGRAL (JEM-X and IBIS) state resolved spectra are well modeled with thermal Comptonization and reflection components. We detect a hard tail in the 0.4–2 MeV range for the hard state only. We extract the state dependent polarigrams of Cyg X-1, which are all compatible with no or an undetectable level of polarization except in the 400–2000 keV range in the hard state where the polarization fraction is 75% ± 32% and the polarization angle 40.°0 ± 14.°3. An upper limit on the 0.4–2 MeV soft state polarization fraction is 70%. Due to the short exposure, we obtain no meaningful constraint for the IS. The likely detection of a >400 keV polarized tail in the hard state, together with the simultaneous presence of a radio jet, reinforce the notion of a compact jet origin of the >400 keV emission.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

The MG-RAST Metagenomics Database and Portal in 2015

MG-RAST (http://metagenomics.anl.gov) is an opensubmission data portal for processing, analyzing, sharing and disseminating metagenomic datasets. Currently, the system hosts over 200 000 datasets and is continuously updated. The volume of submissions has increased 4-fold over the past 24 months, now averaging 4 terabasepairs per month. In addition to several new features, we report changes to the analysis workflow and the technologies used to scale the pipeline up to the required throughput levels. Lastly, to show possible uses for the data from MG-RAST, we present several examples integrating data and analyses from MG-RAST into popular third-party analysis tools or sequence alignment tools.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2024

Automated pipeline processing X-ray diffraction data from dynamic compression experiments on the Extreme Conditions Beamline of PETRA III

Presented and discussed here is the implementation of a software solution that provides prompt X-ray diffraction data analysis during fast dynamic compression experiments conducted within the dynamic diamond anvil cell technique. It includes efficient data collection, streaming of data and metadata to a high-performance cluster (HPC), fast azimuthal data integration on the cluster, and tools for controlling the data processing steps and visualizing the data using the DIOPTAS software package. This data processing pipeline is invaluable for a great number of studies. The potential of the pipeline is illustrated with two examples of data collected on ammonia–water mixtures and multiphase mineral assemblies under high pressure. The pipeline is designed to be generic in nature and could be readily adapted to provide rapid feedback for many other X-ray diffraction techniques, e.g. large-volume press studies, in situ stress/strain studies, phase transformation studies, chemical reactions studied with high-resolution diffraction etc.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

A Data Quality Filter for PMU Measurements: Description, Experience, and Examples

Networks of phasor measurement units (PMUs) continue to grow, and along with them, the amount of data available for analysis. With so much data, it is impractical to identify and remove poor quality data manually. The data quality filter described in this paper was developed for use with the Data Integrity and Situation Awareness Tool (DISAT), which analyzes PMU data to identify anomalous system behavior. The filter operates based only on the information included in the data files, without supervisory control and data acquisition (SCADA) data, state estimator values, or system topology information. Measurements are compared to preselected thresholds to determine if they are reliable. Along with the filter's description, examples of data quality issues from application of the filter to nine months of archived PMU data are provided. The paper is intended to aid the reader in recognizing and properly addressing data quality issues in PMU data.

Phasor Measurement Unit↗
DOE OSTI2019

Implementing efficient data compression and encryption in a persistent key-value store for HPC

Recently, persistent data structures, like key-value stores (KVSs), which are stored in a high-performance computing (HPC) system’s nonvolatile memory, provide an attractive solution for a number of emerging challenges like limited I/O performance. Data compression and encryption are two well-known techniques for improving several properties of such data-oriented systems. This article investigates how to efficiently integrate data compression and encryption into persistent KVSs for HPC with the ultimate goal of hiding their costs and complexity in terms of performance and ease of use. Our compression technique exploits deep memory hierarchy in an HPC system to achieve both storage reduction and performance improvement. Our encryption technique provides a practical level of security and enables sharing of sensitive data securely in complex scientific workflows with nearly imperceptible cost. We implement the proposed techniques on top of a distributed embedded KVS to evaluate the benefits and costs of incorporating these capabilities along different points in the dataflow path, illustrating differences in effective bandwidth, latency, and additional computational expense on Swiss National Supercomputing Centre’s Grand Tavé and National Energy Research Scientific Computing Center’s Cori.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Rock Physics-Based Data Assimilation of Integrated Continuous Active-Source Seismic and Pressure Monitoring Data during Geological Carbon Storage

Summary There has been substantial controversy concerning the role of geological carbon storage (GCS) in sequestering anthropogenic carbon emissions to mitigate climate change and global warming. Arguments center on the inability to monitor a geological storage site precisely and continuously, especially highlighting the associated costs and spatiotemporal trade-offs when using conventional subsurface monitoring techniques (well logs, core samples, chemical tracers, and 4D seismics). Active surveillance of GCS sites is essential for managing and mitigating potential leaks but is also required by regulation. With the goal of enhancing the monitoring capability at GCS sites, we present a rock physics-based joint data assimilation model to study a popular GCS site at Cranfield, Mississippi, USA. Synthetic continuous active-source seismic monitoring (CASSM) data (in the form of Vp and Qp measurements) and wellbore pressure monitoring data are assimilated with an ensemble of reservoir realizations to monitor gas saturation and reservoir pressure changes over a period of 100 years. Synthetic seismic attributes are generated using rock physics models (RPMs) and wellbore pressure monitoring data are extracted from the ground truth. Two assimilation methods, ensemble Kalman filter (EnKF) and ensemble Kalman smoother (EnKS), are tested in an observation system simulation experiment (OSSE) environment to assess the prediction accuracy of the individual and composite observation systems. The joint monitoring system achieves more accurate estimates of gas saturation and pressure, across the time span from start of injection to end of forecast, as compared to a single type of monitoring tool and irrespective of data assimilation algorithm choice. These results indicate that jointly assimilated data from two types of sensors (in this case, crosswell seismic and downhole pressure) may lead to a more risk-reducing monitoring design. One would expect that more data, vis-à-vis inclusion of a new sensor type, will improve the accuracy of any GCS monitoring system. However, from a practical standpoint, one important question is whether such a gain in accuracy is worth the additional cost associated with the new sensor. This paper focuses on quantifying the gain in accuracy, such that a practitioner can answer this question.

Engineering↗
DOE OSTI2014

Establishing MICHCARB, a geological carbon sequestration research and education center for Michigan, implemented through the Michigan Geological Repository for Research and Education, part of the Department of Geosciences at Western Michigan University

The Michigan Geological Repository for Research and Education (MGRRE), part of the Department of Geosciences at Western Michigan University (WMU) at Kalamazoo, Michigan, established MichCarb—a geological carbon sequestration resource center by: • Archiving and maintaining a current reference collection of carbon sequestration published literature • Developing statewide and site-specific digital research databases for Michigan’s deep geological formations relevant to CO2 storage, containment and potential for enhanced oil recovery • Producing maps and tables of physical properties as components of these databases • Compiling all information into a digital atlas • Conducting geologic and fluid flow modeling to address specific predictive uses of CO2 storage and enhanced oil recovery, including compiling data for geological and fluid flow models, formulating models, integrating data, and running the models; applying models to specific predictive uses of CO2 storage and enhanced oil recovery • Conducting technical research on CO2 sequestration and enhanced oil recovery through basic and applied research of characterizing Michigan oil and gas and saline reservoirs for CO2 storage potential volume, injectivity and containment. Based on our research, we have concluded that the Michigan Basin has excellent saline aquifer (residual entrapment) and CO2/Enhanced oil recovery related (CO2/EOR; buoyant entrapment) geological carbon sequestration potential with substantial, associated incremental oil production potential. These storage reservoirs possess at least satisfactory injectivity and reliable, permanent containment resulting from associated, thick, low permeability confining layers. Saline aquifer storage resource estimates in the two major residual entrapment, reservoir target zones (Lower Paleozoic Sandstone and Middle Paleozoic carbonate and sandstone reservoirs) are in excess of 70-80 Gmt (at an overall 10% storage efficiency factor; an approximately P50 probability range for all formations using DOE-NETL, 2010, storage resource estimation methodology). Incremental oil production resulting from successful implementation of CO2/EOR for the highest potential Middle Paleozoic reef reservoirs (Silurian, Northern Niagaran Reef trend) in Michigan is estimated at 130 to over 200 MMBO (22-33 Mm3). In addition, between 200 and 400 Mmt of CO2 could be sequestered in the course of successful deployment of CO2/EOR in the northern reef trend’s largest depleted (primary production) oil fields (those that have produced in excess of 500,000 BO; 80,000 m3of oil). • Effecting technology transfer to members of industry and governmental agencies by establishing an Internet Website at which all data, reports and results are accessible; publishing results in relevant journals; conducting technology transfer workshops as part of our role as the Michigan Center of the Petroleum Technology Transfer Council or any successor organization.

58 GEOSCIENCES↗
DOE OSTI2021

A Nuclear Security Enterprise Study of High-Reliability Systems, Collaboration, and Data

It may seem simple and trivial, but defining the difference between data and information is contested and has implications that may affect the security of United States interests and even cost lives. For security, data are raw facts or figures without context, while information is the compilation or articulation of data that forms context. Security depends on clarity in the differences between data and information and controlling them. Control is necessary to ensure that data and information are not inadvertently released to foreign governments, the public, or those without Need-to-Know. A primary concern in the practice of security is the control of data to avoid the inadvertent conversion to sensitive information. The complexity of this concern is further augmented when institutions are part of tightly coupled networks that informally share data and information. Additionally, those that share data as a function of legislative action—and/or formally integrate data and information system infrastructures—may be a higher security risk. This paper will present a case study that utilizes elements of literature from Knowledge Management and networks to tell a story of an issue in security—specifically, controlling the conversion of data to information.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1984

Method for Accessing Distributed Heterogeneous Databases

A scenario of relational, hierarchial, and network data bases is presented and a distributed access view integrated data base system (DAVID) is described for uniformly accessing data bases which are heterogeneous and physically distributed. The DAVID system is based on data base logic so that the relational approach is generalized to the heterogeneous approach. The global data manager is explained as are global data manipulation languages which can operate on all the data bases and can query the data dictionary and the data directory.

Jacobs, B. E.↗
NASA NTRS1986

Laser fringe anemometry for aero engine components

Advances in flow measurement techniques in turbomachinery continue to be paced by the need to obtain detailed data for use in validating numerical predictions of the flowfield and for use in the development of empirical models for those flow features which cannot be readily modelled numerically. The use of laser anemometry in turbomachinery research has grown over the last 14 years in response to these needs. Based on past applications and current developments, this paper reviews the key issues which are involved when considering the application of laser anemometry to the measurement of turbomachinery flowfields. Aspects of laser fringe anemometer optical design which are applicable to turbomachinery research are briefly reviewed. Application problems which are common to both laser fringe anemometry (LFA) and laser transit anemometry (LTA) such as seed particle injection, optical access to the flowfield, and measurement of rotor rotational position are covered. The efficiency of various data acquisition schemes is analyzed and issues related to data integrity and error estimation are addressed. Real-time data analysis techniques aimed at capturing flow physics in real time are discussed. Finally, data reduction and analysis techniques are discussed and illustrated using examples taken from several LFA turbomachinery applications.

Strazisar, A. J.↗
NASA NTRS1986

Laser fringe anemometry for aero engine components

Advances in flow measurement techniques in turbomachinery continue to be paced by the need to obtain detailed data for use in validating numerical predictions of the flowfield and for use in the development of empirical models for those flow features which cannot be readily modelled numerically. The use of laser anemometry in turbomachinery research has grown over the last 14 years in response to these needs. Based on past applications and current developments, the key issues which are involved when considering the application of laser anemometry to the measurement of turbomachinery flowfields are discussed. Aspects of laser fringe anemometer optical design which are applicable to turbomachinery research are briefly reviewed. Application problems which are common to both laser fringe anemometry (LFA) and laser transit anemometry (LTA) such as seed particle injection, optical access to the flowfield, and measurement of rotor rotational position are covered. The efficiency of various data acquisition schemes is analyzed and issues related to data integrity and error estimation are addressed. Real-time data analysis techniques aimed at capturing flow physics in real time are discussed. Finally, data reduction and analysis techniques are discussed and illustrated using examples taken from several LFA turbomachinery applications.

Strazisar, Anthony J.↗
NASA NTRS2010

Data Access Services that Make Remote Sensing Data Easier to Use

This slide presentation reviews some of the processes that NASA uses to make the remote sensing data easy to use over the World Wide Web. This work involves much research into data formats, geolocation structures and quality indicators, often to be followed by coding a preprocessing program. Only then are the data usable within the analysis tool of choice. The Goddard Earth Sciences Data and Information Services Center is deploying a variety of data access services that are designed to dramatically shorten the time consumed in the data preparation step. On-the-fly conversion to the standard network Common Data Form (netCDF) format with Climate-Forecast (CF) conventions imposes a standard coordinate system framework that makes data instantly readable through several tools, such as the Integrated Data Viewer, Gridded Analysis and Display System, Panoply and Ferret. A similar benefit is achieved by serving data through the Open Source Project for a Network Data Access Protocol (OPeNDAP), which also provides subsetting. The Data Quality Screening Service goes a step further in filtering out data points based on quality control flags, based on science team recommendations or user-specified criteria. Further still is the Giovanni online analysis system which goes beyond handling formatting and quality to provide visualization and basic statistics of the data. This general approach of automating the preparation steps has the important added benefit of enabling use of the data by non-human users (i.e., computer programs), which often make sub-optimal use of the available data due to the need to hard-code data preparation on the client side.

Lynnes, Christopher↗