Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integration.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76
All sourcesNASA NTRSDOE OSTI
NASA NTRS1996

Investigation of Adaptive-threshold Approaches for Determining Area-Time Integrals from Satellite Infrared Data to Estimate Convective Rain Volumes

The principal goal of this project is to establish relationships that would allow application of area-time integral (ATI) calculations based upon satellite data to estimate rainfall volumes. The research is being carried out as a collaborative effort between the two participating organizations, with the satellite data analysis to determine values for the ATIs being done primarily by the STC-METSAT scientists and the associated radar data analysis to determine the 'ground-truth' rainfall estimates being done primarily at the South Dakota School of Mines and Technology (SDSM&T). Synthesis of the two separate kinds of data and investigation of the resulting rainfall-versus-ATI relationships is then carried out jointly. The research has been pursued using two different approaches, which for convenience can be designated as the 'fixed-threshold approach' and the 'adaptive-threshold approach'. In the former, an attempt is made to determine a single temperature threshold in the satellite infrared data that would yield ATI values for identifiable cloud clusters which are closely related to the corresponding rainfall amounts as determined by radar. Work on the second, or 'adaptive-threshold', approach for determining the satellite ATI values has explored two avenues: (1) attempt involved choosing IR thresholds to match the satellite ATI values with ones separately calculated from the radar data on a case basis; and (2) an attempt involved a striaghtforward screening analysis to determine the (fixed) offset that would lead to the strongest correlation and lowest standard error of estimate in the relationship between the satellite ATI values and the corresponding rainfall volumes.

Smith, Paul L.↗
DOE OSTI2012

Use of Stable Isotopes in Forensic Analysis of Microorganisms

The use of isotopic signatures for forensic analysis of biological materials is well-established, and the same general principles that apply to interpretation of stable isotope content of C, N, O, and H apply to the analysis of microorganisms. Heterotrophic microorganisms derive their isotopic content from their growth substrates, which are largely plant and animal products, and the water in their culture medium. Thus the isotope signatures of microbes are tied to their growth environment. The C, N, O, and H isotope ratios of spores have been demonstrated to constitute highly discriminating signatures for sample matching. They can rule out specific samples of media and/or water as possible production media, and can predict isotope ratio ranges of the culture media and water used to produce a given sample. These applications have been developed and tested through analyses of approximately 250 samples of Bacillus subtilis spores and over 500 samples of culture media, providing a strong statistical basis for data interpretation. A Bayesian statistical framework for integrating stable isotope data with other types of signatures derived from microorganisms has been able to characterize the culture medium used to produce spores of various Bacillus species, leveraging isotopic differences in different medium types and demonstrating the power of data integration for forensic investigations.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

VA EDH Advanced Software Pipeline Framework Report: Enhancing Automation and Scalability

The VA Environmental Determinants of Health (EDH) Advanced Software Pipeline Framework is designed to enhance the efficiency, scalability, and security of geospatial data processing workflows. This framework integrates modern data orchestration and containerization technologies, including Prefect for workflow automation, Docker for containerization, and PostgreSQL/PostGIS for geospatial data storage and analysis. It ensures standardized, reproducible, and automated data processing, supporting VA objectives related to substance use risk assessment and recovery research. The pipeline addresses key scalability and performance challenges through horizontal and vertical scaling, high-performance computing (HPC) integration, parallel processing, task caching, and dynamic resource allocation. These optimizations improve throughput and reduce latency, allowing the system to efficiently manage large and complex datasets. Additionally, security and compliance measures—such as data encryption (SSL), Role-Based Access Control (RBAC), and adherence to GDPR and HIPAA standards—safeguard sensitive information throughout data transmission and storage. A key implementation of this framework includes the automation of shelter list geolocation workflows, ensuring that up-to-date data is readily available for VA decision-making. Lessons learned from this project include the transition from in-memory processing to incremental storage writes, improving resource management and reliability. Future enhancements aim to expand automation, integrate AI-driven anomaly detection, and incorporate high-performance computing resources. This framework provides a scalable, secure, and adaptable solution for managing geospatial datasets, reinforcing the VA’s ability to support clinical and strategic initiatives through data-driven decision-making.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

On-the-fly machine-learning for high-throughput experiments: search for rare-earth-free permanent magnets

Advanced materials characterization techniques with ever-growing data acquisition speed and storage capabilities represent a challenge in modern materials science, and new procedures to quickly assess and analyze the data are needed. Machine learning approaches are effective in reducing the complexity of data and rapidly homing in on the underlying trend in multi-dimensional data. Here, we show that by employing an algorithm called the mean shift theory to a large amount of diffraction data in high-throughput experimentation, one can streamline the process of delineating the structural evolution across compositional variations mapped on combinatorial libraries with minimal computational cost. Data collected at a synchrotron beamline are analyzed on the fly, and by integrating experimental data with the inorganic crystal structure database (ICSD), we can substantially enhance the accuracy in classifying the structural phases across ternary phase spaces. We have used this approach to identify a novel magnetic phase with enhanced magnetic anisotropy which is a candidate for rare-earth free permanent magnet.

Kusne, AG↗
DOE OSTI2024

Congruity of genomic and epidemiological data in modelling of local cholera outbreaks

Cholera continues to be a global health threat. Understanding how cholera spreads between locations is fundamental to the rational, evidence-based design of intervention and control efforts. Traditionally, cholera transmission models have used cholera case-count data. More recently, whole-genome sequence data have qualitatively described cholera transmission. Integrating these data streams may provide much more accurate models of cholera spread; however, no systematic analyses have been performed so far to compare traditional case-count models to the phylodynamic models from genomic data for cholera transmission. Here, we use high-fidelity case-count and whole-genome sequencing data from the 1991 to 1998 cholera epidemic in Argentina to directly compare the epidemiological model parameters estimated from these two data sources. We find that phylodynamic methods applied to cholera genomics data provide comparable estimates that are in line with established methods. Our methodology represents a critical step in building a framework for integrating case-count and genomic data sources for cholera epidemiology and other bacterial pathogens.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2009

Integrating and Visualizing Tropical Cyclone Data Using the Real Time Mission Monitor

The Real Time Mission Monitor (RTMM) is a visualization and information system that fuses multiple Earth science data sources, to enable real time decision-making for airborne and ground validation experiments. Developed at the NASA Marshall Space Flight Center, RTMM is a situational awareness, decision-support system that integrates satellite imagery, radar, surface and airborne instrument data sets, model output parameters, lightning location observations, aircraft navigation data, soundings, and other applicable Earth science data sets. The integration and delivery of this information is made possible using data acquisition systems, network communication links, network server resources, and visualizations through the Google Earth virtual globe application. RTMM is extremely valuable for optimizing individual Earth science airborne field experiments. Flight planners, scientists, and managers appreciate the contributions that RTMM makes to their flight projects. A broad spectrum of interdisciplinary scientists used RTMM during field campaigns including the hurricane-focused 2006 NASA African Monsoon Multidisciplinary Analyses (NAMMA), 2007 NOAA-NASA Aerosonde Hurricane Noel flight, 2007 Tropical Composition, Cloud, and Climate Coupling (TC4), plus a soil moisture (SMAP-VEX) and two arctic research experiments (ARCTAS) in 2008. Improving and evolving RTMM is a continuous process. RTMM recently integrated the Waypoint Planning Tool, a Java-based application that enables aircraft mission scientists to easily develop a pre-mission flight plan through an interactive point-and-click interface. Individual flight legs are automatically calculated "on the fly". The resultant flight plan is then immediately posted to the Google Earth-based RTMM for interested scientists to view the planned flight track and subsequently compare it to the actual real time flight progress. We are planning additional capabilities to RTMM including collaborations with the Jet Propulsion Laboratory in the joint development of a Tropical Cyclone Integrated Data Exchange and Analysis System (TC IDEAS) which will serve as a web portal for access to tropical cyclone data, visualizations and model output.

Goodman, H. Michael↗
DOE OSTI2022

A Simple and Accurate Energy-Detector-Based Transient Waveform Detection for Smart Grids: Real-World Field Data Performance

Integration of distributed energy sources, advanced meshed operation, sensors, automation, and communication networks all contribute to autonomous operations and decision-making processes utilized in the grid. Therefore, smart grid systems require sophisticated supporting structures. Furthermore, rapid detection and identification of disturbances and transients are a necessary first step towards situationally aware smart grid systems. This way, high-level monitoring is achieved and the entire system kept operational. Even though smart grid systems are unavoidably sophisticated, low-complexity algorithms need to be developed for real-time sensing on the edge and online applications to alert stakeholders in the event of an anomaly. In this study, the simplest form of anomaly detection mechanism in the absence of any a priori knowledge, namely, the energy detector (also known as radiometer in the field of wireless communications and signal processing), is investigated as a triggering mechanism, which may include automated alerts and notifications for grid anomalies. In contrast to the mainstream literature, it does not rely on transform domain tools; therefore, utmost design and implementation simplicity are attained. Performance results of the proposed energy detector algorithm are validated by real power system data obtained from the DOE/EPRI National Database of power system events and the Grid Signature Library.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Big Data For Operation and Maintenance Cost Reduction

The purpose of this research is to develop a first-of-a-kind framework for integrating Big Data capability into the daily activities of our current fleet of nuclear power plants. Big Data is traditionally defined as data sets with high volume, velocity, and heterogeneity, and the existing Big Data analytics capabilities are now widely popular in fields such as finance, weather, e-commerce, healthcare and sports. In the nuclear industry, while the volume and velocity of data may present computational challenges for existing analytics capabilities, data heterogeneity are seen to present the major challenge. This research project mainly focuses on incorporating the wide range of data heterogeneities in nuclear power plants into an integrated Big Data Analytics capability. The primary end-product of this project is a Big Data framework that is capable of dealing with the large volume and heterogeneity of the data found in nuclear power plants to extract timely and valuable information on equipment performance. The framework can generate system insights that are actionable relations between measurable impacts and the corresponding maintenance action plans and enable optimization of plant operation and maintenance based on the extracted information. The developed framework is capable of handling heterogeneous data including both image data and time-series sensor data. Specifically, this developed framework includes the following components. The first component is an overarching maintenance ontology which includes system insights required by maintenance optimization. The maintenance ontology interacts with other components in the developed framework. The second component handles Piping & Instrumentation Diagram (P&ID) data. It can be used to extract system components and their relations automatically from the P&IDs. This extracted information is stored in the first component, i.e., maintenance ontology, and is also used as input to the third component, i.e., a tool for generating the fault tree for the corresponding system. The generated fault tree in turn is stored in the ontology for assessing risk that is used as a criterion in maintenance policy optimization. The fourth component is a tool for inferring the parameters in the Markov degradation model for a nuclear system. It uses basic information from the ontology. The fifth component is a tool for assessing the degradation level using sensor measurement data, for example, pressure, flowrate. This tool can be used for determining corrective maintenance actions. The results obtained from components four and five are returned to the ontology. The sixth component of the framework is a tool for optimizing the maintenance policy for a nuclear system of interest. It takes certain basic information from the ontology, e.g., costs of maintenance actions and system failures, as input, and returns the optimal maintenance policy to the ontology. This tool can be used for determining predictive maintenance actions. A set of experiments have also been conducted to verify the algorithms developed in this project for nuclear system degradation monitoring. The experiments are based on four solenoid valves, similar to the ones used in nuclear power plants. The analyses based on the experimental data using two algorithms, i.e., the Randomized Window Decomposition (RWD) algorithm and the particle filtering algorithm, and the results are introduced in the report. The Big Data framework developed in this project can be used as a support tool in daily activities of plant operation and maintenance and will reduce current costs while maintaining or improving safety levels. Overall, the project will not only benefit existing reactors, however it will open new frontiers to realize the long overdue value of Big Data Analytics in the nuclear sphere.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2019

Integrating physics-based models with sensor data: An inverse modeling approach

Physics-based building energy models (e.g., EnergyPlus) rely on some unknown input parameters (e.g., zone air infiltration) that are hard to measure, leading to uncertainty in simulation results especially for existing buildings with varying operating conditions. With the increasing deployment of smart thermostats, zone air temperature data are readily available, posing a new opportunity for building energy modeling if such data can be harnessed. This study presents a novel inverse modeling approach which inverses the zone air heat balance equation and uses the measured zone air temperature to analytically calculate the zone air infiltration rate and zone internal thermal mass (e.g., furniture, interior partitions), which are two important model parameters with great variability and difficult to measure. This paper introduces the technical concept and algorithms of the inverse models, their implementation in EnergyPlus, and verification using EnergyPlus simulated building performance data. The inverse modeling approach provides new opportunities for integrating data from massive IoT sensors and devices to enhance the accuracy of simulation results which are used to inform decision making on energy retrofits and efficiency improvements of existing buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2017

Power Hardware-in-the-Loop Evaluation of PV Inverter Grid Support on Hawaiian Electric Feeders

As more grid-connected photovoltaic (PV) inverters become compliant with evolving interconnections requirements, there is increased interest from utilities in understanding how to best deploy advanced grid-support functions (GSF) in the field. One efficient and cost-effective method to examine such deployment options is to leverage power hardware-in-the-loop (PHIL) testing methods, which combine the fidelity of hardware tests with the flexibility of computer simulation. This paper summarizes a study wherein two Hawaiian Electric feeder models were converted to real-time models using an OPAL-RT real-time digital testing platform, and integrated with models of GSF capable PV inverters based on characterization test data. The integrated model was subsequently used in PHIL testing to evaluate the effects of different fixed power factor and volt-watt control settings on voltage regulation of the selected feeders using physical inverters. Selected results are presented in this paper, and complete results of this study were provided as inputs for field deployment and technical interconnection requirements for grid-connected PV inverters on the Hawaiian Islands.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1977

MIUS Integration and Subsystem Test (MIST) data system

A data system for use in testing integrated subsystems of a modular integrated utility system (MIUS) is presented. The MIUS integration and subsystem test (MIST) data system is reviewed from its conception through its checkout and operation as the controlling portion of the MIST facility. The MIST data system provides a real time monitoring and control function that allows for complete evaluation of the performance of the mechanical and electrical subsystems, as well as controls the operation of the various components of the system. In addition to the aforementioned capabilities, the MIST data system provides computerized control of test operations such that minimum manpower is necessary to set up, operate, and shut down subsystems during test periods.

Pringle, L. M.↗
DOE OSTI2014

Tightly integrated single- and multi-crystal data collection strategy calculation and parallelized data processing in JBluIce beamline control system

The calculation of single- and multi-crystal data collection strategies and a data processing pipeline have been tightly integrated into the macromolecular crystallographic data acquisition and beamline control software JBluIce . Both tasks employ wrapper scripts around existing crystallographic software. JBluIce executes scripts through a distributed resource management system to make efficient use of all available computing resources through parallel processing. The JBluIce single-crystal data collection strategy feature uses a choice of strategy programs to help users rank sample crystals and collect data. The strategy results can be conveniently exported to a data collection run. The JBluIce multi-crystal strategy feature calculates a collection strategy to optimize coverage of reciprocal space in cases where incomplete data are available from previous samples. The JBluIce data processing runs simultaneously with data collection using a choice of data reduction wrappers for integration and scaling of newly collected data, with an option for merging with pre-existing data. Data are processed separately if collected from multiple sites on a crystal or from multiple crystals, then scaled and merged. Results from all strategy and processing calculations are displayed in relevant tabs of JBluIce .

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
NASA NTRS1989

International standards activities in image data compression

Integrated Services Digital Network (ISDN); coding for color TV, video conferencing, video conferencing/telephone, and still color images; ISO color image coding standard; and ISO still picture standard are briefly discussed. This presentation is represented by viewgraphs only.

Haskell, Barry↗
NASA NTRS2017

Principal Modes of Precipitation Variability from Preliminary Series of IMERG Data

The Integrated Multi-satellitE Retrievals for the Global Precipitation Measurement (GPM) mission, IMERG, is the unified U.S. algorithm that provides merged Microwave Infrared (IR) satellite precipitation product for the U.S. GPM team. Even though IMERG record is still very short, 2014-2016, it is tempting to test if it captures ENSO and NAO signals as compared to the popular, still on-going, TRMM Multi-satellite Precipitation Analysis, TMPA. El Nino Southern Oscillation (ENSO) is the most significant mode of interannual variability of tropical ocean atmosphere. North Atlantic Oscillation (NAO) impact is on monthly scales and is mostly an atmospheric mode in the North Atlantic. There exist well-defined, multivariate, indexes that represent ENSO and NAO conditions and phase.

climate variability↗
NASA NTRS2018

An “EPIIC” Vision to Evolve Project Integration, Innovation, and Collaboration with Broad Impact for How NASA Executes Complex Projects

Evolving Project Integration, Innovation, and Collaboration (EPIIC) is a vision defined to transform the way projects manage information to support real-time decisions, capture best practices and lessons learned, perform assessments, and manage risk across a portfolio of projects. The foundational project management needs for data and information will be revolutionized through innovations on how we manage and access that data, implement configuration control, and certify compliance. The embedded intelligence of new interactive data interfaces integrate technical and programmatic data such that near real time analytics can be accomplished to more efficiently and accurately complete systems engineering and project management tasks. The system-wide data analytics that are integrated into customized data interfaces allows the growing team of engineers and managers required to develop and implement major NASA missions the ability to access authoritative source(s) of system information while greatly reducing the labor required to complete system assessments. This would allow, for example, much of what is accomplished in a scheduled design review to take place as needed, between any team members, at any time. An intelligent data interface that rigorously integrates systems engineering and project management information in near real time can provide substantially greater insight for systems engineers, project managers, and the large diverse teams required to complete a complex project. System engineers, programmatic personnel (those who focus on cost, schedule, and risk), the technical engineering disciplines, and project management can realize immediate benefit from the shared vision described herein. Implementation of the vision also enables significant improvements in the performance of the engineered system being developed.

Moreland, Robert J.↗
NASA NTRS2015

NASA's ATM Ontology: Semantic Integration and Querying Across NAS Data Sources

NASA is developing an Air Traffic Management (ATM) Ontology as part of an advanced prototyping activity that demonstrates the utility of semantic technologies for integrating, querying, and searching over various sources of heterogeneous ATM data. The ontology encodes an overarching data model that functions as the backbone upon which to overlay data from multiple sources published by FAA, NOAA, NASA, and others. The integrated data can be queried to produce results not achievable using any single source alone. The ontology incorporates flight data, weather data, traffic management advisory data, airport delay data, and national airspace infrastructure data for a very limited spatial and temporal slice of airspace operations (one day of operations at a major airport).

air traffic management↗
DOE OSTI2019

Integration of transcriptomic and proteomic data identifies biological functions in cell populations from human infant lung

Systems biology uses computational approaches to integrate diverse data types to understand cell and organ behavior. Data derived from complementary technologies, for example transcriptomic and proteomic analyses, are providing new insights into development and disease. We compared mRNA and protein profiles from purified endothelial, epithelial, immune, and mesenchymal cells from normal human lung tissue. Signatures for each cell type were identified and compared at both mRNA and protein levels. Cell specific biological processes and pathways were predicted by analysis of concordant and discordant RNA-protein pairs. Cell clustering and gene set enrichment comparisons identified shared versus unique processes associated with transcriptomic and/or proteomic data. Clear cell-cell correlations between mRNA and protein data were obtained from each cell type. Approximately 40% of RNA-protein pairs were coherently expressed. While the correlation between RNA and their protein products was relatively low (approximately 0.4), cell specific signature genes involved in functional processes characteristic of each cell type, were more highly correlated with their protein products. Consistency of cell specific RNA-protein signatures indicated an essential framework for the function of each cell type. Visualization and reutilization of the protein and RNA profiles are supported by a new web application, "LungProteomics," which is incorporated into the LGEA web portal and freely accessible at (https://research.cchmc.org/pbge/lunggens/lungProteomics/human_sorted_profiles.html).

human lung, sorted cells, integrated omics, Transc↗
NASA NTRS2017

Data Reduction Pipeline for the CHARIS Integral-Field Spectrograph I: Detector Readout Calibration and Data Cube Extraction

We present the data reduction pipeline for CHARIS, a high-contrast integral-field spectrograph for the Subaru Telescope. The pipeline constructs a ramp from the raw reads using the measured nonlinear pixel response and reconstructs the data cube using one of three extraction algorithms: aperture photometry, optimal extraction, or chi-squared fitting. We measure and apply both a detector flatfield and a lenslet flatfield and reconstruct the wavelength- and position-dependent lenslet point-spread function (PSF) from images taken with a tunable laser. We use these measured PSFs to implement a chi-squared-based extraction of the data cube, with typical residuals of approximately 5 percent due to imperfect models of the under-sampled lenslet PSFs. The full two-dimensional residual of the chi-squared extraction allows us to model and remove correlated read noise, dramatically improving CHARIS's performance. The chi-squared extraction produces a data cube that has been deconvolved with the line-spread function and never performs any interpolations of either the data or the individual lenslet spectra. The extracted data cube also includes uncertainties for each spatial and spectral measurement. CHARIS's software is parallelized, written in Python and Cython, and freely available on github with a separate documentation page. Astrometric and spectrophotometric calibrations of the data cubes and PSF subtraction will be treated in a forthcoming paper.

Brandt, Timothy D.↗