Search NASA⌕ Search

SEARCH · Search NASA

Results for “Streaming Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

An Integrated Platform for Collaborative Data Analytics

While collaboration among data scientists is a key to organizational productivity, data analysts face significant barriers to achieving this end, including data sharing, accessing and configuring the required computational environment, and a unified method of sharing knowledge. Each of these barriers to collaboration is related to the fundamental question of knowledge management “how can organizations use knowledge more effectively?”. In this paper, we consider the problem of knowledge management in collaborative data analytics and present ShareAL, an integrated knowledge management platform, as a solution to that problem. The ShareAL platform consists of three core components: a full stack web application, a dashboard for analyzing streaming data and a High Performance Computing (HPC) cluster for performing real time analysis. Prior research has not applied knowledge management to collaborative analytics or developed a platform with the same capabilities as ShareAL. ShareAL overcomes the barriers data scientists face to collaboration by providing intuitive sharing of data and analytics via the web application, a shared computing environment via the HPC cluster and knowledge sharing and collaboration via a real time messaging application.

Oesch, T↗
DOE OSTI2025

Modular Autonomous Experimentation for Biological Applications (Full Report)

The Modular Autonomous Research System (MARS) was developed to address the pressing need for faster, more reliable, and more adaptable scientific discovery. Traditional experimentation is limited by manual labor, long cycle times, and fragmented data streams, which constrain the ability to explore complex chemical and materials design spaces. To overcome these limitations, we created an integrated, modular platform that combines laboratory robotics, diverse measurement instruments, and a central data infrastructure with artificial intelligence–driven decision-making. The system links liquid handling robots, robotic arms, and optical plate readers into a closed loop where experiments are executed automatically, data is analyzed in real time, and subsequent experimental conditions are adaptively chosen to maximize information gain. Over the course of the project, MARS was validated on two primary test cases—spectroscopic metal–ligand binding assays and peptide-directed mineralization—which highlighted the system’s ability to handle uncertainty and variability in experimental measurements. To further demonstrate modularity and extensibility, we also established additional testbeds in electrochemistry for catalyst discovery and electrolyte formulation for advanced batteries. The results show that MARS can reliably conduct autonomous campaigns with minimal human intervention, adapt to distinct scientific domains, and provide a scalable model for future self-driving laboratories. This work establishes new capabilities for modular, uncertainty-aware automation and directly supports the need for advanced, data-driven research platforms capable of accelerating discovery across a wide range of scientific and national security missions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Deep learning at the edge enables real-time streaming ptychographic imaging

Abstract Coherent imaging techniques provide an unparalleled multi-scale view of materials across scientific and technological fields, from structural materials to quantum devices, from integrated circuits to biological cells. Driven by the construction of brighter sources and high-rate detectors, coherent imaging methods like ptychography are poised to revolutionize nanoscale materials characterization. However, these advancements are accompanied by significant increase in data and compute needs, which precludes real-time imaging, feedback and decision-making capabilities with conventional approaches. Here, we demonstrate a workflow that leverages artificial intelligence at the edge and high-performance computing to enable real-time inversion on X-ray ptychography data streamed directly from a detector at up to 2 kHz. The proposed AI-enabled workflow eliminates the oversampling constraints, allowing low-dose imaging using orders of magnitude less data than required by traditional methods.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Biologically inspired approaches for biosurveillance anomaly detection and data fusion

This study developed and tested biologically inspired computational methods to detect anomalous signals in data streams that could indicate a pending outbreak or bio-weapon attack. Current large-scale biosurveillance systems are plagued by two principal deficiencies: (1) timely detection of disease-indicating signals in noisy data and (2) anomaly detection across multiple channels. Anomaly detectors and data fusion components modeled after human immune system processes were tested against a variety of natural and synthetic surveillance datasets. A pilot scale immune-system-based biosurveillance system performed at least as well as traditional statistical anomaly detection data fusion approaches. Machine learning approaches leveraging Deep Learning recurrent neural networks were developed and applied to challenging unstructured and multimodal health surveillance data. Within the limits imposed of data availability, both immune systems and deep learning methods were found to improve anomaly detection and data fusion performance for particularly challenging data subsets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

Data Analytics for Electrical Distribution Systems with Micro PMUs (GMLC 1.4.9 Technical Report)

This report describes how electrical utilities with residential, industrial, commercial and government customers make decisions using Data Analytics (DA) applications and discusses how OpenFMB was leveraged to share that data among disparate types of devices for DA. OpenFMB is a data standard developed by utilities for distributed communication and control of power system assets. Here we explored how OpenFMB can be used for sharing analytics and what those analytics could be for a set of representative grid devices. A review of DA applications performed by electrical utilities and customers is presented. An overview of how the testbed we used was built and configured is summarized. Several experiments which were carried out to explore the capabilities of OpenFMB alongside other data sharing mechanisms for gathering sensor measurements and sharing analytics is also discussed. And finally a comparison of OpenFMB and LBNL’s SPARK framework is summarized. Today’s electrical utilities focus on monitoring the real-time customer consumptions, offering new real-time tariff models based on customer usage habits (rewards and punishments), providing energy peak during peak load times (plan energy generation and distribution), improving the load management for transformers and power lines, and controlling the energy and grid resources efficiently. On the other hand, the DA applications for residential customers are based on modifying the customer electricity usage habit to save money, making accurate and timely decisions for electrical billing. However, the DA applications for industrial, commercial and government customers focus on creating efficient programs to improve the environmental condition (reducing CO₂ emission), developing better load forecasting for industrial and/or commercial activities. In this project, a testbed was built to explore how the OpenFMB framework can be used to distribute data among various types of representative grid edge devices. Micro PMUs (Phasor Measurement Unit) were used to collect and transmit data to an OpenPDC server alongside relays with a framework configured with a system using OpenFMB to collect and transmit data. The testbed was composed of a physical asset based on a 60 Hz AC radial power system constructed using SI-GRID components. The power system consisted of a 3-phase 14 volts line-to-ground (24 volts line-to-line) power source, three 3-phase line emulators, a programmable resistive load bank, two smart relays, two micro PMUs, and a (SCADA) and Phasor Data Concentrator (PDC) computer. This testbed configuration allowed us to perform breaker operation tests collecting data from the micro PMUs and from the relays’ sensors. During the micro PMU breaker operation tests we collected the measured and estimated fundamental voltages, currents, frequency as well as the total true, reactive and apparent power. The displacement power factor from two micro PMUs located in parallel power lines was also used. The software steps for the implementation of the Synchrophasor system are explained in detail in this report. The setting (1-2-3) and data collection (4-5) phases were performed with a free download open source software. In the setting phase, the micro PMUs were set (1), the data stream for each micro PMU was validated, and the connection and configuration files were created (2). The PDC computer was set using the connection and configuration files of the micro PMUs (3). In the data collection phase, the binary data from the micro PMUs was collected (4), and the binary data files of the micro PMUs was converted to analyze the test results (5). OpenFMB was configured to use protocol translators to read Modbus and transform the reading into the OpenFMB data model to be read for DA applications.Our experiments showed that by leveraging the OpenFMB data model and the micro PMUs together we could obtain better situational awareness of network faults, which in some cases could lead to failures in the system and would be less perceivable otherwise. For example, in one experiment the fundamental RMS current angle, and the fundamental displacement power factor (estimated by the ratio between the measured real and apparent total power) had a disturbance at the micro PMU connected to the beginning of the parallel power line because the breaker was opened without the operators or SCADA system’s knowledge, and no current was available. Based on the breaker state scenarios and micro PMU locations, the DA applications using the current angle and displacement power factor from the micro PMUs should be studied in detail to avoid a non-desired measurement that could affect the decision from the DA. Also, the sample rates for PMU, Relay and micro PMU devices were compared. The DA application improved the control and operation between the electrical utilities and their customers (residential, industrial, transportation, commercial and government). The higher the sample rate, the better the quality of the measurement for DA estimations.

42 ENGINEERING↗
NASA NTRSDate not supplied

Leveraging VIPER Geotechnical and Mobility Insights for Endurance Rover and Traverse Development

A new generation of lunar surface rover exploration is rapidly approaching. This is evidenced by the rise of commercially developed rovers from private companies (e.g., Lunar Outpost’s MAPP) and government agencies, like NASA’s Volatiles Investigating Polar Exploration Rover (VIPER), the Lunar Terrain Vehicle (LTV), and the Endurance/Intrepid rover concepts, and the INSPIRE mission concept. The drive distance requirements for off-world vehicles are also rapidly increasing. For example, LTV driving requirements dictate a 20 km range on a single battery charge, with an overall lifetime requirement of 13,000 km over a minimum 10-year service life. Similarly, NASA's Endurance-A mission concept requires a 2000 km vehicle traverse distance, which is nearly 50 times further than the USSR's Lunokhod 2, the longest lunar rover mission to date. Previous planetary surface missions on the Moon and Mars have revealed harsh mobility conditions: aggressive rocky terrains on Mars severely damaged the Curiosity rover’s wheels, soft lunar regolith caused the Lunokhod 2 rover to sink and NASA’s Apollo 15 LRV (Lunar Roving Vehicle) spun out due to poor traction, which required manual correction. This demonstrates that mobility challenges like wheel degradation and soft soil driving performance must be accounted for in a mission’s design phase to meet increasingly aggressive mission durations and mobility goals. The success of these long duration missions requires highly reliable and durable mobility platforms in conjunction with a more developed understanding of the geotechnical properties of the lunar surface. The VIPER mission is uniquely posed to inform future missions about mobility performance, wheel durability, and lunar surface soil mechanics of sunlit and shadowed regions through use of data streams that already exist for other mission purposes. VIPER’s mission to the lunar south pole will provide a new perspective into off-world mobility challenges including traversal into permanently shadowed regions. Mobility data from VIPER’s mission will represent a first glimpse into the surface mobility challenges of the lunar South Pole. Utilization of VIPER’s existing capabilities including cameras, inertial measurement unit (IMU) data, motor currents/torque, drill data, and the rover’s suspension state and kinematics provide the opportunity to determine potentially useful information for future rover and traverse designs, as well as on-Earth testing practices. Data products collected during VIPER’s mission could inform design considerations for the Endurance rover much earlier than NASA’s standard “lessons learned” approach as VIPER has a highly tactical mission operations structure. This work outlines the specific links between possible data products from the VIPER mission and their potential impact on the Endurance/Intrepid mission concepts in their early design phases. From VIPER, we will also gain insights for future development of onboard mobility software as VIPER is a way-point driven rover which requires more constant human operation on Earth than some other rover missions (non-lunar surface). Primary VIPER data products and insights of interest include wheel sinkage, wheel slip, wheel degradation, surface soil shear strength, and terrain slope estimation. VIPER data products will be made available via the Planetary Data System’s Geosciences and Cartography and Imaging Sciences nodes, conforming to the PDS4 standard.

mobility↗
DOE OSTI2024

HERMES

HERMES - High-speed Event Retrieval and Management for Enhanced Spectral imaging code. This code is meant to unpack and process neutron imaging data from the TPX3Cam made by Amsterdam Scientific Instruments. The TPX3Cam utilizes the Timepix3 chip in a single photon counting mode of image acquisition. The photon counting data streaming off the TPX3Cam needs to be process and analyzed in order to create final images. Therefore we are developing both python and cpp codes to allow for users of the TPX3Cam to efficiently analyze data and create images.

Long, Alexander↗
DOE OSTI2021

Data Reduction for Science: Brochure from the Advanced Scientific Computing Research Workshop

Data reduction for science holds promise for addressing the challenges of moving, storing, and processing massive data sets produced by the scientific community. Pursuing the PRDs outlined here will enable advances in data streaming, fast feedback and/or autonomous control of experiments, and faster time to scientific insight. These advances will result in a significant improvement in the ability to transport, store, process and interpret experimental, observational, and computational data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Status of the data acquisition, trigger, and slow control systems of the Mu2e experiment at Fermilab

The Mu2e experiment at the Fermilab will search for a coherent neutrinoless conversion of a muon into an electron in the field of an aluminum nucleus with a sensitivity improvement by a factor of 10,000 over existing limits. In this work, the Mu2e Trigger and Data Acquisition System (TDAQ) uses otsdaq framework as the online Data Acquisition System (DAQ) solution. Developed at Fermilab, otsdaq integrates several framework components — an artdaq-based DAQ, an art-based event processing, and an EPICS-based detector control system (DCS), and provides a uniform multi-user interface to its components through a web browser. Data streams from the Mu2e tracker and calorimeter are handled by the artdaq-based DAQ and processed by a one-level software trigger implemented within the art framework. Events accepted by the trigger have their data combined, post-trigger, with the separately read out data from the Mu2e Cosmic Ray Veto system. Foundation of the Mu2e DCS, EPICS – an Experimental Physics and Industrial Control System – is an open-source platform for monitoring, controlling, alarming, and archiving. A prototype of the TDAQ and the DCS systems has been built and tested over the last three years at Fermilab’s Feynman Computing Center, and now the production system installation is underway. This work presents their status and focus on the installation plans and procedures for racks, workstations, network switches, gateway computers, DAQ hardware, slow controls implementation, and testing.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Unsupervised anomaly clustering via offset alignment in multivariate grid sensing data

Modern industries increasingly rely on multi-sensor technologies to acquire complex, high-dimensional data streams, enabling advanced monitoring and control systems. One critical application is online anomaly detection in electrical smart grids, where multivariate and multimodal sensing technologies play a vital role. However, detecting anomalies in such time-series data is challenging due to their inherent temporal dependencies and stochastic behavior. Traditional approaches based on supervised and semi-supervised learning methods depend on labeled datasets, which are often unavailable in real-world scenarios. While unsupervised methods have emerged as promising alternatives, these methods are highly susceptible to noise and outliers commonly present in sensing applications. Furthermore, deep learning-based anomaly detection methods, despite their performance, are often criticized for their black-box nature, limiting their applicability in safety-critical and online environments where interpretability and explainability are paramount. In this work, we propose an unsupervised anomaly clustering method leveraging a cyclic alignment-based offset detection algorithm for multivariate time-series signals. The proposed method is applied to multivariate data collected from vibrational, voltage, and magnetic field sensors deployed in a local grid substation. Our results demonstrate the robustness of the algorithm in accurately clustering various anomalies/events across different sensing modalities. Additionally, we compare the effectiveness of the proposed approach against a simple pattern-based anomaly detection method, which performs well for univariate data but fails to generalize to multivariate and multimodal time-series data.

Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗
DOE OSTI2024

Slow control and TDAQ systems installation and tests in the Mu2e experiment

The Mu2e experiment at Fermilab will attempt to detect a coherent neutrinoless conversion of a muon into an electron in the field of an aluminum nucleus, with a sensitivity that is 10,000 times greater than existing limits. The Mu2e trigger and data acquisition system (TDAQ) uses the otsdaqframework as its online Data Acquisition System (DAQ) solution. Developed at Fermilab, otsdaq integrates several components, such as an artdaq-based DAQ, an art-based event processing, and an EPICS-based detector control system (DCS), and provides a uniform multi-user interface toits components through a web browser. The data streams from the Mu2e tracker and calorimeter are handled by the artdaq-based DAQ and processed by a one-level software trigger implemented within the art framework. Events accepted by the trigger have their data combined, post-trigger, with the separately read-out data from the Mu2e Cosmic Ray Veto system. The foundation of Mu2e DCS, EPICS, an Experimental Physics and Industrial Control System, is an open-source platform for monitoring, controlling, alarming, and archiving. Over the last three years, a prototype ofthe TDAQ and DCS systems has been built and tested at Fermilab’s Feynman Computing Center.Currently, the production system installation is underway. At the end, this work presents a brief update on the installation of racks and DAQ hardware.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1987

Program for Automated Real-Time Monitoring

Process-control decisions made by applying rules to process data streams. Expert System Executive computer program demonstrates feasibility of automated real-time monitoring. Using Executives knowledge of system formally coded as set of rules specifying actions taken under various conditions. Program makes inferences about observed data based on rules. Written in C language.

Goodrum, Glenn R.↗
DOE OSTI2023

Operational Focused Data Analytics for Optimizing Radiation Portal Monitor-Based Nuclear Smuggling Detection Systems at Global Ports of Entry

The National Nuclear Security Administration’s Office of Nuclear Smuggling Detection and Deterrence has deployed a fleet of radiation portal monitors (RPMs) across the world at global ports of entry including seaports, airports, and land border crossings. These RPMs are integrated into radiation detection systems (RDS) that also include fixed cameras, optical character recognition (OCR) systems, primary scanning systems (e.g., X-ray or gamma-ray), and secondary scanning systems (e.g., spectroscopic radiation portal monitors, portable radiation detection systems). The data from these sensing technologies is collected at the Central Alarm Station (CAS) where servers and computers reside to control and operate the system. Operators utilize the data collected by the CAS and declared cargo information to make decisions on how to respond to an alarm.This work explores the use of CAS-located data, looking at both the sensor data streams and operator inputs, to perform analysis which supports customs and border protection agencies to improve training capability and operational effectiveness. We focus on analyzing site level effectiveness and behavior by rolling up CAS-located data collected from individual occurrences. To-date, more than 15 sites (e.g., seaports, airports, border crossings) have been analyzed in this manner with the goal of understanding system operations to verify effectiveness and recommend potential improvements. This work first aims to provide background information on relevant CAS-located data sources and our current operational system analytics process including example results. After summarizing our current analytic techniques, we discuss how the future data analytics systems can provide key benefits to improving operational performance while minimizing the burden these detection systems place on operators.

Kuhn, Michael↗
NASA NTRS1975

Impact of remote sensing upon the planning, management, and development of water resources

Principal water resources users were surveyed to determine the impact of remote data streams on hydrologic computer models. Analysis of responses demonstrated that: most water resources effort suitable to remote sensing inputs is conducted through federal agencies or through federally stimulated research; and, most hydrologic models suitable to remote sensing data are federally developed. Computer usage by major water resources users was analyzed to determine the trends of usage and costs for the principal hydrologic users/models. The laws and empirical relationships governing the growth of the data processing loads were described and applied to project the future data loads. Data loads for ERTS CCT image processing were computed and projected through the 1985 era.

Castruccio, P. A.↗
NASA NTRS1990

A simple low-cost array on the lunar near-side for the early lunar expeditions

A concept of a simple, low-cost array on the lunar near-side is discussed. The concept includes 19 stations laid in a T, with three arms each 36 km long, and separated by 6 km. Each station consists of two 10-m-long orthogonal dipoles, each feeding a receiver of a commercial pocket digital shortwave radio type, and a 3-m spring-loaded mast with a small UHF array at the top for receiving the timing signals and tuning commmands and for sending out the digitized data streams. Each station has 0.25 square meters of solar arrays and 20 kg of lithium batteries. At the array center there is either a digital correlator for processing the signals from each antenna pair to compact the data into a 32-kb/sec channel or a 10-cm telescope for sending back raw data to earth.

Kuiper, T. B. H.↗
DOE OSTI2021

Hysteresis Patterns of Watershed Nitrogen Retention and Loss Over the Past 50 years in United States Hydrological Basins

Patterns of watershed nitrogen (N) retention and loss are shaped by how watershed biogeochemical processes retain, biogeochemically transform, and lose incoming atmospheric deposition of N. Loss patterns represented by concentration, discharge, and their associated stream exports are important indicators of integrated watershed N retention behaviors. We examined continental United States (CONUS) scale N deposition (e.g., wet and dry atmospheric deposition), vegetation trends, and stream trends as potential indicators of watershed N-saturation and retention conditions, and how watershed N retention and losses vary over space and time. By synthesizing changes and modalities in watershed nitrogen loss patterns based on stream data from 2200 U.S. watersheds over a 50 years record, our work revealed two patterns of watershed N-retention and loss. One was a hysteresis pattern that reflects the integrated influence of hydrology, atmospheric inputs, land-use, stream temperature, elevation, and vegetation. The other pattern was a one-way transition to a new state. We found that regions with increasing atmospheric deposition and increasing vegetation health/biomass patterns have the highest N-retention capacity, become increasingly N-saturated over time, and are associated with the strongest declines in stream N exports—a pattern, that is, consistent across all land cover categories. We provide a conceptual model, validated at an unprecedented scale across the CONUS that links instream nitrogen signals to upstream mechanistic landscape processes. Our work can aid in the future interpretation of in-stream concentrations of DOC and DIN as indicators of watershed N-retention status and integrators of watershed hydrobiogeochemical processes.

58 GEOSCIENCES↗
DOE OSTI2023

Quillinan, et al 2018 DOE Geothermal Technology Office REE Report for NEWTS Database and Case Studies

Produced water data processed into the NEWTS data format for easy input into aqueous chemistry modeling software, including oil & gas and coal bed methane produced waters, and geothermal waters. Includes information on rare earth elements (REEs) and critical minerals (CMs). Case studies are included to demonstrate solved streams using aqueous chemistry software. An input template is provided for modeling stream data in OLI Studio. Original data from: Quillinan, Scott, Nye, Charles, Engle, Mark, Bartos, Timothy T., Neupane, Ghanashyam, Brant, Jonathan, Bagdonas, Davin, McLing, Travis, McLaughlin, J. Fred, Phillips, Erin, Hallberg, Laura L., Shahabadi, Mahdi, and Johnson, Matthew. Assessing rare earth element concentrations in geothermal and oil and gas produced waters: A potential domestic source of strategic mineral commodities (Final Report). United States: N. p., 2018. Web. doi:10.2172/1509037.

Aqueous Chemistry↗
NASA NTRS1984

Revised Radiometric Calibration Technique for LANDSAT-4 Thematic Mapper Data by the Canada Centre for Remote Sensing

A technique for the radiometric correction of LANDSAT-4 Thematic Mapper data was proposed by the Canada Center for Remote Sensing. Subsequent detailed observations of raw image data, raw radiometric calibration data and background measurements extracted from the raw data stream on High Density Tape highlighted major shortcomings in the proposed method which if left uncorrected, can cause severe radiometric striping in the output product. Results are presented which correlate measurements of the DC background with variations in both image data background and calibration samples. The effect on both raw data and on data corrected using the earlier proposed technique is explained, and the correction required for these factors as a function of individual scan line number for each detector is described. It is shown how the revised technique can be incorporated into an operational environment.

Murphy, J.↗