Search NASA⌕ Search

SEARCH · Search NASA

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Selecting a Classification Ensemble and Detecting Process Drift in an Evolving Data Stream

We characterize the commercial behavior of a group of companies in a common line of business using a small ensemble of classifiers on a stream of records containing commercial activity information. This approach is able to effectively find a subset of classifiers that can be used to predict company labels with reasonable accuracy. Performance of the ensemble, its error rate under stable conditions, can be characterized using an exponentially weighted moving average (EWMA) statistic. The behavior of the EWMA statistic can be used to monitor a record stream from the commercial network and determine when significant changes have occurred. Results indicate that larger classification ensembles may not necessarily be optimal, pointing to the need to search the combinatorial classifier space in a systematic way. Results also show that current and past performance of an ensemble can be used to detect when statistically significant changes in the activity of the network have occurred. The dataset used in this work contains tens of thousands of high level commercial activity records with continuous and categorical variables and hundreds of labels, making classification challenging.

Ensemble classifiers↗
DOE OSTI2024

ML Classifier Fusion for Three Data Streams with Quality Inversely Proportional to Time Resolution

We consider a monitoring scenario of phenomenon using three different streams of measurements whose quality is proportional to their constant inter-arrival times. Each measurement of a stream needs to be binary-classified to reflect the state of interest of the phenomenon. A set of classifiers is separately trained and fused for each stream at its time resolution using measurements collected under known states. We present a machine learning method to fuse the outputs of these fusers to provide a final classification at the finest time resolution. We show that this fused-fusers method provides decisions with likely superior classification probability compared to the best individual classifiers and fused-classifiers. We derive generalization equations that guarantee a superior classification probability of fused-fusers with a confidence probability specified by the classifiers’ generalization equations. We apply these results to study a practical problem of classifying Pu/Np target dissolution events at a radiochemical processing facility using gamma spectral measurements of effluent flows.

Rao, Nageswara↗
DOE OSTI2023

Toward Statistical Real-Time Power Fault Detection

We propose statistical fault detection methodology based on high-frequency data streams that are becoming available in modern power grids. Our approach can be treated as an online (sequential) change point monitoring methodology. However, due to the mostly unexplored and very nonstandard structure of high-frequency power grid streaming data, substantial new statistical development is required to make this methodology practically applicable. The paper includes development of scalar detectors based on multichannel data streams, determination of data-driven alarm thresholds and investigation of the performance and robustness of the new tools. Due to a reasonably large database of faults, we can calculate frequencies of false and correct fault signals, and recommend implementations that optimize these empirical success rates.

bolted faults↗
DOE OSTI2023

A Three-Tier Incremental Approach for Development of Smart Corridor Digital Twins

Development of an arterial smart corridor digital twin requires the integration of real time data streams, data storage, and simulation model execution. This process may be complex, time consuming, and susceptible to errors. To aid in smart corridor digital twin development this paper seeks to provide a framework, best practices, and development guidance. As such, a three-tier incremental approach to smart corridor digital twin development is presented. The paper highlights practical issues in digital twin construction along with key data challenges based on experiences from the development of digital twins for two large Smart Corridor deployments, one in Chattanooga, Tennessee, and the other in Atlanta, Georgia. The presented three-tier incremental approach includes: 1) development of a prepopulated offline simulation, 2) development of a pseudo digital twin that is driven by archived data, and 3) integration of real time data streams to create the online digital twin model. The three-tiered approach facilitates conducting multiple trials and scenarios with increasing complexity, allowing for incremental error processing and updating of the digital twin.

Saroj, Abhilasha↗
DOE OSTI2010

Leveraging The Open Provenance Model as a Multi-Tier Model for Global Climate Research

Global climate researchers rely upon many forms of sensor data and analytical methods to help profile subtle changes in climate conditions. The U.S. Department of Energy's Atmospheric Radiation Measurement (ARM) program provides researchers with curated Value Added Products (VAPs) resulting from continuous sensor data streams, data fusion, and modeling. The ARM operations staff and software development teams (data producers) rely upon a number of techniques to ensure strict quality control (QC) and quality assurance (QA) standards are maintained. Climate researchers (data consumers) are highly interested in obtaining as much provenance (data quality, data pedigree) as possible to establish data trustworthiness. Currently all the provenance is not easily attainable or identifiable without significant efforts to extract and piece together information from configuration files, log files, codes, and status information from ARM databases. The need for a formalized approach to managing provenance became paramount with the planned addition of 120 new instruments, new data products, and data collection scaling to half a terabyte daily. Last year our research identified the need for a multi-tier provenance model to enable the data consumer easy access to the provenance for their data. This year we are leveraging the Open Provenance Model as a foundational construct that serves the needs of both the VAP producers and consumers, we are organizing the provenance in different tiers of granularity to model VAP lineage, causality at the component level within a VAP, and the causality for each time step as samples are being assembled within the VAP. This paper shares our implementation strategy and how the ARM operations staff and the climate research community can greatly benefit from this approach to more effectively assess and quantify VAP provenance.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Evolutionary NAS in Light of Model Stability for Accurate Continual Learning

Continual learning, the capability to learn new knowledge from streaming data without forgetting the previous knowledge, is a critical requirement for dynamic learning systems, especially for emerging edge devices such as self-driving cars and drones. However, continual learning is still facing the catastrophic forgetting problem. Previous work illustrate that model performance on continual learning is not only related to the learning algorithms but also strongly dependent on the inherited model, i.e., the model where continual learning starts. The better stability of the inherited model, the less catastrophic forgetting and thus, the inherited model should be elaborately selected. Inspired by this finding, we develop an evolutionary neural architecture search (ENAS) algorithm that emphasizes the Stability of the inherited model, namely ENAS-S. ENAS-S aims to find optimal architectures for accurate continual learning on edge devices. On CIFAR-10 and CIFAR-100, we present that ENAS-S achieves competitive architectures with lower catastrophic forgetting and smaller model size when learning from a data stream, as compared with handcrafted DNNs.

Du, Xiaocong↗
DOE OSTI2011

System using data compression and hashing adapted for use for multimedia encryption

A system and method is disclosed for multimedia encryption. Within the system of the present invention, a data compression module receives and compresses a media signal into a compressed data stream. A data acquisition module receives and selects a set of data from the compressed data stream. And, a hashing module receives and hashes the set of data into a keyword. The method of the present invention includes the steps of compressing a media signal into a compressed data stream; selecting a set of data from the compressed data stream; and hashing the set of data into a keyword.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Characterizing the GD-1 Stream with DESI DR2 Data: Thin Stream and Hot Cocoon

GD-1 is among the longest, coldest stellar streams in the Milky Way, making it an ideal target for probing dark matter substructure through dynamical heating. We present a catalog of 608 spectroscopically confirmed GD-1 members from the first three years of Dark Energy Spectroscopic Instrument (DESI) observations. This constitutes the largest homogeneous spectroscopic sample of GD-1, doubling the number of members previously available only through heterogeneous compilations combining multiple surveys with different systematics. Using these data, we derive updated stream tracks in sky position, proper motion, and radial velocity that extend over $100^\circ$ of the stream. We apply a Gaussian mixture model to decompose the stream into a dynamically cold thin component ($σ_V = 2.49\pm 0.28$ km s$^{-1}$, width $= 0.23\pm0.01^\circ$) and a kinematically hot cocoon ($σ_V = 6.13\pm0.75$ km s$^{-1}$, width $= 2.18\pm0.17^\circ$). The cocoon contains $\sim30\%$ of members and its velocity dispersion is consistent with $\sim11$ Gyr of heating by cold dark matter subhalos. We also detect a large proper motion dispersion ($41.36\pm4.98$ km s$^{-1}$) along the stream direction in the cocoon component. This feature indicates a significant line-of-sight distance spread in the cocoon, and its origin will be further explored in a forthcoming paper. These measurements demonstrate the power of DESI spectroscopy for characterizing the multi-component phase-space structure of stellar streams and constraining small-scale dark matter substructure.

Jarvis, Emma [Toronto U.] (ORCID:0009000656127336)↗
DOE OSTI2020

SAMPA Based Streaming Readout Data Acquisition Prototype

We have assembled a small-scale streaming data acquisition system based on the SAMPA front-end ASIC. The 32-channel SAMPA chip was designed for the high-luminosity upgrade of the ALICE Time Projection Chamber (TPC) detector at the CERN Large Hadron Collider. The goals of the prototype system are to determine if the SAMPA chip is appropriate for use in detector systems at Jefferson Lab, and to gain experience with the hardware and software required to deploy streaming data acquisition systems in nuclear physics experiments. The 800 channel system is composed of components used in the ALICE TPC data acquisition upgrade. Five front-end cards (FEC) support five SAMPA chips each. SAMPA data streams on an FEC are concentrated into two high-speed (4.48 Gb/s) serial data streams by a pair Gigabit Transceiver ASICs (GBTx). These ten streams (44.8 Gb/s) are transmitted from the FECs over fibers to a PCIe based readout unit. The FPGA engine on the readout unit compresses data for transmission to a server via 100 Gb ethernet. Components on the FECs are radiation tolerant. High data rates can be handled. The system is by design scalable and thus provides a functional prototype for high-rate streaming readout at Jefferson Lab and the future Electron Ion Collider. We have made fundamental measurements (noise, linearity, time resolution) on the SAMPA ASIC. We have also coupled the readout system to a small Gas Electron Multiplier (GEM) detector and have studied its response to cosmic rays. A beam test of the system is planned.

ABBOTT, David↗
DOE OSTI2016

Video Analysis in Multi-Intelligence

This is a project which was performed by a graduated high school student at Los Alamos National Laboratory (LANL). The goal of the Multi-intelligence (MINT) project is to determine the state of a facility from multiple data streams. The data streams are indirect observations. The researcher is using DARHT (Dual-Axis Radiographic Hydrodynamic Test Facility) as a proof of concept. In summary, videos from the DARHT facility contain a rich amount of information. Distribution of car activity can inform us about the state of the facility. Counting large vehicles shows promise as another feature for identifying the state of operations. Signal processing techniques are limited by the low resolution and compression of the videos. We are working on integrating these features with features obtained from other data streams to contribute to the MINT project. Future work can pursue other observations, such as when the gate is functioning or non-functioning.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Electronic device and method for compressing video data

A computer-implemented method for compressing video data comprises receiving a sequence of video data values, each video data value being a digital value from a successive one of a plurality of pixels that form a video sensor, the sequence of video data values resulting from successive frames of video captured by the video sensor; extracting the video data values for each pixel in turn to create a plurality of pixel data streams, each pixel data stream including the video data value for each frame of captured video for the pixel; and applying data compression to each pixel data stream to create compressed data for each pixel data stream.

Tohlen, Michael Aaron↗
DOE OSTI2023

Electronic device and method for compressing video data

A computer-implemented method for compressing video data comprises receiving a sequence of video data values, each video data value being a digital value from a successive one of a plurality of pixels that form a video sensor, the sequence of video data values resulting from successive frames of video captured by the video sensor; extracting the video data values for each pixel in turn to create a plurality of pixel data streams, each pixel data stream including the video data value for each frame of captured video for the pixel; and applying data compression to each pixel data stream to create compressed data for each pixel data stream.

Tohlen, Michael Aaron↗
DOE OSTI2016

Towards Cache-Enabled, Order-Aware, Ontology-Based Stream Reasoning Framework

While streaming data have become increasingly more popular in business and research communities, semantic models and processing software for streaming data have not kept pace. Traditional semantic solutions have not addressed transient data streams. Semantic web languages (e.g., RDF, OWL) have typically addressed static data settings and linked data approaches have predominantly addressed static or growing data repositories. Streaming data settings have some fundamental differences; in particular, data are consumed on the fly and data may expire. Stream reasoning, a combination of stream processing and semantic reasoning, has emerged with the vision of providing "smart" processing of streaming data. C-SPARQL is a prominent stream reasoning system that handles semantic (RDF) data streams. Many stream reasoning systems including C-SPARQL use a sliding window and use data arrival time to evict data. For data streams that include expiration times, a simple arrival time scheme is inadequate if the window size does not match the expiration period. In this paper, we propose a cache-enabled, order-aware, ontology-based stream reasoning framework. This framework consumes RDF streams with expiration timestamps assigned by the streaming source. Our framework utilizes both arrival and expiration timestamps in its cache eviction policies. In addition, we introduce the notion of "semantic importance" which aims to address the relevance of data to the expected reasoning, thus enabling the eviction algorithms to be more context- and reasoning-aware when choosing what data to maintain for question answering. We evaluate this framework by implementing three different prototypes and utilizing five metrics. The trade-offs of deploying the proposed framework are also discussed.

Data Discovery↗
DOE OSTI2021

Transmitter with fully re-assignable segments for reconfigurable FFE taps

Methods and systems of performing feed forward equalization (FFE) on data streams are described. A circuitry may generate staggered data streams from data streams of an input signal. The staggered data streams may include data in staggered unit intervals. The circuitry may include a plurality of segments. A segment may define a specific unit interval to carve the staggered data streams into one unit interval pulses positioned at the specific unit interval. The specific unit interval to carve the staggered data streams may indicate an assignment of the segment as one of a FFE pre tap, a FFE main tap, and a FFE post tap. The plurality of segments may be assigned to different FFE taps based on different clock signal selection defining different unit intervals to perform the carving. The plurality of segments may output respective one unit interval pulses to reproduce the input signal.

Toprak-Deniz, Zeynep↗
DOE OSTI2023

Data transmission framing

Techniques for framing data in various data transmission contexts are described. A data framing technique may include a transmitter sending a data stream including repeating bits in alternating forward and reverse order. A receiver of the data stream may fold the data stream, and correlate portions of the folded data stream for purposes of validating the data stream and/or identifying an ID in the data stream. In at least some instances, once the receiver validates the data stream, the receiver may accept payload accompanying the data stream.

Palmer, David↗