Search NASA⌕ Search

SEARCH · Search NASA

Results for “Streaming Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Simple and Scalable Streaming: The GRETA Data Pipeline

The Gamma Ray Energy Tracking Array (GRETA) is a state of the art gamma-ray spectrometer being built at Lawrence Berkeley National Laboratory to be first sited at the Facility for Rare Isotope Beams (FRIB) at Michigan State University. A key design requirement for the spectrometer is to perform gamma-ray tracking in near real time. To meet this requirement we have used an inline, streaming approach to signal processing in the GRETA data acquisition system, using a GPU-equipped computing cluster. The data stream will reach 480 thousand events per second at an aggregate data rate of 4 gigabytes per second at full design capacity. We have been able to simplify the architecture of the streaming system greatly by interfacing the FPGA-based detector electronics with the computing cluster using standard network technology. A set of highperformance software components to implement queuing, flow control, event processing and event building have been developed, all in a streaming environment which matches detector performance. Prototypes of all high-performance components have been completed and meet design specifications.

Cromaz, Mario↗
DOE OSTI2021

Real-time nuclear activation detectors for measuring neutron angular distributions at the National Ignition Facility (invited)

The Real Time Nuclear Activation Detector (RTNAD) array at NIF measures the distribution of 14 MeV neutrons emitted by deuterium-tritium (DT) fueled inertial confinement fusion implosions. The uniformity of the neutron distribution is an important indication of implosion symmetry and DT shell integrity. The array consists of 48 LaBr 3 (Ce) crystal gamma-ray spectrometers mounted outside the NIF target chamber, which continuously monitor the slow decay of the 909 keV gamma-ray line from activated 89 Zr located in Zr cups surrounding each crystal. The measured decay rate dramatically increases during a DT implosion in proportion to the number of 14 MeV neutrons striking each Zr cup. The neutrons produce activated 89 Zr through an (n, 2n) reaction on 90 Zr, which is insensitive to low energy neutrons. The neutron flux along the detector line-of-sight at shot time is determined by extrapolating the fitted 909 keV decay curve back to shot time. Automatic analysis algorithms were developed to handle the non-stop data stream. The large number of detectors and the high statistical accuracy of the array enable the spherical harmonic modes of the neutron angular distribution to be measured up to L ≤ 4 to provide a better understanding of implosion dynamics. In addition, these data combined with measurements of the down-scattered neutrons can be used to derive fuel areal density distributions. This paper will describe the RTNAD hardware and analysis procedures.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Multimodal sensor fusion framework for residential building occupancy detection

For several years now, smart building energy systems have been a research area of intensive activity. In light of the increasing need for sustainable buildings and energy systems, this trend motivates an increasing need for a solution to reduce carbon dioxide emissions and improve energy efficiency. This work proposes a high-performing and transferable occupancy detection framework that combines sensor data from different data modalities, including time series environmental data (temperature, humidity, and illuminance), image data, and acoustic energy data using ensemble method. To draw out the best prediction performance in each modality, the proposed framework was developed, including various models that were designed to learn the occupancy patterns reflected in the physical data streams. To tackle the time series environmental data, we designed two variants of an occupancy detection spatiotemporal pattern network (Occ-STPN) that performs both feature level and decision level fusion, respectively. We also propose a new metric; the fading memory mean square error (FMMSE), that provides a fair evaluation and penalization of delayed occupancy predictions. Multiple open-sourced datasets, including the Electricity Consumption and Occupancy and the University of California, Irvine's (UCI) building occupancy detection dataset, along with our own real data collected from six different houses, were used to validate the algorithms' performance. The experimental results presented herein break down the performance for each sensing modality, and a detailed analysis of the performance is also discussed.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Nuclear data uncertainty propagation and modeling uncertainty impact evaluation in neutronics core simulation

Uncertainty analysis is a critical requirement in reactor simulation as it is used to quantify the reliability of best-estimate calculation. A comprehensive uncertainty analysis should characterize all sources of uncertainties in a computationally-feasible and scientifically-defendable manner. Here we employ a well-established reduced order modeling (ROM) based uncertainty quantification methodology to propagate uncertainties throughout neutronic calculations. ROM relies on recent advances in randomized data mining techniques applied to large data streams. In our proposed implementation, the nuclear data uncertainties are first propagated from multi-group level through lattice physics calculation to generate few-group parameter uncertainties, described using a vector of mean values and a covariance matrix. Employing an ROM-based compression of the covariance matrix, the few-group uncertainties are then propagated through downstream core simulation in a computationally efficient manner. This straightforward approach, albeit efficient as compared to brute force forward and/or adjoint-based methods, often employs a number of assumptions that have been unquestioned in the literature of neutronic uncertainty analysis. This manuscript argues that these assumptions could introduce another source of uncertainty referred to as modeling uncertainties, whose magnitude needs to be quantified in tandem with nuclear data uncertainties. Thus, our primary goal is to explore the interactions between these two uncertainty sources in order to assess whether modeling uncertainties have an impact on parameter uncertainties. To explore this endeavor, the impact of a number of modeling assumptions on core attributes uncertainties is quantified. The study employs a CANDU reactor model, with Serpent and NEWT as lattice physics solvers and NESTLE-C as core simulator. The modeling assumptions investigated include those related with the uncertainty propagation method employed, e.g., deterministic vs. stochastic, the few-group energy structure employed to represent the cross-sections, the resonance treatment in lattice physics calculation, the reference values for the cross-section, and the number of samples employed to render ROM compression. Results indicate that some of the modeling assumptions could have a non-negligible impact on the core responses propagated uncertainties, highlighting the need for a more comprehensive approach to combine parameter and modeling uncertainties.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2019

GNU Radio Sandia Utilities v. 1.6

This software adds a data handling module to the GNU Radio (GR) software defined radio (SDR) framework as well as some general-purpose function blocks (filters, metadata control, etc). This software is useful for processing bursty RF transmissions with GR, and serves as a base for applying SDR signal processing techniques to a whole burst of data at a time, as opposed to streaming data which GR.

Gilbert, Jacob↗
DOE OSTI2017

SmartBlock v. 1.0

SmartBlock offers reusable components that can provide data manipulation and transformation operations on different data streams without requiring recompilation for the new data types.

Champsaur, Alexis↗
DOE OSTI2020

Wattile: Probabilistic Deep Learning-based Forecasting of Building Energy Consumption [SWR-20-94]

Accurate energy forecasting is becoming critical due to many reasons: i ) optimal distributed energy resources operations and dispatch, ii) fault detection and diagnostics, and iii) meeting operational energy efficiency targets. Wattile uses deep learning (DL) for the building's short-term load forecasting application. Two specific types of neural networks called, Long Short Term Memory (LSTM) and Sequence-to-Sequence (S2S) models are used to make predictions. Forecasting models are trained using online historical weather and occupancy indicator data streams from the Intelligent Campus Program's data acquisition systems at the National Renewable Energy Laboratory (NREL) for main meters and sub-meters of multiple building types. These models use probabilistic methods to provide quantile-based forecasts in addition to nominal conditional median predictions of electricity consumption.

Frank, Stephen↗
NASA NTRS1999

Science Quality Assessment in Support of EOS MISR Data

The Multi-angle Imaging SpectroRadiometer (MISR) instrument, scheduled to fly on board the first Earth Observing System (EOS) spacecraft, will have a data stream that produces over 100 GB of data per day, once in full operation, of information about the Earth's atmosphere, clouds, and surface.

remote sensing Multi-angle Imaging SpectroRadiomet↗
NASA NTRS2012

NASA GSFC Space Weather Center - Innovative Space Weather Dissemination: Web-Interfaces, Mobile Applications, and More

The NASA GSFC Space Weather Center (http://swc.gsfc.nasa.gov) is committed to providing forecasts, alerts, research, and educational support to address NASA's space weather needs - in addition to the needs of the general space weather community. We provide a host of services including spacecraft anomaly resolution, historical impact analysis, real-time monitoring and forecasting, custom space weather alerts and products, weekly summaries and reports, and most recently - video casts. There are many challenges in providing accurate descriptions of past, present, and expected space weather events - and the Space Weather Center at NASA GSFC employs several innovative solutions to provide access to a comprehensive collection of both observational data, as well as space weather model/simulation data. We'll describe the challenges we've faced with managing hundreds of data streams, running models in real-time, data storage, and data dissemination. We'll also highlight several systems and tools that are utilized by the Space Weather Center in our daily operations, all of which are available to the general community as well. These systems and services include a web-based application called the Integrated Space Weather Analysis System (iSWA http://iswa.gsfc.nasa.gov), two mobile space weather applications for both IOS and Android devices, an external API for web-service style access to data, google earth compatible data products, and a downloadable client-based visualization tool.

Maddox, Marlo↗
NASA NTRS2016

Laser Remote Sensing from ISS: CATS Cloud and Aerosol Level 2 Data Products (Heritage Edition)

With the recent launch of the Cloud-Aerosol Transport System (CATS) we have the opportunity to acquire a continuous record of space based lidar measurements spanning from the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) era to the start of the EarthCARE mission. Utilizing existing well-validated science algorithms from the CALIPSO mission, we will ingest the CATS data stream and deliver high-quality lidar data sets to the user community at the earliest possible opportunity. In this paper we present an overview of procedures necessary to generate CALIPSO-like lidar level 2 data products from the CATS level 1 data products.

cloud-aerosol lidar↗
DOE OSTI2023

Data fusion enhanced multi-modality wellbore integrity inspection system

A downhole multi-modality inspection system includes a first imaging device operable to generate first imaging data and a second imaging device operable to generate second imaging data. The first imaging device includes a first source operable to emit energy of a first modality, and a first detector operable to detect returning energy induced by the emitted energy of the first modality. The second imaging device includes a second source operable to emit energy of a second modality, and a second detector operable to detect returning energy induced by the emitted energy of the second modality. The system further includes a processor configured to receive the first imaging data and the second imaging data, and integrate the first imaging data with the second imaging data into an enhanced data stream. The processor correlates the first imaging data and the second imaging data to provide enhanced data for detecting potential wellbore anomalies.

Kasten, Ansas Matthias↗
NASA NTRS2007

Wireless Data-Acquisition System for Testing Rocket Engines

A prototype wireless data-acquisition system has been developed as a potential replacement for a wired data-acquisition system heretofore used in testing rocket engines. The traditional use of wires to connect sensors, signal-conditioning circuits, and data acquisition circuitry is time-consuming and prone to error, especially when, as is often the case, many sensors are used in a test. The system includes one master and multiple slave nodes. The master node communicates with a computer via an Ethernet connection. The slave nodes are powered by rechargeable batteries and are packaged in weatherproof enclosures. The master unit and each of the slave units are equipped with a time-modulated ultra-wide-band (TMUWB) radio transceiver, which spreads its RF energy over several gigahertz by transmitting extremely low-power and super-narrow pulses. In this prototype system, each slave node can be connected to as many as six sensors: two sensors can be connected directly to analog-to-digital converters (ADCs) in the slave node and four sensors can be connected indirectly to the ADCs via signal conditioners. The maximum sampling rate for streaming data from any given sensor is about 5 kHz. The bandwidth of one channel of the TM-UWB radio communication system is sufficient to accommodate streaming of data from five slave nodes when they are fully loaded with data collected through all possible sensor connections. TM-UWB radios have a much higher spatial capacity than traditional sinusoidal wave-based radios. Hence, this TM-UWB wireless data-acquisition can be scaled to cover denser sensor setups for rocket engine test stands. Another advantage of TM-UWB radios is that it will not interfere with existing wireless transmission. The maximum radio-communication range between the master node and a slave node for this prototype system is about 50 ft (15 m) when the master and slave transceivers are equipped with small dipole antennas. The range can be increased by changing to larger antennas and/or greater transmission power. The battery life of a slave node ranges from about six hours during operation at full capacity to as long as three days when the system is in a "sleep" mode used to conserve battery charge during times between setup and rocket-engine testing. Batteries can be added to prolong operational lifetimes. The radio transceiver dominates the power consumption.

Lin, Chujen↗
DOE OSTI2022

FTICR-MS, Sensor, and Environmental Data from 5 Streams Impacted by the 2020 Holiday Farm Fire Associated with: "Spatiotemporal controls on the delivery of dissolved organic matter to streams following a wildfire"

This data package is associated with the publication "Spatiotemporal Controls on the Delivery of Dissolved Organic Matter to Streams Following a Wildfire" submitted to Geophysical Research Letters (Roebuck et al., 2022). The study aims to understand storm induced transport of pyrogenic materials to streams impacted by varying degrees of burn severity. Time series samples (24 samples in 1-hour intervals) were collected at 5 sites within the McKenzie River Watershed (Oregon, USA) whose catchment were each completely engulfed by the 2020 Holiday Farm Fire. The samples were collected in November 2020 during the first major storm pulse following the conclusion of the wildfire. Samples were characterized for dissolved organic carbon, total dissolved nitrogen, and by ultra-high resolution mass spectrometry. In situ turbidity data also collected.This data package contains 4 primary folders that include the following: 1) Metadata, 2) EnvData (Environmental Data), 3) SensorData, and 4) FTICR_SupportingData. The package contains a single file-level metadata (flmd) file. Each primary folder also contains individual data dictionaries (dd) to define and provide descriptors of column/row headers and data flags. The FTICR_SupportingData, folder 4, contains raw, unprocessed FTICR-MS Data files in addition to a csv containing processed FTICR-MS data. This package contains the following file types: csv, xml, pdf.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2013

Discrete Data Qualification System and Method Comprising Noise Series Fault Detection

A Sensor Data Qualification (SDQ) function has been developed that allows the onboard flight computers on NASA s launch vehicles to determine the validity of sensor data to ensure that critical safety and operational decisions are not based on faulty sensor data. This SDQ function includes a novel noise series fault detection algorithm for qualification of the output data from LO2 and LH2 low-level liquid sensors. These sensors are positioned in a launch vehicle s propellant tanks in order to detect propellant depletion during a rocket engine s boost operating phase. This detection capability can prevent the catastrophic situation where the engine operates without propellant. The output from each LO2 and LH2 low-level liquid sensor is a discrete valued signal that is expected to be in either of two states, depending on whether the sensor is immersed (wet) or exposed (dry). Conventional methods for sensor data qualification, such as threshold limit checking, are not effective for this type of signal due to its discrete binary-state nature. To address this data qualification challenge, a noise computation and evaluation method, also known as a noise fault detector, was developed to detect unreasonable statistical characteristics in the discrete data stream. The method operates on a time series of discrete data observations over a moving window of data points and performs a continuous examination of the resulting observation stream to identify the presence of anomalous characteristics. If the method determines the existence of anomalous results, the data from the sensor is disqualified for use by other monitoring or control functions.

Fulton, Christopher↗
DOE OSTI2025

Real-time data processing for serial crystallography experiments

We report the use of streaming data interfaces to perform fully online data processing for serial crystallography experiments, without storing intermediate data on disk. The system produces Bragg reflection intensity measurements suitable for scaling and merging, with a latency of less than 1 s per frame. Our system uses the CrystFEL software in combination with the ASAP::O data framework. In a series of user experiments at PETRA III, frames from a 16 megapixel Dectris EIGER2 X detector were searched for peaks, indexed and integrated at the maximum full-frame readout speed of 133 frames per second. The computational resources required depend on various factors, most significantly the fraction of non-blank frames ('hits'). The average single-thread processing time per frame was 242 ms for blank frames and 455 ms for hits, meaning that a single 96-core computing node was sufficient to keep up with the data, with ample headroom for unexpected throughput reductions. Further significant improvements are expected, for example by binning pixel intensities together to reduce the pixel count. We discuss the implications of real-time data processing on the `data deluge' problem from recent and future photon-science experiments, in particular on calibration requirements, computing access patterns and the need for the preservation of raw data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Evaluating lightweight unsupervised online IDS for masquerade attacks in CAN

Vehicular controller area networks (CANs) are susceptible to masquerade attacks by malicious adversaries. In masquerade attacks, adversaries silence a targeted ID and then send malicious frames with forged content at the expected timing of benign frames. As masquerade attacks could seriously harm vehicle functionality and are the stealthiest attacks to detect in CAN, recent work has devoted attention to compare frameworks for detecting masquerade attacks in CAN. However, most existing works report offline evaluations using CAN logs already collected using simulations that do not comply with the domain’s real-time constraints. Here we contribute to advance the state of the art by presenting a comparative evaluation of four different non-deep learning (DL)-based unsupervised online intrusion detection systems (IDS) for masquerade attacks in CAN. Our approach differs from existing comparative evaluations in that we analyze the effect of controlling streaming data conditions in a sliding window setting. In doing so, we use realistic masquerade attacks being replayed from the ROAD dataset. We show that although evaluated IDS are not effective at detecting every attack type, the method that relies on detecting changes in the hierarchical structure of clusters of time series produces the best results at the expense of higher computational overhead. We discuss limitations, open challenges, and how the evaluated methods can be used for practical unsupervised online CAN IDS for masquerade attacks.

Anomaly detection↗
NASA NTRSDate not supplied

Anomaly Detection in Flight Operational Data Using Deep Learning

In this session, we demonstrate two recently developed deep learning models for anomaly detection in flight operational data by the Data Sciences Group at NASA Ames Research Center. The first model is Convolutional Variational Auto-Encoder (CVAE) [1], which is an unsupervised deep encoder-decoder model, designed specifically for finding anomalies in heterogeneous multivariate time series data. We will demonstrate its application to finding anomalies in streaming data from NASA’s Digital Information Platform’s Fuser source. CVAE identifies data instances that are not representative of expected nominal behavior as anomalous. Since it is an unsupervised approach, the flagged anomalies will need to be reviewed by the subject matter experts (SMEs) for validation and labeling and is designed to assist with vulnerability discovery within Safety Monitoring System programs. The second model is Robust and Explainable Semi-supervised Anomaly Detection (RESAD) model [2], which builds on CVAE to allow learning from both minimally labeled data (previously reviewed by the SMEs) as well as majority unlabeled data. RESAD takes advantage of graph theoretic techniques to propagate the labels from the labeled data to the unlabeled data based on a pre-defined similarity metric and structures the learned feature space from flight time-series so that data of the same class would cluster tightly together. This model characteristic is enabled by training with an augmented loss function and allows learning of a more informative feature space for down-stream tasks such as search and active learning. We demonstrate RESAD using data from the NASA DASHlink project [3].

anomaly detection↗
DOE OSTI2019

Efficient Modeling of HIL Multi-Grid System for Scalability & Concurrency in CPS Security Testbed

Cyber-event-triggered power grid blackout compels utility operators to intensify cyber-aware and physics-constrained recovery and restoration process. Recently, coordinated cyber attacks on the Ukrainian grid witnessed such a cyber-event-triggered power system blackout. Various cyber-physical system (CPS) testbeds have attempted with multitude designs to analyze such interdependent events and evaluate remedy measures. However, resource constraints and modular integration designs have been significant barriers while modeling large-scale grid models (scalability) and multi-grid isolated models (concurrency) under a single real-time execution environment for the hardware-in-the-loop (HIL) CPS security testbeds. This paper proposes a meticulous design and effective modeling for simulating large-scale grid models and multi-grid isolated models in a HIL realtime digital simulator environment integrated with industry-grade hardware and software systems. We have used our existing HIL CPS security testbed to demonstrate scalability by the realtime performance of a Texas-2000 bus US synthetic grid model and concurrency by the real-time performance of simultaneous ten IEEE-39 bus grid models and an IEEE-118 bus grid model. The experiments demonstrated significant results by 100% realtime performance with zero overruns, low latency while receiving and executing control signals from SEL Relays via IEC-61850 protocol and low latency while computing and transmitting grid data streams including stability measures via IEEE C37.118 synchrophasor data protocol to SEL Phasor Data Concentrators.

24 POWER TRANSMISSION AND DISTRIBUTION↗