Search NASA⌕ Search

SEARCH · Search NASA

Results for “Streaming Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63
All sourcesNASA NTRSDOE OSTI
NASA NTRS2013

Cloud Absorption Radiometer Autonomous Navigation System - CANS

CAR (cloud absorption radiometer) acquires spatial reference data from host aircraft navigation systems. This poses various problems during CAR data reduction, including navigation data format, accuracy of position data, accuracy of airframe inertial data, and navigation data rate. Incorporating its own navigation system, which included GPS (Global Positioning System), roll axis inertia and rates, and three axis acceleration, CANS expedites data reduction and increases the accuracy of the CAR end data product. CANS provides a self-contained navigation system for the CAR, using inertial reference and GPS positional information. The intent of the software application was to correct the sensor with respect to aircraft roll in real time based upon inputs from a precision navigation sensor. In addition, the navigation information (including GPS position), attitude data, and sensor position details are all streamed to a remote system for recording and later analysis. CANS comprises a commercially available inertial navigation system with integral GPS capability (Attitude Heading Reference System AHRS) integrated into the CAR support structure and data system. The unit is attached to the bottom of the tripod support structure. The related GPS antenna is located on the P-3 radome immediately above the CAR. The AHRS unit provides a RS-232 data stream containing global position and inertial attitude and velocity data to the CAR, which is recorded concurrently with the CAR data. This independence from aircraft navigation input provides for position and inertial state data that accounts for very small changes in aircraft attitude and position, sensed at the CAR location as opposed to aircraft state sensors typically installed close to the aircraft center of gravity. More accurate positional data enables quicker CAR data reduction with better resolution. The CANS software operates in two modes: initialization/calibration and operational. In the initialization/calibration mode, the software aligns the precision navigation sensors and initializes the communications interfaces with the sensor and the remote computing system. It also monitors the navigation data state for quality and ensures that the system maintains the required fidelity for attitude and positional information. In the operational mode, the software runs at 12.5 Hz and gathers the required navigation/attitude data, computes the required sensor correction values, and then commands the sensor to the required roll correction. In this manner, the sensor will stay very near to vertical at all times, greatly improving the resulting collected data and imagery. CANS greatly improves quality of resulting imagery and data collected. In addition, the software component of the system outputs a concisely formatted, high-speed data stream that can be used for further science data processing. This precision, time-stamped data also can benefit other instruments on the same aircraft platform by providing extra information from the mission flight.

Kahle, Duncan↗
DOE OSTI2022

Random Forest Regressor-Based Approach for Detecting Fault Location and Duration in Power Systems

Power system failures or outages due to short-circuits or “faults” can result in long service interruptions leading to significant socio-economic consequences. It is critical for electrical utilities to quickly ascertain fault characteristics, including location, type, and duration, to reduce the service time of an outage. Existing fault detection mechanisms (relays and digital fault recorders) are slow to communicate the fault characteristics upstream to the substations and control centers for action to be taken quickly. Fortunately, due to availability of high-resolution phasor measurement units (PMUs), more event-driven solutions can be captured in real time. In this paper, we propose a data-driven approach for determining fault characteristics using samples of fault trajectories. A random forest regressor (RFR)-based model is used to detect real-time fault location and its duration simultaneously. This model is based on combining multiple uncorrelated trees with state-of-the-art boosting and aggregating techniques in order to obtain robust generalizations and greater accuracy without overfitting or underfitting. Four cases were studied to evaluate the performance of RFR: 1. Detecting fault location (case 1), 2. Predicting fault duration (case 2), 3. Handling missing data (case 3), and 4. Identifying fault location and length in a real-time streaming environment (case 4). A comparative analysis was conducted between the RFR algorithm and state-of-the-art models, including deep neural network, Hoeffding tree, neural network, support vector machine, decision tree, naive Bayesian, and K-nearest neighborhood. Experiments revealed that RFR consistently outperformed the other models in detection accuracy, prediction error, and processing time.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1948

Altitude-wind-tunnel investigation of Westinghouse 19B-2, 19B-8, and 19XB-1 jet-propulsion engines III : performance and windmilling drag characteristics

The performance characteristics of the 19B-8 and 19XB-1 turbojet engines and the windmilling drag characteristics of the 19B-8 engine were determined in the NACA Cleveland altitude wind tunnel. The 19B engine is one of the earliest experimental Westinghouse axial-flow engines. The 19XB-1 engine is an experimental prototype of the Westinghouse 19XB series, having a rated thrust of 1400 pounds. Improvements in performance and operational characteristics have resulted in the 19XB-2B engine with a rated thrust of 1600 pounds. The investigations were conducted on the 19B-8 engine at simulated altitudes from 5000 to 25,000 feet with various free-stream ram-pressure ratios and on the 19XB-1 engine at simulated altitudes from 5000 to 30,000 feet with approximately static free-stream conditions. Data for these two engines are presented to show the effect of altitude, free-stream ram-pressure ratio, and tail-pipe-nozzle area on engine performance.

Fleming, William A↗
NASA NTRS1992

The Extrapolation of Elementary Sequences

We study sequence extrapolation as a stream-learning problem. Input examples are a stream of data elements of the same type (integers, strings, etc.), and the problem is to construct a hypothesis that both explains the observed sequence of examples and extrapolates the rest of the stream. A primary objective -- and one that distinguishes this work from previous extrapolation algorithms -- is that the same algorithm be able to extrapolate sequences over a variety of different types, including integers, strings, and trees. We define a generous family of constructive data types, and define as our learning bias a stream language called elementary stream descriptions. We then give an algorithm that extrapolates elementary descriptions over constructive datatypes and prove that it learns correctly. For freely-generated types, we prove a polynomial time bound on descriptions of bounded complexity. An especially interesting feature of this work is the ability to provide quantitative measures of confidence in competing hypotheses, using a Bayesian model of prediction.

Laird, Philip↗
DOE OSTI2021

Streaming Readout of the CLAS12 Forward Tagger Using TriDAS and JANA2

An effort is underway to develop streaming readout data acquisition system for the CLAS12 detector in Jefferson Lab’s experimental Hall-B. Successful beam tests were performed in the spring and summer of 2020 using a 10GeV electron beam from Jefferson Lab’s CEBAF accelerator. The prototype system combined elements of the TriDAS and CODA data acquisition systems with the JANA2 analysis/reconstruction framework. This successfully merged components that included an FPGA stream source, a distributed hit processing system, and software plugins that allowed offline analysis written in C++ to be used for online event filtering. Details of the system design and performance are presented.

Ameli, Fabrizio↗
DOE OSTI2022

Automated System-wide Event Detection and Classification Using Machine Learning on Synchrophasor Data

As the number of phasor measurement units (PMUs) deployed in a power system increases, and their data volume streamed to the control canter intensifies, operators are facing challenges related to the analysis of such data, which need to be observed and responded to as the measurements are displayed in the Control Room. Humans are generally unable to process such large amount of data efficiently and rapidly. There is an apparent need for automated ways to analyze the data, extract actionable information about occurrence of specific events, and characterize the events quickly and cost effectively. This paper discusses the use of machine learning (ML) to facilitate such tasks by providing automated, highly computationally efficient, and cost-effective ways of extracting actionable information from synchrophasor big data in real-time. We developed Big Data Smart (BDSmart) ML-based prototype tool for the Control Room use that automatically analyses data properties from synchrophasor system measurements taken across the three grid Interconnections in the USA (Western, Eastern and ERCOT). The data collected from several hundreds of PMUs located across the Interconnections over a period of two years have been made available for our extensive study. As a result, we were able to identify a number of big data properties that influence how ML methodology is applied to select, develop, train and test the data models that can eventually be used for the tool implementation. The resulting set of candidate algorithms spans unsupervised, supervised, semi-supervised and transfer-learning approaches. Many ML techniques, such as decision trees, multinomial logistic regression, feed-forward neural networks, K-nearest neighbor, multiclass support vector machine, and single and multi-channel convolutional neural networks, are implemented, and their performance is examined. We offer the results from testing the data models. The novelty of our study is in the approaches for bad data detection and mitigation, selection of a simplified feature for event detection, and data label improvements. As a result, we came up with a list of recommendations for the utilities on how to improve the PMU recording practices to cater to the future ML applications aimed at automating the analysis of synchrophasor data.

Synchrophasors, Machine Learning, System-wide Even↗
DOE OSTI2018

A Novel Event Detection Method Using PMU Data with High Precision

To take full advantage of the considerably high reporting rate of phasor measurement units (PMU) data, this paper develops a novel PMU-based event detection methodology. Considering the huge amount of streaming PMU data, a data compression algorithm, swinging door trending (SDT), is first used to compress the PMU data and generate multiple compression intervals. Then dynamic programming is utilized to solve the optimization problem, which is recursively constituted by a score function. Based on predefined PMU event rules, dynamic programming merges adjacent compression intervals with the same slope direction. Finally, all the PMU event features are characterized. A conventional wavelet-based event detection method is compared with the developed dynamic programming based SDT (DPSDT) method. Numerical simulations on the real-time and synthetic PMU data show that the DPSDT method can accurately detect the start-time of an event and the event placement with relatively high precision. Furthermore, the PMU event features, including the magnitude and duration of strokes, are characterized.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

DELVE-ing into the Jet: A Thin Stellar Stream on a Retrograde Orbit at 30 kpc

We perform a detailed photometric and astrometric analysis of stars in the Jet stream using data from the first data release of the DECam Local Volume Exploration Survey DR1 and Gaia EDR3. We discover that the stream extends over ∼ 29° on the sky (increasing the known length by 18°), which is comparable to the kinematically cold Phoenix, ATLAS, and GD-1 streams. Using blue horizontal branch stars, we resolve a distance gradient along the Jet stream of 0.2 kpc deg$^{−1}$, with distances ranging from D $_{⊙}$ ∼ 27–34 kpc. We use natural splines to simultaneously fit the stream track, width, and intensity to quantitatively characterize density variations in the Jet stream, including a large gap, and identify substructure off the main track of the stream. Furthermore, we report the first measurement of the proper motion of the Jet stream and find that it is well aligned with the stream track, suggesting the stream has likely not been significantly perturbed perpendicular to the line of sight. Finally, we fit the stream with a dynamical model and find that it is on a retrograde orbit, and is well fit by a gravitational potential including the Milky Way and Large Magellanic Cloud. These results indicate the Jet stream is an excellent candidate for future studies with deeper photometry, astrometry, and spectroscopy to study the potential of the Milky Way and probe perturbations from baryonic and dark matter substructure.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2012

Revamping Spacecraft Operational Intelligence with Splunk

So what is Splunk? Instead of giving the technical details, which you can find online, I'll tell you what it did for me. Splunk slapped everything into one place, with one uniform format, and gave me the ability to forget about all these annoying details of where it is, how to parse it, and all that. Instead, I only need to interact with Splunk to find the data I need. This sounds simple and obvious, but it's surprising what you can do once you all of your data is indexed in one place. By having your data organized, querying becomes much easier. Let's say that I want to search telemetry for a sensor_name gtemp_1 h and to return all data that is at most five minutes old. And because Splunk can hook into a real ]time stream, this data will always be up-to-date. Extending the previous example, I can now aggregate all types of data into one view based in time. In this picture, I've got transaction logs, telemetry, and downlinked files all in one page, organized by time. Even though the raw data looks completely than this, I've defined interfaces that transform it into this uniform format. This gives me a more complete picture for the question what was the spacecraft doing at this particular time? And because querying data is simple, I can start with a big block of data and whiddle it down to what I need, rather than hunting around for the individual pieces of data that I need. When we have all the data we need, we can begin widdling down the data with Splunk's Unix-like search syntax. These three examples highlights my trial-and-error attempts to find large temperature changes. I begin by showing the first 5 temperatures, only to find that they're sorted chronologically, rather than from highest temperatures to lowest temperatures. The next line shows sorting temperatures by their values, but I find that that fs not really what I want either. I want to know the delta temperatures between readings. Looking through Splunk's user manual, I find the delta function, which lets me dynamically generate new information to use in my query. With that extra piece of information, I can now return only the telemetry readings where the temperature changed by at least 10. One other useful feature I'll mention is that all of these queries can be run through Splunk's API. So any scripting language you can think of can plug right in and make these queries. This gives us the ability to build a lot of new tools.

operational intelligence↗
NASA NTRS1994

Some Practical Universal Noiseless Coding Techniques

Report discusses noiseless data-compression-coding algorithms, performance characteristics and practical consideration in implementation of algorithms in coding modules composed of very-large-scale integrated circuits. Report also has value as tutorial document on data-compression-coding concepts. Coding techniques and concepts in question "universal" in sense that, in principle, applicable to streams of data from variety of sources. However, discussion oriented toward compression of high-rate data generated by spaceborne sensors for lower-rate transmission back to earth.

Rice, Robert F.↗
NASA NTRS1995

An approximate Kalman filter for ocean data assimilation: An example with an idealized Gulf Stream model

A practical method of data assimilation for use with large, nonlinear, ocean general circulation models is explored. A Kalman filter based on approximation of the state error covariance matrix is presented, employing a reduction of the effective model dimension, the error's asymptotic steady state limit, and a time-invariant linearization of the dynamic model for the error integration. The approximations lead to dramatic computational savings in applying estimation theory to large complex systems. We examine the utility of the approximate filter in assimilating different measurement types using a twin experiment of an idealized Gulf Stream. A nonlinear primitive equation model of an unstable east-west jet is studied with a state dimension exceeding 170,000 elements. Assimilation of various pseudomeasurements are examined, including velocity, density, and volume transport at localized arrays and realistic distributions of satellite altimetry and acoustic tomography observations. Results are compared in terms of their effects on the accuracies of the estimation. The approximate filter is shown to outperform an empirical nudging scheme used in a previous study. The examples demonstrate that useful approximate estimation errors can be computed in a practical manner for general circulation models.

Fukumori, Ichiro↗
NASA NTRS1994

An Approximate Kalman Filter for Ocean Data Assimilation; An Example with an Idealized Gulf Stream Model

A practical method of data assimilation for use with large, nonlinear, ocean general circulation models is explored. A Kalman filter based on approximations of the state error covariance matrix is presented, employing a reduction of the effective model dimension, the error's asymptotic steady-state limit, and a time-invariant linearization of the dynamic model for the error integration. The approximations lead to dramatic computational savings in applying estimation theory to large complex systems. We examine the utility of the approximate filter in assimilating different measurement types using a twin experiment of an idealized Gulf Stream. A nonlinear primitive equation model of an unstable east-west jet is studied with a state dimension exceeding 170,000 elements. Assimilation of various pseudo measurements is examined, including velocity, density, and volume transport at localized arrays, and realistic distributions of satellite altimetry and acoustic tomography observations. Results are compared in terms of their effects on the accuracies of the estimation. The approximate filter is shown to outperform a previous study that used an empirical nudging scheme. The examples demonstrate that useful approximate estimation errors can be computed in a practical manner for general circulation models.

['Kalman↗
DOE OSTI2016

A multi-level anomaly detection algorithm for time-varying graph data with interactive visualization

This work presents a novel modeling and analysis framework for graph sequences which addresses the challenge of detecting and contextualizing anomalies in labelled, streaming graph data. We introduce a generalization of the BTER model of Seshadhri et al. by adding flexibility to community structure, and use this model to perform multi-scale graph anomaly detection. Specifically, probability models describing coarse subgraphs are built by aggregating node probabilities, and these related hierarchical models simultaneously detect deviations from expectation. This technique provides insight into a graph's structure and internal context that may shed light on a detected event. Additionally, this multi-scale analysis facilitates intuitive visualizations by allowing users to narrow focus from an anomalous graph to particular subgraphs or nodes causing the anomaly. For evaluation, two hierarchical anomaly detectors are tested against a baseline Gaussian method on a series of sampled graphs. We demonstrate that our graph statistics-based approach outperforms both a distribution-based detector and the baseline in a labeled setting with community structure, and it accurately detects anomalies in synthetic and real-world datasets at the node, subgraph, and graph levels. Furthermore, to illustrate the accessibility of information made possible via this technique, the anomaly detector and an associated interactive visualization tool are tested on NCAA football data, where teams and conferences that moved within the league are identified with perfect recall, and precision greater than 0.786.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1985

Iterative solution of large, sparse linear systems on a static data flow architecture - Performance studies

The applicability of static data flow architectures to the iterative solution of sparse linear systems of equations is investigated. An analytic performance model of a static data flow computation is developed. This model includes both spatial parallelism, concurrent execution in multiple PE's, and pipelining, the streaming of data from array memories through the PE's. The performance model is used to analyze a row partitioned iterative algorithm for solving sparse linear systems of algebraic equations. Based on this analysis, design parameters for the static data flow architecture as a function of matrix sparsity and dimension are proposed.

Reed, D. A.↗
NASA NTRS1995

Effect Of Transition Density On Performance Of A DTTL

One estimates minimum transition density needed to limit symbol-error rate. Theoretical calculations and accompanying computer simulations provide numerical basis for specification of minimum data-transition density needed for adequate performance of radio receiver processing noisy signal containing non-return-to-zero (NRZ) stream of data symbols (bits). By use of digital-data-transition-tracking loop (DTTL), receiver strives to synchronize operation with data symbols. Performance of receiver in terms of ability to keep symbol-error rate (SER) in output of receiver below specified level depends on symbol signal-to-noise ratio (SSNR) of incoming signal and on degree of synchronization. Ability of DTTL to maintain synchronization depends partly on SSNR and partly on data-transition density.

Nguyen, Tien M.↗
DOE OSTI2016

Pre-commercial Demonstration of Direct Non-iterative State Estimator for Operational Use of Synchrophasor Management Systems

The objective of this project was to demonstrate functionality and performance of a Direct Non-iterative State Estimator (DNSE) integrated with NYPA’s Energy Management System (EMS) and with enhanced Real Time Dynamics Monitoring System (RTDMS) synchrophasor platform from Electric Power Group (EPG). DNSE is designed to overcome a major obstacle to operational use of Synchro-Phasor Management Systems (SPMS) by providing to synchrophasor management systems (SPMS) applications a consistent and a complete synchrophasor data foundation in the same way that a traditional EMS State Estimator (SE) provides to EMS applications. Specifically, DNSE is designed to use synchrophasor measurements collected by a central PDC, Supervisory Control and Data Acquisition (SCADA) measurements, and Energy Management System (EMS) network model, to obtain the complete state of the utility’s operating model at rates that are close to the synchrophasor data rates. In this way, the system is comprehensive in that it does not only cover the part of the network that is “visible” via synchrophasors, but also the part that is only “visible” through the SCADA measurements. Visualization needs associated with the use of DNSE results are fulfilled through suitably enhanced Real Time Dynamics Monitoring System (RTDMS), with the enhancements implemented by EPG. This project had the following goals in mind: To advance the deployment of commercial grade DNSE software application that relies on synchrophasor and SCADA data ; Apply DNSE at other utilities, to address a generic and fundamental need for “clean” operational data for synchrophasor applications; Provide means for “live” estimated data access by control system operators; Enhance potential for situational awareness through full system operational model coverage; Sub-second execution rate of the Direct Non-iterative State Estimator, eventually at a near-phasor data rate execution speed, i.e. < 0.1 sec. Anticipated benefits from this projects are: Enhanced reliability and improvements in the economic efficiency of bulk power system planning and operations; Providing “clean” data to other synchrophasor applications; Enhancement of situational awareness by providing the full operational model updated at near synchrophasor rate; A production-grade software tool that incorporate synchrophasor and SCADA data; Provides a basis for development of next generation monitoring and control applications, based on both SCADA and PMU data. Quanta Technology (QT) team worked in collaboration with Electric Power Group (EPG) whose team has enhanced its commercial Real Time Dynamics Monitoring System (RTDMS) to accommodate the requirements posed by DNSE application. EPG also provided its ePDC and Model-less Data Conditioning (PDVC) software for integration with DNSE+. QT developed the system requirements for DNSE; developed system architecture and defined interfaces between internal DNSE components. The core DNSE algorithm with all surrounding interfaces was named DNSE+. Since the DNSE development was done in a simulated system environment, QT used its PMU simulator that was enhanced during this project for development and factory acceptance testing (FAT). SCADA data in this stage was simulated by commercial PSS/e software. The output of DNSE are estimates of System states in C37.118-2 format, sent to RTDMS for further processing and display. As the number of these states is large, it was necessary to expand the C37.111-2 standard to accommodate large data sets. This enhancement was implemented in RTDMS. The demonstration of pre-production DNSE technology was done at NYPA using streaming field data from NYPA PMUs and from its RTUs through their SCADA system. NYPA provided ICCP interface as well as Common Information Model (CIM). The relevance of the DNSE+ application is that it provides state estimation of the power system based on hybrid set of data, consisting of both available PMU data and SCADA measurements. As this is a direct, non-iterative method of calculation of the system states, if does not suffer from convergence issues which is potential problem for conventional state estimators. Also, it can take any available PMU measurements, so it does not need to have a high percentage of PMU coverage needed in the case of Linear State Estimator. As the DNSE calculates synchrophasors of the system states (both phase and absolute value) as sub-second rate, this application can provide a basis for development of next generation of applications based both on SCADA and PMU data.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2018

System for detecting objects in streaming 3D images formed from data acquired with a medium penetrating sensor

A detection system that detects subsurface objects within a medium and estimates various features of the objects is provided. The detection system receives a streaming sequence of image frames of the medium at various along-medium locations. An image frame contains voxel values (intensities) representing characteristics of the medium across the medium and in the depth (range) direction. The detection system depth-compensates the intensities for determining which voxels are part of an object using an unsupervised binary classifier. The detection system then connects object voxels into distinct objects and recursively estimates the features of those objects as the image frames stream based on the locations and intensities of the object voxels.

Paglieroni, David W.↗
NASA NTRS2010

VLBI Data Interchange Format (VDIF)

One important outcome of the 7th International e-VLBI Workshop in Shanghai in June 2008 was the creation of a task force to study and recommend a universal VLBI data format that is suitable for both on-the-wire e-VLBI data transfer, as well as direct disk storage. This task force, called the VLBI Data Interchange Format (VDIF) Task Force, is the first part of a two-part effort, the second of which will address standardization of e-VLBI data-transmission-protocols. The formation of the VDIF Task Force was prompted particularly by increased e-VLBI activity and the difficulties encountered when data arrive at a correlator in different formats from various instruments in various parts of the world. The task force created a streaming packetized data format that may be used for real-time and non-realtime e-VLBI, as well as direct disk storage. The data may contain multiple channels of time-sampled data with an arbitrary number of channels, arbitrary #bits/sample up to 32, and real or complex data; data rates in excess of 100 Gbps are supported. Each data packet is completely self-identifying via a short header, and data may be decoded without reference to any external information. The VDIF task force has completed its work, and the VDIF standard was ratified at the 2009 e-VLBI workshop in Madrid.

Whitney, Alan↗