Search NASA⌕ Search

SEARCH · Search NASA

Results for “Streaming Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Apparatus and amendment of wind turbine blade impact detection and analysis

A multisensory system provides both temporal and spatial coverage capacities for auto-detection of bird collision events. The system includes an apparatus having a first circuitry to capture and store a series of images or video of a blade of a wind turbine; and a memory to store the images from the first circuitry. The apparatus also has one or more sensors to continuously sense vibration of the blade or for acoustic recordings; and a second circuitry to analyze the sensor data stream and/or the series of images or video to identify a cause of the vibration and to trigger the camera(s). A communication interface transmits data from the second circuitry to another device, wherein the second circuitry applies artificial intelligence or machine learning to control sensitivity of the one or more sensors.

Johnston, Matthew↗
DOE OSTI2025

Real-time Anomaly Detection for Liquid Argon Time Projection Chambers

We present a real-time anomaly detection framework for liquid argon time projection chambers (LArTPCs), targeting applications in particle physics experiments such as the Short Baseline Near Detector (SBND) or the future Deep Underground Neutrino Experiment (DUNE). These experiments employ detectors that generate and stream high-resolution but sparse images of neutrino and other particle interactions. Our approach utilizes anomaly detection with autoencoders, compressed through knowledge distillation (KD), to enable the detection of anomalous signals in the data through efficient inference on resource-constrained hardware. The framework is targeted for deployment on computing platforms equipped with field-programmable gate arrays (FPGAs), GPUs, or CPUs, allowing low-latency selection of relevant activity directly from the raw detector data stream. We demonstrate that our approach is suitable for the detection and localization of anomalously "high-multiplicity" activity, and outline promising applications for LArTPC online data filtering and triggering.

FOS: Physical sciences↗
DOE OSTI2023

Machine Learning Enabled Sensor Fusion for In-Situ Defect Detection in Laser Powder Bed Fusion

Laser Powder Bed Fusion (L-PBF) Additive Manufacturing (AM) is among the metal 3D printing technologies most broadly adopted by the manufacturing industry. The current industry qualification paradigm for critical-application L-PBF parts relies heavily on expensive non-destructive inspection techniques such as X-Ray Computed Tomography (XCT), which significantly limits the use-cases of L-PBF. In situ monitoring of the process promises a less expensive alternative to ex situ testing, but existing sensor technologies and data analysis techniques struggle to detect sub-surface flaws (e.g., porosity and cracking) on production-scale L-PBF printers. RTX Technologies Research Center (RTRC) has licensed ORNL’s Peregrine software package – a printer- and camera-agnostic data analytics tool designed specifically for detecting process anomalies using in situ data collected during powder bed printing. The goal of this project was to feed temporally rich, multi-modal sensor data, including visible light, integrated near infrared (NIR), and spatially mapped co-axial melt pool thermal emission data into Peregrine to enable detection of subsurface flaws. XCT data was used as ground truth training data to allow Peregrine’s deep learning algorithms to recognize anomalies in these complex data streams in both test artifacts and industrially relevant geometries. Completion of this program has seen the successful implementation of multi-modal, multi-layer sensor data footprints for training of machine learning models in Peregrine. Flaws detected in XCT data have been successfully detected directly from this in situ data footprint, and initial analyses of the in situ probability-of-detection has been conducted, showing performance levels commensurate with traditional non-destructive evaluation (NDE) methods. The in situ monitoring methodology was then applied to an industrially relevant component that was using post-build NDE, highlighting the utility of the proposed method for hard-to-inspect AM components. As a direct result of this program, two journal manuscripts [1], [2] have been published in Additive Manufacturing, with additional manuscripts planned following program completion.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Measurement of charged-current $\nu_{\mu}$ and $\nu_e$ interactions with wire-cell in MicroBooNE towards a search for low-energy $\nu_e$ excess

This technote summarizes the existing work in searching for $\nu_e$ low-energy excess (eLEE) in MicroBooNE Booster Neutrino Beam (BNB) data stream based on the Wire-Cell event reconstruction paradigm. The charged-current $\nu_{\mu}$ and $\nu_e$ events are selected from the 5.3e19 POT data from the BNB beam and 2.06e20 POT data from Neutrinos at the Main Injector (NuMI) beam. The charged-current $\nu_e$ selection results from the BNB data that are sensitive to the eLEE search are not included. Various comparisons between data and Monte Carlo predictions are performed to validate the overall model and demonstrate the power of the analysis techniques.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1979

Solar maximum ultraviolet spectrometer and polarimeter

The objectives of the UVSP experiment are to study solar ultraviolet radiations, particularly from flares and active regions, and to measure constituents in the terrestrial atmosphere by the extinction of sunlight at satellite dawn and dusk. The instrument is designed to observe the Sun at a variety of spectral and spatial resolutions in the range from 1150 to 3600 A. A Gregorian telescope with effective focal length of 1.8 m is used to feed a 1 m Ebert-Fastie spectrometer. A polarimeter containing rotatable magnesium fluoride waveplates is included behind the spectrometer entrance slit and will allow all four Stokes parameters to be determined. Velocities on the Sun can also be measured. The instrument is controlled by a computer which can interact with the data stream to modify the observing program. The observing modes, including rasters, spectral scans, velocity measurements, and polarimetry, are also described along with plans for mission operations, data handling, and analysis of the observations.

Tandberg-Hanssen, E.↗
NASA NTRS2011

Low Latency DESDynI Data Products for Disaster Response, Resource Management and Other Applications

We are developing onboard processor technology targeted at the L-band SAR instrument onboard the planned DESDynI mission to enable formation of SAR images onboard opening possibilities for near-real-time data products to augment full data streams. Several image processing and/or interpretation techniques are being explored as possible direct-broadcast products for use by agencies in need of low-latency data, responsible for disaster mitigation and assessment, resource management, agricultural development, shipping, etc. Data collected through UAVSAR (L-band) serves as surrogate to the future DESDynI instrument. We have explored surface water extent as a tool for flooding response, and disturbance images on polarimetric backscatter of repeat pass imagery potentially useful for structural collapse (earthquake), mud/land/debris-slides etc. We have also explored building vegetation and snow/ice classifiers, via support vector machines utilizing quad-pol backscatter, cross-pol phase, and a number of derivatives (radar vegetation index, dielectric estimates, etc.). We share our qualitative and quantitative results thus far.

applications↗
NASA NTRSDate not supplied

Generation of Land Surface Reflectance with Combined Geo-KOMPSAT-2A AMI and Himawari 8 AHI Observations

The latest generation of geostationary satellites has opened a new era of Earth observations with unprecedented spatiotemporal resolution and spectral range. Together with GOES 16/17 ABI, FY4-A AGRI, and Himawari-8 AHI, a new Korean geostationary satellite (Geo-KOMPSAT-2A AMI) has operationally collected a full-disk image in 16 channels every ten minutes since July 2019, allowing diurnal land surface monitoring over a large proportion of Asia and all of Oceania. Retrieving accurate surface reflectance (SR) over land from GK-2A/AMI is a challenging but high priority objective. One of the challenges is the absence of a spectral band in the 2.2 m SWIR range from AMI, which is required by many atmospheric correction algorithms to retrieve atmospheric aerosol properties. To remedy this issue, we adopt a strategy that combines concurrent GK-2A/AMI and Himawari 8/AHI observations in order to derive AMI SR. We have adapted the NASA Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm to process the data stream from Himawari 8/AHI. The advantages of the MAIAC algorithm is its capability to exploit the high temporal frequency and varying illumination geometry of the geostationary data for advanced cloud/snow detection, aerosol retrieval, and characterization of surface bidirectional reflectance properties. Leveraging the similarities of spectral bands and the sun-target-sensor geometry between AMI and AHI, we are able to create denser time series of observations and enhanced BRDF samples over most of the spatial coverage of AMI (and AHI). The combined stereo-type observations not only help derive SR for AMI but also enhance retrievals of the corresponding AHI surface products. We evaluate the resulting AMI SR using ground (AERONET) observations and corresponding MODIS products. Further, we discuss potential challenges in utilizing the geostationary satellite data for land surface monitoring.

geostationary satellite↗
NASA NTRSDate not supplied

Actor-based Runtime Verification with MESA

This work presents a runtime verification approach implemented in the tool MESA (MEssage-based System Analysis) which allows for using concurrent monitors to check for properties specified in data parameterized temporal logic and state machines. The tool is implemented as an internal Scala DSL. We employ the actor programming model to implement MESA where monitors are captured by concurrent actors that communicate via messaging. The paper presents a case study in which MESA is used to effectively monitor a large number of flights from live US airspace data streams. We also perform an empirical study by conducting experiments using monitoring systems with different numbers of concurrent monitors and different layers of indexing on the data contained in events. The paper describes the experiments, evaluates the results, and discusses challenges faced during the study. The evaluation shows the value of combining concurrency with indexing to handle data rich events.

runtime verification↗
DOE OSTI2023

ESnet/JLab FPGA Accelerated Transport

To increase the science rate for high data rates/volumes, Thomas Jefferson National Accelerator Facility (JLab) has partnered with Energy Sciences Network (ESnet) to define an edge to data center traffic shaping / steering transport capability featuring data event aware network shaping and forwarding. The keystone of this ESnet+JLab FPGA Accelerated Transport (EJFAT) is the joint development of an AI/ML directed dynamic compute work Load Balancer (LB) of UDP streamed data. The LB is a suite consisting of a Field Programmable Gate Array (FPGA) executing the dynamically configurable, low fixed latency LB data plane featuring real-time packet redirection and high throughput, and a control plane running on the FPGA host computer that monitors network and compute farm telemetry in order to make dynamic AI/ML guided decisions for destination compute host redirection/load balancing and destination resource provisioning. The LB provides for three-tier horizontal scaling across LB suites, core compute hosts, and CPUs within a host. The LB effectively provides seamless integration of edge/core computing to support direct experimental data processing for immediate use by JLab science programs and others such as the EIC as well as data centers of the future requiring high throughput and low latency for both hot and cooled data for both running experiment data acquisition systems and data center use cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

An Event Detection Framework for Virtual Observation System: Anomaly Identification for an ACME Land Simulation

Based on previous work on in-situ data transfer infrastructure and compiler-based software analysis, we have designed a virtual observation system for real time computer simulations. This paper presents an event detection framework for a virtual observation system. By using signal processing and detection approaches to the memory-based data streams, this framework can be reconfigured to capture high-frequency events and low-frequency events. These approaches used in the framework can dramatically reduce the data transfer needed for in-situ data analysis (between distributed computing nodes or between the CPU/GPU nodes). In the paper, we also use a terrestrial ecosystem system simulation within the Earth System Model to demonstrate the practical values of this effort.

Yao, Cindy↗
DOE OSTI2021

Measurement of Charged-Current $ν_μ$ and $ν_e$ Interactions Towards a Search for Low-Energy $ν_Ε$ Excess Using Wire-Cell In Microboone

This technote summarizes the existing work in searching for $ν_e$ low-energy excess (eLEE) in MicroBooNE Booster Neutrino Beam (BNB) data stream based on the Wire-Cell event reconstruction paradigm. The charged-current $ν_μ$ and $ν_e$ events are selected from the 5.3e19 POT open data from the BNB beam, 6.37e20 POT far sideband from the BNB beam data, and 2.10e20 POT data from Neutrinos at the Main Injector (NuMI) beam. The charged-current $ν_e$ selection results from the BNB data that are sensitive to the eLEE search are not included. Various comparisons between data and Monte Carlo predictions are performed to validate the overall model and demonstrate the power of the analysis techniques. Physics sensitivities in terms of the exclusion and the discovery potential are presented.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1984

Revised radiometric calibration technique for LANDSAT-4 Thematic Mapper data by the Canada Centre for Remote Sensing

Observations of raw image data, raw radiometric calibration data, and background measurements extracted from the raw data streams on high density tape reveal major shortcomings in a technique proposed by the Canadian Center for Remote Sensing in 1982 for the radiometric correction of TM data. Results are presented which correlate measurements of the DC background with variations in both image data background and calibration samples. The effect on both raw data and data corrected using the earlier proposed technique is explained and the correction required for these factors as a function of individual scan line number for each detector is described. How the revised technique can be incorporated into an operational environment is demonstrated.

Murphy, J.↗
NASA NTRS2011

Validation of Real-Time Data Processing for the Ground and Air-MSPI Systems

JPL is currently developing the multi-angle spectro-polarimetric imager (MSPI), targeted for the Aerosol-Cloud-Ecosystems (ACE) mission, as defined in the National Academies 2007 Decadal Survey. In preparation for the space instrument, the MSPI team has built two incremental camera systems (Ground- and Air-MSPI) to improve understanding of the proposed architecture. Ground-MSPI is a gimballed instrument used primarily for stationary observation and characterization of the imager and optics. The ER-2 based Air-MSPI operates in a step-and-stare mode, providing multi-angle imaging of a static target. This mode-of-operation simulates the observation scenario of the space instrument. Physically, MSPI is a pushbroom camera with a specialized frontend. Before imaging, light entering the camera passes through a pair of photoelastic modulators and a set of pattern polarizers. These optical elements act on the light to make polarimetric extraction computationally feasible. Calculating polarimetric parameters from the imager's data stream requires a real-time least-squares computation that produces coefficients of a truncated time-series expansion of the image. As reported in, the data processing algorithm can operate in real-time on a Xilinx Virtex-5 FPGA. Moving beyond verification with an onboard data source, the algorithm has been validated on a commercial development board interfaced with the ground camera. In addition, the algorithm has been instantiated within the Air-MSPI electronics board's FPGA, and in situ first-light has been achieved.

real-time data processing↗
NASA NTRS2020

Actor-based Runtime Verification with MESA

This work presents a runtime verification approach implemented in the tool MESA (MEssage-based System Analysis) which allows for using concurrent monitors to check for properties specified in data parameterized temporal logic and state machines. The tool is implemented as an internal Scala DSL. We employ the actor programming model to implement MESA where monitors are captured by concurrent actors that communicate via messaging. The paper presents a case study in which MESA is used to effectively monitor a large number of flights from live US airspace data streams. We also perform an empirical study by conducting experiments using monitoring systems with different num- bers of concurrent monitors and different layers of indexing on the data contained in events. The paper describes the experiments, evaluates the results, and discusses challenges faced during the study. The evaluation shows the value of combining concurrency with indexing to handle data rich events.

Mehlitz, Peter↗
DOE OSTI2017

Two-Year Operational Evaluation Of A Consumer Electronics-Based Data Acquisition System For Equipment Monitoring

Oak Ridge National Laboratory (ORNL) has performed a two-year evaluation of the performance of an in-house developed consumer electronics-based data acquisition system (DAS). The main advantage of this approach compared to conventional instrumentation grade systems is cost; instrument grade data acquisition systems average costs range from $800 to $2,000 per channel compared to a range of $200 - $400 per channel for a consumer electronics-based system. The DAS is operated as a full-time in-situ vibration monitor. The resulting data stream is streamed over the ORNL network, at an aggregate rate of approximately 2 megabytes/s, to a Linux server. The server includes the capability to implement event-triggered data stores, as well as real-time files for the implementation of continuous display monitoring of the spectra. Detailed spectral analysis is performed post event. The DAS is installed on a large industrial chiller and cooling water pump associated with ORNL’s Titan supercomputer. These mechanical systems include rotating components that operate at fundamental frequencies within the range of 30 Hz to over 3 KHz. Evaluation of the DAS data over a two-year operating period leads to the conclusion that for many industrial processes this system could form the basis for a cost effective means of obtaining operating health data in real time from rotating machinery. The deployment has also shown that the DAS technology is reliable. Furthermore, because the cost of the DAS is low, the other significant advantage of this approach is that the DAS can be deployed in a dedicated manner and operated on a full-time basis.

Wetherington Jr, G↗
NASA NTRS2017

Framework for Processing Citizens Science Data for Applications to NASA Earth Science Missions

Citizen science (or crowdsourcing) has drawn much high-level recent and ongoing interest and support. It is poised to be applied, beyond the by-now fairly familiar use of, e.g., Twitter for natural hazards monitoring, to science research, such as augmenting the validation of NASA earth science mission data. This interest and support is seen in the 2014 National Plan for Civil Earth Observations, the 2015 White House forum on citizen science and crowdsourcing, the ongoing Senate Bill 2013 (Crowdsourcing and Citizen Science Act of 2015), the recent (August 2016) Open Geospatial Consortium (OGC) call for public participation in its newly-established Citizen Science Domain Working Group, and NASA's initiation of a new Citizen Science for Earth Systems Program (along with its first citizen science-focused solicitation for proposals). Over the past several years, we have been exploring the feasibility of extracting from the Twitter data stream useful information for application to NASA precipitation research, with both "passive" and "active" participation by the twitterers. The Twitter database, which recently passed its tenth anniversary, is potentially a rich source of real-time and historical global information for science applications. The time-varying set of "precipitation" tweets can be thought of as an organic network of rain gauges, potentially providing a widespread view of precipitation occurrence. The validation of satellite precipitation estimates is challenging, because many regions lack data or access to data, especially outside of the U.S. and in remote and developing areas. Mining the Twitter stream could augment these validation programs and, potentially, help tune existing algorithms. Our ongoing work, though exploratory, has resulted in key components for processing and managing tweets, including the capabilities to filter the Twitter stream in real time, to extract location information, to filter for exact phrases, and to plot tweet distributions. The key step is to process the "precipitation" tweets to be compatible with satellite-retrieved precipitation data. These key components for processing and managing "precipitation" tweets (and additional ones to be developed) are not limited to precipitation, nor are they limited to the Twitter social medium. Indeed, to maximize the value of our work for NASA earth science programs, these components should be generalized and be part of an overall framework for processing citizen science data for science research. In this paper, we outline such a framework.

earth science satellite data↗
NASA NTRS1974

The engineering design integration (EDIN) system

A digital computer program complex for the evaluation of aerospace vehicle preliminary designs is described. The system consists of a Univac 1100 series computer and peripherals using the Exec 8 operating system, a set of demand access terminals of the alphanumeric and graphics types, and a library of independent computer programs. Modification of the partial run streams, data base maintenance and construction, and control of program sequencing are provided by a data manipulation program called the DLG processor. The executive control of library program execution is performed by the Univac Exec 8 operating system through a user established run stream. A combination of demand and batch operations is employed in the evaluation of preliminary designs. Applications accomplished with the EDIN system are described.

Glatt, C. R.↗
DOE OSTI2020

Scalable Data-Intensive Geocomputation: A Design for Real-Time Continental Flood Inundation Mapping

The convergence of data-intensive and extreme-scale computing enables an integrated software and data ecosystem for scientific discovery. Developments in this realm will fuel transformative research in data-driven interdisciplinary domains. Geocomputation provides computing paradigms in Geographic Information Systems (GIS) for interactive computing of geographic data, processes, models, and maps. Because GIS is data-driven, the computational scalability of a geocomputation workflow is directly related to the scale of the GIS data layers, their resolution and extent, as well as the velocity of the geo-located data streams to be processed. Unique in high user interactivity and low end-to-end latency requirements, geocomputation applications will dramatically benefit from the convergence of high-end data analytics (HDA) and high-performance computing (HPC). The application level challenge, however, is to identify and eliminate computational bottlenecks that arise along a geocomputation workflow. Indeed, poor scalability at any of the workflow components is detrimental to the entire end-to-end pipeline. Here, we study a large geocomputation use case in flood inundation mapping that handles multiple national-scale geospatial datasets and targets low end-to-end latency. We discuss benefits and challenges for harnessing both HDA and HPC for data-intensive geospatial data processing and intensive numerical modeling of geographic processes. We propose an HDA+HPC geocomputation architecture design that couples HDA (e.g., Spark)-based spatial data handling and HPC-based parallel data modeling. Key techniques for coupling HDA and HPC to bridge the two different software stacks are reviewed and discussed.

Liu, Yan↗