Search NASA⌕ Search

SEARCH · Search NASA

Results for “Streaming Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Streaming Analytics Machine (SAM) v. 0.0.1

This is software developed in the course of my dissertation work, which was funded by Sandia. The software includes a library (the Streaming Analytics Machine or SAM) for expressing machine learning and subgraph matching over streaming data sources. It also includes a utility that will convert from a high-level language, the Streaming Analytics Language (SAL) to SAM. SAND2019-4322 M

Goodman, Eric↗

Classification and Localization of Fracture-Hit Events in Low-Frequency Distributed Acoustic Sensing Strain Rate with Convolutional Neural Networks

Summary Distributed acoustic sensing (DAS) has been used in the oil and gas industry as an advanced technology for surveillance and diagnostics. Operators use DAS to monitor hydraulic fracturing activities, examine well stimulation efficacy, and estimate complex fracture system geometries. Particularly, low-frequency DAS can detect geomechanical events such as fracture hits because hydraulic fractures propagate and create strain rate variations in the rock. Analysis of DAS data today is mostly done post-job and subject to interpretation methods. However, the continuous and dense data stream generated live by DAS poses the opportunity for more efficient and accurate real-time data-driven analysis. The objective of this study is to develop a machine learning-based workflow that can identify and locate fracture-hit events in simulated strain rate responses correlated with low-frequency DAS data. In this paper, “fracture hit” refers to a hydraulic fracture originating from a stimulated well intersecting an offset well. We start with building a single fracture propagation model to produce strain rate patterns observed at a hypothetical monitoring well. This model is used to generate two sets of strain rate responses with one set containing fracture-hit events. The labeled synthetic data are then used to train a custom convolutional neural network (CNN) model for identifying the presence of fracture-hit events. The same model is trained again for locating the event with the output layer of the model replaced with linear units. We achieved near-perfect predictions for both event classification and localization. These promising results prove the feasibility of using CNN for real-time event detection from fiber-optic sensing data. Additionally, we use edge detection techniques to recognize fracture-hit event patterns in strain rate images. The fracture-hit location can be identified using recognized pixels in the image. The accuracy of edge detection-based location identification is also plausible, but edge detection is dependent on the assumption of pattern shape and image quality, hence it is less robust compared to CNN models. This comparison further supports the need for CNN applications in image-based real-time fiber-optic sensing event detection.

Engineering↗

Design and development of the DAQ and Timing Hub for CMS Phase-2

The CMS detector will undergo a major upgrade for Phase-2 of the LHC program, starting around 2026. The upgraded Level-1 hardware trigger will select events at a rate of 750 kHz. At an expected event size of 7.4 MB this corresponds to a data rate of up to 50 Tbit/s.Optical links will carry the signals from on-detector front-end electronics to back-end electronics in ATCA crates in the service cavern. A DAQ and Timing Hub board aggregates data streams from back-end boards over point-to-point links, provides buffering and transmits the data to the commercial data-to-surface network for processing and storage. This hub board is also responsible for the distribution of timing, control and trigger signals to the back-ends.This paper presents the current development towards the DAQ and Timing Hub and the design of the first prototype, to be used as for validation and integration with the first back-end prototypes in 2019-2020.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Simplifying operations with an uplink/downlink integration toolkit

The Operations Engineering Lab (OEL) at JPL has developed a simple, generic toolkit to integrate the uplink/downlink processes, (often called closing the loop), in JPL's Multimission Ground Data System. This toolkit provides capabilities for integrating telemetry verification points with predicted spacecraft commands and ground events in the Mission Sequence Of Events (SOE) document. In the JPL ground data system, the uplink processing functions and the downlink processing functions are separate subsystems that are not well integrated because of the nature of planetary missions with large one-way light times for spacecraft-to-ground communication. Our new closed-loop monitoring tool allows an analyst or mission controller to view and save uplink commands and ground events with their corresponding downlinked telemetry values regardless of the delay in downlink telemetry and without requiring real-time intervention by the user. An SOE document is a time-ordered list of all the planned ground and spacecraft events, including all commands, sequence loads, ground events, significant mission activities, spacecraft status, and resource allocations. The SOE document is generated by expansion and integration of spacecraft sequence files, ground station allocations, navigation files, and other ground event files. This SOE generation process has been automated within the OEL and includes a graphical, object-oriented SOE editor and real-time viewing tool running under X/Motif. The SOE toolkit was used as the framework for the integrated implementation. The SOE is used by flight engineers to coordinate their operations tasks, serving as a predict data set in ground operations and mission control. The closed-loop SOE toolkit allows simple, automated integration of predicted uplink events with correlated telemetry points in a single SOE document for on-screen viewing and archiving. It automatically interfaces with existing real-time or non real-time sources of information, to display actual values from the telemetry data stream. This toolkit was designed to greatly simplify the user's ability to access and view telemetry data, and also provide a means to view this data in the context of the commands and ground events that are used to interpret it. A closed-loop system can prove especially useful in small missions with limited resources requiring automated monitoring tools. This paper will discuss the toolkit implementation, including design trade-offs and future plans for enhancing the automated capabilities.

Murphy, Susan C.↗

Video event trigger and tracking system using fuzzy comparators

A video observation method and apparatus, the apparatus having a frame storage mechanism, a dividing mechanism, a plurality of fuzzy comparators and a trigger signal mechanism. The frame storage mechanism stores at least one non-current video frame of a viewing field. The dividing mechanism divides a current video frame of the viewing field and the at least one non-current video frame into a plurality of corresponding trigger sections. The plurality of fuzzy comparators each compare and detect a fuzzy logic difference between one trigger section of the current video frame and the corresponding trigger sections of the at least one non-current video frame, the number of fuzzy comparators being selected so that every trigger section of the current video frame is compared. The trigger signal mechanism provides a trigger signal when a fuzzy logic difference is detected between any of the corresponding current and non-current trigger sections. A video observation mechanism and data reducing mechanism may be included with the above apparatus or alone with only a frame storage mechanism, a single generic comparator and a trigger signal mechanism. The video observation mechanism provides a video data stream, wherein each pixel of each frame of a viewing field is provided as multiple bits of data. The data reducing mechanism reduces each set of multiple bits of data which correspond to each pixel to one bit of binary data based on whether the pixel has a level of grey which is above or below a threshold level of grey.

Williams, Glenn L.↗

Spinoff 2013

Topics covered include: Innovative Software Tools Measure Behavioral Alertness; Miniaturized, Portable Sensors Monitor Metabolic Health; Patient Simulators Train Emergency Caregivers; Solar Refrigerators Store Life-Saving Vaccines; Monitors Enable Medication Management in Patients' Homes; Handheld Diagnostic Device Delivers Quick Medical Readings; Experiments Result in Safer, Spin-Resistant Aircraft; Interfaces Visualize Data for Airline Safety, Efficiency; Data Mining Tools Make Flights Safer, More Efficient; NASA Standards Inform Comfortable Car Seats; Heat Shield Paves the Way for Commercial Space; Air Systems Provide Life Support to Miners; Coatings Preserve Metal, Stone, Tile, and Concrete; Robots Spur Software That Lends a Hand; Cloud-Based Data Sharing Connects Emergency Managers; Catalytic Converters Maintain Air Quality in Mines; NASA-Enhanced Water Bottles Filter Water on the Go; Brainwave Monitoring Software Improves Distracted Minds; Thermal Materials Protect Priceless, Personal Keepsakes; Home Air Purifiers Eradicate Harmful Pathogens; Thermal Materials Drive Professional Apparel Line; Radiant Barriers Save Energy in Buildings; Open Source Initiative Powers Real-Time Data Streams; Shuttle Engine Designs Revolutionize Solar Power; Procedure-Authoring Tool Improves Safety on Oil Rigs; Satellite Data Aid Monitoring of Nation's Forests; Mars Technologies Spawn Durable Wind Turbines; Programs Visualize Earth and Space for Interactive Education; Processor Units Reduce Satellite Construction Costs; Software Accelerates Computing Time for Complex Math; Simulation Tools Prevent Signal Interference on Spacecraft; Software Simplifies the Sharing of Numerical Models; Virtual Machine Language Controls Remote Devices; Micro-Accelerometers Monitor Equipment Health; Reactors Save Energy, Costs for Hydrogen Production; Cameras Monitor Spacecraft Integrity to Prevent Failures; Testing Devices Garner Data on Insulation Performance; Smart Sensors Gather Information for Machine Diagnostics; Oxygen Sensors Monitor Bioreactors and Ensure Health and Safety; Vision Algorithms Catch Defects in Screen Displays; and Deformable Mirrors Capture Exoplanet Data, Reflect Lasers.

Source record↗

Towards Reliable Evaluation of Anomaly-Based Intrusion Detection Performance

This report describes the results of research into the effects of environment-induced noise on the evaluation process for anomaly detectors in the cyber security domain. This research was conducted during a 10-week summer internship program from the 19th of August, 2012 to the 23rd of August, 2012 at the Jet Propulsion Laboratory in Pasadena, California. The research performed lies within the larger context of the Los Angeles Department of Water and Power (LADWP) Smart Grid cyber security project, a Department of Energy (DoE) funded effort involving the Jet Propulsion Laboratory, California Institute of Technology and the University of Southern California/ Information Sciences Institute. The results of the present effort constitute an important contribution towards building more rigorous evaluation paradigms for anomaly-based intrusion detectors in complex cyber physical systems such as the Smart Grid. Anomaly detection is a key strategy for cyber intrusion detection and operates by identifying deviations from profiles of nominal behavior and are thus conceptually appealing for detecting "novel" attacks. Evaluating the performance of such a detector requires assessing: (a) how well it captures the model of nominal behavior, and (b) how well it detects attacks (deviations from normality). Current evaluation methods produce results that give insufficient insight into the operation of a detector, inevitably resulting in a significantly poor characterization of a detectors performance. In this work, we first describe a preliminary taxonomy of key evaluation constructs that are necessary for establishing rigor in the evaluation regime of an anomaly detector. We then focus on clarifying the impact of the operational environment on the manifestation of attacks in monitored data. We show how dynamic and evolving environments can introduce high variability into the data stream perturbing detector performance. Prior research has focused on understanding the impact of this variability in training data for anomaly detectors, but has ignored variability in the attack signal that will necessarily affect the evaluation results for such detectors. We posit that current evaluation strategies implicitly assume that attacks always manifest in a stable manner; we show that this assumption is wrong. We describe a simple experiment to demonstrate the effects of environmental noise on the manifestation of attacks in data and introduce the notion of attack manifestation stability. Finally, we argue that conclusions about detector performance will be unreliable and incomplete if the stability of attack manifestation is not accounted for in the evaluation strategy.

cyber defense↗

Global Ocean Evaporation: How Well Can We Estimate Interannual to Decadal Variability?

Evaporation from the world's oceans constitutes the largest component of the global water balance. It is important not only as the ultimate source of moisture that is tied to the radiative processes determining Earth's energy balance but also to freshwater availability over land, governing habitability of the planet. Here we focus on variability of ocean evaporation on scales from interannual to decadal by appealing to three sources of data: the new MERRA-2 (Modern-Era Retrospective analysis for Research and Applications -2); climate models run with historical sea-surface temperatures, ice and atmospheric constituents (so-called AMIP experiments); and state-of-the-art satellite retrievals from the Seaflux and HOAPS (Hamburg Ocean-Atmosphere Parameters and Fluxes from Satellite) projects. Each of these sources has distinct advantages as well as drawbacks. MERRA-2, like other reanalyses, synthesizes evaporation estimates consistent with observationally constrained physical and dynamical models-but data stream discontinuities are a major problem for interpreting multi-decadal records. The climate models used in data assimilation can also be run with lesser constraints such as with SSTs and sea-ice (i.e. AMIPs) or with additional, minimal observations of surface pressure and marine observations that have longer and less fragmentary observational records. We use the new ERA-20C reanalysis produced by ECMWF embodying the latter methodology. Still, the model physics biases in climate models and the lack of a predicted surface energy balance are of concern. Satellite retrievals and comparisons to ship-based measurements offer the most observationally-based estimates, but sensor inter-calibration, algorithm retrieval assumptions, and short records are dominant issues. Our strategy depends on maximizing the advantages of these combined records. The primary diagnostic tool used here is an analysis of bulk aerodynamic computations produced by these sources and uses a first-order Taylor series analysis of wind speed, SST, near-surface stability and relative humidity variations around climatology to gauge the importance of these components. We find that the MERRA-2 evaporation record is strongly influenced by the availability of wind speed and humidity from passive microwave imagers beginning in the late 1980s as well as by the SST record. The trend over the period 1980 to present is nearly 10%. AMIP or the ERA-20C trends are much smaller. We find that ENSO-related signals involving both wind speed and thermodynamic variability remain the primary signal in the latter and are confirmed by satellite retrievals. We present uncertainty estimates based on the various data sources and discuss the implications for GEWEX water and energy budget science challenges.

Robertson, Franklin R.↗

Navigating Exascale Operational Data Analytics: From Inundation to Insight

In this paper, we address the challenges in achieving sustainable data-driven efficiency by providing a detailed exploration of the end-to-end operational data analytics (ODA) framework that evolved through two generations of supercomputer systems at the Oak Ridge Leadership Computing Facility (OLCF). This framework addresses large data streams ingested from heavily instrumented HPC environment that accumulates multi-terabytes per day. We outline the multifaceted data life cycle across HPC procurement, operations, and research & development, identifying key obstacles and design decisions that shape effective strategies in building and supporting data pipelines end-to-end. By sharing key insights and lessons learned from our experience, we offer recommendations for the HPC community on enabling sustainable operational data analytics and beyond. Our contributions aim to bridge the gap between potential and real benefits of operational data, guiding future efforts towards integrated and sustainable operational intelligence in high-performance computing environments.

Shin, Woong↗

Fine structure in RF spectra of lightning return stroke wave forms

The power spectra of the wide-band (10 Hz to 100 kHz) magnetic-field signals for a number of lightning return strokes measured during a thunderstorm which occurred in Lindau in August, 1984 have been calculated. The RF magnetic field data are obtained with the engineering unit of the Galileo Jupiter Probe lightning experiment. Each return stroke data stream is passed through an adaptive filter designed to whiten its spectrum. The spectra of the magnetic field data definitely show fine structure, with two or three distinct peaks in the spectra of many of the waveforms. A peak at f of about 60-70 kHz is often seen in the power spectra of the waveform time segments preceding and following the rise-to-peak amplitude of the return stroke.

Lanzerotti, L. J.↗

Space telemetry degradation due to Manchester data asymmetry induced carrier tracking phase error

The deleterious effects that the Manchester (or Bi-phi) data asymmetry has on the performance of phase-modulated residual carrier communication systems are analyzed. Expressions for the power spectral density of an asymmetric Manchester data stream, the interference-to-carrier signal power ratio (I/C), and the error probability performance are derived. Since data asymmetry can cause undesired spectral components at the carrier frequency, the I/C ratio is given as a function of both the data asymmetry and the telemetry modulation index. Also presented are the data asymmetry and asymmetry-induced carrier tracking loop and the system bit-error rate to various parameters of the models.

Nguyen, Tien M.↗

Commercial potential of remote sensing data from the Earth observing system

The purpose was to assess the market potential of remote sensing value-added products from the Earth Observing System (EOS) platform. Sensors on the EOS platform were evaluated to determine which qualities and capabilities could be useful to the commercial user. The approach was to investigate past and future satellite data distribution programs. A questionnaire was developed for use in a telephone survey. Based on the results of the survey of companies that add value to remotely sensed data, conversations with the principal investigators in charge of each EOS sensor, a study of past commercial satellite data ventures, and reading from the commercial remote sensing industry literature, three recommendations were developed: develop a strategic plan for commercialization of EOS data, define a procedure for commercial users within the EOS data stream, and develop an Earth Observations Commercial Applications Program-like demonstration program within NASA using EOS simulated data.

Merry, Carolyn J.↗

A digital twin platform for building performance monitoring and optimization: Performance simulation and case studies

Advancements in sensor technology, data analytics, affordable compute, and communication infrastructure have paved the way for Digital Twin technology in optimizing building operations and controls. This study presents the development of an open and interoperable web-based Digital Twin platform for integrating diverse data streams and facilitating effective user interactions. The platform utilizes modern technologies for the web framework and time-series data management, ensuring scalability and responsiveness. The backend supports seamless integration of diverse data sources and emulators, incorporating data from building sensors and meters, external weather Application Programming Interfaces, and advanced EnergyPlus simulation models of the building and its energy systems including the Distributed Energy Resources that are formulated in Functional Mockup Units. A simulation case study was conducted with FlexLab, a test facility on Lawrence Berkeley National Laboratory campus. The case study includes normal operations, Distributed Energy Resource integration, and power outage scenarios, to illustrate the Digital Twin’s ability to provide critical insights into energy performance and thermal resilience. The results demonstrated the platform’s potential as a decision-support tool for optimizing building energy performance and enhancing resilience against extreme weather events. Future work will focus on deploying the Digital Twin platform to a real building for field validation, extending its capabilities to cover more scenarios such as bidirectional Electric Vehicle interactions, and enhancing user engagement.

EnergyPlus↗

Pinch Heat Integration Tool (PIT) v0.9

The pinch heat integration tool (PIT) is a multi-module tool designed for identifying and analyzing thermal energy integration at facilities. It generates pinch and grand composite curves to identify optimal streams for heat exchange and industrial heat pumps. Users input heating and cooling stream data via an Excel sheet. The tool comprises four modules: 1. 'Main Pinch' for creating a pinch diagram using user-provided data. 2. 'GCC' for generating a grand composite curve to identify utility optimization opportunities. 3. 'Heat Exchanger' for analyzing stream combinations for heat exchange. 4. 'Heat Pump' for evaluating streams as sources and sinks for heat pumps and calculating the heat pump's coefficient of performance based on the selected refrigerant.

Karki, Unique [Lawrence Berkeley National Laborato↗

Workflows Community Summit 2022: A Roadmap Revolution

Scientific workflows have become integral tools in broad scientific computing use cases. Science discovery is increasingly dependent on workflows to orchestrate large and complex scientific experiments that range from the execution of a cloud-based data preprocessing pipeline to multi-facility instrument-to-edge-to-HPC computational workflows. Given the changing landscape of scientific computing (often referred to as a computing continuum) and the evolving needs of emerging scientific applications, it is paramount that the development of novel scientific workflows and system functionalities seek to increase the efficiency, resilience, and pervasiveness of existing systems and applications. Specifically, the proliferation of machine learning/artificial intelligence (ML/AI) workflows, need for processing large-scale datasets produced by instruments at the edge, intensification of near real-time data processing, support for long-term experiment campaigns, and emergence of quantum computing as an adjunct to HPC, have significantly changed the functional and operational requirements of workflow systems. Workflow systems now need to, for example, support data streams from the edge-to-cloud-to-HPC, enable the management of many small-sized files, allow data reduction while ensuring high accuracy, orchestrate distributed services (workflows, instruments, data movement, provenance, publication, etc.) across computing and user facilities, among others. Further, to accelerate science, it is also necessary that these systems implement specifications/standards and APIs for seamless (horizontal and vertical) integration between systems and applications, as well as enable the publication of workflows and their associated products according to the FAIR principles.

97 MATHEMATICS AND COMPUTING↗

Moving NASA Remote Sensing Data to the GIS Environment for Health Studies

There has been an increasing demand by the health community for improved data on many different environmental factors relevant to the links between the environment and disease occurrence and transmission. These data are important for GIS-based monitoring, risk mapping, and surveillance of epidemiological parameters on a large number of different spatial, temporal, and spectral resolutions. Accordingly, NASA is developing new approaches to data collection and distribution in order to improve access to multiple sources of data streams to increase spatial and temporal coverage. Methods are being developed to incorporate different, scalable capabilities to handle multiple data sources by adding, deleting and replacing components as required as well as associated tools for their management. An approach has been to search for innovative solutions focused on the creation, use and manipulation of data stored in many different archives. These include data transformation and combination as well as data and information tools that can assist the public health and science community to use existing and anticipated products in new and flexible ways. This presentation will provide an inventory of geophysical parameters derived from satellite remote sensing sensors that are useful for GIS-based public health studies. The presentation will also discuss the physical and scientific limitations of access to and use of these data for health applications such as resolution and format differences, lack of software interoperability, data access problems. Finally, there will be a summary of the recent steps the NASA program has taken to bring NASA-generated satellite products to a wider range of users in the GIS community.

Vicente, Gilberto A.↗

Improving Geolocation Accuracy of the Advanced Meteorological Imager on the GEO-KOMPSAT-2A

GeoNEX is a collaborative project led by scientists from NASA and many other international institutes to generate Earth monitoring products using data streams from the latest geostationary (GEO) sensors. Its consistent processing and common gridding systems can produce research-quality data products from GEO sensors and leverage GEO-GEO or GEO-LEO (low earth orbit) synergistic uses. Currently, GeoNEX has produced and disseminated L1G (geometrically corrected Level 1 products) from GOES 16/17 ABIs and Himawari-8 AHI, but a new Korean geostationary sensor (Advanced Meteorological Imager, AMI) onboard Geo-KOMPSAT-2A covering a large proportion of Asia and all of Oceania is in development. Our recent efforts on assessing geolocation accuracy in ABI and AHI suggest a nontrivial residual exists in both level 1B data with varying spatiotemporal patterns. The findings urge us to prioritize identifying and correcting geolocation residuals of AMI to generate accurate and consistent GeoNEX top-of-atmosphere (TOA) reflectance products and following processing chains. Here we implement a phase correlation correction approach to a visible band (0.64 μm, 500 m) using landmarks prepared from finer scale digital terrain models. We characterize spatiotemporal patterns (e.g., diurnal & daily) of geolocation residuals of AMI before and after correction. We then assess stability of datasets and quantify impact of unexpected geolocation errors on terrestrial monitoring. The geolocation corrected AMI data are further compared with GeoNEX AHI L1G products which are able to create unique stereo-type observations with AMI through leveraging the similarities of spectral bands and the sun-target-sensor geometry. Further, we discuss challenges in utilizing the GEO-GEO (e.g., AMI & AHI) satellite data for potential applications.

Geostationary Satellites↗

Multiple Laser Tracker Synchronization for the Vibration Analysis

Measuring the mechanical coupling and rigidity of a large apparatus presents unique challenges for 3D metrology. In this case the motion of well separated points attached to a rotor blade must be rapidly and repeatedly measured and the resulting data synchronized. This presentation describes the methodology and results for the mechanical coupling test of the magnetic field mapping system (FMS) for the Mu2e experiment at Fermi National Accelerator Laboratory (Fermilab). Three laser trackers, 2 API Radians and an API TIII, simultaneously measured the position of three cylindrical corner cube reflectors mounted on the FMS while the system was in motion and after coming to rest. The laser trackers took measurements at 83Hz for the T3 and 100Hz for the Radians, with fine spatial resolution, resulting in over 20,000 points measured per instrument in observation cycle. The laser tracker hardware and utilized software are unable to synchronize the three independent data streams directly during the measurement process. Instead a custom software program was developed to perform this synchronization of the data sequences during post-processing using features of the targets motion common to all three fiducial locations.

43 PARTICLE ACCELERATORS↗