Search NASA⌕ Search

SEARCH · Search NASA

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

AFRL Additive Manufacturing Modeling Series: Challenge 4, In Situ Mechanical Test of an IN625 Sample with Concurrent High-Energy Diffraction Microscopy Characterization

In this work, we describe 3D characterization of an additively manufactured Inconel 625 nickel-base superalloy specimen conducted during a uniaxial tension test using a suite of nondestructive x-ray techniques. High-energy diffraction microscopy in both near- and far-field modalities are employed in situ to track evolution of the material orientation and stress–strain fields at six points during the mechanical test, and these data streams are registered with micro-computed tomography reconstructions which probe the material density. This data volume was matched to a multi-modal serial sectioning characterization of the specimen taken after loading, described in this article’s companion. Twenty-eight grains which were monitored throughout the experiment were selected to form the basis for AFRL AM Modeling Series Challenge 4, Microscale Structure-to-Properties.

36 MATERIALS SCIENCE↗
DOE OSTI2016

CLARA: CLAS12 Reconstruction and Analysis Framework

In this paper we present SOA based CLAS12 event Reconstruction and Analyses (CLARA) framework. CLARA design focus is on two main traits: real-time data stream processing, and service-oriented architecture (SOA) in a flow based programming (FBP) paradigm. Data driven and data centric architecture of CLARA presents an environment for developing agile, elastic, multilingual data processing applications. The CLARA framework presents solutions capable of processing large volumes of data interactively and substantially faster than batch systems.

Gyurjyan, Vardan↗
DOE OSTI2015

Component based dataflow processing framework

In this paper we present SOA based CLAs12 event Reconstruction and Analyses (CLARA) framework used to develop Earth Science multi-sensor data fusion, processing, and analytics applications (NAIADS: NASA JLAB collaboration). CLARA design focus is on two main traits: a) real-time data stream processing, and b) service oriented architecture (SOA) in a flow based programming (FBP) paradigm. Data driven and data centric architecture of CLARA presents an environment for developing agile, elastic, multilingual data processing applications. The CLARA framework presents solutions, capable of processing large volumes of data interactively and substantially faster than batch systems.

Gyurjyan, V.↗
DOE OSTI2018

A Multiscale Approach to Modeling Carbon and Nitrogen Cycling within a High Elevation Watershed

This funding represents a small sub-award related the larger project titled: A Multiscale Approach to Modeling Carbon and Nitrogen Cycling within a High Elevation Watershed. The goal of the sub-award was to facilitate the characterization of carbon and radiocarbon data collected from the East River watershed outside Gothic, Colorado USA. During the period of funding from 8/1/15 until 7/31/17, we sampled 40 soil profiles and collected ~325 soil samples. This funding supported the collection, processing, and elemental analysis of each of these samples. In addition, the funding allowed for the further density separation of a subset of soil resulting in 60 measurements of 13 C and 14 C of bulk soil and density separates. Funding also supported installation of temperature and moisture data sensors arrays, soil gas wells, and soil water lysimeters. From this infrastructure, a steady stream data including soil gas, water, and physical information have been generated to support the larger research project.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Event Definition for the Automated Detection of Nuclear Proliferation Activities

In FY2020, Savannah River National Laboratory (SRNL) in collaboration with the Discovery Analytics Center (DAC) at Virginia Polytechnic Institute and State University (VT) began developing a demonstration prototype system that uses multiple machine learning and data analytic methods on largescale open data sources to identify new, developing, or undeclared nuclear programs. One of the most challenging aspects of applying machine learning techniques to such a problem is the high likelihood of extremely sparse data from disparate sources. To overcome this challenge, the current work will use a strategic combination of supervised, semi-supervised, and unsupervised learning techniques to ingest and fuse data streams to make a forecast of nuclear activities in a targeted geospatial location. Identifying potential data sources and training supervised learning algorithms is dependent upon the development of a robust foundation of targeted event domains that fundamentally define the nuclear activities of interest. This report documents the definition of a hierarchical structure for both nuclear activity and event domains that will be used to guide the research team in development or use of existing semantic dictionaries that are instrumental to searching, parsing, and categorizing events for the forecasting system’s use.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Using Data-Driven Prediction of Downstream 1D River Flow to Overcome the Challenges of Hydrologic River Modeling

Methods for downstream river flow prediction can be categorized into physics-based and empirical approaches. Although based on well-studied physical relationships, physics-based models rely on numerous hydrologic variables characteristic of the specific river system that can be costly to acquire. Moreover, simulation is often computationally intensive. Conversely, empirical models require less information about the system being modeled and can capture a system’s interactions based on a smaller set of observed data. This article introduces two empirical methods to predict downstream hydraulic variables based on observed stream data: a linear programming (LP) model, and a convolutional neural network (CNN). We apply both empirical models within the Colorado River system to a site located on the Green River, downstream of the Yampa River confluence and Flaming Gorge Dam, and compare it to the physics-based model Streamflow Synthesis and Reservoir Regulation (SSARR) currently used by federal agencies. Results show that both proposed models significantly outperform the SSARR model. Moreover, the CNN model outperforms the LP model for hourly predictions whereas both perform similarly for daily predictions. Although less accurate than the CNN model at finer temporal resolution, the LP model is ideal for linear water scheduling tools.

13 HYDRO ENERGY↗
NASA NTRS1973

Data compression by a decreasing slope-threshold test

Resolution can be obtained at large compression ratios with method for selecting data points for transmission by telemetry in television compressed-data system. Test slope of raw data stream and compare it to symmetric pair of decreasing thresholds. When either threshold is exceeded, data are sampled and transmitted; thresholds are reset, and test begins again.

Kleinrock, L.↗
NASA NTRS1972

The Mariner Venus Mercury flight data subsystem.

The flight data subsystem (FDS) discussed handles both the engineering and scientific measurements performed on the MVM'73. It formats the data into serial data streams, and sends it to the modulation/demodulation subsystem for transmission to earth or to the data storage subsystem for storage on a digital tape recorder. The FDS is controlled by serial digital words, called coded commands, received from the central computer sequencer of from the ground via the modulation/demodulation subsystem. The eight major blocks of the FDS are: power converter, timing and control, engineering data, memory, memory input/output and control, nonimaging data, imaging data, and data output. The FDS incorporates some 4000 components, weighs 17 kg, and uses 35 W of power. General data on the mission and spacecraft are given.

Whitehead, P. B.↗
NASA NTRS1995

A satellite based telemetry link for a UAV application

The requirements for a satellite based communication facility to service the needs of the Geographical Information System (GIS) data collection community are addressed in this paper. GIS data is supplied in the form of video imagery at sub-television rates in one or more spectral bands / polarizations laced with a position correlated data stream. The limitations and vicissitudes of using a terrestrial based telecommunications link to collect GIS data are illustrated from actual mission scenarios. The expectations from a satellite based communications link by the geophysical data collection community concerning satellite architecture, operating bands, bandwidth, footprint agility, up link and down link hardware configurations on the UAV, the Mobile Control Vehicle and at the Central Command and Data Collection Facility comprise the principle issues discussed in the first section of this paper. The final section of the paper discusses satellite based communication links would have an increased volume and scope of services the GIS data collection community could make available to the GIS user community, and the price the data collection community could afford to pay for access to the communication satellite described in the paper.

Bloise, Anthony↗
NASA NTRS2002

Onboard Processor for Compressing HSI Data

With EO-1 Hyperion and MightySat in orbit NASA and the DoD are showing their continued commitment to hyperspectral imaging (HSI). As HSI sensor technology continues to mature, the ever-increasing amounts of sensor data generated will result in a need for more cost effective communication and data handling systems. Lockheed Martin, with considerable experience in spacecraft design and developing special purpose onboard processors, has teamed with Applied Signal & Image Technology (ASIT), who has an extensive heritage in HSI, to develop a real-time and intelligent onboard processing (OBP) system to reduce HSI sensor downlink requirements. Our goal is to reduce the downlink requirement by a factor greater than 100, while retaining the necessary spectral fidelity of the sensor data needed to satisfy the many science, military, and intelligence goals of these systems. Our initial spectral compression experiments leverage commercial-off-the-shelf (COTS) spectral exploitation algorithms for segmentation, material identification and spectral compression that ASIT has developed. ASIT will also support the modification and integration of this COTS software into the OBP. Other commercially available COTS software for spatial compression will also be employed as part of the overall compression processing sequence. Over the next year elements of a high-performance reconfigurable OBP will be developed to implement proven preprocessing steps that distill the HSI data stream in both spectral and spatial dimensions. The system will intelligently reduce the volume of data that must be stored, transmitted to the ground, and processed while minimizing the loss of information.

Cook, Sid↗
NASA NTRSDate not supplied

Biological Research and Space Health Enabled by Machine Learning to Support Deep Space Missions

A key science goal of the NASA “Moon to Mars” campaign is to understand how biology responds to the Lunar, Martian, and deep space environments in order to advance fundamental knowledge, reduce risk, and support safe, productive human space missions. Through the powerful emerging approaches of artificial intelligence (AI) and machine learning (ML), a paradigm shift has begun in biomedical science and engineered astronaut health systems, to enable Earth independence and autonomy of mission operations. Here we present an overview of AI/ML architecture to support deep space mission goals, developed with leaders in the field. First, we focus on the fundamental biological research that supports our understanding of physiological responses to spaceflight, and we describe current efforts to support AI/ML research including data standardization and data engineering through maximally open and FAIR (findable, accessible, interoperable, reusable) databases and the generation of AI-ready datasets for reuse and analysis. We also discuss remote data management frameworks for research data as well as environmental and health data that are generated during deep space missions. We highlight several research projects that leverage data standardization and management for fundamental biological discovery to uncover the complex effects of space travel on living systems. Next, we provide an overview of cutting-edge AI/ML approaches that can be integrated to support remote monitoring and analysis during deep space missions, including generative models and large language models to learn the underlying biomedical patterns and predict outcomes or answer questions during off world medical scenarios. We also describe current AI/ML methods to support this research and monitoring through automated cloud-based labs which enable limited human intervention and closed-loop experimentation in remote settings. These labs could support mission autonomy by analyzing environmental data streams, and would be facilitated through in situ analytics capabilities to avoid sending large raw data files through low bandwidth communications. Finally, in the context of deep space missions with limited communications or access to medical advice from Earth, we describe a solution for integrated, real-time mission biomonitoring across hierarchical levels from continuous environmental monitoring, to wearables and point-of-care devices, to molecular and physiological monitoring. We introduce a precision space health system that will ensure that the future of space health is predictive, preventative, participatory and personalized.

artificial intelligence↗
DOE OSTI2022

Sensing Electrical Networks Securely & Economically (SENSE)

The growing adoption of distributed energy resources (DERs) like battery energy storage systems and roof top solar/PV and the rapid penetration of electric vehicles (EVs), the electric grid is undergoing a major transformation with elevated stress on legacy grid assets. Despite a lot of expenditure to address these challenges, both in dollars and manpower, utilities have not been able to receive the value that was promised. The gains have been most visible at the transmission and substation level, especially where the main objective was improving operational and economic efficiency for the utility. Improving visibility and control at a few select points enhances the existing and established paradigm of centralized command and control. With changing load patterns, load types and the overall transition to an “active grid”, the centralized control and coordination paradigm gets challenged. To address the challenges, a new architecture and mechanism is needed, one that supports decentralized control and decision making, extracting value streams at the grid edge, particularly as the changes are fueled by transitions occurring in the distribution system. To address this, a communications and data processing platform, “GAMMA” was developed and demonstrated through the project. At the heart of the platform, are distributed, intelligent edge nodes with sensing and compute capabilities, that can record and analyze information locally. They are embedded in sensors and actuators specific to different distribution system applications. Phase 1 of the project focused on developing novel sensor technology that can be used for monitoring utility pole top distribution transformers. The sensors were designed with the objective of being low-cost, communicating with the GAMMA cloud using novel “delay-tolerant” networking using Bluetooth and a secure mobile application. They were non-intrusive in nature so that they can be installed quickly in the field, resulting in overall low cost of deployment and operations. Following the successful completion of Phase 1, the team manufactured 100 units for a field demonstration in Phase 2. The field demonstration was carried out on two real feeder systems with the local utility partner. In total, 100 sensors were installed and operated over a period of 6 months in the state of Georgia. The platform is operational end to end, with the cloud infrastructure deployed on a distributed, serverless environment that can serve multiple data streams, an analytics engine and a portal to securely view the data from multiple assets. The data collected through the GAMMA Mobile Phone app showcased the viability of the novel delay tolerant networking architecture, and the data processing algorithms developed through the course of the project, were successful in extracting important information about the overall network, improving the utility’s visibility and situational awareness in the distribution feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Integrated photonic encoder for low power and high-speed image processing

Abstract Modern lens designs are capable of resolving greater than 10 gigapixels, while advances in camera frame-rate and hyperspectral imaging have made data acquisition rates of Terapixel/second a real possibility. The main bottlenecks preventing such high data-rate systems are power consumption and data storage. In this work, we show that analog photonic encoders could address this challenge, enabling high-speed image compression using orders-of-magnitude lower power than digital electronics. Our approach relies on a silicon-photonics front-end to compress raw image data, foregoing energy-intensive image conditioning and reducing data storage requirements. The compression scheme uses a passive disordered photonic structure to perform kernel-type random projections of the raw image data with minimal power consumption and low latency. A back-end neural network can then reconstruct the original images with structural similarity exceeding 90%. This scheme has the potential to process data streams exceeding Terapixel/second using less than 100 fJ/pixel, providing a path to ultra-high-resolution data and image acquisition systems.

47 OTHER INSTRUMENTATION↗
DOE OSTI2017

Pattern Discovery in Internet Background Radiation

Internet Background Radiation (IBR) is observed in empty network address spaces. No traffic should arrive there, but it does in overwhelming quantities, gathering evidences of attacks, malwares and misconfigurations. The study of IBR helps to detect spreading network problems, common vulnerabilities and attack trends. However, network traffic data evolves quickly and is of high volume and diversity, i.e., an outstanding big data challenge. When used to assist network security, it also requires the online classification of dynamic streaming data. In this paper, we introduce an AGgregation & Mode (AGM) vector to represent network traffic. The AGM format characterizes IP hosts by extracting aggregated and mode values of IP header fields, and without inspecting payloads. We performed clustering and statistical analysis to explore six months of IBR from 2012 with the AGM mapping. The discovered patterns allow building a classification of IBR, which identifies phenomena that have been actively polluting the Internet for years. The AGM representation is light and tailored for monitoring and pattern discovery. We show that AGM vectors are suitable to analyze large volumes of network traffic: they capture permanent operations, such as long term scanning, as well as bursty events from targeted attacks and short term incidents.

Iglesias, Felix↗
DOE OSTI2021

Online Voltage Event Detection Using Synchrophasor Data with Structured Sparsity-Inducing Norms

This paper develops an accurate and computationally efficient data-driven framework to detect voltage events from PMU data streams. It develops an innovative Proximal Bilateral Random Projection (PBRP) algorithm to quickly decompose the PMU data matrix into a low-rank matrix, a row-sparse event-pattern matrix and a noise matrix. Here, the row-sparse pattern matrix significantly distinguishes events from normal behavior. These matrices are then fed into a clustering algorithm to separate voltage events from normal operating conditions. Large-scale numerical study results on real-world PMU data show that the proposed algorithm is computationally more efficient and achieves higher F scores than state-of-the-art benchmarks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2016

Use of Integrated Instrumentation to Detect and Classify Targets in Shallow Water

The detection and classification of targets in shallow water environments (e.g., depth < 20 m) is of increasing interest, both in security applications and more general environmental studies. Here, we report the ability of a cabled integrated instrumentation system to detect and classify targets in a shallow (15 m deep), narrow (150 m wide) channel influenced by moderate tidal currents (peak currents < 2 m/s). The instrumentation system incorporates a passive acoustic array, multibeam sonar, acoustic camera, stereo-optical camera, and acoustic Doppler current profiler. The system effectiveness is benchmarked using autonomous “cooperative” targets (e.g., drifters with known position and acoustic characteristics), divers, and opportunistic observations of fish and marine mammals. By fusing data streams from multiple sensors, detection and classification rates are improved, particularly for smaller targets. Approaches to data management (data rates exceed 80 MB/s), real-time classification, and de-noising of acoustic imagery are also discussed.

16 TIDAL AND WAVE POWER↗
DOE OSTI2017

The CompTox Chemistry Dashboard: a community data resource for environmental chemistry

Despite an abundance of online databases providing access to chemical data, there is increasing demand for highquality, structure-curated, open data to meet the various needs of the environmental sciences and computational toxicology communities. The U.S. Environmental Protection Agency’s (EPA) web-based CompTox Chemistry Dashboard is addressing these needs by integrating diverse types of relevant domain data through a cheminformatics layer, built upon a database of curated substances linked to chemical structures. These data include physicochemical, environmental fate and transport, exposure, usage, in vivo toxicity, and in vitro bioassay data, surfaced through an integration hub with link-outs to additional EPA data and public domain online resources. Batch searching allows for direct chemical identifer (ID) mapping and downloading of multiple data streams in several diferent formats. This facilitates fast access to available structure, property, toxicity, and bioassay data for collections of chemicals (hundreds to thousands at a time). Advanced search capabilities are available to support, for example, non-targeted analysis and identifcation of chemicals using mass spectrometry. The contents of the chemistry database, presently containing ~ 760,000 substances, are available as public domain data for download. The chemistry content underpinning the Dashboard has been aggregated over the past 15 years by both manual and auto-curation techniques within EPA’s DSSTox project. DSSTox chemical content is subject to strict quality controls to enforce consistency among chemical substance-structure identifers, as well as list curation review to ensure accurate linkages of DSSTox substances to chemical lists and associated data. The Dashboard, publicly launched in April 2016, has expanded considerably in content and user trafc over the past year. It is continuously evolving with the growth of DSSTox into high-interest or data-rich domains of interest to EPA, such as chemicals on the Toxic Substances Control Act listing, while providing the user community with a fexible and dynamic web-based platform for integration, processing, visualization and delivery of data and resources. The Dashboard provides support for a broad array of research and regulatory programs across the worldwide community of toxicologists and environmental scientists.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

"Sintering" Models and In-Situ Experiments: Data Assimilation for Microstructure Prediction in SLS Additive Manufacturing of Nylon Components

Selective laser sintering methods are workhorses for additively manufacturing polymer-based components. The ease of rapid prototyping also means it is easy to produce illicit components. It is necessary to have a data-calibrated in-situ physical model of the build process in order to predict expected and defective microstructure characteristics that inform component provenance. Toward this end, sintering models are calibrated and characteristics such as component defects are explored. This is accomplished by assimilating multiple data streams, imaging analysis, and computational model predictions in an adaptive Bayesian parameter estimation algorithm. From these data sources, along with a phase-field model, bulk porosity distributions are inferred. Model parameters are constrained to physically-relevant search directions by sensitivity analysis, and then matched to predictions using adaptive sampling. Using this feedback loop, data-constrained estimates of sintering model parameters along with uncertainty bounds are obtained.

3D printing, Additive Manufacturing, polymer, sint↗