Search NASA⌕ Search

SEARCH · Search NASA

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

HC-NIDS: signatures and simulations for detecting cyber-attacks aiming to cause damage against cyber-physical energy systems

Physical device safety is typically implemented locally using embedded controllers, while operations safety is primarily performed in control centers. Safe operations can be enhanced by correct design of device-level control algorithms, and protocols, procedures and operator training at the control-room level, but all can fail. Moreover, these elements exchange data and issue commands via vulnerable communication layers. In order to secure these gaps and enhance operational safety, we believe monitoring of command sequences must be combined with an awareness of physical device limitations and automata models that capture safety mechanisms. One way of doing this is by leveraging specification-based intrusion detection to monitor for physical constraint violations. The method can also verify that physical infrastructure state is consistent with monitoring information and control commands exchanged between field devices and control centers. This additional security layer enhances protection from both outsider attacks and insider mistakes. We implemented specification-based SCADA command analyzers using physical constraint algorithms directly in the Bro framework and Broccoli APIs for three separate scenarios: a water heater, an automated distribution system, and an over-current protection scheme. To accomplish this, we added low-level analyzers capable of examining control system-specific protocol packets for both Modbus TCP and DNP3, and also higher-level analyzers able to interpret device command and data streams within the context of each device's physical capabilities and present operational state. Thus the software that we are making available includes the Bro/Broccoli scripts for these three scenarios, as well as simulators, written in C, of those scenarios that generate sample traffic that is monitored by the Bro/Broccoli scripts. In addition, we have also implemented systems to directly pull cyber-physical information from the OSIsoft PI historian system. We have included the Python scripts used to perform that monitoring.

Source record↗

Sonar

Sonar contains software for managing a HPC performance-monitoring software stack. This includes streaming data into a distributed Kafka message queue, storing/querying from a distributed Cassandra database, and front-end tools for secure access, visualization, and analytics of facility-wide performance data.

Gimenez, AlfredoA.↗

bokeh streaming [SWR 19-08]

Simple examples of bokeh streaming data. The plots are intended to be generic and able to infer some of the plot data from the structure of the data sent. The receiver accepts zlib compressed pickled python dictionaries.

Simpson, Jeffery↗

Commercial Routing Assistance

The Commercial Routing Assistance (CRA) tool merges coordinated and vetted data streams, plots multiple automated or custom routing options, and visualizes the wide variety of state regulations and actions that a driver would encounter along a route. With CRA, commercial operators can plan vehicle movements across multiple states quickly, particularly during emergencies.

Klett, TimothyR.↗

DER Cybersecurity Detection and Response Suite

SAND2024-08475O The Distributed Energy Resource (DER) Cybersecurity Detection and Response Suite is a solution for distributed energy resource (DER) systems. The DER Security Orchestration, Automation, and Response (SOAR) solution that uses alerts from signature- and behavior-based Intrusion Detection Systems are intended to be deployed as bump-in-the-wire (BITW) devices in front of DER equipment. The fielded application would use multiple intrusion detection systems that report data to SOAR to respond to cyberattacks. The suite consists of two software components: • The proactive intrusion detection and mitigation system (PIDMS) secures grid-edge photovoltaic smart inverters and other equipment in distributed energy resource systems. It is a distributed BITW solution; cyber and physical data are automatically processed using network inspection tools and custom machine learning algorithms to detect abnormal events and correlate cyber-physical events. • The Security Orchestration, Automation, and Response for Distributed Energy Resources (SOAR4DER) application ingests data from several intrusion detection systems to quickly block attacks and revert DER systems to good states. Using a collection of intrusion detection system technologies on a BITW device, it incorporates physical and cyber data to detect abnormal and potential malicious behaviors. Multiple SOAR playbooks then use the intrusion detection system data streams to automatically defend the system. SOAR4DER system testing showed detection and response times under 30 seconds for all adversary reconnaissance, denial-of-service attacks, malicious Modbus commands, brute-force logins, and machine-in-the-middle attacks. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Johnson, Jay↗

A Geometry-Driven Longitudal Topic Model

A simple and scalable framework for longitudinal analysis of Twitter data is developed that combines latent topic models with computational geometric methods. Dimensionality reduction tools from computational geometry are applied to learn the intrinsic manifold on which the latent, temporal topics reside. Then shortest path distances on the manifold are used to link together these topics. The proposed framework permits visualization of the low-dimensional embedding which provides clear interpretation of the complex, high-dimensional trajectories that may exist among latent topics. Practical application of the proposed framework is demonstrated through its ability to capture and effectively visualize natural progression of latent COVID-19 related topics learned from Twitter data. Interpretability of the trajectories is achieved by comparing to real-world events. In addition, the framework permits study of spatial variation in Twitter behavior for learned topics. The analysis demonstrates that the proposed framework is able to capture granular-level impact of COVID-19 on public discussions. We end by arguing that Twitter data, when analyzed within the proposed framework, can serve as a valuable supplementary data stream for COVID-related studies.

97 MATHEMATICS AND COMPUTING↗

The Surface Atmosphere Integrated Field Laboratory (SAIL) Campaign

The science of mountainous hydrology spans the atmosphere through the bedrock and inherently crosses physical and disciplinary boundaries: land-atmosphere interactions in complex terrain enhance clouds and precipitation, while watersheds retain and release water over a large range of spatial and temporal scales. Limited observations in complex terrain challenge efforts to improve predictive models of the hydrology in the face of rapid changes. The Upper Colorado River exemplifies these challenges, especially with ongoing mismatches between precipitation, snowpack, and discharge. Consequently, the U.S. Department of Energy’s (DOE) Atmospheric Radiation Measurement (ARM) user facility has deployed an observatory to the East River Watershed near Crested Butte, Colorado between September 2021 and June 2023 to measure the main atmospheric drivers of water resources, including precipitation, clouds, winds, aerosols, radiation, temperature and humidity. This effort, called the Surface Atmosphere Integrated Field Laboratory (SAIL), is also working in tandem with DOE-sponsored surface and subsurface hydrologists and other federal, state, and local partners. SAIL data can be benchmarks for model development by producing a wide range of observational information on precipitation and its associated processes, including those processes that impact snowpack sublimation and redistribution, aerosol direct radiative effects in the atmosphere and in the snowpack, aerosol impacts on clouds and precipitation, and processes controlling surface fluxes of energy and mass. Preliminary data from SAIL’s first year showcase the rich information content in SAIL’s many data-streams and support testing hypotheses that will ultimately improve scientific understanding and predictability of Upper Colorado River hydrology in 2023 and beyond.

54 ENVIRONMENTAL SCIENCES↗

HUMAN-ARTIFICIAL INTELLIGENCE TEAMING FOR THE U.S. NAVY: DEVELOPING A HOLISTIC RESEARCH ROADMAP

With the ever-increasing deluge of data and demand for warfighters to make decisions upon its analysis, U.S. defense strategy has prioritized the development of artificial intelligence (AI)/machine learning (ML) systems that can analyze multi-source data streams and suggest courses of action. However, there is a history of systems that have failed to be adopted by the warfighter due not only to unsolved technical challenges, but also a lack of usability or contributions to mission effectiveness, perceived or otherwise. To avoid this, research into human-AI teaming shows promise for developing AI systems that work with frontline operators. A recent National Academies of Sciences, Engineering, and Medicine report (NASEM, 2022) presented 57 research objectives in this area; however, the U.S. Navy requires a more-focused set of priorities, as it is impossible to tackle every priority. A workshop involving 23 human factors scientists, computer scientists, and active-duty sailors was organized at the Naval Information Warfare Center Pacific, resulting in a set of five research priorities spanning near-, mid-, and far-term time frames. This panel will summarize the results of this workshop, with a focus on the big questions both going into this workshop and coming out of it. The panel participants come from government, academia, and industry, providing perspective from the different kinds of organizations required to accomplish these research goals.

Wong, Jason↗

Evaluation of an open forecasting challenge to assess skill of West Nile virus neuroinvasive disease prediction

Abstract Background West Nile virus (WNV) is the leading cause of mosquito-borne illness in the continental USA. WNV occurrence has high spatiotemporal variation, and current approaches to targeted control of the virus are limited, making forecasting a public health priority. However, little research has been done to compare strengths and weaknesses of WNV disease forecasting approaches on the national scale. We used forecasts submitted to the 2020 WNV Forecasting Challenge, an open challenge organized by the Centers for Disease Control and Prevention, to assess the status of WNV neuroinvasive disease (WNND) prediction and identify avenues for improvement. Methods We performed a multi-model comparative assessment of probabilistic forecasts submitted by 15 teams for annual WNND cases in US counties for 2020 and assessed forecast accuracy, calibration, and discriminatory power. In the evaluation, we included forecasts produced by comparison models of varying complexity as benchmarks of forecast performance. We also used regression analysis to identify modeling approaches and contextual factors that were associated with forecast skill. Results Simple models based on historical WNND cases generally scored better than more complex models and combined higher discriminatory power with better calibration of uncertainty. Forecast skill improved across updated forecast submissions submitted during the 2020 season. Among models using additional data, inclusion of climate or human demographic data was associated with higher skill, while inclusion of mosquito or land use data was associated with lower skill. We also identified population size, extreme minimum winter temperature, and interannual variation in WNND cases as county-level characteristics associated with variation in forecast skill. Conclusions Historical WNND cases were strong predictors of future cases with minimal increase in skill achieved by models that included other factors. Although opportunities might exist to specifically improve predictions for areas with large populations and low or high winter temperatures, areas with high case-count variability are intrinsically more difficult to predict. Also, the prediction of outbreaks, which are outliers relative to typical case numbers, remains difficult. Further improvements to prediction could be obtained with improved calibration of forecast uncertainty and access to real-time data streams (e.g. current weather and preliminary human cases). Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗

Using heterogeneous data to identify signatures of dengue outbreaks at fine spatio-temporal scales across Brazil

Dengue virus remains a significant public health challenge in Brazil, and seasonal preparation efforts are hindered by variable intra- and interseasonal dynamics. Here, we present a framework for characterizing weekly dengue activity at the Brazilian mesoregion level from 2010–2016 as time series properties that are relevant to forecasting efforts, focusing on outbreak shape, seasonal timing, and pairwise correlations in magnitude and onset. In addition, we use a combination of 18 satellite remote sensing imagery, weather, clinical, mobility, and census data streams and regression methods to identify a parsimonious set of covariates that explain each time series property. The models explained 54% of the variation in outbreak shape, 38% of seasonal onset, 34% of pairwise correlation in outbreak timing, and 11% of pairwise correlation in outbreak magnitude. Regions that have experienced longer periods of drought sensitivity, as captured by the “normalized burn ratio,” experienced less intense outbreaks, while regions with regular fluctuations in relative humidity had less regular seasonal outbreaks. Both the pairwise correlations in outbreak timing and outbreak trend between mesoresgions were best predicted by distance. Our analysis also revealed the presence of distinct geographic clusters where dengue properties tend to be spatially correlated. Forecasting models aimed at predicting the dynamics of dengue activity need to identify the most salient variables capable of contributing to accurate predictions. Our findings show that successful models may need to leverage distinct variables in different locations and be catered to a specific task, such as predicting outbreak magnitude or timing characteristics, to be useful. This advocates in favor of “adaptive models” rather than “one-size-fits-all” models. The results of this study can be applied to improving spatial hierarchical or target-focused forecasting models of dengue activity across Brazil.

59 BASIC BIOLOGICAL SCIENCES↗

COMPASS-FME Terrestrial Ecosystem Manipulation to Probe the Effects of Storm Treatments (TEMPEST) Experiment Level 2 Sensor Data v2-1

This is the version v2-1 Level 2 (L2) data release for COMPASS-FME environmental sensors located at our Terrestrial Ecosystem Manipulation to Probe the Effects of Storm Treatments (TEMPEST) experimental site. This manipulative, ecosystem-scale TEMPEST experiment addresses the potential for freshwater and estuarine-water disturbance events to alter tree function, species composition, and ecosystem processes in a deciduous coastal forest in MD, USA. The experiment uses a large-unit (2000 m2), un-replicated experimental design, with three 50 m × 40 m plots serving as control, freshwater, and estuarine-water treatments. Level 2 (L2) data consist of sensor observations from the COMPASS-FME synoptic sites, TEMPEST, and DELUGE. Compared to the L1 data, these are more consistent (always 15-minute timestamps for the entire year); better QA/QC’d (out of bounds, out of service, and extreme outlier values are removed); and more complete, with a gap-filled time series available alongside the main observations, and additional derived (calculated) variables. L2 data are intended to be rapidly and easily usable in analyses and simulations. However, algorithmic outlier identification always carries the risk of removing valid data, and Level 1 data may be more suitable for analyses that focus on variability or extreme events. This dataset includes: - An overall dataset README file that describes the current version, gives citation and contact information, etc. - Site- and year-specific folders, each holding variable-specific Parquet (a high performance, space efficient format; see https://parquet.apache.org) data files for each site and plot in that year. - Metadata files within each site-year folder provide full information on data units, expected ranges, contact information, detailed flood times, as well as a general description of the site. - Environmental sensor types that appear in the data files include weather (ClimaVUE50, CS, RM Young, and LI instruments in the graphs below); soil conditions (TEROS12); soil redox state (Redox); groundwater variables (AquaTROLL200 and AquaTROLL600); open water sondes (Exo); tree sap velocity (Sapflow); and system voltage and state (Datalogger). Data are reported every 15 minutes. Please see v2-1 TEMPEST L2 Sensor Package Quick Start.pdf for detailed information on data package structure, temporal coverage, and versioning. Data files are in Apache Parquet, a high performance, space efficient format for tabular data. These files can be read using R's `arrow` package (https://arrow.apache.org/docs/r/), with similar tools available in other languages. The TEMPEST flood events occurred on the following dates. They lasted for ~10 hours each day and delivered ~80,000 gallons to each plot; many data streams are available at 1 or 5 minute frequency during these periods. * Tests: Aug 25 (fresh plot) and Sep 9 (salt plot), 2021 * TEMPEST 1: June 22, 2022 * TEMPEST 2: June 6-7, 2023 * TEMPEST 3: June 11-13, 2024

EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC TEMPERATU↗

International Monitoring System Correlation Detection at the North Korean Nuclear Test Site at Punggye-ri with Insights from the Source Physics Experiment

Seismic waveform correlation offers the prospect of greatly reducing event detection thresholds when compared with more conventional processing methods. Correlation is applicable for seismic events that in some sense repeat, that is they have very similar waveforms. A number of recent studies have shown that correlated seismic signals may form a significant fraction of seismicity at regional distances. For the particular case of multiple nuclear explosions at the same test site, regional distance correlation also allows very precise relative location measurements and could offer the potential to lower thresholds when multiple events exist. Using the Comprehensive Nuclear-Test-Ban Treaty (CTBT) International Monitoring System (IMS) seismic array at Matsushiro, Japan (MJAR), Gibbons and Ringdal (2012) were able to create a multichannel correlation detector with a very low false alarm rate and a threshold below magnitude 3.0. They did this using the 2006 or 2009 Democratic People’s Republic of Korea (DPRK) nuclear explosion as a template to search through a data stream from the same station to find a match via waveform correlation. In this paper, we extend the work of Gibbons and Ringdal (2012) and measure the correlation detection threshold at several other IMS arrays. We use this to address three main points. First, we show the IMS array station at Mina, Nevada (NVAR), which is closest to the Nevada National Security Site (NNSS), is able to detect a chemical explosion that is well under 1 ton with the right template. Second, we examine the two IMS arrays closest to the North Korean (DPRK) test site (at Ussuriysk, Russian Federation [USRK] and Wonju, Republic of Korea [KSRS]) to show that similarly low thresholds are possible when the right templates exist. We also extend the work of Schaff et al. (2012) and measure the correlation detection threshold at the nearest Global Seismic Network (GSN) three-component station (MDJ) at Mudanjiang, Heilongjiang Province, China, from the New China Digital Seismograph Network (IC). To conclude, we use these results to explore the recent claim by Zhang and Wen (2015) that the DPRK conducted “…a low-yield nuclear test…” on 12 May 2010.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Machine learning at the Spallation Neutron Source accelerator and target

We describe the ongoing efforts to apply Machine Learning techniques to improve the performance of our accelerator and target. Specially, we are looking to minimize halo beam losses in the absence of a proper physics model, automatically detect and log anomalies in the target support systems such as cooling, and detect and prevent errant beam pulses in the linac. We also describe the infrastructure we use to acquire and stream data to the GPU cluster for training, our code development cycle, and edge computing for model inference. To minimize halo beam losses, we use a Reinforcement Learning technique tested on a virtual accelerator. The target anomaly detection is trained on archived data using incomplete physics models and is made part of the existing target reporting system. The errant beam prevention analyzes beam current and beam phase waveforms as well as accelerator configuration data to predict errant pulses. We also develop continual learning to adapt to changes in the accelerator.

Accelerator Physics↗

Weighting and Bayes Nets for Rollup of Surveillance Metrics

The LANL IKE team proposes that the surveillance metrics for several data stream that are used to detect the same failure mode be weighted. Similarly, the failure mode metrics are weighted to obtain a subsystem metric.

97 MATHEMATICS AND COMPUTING↗

The Los Alamos Science Pillars The Science of Signatures

As a national security science laboratory, Los Alamos is often asked to detect and measure the characteristics of complex systems and to use the resulting information to quantify the system's behavior. The Science of Signatures (SoS) pillar is the broad suite of technical expertise and capability that we use to accomplish this task. With it, we discover new signatures, develop new methods for detecting or measuring signatures, and deploy new detection technologies. The breadth of work at Los Alamos National Laboratory (LANL) in SoS is impressive and spans from the initial understanding of nuclear weapon performance during the Manhattan Project, to unraveling the human genome, to deploying laser spectroscopy instrumentation on Mars. Clearly, SoS is a primary science area for the Laboratory and we foresee that as it matures, new regimes of signatures will be discovered and new ways of extracting information from existing data streams will be developed.

Applied Life Science(60)↗

Determination of the standard deviation on multiplication based on count distributions

The multiplication M of a multiplying object can be determined by analyzing the arrival times of thermal neutrons in detectors such as helium tubes. A source emitting neutrons randomly will lead to neutron inter arrival times obeying Poisson process statistics. By observing how the inter arrival times of detected counts differ from Poisson statistics, one can assess properties of the neutron source such as the source intensity and multiplication.The results of this study show that analyzing the data stream using non-overlapping time gates is an improvement over the current analysis using overlapping time gates. The new method enables us to compute the standard deviation on the multiplication parameter M accurately. Also, this standard deviation on M decreases as the largest time gate is about 10 times the characteristic time constant λ -1 of the system.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Gary Laughlin IHNS Intro to 2015 HPC Annual Report

The International, Homeland, and Nuclear Security (IHNS) Program Management Unit (PMU) oversees a broad portfolio of Sandia’s programs in areas ranging from global nuclear security to critical asset protection. We use science and technology, innovative research, and global engagement to counter threats, reduce dangers, and respond to disasters. The PMU draws on the skills of scientists and engineers from across Sandia. Our programs focus on protecting US government installations, safeguarding nuclear weapons and materials, facilitating nonproliferation activities, securing infrastructures, countering chemical and biological dangers, and reducing the risk of terrorist threats. We conduct research in risk and threat analysis, monitoring and detection, decontamination and recovery, and situational awareness. We develop technologies for verifying arms control agreements, neutralizing dangerous materials, detecting intruders, and strengthening resiliency. Our programs use Sandia’s High-Performance Computing resources for predictive modeling and simulation of interdependent systems, for modeling dynamic threats and forecasting adaptive behavior, and for enabling decision support and processing large cyber data streams. In this report, we highlight four advanced computation projects that illustrate the breadth of the IHNS mission space.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Biosurveillance Using Clinical Diagnoses and Social Media Indicators in Military Populations

U.S. military influenza surveillance uses electronic reporting of clinical diagnoses to monitor health of military personnel and detect naturally occurring and bioterrorism-related epidemics. While accurate, these systems lack in timeliness. More recently, researchers have used novel data sources to detect influenza in real time and capture nontraditional populations. With data-mining techniques, military social media users are identified and influenza-related discourse is integrated along with medical data into a comprehensive disease model. By leveraging heterogeneous data streams and developing dashboard biosurveillance analytics, the researchers hope to increase the speed at which outbreaks are detected and provide accurate disease forecasting among military personnel.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗