Search NASA⌕ Search

SEARCH · Search NASA

Results for “detection network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2014

Anomaly Detection in Dynamic Networks

Anomaly detection in dynamic communication networks has many important security applications. These networks can be extremely large and so detecting any changes in their structure can be computationally challenging; hence, computationally fast, parallelisable methods for monitoring the network are paramount. For this reason the methods presented here use independent node and edge based models to detect locally anomalous substructures within communication networks. As a first stage, the aim is to detect changes in the data streams arising from node or edge communications. Throughout the thesis simple, conjugate Bayesian models for counting processes are used to model these data streams. A second stage of analysis can then be performed on a much reduced subset of the network comprising nodes and edges which have been identified as potentially anomalous in the first stage. The first method assumes communications in a network arise from an inhomogeneous Poisson process with piecewise constant intensity. Anomaly detection is then treated as a changepoint problem on the intensities. The changepoint model is extended to incorporate seasonal behavior inherent in communication networks. This seasonal behavior is also viewed as a changepoint problem acting on a piecewise constant Poisson process. In a static time frame, inference is made on this extended model via a Gibbs sampling strategy. In a sequential time frame, where the data arrive as a stream, a novel, fast Sequential Monte Carlo (SMC) algorithm is introduced to sample from the sequence of posterior distributions of the change points over time. A second method is considered for monitoring communications in a large scale computer network. The usage patterns in these types of networks are very bursty in nature and don’t fit a Poisson process model. For tractable inference, discrete time models are considered, where the data are aggregated into discrete time periods and probability models are fitted to the communication counts. In a sequential analysis, anomalous behavior is then identified from outlying behavior with respect to the fitted predictive probability models. Seasonality is again incorporated into the model and is treated as a changepoint model on the transition probabilities of a discrete time Markov process. Second stage analytics are then developed which combine anomalous edges to identify anomalous substructures in the network.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1991

NMESys: An expert system for network fault detection

The problem of network management is becoming an increasingly difficult and challenging task. It is very common today to find heterogeneous networks consisting of many different types of computers, operating systems, and protocols. The complexity of implementing a network with this many components is difficult enough, while the maintenance of such a network is an even larger problem. A prototype network management expert system, NMESys, implemented in the C Language Integrated Production System (CLIPS). NMESys concentrates on solving some of the critical problems encountered in managing a large network. The major goal of NMESys is to provide a network operator with an expert system tool to quickly and accurately detect hard failures, potential failures, and to minimize or eliminate user down time in a large network.

Nelson, Peter C.↗
DOE OSTI2022

Evaluating cloud liquid detection against Cloudnet using cloud radar Doppler spectra in a pre-trained artificial neural network

Detection of liquid-containing cloud layers in thick mixed-phase clouds or multi-layer cloud situations from ground-based remote-sensing instruments still poses observational challenges, yet improvements are crucial since the existence of multi-layer liquid layers in mixed-phase cloud situations influences cloud radiative effects, cloud lifetime, and precipitation formation processes. Hydrometeor target classifications such as from Cloudnet that require a lidar signal for the classification of liquid are limited to the maximum height of lidar signal penetration and thus often lead to underestimations of liquid-containing cloud layers. Here we evaluate the Cloudnet liquid detection against the approach of Luke et al. (2010) which extracts morphological features in cloud-penetrating cloud radar Doppler spectra measurements in an artificial neural network (ANN) approach to classify liquid beyond full lidar signal attenuation based on the simulation of the two lidar parameters particle backscatter coefficient and particle depolarization ratio. We show that the ANN of Luke et al. (2010) which was trained under Arctic conditions can successfully be applied to observations at the mid-latitudes obtained during the 7-week-long ACCEPT field experiment in Cabauw, the Netherlands, in 2014. In a sensitivity study covering the whole duration of the ACCEPT campaign, different liquid-detection thresholds for ANN-predicted lidar variables are applied and evaluated against the Cloudnet target classification. Independent validation of the liquid mask from the standard Cloudnet target classification against the ANN-based technique is realized by comparisons to observations of microwave radiometer liquid-water path, ceilometer liquid-layer base altitude, and radiosonde relative humidity. In addition, a case-study comparison against the cloud feature mask detected by the space-borne lidar aboard the CALIPSO satellite is presented. Three conclusions were drawn from the investigation. First, it was found that the threshold selection criteria of liquid-related lidar backscatter and depolarization alone control the liquid detection considerably. Second, all threshold values used in the ANN framework were found to outperform the Cloudnet target classification for deep or multi-layer cloud situations where the lidar signal is fully attenuated within low liquid layers and the cloud radar is able to detect the microphysical fingerprint of liquid in higher cloud layers. Third, if lidar data are available, Cloudnet is at least as good as the ANN. The times when Cloudnet outperforms the ANN in liquid detections are often associated with situations where cloud dynamics smear the imprint of cloud microphysics on the radar Doppler spectra.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Power System Event Classification and Localization Using a Convolutional Neural Network

Detection and timely identification of power system disturbances are essential for situation awareness and reliable electricity grid operation. Because records of actual events in the system are limited, ensemble simulation-based events are needed to provide adequate data for building event-detection models through deep learning; e.g., a convolutional neural network (CNN). An ensemble numerical simulation-based training data set have been generated through dynamic simulations performed on the Polish system with various types of faults in different locations. Such data augmentation is proven to be able to provide adequate data for deep learning. The synchronous generators’ frequency signals are used and encoded into images for developing and evaluating CNN models for classification of fault types and locations. With a time-domain stacked image set as the benchmark, two different time-series encoding approaches, i.e., wavelet decomposition-based frequency-domain stacking and polar coordinate system-based Gramian Angular Field (GAF) stacking, are also adopted to evaluate and compare the CNN model performance and applicability. The various encoding approaches are suitable for different fault types and spatial zonation. With optimized settings of the developed CNN models, the classification and localization accuracies can go beyond 84 and 91%, respectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2016

GraphPrints: Towards a Graph Analytic Method for Network Anomaly Detection

This paper introduces a novel graph-analytic approach for detecting anomalies in network flow data called \textit{GraphPrints}. Building on foundational network-mining techniques, our method represents time slices of traffic as a graph, then counts graphlets\textemdash small induced subgraphs that describe local topology. By performing outlier detection on the sequence of graphlet counts, anomalous intervals of traffic are identified, and furthermore, individual IPs experiencing abnormal behavior are singled-out. Initial testing of GraphPrints is performed on real network data with an implanted anomaly. Evaluation shows false positive rates bounded by 2.84\% at the time-interval level, and 0.05\% at the IP-level with 100\% true positive rates at both.

anomaly detection↗
DOE OSTI2013

HPNAIDM: The High-Performance Network Anomaly/Intrusion Detection and Mitigation System

Identifying traffic anomalies and attacks rapidly and accurately is critical for large network operators. With the rapid growth of network bandwidth, such as the next generation DOE UltraScience Network, and fast emergence of new attacks/virus/worms, existing network intrusion detection systems (IDS) are insufficient because they: • Are mostly host-based and not scalable to high-performance networks; • Are mostly signature-based and unable to adaptively recognize flow-level unknown attacks; • Cannot differentiate malicious events from the unintentional anomalies. To address these challenges, we proposed and developed a new paradigm called high-performance network anomaly/intrustion detection and mitigation (HPNAIDM) system. The new paradigm is significantly different from existing IDSes with the following features (research thrusts). • Online traffic recording and analysis on high-speed networks; • Online adaptive flow-level anomaly/intrusion detection and mitigation; • Integrated approach for false positive reduction. Our research prototype and evaluation demonstrate that the HPNAIDM system is highly effective and economically feasible. Beyond satisfying the pre-set goals, we even exceed that significantly (see more details in the next section). Overall, our project harvested 23 publications (2 book chapters, 6 journal papers and 15 peer-reviewed conference/workshop papers). Besides, we built a website for technique dissemination, which hosts two system prototype release to the research community. We also filed a patent application and developed strong international and domestic collaborations which span both academia and industry.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Network Anomaly Detection in Distributed Edge Computing Infrastructure

As networks continue to grow in complexity and scale, detecting anomalies has become increasingly challenging, particularly in diverse and geographically dispersed environments. Traditional approaches often struggle with managing the computational burden associated with analyzing large-scale network traffic to identify anomalies. This paper introduces a distributed edge computing framework that integrates federated learning with Apache Spark and Kubernetes to address these challenges. We hypothesize that our approach, which enables collaborative model training across distributed nodes, significantly enhances the detection accuracy of network anomalies across different network types. We show that by leveraging distributed computing and containerization technologies, our framework not only improves scalability and fault tolerance but also achieves superior detection performance compared to state-of-the-art methods. Extensive experiments on the UNSW-NB15 and ROAD datasets validate the effectiveness of our approach, demonstrating statistically significant improvements in detection accuracy and training efficiency over baseline models, as confirmed by MannWhitney U and Kolmogorov-Smirnov tests (p<0.05).

Marfo, William [University of Texas at El Paso,Dep↗
DOE OSTI2025

Reducing Communication Overhead in Federated Learning for Network Anomaly Detection with Adaptive Client Selection

Communication overhead in federated learning (FL) poses a significant challenge for network anomaly detection systems, where the myriad of client configurations and network conditions can severely impact system efficiency and detection accuracy. While existing approaches attempt to address this through individual optimization techniques, they often fail to maintain the delicate balance between reduced overhead and detection performance. This paper presents an adaptive FL framework that dynamically combines batch size optimization, client selection, and asynchronous updates to achieve efficient anomaly detection. Through extensive profiling and experimental analysis on two distinct datasets-UNSW-NBIS for general network traffic and ROAD for automotive networks-our framework reduces communication overhead by 97.6%; (from 700.0s to 16.8s) compared to synchronous baseline approaches while maintaining comparable detection accuracy (95.10%; vs. 95.12%;). Statistical validation using Mann-Whitney U test confirms significant improvements (p < 0.05) over existing FL approaches across both datasets, demonstrating the framework's adaptability to different network security contexts. Detailed profiling analysis reveals the efficiency gains through dramatic reductions in GPU operations and memory transfers while maintaining robust detection performance under varying client conditions.

Marfo, William [University of Texas at El Paso]↗
DOE OSTI2023

Systems and methods for detecting large network flows

In a system for efficiently detecting large/elephant flows in a network, the rate at which the received packets are sampled is adjusted according to the measured heavy tailedness of the arriving traffic, such that the measured heavy tailedness reaches a specified target level. The heavy tailedness is measured using the estimated sizes of different flows associated with the arriving packets. When the measured heavy tailedness reaches and remains at the specified target level, the flows having the largest estimated sizes are likely to be the largest/elephant flows in the network.

Ros-Giralt, Jordi↗
NASA NTRS1991

Electrification in winter storms and the analysis of thunderstorm overflight

The emergence of 24 hr operational lightning detection networks has led to the finding that positive lightning strokes, although still much fewer in number than the normal negative strokes, are present in summer and winter storms. Recent papers address the importance of understanding the meteorological conditions which lead to a dominance of one polarity of stroke over another; the appearance of positive strokes at the end of a storm appeared to presage the end-of-storm downdraft and subsidence leading to downburst activity. It is beginning to appear that positive strokes may be important meteorological indicators. Significant research accomplishments on the following topics are addressed: (1) a study to verify that the black boxes used in the lightning networks to detect both negative and positive strokes to ground were accurate; (2) the use of slow tails to determine the polarity of distant lightning; (3) lightning initiation in winter vs. summer storms; (4) the upgrade of sensors for the measurement of electric field signals associated with lightning; (5) the analysis of lightning flash records from storms between 40 and 125 km from the sensor; and (6) an interesting aspect of the initiation process which involves the physical processes driving the stepped leader. The focus of current research and future research plans are presented.

Brook, Marx↗
DOE OSTI2011

Localization-Based detection under network losses

We consider a networked system of sensors that measure the intensity of a source amidst background inside a two-dimensional monitoring area. The source intensity decays away from it, and the corresponding sensor measurements are random with a parameter determined by the intensity at sensor location. The detection problem is to infer the presence of a source based on measurements. Under a statistical independence condition, we show that a detection method based on maximum likelihood fuser performs below the individual sensors in presence of network losses. It has been previously shown that the localization of a source using measurements from multiple sensors leads to an improved detection, thereby establishing the effectiveness of a network over single or co-located sensors. We show that the communication losses degrade such network detection performance, particularly to levels below that of a single sensor under heavy losses. Under fairly general conditions on the source intensity decay functions and underlying measurement distributions, we quantify the loss of performance of the localization-based detection as a function of loss rate and packing number of state space. We present simulation and experimental results that illustrate the performance degradations due to network losses in detecting radiation sources.

cyber-physical networks↗
NASA NTRS2012

Developing an Open Source, Reusable Platform for Distributed Collaborative Information Management in the Early Detection Research Network

For the past decade, the NASA Jet Propulsion Laboratory, in collaboration with Dartmouth University has served as the center for informatics for the Early Detection Research Network (EDRN). The EDRN is a multi-institution research effort funded by the U.S. National Cancer Institute (NCI) and tasked with identifying and validating biomarkers for the early detection of cancer. As the distributed network has grown, increasingly formal processes have been developed for the acquisition, curation, storage, and dissemination of heterogeneous research information assets, and an informatics infrastructure has emerged. In this paper we discuss the evolution of EDRN informatics, its success as a mechanism for distributed information integration, and the potential sustainability and reuse benefits of emerging efforts to make the platform components themselves open source. We describe our experience transitioning a large closed-source software system to a community driven, open source project at the Apache Software Foundation, and point to lessons learned that will guide our present efforts to promote the reuse of the EDRN informatics infrastructure by a broader community.

Early Detection Research Network (EDRN)↗
DOE OSTI2015

Constructing rigorous and broad biosurveillance networks for detecting emerging zoonotic outbreaks

Determining optimal surveillance networks for an emerging pathogen is difficult since it is not known beforehand what the characteristics of a pathogen will be or where it will emerge. The resources for surveillance of infectious diseases in animals and wildlife are often limited and mathematical modeling can play a supporting role in examining a wide range of scenarios of pathogen spread. We demonstrate how a hierarchy of mathematical and statistical tools can be used in surveillance planning help guide successful surveillance and mitigation policies for a wide range of zoonotic pathogens. The model forecasts can help clarify the complexities of potential scenarios, and optimize biosurveillance programs for rapidly detecting infectious diseases. Using the highly pathogenic zoonotic H5N1 avian influenza 2006-2007 epidemic in Nigeria as an example, we determined the risk for infection for localized areas in an outbreak and designed biosurveillance stations that are effective for different pathogen strains and a range of possible outbreak locations. We created a general multi-scale, multi-host stochastic SEIR epidemiological network model, with both short and long-range movement, to simulate the spread of an infectious disease through Nigerian human, poultry, backyard duck, and wild bird populations. We chose parameter ranges specific to avian influenza (but not to a particular strain) and used a Latin hypercube sample experimental design to investigate epidemic predictions in a thousand simulations. We ranked the risk of local regions by the number of times they became infected in the ensemble of simulations. These spatial statistics were then complied into a potential risk map of infection. Finally, we validated the results with a known outbreak, using spatial analysis of all the simulation runs to show the progression matched closely with the observed location of the farms infected in the 2006-2007 epidemic.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2012

Network Intrusion Detection and Visualization using Aggregations in a Cyber Security Data Warehouse

The challenge of achieving situational understanding is a limiting factor in effective, timely, and adaptive cyber-security analysis. Anomaly detection fills a critical role in network assessment and trend analysis, both of which underlie the establishment of comprehensive situational understanding. To that end, we propose a cyber security data warehouse implemented as a hierarchical graph of aggregations that captures anomalies at multiple scales. Each node of our pro-posed graph is a summarization table of cyber event aggregations, and the edges are aggregation operators. The cyber security data warehouse enables domain experts to quickly traverse a multi-scale aggregation space systematically. We describe the architecture of a test bed system and a summary of results on the IEEE VAST 2012 Cyber Forensics data.

anomaly detection↗
NASA NTRS2010

Effects of Lightning and Other Meteorological Factors on Fire Activity in the North American Boreal Forest: Implications for Fire Weather Forecasting

The effects of lightning and other meteorological factors on wildfire activity in the North American boreal forest are statistically analyzed during the fire seasons of 2000-2006 through an integration of the following data sets: the MODerate Resolution Imaging Spectroradiometer (MODIS) level 2 fire products, the 3-hourly 32-kin gridded meteorological data from North American Regional Reanalysis (NARR), and the lightning data collected by the Canadian Lightning Detection Network (CLDN) and the Alaska Lightning Detection Network (ALDN). Positive anomalies of the 500 hPa geopotential height field, convective available potential energy (CAPE), number of cloud-to-ground lightning strikes, and the number of consecutive dry days are found to be statistically important to the seasonal variation of MODIS fire counts in a large portion of Canada and the entirety of Alaska. Analysis of fire occurrence patterns in the eastern and western boreal forest regions shows that dry (in the absence of precipitation) lightning strikes account for only 20% of the total lightning strikes, but are associated with (and likely cause) 40% of the MODIS observed fire counts in these regions. The chance for ignition increases when a threshold of at least 10 dry strikes per NARR grid box and at least 10 consecutive dry days is reached. Due to the orientation of the large-scale pattern, complex differences in fire and lightning occurrence and variability were also found between the eastern and western sub-regions. Locations with a high percentage of dry strikes commonly experience an increased number of fire counts, but the mean number of fire counts per dry strike is more than 50% higher in western boreal forest sub-region, suggesting a geographic and possible topographic influence. While wet lightning events are found to occur with a large range of CAPE values, a high probability for dry lightning occurs only when 500 hPa geopotential heights are above 5700m and CAPE values are near the maximum observed level, underscoring the importance of low-level instability to boreal fire weather forecasts-

Peterson, D.↗