Search NASA⌕ Search

SEARCH · Search NASA

Results for “False Positive Rate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Improved Distributed-memory Triangle Counting by Exploiting the Graph Structure

Graphs are ubiquitous in modeling complex systems and representing interactions between entities to uncover structural information of the domain. Traditionally, graph analytics workloads are challenging to efficiently scale (both strong and weak cases) on distributed memory due to the irregular memory-access driven nature (with little or no computations) of the methods. The structure of graphs and their relative distribution over the processing elements poses another level of complexity, making it difficult to attain sustainable scalability across platforms. In this paper, we discuss enhancements to TriC, a distributed-memory implementation of graph triangle counting using Message Passing Interface (MPI), which was featured in the 2020 Graph Challenge competition. We have made some incremental enhancements to TriC, primarily adopting a user-defined buffering strategy to overcome the startup problem for large graphs (by fixing the memory for intermediate data), and experimenting with probabilistic data structures such as bloom filter to improve the query response time for assessing edge existence, at the expense of increasing the overall false positive rate. These adjustments have led to a modest improvements in most cases, as compared to the previous version.

Graph Analytics, HPC↗

Correlations Between Panoramic Imagery and Gamma-Ray Background in an Urban Area

When searching for radiological sources in an urban area, a vehicle-borne detector system will often measure complex, varying backgrounds primarily from natural gamma-ray sources. Much work has been focused on developing spectral algorithms that retain sensitivity and minimize the false-positive rate even in the presence of such spectral and temporal variability. However, information about the environment surrounding the detector system might also provide useful clues about the expected background, which if incorporated into an algorithm, could improve performance. Recent work has focused on extensive measuring and modeling of urban areas with the goal of understanding how these complex backgrounds arise. This work presents an analysis of panoramic video images and gamma-ray background data collected in Oakland, California, by the radiological multisensor analysis platform (RadMAP) vehicle. Features were extracted from the panoramic images by semantically labeling the images and then convolving the labeled regions with the detector response. A linear model was used to relate the image-derived features to gamma-ray spectral features obtained using nonnegative matrix factorization (NMF) under different regularizations. Here we find some gamma-ray background features correlate strongly with image-derived features that measure the response-adjusted solid angle subtended by sky and buildings, and we discuss the implications for the development of future, contextually aware detection algorithms.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Cy-Phy ADS: Cyber Physical Anomaly Detection Framework for EV Charging Systems

Today’s large-scale Electric Vehicle (EV) infrastructures are heavily dependent on information communication technologies to maintain their operation and to support communication within sub-system components as well as the outside world. These technologies are vulnerable to various cyber and physical threats. Timely identification and mitigation of these threats are critical for improving human safety, avoiding economic losses, and preventing catastrophic system failures. By addressing this, our work presents a ResNet Autoencoder (AE) based Cyber-Physical Anomaly Detection System (Cy-Phy ADS) for detecting anomalies in EV Controller Area Network (CAN) protocol communication. It consists of four main components: Cyber-Physical Feature Extractor, ResNet AE-based Anomaly Detection Framework, Cyber-Physical Health Metric (CPHM), and Visualization Dashboard. The presented framework was trained and tested using CAN data collected from the EV charging system testbed at the Idaho National Laboratory. The presented Cy-Phy ADS compared against six widely used unsupervised anomaly detection algorithms: One Class Support Vector Machine (OCSVM), Variational Autoencoder (VAE), LSTM Autoencoder (LSTM AE), Isolation Forest (IForest), Principle Component Analysis (PCA) and Local Outlier Factor (LOF). Here the presented approach showed the highest accuracy among the compared methods. Further, the proposed approach showed comparable performance in terms of precision, F1, and False positive rate. It also showed the lowest training and inference time compared to the neural network-based baseline algorithms compared against with. Additionally, the Cy-Phy ADS has advantages such as unsupervised training, the ability to provide a holistic metric for system health characterization, and non-linear feature extraction.

99 GENERAL AND MISCELLANEOUS↗

The interplay of machine learning-based resonant anomaly detection methods

Abstract Machine learning-based anomaly detection (AD) methods are promising tools for extending the coverage of searches for physics beyond the Standard Model (BSM). One class of AD methods that has received significant attention is resonant anomaly detection, where the BSM physics is assumed to be localized in at least one known variable. While there have been many methods proposed to identify such a BSM signal that make use of simulated or detected data in different ways, there has not yet been a study of the methods’ complementarity. To this end, we address two questions. First, in the absence of any signal, do different methods pick the same events as signal-like? If not, then we can significantly reduce the false-positive rate by comparing different methods on the same dataset. Second, if there is a signal, are different methods fully correlated? Even if their maximum performance is the same, since we do not know how much signal is present, it may be beneficial to combine approaches. Using the Large Hadron Collider (LHC) Olympics dataset, we provide quantitative answers to these questions. We find that there are significant gains possible by combining multiple methods, which will strengthen the search program at the LHC and beyond.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Pushing the limits of pulse shape discrimination in a large liquid xenon detector

Abstract The LUX-ZEPLIN (LZ) experiment is a direct-detection dark matter experiment, optimized to search for weakly interacting massive particles (WIMPs) through WIMP-nucleon interactions. The main challenge in dark matter detection is differentiating between WIMP signals and background events. In LZ, the ratio of ionization to scintillation signals (charge-to-light) is the primary method for rejecting electronic recoil (ER) background. Pulse shape discrimination (PSD) offers a method for additional ER backgrounds rejection in liquid xenon detectors. In this paper, the discrimination power of PSD with the LZ experiment is discussed. To precisely characterize the scintillation pulse shape, an analysis framework is developed to reconstruct the detection time of individual photons. Using LZ calibration data, the photon-timing tail fraction discriminator is optimized and achieves ER leakage as low as $$15\%$$ 15 % . For specific background processes such as $$^{124}$$ 124 Xe double electron capture, the leakage is reduced further to about $$5\%$$ 5 % . PSD is combined with charge-to-light to form two-factor discrimination (TFD). The optimized TFD performance is compared with the performance of the charge-to-light method, with the corresponding false positive rate reduced by up to a factor of two for large scintillation pulses. Finally, PSD and TFD are applied to data from LZ’s WS2024 run and their performance is summarized.

Akerib, D. S. [SLAC National Accelerator Laborator↗

Searching for a Pulse: Evaluating the Use of Rapid DC Pulses for Diagnosing Battery Health, State-of-Charge, and Safety

Rapid electrochemical diagnostics, like DC pulse sequences or electrochemical impedance spectroscopy, are known to be useful for capacity prediction. However, it is unclear how previous results will map to different cell types and use cases and whether rapid diagnostics are useful for remaining useful life prediction or for detecting potential safety issues. To that end, we have collected a data set with ∼50,000 DC pulse measurements from four types of commercial lithium-ion batteries to enable training of state-of-charge, health, and safety diagnostic models via machine-learning. We demonstrate that 120-second DC pulse sequences can be used to predict capacity with 2%–9% average error, which can separate high- from low-capacity cells with only a 0.3% false positive rate but is not accurate enough to estimate remaining useful life. We also find that no safety related targets can be accurately predicted, highlighting the critical need for other non-invasive methods to diagnose battery safety.

25 ENERGY STORAGE↗

Hyper Spectral Anomaly Detection

The HSA is a statistics based anomaly detection model. The model performs unsupervised anomaly detection, based on a datapoint's density and similarity within a dataset. Density and similarity data are encoded into an affinity matrix. The affinity matrix is evolved to summarize the data's structure on greater topographical scales within the data's function space. The set of evolved affinity matrices and an anomaly score vector are passed to a user defined penalized objective function. The penalized objective function of anomaly scores is then minimized. Data points where the absolute value of the z-scores of anomaly scores greater than a specified threshold are predicted as anomalies. A novel multi-filter feature has also been implemented. To reduce false positive rates, the multi-filter records the indexes of the HSA predictions. A new dataset and data loader are instantiated consisting of all the initial HSA predictions and non-anomalous data points in a 10% and 90% split respectively. The HSA is then run through this data set and a count of number of times a data point is predicted is kept. In this way the initial predictions may be compared with data spanning the entire dataset. After the multi-filter is complete, all datapoints will have an associated anomaly score, as well as a multi-filter prediction count to further filter the anomalous predictions.

Rogers, DempseyD [Idaho National Laboratory (INL),↗

Bloom filters for molecules

Abstract Ultra-large chemical libraries are reaching 10s to 100s of billions of molecules. A challenge for these libraries is to efficiently check if a proposed molecule is present. Here we propose and study Bloom filters for testing if a molecule is present in a set using either string or fingerprint representations. Bloom filters are small enough to hold billions of molecules in just a few GB of memory and check membership in sub milliseconds. We found string representations can have a false positive rate below 1% and require significantly less storage than using fingerprints. Canonical SMILES with Bloom filters with the simple FNV (Fowler-Noll-Voll) hashing function provide fast and accurate membership tests with small memory requirements. We provide a general implementation and specific filters for detecting if a molecule is purchasable, patented, or a natural product according to existing databases at https://github.com/whitead/molbloom .

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Stacked filters: learning to filter by structure

We present Stacked Filters, a new probabilistic filter which is fast and robust similar to query-agnostic filters (such as Bloom and Cuckoo filters), and at the same time brings low false positive rates and sizes similar to classifier-based filters (such as Learned Filters). The core idea is that Stacked Filters incorporate workload knowledge about frequently queried non-existing values. Instead of learning, they structurally incorporate that knowledge using hashing and several sequenced filter layers, indexing both data and frequent negatives. Stacked Filters can also gather workload knowledge on-the-fly and adaptively build the filter. We show experimentally that for a given memory budget, Stacked Filters achieve end-to-end query throughput up to 130x better than the best alternative for a workload, either query-agnostic or classifier-based filters, and depending on where data is (SSD or HDD).

Computer Science↗

Prototype for Unclassified Algorithm

Continuing previous efforts to investigate and develop the Unclassified Radioisotope Algorithm, the goal of the FY19-FY20 effort was to develop a prototype detector system which uses the algorithm to confirm warhead attributes related to the presence of either weapons grade plutonium (WGPu) or highly enriched uranium (HEU). The final deliverable is a prototype attribute measurement system built with common, commercially available gamma radiation detector components, capable of confirming the presence of specific, complex radioactive sources of interest, without the collection and storage of gamma energy spectra. This is accomplished by processing each gamma pulse as it is received, applying weight values based on the energy and incrementing or decrementing scalar counters which can be compared with expected values to determine if the measured source is consistent with WGPu or HEU. This report documents the design of the prototype system as well as the development of the algorithm and performance testing results. While the previously conceptualized, simple algorithm resulted in a prohibitive amount of false positives, the goal for a simple attribute measurement system capable of verifying Ba-133 and Ra-226 (weapons grade plutonium and highly enriched uranium surrogate testing sources) at over 95% accuracy with sub 5% false positive rate was demonstrated.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Breast Cancer Imaging and Characterization

Researchers at Los Alamos National Laboratory are developing a suite of ultrasound-waveform tomography and super-resolution ultrasound imaging technology that, while investigational at this time, may be able to provide significantly improved image resolution and characterization of early-stage cancers in a non-invasive and cost-effective manner. Breast cancer is the most frequently diagnosed cancer in U.S. women. Although current medical imaging technologies have improved, false positive rates for mammograms still range 50-60% after 10 yearly mammograms. This innovative ultrasound-imaging technology is patented and copyright protected. It has great potential to differentiate between cancerous and healthy tissue without the negative side effects of mammography. We are seeking a commercialization partner to license these technologies or participate in a Cooperative Research and Development Agreement (CRADA) to further develop a practical suite of products for commercial purposes, and where applicable seek any necessary FDA authorization of the technology.

59 BASIC BIOLOGICAL SCIENCES↗

Cyber-Attack Detection and Accommodation for the Energy Delivery System

The goals of this project were to create a software system with a suite of key algorithms for cyber-attack detection and accommodation providing domain layer protection for critical power generation assets. Example assets included gas and steam turbines, heat recovery steam generators, and electrical generators. The aggressive algorithm goals were aimed at reducing the false positive rates in threat detection to <1% using learnings from many evolving disciplines (power turbine and generator physics, power system modeling, modern control theory, system identification, machine learning, deep learning, mathematics and data science). Additional goals for the algorithms involved localizing threats on-the-fly to know in which monitoring node the effects of attacks are present, and then providing accommodation to keep the system running uninterrupted much of the time in the presence of the attack. Accommodation had a performance goal of providing resiliency when up to 50% of monitoring nodes are in an attack state.

cybersecurity, cyber-physical↗

OR22-Neuromorphic Rad Detector-PD3Ra (Final Report)

In unattended monitoring scenarios, automated radiation detection algorithms must be able to detect low signal-to-noise ratio (SNR) anomalies in a potentially dynamic and noisy background and report these anomalies in a timely fashion. Dynamic and noisy backgrounds complicate the use of simple gross-counting algorithms because they can lead to either high false positive rates or low sensitivity. Algorithms that use the entire spectrum have been the most successful in this area; notable examples are the NSCRAD algorithm developed at Pacific Northwest National Laboratory and recently the nonnegative matrix factorization approach developed at Lawrence Berkeley National Laboratory (LBNL). These approaches use either spectral regions of interest or spectral decomposition to detect threat isotopes in the background.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Explosion Detection using Transfer Learning via YAMNet [Poster]

The acoustic data of explosions and noise collected on smartphones were fed to the YAMNet model to obtain the scores of each class. The data was split 60/20/20 for training, validation, and testing. A single-layered neural network was trained using the computed scores to predict if the data was an explosion or noise. The trained model was then combined with the YAMNet model. The combined model was tested with the ESC-50 dataset to investigate overall accuracy and class based false positive rates.

98 - NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL↗

Machine Learning for Anomaly Detection in Neural Network Security and SRF Cavities

This dissertation explores the development and deployment of machine learning approaches to address critical challenges in anomaly detection across two distinct domains: neural network security in federated learning settings and cavity behavior analysis in particle accelerator operations at Jefferson Lab in Newport News, Virginia. Anomaly detection identifies deviations from expected patterns, safeguarding systems in cybersecurity, industry, and research against malicious activities and failures. This dissertation demonstrates how our machine learning approaches enhance detection accuracy and efficiency in both neural network security and industrial applications. First, we investigate vulnerabilities in deep neural networks deployed in federated learning. Although federated learning preserves user privacy by training models locally, it remains vulnerable to backdoor attacks, in which malicious participants embed hidden triggers that induce targeted misbehavior. We propose a self-supervised contrastive learning framework to detect and mitigate such backdoor attacks. In our experiments, this method achieves higher detection accuracy and lower false positive rates than existing defenses, while operating without access to local model updates or original training data and thus preserving the privacy guarantees of the federated setting. Second, we address the operational reliability of superconducting radio-frequency (SRF) cavities at the Continuous Electron Beam Accelerator Facility (CEBAF). Our research leverages an unsupervised learning approach, combined with Principal Component Analysis (PCA) and k-means clustering, to identify anomalous behaviors in SRF cavities. Our method detects subtle anomalous behavior by analyzing SRF signal data. This knowledge allows for the early detection and resolution of potential faults, significantly improving the efficiency and reliability of operations. Third, we extend these insights to time-series anomaly detection more broadly. We design a contrastive-learning based model tailored to increasingly dynamic environments and academic research. This model improves detection accuracy in settings that require real-time monitoring and predictive maintenance. Our research underscores the broader applicability and impact of advanced machine learning techniques in anomaly detection. By extracting meaningful patterns from complex data, machine learning can significantly enhance security in distributed neural networks and improve the efficiency of particle accelerator operations. This dissertation serves as a stepping stone for future investigations into the vast possibilities of anomaly detection, inspiring further exploration and development of machine learning techniques in this field.

Ferguson, Hal [Old Dominion University]↗

Predictions for the Detectability of Milky Way Satellite Galaxies and Outer-Halo Star Clusters with the Vera C. Rubin Observatory

We predict the sensitivity of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) to faint, resolved Milky Way satellite galaxies and outer-halo star clusters. We characterize the expected sensitivity using simulated LSST data from the LSST Dark Energy Science Collaboration (DESC) Data Challenge 2 (DC2) accessed and analyzed with the Rubin Science Platform as part of the Rubin Early Science Program. We simulate resolved stellar populations of Milky Way satellite galaxies and outer-halo star clusters over a wide range of sizes, luminosities, and heliocentric distances, which are broadly consistent with expectations for the Milky Way satellite system. We inject simulated stars into the DC2 catalog with realistic photometric uncertainties and star/galaxy separation derived from the DC2 data itself. We assess the probability that each simulated system would be detected by LSST using a conventional isochrone matched-filter technique. We find that assuming perfect star/galaxy separation enables the detection of resolved stellar systems with $M_V$ = 0 mag and $r_{1/2}$ = 10 pc with >50% efficiency out to a heliocentric distance of ~250 kpc. Similar detection efficiency is possible with a simple star/galaxy separation criterion based on measured quantities, although the false positive rate is higher due to leakage of background galaxies into the stellar sample. When assuming perfect star/galaxy classification and a model for the galaxy-halo connection fit to current data, we predict that 89 +/- 20 Milky Way satellite galaxies will be detectable with a simple matched-filter algorithm applied to the LSST wide-fast-deep data set. Different assumptions about the performance of star/galaxy classification efficiency can decrease this estimate by ~7%-25%, which emphasizes the importance of high-quality star/galaxy separation for studies of the Milky Way satellite population with LSST.

79 ASTRONOMY AND ASTROPHYSICS↗

Statistical Significance Testing for Mixed Priors: A Combined Bayesian and Frequentist Analysis

In many hypothesis testing applications, we have mixed priors, with well-motivated informative priors for some parameters but not for others. The Bayesian methodology uses the Bayes factor and is helpful for the informative priors, as it incorporates Occam’s razor via the multiplicity or trials factor in the look-elsewhere effect. However, if the prior is not known completely, the frequentist hypothesis test via the false-positive rate is a better approach, as it is less sensitive to the prior choice. We argue that when only partial prior information is available, it is best to combine the two methodologies by using the Bayes factor as a test statistic in the frequentist analysis. We show that the standard frequentist maximum likelihood-ratio test statistic corresponds to the Bayes factor with a non-informative Jeffrey’s prior. We also show that mixed priors increase the statistical power in frequentist analyses over the maximum likelihood test statistic. We develop an analytic formalism that does not require expensive simulations and generalize Wilks’ theorem beyond its usual regime of validity. In specific limits, the formalism reproduces existing expressions, such as the p-value of linear models and periodograms. We apply the formalism to an example of exoplanet transits, where multiplicity can be more than 10 7 . We show that our analytic expressions reproduce the $p$-values derived from numerical simulations. We offer an interpretation of our formalism based on the statistical mechanics. We introduce the counting of states in a continuous parameter space using the uncertainty volume as the quantum of the state. We show that both the $p$-value and Bayes factor can be expressed as an energy versus entropy competition.

97 MATHEMATICS AND COMPUTING↗

Rocket Launch Detection with Smartphone Audio and Transfer Learning

Rocket launches generate infrasound signatures that have been detected at great distances. Due to the sparsity of the networks that have made these detections, however, most signals are detected tens of minutes to hours after the rocket launch. In this work, a method of near-real-time detection of rocket launches using data from a network of smartphones located 10–70 km from launch sites is presented. A machine learning model is trained and tested on the open-access Aggregated Smartphone Timeseries of Rocket-generated Acoustics (ASTRA), Smartphone High-explosive Audio Recordings Dataset (SHAReD), and ESC-50 datasets, resulting in a final accuracy of 97% and a false positive rate of <1%. The performance and behavior of the model are summarized, and its suitability for persistent monitoring applications is discussed.

acoustics↗