Search NASA⌕ Search

Engineering topics

Ganter, Tyler

Publications and source records attributed to Ganter, Tyler.

16 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Quantifying Epistemic Uncertainty in Binary Classification via Accuracy Gain

ABSTRACT Recently, a surge of interest has been given to quantifying epistemic uncertainty (EU), the reducible portion of uncertainty due to lack of data. We propose a novel EU estimator in the binary classification setting, as the posterior expected value of the empirical gain in accuracy between the current prediction and the optimal prediction. In order to validate the performance of our EU estimator, we introduce an experimental procedure where we take an existing dataset, remove a set of points, and compare the estimated EU with the observed change in accuracy. Through real and simulated data experiments, we demonstrate the effectiveness of our proposed EU estimator.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Improving and Assessing the Quality of Uncertainty Quantification in Deep Learning

Deep learning (DL) models have enjoyed increased attention in recent years because of their powerful predictive capabilities. While many successes have been achieved, standard deep learning methods suffer from a lack of uncertainty quantification (UQ). While the development of methods for producing UQ from DL models is an active area of current research, little attention has been given to the quality of the UQ produced by such methods. In order to deploy DL models to high-consequence applications, high-quality UQ is necessary. This report details the research and development conducted as part of a Laboratory Directed Research and Development (LDRD) project at Sandia National Laboratories. The focus of this project is to develop a framework of methods and metrics for the principled assessment of UQ quality in DL models. This report presents an overview of UQ quality assessment in traditional statistical modeling and describes why this approach is difficult to apply in DL contexts. An assessment on relatively simple simulated data is presented to demonstrate that UQ quality can differ greatly between DL models trained on the same data. A method for simulating image data that can then be used for UQ quality assessment is described. A general method for simulating realistic data for the purpose of assessing a model’s UQ quality is also presented. A Bayesian uncertainty framework for understanding uncertainty and existing metrics is described. Research that came out of collaborations with two university partners are discussed along with a software toolkit that is currently being developed to implement the UQ quality assessment framework as well as serve as a general guide to incorporating UQ into DL applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Machine Learning for Correlated Intelligence. LDRD SAND Report

The Machine Learning for Correlated Intelligence Laboratory Directed Research & Development (LDRD) Project explored competing a variety of machine learning (ML) classification techniques against a known, open source dataset through the use of a rapid and automated algorithm research & development (RD) infrastructure. This approach relied heavily on creating an infrastructure in which to provide a pipeline for automatic target recognition (ATR) ML algorithm competition. Results are presented for nine ML classifiers against a primary dataset using the pipeline infrastructure developed for this project. New approaches to feature set extraction are presented and discussed as well.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Keras CT Segmentation v.1

SAND2020-14119 O A Tensorflow and Keras-backed framework for learned segmentation methods of 3D CT scan volumes. Supported functionality includes training models, running inference and quantifying uncertainty. The main underlying model architecture is V-Net. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Ganter, Tyler↗
DOE OSTI2018

Alternate Null Hypothesis Correlation: A New Approach to Automatic Seismic Event Detection

This paper describes a new method of seismic signal detection that improves upon the conventional waveform correlation method. Recent studies suggested that a significant limiting factor in the application of waveform correlation to regional and global scale monitoring is the false alarm rate. The false alarms do not originate from detections on noise but rather from seismic arrivals with unrelated source locations. This article presents results from an approach to waveform correlation that exploits techniques from signal processing and machine learning to improve the accuracy of detecting seismic arrivals. We modify the detection model for waveform correlation such that transient signals from noncollocated seismicity are considered when designing the detectors. The new approach uses waveform templates from known catalog events to train a supervised machine learning algorithm that derives a new set of detectors to represent the unique characteristics of the template waveforms; these new detectors maximize the likelihood of detecting only the desired events, thereby minimizing false alarms. We train a waveform correlation template library for a single three-component seismic monitoring station. We then review results from applying the new detectors, known as alternate null hypothesis correlation (ANCorr) templates, to a test set of seismic waveforms. Finally, we compare ANCorr results with those from application of the conventional waveform correlation matched filter technique.

58 GEOSCIENCES↗