Search NASA⌕ Search

SEARCH · Search NASA

Results for “false data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

Signal Detection Theory Applied to Helicopter Transmission Diagnostic Thresholds

Helicopter Health Usage Monitoring Systems (HUMS) have potential for providing data to support increasing the service life of a dynamic mechanical component in the transmission of a helicopter. Data collected can demonstrate the HUMS condition indicator responds to a specific component fault with appropriate alert limits and minimal false alarms. Defining thresholds for specific faults requires a tradeoff between the sensitivity of the condition indicator (CI) limit to indicate damage and the number of false alarms. A method using Receiver Operating Characteristic (ROC) curves to assess CI performance was demonstrated using CI data collected from accelerometers installed on several UH60 Black Hawk and AH64 Apache helicopters and an AH64 helicopter component test stand. Results of the analysis indicate ROC curves can be used to reliably assess the performance of commercial HUMS condition indicators to detect damaged gears and bearings in a helicopter transmission.

Dempsey, Paula J.↗

Performance analysis of a fault inferring nonlinear detection system algorithm with integrated avionics flight data

This paper presents the performance analysis results of a fault inferring nonlinear detection system (FINDS) using integrated avionics sensor flight data for the NASA ATOPS B-737 aircraft in a Microwave Landing System (MLS) environment. First, an overview of the FINDS algorithm structure is given. Then, aircraft state estimate time histories and statistics for the flight data sensors are discussed. This is followed by an explanation of modifications made to the detection and decision functions in FINDS to improve false alarm and failure detection performance. Next, the failure detection and false alarm performance of the FINDS algorithm are analyzed by injecting bias failures into fourteen sensor outputs over six repetitive runs of the five minutes of flight data. Results indicate that the detection speed, failure level estimation, and false alarm performance show a marked improvement over the previously reported simulation runs. In agreement with earlier results, detection speed is faster for filter measurement sensors such as MLS than for filter input sensors such as flight control accelerometers. Finally, the progress in modifications of the FINDS algorithm design to accommodate flight computer constraints is discussed.

Caglayan, A. K.↗

Impacts of benchmarking choices on inferred model skill of the Arctic–Boreal terrestrial carbon cycle

Abstract Land surface models require continuous validation against observations to improve and reduce simulation uncertainty. However, inferred model performance can be heavily influenced by subjective choices made in the selection and application of observational data products. A key area often misrepresented by models is the Arctic–Boreal region, which is a potential tipping point region in Earth’s climate system due to large permafrost carbon stocks that are vulnerable to release with climate warming. We use the International Land Model Benchmarking (ILAMB) framework to evaluate how the model skill of TRENDY-v9 models varies based on the choice of observational-based benchmark and how benchmarks are applied in model evaluation. This analysis uses global datasets integrated into ILAMB and new, regionally-specific observational products from the Arctic–Boreal Vulnerability Experiment. Our results cover the overall time period of 1979–2019 and show that model scores can vary substantially depending on the data product applied, with higher model scores indicating better model performance against observations. The lowest model scores occur when benchmarked against regional, compared to global, datasets. We also evaluate observed and modeled functional relationships between ecosystem respiration and air temperature and between gross primary production and precipitation. Here, we find that the magnitude and shape of the responses are strongly impacted by the choice of observational dataset and the approach used to construct the functional relationship benchmark. These results suggest that model evaluation studies could conclude a false sense of model skill if only using a single benchmark data product or if not applying regional data products when performing a regional model analysis. Collectively, our findings highlight the influence of benchmarking choices on model evaluation and point to the need for benchmarking guidelines when assessing model skill.

Poe, Jeralyn (ORCID:0000000318495278)↗

Review of Grey Box/Black Box Data Contamination Metrics on Open and Commercial Models

Dataset contamination is a problem where benchmarks and tasks used to evaluate the capabilities of Large Language Models (LLMs) have been incorporated into the training dataset of the models. This gives a false sense of performance that can overestimate how these models will function on truly unseen data. This problem becomes worse with commercial LLMs with larger and non-accessible training data, so techniques have been developed to try to measure the degree to which a model is contaminated with a benchmark’s data. To understand the effectiveness of these techniques, particularly when evaluating contamination on coding tasks, we review trends and categorize techniques by the degree of access to the model that is required. The research literature on this topic has reported mixed effectiveness of these techniques, so we select a set of black box (text access only) and grey box (access to model loss/probabilities required) techniques and apply them to both commercial and non-commercial models. We implement these metrics as part of a framework to test the contamination of Python code in LLMs to see to what extent we can replicate the effectiveness (or ineffectiveness) of these contamination detection techniques. Though we find mixed results in the capabilities of these metrics to identify contamination, we do observe evidence that they can identify contamination (broadly) in fine-tuned models when both a baseline and fine-tuned model is present. Additionally, similarity metrics were able to identify between contaminated and uncontaminated data even in situations where the data is distributionally similar (e.g., drawn from the same set of code projects).

97 MATHEMATICS AND COMPUTING↗

Estimating Alarm Thresholds for Process Monitoring Data under Different Assumptions about the Data Generating Mechanism

Process monitoring (PM) for nuclear safeguards sometimes requires estimation of thresholds corresponding to small false alarm rates. Threshold estimation dates to the 1920s with the Shewhart control chart; however, because possible new roles for PM are being evaluated in nuclear safeguards, it is timely to consider modern model selection options in the context of threshold estimation. One of the possible new PM roles involves PM residuals, where a residual is defined as residual = data − prediction. This paper reviews alarm threshold estimation, introduces model selection options, and considers a range of assumptions regarding the data-generating mechanism for PM residuals. Two PM examples from nuclear safeguards are included to motivate the need for alarm threshold estimation. The first example involves mixtures of probability distributions that arise in solution monitoring, which is a common type of PM. The second example involves periodic partial cleanout of in-process inventory, leading to challenging structure in the time series of PM residuals.

Burr, Tom↗

Multi phenomena melt pool sensor data fusion for enhanced process monitoring of laser powder bed fusion additive manufacturing

Finding actionable trends in laser-based metal additive manufacturing process monitoring data is challenging owing to the diversity and complexity of the underlying physical interactions. A single monitoring solution that captures a particular process phenomenon, such as a photodiode that tracks melt pool intensity, is not alone capable of evaluating process stability or detecting flaw formation with sufficient precision for routine application in industry. In this work, to improve flaw detection performance, we adopted a data fusion approach that captures multiple process phenomena. To demonstrate this, we acquired data from laser powder bed fusion (LPBF) builds of cylindrical specimens produced with different laser spot sizes, emulating defocusing due to process faults such as thermal lensing. The resulting specimens had porosity of varying types and severity, quantified by post-build non-destructive X-ray computed tomography, Archimedes density measurements, and destructive metallographic characterization. During the build, the melt pool state was monitored with two coaxial high-speed video cameras and a temperature field imaging system. Physically intuitive low-level melt pool signatures, such as melt pool temperature, shape and size, and spatter intensity were extracted from this high-dimensional, image-based sensor data. These process signatures were subsequently used as input features in relatively simple machine learning models, such as a support vector machine, which were trained to detect laser defocusing, and in addition, predict porosity type and severity. The results show that the data fusion approach significantly enhanced system performance by reducing the overall false positive rate from ~ 0.1 to ~ 0.001 without sacrificing the true positive rate (~0.90). These results were at par with a black-box, deep machine learning approach (convolutional neural network).

36 MATERIALS SCIENCE↗

Flight experience with windshear detection

Windshear alerts resulting from the Honeywell Windshear Detection and Guidance System are presented based on data from approximately 248,000 revenue flights at Piedmont Airlines. The data indicate that the detection system provides a significant benefit to the flight crew of the aircraft. In addition, nuisance and false alerts were found to occur at an acceptably low rate to maintain flight crew confidence in the system. Data from a digital flight recorder is also presented which shows the maximum and minimum windshear magnitudes recorded for a representative number of flights in February, 1987. The effect of the boundary layer of a steady state wind is also discussed.

Zweifel, Terry↗

Mapping systematic errors in helium abundance determinations using Markov Chain Monte Carlo

Monte Carlo techniques have been used to evaluate the statistical and systematic uncertainties in the helium abundances derived from extragalactic H II regions. The helium abundance is sensitive to several physical parameters associated with the H II region. In this work, we introduce Markov Chain Monte Carlo (MCMC) methods to efficiently explore the parameter space and determine the helium abundance, the physical parameters, and the uncertainties derived from observations of metal poor nebulae. Experiments with synthetic data show that the MCMC method is superior to previous implementations (based on flux perturbation) in that it is not affected by biases due to non-physical parameter space. The MCMC analysis allows a detailed exploration of degeneracies, and, in particular, a false minimum that occurs at large values of optical depth in the He I emission lines. We demonstrate that introducing the electron temperature derived from the [O III] emission lines as a prior, in a very conservative manner, produces negligible bias and effectively eliminates the false minima occurring at large optical depth. We perform a frequentist analysis on data from several ''high quality'' systems. Likelihood plots illustrate degeneracies, asymmetries, and limits of the determination. In agreement with previous work, we find relatively large systematic errors, limiting the precision of the primordial helium abundance for currently available spectra.

79 ASTRONOMY AND ASTROPHYSICS↗

A protection system for the JET ITER-like wall based on imaging diagnostics

The new JET ITER-like wall (made of beryllium and tungsten) is more fragile than the former carbon fiber composite wall and requires active protection to prevent excessive heat loads on the plasma facing components (PFC). Analog CCD cameras operating in the near infrared wavelength are used to measure surface temperature of the PFCs. Region of interest (ROI) analysis is performed in real time and the maximum temperature measured in each ROI is sent to the vessel thermal map. The protection of the ITER-like wall system started in October 2011 and has already successfully led to a safe landing of the plasma when hot spots were observed on the Be main chamber PFCs. Divertor protection is more of a challenge due to dust deposits that often generate false hot spots. In this contribution we describe the camera, data capture and real time processing systems. We discuss the calibration strategy for the temperature measurements with cross validation with thermal IR cameras and bi-color pyrometers. Most importantly, we demonstrate that a protection system based on CCD cameras can work and show examples of hot spot detections that stop the plasma pulse. The limits of such a design and the associated constraints on the operations are also presented.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

{chi}{sup 2} DISCRIMINATORS FOR TRANSITING PLANET DETECTION IN KEPLER DATA

The Kepler spacecraft observes a host of target stars to detect transiting planets. Requiring a 7.1{sigma} detection in three years of data yields over 100,000 detections, many of which are false alarms. After a second cut is made on a robust detection statistic, some 50,000 or more targets still remain. These false alarms waste resources as they propagate through the remainder of the software pipeline and so a method to discriminate against them is crucial in maintaining the desired sensitivity to true events. This paper describes a {chi}{sup 2} test which represents a novel application of an existing formalism developed for false alarm mitigation in searches for gravitational waves. Using this technique, the false alarm rate can be lowered to {approx}5%.

79 ASTRONOMY AND ASTROPHYSICS↗

Nature and Consequences of Biological Reductionism for the Immunological Study of Infectious Diseases

Evolution has conserved “economic” systems that perform many functions, faster or better, with less. For example, three to five leukocyte types protect from thousands of pathogens. In order to achieve so much with so little, biological systems combine their limited elements, creating complex structures. Yet, the prevalent research paradigm is reductionist. Focusing on infectious diseases, reductionist and non-reductionist views are here described. Furthermore, the literature indicates that reductionism is associated with information loss and errors, while non-reductionist operations can extract more information from the same data. When designed to capture one-to-many/many-to-one interactions—including the use of arrows that connect pairs of consecutive observations—non-reductionist (spatial–temporal) constructs eliminate data variability from all dimensions, except along one line, while arrows describe the directionality of temporal changes that occur along the line. To validate the patterns detected by non-reductionist operations, reductionist procedures are needed. Integrated (non-reductionist and reductionist) methods can (i) distinguish data subsets that differ immunologically and statistically; (ii) differentiate false-negative from -positive errors; (iii) discriminate disease stages; (iv) capture in vivo, multilevel interactions that consider the patient, the microbe, and antibiotic-mediated responses; and (v) assess dynamics. Integrated methods provide repeatable and biologically interpretable information.

59 BASIC BIOLOGICAL SCIENCES↗

Conditional Generative Adversarial Networks (cGANs) for Near Real-Time Precipitation Estimation from Multispectral GOES-16 Satellite Imageries—PERSIANN-cGAN

In this paper, we present a state-of-the-art precipitation estimation framework which leverages advances in satellite remote sensing as well as Deep Learning (DL). The framework takes advantage of the improvements in spatial, spectral and temporal resolutions of the Advanced Baseline Imager (ABI) onboard the GOES-16 platform along with elevation information to improve the precipitation estimates. The procedure begins by first deriving a Rain/No Rain (R/NR) binary mask through classification of the pixels and then applying regression to estimate the amount of rainfall for rainy pixels. A Fully Convolutional Network is used as a regressor to predict precipitation estimates. The network is trained using the non-saturating conditional Generative Adversarial Network (cGAN) and Mean Squared Error (MSE) loss terms to generate results that better learn the complex distribution of precipitation in the observed data. Common verification metrics such as Probability Of Detection (POD), False Alarm Ratio (FAR), Critical Success Index (CSI), Bias, Correlation and MSE are used to evaluate the accuracy of both R/NR classification and real-valued precipitation estimates. Statistics and visualizations of the evaluation measures show improvements in the precipitation retrieval accuracy in the proposed framework compared to the baseline models trained using conventional MSE loss terms. This framework is proposed as an augmentation for PERSIANN-CCS (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network- Cloud Classification System) algorithm for estimating global precipitation.

42 ENGINEERING↗

Expediting DECam Multimessenger Counterpart Searches with Convolutional Neural Networks

Searches for counterparts to multimessenger events with optical imagers use difference imaging to detect new transient sources. However, even with existing artifact-detection algorithms, this process simultaneously returns several classes of false positives: false detections from poor-quality image subtractions, false detections from low signal-to-noise images, and detections of preexisting variable sources. Currently, human visual inspection to remove the false positives is a central part of multimessenger follow-up observations, but when next generation gravitational wave and neutrino detectors come online and increase the rate of multimessenger events, the visual inspection process will be prohibitively expensive. We approach this problem with two convolutional neural networks operating on the difference imaging outputs. The first network focuses on removing false detections and demonstrates an accuracy of 92% on our data set. The second network focuses on sorting all real detections by the probability of being a transient source within a host galaxy and distinguishes between various classes of images that previously required additional human inspection. We find the number of images requiring human inspection will decrease by a factor of 1.5 using our approach alone and a factor of 3.6 using our approach in combination with existing algorithms, facilitating rapid multimessenger counterpart identification by the astronomical community.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

EVALUATING THE USE OF ULTRASONIC TESTING IN LIEU OF RADIOGRAPHY FOR INSPECTING WELDS IN NUCLEAR POWER PLANTS

The nuclear power industry has recently proposed using ultrasonic testing (UT) in lieu of radiographic testing (RT) on new construction welds and on the repairs of operating reactor component welds. Advantages of UT include a reduction in inspection time, costs, and plant personnel exposure to radiation fields. The replacement of one nondestructive testing method with another, however, requires a detailed analysis of the capabilities of the replacement method as compared to the existing method. Capabilities of interest in this context include detection reliability, false call rates, flaw characterization and sizing accuracy, human factors, data recording capabilities, and cost. The interchangeability of UT and RT has been studied by several institutions, but the evidence found to date in a literature search is not conclusive and therefore requires further investigation to assess the ability of UT to meet nuclear power industry fabrication and pre-service inspection standards. This paper reviews relevant literature on the interchangeability of UT and RT, and identifies potential gaps that may need to be addressed in this area.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Error minimizing algorithms for nearest eighbor classifiers

Stack Filters define a large class of discrete nonlinear filter first introd uced in image and signal processing for noise removal. In recent years we have suggested their application to classification problems, and investigated their relationship to other types of discrete classifiers such as Decision Trees. In this paper we focus on a continuous domain version of Stack Filter Classifiers which we call Ordered Hypothesis Machines (OHM), and investigate their relationship to Nearest Neighbor classifiers. We show that OHM classifiers provide a novel framework in which to train Nearest Neighbor type classifiers by minimizing empirical error based loss functions. We use the framework to investigate a new cost sensitive loss function that allows us to train a Nearest Neighbor type classifier for low false alarm rate applications. We report results on both synthetic data and real-world image data.

97 MATHEMATICS AND COMPUTING↗

Lightning Jump Algorithm Development for the GOES·R Geostationary Lightning Mapper

Current work on the lightning jump algorithm to be used in GOES‐R Geostationary Lightning Mapper (GLM)'s data stream is multifaceted due to the intricate interplay between the storm tracking, GLM proxy data, and the performance of the lightning jump itself. This work outlines the progress of the last year, where analysis and performance of the lightning jump algorithm with automated storm tracking and GLM proxy data were assessed using over 700 storms from North Alabama. The cases analyzed coincide with previous semi‐objective work performed using total lightning mapping array (LMA) measurements in Schultz et al. (2011). Analysis shows that key components of the algorithm (flash rate and sigma thresholds) have the greatest influence on the performance of the algorithm when validating using severe storm reports. Automated objective analysis using the GLM proxy data has shown probability of detection (POD) values around 60% with false alarm rates (FAR) around 73% using similar methodology to Schultz et al. (2011). However, when applying verification methods similar to those employed by the National Weather Service, POD values increase slightly (69%) and FAR values decrease (63%). The relationship between storm tracking and lightning jump has also been tested in a real‐time framework at NSSL. This system includes fully automated tracking by radar alone, real‐time LMA and radar observations and the lightning jump. Results indicate that the POD is strong at 65%. However, the FAR is significantly higher than in Schultz et al. (2011) (50‐80% depending on various tracking/lightning jump parameters) when using storm reports for verification. Given known issues with Storm Data, the performance of the real‐time jump algorithm is also being tested with high density radar and surface observations from the NSSL Severe Hazards Analysis & Verification Experiment (SHAVE).

Schultz. E.↗

Corrections for the polarization-dependent efficiency and new neutron-proton analyzing power data at 7.6 MeV

We present new corrections for the polarization-dependent efficiency (PDE), which introduces a false asymmetry into measurements of n-p analyzing power A{sub y}({theta}) caused by double scattering in the neutron side detectors. To accomplish this, we created a new database of {sup 12}C(n-vector,n) A{sub y}({theta}) by using a combination of fits to data, phase-shift analysis, and R-matrix analysis. Our recorrection for PDE of previously reported n-p A{sub y}({theta}) data at 7.6 and 12.0 MeV and new data at 7.6 MeV indicate that we have achieved a superior representation of {sup 12}C(n-vector,n). Our results continue to suggest a possible charge dependence of the pion-nucleon coupling constant.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗