Search NASA⌕ Search

SEARCH · Search NASA

Results for “outlier detection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Stragglers of the thick disc

Young alpha-rich (YAR) stars have been detected in the past as outliers to the local age - [α/Fe] relation. These objects are enhanced in α-elements, but they are apparently younger than typical thick disc stars. Here, we study the global kinematics and chemical properties of YAR giant stars in the APOGEE DR17 survey and show that they have properties similar to those of the standard thick disc stellar population. This leads us to conclude that YAR are rejuvenated thick disc objects, and the most likely explanation is that they are evolved blue stragglers. This is confirmed by their position in the Hertzsprung–Russel diagram (HRD). Extending our selection to dwarfs allowed us to obtain the first general straggler distribution in an HRD of field stars. We also compared the elemental abundances of our sample with those of standard thick disc stars and found that our YAR stars are shifted in oxygen, magnesium, sodium, and the slow neutron-capture element cerium. Although we detected no sign of binarity for most objects, the enhancement in cerium may be a signature of a mass transfer from an asymptotic giant branch companion. The most massive YAR stars suggest that mass transfer from an evolved star may not be the only plausible formation pathway and that other scenarios, such as collision or coalescence, should be considered.

79 ASTRONOMY AND ASTROPHYSICS↗

Challenging problems of quality assurance and quality control (QA/QC) of meteorological time series data

Abstract Representativeness and quality of collected meteorological data impact accuracy and precision of climate, hydrological, and biogeochemical analyses and predictions. We developed a comprehensive Quality Assurance (QA) and Quality Control (QC) statistical framework, consisting of three major phases: Phase I—Preliminary data exploration, i.e., processing of raw datasets, with the challenging problems of time formatting and combining datasets of different lengths and different time intervals; Phase II—QA of the datasets, including detecting and flagging of duplicates, outliers, and extreme data; and Phase III—the development of time series of a desired frequency, imputation of missing values, visualization and a final statistical summary. The paper includes two use cases based on the time series data collected at the Billy Barr meteorological station (East River Watershed, Colorado), and the Barro Colorado Island (BCI, Panama) meteorological station. The developed statistical framework is suitable for both real-time and post-data-collection QA/QC analysis of meteorological datasets.

54 ENVIRONMENTAL SCIENCES↗

Quality Assurance and Quality Control (QA/QC) of Meteorological Time Series Data for Billy Barr, East River, Colorado USA

A comprehensive Quality Assurance (QA) and Quality Control (QC) statistical framework consists of three major phases: Phase 1—Preliminary raw data sets exploration, including time formatting and combining datasets of different lengths and different time intervals; Phase 2—QA of the datasets, including detecting and flagging of duplicates, outliers, and extreme values; and Phase 3—the development of time series of a desired frequency, imputation of missing values, visualization and a final statistical summary. The time series data collected at the Billy Barr meteorological station (East River Watershed, Colorado) were analyzed. The developed statistical framework is suitable for both real-time and post-data-collection QA/QC analysis of meteorological datasets.The files that are in this data package include one excel file, converted to CSV format (Billy_Barr_raw_qaqc.csv) that contains the raw meteorological data, i.e., input data used for the QA/QC analysis. The second CSV file (Billy_Barr_1hr.csv) is the QA/QC and flagged meteorological data, i.e., output data from the QA/QC analysis. The last file (QAQC_Billy_Barr_2021-03-22.R) is a script written in R that implements the QA/QC and flagging process. The purpose of the CSV data files included in this package is to provide input and output files implemented in the R script.

54 ENVIRONMENTAL SCIENCES↗

QA/QC-ed BCI meteorological drivers

Time series raw datasets of Temperature, Solar Radiation, Relative Humidity, Rainfall, Barometric Pressure, Wind Speed, and Wind Direction for the Barro Colorado (PA-BCI) site for the period from 2003-01-01 to 2016-12-31 were downloaded from the STRI website. The QA/QC protocol of raw datasets included the detection and removal of NAs, outliers and bad data points, imputation of missing data, and creation of the time series with equidistant time stamps. The time series for different meteorological parameters were aligned into a single database to be used for modeling. Statistical QA/QC analysis was performed using a series of R libraries in Rstudio. The changes in the dataset were flagged. The file BCI_met_drivers_2003-2016_QAQC_summary_report.docx provides the details of the QA/QC data analysis.

54 ENVIRONMENTAL SCIENCES↗

Spatial Distribution of Accuracy of Aerosol Retrievals from Multiple Satellite Sensors

Remote sensing of aerosols from space has been a subject of extensive research, with multiple sensors retrieving aerosol properties globally on a daily or weekly basis. The diverse algorithms used for these retrievals operate on different types of reflected signals based on different assumptions about the underlying physical phenomena. Depending on the actual retrieval conditions and especially on the geographical location of the sensed aerosol parcels, the combination of these factors might be advantageous for one or more of the sensors and unfavorable for others, resulting in disagreements between similar aerosol parameters retrieved from different sensors. In this presentation, we will demonstrate the use of the Multi-sensor Aerosol Products Sampling System (MAPSS) to analyze and intercompare aerosol retrievals from multiple spaceborne sensors, including MODIS (on Terra and Aqua), MISR, OMI, POLDER, CALIOP, and SeaWiFS. Based on this intercomparison, we are determining geographical locations where these products provide the greatest accuracy of the retrievals and identifying the products that are the most suitable for retrieval at these locations. The analyses are performed by comparing quality-screened satellite aerosol products to available collocated ground-based aerosol observations from the Aerosol Robotic Network (AERONET) stations, during the period of 2006-2010 when all the satellite sensors were operating concurrently. Furthermore, we will discuss results of a statistical approach that is applied to the collocated data to detect and remove potential data outliers that can bias the results of the analysis.

Petrenko, Maksym↗

PHOTOMETRY AND PHOTOMETRIC REDSHIFT CATALOGS FOR THE LOCKMAN HOLE DEEP FIELD

We present broadband photometry and photometric redshifts for 187,611 sources located in {approx}0.5 deg{sup 2} in the Lockman Hole area. The catalog includes 388 X-ray-detected sources identified with the very deep XMM-Newton observations available for an area of 0.2 deg{sup 2}. The source detection was performed on the R{sub c}-, z'-, and B-band images and the available photometry is spanning from the far-ultraviolet to the mid-infrared, reaching in the best-case scenario 21 bands. Astrometry corrections and photometric cross-calibrations over the entire data set allowed the computation of accurate photometric redshifts. Special treatment is undertaken for the X-ray sources, the majority of which are active galactic nuclei (AGNs). For normal galaxies, comparing the photometric redshifts to the 253 available spectroscopic redshifts, we achieve an accuracy of {sigma}{sub {Delta}z/(1+z)} = 0.036, with 12.6% outliers. For the X-ray-detected sources, compared to 115 spectroscopic redshifts, the accuracy is {sigma}{sub {Delta}z/(1+z)} = 0.069, with 18.3% outliers, where the outliers are defined as sources with |z{sub phot} - z{sub spec}| > 0.15 Multiplication-Sign (1 + z{sub spec}). These results are a significant improvement over the previously available photometric redshifts for normal galaxies in the Lockman Hole, while it is the first time that photometric redshifts are computed and made public for AGNs for this field.

79 ASTRONOMY AND ASTROPHYSICS↗

Cognitive IoT and Edge Computing for Intrusion Detection with Federated TinyML

Internet of Things (IoT) and Edge Computing (EC) are rapidly becoming an integral part of the modern society. By 2030, there is estimated to be over 40 billion active and connected IoT devices [1]. This rapid progress also comes with a significant implication on cybersecurity. Back-end infrastructure and systems have a much broader attack than they did previously due to vulnerable IoT/EC devices being connected to wireless networks. This expanding attack surface is a growing concern because IoT/EC are increasingly being used in critical systems such as power grids, health care, and smart homes. To effectively address a problem of this scale, cognitive cyber methods—which can autonomously detect and react to cyber attacks as they develop—are needed. To address this, we bring Artificial Intelligence (AI) and Machine Learning (ML) to IoT/EC devices, using tinyML to monitor voluminous IoT data against cyber threats, and using Federated Learning (FL) to share local detection knowledge across the system while preserving privacy. We propose a novel three-layer architecture: (1) an IoT layer for tinyML-based inference, (2) an edge layer for ML model training, and (3) a cloud layer for FL operations. Using the publicly available 11-class N-BaIoT dataset [2], we demonstrate that this architecture mitigates resource constraints at the IoT layer while improving detection accuracy over standard two-layer designs. An outlier-resistant scaler, feature reduction, and quantization enable the tinyML model to maintain detection accuracy with a reduced model size. Additionally, federated learning that only utilizes the intersection (across heterogenous devices) of the reduced feature set achieves superior detection accuracy compared to locally trained models.

Li, Mingyan [ORNL] (ORCID:0009000569532640)↗

All-Sky Search for Periodic Gravitational Waves in the Full S5 LIGO Data

We report on an all-sky search for periodic gravitational waves in the frequency band 50-800 Hz and with the frequency time derivative in the range of 0 through -6 x 10(exp -9) Hz/s. Such a signal could be produced by a nearby spinning and slightly non-axisymmetric isolated neutron star in our galaxy. After recent improvements in the search program that yielded a 10x increase in computational efficiency, we have searched in two years of data. collected during LIGO's fifth science run and have obtained the most sensitive all-sky upper limits on gravitational wave strain to date. Near 150 Hz our upper limit on worst-case linearly polarized strain amplitude h(sub 0) is 1 x 10(exp -24), while at the high end of our frequency ra.nge we achieve a worst-case upper limit of 3.8 x 10(exp -24) for all polarizations and sky locations. These results constitute a factor of two improvement upop. previously published data. A new detection pipeline utilizing a Loosely Coherent algorithm was able to follow up weaker outliers, increasing the volume of space where signals can be detected by a factor of 10, but has not revealed any gravitational wave signals. The pipeline has been tested for robustness with respect to deviations from the model of an isolated neutron star, such as caused by a low-mass or long.period binary companion.

Abadie, J.↗

Out of Distribution Detection with Neural Network Anchoring

This is code to reproduce and build on OOD detection from the paper "Out of Distribution Detection with Neural Network Anchoring". Our goal here is to exploit heteroscedastic temperature scaling as a calibration strategy for out of distribution (OOD) detection. Heteroscedasticity here refers to the fact that the optimal temperature parameter for each sample can be different, as opposed to conventional approaches that use the same value for the entire distribution. To enable this, we propose a new training strategy called anchoring that can estimate appropriate temperature values for each sample, leading to state-of-the-art OOD detection performance across several benchmarks. Using NTK theory, we show that this temperature function estimate is closely linked to the epistemic uncertainty of the classifier, which explains its behavior. In contrast to some of the best-performing OOD detection approaches, our method does not require exposure to additional outlier datasets, custom calibration objectives, or model ensembling. Through empirical studies with different OOD detection settings - far OOD, near OOD, and semantically coherent OOD - we establish a highly effective OOD detection approach.

Thiagarajan, Jayaraman↗

Quality Control Analysis in Real-time (QC-ART): A Tool for Real-time Quality Control Assessment of Mass Spectrometry-based Proteomics Data

Liquid chromatography-mass spectrometry (LC-MS)-based proteomics studies of large sample cohorts can easily require from months to years to complete. Acquiring consistent, high-quality data in such large-scale studies is challenging because of normal variations in instrumentation performance over time, as well as artifacts introduced by the samples themselves, such as those due to collection, storage and processing. Existing quality control methods for proteomics data primarily focus on post-hoc analysis to remove low-quality data that would degrade downstream statistics; they are not designed to evaluate the data in near real-time, which would allow for interventions as soon as deviations in data quality are detected. In addition to flagging analyses that demonstrate outlier behavior, evaluating how the data structure changes over time can aide in understanding typical instrument performance or identify issues such as a degradation in data quality due to the need for instrument cleaning and/or re-calibration. To address this gap for proteomics, we developed Quality Control Analysis in Real-Time (QC-ART), a tool for evaluating data as they are acquired in order to dynamically flag potential issues with instrument performance or sample quality. QC-ART has similar accuracy as standard post-hoc analysis methods with the additional benefit of real-time analysis. We demonstrate the utility and performance of QC-ART in identifying deviations in data quality due to both instrument and sample issues in near real-time for LC-MS-based plasma proteomics analyses of a sample subset of The Environmental Determinants of Diabetes in the Young cohort. We also present a case where QC-ART facilitated the identification of oxidative modifications, which are often underappreciated in proteomic experiments.

Stanfill, Bryan A.↗

Time series anomaly detection in power electronics signals with recurrent and ConvLSTM autoencoders

The anomalies in the high voltage converter modulator (HVCM) remain a major down time for the spallation neutron source facility, that delivers the most intense neutron beam in the world for scientific materials research. In this work, we propose neural network architectures based on Recurrent AutoEncoders (RAE) to detect anomalies ahead of time in the power signals coming from the HVCM. Bi-directional gated recurrent unit, bi-directional long-short term memory (LSTM), and convolutional LSTM (ConvLSTM) are developed, trained, and tested using real experimental signals from the HVCM module. The results show a good performance of the proposed RAE models, achieving precision up to 91%, recall up to 88%, false omission rate as low as 20% (i.e. 80% of the anomalies were detected), and area under the ROC curve up to 0.9. The three RAE models provide very comparable performance, with LSTM showing slightly better performance than GRU and ConvLSTM. The RAE models are benchmarked against other anomaly detection methods, including isolation forest, support vector machine, local outlier factor, feedforward and convolutional autoencoders, and others; showing a better performance. Here, the results of this study demonstrate the promising potential of RAE in anomaly detection for real-world power systems, and for increasing the reliability of the HVCM modules in the spallation neutron source.

42 ENGINEERING↗

A study of the central stellar populations of galaxies in SDSS-IV MaNGA: identification of a subsample with unusually young and massive stars

ABSTRACT This paper describes a search for galaxy centres with clear indications of unusual stellar populations with an initial mass function flatter than Salpeter at high stellar masses. Out of a sample of 668 face-on galaxies with stellar masses in the range $10^{10}\!-\!10^{11} \, \mathrm{M}_{\odot }$, I identify 15 galaxies with young- to intermediate-age central stellar populations with unusual stellar population gradients in the inner regions of the galaxy. In these galaxies, the 4000-Å break is either flat or rising towards the centre of the galaxy, indicating that the central regions host evolved stars, but the H α equivalent width also rises steeply in the central regions. The ionization parameter [O iii]/[O ii] is typically low in these Galactic Centres, indicating that ionizing sources are stellar rather than AGN. Wolf–Rayet features characteristic of hot young stars are often found in the spectra and these also get progressively stronger at smaller galactocentric radii. These outliers are compared to a control sample of galaxies of similar mass with young inner stellar populations, but where the gradients in H α equivalent width and 4000-Å break follow each other more closely. The outliers exhibit central Wolf–Rayet red bump excesses much more frequently, they have higher central stellar and ionized gas metallicities, and they are also more frequently detected at 20-cm radio wavelengths. I highlight one outlier where the ionized gas is clearly being strongly perturbed and blown out either by massive stars after they explode as supernovae, or by energy injection from matter falling on to a black hole.

Kauffmann, Guinevere↗

The good, the bad, and the ugly: Data-driven load profile discord identification in a large building portfolio

Reducing the overall energy consumption and associated greenhouse gas emissions in the building sector is essential for meeting our future sustainability goals. Recently, smart energy metering facilities have been deployed to enable monitoring of energy consumption data with hourly or subhourly temporal resolution. This unprecedented data collection has created various opportunities for advanced data analytics involving load profiles (e.g., building energy benchmarking programs, building-to-grid integration, and calibration of urban-scale energy models). These applications often need preprocessing steps to detect daily load profile discords, such as: 1) outliers due to system malfunctions (the bad) and 2) irregular energy consumption patterns, such as those resulting from holidays (the ugly) compared to normal consumption patterns (the good). However, current preprocessing methods predominantly focus on filtering using statistical threshold values, which fail to capture the contextual discords of daily profiles. In addition, discord detection algorithms in building research are often aimed at finding individual building-level discords, which are not suitable at a large scale. Thus, here, we develop a method for automated load profile discord identification (ALDI) in a large portfolio of buildings (more than 100 buildings). Specifically, ALDI 1) uses the matrix profile (MP) method to quantify the similarities of daily subsequences in time series meter data, 2) compares daily MP values with typical-day MP distributions using the Kolmogorov-Smirnov test, and 3) identifies daily load profile discords in a large building portfolio. We evaluate ALDI using the metering data of both an academic campus and a residential neighborhood. Our results demonstrate that ALDI efficiently discovers measurement errors by system malfunctions and low energy consumption days in the academic campus portfolio, and it detects unique load shape patterns likely driven by occupant behavior and extreme weather conditions in the residential neighborhood.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Quantification of Trace Chemicals Using Vehicle Cabin Atmosphere Monitor

A system to monitor the concentrations of trace chemicals in cabin atmosphere is one of the most critical components in long-duration human flight missions. The Vehicle Cabin Atmosphere Monitor (VCAM) is a miniature gas chromatograph mass spectrometer system to be used to detect and quantify trace chemicals in the International Space Station. We developed an autonomous computational process to quantify trace chemicals for use in VCAM. The process involves the design of a measured signal quantification scheme, the construction of concentration curves (i.e. the relationship between concentration and ion count measured by VCAM), the decision rule of applying high- or low-gain concentration curves, and the detection of saturation, low-signals, and outliers. When the developed quantification process is applied, the average errors of concentration for most of trace chemicals are found to be between 14% and 66%.

Lee, Seungwon↗

A Universal Threshold for the Assessment of Load and Output Residuals of Strain-Gage Balance Data

A new universal residual threshold for the detection of load and gage output residual outliers of wind tunnel strain{gage balance data was developed. The threshold works with both the Iterative and Non{Iterative Methods that are used in the aerospace testing community to analyze and process balance data. It also supports all known load and gage output formats that are traditionally used to describe balance data. The threshold's definition is based on an empirical electrical constant. First, the constant is used to construct a threshold for the assessment of gage output residuals. Then, the related threshold for the assessment of load residuals is obtained by multiplying the empirical electrical constant with the sum of the absolute values of all first partial derivatives of a given load component. The empirical constant equals 2.5 microV/V for the assessment of balance calibration or check load data residuals. A value of 0.5 microV/V is recommended for the evaluation of repeat point residuals because, by design, the calculation of these residuals removes errors that are associated with the regression analysis of the data itself. Data from a calibration of a six-component force balance is used to illustrate the application of the new threshold definitions to real{world balance calibration data.

wind tunnel balance↗

Simulated performance of an order statistic threshold strategy for detection of narrowband signals

The application of order statistics to signal detection is becoming an increasingly active area of research. This is due to the inherent robustness of rank estimators in the presence of large outliers that would significantly degrade more conventional mean-level-based detection systems. A detection strategy is presented in which the threshold estimate is obtained using order statistics. The performance of this algorithm in the presence of simulated interference and broadband noise is evaluated. In this way, the robustness of the proposed strategy in the presence of the interference can be fully assessed as a function of the interference, noise, and detector parameters.

Satorius, E.↗

THE FMOS-COSMOS SURVEY OF STAR-FORMING GALAXIES AT z ∼ 1.6. III. SURVEY DESIGN, PERFORMANCE, AND SAMPLE CHARACTERISTICS

We present a spectroscopic survey of galaxies in the COSMOS field using the Fiber Multi-object Spectrograph (FMOS), a near-infrared instrument on the Subaru Telescope. Our survey is specifically designed to detect the Hα emission line that falls within the H-band (1.6–1.8 μm) spectroscopic window from star-forming galaxies with 1.4 < z < 1.7 and M{sub stellar} ≳ 10{sup 10} M{sub ⊙}. With the high multiplex capability of FMOS, it is now feasible to construct samples of over 1000 galaxies having spectroscopic redshifts at epochs that were previously challenging. The high-resolution mode (R ∼ 2600) effectively separates Hα and [N ii]λ6585, thus enabling studies of the gas-phase metallicity and photoionization state of the interstellar medium. The primary aim of our program is to establish how star formation depends on stellar mass and environment, both recognized as drivers of galaxy evolution at lower redshifts. In addition to the main galaxy sample, our target selection places priority on those detected in the far-infrared by Herschel/PACS to assess the level of obscured star formation and investigate, in detail, outliers from the star formation rate (SFR)—stellar mass relation. Galaxies with Hα detections are followed up with FMOS observations at shorter wavelengths using the J-long (1.11–1.35 μm) grating to detect Hβ and [O iii]λ5008 which provides an assessment of the extinction required to measure SFRs not hampered by dust, and an indication of embedded active galactic nuclei. With 460 redshifts measured from 1153 spectra, we assess the performance of the instrument with respect to achieving our goals, discuss inherent biases in the sample, and detail the emission-line properties. Our higher-level data products, including catalogs and spectra, are available to the community.

79 ASTRONOMY AND ASTROPHYSICS↗

Online anomaly detection for multi‐source VMware using a distributed streaming framework

Summary Anomaly detection refers to the identification of patterns in a dataset that do not conform to expected patterns. Such non‐conformant patterns typically correspond to samples of interest and are assigned to different labels in different domains, such as outliers, anomalies, exceptions, and malware. A daunting challenge is to detect anomalies in rapid voluminous streams of data. This paper presents a novel, generic real‐time distributed anomaly detection framework for multi‐source stream data. As a case study, we investigate anomaly detection for a multi‐source VMware‐based cloud data center, which maintains a large number of virtual machines (VMs). This framework continuously monitors VMware performance stream data related to CPU statistics (e.g., load and usage). It collects data simultaneously from all of the VMs connected to the network and notifies the resource manager to reschedule its CPU resources dynamically when it identifies any abnormal behavior from its collected data. A semi‐supervised clustering technique is used to build a model from benign training data only. During testing, if a data instance deviates significantly from the model, then it is flagged as an anomaly. Effective anomaly detection in this case demands a distributed framework with high throughput and low latency. Distributed streaming frameworks like Apache Storm, Apache Spark, S4, and others are designed for a lower data processing time and a higher throughput than standard centralized frameworks. We have experimentally compared the average processing latency of a tuple during clustering and prediction in both Spark and Storm and demonstrated that Spark processes a tuple much quicker than storm on average. Copyright © 2016 John Wiley & Sons, Ltd.

Solaimani, Mohiuddin↗