Search NASA⌕ Search

SEARCH · Search NASA

Results for “outlier detection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Robust Spectral Unmixing of Sparse Multispectral Lidar Waveforms using Gamma Markov Random Fields

Here, this paper presents a new Bayesian spectral un-mixing algorithm to analyse remote scenes sensed via sparse multispectral Lidar measurements. To a first approximation, in the presence of a target, each Lidar waveform consists of a main peak, whose position depends on the target distance and whose amplitude depends on the wavelength of the laser source considered (i.e, on the target reflectivity). Besides, these temporal responses are usually assumed to be corrupted by Poisson noise in the low photon count regime. When considering multiple wavelengths, it becomes possible to use spectral information in order to identify and quantify the main materials in the scene, in addition to estimation of the Lidar-based range profiles. Due to its anomaly detection capability, the proposed hierarchical Bayesian model, coupled with an efficient Markov chain Monte Carlo algorithm, allows robust estimation of depth images together with abundance and outlier maps associated with the observed 3D scene. The proposed methodology is illustrated via experiments conducted with real multispectral Lidar data acquired in a controlled environment. The results demonstrate the possibility to unmix spectral responses constructed from extremely sparse photon counts (less than 10 photons per pixel and band).

97 MATHEMATICS AND COMPUTING↗

Cy-Phy ADS: Cyber Physical Anomaly Detection Framework for EV Charging Systems

Today’s large-scale Electric Vehicle (EV) infrastructures are heavily dependent on information communication technologies to maintain their operation and to support communication within sub-system components as well as the outside world. These technologies are vulnerable to various cyber and physical threats. Timely identification and mitigation of these threats are critical for improving human safety, avoiding economic losses, and preventing catastrophic system failures. By addressing this, our work presents a ResNet Autoencoder (AE) based Cyber-Physical Anomaly Detection System (Cy-Phy ADS) for detecting anomalies in EV Controller Area Network (CAN) protocol communication. It consists of four main components: Cyber-Physical Feature Extractor, ResNet AE-based Anomaly Detection Framework, Cyber-Physical Health Metric (CPHM), and Visualization Dashboard. The presented framework was trained and tested using CAN data collected from the EV charging system testbed at the Idaho National Laboratory. The presented Cy-Phy ADS compared against six widely used unsupervised anomaly detection algorithms: One Class Support Vector Machine (OCSVM), Variational Autoencoder (VAE), LSTM Autoencoder (LSTM AE), Isolation Forest (IForest), Principle Component Analysis (PCA) and Local Outlier Factor (LOF). Here the presented approach showed the highest accuracy among the compared methods. Further, the proposed approach showed comparable performance in terms of precision, F1, and False positive rate. It also showed the lowest training and inference time compared to the neural network-based baseline algorithms compared against with. Additionally, the Cy-Phy ADS has advantages such as unsupervised training, the ability to provide a holistic metric for system health characterization, and non-linear feature extraction.

99 GENERAL AND MISCELLANEOUS↗

Atmospheric condition identification in multivariate data through a metric for total variation

Identification of atmospheric conditions within a multivariable atmospheric data set is a necessary step in the validation of emerging and existing high-fidelity models used to simulate wind plant flows and operation.Atmospheric conditions relevant for wind energy research include stationary conditions, given the need for well-converged statistics for model validation, as well as conditions observed less frequently, such as extreme atmospheric events, which are used in wind turbine and wind plant design.Aggregation of observations without regard to covariance between time series discounts the dynamical nature of the atmosphere and is not sufficiently representative of atmospheric conditions.Identification and characterization of continuous time periods with atmospheric conditions that have a high value for analysis or simulation set the stage for more advanced model validation and the development of real-time control and operational strategies.The current work explores a single metric for variation in a multivariate data sample that quantifies variability within each channel as well as covariance between channels.The total variation is used to identify conditions of interest that conform to desired objective functions, such as stationary conditions, ramps or waves of wind speed, and changes in wind direction.Total variation is somewhat sensitive to the presence of outliers in the input data, and the method is best complemented by quality-control procedures to ensure reliable results.The direct detection and classification of events or conditions of interest within atmospheric data sets is vital to developing our understanding of wind plant response and to the formulation of forecasting and control models.

17 WIND ENERGY↗

Anomaly detection in scientific data using joint statistical moments

We propose an anomaly detection method for multi-variate scientific data based on analysis of high-order joint moments. Using kurtosis as a reliable measure of outliers, we suggest that principal kurtosis vectors, by analogy to principal component analysis (PCA) vectors, signify the principal directions along which outliers appear. The inception of an anomaly, then, manifests as a change in the principal values and vectors of kurtosis. Obtaining the principal kurtosis vectors requires decomposing a fourth order joint cumulant tensor for which we use a simple, computationally less expensive approach that involves performing a singular value decomposition (SVD) over the matricized tensor. We demonstrate the efficacy of this approach on synthetic data, and develop an algorithm to identify the occurrence of a spatial and/or temporal anomalous event in scientific phenomena. The algorithm decomposes the data into several spatial sub-domains and time steps to identify regions with such events. Feature moment metrics, based on the alignments of the principal kurtosis vectors, are computed at each sub-domain and time step for all features to quantify their relative importance towards the overall kurtosis in the data. Accordingly, spatial and temporal anomaly metrics for each sub-domain are proposed using the Hellinger distance of the feature moment metric distribution from a suitable nominal distribution. Finally, we apply the algorithm to two turbulent auto-ignition combustion cases and demonstrate that the anomaly metrics reliably capture the occurrence of auto-ignition in relevant spatial sub-domains at the right time steps.

97 MATHEMATICS AND COMPUTING↗

Identification of characteristic molecular signature for volatile organic compounds in peripheral blood of rat

In a previous report we demonstrated that the transcriptomic response of liver tissue was specific to toxicants, and a characteristic molecular signature could be used as an early prognostic biomarker in rats. It is necessary to determine the transcriptomic response to toxicants in peripheral blood for application to the human system. Volatile organic compounds (VOCs) comprise a major group of pollutants which significantly affect the chemistry of the atmosphere and human health. In this study we identified and validated the specific molecular signatures of toxicants in rat whole blood as early predictors of environmental toxicants. VOCs (dichloromethane, ethylbenzene, and trichloroethylene) were administered to 11-week-old SD male rats after 48 h of exposure, peripheral whole blood was subjected to expression profiling analysis. Unsupervised gene expression analysis resulted in a characteristic molecular signature for each toxicant, and supervised analysis identified 1,217 outlier genes as distinct molecular signatures discerning VOC exposure from healthy controls. Further analysis of multi-classification suggested 337 genes as early detective molecular markers for three VOCs with 100% accuracy. A large-scale gene expression analysis of a different VOC exposure animal model suggested that characteristic expression profiles exist in blood cells and multi-classification of this VOC-specific molecular signature can discriminate each toxicant at an early exposure time. This blood expression signature can thus be used as discernable surrogate marker for detection of biological responses to VOC exposure in an environment.

60 APPLIED LIFE SCIENCES↗

Limits on Stellar-Mass Compact Objects as Dark Matter from Gravitational Lensing of Type Ia Supernovae

We present that the nature of dark matter (DM) remains unknown despite very precise knowledge of its abundance in the Universe. An alternative to new elementary particles postulates DM as made of macroscopic compact halo objects (MACHO) such as black holes formed in the very early Universe. Stellar-mass primordial black holes (PBHs) are subject to less robust constraints than other mass ranges and might be connected to gravitational-wave signals detected by the Laser Interferometer Gravitational-Wave Observatory (LIGO). New methods are therefore necessary to constrain the viability of compact objects as a DM candidate. Here we report bounds on the abundance of compact objects from gravitational lensing of type Ia supernovae (SNe). Current SNe data sets constrain compact objects to represent less than 35.2% (Joint Lightcurve Analysis) and 37.2% (Union 2.1) of the total matter content in the Universe, at 95% confidence level. The results are valid for masses larger than ~ 0.01 M ⊙ (solar masses), limited by the size SNe relative to the lens Einstein radius. We demonstrate the mass range of the constraints by computing magnification probabilities for realistic SNe sizes and different values of the PBH mass. Our bounds are sensitive to the total abundance of compact objects with M ≳ 0.01 M ⊙ and complementary to other observational tests. These results are robust against cosmological parameters, outlier rejection, correlated noise, and selection bias. PBHs and other MACHOs are therefore ruled out as the dominant form of DM for objects associated to LIGO gravitational wave detections. Lastly, these bounds constrain early-Universe models that predict stellar-mass PBH production and strengthen the case for lighter forms of DM, including new elementary particles.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

THE ULTRA-LONG GAMMA-RAY BURST 111209A: THE COLLAPSE OF A BLUE SUPERGIANT?

We present optical, X-ray and gamma-ray observations of GRB 111209A, observed at a redshift of z = 0.677. We show that this event was active in its prompt phase for about 25000 s, making it the longest burst ever observed. This rare event could have been detected up to z {approx} 1.4 in gamma-rays. Compared to other long gamma-ray bursts (GRBs), GRB 111209A is a clear outlier in the energy-fluence and duration plane. The high-energy prompt emission shows no sign of a strong blackbody component, the signature of a tidal disruption event, or a supernova shock breakout. Given the extreme longevity of this event, and lack of any significant observed supernova signature, we propose that GRB 111209A resulted from the core-collapse of a low-metallicity blue supergiant star. This scenario is favored because of the necessity to supply enough mass to the central engine over a duration of thousands of seconds. Hence, we suggest that GRB 111209A could have more in common with population III stellar explosions, rather than those associated with normal long GRBs.

79 ASTRONOMY AND ASTROPHYSICS↗

Fast and efficient identification of anomalous galaxy spectra with neural density estimation

ABSTRACT Current large-scale astrophysical experiments produce unprecedented amounts of rich and diverse data. This creates a growing need for fast and flexible automated data inspection methods. Deep learning algorithms can capture and pick up subtle variations in rich data sets and are fast to apply once trained. Here, we study the applicability of an unsupervised and probabilistic deep learning framework, the probabilistic auto-encoder, to the detection of peculiar objects in galaxy spectra from the SDSS survey. Different to supervised algorithms, this algorithm is not trained to detect a specific feature or type of anomaly, instead it learns the complex and diverse distribution of galaxy spectra from training data and identifies outliers with respect to the learned distribution. We find that the algorithm assigns consistently lower probabilities (higher anomaly score) to spectra that exhibit unusual features. For example, the majority of outliers among quiescent galaxies are E+A galaxies, whose spectra combine features from old and young stellar population. Other identified outliers include LINERs, supernovae, and overlapping objects. Conditional modelling further allows us to incorporate additional information. Namely, we evaluate the probability of an object being anomalous given a certain spectral class, but other information such as metrics of data quality or estimated redshift could be incorporated as well. We make our code publicly available.

Böhm, Vanessa↗

Scheduled Sequential Compressed Spectrum Sensing for Wideband Cognitive Radios

The support for high data rate applications with the cognitive radio technology necessitates wideband spectrum sensing. However, it is costly to apply long-term wideband sensing and is especially difficult in the presence of uncertainty, such as high noise, interference, outliers, and channel fading. In this paper, we propose scheduling of sequential compressed spectrum sensing which jointly exploits compressed sensing (CS) and sequential periodic detection techniques to achieve more accurate and timely wideband sensing. Instead of invoking CS to reconstruct the signal in each period, our proposed scheme performs backward grouped-compressed-data sequential probability ratio test (backward GCD-SPRT) using compressed data samples in sequential detection, while CS recovery is only pursued when needed. This method on one hand significantly reduces the CS recovery overhead, and on the other takes advantage of sequential detection to improve the sensing quality. Furthermore, we propose (a) an in-depth sensing scheme to accelerate sensing decision-making when a change in channel status is suspected, (b) a block-sparse CS reconstruction algorithm to exploit the block sparsity properties of wide spectrum, and (c) a set of schemes to fuse results from the recovered spectrum signals to further improve the overall sensing accuracy. Finally, extensive performance evaluation results show that our proposed schemes can significantly outperform peer schemes under sufficiently low SNR settings.

42 ENGINEERING↗

On The Processing of Log Files for Monitoring Antenna Health

In order to improve the quality of geodetic results, we have developed an infrastructure for timely processing of telemetry from IVS observing stations. We check every hour for new log files with telemetry from both VLBI observing sessions, single dish experiments, and stow-in data collection and automatically process them. The telemetry data we use is the system temperature, phase calibration phases and amplitudes, system equivalent flux density, and the differences between formatter clock and GPS clock. For the system temperature and phase calibration, processing includes filtering out outliers and computing averages and rms of the scatter in each scan. Furthermore, for the phase calibration we also compute the group delay and detect spurious signals. Cleaned and post-processed telemetry is archived. Our process detects abnormalities, such as, anomalously high system temperature, unstable phase calibration phases, jumps in the GPS and formatter clock differences, and others. With our procedure, the latency of detection of station abnormalities is reduced to less than two hours. Early detection of abnormalities reduces the amount of affected data since station personnel get early alerts. We discuss our experience of running this system since 2022.

Phase Calibration↗

On The Processing of Log Files for Monitoring Antenna Health

In order to improve the quality of geodetic results, we have developed an infrastructure for timely processing of telemetry from IVS observing stations. We check every hour for new log files with telemetry from both VLBI observing sessions, single dish experiments, and stow-in data collection and automatically process them. The telemetry data we use is the system temperature, phase calibration phases and amplitudes, system equivalent flux density, and the differences between formatter clock and GPS clock. For the system temperature and phase calibration, processing includes filtering out outliers and computing averages and rms of the scatter in each scan. Furthermore, for the phase calibration we also compute the group delay and detect spurious signals. Cleaned and post-processed telemetry is archived. Our process detects abnormalities, such as, anomalously high system temperature, unstable phase calibration phases, jumps in the GPS and formatter clock differences, and others. With our procedure, the latency of detection of station abnormalities is reduced to less than two hours. Early detection of abnormalities reduces the amount of affected data since station personnel get early alerts. We discuss our experience of running this system since 2022.

VLBI↗

Coincident learning for unsupervised anomaly detection of scientific instruments

Abstract Anomaly detection is an important task for complex scientific experiments and other complex systems (e.g. industrial facilities, manufacturing), where failures in a sub-system can lead to lost data, poor performance, or even damage to components. While scientific facilities generate a wealth of data, labeled anomalies may be rare (or even nonexistent), and expensive to acquire. Unsupervised approaches are therefore common and typically search for anomalies either by distance or density of examples in the input feature space (or some associated low-dimensional representation). This paper presents a novel approach called coincident learning for anomaly detection (CoAD), which is specifically designed for multi-modal tasks and identifies anomalies based on coincident behavior across two different slices of the feature space. We define an unsupervised metric, F ^ β , out of analogy to the supervised classification F β statistic. CoAD uses F ^ β to train an anomaly detection algorithm on unlabeled data , based on the expectation that anomalous behavior in one feature slice is coincident with anomalous behavior in the other. The method is illustrated using a synthetic outlier data set and a MNIST-based image data set, and is compared to prior state-of-the-art on two real-world tasks: a metal milling data set and our motivating task of identifying RF station anomalies in a particle accelerator.

43 PARTICLE ACCELERATORS↗

The metal-poor stellar halo in RAVE-TGAS and its implications for the velocity distribution of dark matter

The local velocity distribution of dark matter plays an integral role in interpreting the results from direct detection experiments. We previously showed that metal-poor halo stars serve as excellent tracers of the virialized dark matter velocity distribution using a high-resolution hydrodynamic simulation of a Milky Way-like halo. In this paper, we take advantage of the first Gaia data release, coupled with spectroscopic measurements from the RAdial Velocity Experiment (RAVE), to study the kinematics of stars belonging to the metal-poor halo within an average distance of ~5 kpc of the Sun. We study stars with iron abundances [Fe/H] < -1.5 and -1.8 that are located more than 1.5 kpc from the Galactic plane. Using a Gaussian mixture model analysis, we identify the stars that belong to the halo population, as well as some kinematic outliers. Here we find that both metallicity samples have similar velocity distributions for the halo component, within uncertainties. Assuming that the stellar halo velocities adequately trace the virialized dark matter, we study the implications for direct detection experiments. The Standard Halo Model, which is typically assumed for dark matter, is discrepant with the empirical distribution by ~6σ, predicts fewer high-speed particles, and is anisotropic. As a result, the Standard Halo Model overpredicts the nuclear scattering rate for dark matter masses below ~10 GeV. The kinematic outliers that we identify may potentially be correlated with dark matter substructure, though further study is needed to establish this correspondence.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

MO-F-CAMPUS-T-04: Implementation of a Standardized Monthly Quality Check for Linac Output Management in a Large Multi-Site Clinic

Purpose: This work is to investigate the feasibility of a standardized monthly quality check (QC) of LINAC output determination in a multi-site, multi-LINAC institution. The QC was developed to determine individual LINAC output using the same optimized measurement setup and a constant calibration factor for all machines across the institution. Methods: The QA data over 4 years of 7 Varian machines over four sites, were analyzed. The monthly output constancy checks were performed using a fixed source-to-chamber-distance (SCD), with no couch position adjustment throughout the measurement cycle for all the photon energies: 6 and 18MV, and electron energies: 6, 9, 12, 16 and 20 MeV. The constant monthly output calibration factor (Nconst) was determined by averaging the machines’ output data, acquired with the same monthly ion chamber. If a different monthly ion chamber was used, Nconst was then re-normalized to consider its different NDW,Co-60. Here, the possible changes of Nconst over 4 years have been tracked, and the precision of output results based on this standardized monthly QA program relative to the TG-51 calibration for each machine was calculated. Any outlier of the group was investigated. Results: The possible changes of Nconst varied between 0–0.9% over 4 years. The normalization of absorbed-dose-to-water calibration factors corrects for up to 3.3% variations of different monthly QA chambers. The LINAC output precision based on this standardized monthly QC relative to the TG-51 output calibration is within 1% for 6MV photon energy and 2% for 18MV and all the electron energies. A human error in one TG-51 report was found through a close scrutiny of outlier data. Conclusion: This standardized QC allows for a reasonably simplified, precise and robust monthly LINAC output constancy check, with the increased sensitivity needed to detect possible human errors and machine problems.

60 APPLIED LIFE SCIENCES↗

DISSECTING PHOTOMETRIC REDSHIFT FOR ACTIVE GALACTIC NUCLEUS USING XMM- AND CHANDRA-COSMOS SAMPLES

In this paper, we release accurate photometric redshifts for 1692 counterparts to Chandra sources in the central square degree of the Cosmic Evolution Survey (COSMOS) field. The availability of a large training set of spectroscopic redshifts that extends to faint magnitudes enabled photometric redshifts comparable to the highest quality results presently available for normal galaxies. We demonstrate that morphologically extended, faint X-ray sources without optical variability are more accurately described by a library of normal galaxies (corrected for emission lines) than by active galactic nucleus (AGN) dominated templates, even if these sources have AGN-like X-ray luminosities. Preselecting the library on the bases of the source properties allowed us to reach an accuracy {sigma}{sub {Delta}z/(1+z{sub s{sub p{sub e{sub c)}}}}}{approx}0.015 with a fraction of outliers of 5.8% for the entire Chandra-COSMOS sample. In addition, we release revised photometric redshifts for the 1735 optical counterparts of the XMM-detected sources over the entire 2 deg{sup 2} of COSMOS. For 248 sources, our updated photometric redshift differs from the previous release by {Delta}z > 0.2. These changes are predominantly due to the inclusion of newly available deep H-band photometry (H{sub AB} = 24 mag). We illustrate once again the importance of a spectroscopic training sample and how an assumption about the nature of a source together, with the number and the depth of the available bands, influences the accuracy of the photometric redshifts determined for AGN. These considerations should be kept in mind when defining the observational strategies of upcoming large surveys targeting AGNs, such as eROSITA at X-ray energies and the Australian Square Kilometre Array Pathfinder Evolutionary Map of the Universe in the radio band.

79 ASTRONOMY AND ASTROPHYSICS↗

Anomaly detection in the Zwicky Transient Facility DR3

We present results from applying the SNAD anomaly detection pipeline to the third public data release of the Zwicky Transient Facility (ZTF DR3). The pipeline is composed of three stages: feature extraction, search of outliers with machine learning algorithms, and anomaly identification with followup by human experts. Our analysis concentrates in three ZTF fields, comprising more than 2.25 million objects. A set of four automatic learning algorithms was used to identify 277 outliers, which were subsequently scrutinized by an expert. From these, 188 (68 per cent) were found to be bogus light curves – including effects from the image subtraction pipeline as well as overlapping between a star and a known asteroid, 66 (24 per cent) were previously reported sources whereas 23 (8 per cent) correspond to non-catalogued objects, with the two latter cases of potential scientific interest (e.g. one spectroscopically confirmed RS Canum Venaticorum star, four supernovae candidates, one red dwarf flare). Moreover, using results from the expert analysis, we were able to identify a simple bi-dimensional relation that can be used to aid filtering potentially bogus light curves in future studies. We provide a complete list of objects with potential scientific application so they can be further scrutinised by the community. These results confirm the importance of combining automatic machine learning algorithms with domain knowledge in the construction of recommendation systems for astronomy. Our code is publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗

Correction of Dual-PRF Doppler Velocity Outliers in the Presence of Aliasing

Abstract In Doppler weather radars, the presence of unfolding errors or outliers is a well-known quality issue for radial velocity fields estimated using the dual–pulse repetition frequency (PRF) technique. Postprocessing methods have been developed to correct dual-PRF outliers, but these need prior application of a dealiasing algorithm for an adequate correction. This paper presents an alternative procedure based on circular statistics that corrects dual-PRF errors in the presence of extended Nyquist aliasing. The correction potential of the proposed method is quantitatively tested by means of velocity field simulations and is exemplified in the application to real cases, including severe storm events. The comparison with two other existing correction methods indicates an improved performance in the correction of clustered outliers. The technique proposed is well suited for real-time applications requiring high-quality Doppler radar velocity fields, such as wind shear and mesocyclone detection algorithms, or assimilation in numerical weather prediction models.

58 GEOSCIENCES↗

Quantifying the Impact of LSST u -band Survey Strategy on Photometric Redshift Estimation and the Detection of Lyman-break Galaxies

The Vera C. Rubin Observatory will conduct the Legacy Survey of Space and Time (LSST), promising to discover billions of galaxies out to redshift 7, using six photometric bands (ugrizy) spanning the near-ultraviolet to the near-infrared. The exact number of and quality of information about these galaxies will depend on survey depth in these six bands, which in turn depends on the LSST survey strategy, i.e., how often and how long to expose in each band. u-band depth is especially important for photometric redshift (photo-z) estimation and for detection of high-redshift Lyman-break galaxies (LBGs). In this paper, we use a simulated galaxy catalog and an analytic model for the LBG population to study how recent updates and proposed changes to Rubin’s u-band throughput and LSST survey strategy impact photo-z accuracy and LBG detection. We find that proposed variations in u-band strategy have a small impact on photo-z accuracy for z < 1.5 galaxies, but the outlier fraction, scatter, and bias for higher-redshift galaxies vary by up to 50%, depending on the survey strategy considered. The number of u-band dropout LBGs at z ∼ 3 is also highly sensitive to the u-band depth, varying by up to 500%, while the number of griz-band dropouts is only modestly affected. Under the new u-band strategy recommended by the Rubin Survey Cadence Optimization Committee, we predict u-band dropout number densities of 110 deg −2 (3200 deg −2 ) in year 1 (10) of LSST. We discuss the implications of these results for LSST cosmology.

79 ASTRONOMY AND ASTROPHYSICS↗