Results for “Unsupervised models” — Page 19

DOE OSTI2022

The Power of Priors: Improved Enrichment Safeguards

International safeguards currently rely on material accountancy to verify that declared nuclear material is present and unmodified. Although effective, material accountancy for large bulk facilities can be expensive to implement due to the high precision instrumentation required to meet regulatory targets. Process monitoring has long been considered to improve material accountancy. However, effective integration of process monitoring has been met with mixed results. Given the large successes in other domains, machine learning may present a solution for process monitoring integration. Past work has shown that unsupervised approaches struggle due to measurement error. Although not studied in depth for a safeguards context, supervised approaches often have poor generalization for unseen classes of data (e.g., unseen material loss patterns). This work shows that engineered datasets, when used for training, can improve the generalization of supervised approaches. Further, the underlying models needed to generate these datasets need only accurately model certain high importance features.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

NASA NTRS2023

Aurora Detection From Nighttime Lights for Earth and Space Science Applications

This research leverages data from the Day/Night Band (DNB) of the Visible Infrared Imaging Radiometer (VIIRS) instrument onboard the Suomi National Polar-orbiting Partnership (S-NPP) satellite. We demonstrate the value of mining the VIIRS DNB for aurora and describe our use of unsupervised machine learning to create a binary mask for aurora occurrence. This mask can be used to flag aurora-contaminated observations for NASA's nighttime lights products for Earth science applications. The identification of auroral regions can also be used for Space Weather applications, for example, for comparison with aurora forecast model and with other satellite- or ground-based aurora observations. The DNB is a broadband channel that is sensitive to wavelengths from 500 to 900 nm, which covers most of the visible light spectrum, and as the name implies, captures light even at night with a sensitivity at the nanowatt level. This band is suitable for aurora observations since the light emitted by the aurora tends to be dominated by emissions from atomic oxygen, resulting in a greenish glow at a wavelength of 557.7 nm, especially at an altitude of 110 km. This study compares the global nighttime derived aurora regions for 17 and 18 March with the NOAA Space Weather Prediction Center's (SWPC) probability product for the St. Patrick's Day geomagnetic storm in 2015. VIIRS sensors are slated to be added to the next generation of polar-orbiting operational satellites. Our novel automated approach to aurora identification opens up an efficient way to leverage this unique data source.

Aurora↗

DOE OSTI2020

Big Data and AI at DoE's Legacy Sites - 20546

More than 30 years have passed since DOE started the decommissioning of nuclear weapon complexes and the clean-up of soil and groundwater. All the sites have been collecting and archiving soil and groundwater monitoring datasets; particularly contaminant concentration time-series. These datasets provide unparalleled opportunities to understand the system behavior (including more fundamental hydrological and geochemical processes, the response to various perturbations, the long-term trend and environmental decay rate towards the regulatory limit). This understanding is critical for providing multiple lines of evidences that can support site closure. In this study, we explore the machine learning (ML) and artificial intelligence (AI) applications to the long-term soil and groundwater management at DoE's legacy sites. ML can improve our understanding of the subsurface systems, which is critical for long-term monitoring and management of the sites, while AI can automate or support some of decision-making processes (e.g., anomaly detection, monitoring well placements). The particular focuses are to develop general algorithms to: (1) to identify distinct spatiotemporal patterns and to identify several groups that have similar temporal behaviors, using unsupervised clustering methods, (2) identify the different temporal scales of hydrological responses to climate perturbations by time-series analysis, and (3) reduce the number of monitoring wells by identifying the minimum sufficient number of wells to capture the heterogeneity of the groundwater contaminant plume and concentration distribution, using the Gaussian Process model. We demonstrate our methodology at the Savannah River Site F-Area. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2026

Structure-Aware Unsupervised, Transformational Machine Learning for Drug Discovery (DTRA Basic Research Final Report)

The major goal of this project is to develop machine learning (ML) methods to enable improved predictive power on real drug discovery for novel targets. More specifically, we planned to demonstrate the capability and effectiveness of ML tools utilizing unlabeled large-volume protein-ligand datasets. We investigated multiple pre-training approaches for 3D protein-ligand structure-based foundation models, without relying on experimental binding data. We also addressed scenarios in which crystal structures are unavailable or binding data are limited. We also planned to develop a complete pipeline to screen novel compounds as well as to demonstrate the capability and effectiveness of the developed methods by testing on a realistic drug discovery task such as SARS-CoV-2. While the major goals and milestones remain consistent with the original proposal, certain technical details have been adjusted, based on the experimental results and related outcomes.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Copula based Damage Detection for Structural Health Monitoring

This project utilizes copulas for damage detection in a Structural Health Monitoring (SHM) application. A copula-based system was chosen for the benefit of multivariate joint distribution with a goal to detect damage based on how the system as a whole reacts rather than one or two sensors by themselves. Copulas are commonly used in the field of finance for risk modeling based on two or more random inputs. A few applications in the field of SHM and Non-Destructive Evaluation (NDE) have been researched mostly on risk or reliability of the structure. The goal of this project is to determine if a copula-based approach can be used for damage detection. An unsupervised learning method was desired to reduce the dimensionality, minimal training, and be a faster evaluation method than other unsupervised methods. If a copula method can be used to detect damage what additional information on the damage can be interpreted. The remainder of this report will go over the background needed, SHM methodology, SHM applications, conclusions, and future developments.

47 OTHER INSTRUMENTATION↗

NASA NTRS2012

Obtaining Accurate Change Detection Results from High-Resolution Satellite Sensors

Multi-date acquisitions of high-resolution imaging satellites (e.g. GeoEye and WorldView), can display local changes of current economic interest. However, their large data volume precludes effective manual analysis, requiring image co-registration followed by image-to-image change detection, preferably with minimal analyst attention. We have recently developed an automatic change detection procedure that minimizes false-positives. The processing steps include: (a) Conversion of both the pre- and post- images to reflectance values (this step is of critical importance when different sensors are involved); reflectance values can be either top-of-atmosphere units or have full aerosol optical depth calibration applied using bi-directional reflectance knowledge. (b) Panchromatic band image-to-image co-registration, using an orthorectified base reference image (e.g. Digital Orthophoto Quadrangle) and a digital elevation model; this step can be improved if a stereo-pair of images have been acquired on one of the image dates. (c) Pan-sharpening of the multispectral data to assure recognition of change objects at the highest resolution. (d) Characterization of multispectral data in the post-image ( i.e. the background) using unsupervised cluster analysis. (e) Band ratio selection in the post-image to separate surface materials of interest from the background. (f) Preparing a pre-to-post change image. (g) Identifying locations where change has occurred involving materials of interest.

multispectral↗

NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗

NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗

DOE OSTI2026

Machine Learning-Based Anomaly Detection for PMT Data Quality Monitoring in the SBN and DUNE

Maintaining high-quality detector data is essential for achieving the scientific objectives of the Short-Baseline Neutrino (SBN) Program at Fermilab. Current data quality monitoring (DQM) procedures rely primarily on threshold-based metrics and manual inspection of detector monitoring plots, making the detection of subtle or gradually developing anomalies both time-consuming and dependent on expert interpretation. This project developed and evaluated a machine-learning workflow for automatically identifying anomalous photomultiplier tube (PMT) channels in the Short-Baseline Near Detector (SBND) using optical-hit amplitude data. A Python-based analysis program was developed to process ROOT files, extract statistical features describing individual PMT amplitude distributions, and generate feature vectors for anomaly detection. These features were used to train an Isolation Forest model using data representing normal detector operation. The trained model was subsequently applied to independent detector runs to identify channels exhibiting statistically unusual behavior relative to the learned reference response. To support expert interpretation, the workflow generated complementary diagnostic products, including anomaly score distributions, normalized amplitude comparisons, decision-tree visualizations, and principal component analysis (PCA) projections. This project demonstrated the feasibility of integrating unsupervised machine learning into detector data-quality monitoring and developed a complete workflow for automated PMT performance assessment to aid expert-driven review. Beyond its technical contributions, the VFP appointment fostered a research collaboration between Aurora University and Fermilab and provided direct workforce development benefits by training the visiting faculty member in detector-scale machine-learning methods that are now being incorporated into undergraduate coursework and research. The methodology developed here provides a foundation for future applications to ProtoDUNE and other liquid argon time projection chamber (LArTPC) detectors, contributing to ongoing efforts to improve detector reliability, reduce manual monitoring requirements, and enable scalable data quality monitoring for future large-scale neutrino experiments, including the Deep Underground Neutrino Experiment (DUNE).

Colón Santana, Juan A. [Unlisted, US, IL]↗

NASA NTRS2009

Parametric Analysis of a Hover Test Vehicle using Advanced Test Generation and Data Analysis

Large complex aerospace systems are generally validated in regions local to anticipated operating points rather than through characterization of the entire feasible operational envelope of the system. This is due to the large parameter space, and complex, highly coupled nonlinear nature of the different systems that contribute to the performance of the aerospace system. We have addressed the factors deterring such an analysis by applying a combination of technologies to the area of flight envelop assessment. We utilize n-factor (2,3) combinatorial parameter variations to limit the number of cases, but still explore important interactions in the parameter space in a systematic fashion. The data generated is automatically analyzed through a combination of unsupervised learning using a Bayesian multivariate clustering technique (AutoBayes) and supervised learning of critical parameter ranges using the machine-learning tool TAR3, a treatment learner. Covariance analysis with scatter plots and likelihood contours are used to visualize correlations between simulation parameters and simulation results, a task that requires tool support, especially for large and complex models. We present results of simulation experiments for a cold-gas-powered hover test vehicle.

Gundy-Burlet, Karen↗

NASA NTRS2021

A Method for Landsat and Sentinel 2 (HLS) BRDF Normalization

The Harmonized Landsat/Sentinel-2 (HLS) project aims to generate a seamless surface reflectance product by combining observations from USGS/NASA Landsat-8 and ESA Sentinel-2 remote sensing satellites. These satellites’ sampling characteristics provide nearly constant observation geometry and low illumination variation through the scene. However, the illumination variation throughout the year impacts the surface reflectance by producing higher values for low solar zenith angles and lower reflectance for large zenith angles. In this work, we present a model to derive the bidirectional reflectance distribution function (BRDF) normalization and apply it to the HLS product at 30m spatial resolution. It is based on the BRDF parameters estimated from the MODerate Resolution Imaging Spectroradiometer (MODIS) surface reflectance product (M{O,Y}D09) at 1 km spatial resolution using the VJB method (Vermote et al., 2009). Unsupervised classification (segmentation) of HLS images is used to disaggregate the BRDF parameters to the HLS spatial resolution and to build a BRDF parameters database at HLS scale. We first test the proposed BRDF normalization for different solar zenith angles over two homogeneous sites, in particular one desert and one Peruvian Amazon forest. The proposed method reduces both the correlation with the solar zenith angle and the coefficient of variation (CV) of the reflectance time series in the red and near infrared bands to 4% in forest and keeps a low CV of 3% to 4% for the deserts. Additionally, we assess the impact of the view zenith angle (VZA) in an area of the Brazilian Amazon forest close to the equator, where impact of the angular variation is stronger because it occurs in the principal plane. The directional reflectance shows a strong dependency with the VZA. The current HLS BRDF correction reduces this dependency but still shows an under-correction, especially in the near infrared, while the proposed method shows no dependency with the view angles. We also evaluate the BRDF parameters using field surface albedo measurements as a reference over seven different sites of the US surface radiation budget observing network (SURFRAD) and five sites of the Australian OzFlux network.

Belen Franch↗

DOE OSTI2026

Unsupervised Detection of SOC Spoofing in OCPP 2.0.1 EV Charging Communication Protocol Using One-Class SVM

The electric vehicles (EVs) market keeps growing globally; thus, it is critical to secure the EV charging communication protocols in order to guarantee reliable and fair charging operations among the customers. The Open Charge Point Protocol (OCPP) 2.0.1 supports the communication between the Electric Vehicle Supply Equipment (EVSE) and Charging Station Management Systems (CSMSs); therefore, it becomes vulnerable to several types of attacks, which aim to jeopardize smart charging, billing, and energy management. Specifically, OCPP 2.0.1 allows the self-reporting of the State of Charge (SOC) values, which makes it vulnerable to spoofing-based cyberattacks, which target manipulating the scheduling priorities, distorting the load forecasts, and extending the charging sessions in an unfair manner. In this paper, we try to address this type of attack by providing a comprehensive analysis of the SOC spoofing attacks and introducing a novel unsupervised detection framework based on the One-Class Support Vector Machine (OCSVM) algorithm. Specifically, two types of attack scenarios are analyzed (i.e., priority manipulation and session extension) by deriving engineered features that capture the nonlinear relationships under normal charging behavior. Detailed simulation-based results are derived by utilizing the DESL-EPFL Level 3 EV charging dataset. Our results demonstrate high F1-score and recall in identifying spoofed SOC values and that the proposed OCSVM model demonstrates superior performance compared to alternative clustering and deep-learning based detectors.

EV charging↗

DOE OSTI2024

AI-NERD: Elucidation of relaxation dynamics beyond equilibrium through AI-informed X-ray photon correlation spectroscopy

Abstract Understanding and interpreting dynamics of functional materials in situ is a grand challenge in physics and materials science due to the difficulty of experimentally probing materials at varied length and time scales. X-ray photon correlation spectroscopy (XPCS) is uniquely well-suited for characterizing materials dynamics over wide-ranging time scales. However, spatial and temporal heterogeneity in material behavior can make interpretation of experimental XPCS data difficult. In this work, we have developed an unsupervised deep learning (DL) framework for automated classification of relaxation dynamics from experimental data without requiring any prior physical knowledge of the system. We demonstrate how this method can be used to accelerate exploration of large datasets to identify samples of interest, and we apply this approach to directly correlate microscopic dynamics with macroscopic properties of a model system. Importantly, this DL framework is material and process agnostic, marking a concrete step towards autonomous materials discovery.

36 MATERIALS SCIENCE↗

DOE OSTI2020

MONACO: accurate biological network alignment through optimal neighborhood matching between focal nodes

Motivation: Alignment of protein–protein interaction networks can be used for the unsupervised prediction of functional modules, such as protein complexes and signaling pathways, that are conserved across different species. To date, various algorithms have been proposed for biological network alignment, many of which attempt to incorporate topological similarity between the networks into the alignment process with the goal of constructing accurate and biologically meaningful alignments. Especially, random walk models have been shown to be effective for quantifying the global topological relatedness between nodes that belong to different networks by diffusing node-level similarity along the interaction edges. However, these schemes are not ideal for capturing the local topological similarity between nodes. Results: Here, we propose MONACO, a novel and versatile network alignment algorithm that finds highly accurate pairwise and multiple network alignments through the iterative optimal matching of ‘local’ neighborhoods around focal nodes. Extensive performance assessment based on real networks as well as synthetic networks, for which the ground truth is known, demonstrates that MONACO clearly and consistently outperforms all other state-of-the-art network alignment algorithms that we have tested, in terms of accuracy, coherence and topological quality of the aligned network regions. Furthermore, despite the sharply enhanced alignment accuracy, MONACO remains computationally efficient and it scales well with increasing size and number of networks.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Intrinsic synaptic plasticity of ferroelectric field effect transistors for online learning

Nanoelectronic devices emulating neuro-synaptic functionalities through their intrinsic physics at low operating energies is imperative toward the realization of brain-like neuromorphic computers. In this work, we leverage the non-linear voltage dependent partial polarization switching of a ferroelectric field effect transistor to mimic plasticity characteristics of biological synapses. Here, we provide experimental measurements of the synaptic characteristics for a 28nm high-k metal gate technology based device and develop an experimentally calibrated device model for large-scale system performance prediction. Decoupled read-write paths, ultra-low programming energies and the possibility of arranging such devices in a cross-point architecture demonstrate the synaptic efficacy of the device. Our hardware-algorithm co-design analysis reveals that the intrinsic plasticity of the ferroelectric devices has potential to enable unsupervised local learning in edge devices with limited training data

42 ENGINEERING↗

DOE OSTI2022

Report on PTT Imaging of Defects in AM Metallic Materials-Part 2

Metal Additive Manufacturing (AM) is a promising method for cost-efficient fabrication of complex shape structures for applications in harsh environment, such as in a nuclear reactor. However, internal defects (pores) occur in high-strength AM alloys, which are manufactured with Laser Powder Bed Fusion (LPBF) AM method. Pulsed Infrared Thermography (PIT) is an efficient nondestructive evaluation (NDE) method to examine actual structures, because this method offers one-sided non-contact measurements, and fast processing of large sample areas. However, imaging of material defects, particularly defects with sizes at microscopic level, is challenging. In this report, we benchmark the performance of several Unsupervised Learning (UL) algorithms designed to enhance imaging of microscopic defects in metals with PIT. UL aims to learn the latent principal patterns (dictionaries) in PIT data to detect defects with minimal human supervision. Performance of Independent Component Analysis (ICA), Sparse Coding (SC), Principal Component Analysis (PCA) and Exploratory Factor Analysis (EFA) was compared using F-score, UL model training time and defects reconstruction time. We obtained the average F-score of 0.75, and a highest F-score of 0.89 for the EFA algorithm. Overall, EFA outperforms other UL algorithms considered in this study.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Pulsed Thermal Tomography Nondestructive Examination of Additively Manufactured Reactor Materials and Components (Final Technical Report)

Metal Additive Manufacturing (AM) is a promising method for cost-efficient fabrication of complex shape structures for applications in harsh environment, such as in a nuclear reactor. However, internal defects (pores) occur in high-strength AM alloys, which are manufactured with Laser Powder Bed Fusion (LPBF) AM method. Pulsed Infrared Thermography (PIT) is an efficient nondestructive evaluation (NDE) method to examine actual structures, because this method offers one-sided non-contact measurements, and fast processing of large sample areas. However, imaging of material defects, particularly defects with sizes at microscopic level, is challenging. In this report, we benchmark the performance of several Unsupervised Learning (UL) algorithms designed to enhance imaging of microscopic defects in metals with PIT. UL aims to learn the latent principal patterns (dictionaries) in PIT data to detect defects with minimal human supervision. Performance of Independent Component Analysis (ICA), Sparse Coding (SC), Principal Component Analysis (PCA) and Exploratory Factor Analysis (EFA) was compared using F-score, UL model training time and defects reconstruction time. We obtained the average F-score of 0.75, and a highest F-score of 0.89 for the EFA algorithm. Overall, EFA outperforms other UL algorithms considered in this study. In another approach, we investigate Thermal Tomography (TT), which is a computational method for reconstruction of depth profile of internal material defects from PIT nondestructive evaluation (NDE). TT algorithm obtains depth reconstructions of thermal effusivity, which has been shown to provide visualization of subsurface internals defects in metals. In many applications, one needs to determine the defect shape and orientation from reconstructed effusivity images. Interpretation of TT images is non-trivial because of blurring, which increases with depth due to heat diffusion-based nature of image formation. We have developed a deep learning convolutional neural network (CNN) to classify size and orientation of subsurface material defects in TT images. CNN was trained with TT images produced with computer simulations of 2D metallic structures (thin plates) containing elliptical subsurface voids. Performance of CNN was investigated using test TT images developed with computer simulations of plates containing elliptical defects, and defects with shape imported from scanning electron microscopy (SEM) images. CNN demonstrated the ability to classify radii and angular orientation of elliptical defects in previously unseen test TT images. We have also demonstrated that CNN trained on TT images of elliptical defects is capable of classifying shape and orientation of irregular defects. Training the CNN on irregular defect shapes instead of on elliptical shapes would make the resulting classifications more descriptive of actual defect shapes. However, this requires a much higher volume of SEM images of material defects, which are difficult to obtain because of random occurrence of defects in LPBF. To address this challenge, we developed a generative adversarial network (GAN) to augment the existing dataset of SEM defect images. The GAN model is demonstrated to create novel yet realistic defect shapes that can be used as input for simulated PTT images to train CNN. We also investigate several approaches based on Gaussian Random Circle and Bezier Curves for constructing parametric models of irregular-shape defects.

36 MATERIALS SCIENCE↗

DOE OSTI2024

A Suppression-based STDP Rule Resilient to Jitter Noise in Spike Patterns for Neuromorphic Computing

Multi-spike models of synaptic plasticity, such as the triplet and suppression spike-timing-dependent plasticity (STDP) rules, exhibit better alignment with neurophysiological data in the brain compared to the pair-based STDP rule. Previous studies have empirically shown that the pair-based STDP rule can detect spatiotemporal spike patterns hidden in equally dense distractor spike trains in an unsupervised manner. However, it fails to detect spike patterns influenced by jitter noise. Given that spiking neural networks (SNNs) exhibit variability in generated spike trains in response to the same inputs, it becomes imperative to have learning rules capable of detecting spike patterns even in the presence of jitter noise. In this study, we introduce a simplified suppression-based STDP rule that demonstrates significantly enhanced tolerance to jitter in spike patterns compared to the pair-based STDP rule. Unlike the ideal suppression STDP rule, characterized by an exponential learning window and requiring high-resolution synapses, the simplified rule limits the synaptic efficacy update to a single bit at any given instant. Moreover, it employs 4-bit fixed-point synapses, facilitating straightforward implementation in neuromorphic hardware.

Gautam, Ashish [ORNL]↗