Search NASA⌕ Search

SEARCH · Search NASA

Results for “Unsupervised machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Multilabel proportion prediction and out-of-distribution detection on gamma spectra of short-lived fission products

In the machine learning problem of multilabel classification, the objective is to determine for each test instance which classes the instance belongs to. In this work, we consider an extension of multilabel classification, called multilabel proportion prediction, in the context of radioisotope identification (RIID) using gamma spectra data. We aim to not only predict radioisotope proportions, but also identify out-of-distribution (OOD) spectra. We achieve this goal by viewing gamma spectra as discrete probability distributions, and based on this perspective, we develop a custom semi-supervised loss function that combines a traditional supervised loss with an unsupervised reconstruction error function. Our approach was motivated by its application to the analysis of short-lived fission products from spent nuclear fuel. In particular, we demonstrate that a neural network model trained with our loss function can successfully predict the relative proportions of 37 radioisotopes simultaneously. The model trained with synthetic data was then applied to measurements taken by Pacific Northwest National Laboratory (PNNL) to conduct analysis typically done by subject-matter experts. Here, we also extend our approach to successfully identify when measurements are OOD, and thus should not be trusted, whether due to the presence of a novel source or novel proportions.

Anomaly detection↗

Enhancing Network Visibility and Security through Tensor Analysis

The increasing size, variety, rate of growth and change, and complexity of network data has warranted advanced network analysis and services. Tools that provide automated analysis through traditional or advanced signature-based systems or machine learning classifiers suffer from practical difficulties. These tools fail to provide comprehensive and contextual insights into the network when put to practical use in operational cyber security. In this paper, we present an effective tool for network security and traffic analysis that uses high-performance data analytics based on a class of unsupervised learning algorithms called tensor decompositions. The tool aims to provide a scalable analysis of the network traffic data and also reduce the cognitive load of network analysts and be network-expert-friendly by presenting clear and actionable insights into the network. With this being said, In this paper, we demonstrate the successful use of the tool in two completely diverse operational cyber security environments, namely, (1) security operations center (SOC) for the SCinet network at the SuperComputing (SC) Conference in 2016 and 2017 and (2) Reservoir Labs’ Local Area Network (LAN). In each of these environments, we produce actionable results for cyber security specialists including (but not limited to) (1) finding malicious network traffic involving internal and external attackers using port scans, SSH brute forcing, and NTP amplification attacks, (2) uncovering obfuscated network threats such as data exfiltration using DNS port and using ICMP traffic, and (3) finding network misconfiguration and performance degradation patterns.

97 MATHEMATICS AND COMPUTING↗

Exploring Advanced Computational Tools and Techniques with Artificial Intelligence and Machine Learning in Operating Nuclear Plants

This report presents the project Idaho National Laboratory conducted for Nuclear Regulatory Commission to explore the advanced computational tools and techniques, such as artificial intelligence (AI) and machine learning (ML), for operating nuclear plants. The report reviews the nuclear data sources, with the focus on the operating experience data, that could be applied by advanced computational tools and techniques. Plant-specific and generic (national and international) data from different sources are described. The report describes the relationships between statistics and AI/ML and then introduces the most widely used AI/ML algorithms in both supervised and unsupervised learning. The report reviews the recent applications of advanced computational tools and techniques in various fields of nuclear industry, such as reactor system design and analysis, plant operation and maintenance, and nuclear safety and risk analysis. Finally, the report presents the insights from the project on the potential applicability of AI/ML techniques in improving advanced computational capabilities, how the advanced tools and techniques could contribute to the understanding of safety and risk, and what information would be needed to provide meaningful insights to decision makers. The report also documents an NRC survey on the current state of commercial nuclear power operations relative to the use of AI and ML tools as well as the role of AI/ML tools in nuclear power operations was published by the NRC as in FRN NRC-2021-0048 in April 2021. A summary of the survey including the survey questions, survey participants, survey responses, and the conclusions and insights derived from the survey is provided in the report. Finally, the report investigates potential applications of using AI/ML in operating NPPs and advanced reactors (both advanced LWRs and advanced NLWRs) to improve nuclear plant safety and efficiency. Three main application fields are defined and discussed: (1) plant safety and security assessments; (2) plant degradation modeling, fault and accident diagnosis and prognosis; and (3) plant operation and maintenance efficiency improvement.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Artificial Intelligence/Machine Learning Technology in Power System Applications

The primary purpose of this report is to provide an overview of the advancement in artificial intelligence and machine learning (AI/ML) technologies and their applications in power systems. It offers a foundation for understanding the transformative role of AI/ML in power systems and aims to stimulate further research and development in this area. This report begins with a historical perspective of AI/ML technologies, then explores their advancement to today’s prominence. The document highlights key contributors to the success of AI/ML technologies, including increased computational power, greater data availability, innovative algorithms, and advanced tools. It further introduces various AI/ML techniques, including supervised, unsupervised and reinforcement learning, graph neural networks, and generative AI. It also emphasizes the critical importance of ensuring the safety, security, and trustworthiness of these AI/ML techniques within this sector. The report reviews the recent representative advancements in various power system applications enhanced by AI/ML techniques, underscoring key developments and their transformative impact as evidenced by numerous studies. It also explores both the opportunities and challenges associated with the application of AI/ML technologies to improve power system applications. While the report extensively covers AI/ML applications in power systems, focusing primarily on the technical and operational aspects, it may not thoroughly explore the sociopolitical, economic, and broader regulatory implications of AI/ML integration in power systems. AI/ML techniques hold significant potential for enhancing power system applications; however, they are not omnipotent. It is crucial to acknowledge their limitations and understand that they may not be able to address all challenges in the power system domain. Various factors must be considered that influence the implementation, adoption, and effectiveness of AI/ML solutions, including but not limited to safety, security, transparency, and trustworthiness. Additionally, the incorporation of advanced human–machine interfaces is essential, as it enables humans to validate the effectiveness of AI/ML solutions while remaining actively engaged, fostering trust in AI/ML deployment. Finally, the report summarizes AI/ML research activities supported by the Department of Energy (DOE) Office of Electricity (OE) through the Advanced Grid Modeling (AGM) program. The work aligns with the interests and mission of DOE-OE AGM, with the report serving as a resource for identifying existing progress and for pinpointing future applications within AI/ML that need further exploration and support.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Machine Learning for Anomaly Detection in Neural Network Security and SRF Cavities

This dissertation explores the development and deployment of machine learning approaches to address critical challenges in anomaly detection across two distinct domains: neural network security in federated learning settings and cavity behavior analysis in particle accelerator operations at Jefferson Lab in Newport News, Virginia. Anomaly detection identifies deviations from expected patterns, safeguarding systems in cybersecurity, industry, and research against malicious activities and failures. This dissertation demonstrates how our machine learning approaches enhance detection accuracy and efficiency in both neural network security and industrial applications. First, we investigate vulnerabilities in deep neural networks deployed in federated learning. Although federated learning preserves user privacy by training models locally, it remains vulnerable to backdoor attacks, in which malicious participants embed hidden triggers that induce targeted misbehavior. We propose a self-supervised contrastive learning framework to detect and mitigate such backdoor attacks. In our experiments, this method achieves higher detection accuracy and lower false positive rates than existing defenses, while operating without access to local model updates or original training data and thus preserving the privacy guarantees of the federated setting. Second, we address the operational reliability of superconducting radio-frequency (SRF) cavities at the Continuous Electron Beam Accelerator Facility (CEBAF). Our research leverages an unsupervised learning approach, combined with Principal Component Analysis (PCA) and k-means clustering, to identify anomalous behaviors in SRF cavities. Our method detects subtle anomalous behavior by analyzing SRF signal data. This knowledge allows for the early detection and resolution of potential faults, significantly improving the efficiency and reliability of operations. Third, we extend these insights to time-series anomaly detection more broadly. We design a contrastive-learning based model tailored to increasingly dynamic environments and academic research. This model improves detection accuracy in settings that require real-time monitoring and predictive maintenance. Our research underscores the broader applicability and impact of advanced machine learning techniques in anomaly detection. By extracting meaningful patterns from complex data, machine learning can significantly enhance security in distributed neural networks and improve the efficiency of particle accelerator operations. This dissertation serves as a stepping stone for future investigations into the vast possibilities of anomaly detection, inspiring further exploration and development of machine learning techniques in this field.

Ferguson, Hal [Old Dominion University]↗

Auto-deconvolution and molecular networking of gas chromatography–mass spectrometry data

We engineered a machine learning approach, MSHub, to enable auto-deconvolution of gas chromatography–mass spectrometry (GC–MS) data. We then designed workflows to enable the community to store, process, share, annotate, compare and perform molecular networking of GC–MS data within the Global Natural Product Social (GNPS) Molecular Networking analysis platform. MSHub/GNPS performs auto-deconvolution of compound fragmentation patterns via unsupervised non-negative matrix factorization and quantifies the reproducibility of fragmentation patterns across samples.

47 OTHER INSTRUMENTATION↗

Watch and learn—a generalized approach for transferrable learning in deep neural networks via physical principles

Transfer learning refers to the use of knowledge gained while solving a machine learning task and applying it to the solution of a closely related problem. Such an approach has enabled scientific breakthroughs in computer vision and natural language processing where the weights learned in state-of-the-art models can be used to initialize models for other tasks which dramatically improve their performance and save computational time. Here we demonstrate an unsupervised learning approach augmented with basic physical principles that achieves fully transferrable learning for problems in statistical physics across different physical regimes. By coupling a sequence model based on a recurrent neural network to an extensive deep neural network, we are able to learn the equilibrium probability distributions and inter-particle interaction models of classical statistical mechanical systems. Our approach, distribution-consistent learning, DCL, is a general strategy that works for a variety of canonical statistical mechanical models (Ising and Potts) as well as disordered interaction potentials. Using data collected from a single set of observation conditions, DCL successfully extrapolates across all temperatures, thermodynamic phases, and can be applied to different length-scales. This constitutes a fully transferrable physics-based learning in a generalizable approach.

97 MATHEMATICS AND COMPUTING↗

To Fail or not to Fail: An Exploration of Machine Learning Techniques for Predictive Maintenance

Predictive maintenance refers to the ability to predict when machinery or systems need to be maintained. Making an accurate prediction is quite challenging given the costs for both over-estimating (unnecessary maintenance and reduction in availability of assets) and under-estimating (untimely breakdowns and possible loss of equipment or lives). To address these challenges researchers were able to develop new approaches for analyzing oil samples taken extracting samples from oil-wetted machinery that may provide information critical to developing predictive capabilities. We consider the problem from both supervised (though data limited) and unsupervised approaches and provide a first look into a data driven approach for identification of condition indicators. Through this work we identify a collection of candidate features that can form the basis of condition indicators for both a high level discrimination of failure vs. normal operation as well as a set for potential failure mode identification. Finally, we present an anomaly detection framework for detecting failures which can be a viable solution for an onboard analysis tool in deployed systems.

predictive maintenance, anomaly detection, Laserne↗

Universal and interpretable classification of atomistic structural transitions via unsupervised graph learning

Materials processing often occurs under extreme dynamic conditions leading to a multitude of unique structural environments. These structural environments generally occur at high temperatures and/or high pressures, often under non-equilibrium conditions, which results in drastic changes in the material's structure over time. Computational techniques, such as molecular dynamics simulations, can probe the atomic regime under these extreme conditions. However, characterizing the resulting diverse atomistic structures as a material undergoes extreme changes in its structure has proved challenging due to the inherently non-linear relationship between structures as large-scale changes occur. Here, we introduce SODAS++, a universal graph neural network framework, that can accurately and intuitively quantify the atomistic structural evolution corresponding to the transition between any two arbitrary phases. We showcase SODAS++ for both solid–solid and solid–liquid transitions for systems of increasing geometric and chemical complexity, such as colloidal systems, elemental Al, rutile and amorphous TiO 2 , and the non-stoichiometric ternary alloy Ag 26 Au 5 Cu 19 . Finally, we show that SODAS++ can accurately quantify all transitions in a physically interpretable manner, showcasing the power of unsupervised graph neural network encodings for capturing the complex and non-linear pathway, a material's structure takes as it evolves.

36 MATERIALS SCIENCE↗

Molecular dipole moment learning via rotationally equivariant derivative kernels in molecular-orbital-based machine learning

This study extends the accurate and transferable molecular-orbital-based machine learning (MOB-ML) approach to modeling the contribution of electron correlation to dipole moments at the cost of Hartree–Fock computations. A MOB pairwise decomposition of the correlation part of the dipole moment is applied, and these pair dipole moments could be further regressed as a universal function of MOs. The dipole MOB features consist of the energy MOB features and their responses to electric fields. An interpretable and rotationally equivariant derivative kernel for Gaussian process regression (GPR) is introduced to learn the dipole moment more efficiently. The proposed problem setup, feature design, and ML algorithm are shown to provide highly accurate models for both dipole moments and energies on water and 14 small molecules. To demonstrate the ability of MOB-ML to function as generalized density-matrix functionals for molecular dipole moments and energies of organic molecules, we further apply the proposed MOB-ML approach to train and test the molecules from the QM9 dataset. The application of local scalable GPR with Gaussian mixture model unsupervised clustering GPR scales up MOB-ML to a large-data regime while retaining the prediction accuracy. In addition, compared with the literature results, MOB-ML provides the best test mean absolute errors of 4.21 mD and 0.045 kcal/mol for dipole moment and energy models, respectively, when training on 110 000 QM9 molecules. The excellent transferability of the resulting QM9 models is also illustrated by the accurate predictions for four different series of peptides.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Online data-driven changepoint detection for high-dimensional dynamical systems

In this study, the detection of anomalies or transitions in complex dynamical systems is of critical importance to various applications. In this study, we propose the use of machine learning to detect changepoints for high-dimensional dynamical systems. Here, changepoints indicate instances in time when the underlying dynamical system has a fundamentally different characteristic—which may be due to a change in the model parameters or due to intermittent phenomena arising from the same model. We propose two complementary approaches to achieve this, with the first devised using arguments from probabilistic unsupervised learning and the latter devised using supervised deep learning. To accelerate the deployment of transition detection algorithms in high-dimensional dynamical systems, we introduce dimensionality reduction techniques. Our experiments demonstrate that transitions can be detected efficiently, in real-time, for the two-dimensional forced Kolmogorov flow and the Rössler dynamical system, which are characterized by anomalous regimes in phase space where dynamics are perturbed off the attractor at potentially uneven intervals. Finally, we also demonstrate how variations in the frequency of detected changepoints may be utilized to detect a significant modification to the underlying model parameters by utilizing the Lorenz-63 dynamical system.

97 MATHEMATICS AND COMPUTING↗

Preface to the Focus Section on Machine Learning in Seismology

Machine learning (ML) is a collection of algorithms and statistical models that enable computers to extract relevant patterns and information from large data sets. Unlike physical modeling approaches, in which scientists develop theories based on physical laws to compare with real–world observations, ML approaches learn directly from data without explicitly reasoning about the underlying physical mechanisms. ML algorithms are often categorized into supervised and unsupervised learning (see fig. 2 in Kong et al., 2018). Supervised learning algorithms build a model from existing labeled input data with the goal of predicting the labels of new data.

58 GEOSCIENCES↗

Machine learning methods for probabilistic locked-mode predictors in tokamak plasmas

A rotating tokamak plasma can interact resonantly with the external helical magnetic perturbations, also known as error fields. This can lead to locking and then to disruptions. We leverage machine learning (ML) methods to predict the locking events. We use a coupled third-order nonlinear ordinary differential equation model to represent the interaction of the magnetic perturbation and the plasma rotation with the error field. This model is sufficient to describe qualitatively the locking and unlocking bifurcations. Here, we explore using ML algorithms with the simulation data and experimental data, focusing on the methods that can be used with sparse datasets. These methods lead to the possibility of the avoidance of locking in real-time operations. We describe the operational space in terms of two control parameters: the magnitude of the error field and the rotation frequency associated with the momentum source that maintains the plasma rotation. The outcomes are quan- tified by order parameters that completely characterize the state, whether locked or unlocked. We use unsupervised ML methods to classify locked/unlocked states and note the usefulness of a certain normalization of the order parameters. Three supervised ML classifiers are used in suite to estimate the probability of locking in the region of control parameter space with hysteresis, i.e., the set of control parameters for which both locked and unlocked states can exist. The results show that a neural network gives the best estimate of the locking probability. An analogy of the present locking model with the van der Waals equation of state is also provided.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Unsupervised Detection of SOC Spoofing in OCPP 2.0.1 EV Charging Communication Protocol Using One-Class SVM

The electric vehicles (EVs) market keeps growing globally; thus, it is critical to secure the EV charging communication protocols in order to guarantee reliable and fair charging operations among the customers. The Open Charge Point Protocol (OCPP) 2.0.1 supports the communication between the Electric Vehicle Supply Equipment (EVSE) and Charging Station Management Systems (CSMSs); therefore, it becomes vulnerable to several types of attacks, which aim to jeopardize smart charging, billing, and energy management. Specifically, OCPP 2.0.1 allows the self-reporting of the State of Charge (SOC) values, which makes it vulnerable to spoofing-based cyberattacks, which target manipulating the scheduling priorities, distorting the load forecasts, and extending the charging sessions in an unfair manner. In this paper, we try to address this type of attack by providing a comprehensive analysis of the SOC spoofing attacks and introducing a novel unsupervised detection framework based on the One-Class Support Vector Machine (OCSVM) algorithm. Specifically, two types of attack scenarios are analyzed (i.e., priority manipulation and session extension) by deriving engineered features that capture the nonlinear relationships under normal charging behavior. Detailed simulation-based results are derived by utilizing the DESL-EPFL Level 3 EV charging dataset. Our results demonstrate high F1-score and recall in identifying spoofed SOC values and that the proposed OCSVM model demonstrates superior performance compared to alternative clustering and deep-learning based detectors.

EV charging↗

A Strategy for NACS investment in Machine Learning

The Nuclear and Chemical Sciences (NACS) Division furnishes the expertise in the scientific areas of chemical, nuclear and isotopic sciences that are foundational in the Laboratory’s national security missions. This expertise is maintained and advanced through identification, development and application of state-of-the-art theoretical, computational and experimental methods and tools. Recent developments in artificial intelligence and machine learning (AI/ML) techniques enabled by advances in computing capabilities and widespread availability of powerful software implementations have made use of these techniques ubiquitous across both science and industry. While the scope of AI/ML applications is incredibly large and evolves very rapidly, the topics most relevant to NACS missions fall into the general category of detecting, categorizing or identifying features in large, complex datasets using either supervised or unsupervised learning. This covers both basic scientific data analysis and the development of efficient surrogate models of real-life technological systems, experimental detectors, or theoretical models. To remain at the forefront of its core scientific disciplines, NACS must both cultivate ML expertise as well as continuously explore applying this expertise to new problems or utilizing new methods. This document identifies the key areas where this support is critical and provides a strategy for investing in them.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

Deep Learning for Unsupervised Insider Threat Detection in Structured Cyber Security Data Streams

Analysis of an organization's computer network activity is a key component of early detection and mitigation of insider threat, a growing concern for many organizations. Raw system logs are a prototypical example of streaming data that can quickly scale beyond the cognitive power of a human analyst. As a prospective filter for the human analyst, we present an online unsupervised deep learning approach to detect anomalous network activity from system logs in real time. Our models decompose anomaly scores into the contributions of individual user behavior features for increased interpretability to aid analysts reviewing potential cases of insider threat. Using the CERT Insider Threat Dataset v6.2 and threat detection recall as our performance metric, our novel deep and recurrent neural network models outperform Principal Component Analysis, Support Vector Machine and Isolation Forest based anomaly detection baselines. For our best model, the events labeled as insider threat activity in our dataset had an average anomaly score in the 95.53 percentile, demonstrating our approach's potential to greatly reduce analyst workloads.

Deep Learning, Insider Threat↗

Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams

Analysis of an organization’s computer network activity is a key component of early detection and mitigation of insider threat, a growing concern for many organizations. Raw system logs are a prototypical example of streaming data that can quickly scale beyond the cognitive power of a human analyst. As a prospective filter for the human analyst, we present an online unsupervised deep learning approach to detect anomalous network activity from system logs in real time. Our models decompose anomaly scores into the contributions of individual user behavior features for increased interpretability to aid analysts reviewing potential cases of insider threat. Using the CERT Insider Threat Dataset v6.2 and threat detection recall as our performance metric, our novel deep and recurrent neural network models outperform Principal Component Analysis, Support Vector Machine and Isolation Forest based anomaly detection baselines. For our best model, the events labeled as insider threat activity in our dataset had an average anomaly score in the 95.53 percentile, demonstrating our approach’s potential to greatly reduce analyst workloads.

Tuor, Aaron R.↗

Revealing Fundamental Physics from the Daya Bay Neutrino Experiment Using Deep Neural Networks

Experiments in particle physics produce enormous quantities of data that must be analyzed and interpreted by teams of physicists. This analysis is often exploratory, where scientists are unable to enumerate the possible types of signal prior to performing the experiment. Thus, tools for summarizing, clustering, visualizing and classifying high-dimensional data are essential. Here in this work, we show that meaningful physical content can be revealed by transforming the raw data into a learned high-level representation using deep neural networks, with measurements taken at the Daya Bay Neutrino Experiment as a case study. We further show how convolutional deep neural networks can provide an effective classification filter with greater than 97% accuracy across different classes of physics events, significantly better than other machine learning approaches.

60 APPLIED LIFE SCIENCES↗