Search NASA⌕ Search

SEARCH · Search NASA

Results for “supervised learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Foundation AI Models for Science

Foundation Models (FM) are AI models that are designed to replace a task or an application specific model. These FM can be applied to many different downstream applications. These FM are trained using self supervised techniques and can be built on any type of sequence data. The use of self supervised learning removes the hurdle for developing a large labeled dataset for training. Most FM use transformer architecture utilizes the notion of self attention which allows the network to model the influence of distant data points to each other both in space and time. The FM models exhibit emergent properties that are induced from the data. FM can be an important tool for science. The scale of these models results in better performance for different downstream applications and these applications show better accuracy over models built from scratch. FM drastically reduces the cost of entry to build different downstream applications both in time and effort. FM for selected science datasets such as optical satellite data, can accelerate applications ranging from data quality monitoring, feature detection and prediction. FM can make it easier to infuse AI into scientific research by removing the training data bottleneck and increasing the use of science data.

Manil Maskey↗

Revolutionizing Earth Science with Generalized AI Models

Foundation Models (FM) are generalized Artificial Intelligence (AI) models that are designed to replace a task or an application-specific model and can be used for many downstream applications. These FM can be built on any sequence data and are trained utilizing self-supervised approaches. The obstacle of creating a sizable labeled dataset for training is removed by using self-supervised learning. Most FM employ transformer design that takes advantage of the idea of self-attention, allowing the network to represent the impact of distant data points on one another in space and time. The FM models show emergent qualities that are induced from the data. FM can become a valuable tool for Earth science researchers. Due to the size of these models, downstream applications built fine-tuning these FM perform better and exhibit greater accuracy than models created from scratch. FM significantly lowers the entry barrier in terms of both the time and effort required to develop various downstream applications. For some scientific datasets, such as optical remote sensing data, FM can speed up processes like classification, object detection and prediction. By eliminating the training data bottleneck and maximizing the usage of science data, FM can make it simpler to integrate AI into scientific research. Initial results for three different FMs will be presented.

Rahul Ramachandran↗

AUTOMATIC RETINA EXUDATES SEGMENTATION WITHOUT A MANUALLY LABELLED TRAINING SET

Diabetic macular edema (DME) is a common vision threatening complication of diabetic retinopathy which can be assessed by detecting exudates (a type of bright lesion) in fundus images. In this work, two new methods for the detection of exudates are presented which do not use a supervised learning step and therefore do not require ground-truthed lesion training sets which are time consuming to create, difficult to obtain, and prone to human error. We introduce a new dataset of fundus images from various ethnic groups and levels of DME which we have made publicly available. We evaluate our algorithm with this dataset and compare our results with two recent exudate segmentation algorithms. In all of our tests, our algorithms perform better or comparable with an order of magnitude reduction in computational time.

59 BASIC BIOLOGICAL SCIENCES↗

Characterizing mammography reports for health analytics

As massive collections of digital health data are becoming available, the opportunities for large scale automated analysis increase. In particular, the widespread collection of detailed health information is expected to help realize a vision of evidence-based public health and patient-centric health care. Within such a framework for large scale health analytics we describe several methods to characterize and analyze free-text mammography reports, including their temporal dimension, using information retrieval, supervised learning, and classical statistical techniques. We present experimental results with a large collection of mostly unlabeled reports that demonstrate the validity and usefulness of the approach, since these results are consistent with the known features of the data and provide novel insights about it.

60 APPLIED LIFE SCIENCES↗

APPLICATION OF NEURAL NETWORK ALGORITHMS FOR BPM LINEARIZATION

Stripline BPM sensors contain inherent non-linearities, as a result of field distortions from the pickup elements. Many methods have been devised to facilitate corrections, often employing polynomial fitting. The cost of computation makes real-time correction difficult, particulalry when integer math is utilized. The application of neural-network technology, particularly the multi-layer perceptron algorithm, is proposed as an efficient alternative for electrode linearization. A process of supervised learning is initially used to determine the weighting coefficients, which are subsequently applied to the incoming electrode data. A non-linear layer, known as an activation layer, is responsible for the removal of saturation effects. Implementation of a perceptron in an FPGA-based software-defined radio (SDR) is presented, along with performance comparisons. In addition, efficient calculation of the sigmoidal activation function via the CORDIC algorithm is presented.

43 PARTICLE ACCELERATORS↗

RAVEN, a New Software for Dynamic Risk Analysis

RAVEN is a generic software driver to perform parametric and probabilistic analysis of code simulating complex systems. Initially developed to provide dynamic risk analysis capabilities to the RELAP-7 code [1] is currently being generalized with the addition of Application Programming Interfaces (APIs). These interfaces are used to extend RAVEN capabilities to any software as long as all the parameters that need to be perturbed are accessible by inputs files or directly via python interfaces. RAVEN is capable to investigate the system response probing the input space using Monte Carlo, grid strategies, or Latin Hyper Cube schemes, but its strength is its focus toward system feature discovery like limit surfaces separating regions of the input space leading to system failure using dynamic supervised learning techniques. The paper will present an overview of the software capabilities and their implementation schemes followed by same application examples.

97 MATHEMATICS AND COMPUTING↗

Approximate Inverse Chain Preconditioner: Iteration Count Case Study for Spectral Support Solvers

As the growing availability of computational power slows, there has been an increasing reliance on algorithmic advances. However, faster algorithms alone will not necessarily bridge the gap in allowing computational scientists to study problems at the edge of scientific discovery in the next several decades. Often, it is necessary to simplify or precondition solvers to accelerate the study of large systems of linear equations commonly seen in a number of scientific fields. Preconditioning a problem to increase efficiency is often seen as the best approach; yet, preconditioners which are fast, smart, and efficient do not always exist. Following the progress of [1], we present a new preconditioner for symmetric diagonally dominant (SDD) systems of linear equations. These systems are common in certain PDEs, network science, and supervised learning among others. Based on spectral support graph theory, this new preconditioner builds off of the work of [2], computing and applying a V-cycle chain of approximate inverse matrices. This preconditioner approach is both algebraic in nature as well as hierarchically-constrained depending on the condition number of the system to be solved. Due to its generation of an Approximate Inverse Chain of matrices, we refer to this as the AIC preconditioner. We further accelerate the AIC preconditioner by utilizing precomputations to simplify setup and multiplications in the con-text of an iterative Krylov-subspace solver. While these iterative solvers can greatly reduce solution time, the number of iterations can grow large quickly in the absence of good preconditioners. Initial results for the AIC preconditioner have shown a very large reduction in iteration counts for SDD systems as compared to standard preconditioners such as Incomplete Cholesky (ICC) and Multigrid (MG). We further show significant reduction in iteration counts against the more advanced Combinatorial Multigrid (CMG) preconditioner. We have further developed no-fill sparsification techniques to ensure that the computational cost of applying the AIC preconditioner does not grow prohibitively large as the depth of the V-cycle grows for systems with larger condition numbers. Our numerical results have shown that these sparsifiers maintain the sparsity structure of our system while also displaying significant reductions in iteration counts.1 2

97 MATHEMATICS AND COMPUTING↗

Approximate Inverse Chain Preconditioner: Iteration Count Case Study for Spectral Support Solvers

As the growing availability of computational power slows, there has been an increasing reliance on algorithmic advances. However, faster algorithms alone will not necessarily bridge the gap in allowing computational scientists to study problems at the edge of scientific discovery in the next several decades. Often, it is necessary to simplify or precondition solvers to accelerate the study of large systems of linear equations commonly seen in a number of scientific fields. Preconditioning a problem to increase efficiency is often seen as the best approach; yet, preconditioners which are fast, smart, and efficient do not always exist. Following the progress of [1], we present a new preconditioner for symmetric diagonally dominant (SDD) systems of linear equations. These systems are common in certain PDEs, network science, and supervised learning among others. Based on spectral support graph theory, this new preconditioner builds off of the work of [2], computing and applying a V-cycle chain of approximate inverse matrices. This preconditioner approach is both algebraic in nature as well as hierarchically-constrained depending on the condition number of the system to be solved. Due to its generation of an Approximate Inverse Chain of matrices, we refer to this as the AIC preconditioner. We further accelerate the AIC preconditioner by utilizing precomputations to simplify setup and multiplications in the con-text of an iterative Krylov-subspace solver. While these iterative solvers can greatly reduce solution time, the number of iterations can grow large quickly in the absence of good preconditioners. Initial results for the AIC preconditioner have shown a very large reduction in iteration counts for SDD systems as compared to standard preconditioners such as Incomplete Cholesky (ICC) and Multigrid (MG). We further show significant reduction in iteration counts against the more advanced Combinatorial Multigrid (CMG) preconditioner. We have further developed no-fill sparsification techniques to ensure that the computational cost of applying the AIC preconditioner does not grow prohibitively large as the depth of the V-cycle grows for systems with larger condition numbers. Our numerical results have shown that these sparsifiers maintain the sparsity structure of our system while also displaying significant reductions in iteration counts.1 2

97 MATHEMATICS AND COMPUTING↗

An Agenda for Multimodal Foundation Models for Earth Observation

Archives of remote sensing (RS) data are increasing swiftly as new sensing modalities with enhanced spatiotemporal resolution become operational. While promising new breakthroughs, the sheer volume of RS archives stretches the limits of human analysts and existing AI tools, as most models are: i) limited to single data modalities; ii) task-specific; iii) heavily reliant on labeled data. The emerging Foundation Models (FMs) have the potential to address these limitations. Trained on vast unlabeled datasets through self-supervised learning, FMs enable generic feature extraction that facilitate specialization to a wide variety of downstream tasks. This paper describes a vision towards an FM for multimodal Earth Observation data (FM4EO), discussing key building blocks and open challenges. We put particular emphasis on multimodal reasoning, a topic underexplored in EO. Our ultimate goal is a practical path toward FM4EO with capacity to unlock breakthroughs in few-shot learning scenarios, multimodal geographic knowledge integration, synthesis, and hypothesis generation.

Ambrozio Dias, Philipe↗

Assembling Multiple Models Within the RAVEN Framework

RAVEN (Risk Analysis Virtual Environment) is an INL-developed software tool that can be used to identify and increase the safety margin in complex systems. As a generic software framework, RAVEN is designed to perform parametric and probabilistic analysis based on the response of complex system codes. RAVEN is capable of investigating the system response as well as the input space using Monte Carlo, Grid, or Latin Hyper Cube sampling schemes, but its strength is focused toward system feature discovery, such as limit surfaces, separating regions of the input space leading to system failure, using dynamic supervised learning techniques. RAVEN is currently able to construct multi-targets Reduced Order Models (ROMs), which are aimed to represent the response of a system (in a fixed configuration) for multiple Figures of Merits (FOMs) and time-dependent ROMs. These capabilities represent the initial steps for a larger implementation about the interaction of multiple models. In fact, in several cases, multiple models need to interface with each other since the initial conditions of one are dependent on the outcomes of another. To better understand the problem that here is solved, it is useful to consider a simple example: The following problem is considered: a weather forecast simulation code 'a' is used to compute the external (i.e. ambient) temperature in a certain location. A second model 'b' is inquired to compute the average temperature in a room having as boundary condition, among several others, the external ambient temperature. The response of the model 'b' depends on the outcome of the model 'a'. The reported example is only aimed to illustrate the reason why the creation of a framework to make interact different models is a key development for the advancement of RAVEN as a comprehensive calculation flow driver. Before reporting how the ensemble-models have been implemented, it is necessary to briefly describe the representative Model 'entities' that are available in RAVEN.

42 ENGINEERING↗

Neural Scaling Laws for Jet Generation

Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.

Amram, Oz [Fermilab]↗

Evaluating the Computational Requirements of using SVM software to train Data-Intensive Problems

Support vector machine (SVM) technology is a growing class of computational algorithms for solving classification problems using supervised learning. There are many freely available SVM codes with implementations of varying granularity in the core optimization task. The performance of SVM implementations is related to four key elements; (1) the size and dimension of the data set on which they are operating, (2) the granularity of their core SVM optimization implementation, (3) the kernel transformation applied to the data, and the (4) underlying hardware on which the implementation is running. To assess the performance of different SVM implementations, several freely available codes representing the spectrum of optimization granularity were built on a variety of hardware—two linux clusters, and a shared memory machine. Binary classifiers were trained on datasets of varying size containing anywhere from a few hundred to over 50,000 vectors. Performance of the methods was measured in terms of wall-clock time for training, statistical quality of the resulting classifier, hardware performance (memory footprint, memory bandwidth, floating point unit usage, I/O bandwidth, and instruction stalls), robustness and portability.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Task-Based Neuromodulation Architecture for Lifelong Learning

The conventional supervised deep learning frameworks for image classification focus on finding the minimizer of an objective function. However, for a lifelong learning system, a network has to be able to dynamically learn new tasks from very few examples. In biological neural networks, it is observed that neuromodulation acts as a key factor in continual learning. In this work, we propose a dynamic learning system, ModNet, wherein a modulatory unit regulates the learning depending on the context and the internal state of the system. The modulatory unit, which consists of a group of neurons, exercises the use of neuromodulation and Hebbian plasticity mechanisms over a population of neurons instead of relying on backpropagation of error gradients. An efficient digital architecture with on-chip training and resource sharing is proposed to facilitate the efficacy of task-based processing.

brain-inspired↗

Supervised machine learning-based multivariate regression of parallel closures for a high-collisionality deuterium-carbon plasma

Many plasmas of interest in laboratory experiments and space consist of multiple ion species. In tokamak edge plasmas, for instance, ionized impurities expelled from the vessel wall influence plasma transport. When describing multi-species plasmas using fluid equations, we need accurate closure relations to close the set of fluid equations. In this study, we introduce the development of fitting formulas for parallel closures using supervised machine learning, in conjunction with the recent closure theory, considering multi-ion collisions and arbitrary ion temperatures. We apply this approach to a high-collisionality deuterium-carbon plasma and demonstrate its effectiveness. As a result, the machine learning-based method for developing practical and accurate closures can be extended to a wider range of plasmas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Semi-supervised machine-learning classification of materials synthesis procedures

Digitizing large collections of scientific literature can enable new informatics approaches for scientific analysis and meta-analysis. However, most content in the scientific literature is locked-up in written natural language, which is difficult to parse into databases using explicitly hard-coded classification rules. In this work, we demonstrate a semi-supervised machine-learning method to classify inorganic materials synthesis procedures from written natural language. Without any human input, latent Dirichlet allocation can cluster keywords into topics corresponding to specific experimental materials synthesis steps, such as “grinding” and “heating”, “dissolving” and “centrifuging”, etc. Guided by a modest amount of annotation, a random forest classifier can then associate these steps with different categories of materials synthesis, such as solid-state or hydrothermal synthesis. Finally, we show that a Markov chain representation of the order of experimental steps accurately reconstructs a flowchart of possible synthesis procedures. Our machine-learning approach enables a scalable approach to unlock the large amount of inorganic materials synthesis information from the literature and to process it into a standardized, machine-readable database.

36 MATERIALS SCIENCE↗

A Weakly-Supervised, Multitask Deep Learning Framework for Shadow Mitigation in Remote Sensing Imagery

We propose a weakly-supervised, multitask framework for training a convolutional neural network to solve the problem of cloud shadow mitigation given only cloud and shadow masks as labels. The network minimizes the Wasserstein distance between shadows and their proximal sunlit neighborhoods, generating a supervisory signal directly from within the input image. We extract further utility from the shadow mask through multitask learning by introducing an auxiliary task of shadow segmentation. Our approach is advantageous since it performs mitigation in an end-to-end framework which requires only a shadowed image for inference. We apply this process to the Landsat 8 OLI SPARCS validation data set and demonstrate plausible results.

Couwenhoven, Scott↗

Multi-Class Anomaly Detection in Flight Data using Semi-Supervised Explainable Deep Learning Model

Identifying precursor for safety incidents in aviation data is a crucial task, yet extremely challenging. The main approach, in practice, leverages domain expertise to define expected tolerances in system’s behavior and alarm exceedance from such safety margins. However, this approach is incapable of identifying unknown risk and vulnerabilities. Machine learning has been long studied and deployed to identify precursors for such anomalies, with the great challenge of the need for sufficient labelled set of data to achieve a reliable and accurate performance. In this article, we develop an explainable deep semi-supervised model for anomaly detection in aviation, building upon recent advancements in the machine learning literature. The proposed model combines feature engineering and classification in the feature space, while leveraging all available data (labelled and unlabeled). Validating on two case studies of anomaly detection in take-off and landing phases of commercial aircraft, we show that our model is able to outperform state-of-the-art supervised anomaly detection model and reach significantly high accuracy and low false alarm with minimum amount of available labelled data.

Anomaly Detection↗

Prediction of inherited genomic susceptibility to 20 common cancer types by a supervised machine-learning method

Prevention and early intervention are the most effective ways of avoiding or minimizing psychological, physical, and financial suffering from cancer. However, such proactive action requires the ability to predict the individual’s susceptibility to cancer with a measure of probability. Of the triad of cancer-causing factors (inherited genomic susceptibility, environmental factors, and lifestyle factors), the inherited genomic component may be derivable from the recent public availability of a large body of whole-genome variation data. However, genome-wide association studies have so far showed limited success in predicting the inherited susceptibility to common cancers. We present here a multiple classification approach for predicting individuals’ inherited genomic susceptibility to acquire the most likely phenotype among a panel of 20 major common cancer types plus 1 “healthy” type by application of a supervised machine-learning method under competing conditions among the cohorts of the 21 types. This approach suggests that, depending on the phenotypes of 5,919 individuals of “white” ethnic population in this study, (i) the portion of the cohort of a cancer type who acquired the observed type due to mostly inherited genomic susceptibility factors ranges from about 33 to 88% (or its corollary: the portion due to mostly environmental and lifestyle factors ranges from 12 to 67%), and (ii) on an individual level, the method also predicts individuals’ inherited genomic susceptibility to acquire the other types ranked with associated probabilities. These probabilities may provide practical information for individuals, heath professionals, and health policymakers related to prevention and/or early intervention of cancer.

59 BASIC BIOLOGICAL SCIENCES↗