Search NASA⌕ Search

SEARCH · Search NASA

Results for “statistical learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Chaconne: A Statistical Approach to Nonlocal Compression for Supervised Learning, Semi-Supervised Learning, and Anomaly Detection

This project developed a novel statistical understanding of compression analytics (CA), which has challenged and clarified some core assumptions about CA, and enabled the development of novel techniques that address vital challenges of national security. Specifically, this project has yielded the development of novel capabilities including 1. Principled metrics for model selection in CA, 2. Techniques for deriving/applying optimal classification rules and decision theory to supervised CA, including how to properly handle class imbalance and differing costs of misclassification, 3. Two techniques for handling nonlocal information in CA, 4. A novel technique for unsupervised CA that is agnostic with regard to the underlying compression algorithm, 5. A framework for semisupervised CA when a small number of labels are known in an otherwise large unlabeled dataset. 6. The academic alliance component of this project has focused on the development of a novel exemplar-based Bayesian technique for estimating variable length Markov models (closely related to PPM [prediction by partial matching] compression techniques). We have developed examples illustrating the application of our work to text, video, genetic sequences, and unstructured cybersecurity log files.

99 GENERAL AND MISCELLANEOUS↗

Extracting Knowledge from Data through Catalysis Informatics

Catalysis informatics is a distinct sub-field that lies at the intersection of cheminformatics and materials informatics, but with unique challenges arising from the dynamic, surface-sensitive, multi-scale nature of heterogeneous catalysis. The ideas behind catalysis informatics can be traced back decades, but the field is only recently emerging due to advances in data infrastructure, statistics, machine learning, and computational methods. In this work we review the field from early works on expert systems and knowledge engines to more recent approaches utilizing machine-learning and uncertainty quantification. The data-information-knowledge hierarchy is introduced and used to classify various developments. The chemical master equation and micro-kinetic models are proposed as a quantitative representation of catalysis knowledge, which can be used to generate explanative and predictive hypotheses for the understanding and discovery of catalytic materials. We discuss future prospects for the field, including improved quantitative coupling of experiment/theory, advanced micro-kinetic models, and the development of open-source software tools. As a result, integration of existing chemical and physical models with emerging statistical and computational tools presents a promising route toward the automated design, discovery, and optimization of heterogeneous catalytic processes.

97 MATHEMATICS AND COMPUTING↗

Local Bayesian Dirichlet mixing of imperfect models

Abstract To improve the predictability of complex computational models in the experimentally-unknown domains, we propose a Bayesian statistical machine learning framework utilizing the Dirichlet distribution that combines results of several imperfect models. This framework can be viewed as an extension of Bayesian stacking. To illustrate the method, we study the ability of Bayesian model averaging and mixing techniques to mine nuclear masses. We show that the global and local mixtures of models reach excellent performance on both prediction accuracy and uncertainty quantification and are preferable to classical Bayesian model averaging. Additionally, our statistical analysis indicates that improving model predictions through mixing rather than mixing of corrected models leads to more robust extrapolations.

97 MATHEMATICS AND COMPUTING↗

Veridical data science

Building and expanding on principles of statistics, machine learning, and scientific inquiry, we propose the predictability, computability, and stability (PCS) framework for veridical data science. Our framework, composed of both a workflow and documentation, aims to provide responsible, reliable, reproducible, and transparent results across the data science life cycle. The PCS workflow uses predictability as a reality check and considers the importance of computation in data collection/storage and algorithm design. It augments predictability and computability with an overarching stability principle. Stability expands on statistical uncertainty considerations to assess how human judgment calls impact data results through data and model/algorithm perturbations. As part of the PCS workflow, we develop PCS inference procedures, namely PCS perturbation intervals and PCS hypothesis testing, to investigate the stability of data results relative to problem formulation, data cleaning, modeling decisions, and interpretations. We illustrate PCS inference through neuroscience and genomics projects of our own and others. Moreover, we demonstrate its favorable performance over existing methods in terms of receiver operating characteristic (ROC) curves in high-dimensional, sparse linear model simulations, including a wide range of misspecified models. Finally, we propose PCS documentation based on R Markdown or Jupyter Notebook, with publicly available, reproducible codes and narratives to back up human choices made throughout an analysis. The PCS workflow and documentation are demonstrated in a genomics case study available on Zenodo.

97 MATHEMATICS AND COMPUTING↗

Recurrent convolutional neural networks for modeling nonadiabatic dynamics of quantum-classical systems

Recurrent neural networks (RNNs) have recently been extensively applied to model the time evolution in fluid dynamics, weather predictions, and even chaotic systems due to their ability to capture temporal dependencies and sequential patterns in data. Here we present an RNN model based on convolutional neural networks for modeling the nonlinear nonadiabatic dynamics of hybrid quantum-classical systems. The dynamical evolution of the hybrid systems is governed by equations of motion for classical degrees of freedom and von Neumann equation for electrons. The Physics-Aware Recurrent Convolution (PARC) neural network structure incorporates a differentiator-integrator architecture that inductively models the spatiotemporal dynamics of generic physical systems. Here, we apply our RNN approach to learn the space-time evolution of a one-dimensional semiclassical Holstein model after an interaction quench. For shallow quenches (small changes in electron-lattice coupling), the deterministic dynamics can be accurately captured using a single-CNN-based recurrent network. In contrast, deep quenches induce chaotic evolution, making long-term trajectory prediction significantly more challenging. Nonetheless, we demonstrate that the PARC-CNN architecture can effectively learn the statistical climate of the Holstein model under deep-quench conditions.

Holstein model↗

Classification of Unintended Radiated Emissions in a Multi-Device Environment

Here, unintended radiated emissions (URE) from electronic devices are conducted on to the power infrastructure and can be collected and analyzed for non-intrusive load monitoring (NILM) applications. Dimensionally aligned signal projections (DASP) were previously introduced as specialized signal transforms to enable device classification using machine learning on statistical features derived from the DASP transforms. In this work, we explore multi-device classification which requires more sophisticated methods owing to the complexity of the feature space such as large dynamic range, non-contiguous features, over-lapping features, and nonlinear interactions of features. In particular, we introduce an additional DASP algorithm for increased resolution and sensitivity to intermodulation products and examine the utility of convolutional neural networks (CNNs) to extract and learn features directly from DASP images for classification of single and multiple device URE captures.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Stack filter classifiers

Just as linear models generalize the sample mean and weighted average, weighted order statistic models generalize the sample median and weighted median. This analogy can be continued informally to generalized additive modeels in the case of the mean, and Stack Filters in the case of the median. Both of these model classes have been extensively studied for signal and image processing but it is surprising to find that for pattern classification, their treatment has been significantly one sided. Generalized additive models are now a major tool in pattern classification and many different learning algorithms have been developed to fit model parameters to finite data. However Stack Filters remain largely confined to signal and image processing and learning algorithms for classification are yet to be seen. This paper is a step towards Stack Filter Classifiers and it shows that the approach is interesting from both a theoretical and a practical perspective.

99 GENERAL AND MISCELLANEOUS↗

Creating Feature Representations of Antibody-Antigen Complexes for Fast Binding Prediction with Machine Learning

Antibody-antigen interactions underlie the function of the immune system. Numerous computational tools exist that aim to predict important quantities associated with antibodyantigen interactions, such as binding energy. Efficient and accurate prediction of such quantities would greatly accelerate vaccine (antigen) and therapeutic antibody design. Current methods and tools, however, rely on varying degrees of computationally expensive steps that limit their use on problems that require many predictions, and moreover, no single method produces consistently accurate estimates. In this project, we determined the feasibility of creating feature representations that concisely describe interacting pairs of antibody and antigen as rapidly computable numerical vectors, given the antibody-antigen co-complex. Such a featurization will allow the use of a statistical machine learning model that jointly models the results of the disparate existing computational tools available. In turn, this will greatly enhance their combined predictive performance. We developed a feature representation consisting of 86 descriptors of the antigen-antibody interface which is then reduced into a lower dimensional representation using a neural network dimensionality reduction step. We have demonstrated that this feature representation serves its intended purpose of rapidly describing antigen-antibody pairs for the task of binding strength prediction. While this feasibility study has proven that such a feature representation is feasible, additional research is likely improve its ability to support various tasks, including binding strength prediction.

59 BASIC BIOLOGICAL SCIENCES↗

Metallic Metal–Organic Frameworks Predicted by the Combination of Machine Learning Methods and Ab Initio Calculations

Emerging applications of metal–organic frameworks (MOFs) in electronic devices will benefit from the design and synthesis of intrinsically, highly electronically conductive MOFs. However, very few are known to exist. It is a challenging task to search for electronically conductive MOFs within the tens of thousands of reported MOF structures. Using a new strategy (i.e., transfer learning) of combining machine learning techniques, statistical multivoting, and ab initio calculations, we screened 2932 MOFs and identified 6 MOF crystal structures that are metallic at the level of semilocal DFT band theory: Mn 2 [Re 6 X 8 (CN) 6 ] 4 (X = S, Se,Te), Mn[Re 3 Te 4 (CN) 3 ], Hg[SCN] 4 Co[NCS] 4 , and CdC 4 . Five of these structures have been synthesized and reported in the literature, but their electrical characterization has not been reported. In conclusion, our work demonstrates the potential power of machine learning in materials science to aid in down-selecting from large numbers of potential candidates and provides the information and guidance to accelerate the discovery of novel advanced materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Improving Seasonal Forecast Using Probabilistic Deep Learning

The path toward realizing the potential of seasonal forecasting and its socioeconomic benefits relies on improving general circulation model (GCM) based dynamical forecast systems. To improve dynamical seasonal forecasts, it is crucial to set up forecast benchmarks, and clarify forecast limitations posed by model initialization errors, formulation deficiencies, and internal climate variability. With huge costs in generating large forecast ensembles, and limited observations for forecast verification, the seasonal forecast benchmarking and diagnosing task proves challenging. Here, we develop a probabilistic deep learning-based statistical forecast methodology, drawing on a wealth of climate simulations to enhance seasonal forecast capability and forecast diagnosis. By explicitly modeling the internal climate variability and GCM formulation differences, the proposed Conditional Generative Forecasting (CGF) methodology enables bypassing crucial barriers in dynamical forecast, and offers a top-down viewpoint to examine how complicated GCMs encode the seasonal predictability information. We apply the CGF methodology for global seasonal forecast of precipitation and 2 m air temperature, based on a unique data set consisting 52,201 years of climate simulation. Results show that the CGF methodology can faithfully represent the seasonal predictability information encoded in GCMs. We successfully apply this learned relationship in real-world seasonal forecast, achieving competitive performance compared to dynamical forecasts. Using this CGF as benchmark, we reveal the impact of insufficient forecast spread sampling that limits the skill of the considered dynamical forecast system. Finally, we introduce different strategies for composing ensembles using the CGF methodology, highlighting the potential for leveraging the strengths of multiple GCMs to achieve advantgeous seasonal forecast.

54 ENVIRONMENTAL SCIENCES↗

Predicting RNA structure and dynamics with deep learning and solution scattering

Advanced deep learning and statistical methods can predict structural models for RNA molecules. However, RNAs are flexible, and it remains difficult to describe their macromolecular conformations in solutions where varying conditions can induce conformational changes. Small-angle x-ray scattering (SAXS) in solution is an efficient technique to validate structural predictions by comparing the experimental SAXS profile with those calculated from predicted structures. There are two main challenges in comparing SAXS profiles to RNA structures: the absence of cations essential for stability and charge neutralization in predicted structures and the inadequacy of a single structure to represent RNA’s conformational plasticity. We introduce a solution conformation predictor for RNA (SCOPER) to address these challenges. This pipeline integrates kinematics-based conformational sampling with the innovative deep learning model, IonNet, designed for predicting Mg 2+ ion binding sites. Validated through benchmarking against 14 experimental data sets, SCOPER significantly improved the quality of SAXS profile fits by including Mg 2+ ions and sampling of conformational plasticity. We observe that an increased content of monovalent and bivalent ions leads to decreased RNA plasticity. Therefore, carefully adjusting the plasticity and ion density is crucial to avoid overfitting experimental SAXS data. SCOPER is an efficient tool for accurately validating the solution state of RNAs given an initial, sufficiently accurate structure and provides the corrected atomistic model, including ions.

59 BASIC BIOLOGICAL SCIENCES↗

Optimization of carbon capture systems using surrogate models of simulated processes.

With increasing demand placed on power generation plants to reduce carbon dioxide (CO2) emissions, processes to separate and capture CO2 for eventual sequestration are highly sought after. Carbon capture processes impart a parasitic load on the power plants; it is estimated that this would increase the cost of electricity from existing pulverized coal plants anywhere from 71-85 percent [1]. The National Energy and Technology Lab (NETL) is working to lower this to below a 30 percent increase. To reach this goal, work is being done not only to accurately simulate these processes, but also to leverage those accurate and detailed simulations to design optimal carbon capture processes. The major challenges include the lack of accurate algebraic models of the processes, computationally costly simulations, and insufficiently robust simulations. The first challenge bars the use of provable derivative-based optimization algorithms. The latter two can either lead to difficult or impossible direct derivative-free optimization. To overcome these difficulties, we take a more indirect method to solving this problem by, first, generating an accurate set of algebraic surrogate models from the simulation then using derivative-based solvers to optimize the surrogate models. We developed a method that uses derivative-based and derivative-free optimization alongside machine learning and statistical techniques to generate the set of low-complexity surrogate models using data sampled from detailed simulations. The models are validated and improved through the use of derivative-free solvers to adaptively sample new simulation points. The resulting surrogate models can then be used in a superstructure-based process synthesis and solved using derivative-based methods to optimize carbon capture processes.

01 COAL, LIGNITE, AND PEAT↗

A deep generative model enables automated structure elucidation of novel psychoactive substances

Over the past decade, the illicit drug market has been reshaped by the proliferation of clandestinely produced designer drugs. These agents, referred to as new psychoactive substances (NPSs), are designed to mimic the physiological actions of better-known drugs of abuse while skirting drug control laws. The public health burden of NPS abuse obliges toxicological, police and customs laboratories to screen for them in law enforcement seizures and biological samples. However, the identification of emerging NPSs is challenging due to the chemical diversity of these substances and the fleeting nature of their appearance on the illicit market. Here, in this study, we present DarkNPS, a deep learning-enabled approach to automatically elucidate the structures of unidentified designer drugs using only mass spectrometric data. Our method employs a deep generative model to learn a statistical probability distribution over unobserved structures, which we term the structural prior. We show that the structural prior allows DarkNPS to elucidate the exact chemical structure of an unidentified NPS with an accuracy of 51% and a top-10 accuracy of 86%. Our generative approach has the potential to enable de novo structure elucidation for other types of small molecules that are routinely analysed by mass spectrometry.

Cheminformatics↗

Data-driven occupant-behavior analytics for residential buildings

Many advances have been made in building technology to help save energy, but influencing the behavior of the occupants is still necessary to achieve low-energy use targets. One of the most practical ways to influence and change occupant behaviors is through incentives. Developing incentives for energy-saving and quantifying the impact of occupant behaviors are both active areas of research. Here, we propose a data analytics framework for detecting changes in occupant behaviors, which will help build an analytics feedback loop from behavior impact to incentive design. The framework has two major parts. The first forecasts energy consumption for each occupant, while the second determines a probability distribution for changes in energy consumption. The parts are interchangeable with other existing machine learning and statistical methods. A specific instantiation of the framework, using kernel ridge-regression for forecasting and k-means to find an empirical behavior distribution, is described in detail. An HVAC use-case with 5 different incentivized behaviors is used as an example to show that the framework can detect behavior changes induced by incentives. Furthermore, we show that some simpler behavior-change detection methods do not work, further justifying the use of advanced analytics.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Model orthogonalization and Bayesian forecast mixing via principal component analysis

One can improve predictability in the unknown domain by combining forecasts of imperfect complex computational models using a Bayesian statistical machine learning framework. In many cases, however, the models used in the mixing process are similar. In addition to contaminating the model space, the existence of such similar, or even redundant, models during the multimodeling process can result in misinterpretation of results and deterioration of predictive performance. In this paper we describe a method based on the principal component analysis that eliminates model redundancy. We show that by adding model orthogonalization to the proposed Bayesian model combination framework, one can arrive at better prediction accuracy and reach excellent uncertainty quantification performance.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A computational methodology for learning low-complexity surrogate models of process from experiments or simulations. (Paper 679a)

Costly and/or insufficiently robust simulations or experiments can often pose difficulties when their use extends well beyond a single evaluation. This is case with the numerous evaluations of uncertainty quantification, when an algebraic model is needed for optimization, as well as numerous other areas. To overcome these difficulties, we generate an accurate set of algebraic surrogate models of disaggregated process blocks of the experiment or simulation. We developed a method that uses derivative-based and derivative-free optimization alongside machine learning and statistical techniques to generate the set of surrogate models using data sampled from experiments or detailed simulations. Our method begins by building a low-complexity surrogate model for each block from an initial sample set. The model is built using a best subset technique that leverages a mixed-integer linear problem formulation to allow for very large initial basis sets. The models are then tested, exploited, and improved through the use of derivative-free solvers to adaptively sample new simulation or experimental points. The sets of surrogate models from each disaggregated process block are then combined with heat and mass balances around each disaggregated block to generate a full algebraic model of the process. The full model can be used for cheap and accurate evaluations of the original simulation or experiment or combined with design specifications and an objective for nonlinear optimization.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Neural networks for parameter estimation in intractable models

The goal is to use deep learning models to estimate parameters in statistical models when standard likelihood estimation methods are computationally infeasible. For instance, inference for max-stable processes is exceptionally challenging even with small datasets, but simulation is straightforward. Data from model simulations are used to train deep neural networks and learn statistical parameters from max-stable models. The proposed neural network-based method provides a competitive alternative to current approaches, as demonstrated by considerable accuracy and computational time improvements. Finally, it serves as a proof of concept for deep learning in statistical parameter estimation and can be extended to other estimation problems.

97 MATHEMATICS AND COMPUTING↗

Density estimation via measure transport: Outlook for applications in the biological sciences

Abstract One among several advantages of measure transport methods is that they allow or a unified framework for processing and analysis of data distributed according to a wide class of probability measures. Within this context, we present results from computational studies aimed at assessing the potential of measure transport techniques, specifically, the use of triangular transport maps, as part of a workflow intended to support research in the biological sciences. Scenarios characterized by the availability of limited amount of sample data, which are common in domains such as radiation biology, are of particular interest. We find that when estimating a distribution density function given limited amount of sample data, adaptive transport maps are advantageous. In particular, statistics gathered from computing series of adaptive transport maps, trained on a series of randomly chosen subsets of the set of available data samples, leads to uncovering information hidden in the data. As a result, in the radiation biology application considered here, this approach provides a tool for generating hypotheses about gene relationships and their dynamics under radiation exposure.

gene expression data↗