Search NASA⌕ Search

SEARCH · Search NASA

Results for “Probabilistic learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Logical Activation Functions v.1.1

SAND2024-01501O Logical Activation Functions software is a PyTorch implementation from the paper, "Logical Activation Functions for Training Arbitrary Probabilistic Boolean Logic." The activation functions approximate logit-space marginalization of probabilistic truth tables from probabilistic interpretations of inputs. They also provide a general methodology to approximate logical relationships between abstract antecedents and consequents for machine learning architectures. They do not target any specific application or use-case. By training probabilistic truth tables, these activation functions can capture more expressive relationships in a neural network than typical elementwise activation functions. This code is only designed for a single compute node with a GPU and is limited to machine learning architectures than can fit within the memory of a single GPU. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Duersch, Jed↗

Operational Probabilistic Tools for Solar Uncertainty (OPTSUN) (Final Project Report for DOE Solar Forecasting II Project)

Increasing levels of solar PV can challenge system operations and may require novel methods to operate the power system reliably and efficiently. Power system operating plans generally use deterministic forecasts, in which the variable energy resources are represented by the expected value for each interval of the decision horizon. Probabilistic forecasts are relatively new but have the potential to address the shortfalls of deterministic forecasts. However, understanding how best to use such forecasts is still a key gap in industry and was the focus of this project. The project had three workstreams. In a forecasting workstream, improvements were made to baseline probabilistic forecasts using a number of new approaches such as machine learning methods and improved input data. In a design workstream, advanced simulation tools used these forecasts to investigate newly proposed reserve determination methods. Lastly, in a demonstration workstream a scheduling management platform (SMP) was developed to leverage probabilistic forecasts in a modular and customizable manner. In order to study the benefits that could be accrued, the project team collaborated with three utility partners (Duke Energy, Southern Company and Hawaiian Electric) to deliver improved probabilistic forecasts for each region and to model each region in case studies using advanced production cost modeling tools. Different methods to determine operating reserve requirements from probabilistic forecasts were developed, simulated, and tested across each region. The benefits of using these newly proposed methods varied by utility, but, in general, using probabilistic forecasts as well as historical data to set the reserve requirements seems to improve reliability related results, with less risk of reserve or supply shortfalls. The cost implications were not always straightforward; in some cases the new methods could show a reduction in expected operating costs, but often the increase in reserves associated with better risk mitigation using probabilistic forecasts could result in an increase in operating costs in the simulations. The SMP tool was developed to process probabilistic forecasts from their initial receipt through to scheduling decisions. This open-source tool consists of several modules for scenario development, reserve requirements calculation, and visualization. The SMP tool was demonstrated to a wide range of operators and stakeholders at all three utilities and further improved based on their feedback. The tool will be available on www.epri.com/optsun. The proposed probabilistic information-based reserve determination approaches have the potential to be implemented by different regions to ensure an economic and reliable power system operation on power systems integrating increasing levels of variable renewable resources. The innovative yet practical methods developed in this project demonstrated tangible benefits from using probabilistic forecasts beyond just study-based assessments to include three unique balancing areas. The demonstrated benefits across the multiple utility environments, are expected to provide system operators in all regions the confidence required and a platform to adopt the new forecasting and operating methods.

14 SOLAR ENERGY↗

A two-step short-term probabilistic wind forecasting methodology based on predictive distribution optimization

With increasing wind penetrations into electric power systems, probabilistic wind forecasting becomes more critical to power system operations because of its capability of quantifying wind uncertainties. In this paper, a two-step probabilistic wind forecasting approach based on pinball loss optimization is developed. First, a multimodel machine learning-based ensemble deterministic forecasting framework is adopted to generate deterministic forecasts. The deterministic forecast is assumed to be the mean value of the predictive distribution at each forecasting time stamp. Then, the optimal unknown parameter (i.e., standard deviation) of the predictive distribution is estimated by a support vector regression surrogate model based on the deterministic forecasts. Lastly, probabilistic forecasts are generated from the predictive distribution. Numerical results of case studies at eight locations show that the developed two-step probabilistic forecasting methodology has improved the pinball loss metric score by up to 35% compared to a baseline quantile regression forecasting model.

17 WIND ENERGY↗

Skluma: An extensible metadata extraction pipeline for disorganized data

To mitigate the effects of high-velocity data expansion and to automate the organization of filesystems and data repositories, we have developed Skluma-a system that automatically processes a target filesystem or repository, extracts content-and context-based metadata, and organizes extracted metadata for subsequent use. Skluma is able to extract diverse metadata, including aggregate values derived from embedded structured data; named entities and latent topics buried within free-text documents; and content encoded in images. Skluma implements an overarching probabilistic pipeline to extract increasingly specific metadata from files. It applies machine learning methods to determine file types, dynamically prioritizes and then executes a suite of metadata extractors, and explores contextual metadata based on relationships among files. The derived metadata, represented in JSON, describes probabilistic knowledge of each file that may be subsequently used for discovery or organization. Skluma's architecture enables it to be deployed both locally and used as an on-demand, cloud-hosted service to create and execute dynamic extraction workflows on massive numbers of files. It is modular and extensible-allowing users to contribute their own specialized metadata extractors. Thus far we have tested Skluma on local filesystems, remote FTP-accessible servers, and publicly-accessible Globus endpoints. We have demonstrated its efficacy by applying it to a scientific environmental data repository of more than 500,000 files. We show that we can extract metadata from those files with modest cloud costs in a few hours.

data swamp↗

LandScan Mosaic

The LandScan program at Oak Ridge National Laboratory (ORNL), in collaboration with the National Geospatial-Intelligence Agency (NGA), continues to deliver the most accurate and up to date global, high resolution gridded population data. Additionally, the latest advancements in the LandScan HD methodology led to reduced latency in development of rapid updates for geopolitical events. With momentum towards reporting more up to date population estimates, feedback from the user community expressed interest in reporting population estimates in ranges - whether to express a level of uncertainty or confirm to leadership and stakeholders the modeled data are estimates. Building upon the need to understand uncertainty or confidence in the modeled data and report ranges at the global scale, LandScan Mosaic was developed. LandScan Mosaic represents the next generation of high-resolution population modeling, building upon the established success of previous LandScan HD iterations. While LandScan HD employed a deterministic big data fusion approach, LandScan Mosaic enhances this methodology by integrating advanced machine learning techniques to impute missing, yet crucial, population model parameters. This advancement allows for probabilistic modeling of building occupancy and population distribution, incorporating uncertainty quantification through Monte Carlo sampling methods. By combining big data fusion with machine learning-driven imputation and stochastic modeling, LandScan Mosaic provides a more comprehensive and robust representation of population dynamics. LandScan Mosaic will be following the in the footsteps of its longstanding counterpart LandScan Global and releasing a global gridded population raster, at the 3-arcsecond resolution. This technical report documents the current stage of development of LandScan Mosaic, detailing the methodologies and data sources behind the modeling. Stakeholders are encouraged to use this document as an authoritative reference for insight into Mosaic’s data development processes. However, readers should note that LandScan Mosaic remains in a late-stage research and development phase, and methodologies and data presented here are subject to refinements ahead of the anticipated global release in Summer 2025. Feedback and inquiries from users and stakeholders are welcomed as we continue to refine and enhance this important population resource.

97 MATHEMATICS AND COMPUTING↗

Wasserstein normalized autoencoder for anomaly detection

A novel anomaly detection algorithm is presented. The Wasserstein normalized autoencoder (WNAE) is a normalized probabilistic model that minimizes the Wasserstein distance between the learned probability distribution—a Boltzmann distribution where the energy is the reconstruction error of the autoencoder (AE)—and the distribution of the training data. This algorithm has been developed and applied to the identification of semivisible jets—conical sprays of visible standard model (SM) particles and invisible dark matter states—with the CMS experiment at the CERN LHC. Trained on jets of particles from simulated SM processes, the WNAE is shown to learn the probability distribution of the input data in a fully unsupervised fashion, such that it effectively identifies new physics jets as anomalies. The model exhibits stable, convergent training and recovers strong classification performance for a wide range of signals against the selected background process, for which a standard AE fails because of outlier reconstruction. In addition, the model improves upon standard normalized autoencoders while remaining fully agnostic to the signal. The WNAE directly tackles the problem of outlier reconstruction, a common failure mode of autoencoders in anomaly detection tasks.

Hayrapetyan, Aram [Yerevan Phys. Inst.]↗

Probabilistic neural networks for fluid flow surrogate modeling and data recovery

We consider the use of probabilistic neural networks for fluid flow surrogate modeling and data recovery. This framework is constructed by assuming that the target variables are sampled from a Gaussian distribution conditioned on the inputs. Consequently, the overall formulation sets up a procedure to predict the hyperparameters of this distribution which are then used to compute an objective function given training data. We demonstrate that this framework has the ability to provide for prediction confidence intervals based on the assumption of a probabilistic posterior, given an appropriate model architecture and adequate training data. The applicability of the present framework to cases with noisy measurements and limited observations is also assessed. To demonstrate the capabilities of this framework, we consider canonical regression problems of fluid dynamics from the viewpoint of reduced-order modeling and spatial data recovery for four canonical data sets. The examples considered in this study arise from (i) the shallow-water equations, (ii) a two-dimensional cylinder flow, (iii) the wake of a NACA0012 airfoil with a Gurney flap, and (iv) the NOAA sea surface temperature data set. Furthermore, the present results indicate that the probabilistic neural network not only produces a machine-learning-based fluid flow surrogate model but also systematically quantifies the uncertainty therein to assist with model interpretability.

42 ENGINEERING↗

Accurate and Fast Anomaly Detection in Additive Composite-Based Manufacturing using Thermal Cameras

Today, large-scale additive manufacturing with plastics and composite materials requires continuous monitoring by experienced staff to prevent, detect and correct anomalous events affecting the performance of the printed part. We address the complexity of this demanding task by designing a camera-based anomaly detection system utilizing probabilistic principal component analysis (PPCA). This is a machine learning technique is trained with thermal images collected during normal operation of the large-scale printer (Cincinnati BAAM). This technique is advantageous for practical applications as there is no need to artificially introduce anomalous conditions into model training. During deployment, we challenge this model by introducing deliberate variations of the extruder speed. We reduce extrusion speed to a lower level, between 70 and 95% of the nominal value to collected test images. Our results show that images are easily identified as anomalous for extruder speeds at or below 85% of the nominal speed, meaning that an anomalous reduction of the material deposition rate can be detected within seconds of its onset. We show that our results are robust to (a) camera-to-camera variability and (b) print-to-print variability.

Pike, John [ORNL]↗

A Flexible Framework for Building Occupancy Detection Using Spatiotemporal Pattern Networks: Preprint

This paper presents a reliable, scalable, and transferable framework to predict occupancy in a building utilizing diverse, multi-modal information. We propose a new methodology for learning-driven occupancy detection built on the concepts of probabilistic graphical modeling and observable Markov chain modeling. To capture the relationship between multi-sensor data and occupancy, we propose this Occ-STPN framework that is flexible to support both multivariate and univariate formulations. While the multivariate Occ-STPN performs feature-level fusion of multiple predictors and occupancy time-series data, the univariate Occ-STPN involves decision fusion of occupancy predictions using individual predictors based on a mutual information weighted fusion scheme. We also propose a new metric to evaluate the performance of occupancy prediction algorithms. Two popular datasets are used to validate our approach and demonstrate that our framework is scalable in terms of the number of information sources (e.g., sensors) as well as it is possible to transfer trained models from one building to another without significant reduction in performance. Reliability of the algorithm is also tested by injecting noise into the datasets.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A Flexible Framework for Building Occupancy Detection Using Spatiotemporal Pattern Networks

This paper presents a reliable, scalable, and transferable framework to predict occupancy in a building utilizing diverse, multi-modal information. We propose a new methodology for learning-driven occupancy detection built on the concepts of probabilistic graphical modeling and observable Markov chain modeling. To capture the relationship between multi-sensor data and occupancy, we propose this Occ-STPN framework that is flexible to support both multivariate and univariate formulations. While the multivariate Occ-STPN performs feature-level fusion of multiple predictors and occupancy time-series data, the univariate Occ-STPN involves decision fusion of occupancy predictions using individual predictors based on a mutual information weighted fusion scheme. We also propose a new metric to evaluate the performance of occupancy prediction algorithms. Two popular datasets are used to validate our approach and demonstrate that our framework is scalable in terms of the number of information sources (e.g., sensors) as well as it is possible to transfer trained models from one building to another without significant reduction in performance. Reliability of the algorithm is also tested by injecting noise into the datasets.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Bayesian Deep Learning Approach to Near-Term Climate Prediction

Since model bias and associated initialization shock are serious shortcomings that reduce prediction skills in state-of-the-art decadal climate prediction efforts, we pursue a complementary machine-learning-based approach to climate prediction. The example problem setting we consider consists of predicting natural variability of the North Atlantic sea surface temperature on the interannual timescale in the pre-industrial control simulation of the Community Earth System Model. While previous works have considered the use of recurrent networks such as convolutional LSTMs and reservoir computing networks in this and other similar problem settings, we currently focus on the use of feedforward convolutional networks. In particular, we find that a feedforward convolutional network with a Densenet architecture is able to outperform a convolutional LSTM in terms of predictive skill. Next, we go on to consider a probabilistic formulation of the same network based on Stein variational gradient descent and find that in addition to providing useful measures of predictive uncertainty, the probabilistic (Bayesian) version improves on its deterministic counterpart in terms of predictive skill. Finally, we characterize the reliability of the ensemble of machine learning models obtained in the probabilistic setting by using analysis tools developed in the context of ensemble numerical weather prediction.

54 ENVIRONMENTAL SCIENCES↗

Scalable statistical inference of photometric redshift via data subsampling

Handling big data has largely been a major bottleneck in traditional statistical models. Consequently, when accurate point prediction is the primary target, machine learning models are often preferred over their statistical counterparts for bigger problems. But full probabilistic statistical models often outperform other models in quantifying uncertainties associated with model predictions. We develop a data-driven statistical modeling framework that combines the uncertainties from an ensemble of statistical models learned on smaller subsets of data carefully chosen to account for imbalances in the input space. We demonstrate this method on a photometric redshift estimation problem in cosmology, which seeks to infer a distribution of the redshift—the stretching effect in observing the light of far-away galaxies—given multivariate color information observed for an object in the sky. Our proposed method performs balanced partitioning, graph-based data subsampling across the partitions, and training of an ensemble of Gaussian process models.

data subsampling↗

Probabilistic Short-Term Wind Forecasting Based on Pinball Loss Optimization: Preprint

Probabilistic wind power forecasts that quantify the uncertainty in wind output have the potential to aid in the economic grid integration of wind power at large penetration levels. In this paper, a novel probabilistic wind forecasting approach based on pinball loss optimization is proposed, in conjunction with a multi-model machine learning based ensemble deterministic forecasting framework. By assuming the pointforecasted value as the mean at each point, one unknown parameter (i.e., standard deviation) of a predictive distribution at each forecasting point is determined by minimizing the pinball loss. A surrogate model is developed to represent the unknown distribution parameter as a function of deterministic forecasts. This surrogate model can be used together with deterministic forecasts to predict the unknown distribution parameter and thereby generate probabilistic forecasts. Numerical results of case studies show that the proposed method has improved the pinball loss by up to 35% compared to a baseline quantile regression forecasting model.

17 WIND ENERGY↗

Probabilistic Short-Term Wind Forecasting Based on Pinball Loss Optimization

Probabilistic wind power forecasts that quantify the uncertainty in wind output have the potential to aid in the economic grid integration of wind power at large penetration levels. In this paper, a novel probabilistic wind forecasting approach based on pinball loss optimization is proposed, in conjunction with a multi-model machine learning based ensemble deterministic forecasting framework. By assuming the point-forecasted value as the mean at each point, one unknown parameter (i.e., standard deviation) of a predictive distribution at each forecasting point is determined by minimizing the pinball loss. A surrogate model is developed to represent the unknown distribution parameter as a function of deterministic forecasts. This surrogate model can be used together with deterministic forecasts to predict the unknown distribution parameter and thereby generate probabilistic forecasts. Numerical results of case studies show that the proposed method has improved the pinball loss by up to 35% compared to a baseline quantile regression forecasting model.

17 WIND ENERGY↗

Industrial Internet-of-Things & Data Analytics for Nuclear Power & Safeguards

Data analytics applied to nuclear power operations and nuclear safeguards is in a nascent state, yet some significant initial efforts are being undertaken by industry and academia. This report highlights our findings as to the current state-of-the-art of such efforts, in particular considering the Industrial Internet-of-Things aspect of this work, as well as an investigation into the utility of machine learning tools being developed for other industries. Blockchain applications were also studied. A consideration was undertaken into how to apply data analytics and machine learning to nuclear power and safeguards within the realm of Probabilistic Risk Assessments (PRAs), predictive maintenance & edge analytics, and proprietary data sharing.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Power System Feature-Based Event Classification by Means of Multiple PMU Data

Abstract—Phasor Measurement Units (PMUs) provide time synchronized measurements across the power grid, enabling data driven event detection and classification for enhanced system monitoring and situational awareness. However, variations in event duration, spatial extent, and severity, along with coincident events, pose challenges for conventional classification models that require fixed-size inputs. This paper presents a feature-based framework that aggregates diverse attributes from all available PMUs for each event into a fixed-length vector, facilitating the application of standard machine learning classifiers, including Random Forest, XGBoost, and Multilayer Perceptron. A probabilistic post-processing scheme is further introduced to enable multi-label classification in the presence of overlapping events. Experiments using real-world PMU data demonstrate that the Random Forest model achieves 95% accuracy, while the proposed post-processing method yields an additional 3% improvement.

Nematirad, Reza↗

Hybrid stochastic synapses enabled by scaled ferroelectric field-effect transistors

Achieving brain-like density and performance in neuromorphic computers necessitates scaling down the size of nanodevices emulating neuro-synaptic functionalities. However, scaling nanodevices results in reduction of programming resolution and emergence of stochastic non-idealities. While prior work has mainly focused on binary transitions, in this work, we leverage the stochastic switching of a three-state ferroelectric field-effect transistor to implement a long-term and short-term two-tier stochastic synaptic memory with a single device. Experimental measurements are performed on a scaled 28 nm high-k metal gate technology-based device to develop a probabilistic model of the hybrid stochastic synapse. In addition to the advantage of ultra-low programming energies afforded by scaling, our hardware–algorithm co-design analysis reveals the efficacy of the two-tier memory in comparison to binary stochastic synapses in on-chip learning tasks—paving the way for algorithms exploiting multi-state devices with probabilistic transitions beyond deterministic ones.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Particle Markov Chain Monte Carlo Approach to Inference in Transient Surface Kinetics

Here, in this work, we develop a novel Bayesian approach to study the adsorption and desorption of CO onto a Pd(111) surface, a process of great importance in natural sciences. The motivation for this work comes from the recent availability of time-resolved infrared spectroscopy data and the need for model interpretability and uncertainty quantification in chemical processes. The objective is to learn the relevant parameters that characterize the process: coverage with time, rate constants, activation energies, and pre-exponential factors. Our approach consists of three main schemes: (i) a problem design and probabilistic model for the whole system, (ii) a particle Markov chain Monte Carlo sampler to learn the hidden coverages and rate constant parameters, and (iii) two Bayesian formulations to infer the activation energies and pre-exponential factors. The flexibility of the Bayesian framework allows for uncertainty quantification where possible and integration of mathematical constraints in the model to reflect the system physically. We found that our results for the activation energies and pre-exponential factor are in agreement with those reported in the experimental literature, independently, and we provide discussions on the advantages and disadvantages as well as applicability to other systems.

36 MATERIALS SCIENCE↗