Logical activation functions for training arbitrary probabilistic Boolean operations
Not Available
96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
SEARCH · Search NASA
Results for “Probabilistic training”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
DeepONet-grid-UQ: A trustworthy deep operator framework for predicting the power grid’s post-fault trajectories
This paper proposes a novel data-driven method for the reliable prediction of the power grid’s post-fault trajectories, i.e., the power grid’s dynamic response after a disturbance or fault. Here, the proposed method is based on the recently proposed concept of Deep Operator Networks (DeepONets). Unlike traditional neural networks that learn to approximate functions, DeepONets are designed to approximate nonlinear operators, i.e., mappings between infinite-dimensional spaces. Under this operator framework, we design a novel and efficient DeepONet that (i) takes as inputs the trajectories collected before and during the fault and (ii) outputs the predicted post-fault trajectories. In addition, we endow our method with the much-needed ability to balance efficiency with reliable/trustworthy predictions via uncertainty quantification. To this end, we propose and compare two novel methods that enable quantifying the predictive uncertainty. First, we propose a Bayesian DeepONet (B-DeepONet) that uses stochastic gradient Hamiltonian Monte-Carlo to sample from the posterior distribution of the DeepONet trainable parameters. Then, we design a Probabilistic DeepONet (Prob-DeepONet) that uses a probabilistic training strategy to enable quantifying uncertainty at virtually no extra computational cost. Finally, we validate the proposed methods’ predictive power and uncertainty quantification capability using the New York-New England power grid model.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Enriching Load Data Using Micro-PMUs and Smart Meters
In modern distribution systems, load uncertainty can be fully captured by micro-PMUs, which can record high-resolution data; however, in practice, micro-PMUs are installed at limited locations in distribution networks due to budgetary constraints. In contrast, smart meters are widely deployed but can only measure relatively low-resolution energy consumption, which cannot sufficiently reflect the actual instantaneous load volatility within each sampling interval. In this paper, we have proposed a novel approach for enriching load data for service transformers that only have low-resolution smart meters. The key to our approach is to statistically recover the high-resolution load data, which is masked by the low-resolution data, using trained probabilistic models of service transformers that have both high- and low-resolution data sources, i.e., micro-PMUs and smart meters. The overall framework consists of two steps: first, for the transformers with micro-PMUs, a Gaussian Process is leveraged to capture the relationship between the maximum/minimum load and average load within each low-resolution sampling interval of smart meters; a Markov chain model is employed to characterize the transition probability of known high-resolution load. Next, the trained models are used as teachers for the transformers with only smart meters to decompose known low-resolution load data into targeted high-resolution load data. The enriched data can recover instantaneous load uncertainty and significantly enhance distribution system observability and situational awareness. Here, we have verified the proposed approach using real high- and low-resolution load data.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Logical Activation Functions v.1.1
SAND2024-01501O Logical Activation Functions software is a PyTorch implementation from the paper, "Logical Activation Functions for Training Arbitrary Probabilistic Boolean Logic." The activation functions approximate logit-space marginalization of probabilistic truth tables from probabilistic interpretations of inputs. They also provide a general methodology to approximate logical relationships between abstract antecedents and consequents for machine learning architectures. They do not target any specific application or use-case. By training probabilistic truth tables, these activation functions can capture more expressive relationships in a neural network than typical elementwise activation functions. This code is only designed for a single compute node with a GPU and is limited to machine learning architectures than can fit within the memory of a single GPU. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
Duersch, Jed↗
Bayesian averaging for ground state masses of atomic nuclei in a Machine Learning approach
We present global predictions of the ground state mass of atomic nuclei based on a novel Machine Learning algorithm. We combine precision nuclear experimental measurements together with theoretical predictions of unmeasured nuclei. This hybrid data set is used to train a probabilistic neural network. In addition to training on this data, a physics-based loss function is employed to help refine the solutions. The resultant Bayesian averaged predictions have excellent performance compared to the testing set and come with well-quantified uncertainties which are critical for contemporary scientific applications. We assess extrapolations of the model’s predictions and estimate the growth of uncertainties in the region far from measurements.
74 ATOMIC AND MOLECULAR PHYSICS↗
Improving the Quasi‐Biennial Oscillation via a Surrogate‐Accelerated Multi‐Objective Optimization
Accurate simulation of the quasi-biennial oscillation (QBO) is challenging due to uncertainties in representing convectively generated gravity waves. We develop an end-to-end uncertainty quantification workflow that calibrates these gravity wave processes in E3SM for a realistic QBO. Central to our approach is a domain knowledge-informed, compressed representation of high-dimensional spatio-temporal wind fields. By employing a parsimonious statistical model that learns the fundamental frequency from complex observations, we extract interpretable and physically meaningful quantities capturing key attributes. Building on this, we train a probabilistic surrogate model that approximates the fundamental characteristics of the QBO as functions of critical physics parameters governing gravity wave generation. Leveraging the Karhunen–Loève decomposition, our surrogate efficiently represents these characteristics as a set of orthogonal features, capturing cross-correlations among multiple physics quantities evaluated at different pressure levels and enabling rapid surrogate-based inference at a fraction of the computational cost of full-scale simulations. Finally, we analyze the inverse problem using a multi-objective approach. Our study reveals a tension between amplitude and period that constrains the QBO representation, precluding a single optimal solution. To navigate this, we quantify the bi-criteria trade-off and generate a set of Pareto optimal parameter values that balance the conflicting objectives. This integrated workflow improves the fidelity of QBO simulations and offers a versatile template for uncertainty quantification in complex geophysical models.
54 ENVIRONMENTAL SCIENCES↗
Learning to simulate high energy particle collisions from unlabeled data
In many scientific fields which rely on statistical inference, simulations are often used to map from theoretical models to experimental data, allowing scientists to test model predictions against experimental results. Experimental data is often reconstructed from indirect measurements causing the aggregate transformation from theoretical models to experimental data to be poorly-described analytically. Instead, numerical simulations are used at great computational cost. We introduce Optimal-Transport-based Unfolding and Simulation (OTUS), a fast simulator based on unsupervised machine-learning that is capable of predicting experimental data from theoretical models. Without the aid of current simulation information, OTUS trains a probabilistic autoencoder to transform directly between theoretical models and experimental data. Identifying the probabilistic autoencoder’s latent space with the space of theoretical models causes the decoder network to become a fast, predictive simulator with the potential to replace current, computationally-costly simulators. Here, we provide proof-of-principle results on two particle physics examples, Z-boson and top-quark decays, but stress that OTUS can be widely applied to other fields.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
A Probabilistic Autoencoder for Type Ia Supernova Spectral Time Series
We construct a physically parameterized probabilistic autoencoder (PAE) to learn the intrinsic diversity of Type Ia supernovae (SNe Ia) from a sparse set of spectral time series. The PAE is a two-stage generative model, composed of an autoencoder that is interpreted probabilistically after training using a normalizing flow. We demonstrate that the PAE learns a low-dimensional latent space that captures the nonlinear range of features that exists within the population and can accurately model the spectral evolution of SNe Ia across the full range of wavelength and observation times directly from the data. By introducing a correlation penalty term and multistage training setup alongside our physically parameterized network, we show that intrinsic and extrinsic modes of variability can be separated during training, removing the need for the additional models to perform magnitude standardization. We then use our PAE in a number of downstream tasks on SNe Ia for increasingly precise cosmological analyses, including the automatic detection of SN outliers, the generation of samples consistent with the data distribution, and solving the inverse problem in the presence of noisy and incomplete data to constrain cosmological distance measurements. We find that the optimal number of intrinsic model parameters appears to be three, in line with previous studies, and show that we can standardize our test sample of SNe Ia with an rms of 0.091 ± 0.010 mag, which corresponds to 0.074 ± 0.010 mag if peculiar velocity contributions are removed.
79 ASTRONOMY AND ASTROPHYSICS↗
Particle Filter Based Inference Testing
The primary intent of PAR-FIT (Particle Filter based Inference Testing) is to provide hard inductive evidence that a machine learning model is capable and proven for an individual test input. By examining training data used to form the underlying model functional correlation, an estimate of the reliability that a model will make the correct prediction can be made. The Sequential Probability Ratio Test is used to derive a qualitative evaluation for reliability based on hypothesis testing. The PAR-FIT framework achieves this by implementing a particle filter and the sequential probability ratio test algorithms on the machine learning model training data to determine relevancy of new individual test samples to the training dataset. The kernel function evaluates the local proximity and density of training data used to derive a prediction outcome. Particles are used to probabilistically determine which training data to evaluate for proximity. For test samples that are within a close proximity to and surrounded by multiple training data points, the evaluated reliability of the prediction is high. For test samples that are anomalies not represented by the training dataset, in low density data clusters, or are far from existing data points, the evaluated reliability is low as insufficient training evidence exists to suggest the model is capable of making the correct prediction. Sequential Probability Ratio Test is further used to determine when a hypothesis on whether a signal can be rejected or accepted for use. The ratio test collects sequence information from the particle filter to test whether the signal is anomalous or normal via hypothesis testing of the underlying distributions.
Chen, Edward [Idaho National Laboratory (INL), Ida↗
Trajectory design via unsupervised probabilistic learning on optimal manifolds
Abstract This article illustrates the use of unsupervised probabilistic learning techniques for the analysis of planetary reentry trajectories. A three-degree-of-freedom model was employed to generate optimal trajectories that comprise the training datasets. The algorithm first extracts the intrinsic structure in the data via a diffusion map approach. We find that data resides on manifolds of much lower dimensionality compared to the high-dimensional state space that describes each trajectory. Using the diffusion coordinates on the graph of training samples, the probabilistic framework subsequently augments the original data with samples that are statistically consistent with the original set. The augmented samples are then used to construct conditional statistics that are ultimately assembled in a path planning algorithm. In this framework, the controls are determined stage by stage during the flight to adapt to changing mission objectives in real-time.
42 ENGINEERING↗
Trajectory Optimization via Unsupervised Probabilistic Learning On Manifolds
This report investigates the use of unsupervised probabilistic learning techniques for the analysis of hypersonic trajectories. The algorithm first extracts the intrinsic structure in the data via a diffusion map approach. Using the diffusion coordinates on the graph of training samples, the probabilistic framework augments the original data with samples that are statistically consistent with the original set. The augmented samples are then used to construct conditional statistics that are ultimately assembled in a path-planing algorithm. In this framework the controls are determined stage by stage during the flight to adapt to changing mission objectives in real-time. A 3DOF model was employed to generate optimal hypersonic trajectories that comprise the training datasets. The diffusion map algorithm identfied that data resides on manifolds of much lower dimensionality compared to the high-dimensional state space that describes each trajectory. In addition to the path-planing worflow we also propose an algorithm that utilizes the diffusion map coordinates along the manifold to label and possibly remove outlier samples from the training data. This algorithm can be used to both identify edge cases for further analysis as well as to remove them from the training set to create a more robust set of samples to be used for the path-planing process.
42 ENGINEERING↗
Synthesizing realistic sand assemblies with denoising diffusion in latent space
Abstract The shapes and morphological features of grains in sand assemblies have far‐reaching implications in many engineering applications, such as geotechnical engineering, computer animations, petroleum engineering, and concentrated solar power. Yet, our understanding of the influence of grain geometries on macroscopic response is often only qualitative, due to the limited availability of high‐quality 3D grain geometry data. In this paper, we introduce a denoising diffusion algorithm that uses a set of point clouds collected from the surface of individual sand grains to generate grains in the latent space. By employing a point cloud autoencoder, the three‐dimensional point cloud structures of sand grains are first encoded into a lower‐dimensional latent space. A generative denoising diffusion probabilistic model is trained to produce synthetic sand that maximizes the log‐likelihood of the generated samples belonging to the original data distribution measured by a Kullback‐Leibler divergence. Numerical experiments suggest that the proposed method is capable of generating realistic grains with morphology, shapes and sizes consistent with the training data inferred from an F50 sand database. We then use a rigid contact dynamic simulator to pour the synthetic sand in a confined volume to form granular assemblies in a static equilibrium state with targeted distribution properties. To ensure third‐party validation, 50,000 synthetic sand grains and the 1542 real synchrotron microcomputed tomography (SMT) scans of the F50 sand, as well as the granular assemblies composed of synthetic sand grains are made available in an open‐source repository.
Vlassis, Nikolaos N.↗
Generative diffusion model surrogates for mechanistic agent-based biological models
Mechanistic, multicellular, agent-based models are commonly used to investigate tissue, organ, and organism-scale biology at single-cell resolution. The Cellular-Potts Model (CPM) is a powerful and popular framework for developing and interrogating these models. CPMs become computationally expensive at large space- and time- scales making application and investigation of developed models difficult. Surrogate models may allow for the accelerated evaluation of CPMs of complex biological systems. However, the stochastic nature of these models means each set of parameters may give rise to different model configurations, complicating surrogate model development. In this work, we leverage denoising diffusion probabilistic models (DDPMs) to train a generative AI surrogate of a CPM used to investigate in vitro vasculogenesis. We describe the use of an image classifier to learn the characteristics that define unique areas of a 2-dimensional parameter space. We then apply this classifier to aid in surrogate model selection and verification. Our CPM model surrogate generates model configurations 20,000 timesteps ahead of a reference configuration and demonstrates approximately a 22x reduction in computational time as compared to native code execution. Our work represents a step towards the implementation of DDPMs to develop digital twins of stochastic biological systems.
97 MATHEMATICS AND COMPUTING↗
Machine Learning Emulation of Spatial Deposition from a Multi-Physics Ensemble of Weather and Atmospheric Transport Models
In the event of an accidental or intentional hazardous material release in the atmosphere, researchers often run physics-based atmospheric transport and dispersion models to predict the extent and variation of the contaminant spread. These predictions are imperfect due to propagated uncertainty from atmospheric model physics (or parameterizations) and weather data initial conditions. Ensembles of simulations can be used to estimate uncertainty, but running large ensembles is often very time consuming and resource intensive, even using large supercomputers. In this paper, we present a machine-learning-based method which can be used to quickly emulate spatial deposition patterns from a multi-physics ensemble of dispersion simulations. We use a hybrid linear and logistic regression method that can predict deposition in more than 100,000 grid cells with as few as fifty training examples. Logistic regression provides probabilistic predictions of the presence or absence of hazardous materials, while linear regression predicts the quantity of hazardous materials. The coefficients of the linear regressions also open avenues of exploration regarding interpretability—the presented model can be used to find which physics schemes are most important over different spatial areas. A single regression prediction is on the order of 10,000 times faster than running a weather and dispersion simulation. However, considering the number of weather and dispersion simulations needed to train the regressions, the speed-up achieved when considering the whole ensemble is about 24 times. Ultimately, this work will allow atmospheric researchers to produce potential contamination scenarios with uncertainty estimates faster than previously possible, aiding public servants and first responders.
97 MATHEMATICS AND COMPUTING↗
Probabilistic projections of the Amery Ice Shelf catchment, Antarctica, under conditions of high ice-shelf basal melt
Abstract. Antarctica's Lambert Glacier drains about one-sixth of the ice from the East Antarctic Ice Sheet and is considered stable due to the strong buttressing provided by the Amery Ice Shelf. While previous projections of the sea-level contribution from this sector of the ice sheet have predicted significant mass loss only with near-complete removal of the ice shelf, the ocean warming necessary for this was deemed unlikely. Recent climate projections through 2300 indicate that sufficient ocean warming is a distinct possibility after 2100. This work explores the impact of parametric uncertainty on projections of the response of the Lambert–Amery system (hereafter “the Amery sector”) to abrupt ocean warming through Bayesian calibration of a perturbed-parameter ice-sheet model ensemble. We address the computational cost of uncertainty quantification for ice-sheet model projections via statistical emulation, which employs surrogate models for fast and inexpensive parameter space exploration while retaining critical features of the high-fidelity simulations. To this end, we build Gaussian process (GP) emulators from simulations of the Amery sector at a medium resolution (4–20 km mesh) using the Model for Prediction Across Scales (MPAS)-Albany Land Ice (MALI) model. We consider six input parameters that control basal friction, ice stiffness, calving, and ice-shelf basal melting. From these, we generate 200 perturbed input parameter initializations using space filling Sobol sampling. For our end-to-end probabilistic modeling workflow, we first train emulators on the simulation ensemble and then calibrate the input parameters using observations of the mass balance, grounding line movement, and calving front movement with priors assigned via expert knowledge. Next, we use MALI to project a subset of simulations to 2300 using ocean and atmosphere forcings from a climate model for both low- and high-greenhouse-gas-emission scenarios. From these simulation outputs, we build multivariate emulators by combining GP regression with principal component dimension reduction to emulate multivariate sea-level contribution time series data from the MALI simulations. We then use these emulators to propagate uncertainty from model input parameters to predictions of glacier mass loss through 2300, demonstrating that the calibrated posterior distributions have both greater mass loss and reduced variance compared to the uncalibrated prior distributions. Parametric uncertainty is large enough through about 2130 that the two projections under different emission scenarios are indistinguishable from one another. However, after rapid ocean warming in the first half of the 22nd century, the projections become statistically distinct within decades. Overall, this study demonstrates an efficient Bayesian calibration and uncertainty propagation workflow for ice-sheet model projections and identifies the potential for large sea-level rise contributions from the Amery sector of the Antarctic Ice Sheet after 2100 under high-greenhouse-gas-emission scenarios.
54 ENVIRONMENTAL SCIENCES↗
Converting sWeights to probabilities with density ratios
The use of machine learning approaches continues to have many benefits in experimental nuclear and particle physics. One common issue is generating training data which is sufficiently realistic to give reliable results. Here we advocate using real experimental data as the source of training data and demonstrate how one might subtract background contributions through the use of probabilistic weights which can be readily applied to training data. The sPlot formalism is a common tool used to isolate distributions from different sources. However, the negative sWeights produced by the sPlot technique can cause training problems and poor predictive power. This article demonstrates how density ratio estimation can be applied to convert sWeights to event probabilities, which we call drWeights. The drWeights can then be applied to produce the distributions of interest and are consistent with direct use of the sWeights. This article will also show how decision trees are particularly well suited to convert sWeights, with the benefit of fast prediction rates and adaptability to aspects of experimental data such as the data sample size and proportions of different event sources. We also show that a density ratio product approach in which the initial drWeights are reweighted by an additional converter gives substantially better results.
Glazier, Derek I. [Univ. of Glasgow, Scotland (Uni↗
Deblending galaxies with variational autoencoders: A joint multiband, multi-instrument approach
ABSTRACT Blending of galaxies has a major contribution in the systematic error budget of weak-lensing studies, affecting photometric and shape measurements, particularly for ground-based, deep, photometric galaxy surveys, such as the Rubin Observatory Legacy Survey of Space and Time (LSST). Existing deblenders mostly rely on analytic modelling of galaxy profiles and suffer from the lack of flexible yet accurate models. We propose to use generative models based on deep neural networks, namely variational autoencoders (VAE), to learn probabilistic models directly from data. We train a VAE on images of centred, isolated galaxies, which we reuse, as a prior, in a second VAE-like neural network in charge of deblending galaxies. We train our networks on simulated images including six LSST bandpass filters and the visible and near-infrared bands of the Euclid satellite, as our method naturally generalizes to multiple bands and can incorporate data from multiple instruments. We obtain median reconstruction errors on ellipticities and r-band magnitude between ±0.01 and ±0.05, respectively, in most cases, and ellipticity multiplicative bias of 1.6 per cent for blended objects in the optimal configuration. We also study the impact of decentring and prove the method to be robust. This method only requires the approximate centre of each target galaxy, but no assumptions about the number of surrounding objects, pointing to an iterative detection/deblending procedure we leave for future work. Finally, we discuss future challenges about training on real data and obtain encouraging results when applying transfer learning.
Arcelin, Bastien↗
Efficient Probabilistic Prediction and Uncertainty Quantification of Tropical Cyclone–Driven Storm Tides and Inundation
Abstract This study proposes and assesses a methodology to obtain high-quality probabilistic predictions and uncertainty information of near-landfall tropical cyclone–driven (TC-driven) storm tide and inundation with limited time and resources. Forecasts of TC track, intensity, and size are perturbed according to quasi-random Korobov sequences of historical forecast errors with assumed Gaussian and uniform statistical distributions. These perturbations are run in an ensemble of hydrodynamic storm tide model simulations. The resulting set of maximum water surface elevations are dimensionality reduced using Karhunen–Loève expansions and then used as a training set to develop a polynomial chaos (PC) surrogate model from which global sensitivities and probabilistic predictions can be extracted. The maximum water surface elevation is extrapolated over dry points incorporating energy head loss with distance to properly train the surrogate for predicting inundation. We find that the surrogate constructed with third-order PCs using elastic net penalized regression with leave-one-out cross validation provides the most robust fit across training and test sets. Probabilistic predictions of maximum water surface elevation and inundation area by the surrogate model at 48-h lead time for three past U.S. landfalling hurricanes (Irma in 2017, Florence in 2018, and Laura in 2020) are found to be reliable when compared to best track hindcast simulation results, even when trained with as few as 19 samples. The maximum water surface elevation is most sensitive to perpendicular track-offset errors for all three storms. Laura is also highly sensitive to storm size and has the least reliable prediction. Significance Statement The purpose of this study is to develop and evaluate a methodology that can be used to provide high-quality probabilistic predictions of hurricane-induced storm tide and inundation with limited time and resources. This is important for emergency management purposes during or after the landfall of hurricanes. Our results show that sampling forecast errors using quasi-random sequences combined with machine learning techniques that fit polynomial functions to the data are well suited to this task. The polynomial functions also have the benefit of producing exact sensitivity indices of storm tide and inundation to the forecasted hurricane properties such as path, intensity, and size, which can be used for uncertainty estimation. The code implementing the presented methodology is publicly available on GitHub.
54 ENVIRONMENTAL SCIENCES↗