Search NASA⌕ Search

SEARCH · Search NASA

Results for “Stochastic numerical methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

How plane are plane shock waves in solids

The notion of plane shock waves is a macroscopic, very fruitful idealization of near discontinuous disturbance propagating at supersonic speed. Such a picture is comparable to the picture of shorelines seen from a very high altitude. When viewed at the grain scale where the structure of solids is inherently heterogeneous and stochastic, features of shock waves are non-laminar and field variables, such as particle velocity and pressure, fluctuate. This paper reviews select aspects of such fluctuating nonequilibrium features of plane shock waves in solids with focus on grain scale phenomena and raises the need for a paradigm change to achieve a deeper understanding of plane shock waves in solids.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stochastic average model methods

We consider the solution of finite-sum minimization problems, such as those appearing in nonlinear least-squares or general empirical risk minimization problems. We are motivated by problems in which the summand functions are computationally expensive and evaluating all summands on every iteration of an optimization method may be undesirable. Here we present the idea of stochastic average model (SAM) methods, inspired by stochastic average gradient methods. SAM methods sample component functions on each iteration of a trust-region method according to a discrete probability distribution on component functions; the distribution is designed to minimize an upper bound on the variance of the resulting stochastic model. We present promising numerical results concerning an implemented variant extending the derivative-free model-based trust-region solver POUNDERS, which we name SAM-POUNDERS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Comparative Evaluation of Spectral Methods for Robust Reactor Noise Estimation

Reactor noise analysis provides a noninvasive means to determine neutron kinetic parameters from stochastic fluctuations in detector signals. However, standard cross-power spectral density (CPSD) analyses can be sensitive to numerical processing choices, which may introduce processing-dependent systematic shifts in estimates of the prompt neutron decay constant (α) and limit reproducibility. This study uses a hybrid multitaper–Welch spectral estimator to analyze subcritical noise measurements from a fast-spectrum critical assembly. The decay constant α was extracted using three frequency-domain methods: the CPSD, the magnitude-squared coherence (MSC), and the generalized magnitude-squared coherence (GMSC). These coherence-based estimators normalize detector auto-spectral structure and are expected to reduce the sensitivity of fitted α values to processing parameters. A Sobol global sensitivity analysis identified which numerical inputs most strongly influence the fitted values of α. All estimators produced a linear dependence of α on inverse count rate, with delayed-critical extrapolations near 1.7 × 10 4 s −1 , in agreement within 8% of MCNP6.3 KOPTS benchmark calculations. Sensitivity results show that while the CPSD depends on both time-bin width and taper selection, the MSC and GMSC are dominated by time-bin width alone, indicating reduced parameter coupling and greater robustness to processing variability. These findings demonstrate the feasibility and practical value of coherence-based spectral estimators for extracting α from reactor noise and support their broader application to multi-detector and irregular datasets in subcritical system characterization.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker–Planck Equations

The Fokker-Planck (FP) equation is a foundational partial differential equation (PDE) in stochastic processes involving Brownian motions. However, the curse of dimensionality (CoD) poses a formidable challenge when dealing with high-dimensional FP equations. Although Monte Carlo simulation and (vanilla) Physics-Informed Neural Networks (PINNs) have shown the potential to tackle CoD, both methods exhibit significant numerical errors in high dimensions when dealing with the probability density function (PDF) associated with Brownian motion. The point-wise PDF values tend to decrease exponentially as dimensionality increases, surpassing the precision of numerical simulations and resulting in substantial errors. In addition, due to its massive sampling, Monte Carlo fails to offer fast sampling. Modeling the logarithm likelihood (LL) via vanilla PINNs transforms the FP equation into a notoriously difficult Hamilton-Jacobi-Bellman (HJB) equation, which is impractical for PINN learning, whose error grows rapidly with dimension. To this end, we propose a novel approach utilizing a score-based solver to fit the score function in stochastic differential equations (SDEs). The score function, defined as the gradient of the LL, plays a fundamental role in inferring LL and PDF and enables fast SDE sampling, offering an effective means to overcome the CoD. Three fitting methods, Score Matching (SM), Sliced Score Matching (SSM), and Score-PINN, are introduced, each contributing unique advantages in computational complexity, accuracy, and generality. The proposed score-based SDE solver operates in two stages: first, employing score matching or Score-PINN to acquire the score function; and second, solving the LL via an ordinary differential equation (ODE) using the obtained score function. Comparative evaluations across these methods showcase varying trade-offs. The proposed methodology is evaluated across diverse SDEs, including anisotropic Ornstein-Uhlenbeck processes, geometric Brownian motion, and Brownian motion with varying eigenspace. We also test various distributions, including Gaussian, Log-normal, Laplace, and Cauchy distributions. The numerical results demonstrate the score-based SDE solver’s stability, speed, and performance across different experimental settings, solidifying its potential as a solution to CoD for high-dimensional FP equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stochastic finite volume method for uncertainty quantification of transient flow in gas pipeline networks

We develop a weakly intrusive framework to simulate the propagation of uncertainty in solutions of generic hyperbolic partial differential equation systems on graph-connected domains with nodal coupling and boundary conditions. The method is based on the Stochastic Finite Volume (SFV) approach and can be applied for uncertainty quantification (UQ) of the dynamical state of fluid flow over actuated transport networks. The numerical scheme has specific advantages for modeling intertemporal uncertainty in time-varying boundary parameters, which cannot be characterized by strict upper and lower (interval) bounds. We describe the scheme for a single pipe, and then formulate the controlled junction Riemann problem (JRP) that enables the extension to general network structures. In conclusion, we demonstrate the method's capabilities and performance characteristics using a standard benchmark test network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Integral Kernel Methods for Nonlinear Parabolic-Elliptic Systems

Nonlinear parabolic-elliptic systems arise in many physical, biological, and chemical phenomena such as chemotaxis, ion transport, self-gravitating particles, and Brownian vortices. Existing methods struggle with the strong coupling and high nonlinearity and nonlocality of some of these systems, especially the ill-conditioned, convection-dominated problems. To overcome numerical difficulties, current approaches rely on initial guesses, preconditioning, or iterative techniques with no convergence guarantees. They might suffer from poor scalability, large memory usage, and difficulty to parallelize. Inspired by the connection of parabolic-elliptic systems to stochastic processes, we introduce a novel meshless, monolithic, and fully explicit method that naturally encapsulates the elliptic and parabolic operators into a single step which updates each node deterministically with global information. By being fully quadrature-based, it avoids solving systems of discretized equations and does not utilize initial guesses or preconditioning, while requiring little memory and being easy to parallelize. We first derive the method in an integral kernel formulation with quadratic complexity in the number of integration nodes and then leverage kernel-independent fast multipole methods (FMM) to present a scalable algorithm with linear complexity. We provide numerical examples for the Poisson-Nernst-Planck equations in one, two, and three dimensions, together with the derivation of the integral kernel for each case. Furthermore, the examples demonstrate the fast convergence and scalability of the FMM-accelerated algorithm, as well as its suitability for convection-dominated problems, making it competitive against traditional PDE solvers.

PDE systems↗
DOE OSTI2024

Stochastic minibatch approach to the ptychographic iterative engine

The ptychographic iterative engine (PIE) is a widely used algorithm that enables phase retrieval at nanometer-scale resolution over a wide range of imaging experiment configurations. By analyzing diffraction intensities from multiple scanning locations where a probing wavefield interacts with a sample, the algorithm solves a difficult optimization problem with constraints derived from the experimental geometry as well as sample properties. The effectiveness at which this optimization problem is solved is highly dependent on the ordering in which we use the measured diffraction intensities in the algorithm, and random ordering is widely used due to the limited ability to escape from stagnation in poor-quality local solutions. In this study, we introduce an extension to the PIE algorithm that uses ideas popularized in recent machine learning training methods, in this case minibatch stochastic gradient descent. Our results demonstrate that these new techniques significantly improve the convergence properties of the PIE numerical optimization problem.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Score-based deterministic density sampling

We propose a deterministic sampling framework using Score-Based Transport Modeling for sampling an unnormalized target density π given only its score ∇ log π. Our method approximates the Wasserstein gradient flow on KL($f_t$∥π) by learning the time-varying score ∇ log $f_t$ on the fly using score matching. While having the same marginal distribution as Langevin dynamics, our method produces smooth deterministic trajectories, resulting in monotone noise-free convergence. We prove that our method dissipates relative entropy at the same rate as the exact gradient flow, provided sufficient training. Numerical experiments validate our theoretical findings: our method converges at the optimal rate, has smooth trajectories, and is often more sample efficient than its stochastic counterpart. Experiments on high-dimensional image data show that our method produces high-quality generations in as few as 15 steps and exhibits natural exploratory behavior. The memory and runtime scale linearly in the sample size.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Multistage distributionally robust mixed-integer programming with decision-dependent moment-based ambiguity sets

We study multistage distributionally robust mixed-integer programs under endogenous uncertainty, where the probability distribution of stage-wise uncertainty depends on the decisions made in previous stages. We first consider two ambiguity sets defined by decision-dependent bounds on the first and second moments of uncertain parameters and by mean and covariance matrix that exactly match decision-dependent empirical ones, respectively. For both sets, we show that the subproblem in each stage can be recast as a mixed-integer linear program (MILP). Moreover, we extend the general moment-based ambiguity set in to the multistage decision-dependent setting, and derive mixed-integer semidefinite programming (MISDP) reformulations of stage-wise subproblems. We develop methods for attaining lower and upper bounds of the optimal objective value of the multistage MISDPs, and approximate them using a series of MILPs. We deploy the Stochastic Dual Dynamic integer Programming (SDDiP) method for solving the problem under the three ambiguity sets with risk-neutral or risk-averse objective functions, and conduct numerical studies on multistage facility-location instances having diverse sizes under different parameter and uncertainty settings. Furthermore, our results show that the SDDiP quickly finds optimal solutions for moderate-sized instances under the first two ambiguity sets, and also finds good approximate bounds for the multistage MISDPs derived under the third ambiguity set. We also demonstrate the efficacy of incorporating decision-dependent distributional ambiguity in multistage decision-making processes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

An explicitly solvable energy-conserving algorithm for pitch-angle scattering in magnetized plasmas

In this work, we develop an Explicitly Solvable Energy-Conserving (ESEC) algorithm for the Stochastic Differential Equation (SDE) describing the pitch-angle scattering process in magnetized plasmas. The Cayley transform is used to calculate both the deterministic gyromotion and stochastic scattering, affording the algorithm to be explicitly solvable and exactly energy conserving. An unusual property of the SDE for pitch-angle scattering is that its coefficients diverge at the zero velocity and do not satisfy the global Lipschitz condition. Consequently, when standard numerical methods, such as the Euler-Maruyama (EM), are applied, numerical convergence is difficult to establish. For the proposed ESEC algorithm, its energy-preserving property enables us to overcome this obstacle. We rigorously prove that the ESEC algorithm is order 1/2 strongly convergent. This result is confirmed by detailed numerical studies. For the case of pitch-angle scattering in a magnetized plasma with a constant magnetic field, the numerical solution is benchmarked against the analytical solution, and excellent agreements are found.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

An adaptive Hessian approximated stochastic gradient MCMC method

Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their ability to handle large datasets and the potential to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. Here, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation (SA) to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the samples and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix via SA, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and demonstrates better learning results compared to standard SG-MCMC updating rules. The approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. Numerical experiments are performed on several problems, including sampling from 2D correlated distribution, synthetic regression problems, and learning the numerical solutions of heterogeneous elliptic PDE. The numerical results demonstrate great improvement in both the convergence rate and accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Numerical analysis of a time discretized method for nonlinear filtering problem with Lévy process observations

Abstract In this paper, we consider a nonlinear filtering model with observations driven by correlated Wiener processes and point processes. We first derive a Zakai equation whose solution is an unnormalized probability density function of the filter solution. Then, we apply a splitting-up technique to decompose the Zakai equation into three stochastic differential equations, based on which we construct a splitting-up approximate solution and prove its half-order convergence. Furthermore, we apply a finite difference method to construct a time semi-discrete approximate solution to the splitting-up system and prove its half-order convergence to the exact solution of the Zakai equation. Finally, we present some numerical experiments to demonstrate the theoretical analysis.

Mathematics↗
DOE OSTI2020

Dynamic Performance Enhancement for Nonlinear Stochastic Systems Using RBF Driven Nonlinear Compensation with Extended Kalman Filter

In this paper, a novel hybrid control method is proposed to enhance the control performance of the PI based control system for a class of nonlinear and non-Gaussian stochastic dynamic processes with unmeasurable states. Firstly, to enhance the tracking performance of the PI controller where the PI parameters are fixed in many actual control systems, the compensative signal is formed using the extended Kalman filter(EKF) based state estimator and driven by the radial basis function (RBF) neural network based compensator. In addition, the weights of RBF is trained to minimize the entropy criterion of tracking error as the process is subjected to non-Gaussian disturbances. Meanwhile, since the precise statistical property of noises is hard to obtain for many industrial processes, the kernel density estimation (KDE) technique is employed in this paper to estimate the entropy. The convergence of RBF network is discussed and the stability of the resulted closed-loop hybrid control system is analyzed in mean square sense. Finally, a numerical example and a practical system testing are given to illustrate the effectiveness of the proposed control method.

stochastic systems, PI Controller, Entropy, Extend↗
DOE OSTI2024

Multiscale simulation of spatially correlated microstructure via a latent space representation

When deformation gradients act on the scale of the microstructure of a part due to geometry and loading, spatial correlations and finite-size effects in simulation cells cannot be neglected. We propose a multiscale method that accounts for these effects using a variational autoencoder to encode the structure–property map of the stochastic volume elements making up the statistical description of the part. In this paradigm the autoencoder can be used to directly encode the microstructure or, alternatively, its latent space can be sampled to provide likely realizations. Furthermore, we demonstrate the method on three examples using the common additively manufactured material AlSi10Mg in: (a) a comparison with direct numerical simulation of the part microstructure, (b) a push forward of microstructural uncertainty to performance quantities of interest, and (c) a simulation of functional gradation of a part with stochastic microstructure.

Elastoplasticity↗
DOE OSTI2024

Explicit simulation of the Brownian rotation of arbitrary shaped aerosol particles using quaternions

The shape of an aerosol particle strongly influences its mass and momentum transfer cross-sections, charging properties, and other physical properties. Here, we present an explicit time-stepping procedure to simulate the rotational Brownian motion of arbitrary shaped aerosol particles by solving Euler’s equation of rotation. A Langevin formulation of the rotation equations is used, wherein Brownian motion due to thermal collisions between a particle and background gas molecules is represented using a stochastic fluctuating torque and fluid resistance is included as a drag torque. To avoid singularities associated with describing the orientation of a shape with Euler angles, we employ a quaternion formulation that leads to first-order stochastic differential equations to describe the evolution of the angular position and angular velocity of a rigid body. We perform all the rotational dynamics calculations in the body-fixed frame of reference attached to the rotating shape whose basis vectors are the normalized eigenvectors of the inertia tensor of the particle. Numerical solutions to rotation under torque-free conditions, damped rotation without Brownian motion, and stochastic rotation for arbitrary shapes are presented and discussed. The presented method enables time-resolved simulation of Brownian rotation for direct comparison with experimentally measured trajectories or statistical measures. The second order accuracy of the used time-stepping procedure places a severe restriction on the timestep that can be used for obtaining accurate results. Animations of presented simulations are included for visualizing rotational motion at various gas pressures. To aid implementation, MATLAB ® codes are also provided. Extension to include translation Brownian motion is straightforward.

Roy, Mrittika↗
DOE OSTI2024

A meshless stochastic method for Poisson–Nernst–Planck equations

A plethora of biological, physical, and chemical phenomena involve transport of charged particles (ions). Its continuum-scale description relies on the Poisson–Nernst–Planck (PNP) system, which encapsulates the conservation of mass and charge. The numerical solution of these coupled partial differential equations is challenging and suffers from both the curse of dimensionality and difficulty in efficiently parallelizing. We present a novel particle-based framework to solve the full PNP system by simulating a drift–diffusion process with time- and space-varying drift. We leverage Green’s functions, kernel-independent fast multipole methods, and kernel density estimation to solve the PNP system in a meshless manner, capable of handling discontinuous initial states. The method is embarrassingly parallel, and the computational cost scales linearly with the number of particles and dimension. We use a series of numerical experiments to demonstrate both the method’s convergence with respect to the number of particles and computational cost vis-à-vis a traditional partial differential equation solver.

Chemistry↗
DOE OSTI2020

On Transfer Learning Of Neural Networks Using Bi-Fidelity Data For Uncertainty Propagation

Due to their high degree of expressiveness, neural networks have recently been used as surrogate models for mapping inputs of an engineering system to outputs of interest. Once trained, neural networks are computationally inexpensive to evaluate and remove the need for repeated evaluations of computationally expensive models in uncertainty quantification applications. However, given the highly parameterized construction of neural networks, especially deep neural networks, accurate training often requires large amounts of simulation data that may not be available in the case of computationally expensive systems. In this paper, to alleviate this issue for uncertainty propagation, we explore the application of transfer learning techniques using training data generated from both high- and low-fidelity models. We explore two strategies for coupling these two datasets during the training procedure, namely, the standard transfer learning and the bi-fidelity-weighted learning. In the former approach, a neural network model mapping the inputs to the outputs of interest is trained based on the low-fidelity data. The high-fidelity data are then used to adapt the parameters of the upper layer(s) of the low-fidelity network, or train a simpler neural network to map the output of the low-fidelity network to that of the high-fidelity model. In the latter approach, the entire low-fidelity network parameters are updated using data generated via a Gaussian process model trained with a small high-fidelity dataset. The parameter updates are performed via a variant of stochastic gradient descent with learning rates given by the Gaussian process model. Using three numerical examples, we illustrate the utility of these bi-fidelity transfer learning methods where we focus on accuracy improvement achieved by transfer learning over standard training approaches.

97 MATHEMATICS AND COMPUTING↗