Search NASA⌕ Search

SEARCH · Search NASA

Results for “Stochastic Sampling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

Parallel and interacting stochastic approximation annealing algorithms for global optimisation

We present the parallel and interacting stochastic approximation annealing (PISAA) algorithm, a stochastic simulation procedure for global optimisation, that extends and improves the stochastic approximation annealing (SAA) by using population Monte Carlo ideas. The efficiency of standard SAA algorithm crucially depends on its selfadjusting mechanism which presents stability issues in high dimensional or rugged optimisation problems. The proposed algorithm involves simulating a population of SAA chains that interact each other in a manner that significantly improves the stability of the self-adjusting mechanism and the search for the global optimum in the sampling space, as well as it inherits SAA desired convergence properties when a square-root cooling schedule is used. It can be implemented in parallel computing environments in order to mitigate the computational overhead. As a result, PISAA can address complex optimisation problems that it would be difficult for SAA to satisfactory address. We demonstrate the good performance of the proposed algorithm on challenging applications including Bayesian network learning and protein folding. Our numerical comparisons suggest that PISAA outperforms the simulated annealing, stochastic approximation annealing, and annealing evolutionary stochastic approximation Monte Carlo.

Local trap↗

A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networks

To fit sparse linear associations, a LASSO sparsity inducing penalty with a single hyperparameter provably allows to recover the important features (needles) with high probability in certain regimes even if the sample size is smaller than the dimension of the input vector (haystack). More recently learners known as artificial neural networks (ANN) have shown great successes in many machine learning tasks, in particular fitting nonlinear associations. Small learning rate, stochastic gradient descent algorithm and large training set help to cope with the explosion in the number of parameters present in deep neural networks. Yet few ANN learners have been developed and studied to find needles in nonlinear haystacks. Driven by a single hyperparameter, our ANN learner, like for sparse linear associations, exhibits a phase transition in the probability of retrieving the needles, which we do not observe with other ANN learners. To select our penalty parameter, we generalize the universal threshold of Donoho and Johnstone (Biometrika 81(3):425–455, 1994) which is a better rule than the conservative (too many false detections) and expensive cross-validation. In the spirit of simulated annealing, we propose a warm-start sparsity inducing algorithm to solve the high-dimensional, non-convex and non-differentiable optimization problem. We perform simulated and real data Monte Carlo experiments to quantify the effectiveness of our approach.

97 MATHEMATICS AND COMPUTING↗

Optimization of stochastic feature properties in laser powder bed fusion

Process parameter selection in laser powder bed fusion (LPBF) controls the as-printed dimensional tolerances, pore formation, surface quality and microstructure of printed metallic structures. Measuring the stochastic mechanical performance for a wide range of process parameters is cumbersome both in time and cost. As such, in this study, we overcome these hurdles by using high-throughput tensile (HTT) testing of over 250 dogbone samples to examine process-driven performance of strut-like small features, ~1 mm 2 in austenitic stainless steel (316 L). The output mechanical properties, porosity, surface roughness and dimensional accuracy were mapped across the printable range of laser powers and scan speeds using a continuous wave laser LPBF machine. Tradeoffs between ductility and strength are shown across the process space and their implications are discussed. While volumetric energy density deposited onto a substrate to create a melt-pool can be a useful metric for determining bulk properties, it was not found to directly correlate with output small feature performance.

316 L stainless steel↗

A comparison of generalized hybrid Monte Carlo methods with and without momentum flip

The generalized hybrid Monte Carlo (GHMC) method combines Metropolis corrected constant energy simulations with a partial random refreshment step in the particle momenta. The standard detailed balance condition requires that momenta are negated upon rejection of a molecular dynamics proposal step. The implication is a trajectory reversal upon rejection, which is undesirable when interpreting GHMC as thermostated molecular dynamics. We show that a modified detailed balance condition can be used to implement GHMC without momentum flips. The same modification can be applied to the generalized shadow hybrid Monte Carlo (GSHMC) method. Numerical results indicate that GHMC/GSHMC implementations with momentum flip display a favorable behavior in terms of sampling efficiency, i.e., the traditional GHMC/GSHMC implementations with momentum flip got the advantage of a higher acceptance rate and faster decorrelation of Monte Carlo samples. The difference is more pronounced for GHMC. We also numerically investigate the behavior of the GHMC method as a Langevin-type thermostat. We find that the GHMC method without momentum flip interferes less with the underlying stochastic molecular dynamics in terms of autocorrelation functions and it to be preferred over the GHMC method with momentum flip. The same finding applies to GSHMC.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Integration of multiple coinflip devices for high-quality random sampling

Artificial intelligence, scientific computing, and probabilistic computing use random sampling to approximate solutions to various problems, with larger models requiring a substantial quantity of random numbers. To generate the required vast quantity of random numbers at high rates, we explore so-called “coinflip” devices, which are stochastic microelectronic devices ideally capable of independently generating random bits with a tunable weight at a high rate. However, coinflip devices are inherently analog and demonstrate nonidealities, like temperature dependence and drift, that can introduce determinism into the outputs. We present important considerations for building systems of multiple coinflip devices to produce high-quality bitstreams with low error and little dependency on previous bits. Using tunnel diodes as coinflip devices, we implement a control loop to adapt to temperature dependence and generate fair bitstreams with each device. While this can lead to dependencies between bits in a single bitstream, we demonstrate that combining results generated in parallel with individual tunnel diodes can produce fair and unpredictable bitstreams. The suitability of these bitstreams for use in probabilistic computing is then demonstrated through a Monte Carlo approximation of π.

Taylor, Brady Garland [Sandia National Laboratorie↗

Variational deep learning of equilibrium transition path ensembles

Here, we present a time-dependent variational method to learn the mechanisms of equilibrium reactive processes and efficiently evaluate their rates within a transition path ensemble. This approach builds off of the variational path sampling methodology by approximating the time-dependent commitment probability within a neural network ansatz. The reaction mechanisms inferred through this approach are elucidated by a novel decomposition of the rate in terms of the components of a stochastic path action conditioned on a transition. This decomposition affords an ability to resolve the typical contribution of each reactive mode and their couplings to the rare event. The associated rate evaluation is variational and systematically improvable through the development of a cumulant expansion. We demonstrate this method in both over- and under-damped stochastic equations of motion, in low-dimensional model systems, and in the isomerization of a solvated alanine dipeptide. In all examples, we find that we can obtain quantitatively accurate estimates of the rates of the reactive events with minimal trajectory statistics and gain unique insights into transitions through the analysis of their commitment probability.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NUMERICAL FLOW AND TRANSPORT SIMULATIONS SUPPORTING THE SALTSTONE FACILITY PERFORMANCE ASSESSMENT

The Saltstone Disposal Facility Performance Assessment (PA) is being revised to incorporate requirements of Section 3116 of the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 (NDAA), and updated data and understanding of vault performance since the 1992 PA (Cook and Fowler 1992) and related Special Analyses. A hybrid approach was chosen for modeling contaminant transport from vaults and future disposal cells to exposure points. A higher resolution, largely deterministic, analysis is performed on a best-estimate Base Case scenario using the PORFLOW numerical analysis code. a few additional sensitivity cases are simulated to examine alternative scenarios and parameter settings. Stochastic analysis is performed on a simpler representation of the SDF system using the GoldSim code to estimate uncertainty and sensitivity about the Base Case. This report describes development of PORFLOW models supporting the SDF PA, and presents sample results to illustrate model behaviors and define impacts relative to key facility performance objectives. The SDF PA document, when issued, should be consulted for a comprehensive presentation of results.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

A Stochastic Calculus Approach to Boltzmann Transport

Traditional Monte Carlo methods for particle transport utilize source iteration to express the solution, the flux density, of the transport equation as a Neumann series. Our contribution is to show that the particle paths simulated within source iteration are associated with the adjoint flux density and the adjoint particle paths are associated with the flux density. Here, we make our assertion rigorous through the use of stochastic calculus by representing the particle path used in source iteration as a solution to a stochastic differential equation (SDE). The solution to the adjoint Boltzmann equation is then expressed in terms of the same SDE, and the solution to the Boltzmann equation is expressed in terms of the SDE associated with the adjoint particle process. An important consequence is that the particle paths used within source iteration simultaneously provide Monte Carlo samples of the flux density and adjoint flux density in the detector and source regions, respectively. The significant practical implication is that particle trajectories can be reused to obtain both forward and adjoint quantities of interest. To the best our knowledge, the reuse of entire particles paths has not appeared in the literature. Monte Carlo simulations are presented to support the reuse of the particle paths.

Boltzmann transport↗

Approximate method for stochastic chemical kinetics with two-time scales by chemical Langevin equations

The frequently used reduction technique is based on the chemical master equation for stochastic chemical kinetics with two-time scales, which yields the modified stochastic simulation algorithm (SSA). For the chemical reaction processes involving a large number of molecular species and reactions, the collection of slow reactions may still include a large number of molecular species and reactions. Consequently, the SSA is still computationally expensive. Because the chemical Langevin equations (CLEs) can effectively work for a large number of molecular species and reactions, this paper develops a reduction method based on the CLE by the stochastic averaging principle developed in the work of Khasminskii and Yin [SIAM J. Appl. Math. 56, 1766–1793 (1996); ibid. 56, 1794–1819 (1996)] to average out the fast-reacting variables. This reduction method leads to a limit averaging system, which is an approximation of the slow reactions. Because in the stochastic chemical kinetics, the CLE is seen as the approximation of the SSA, the limit averaging system can be treated as the approximation of the slow reactions. As an application, we examine the reduction of computation complexity for the gene regulatory networks with two-time scales driven by intrinsic noise. For linear and nonlinear protein production functions, the simulations show that the sample average (expectation) of the limit averaging system is close to that of the slow-reaction process based on the SSA. It demonstrates that the limit averaging system is an efficient approximation of the slow-reaction process in the sense of the weak convergence.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Stochastic Simulation of Daily Suspended Sediment Concentration Using Multivariate Copulas

Estimation of daily suspended sediment concentration (SSC) is required for water resources and environment management. In this paper, a copula-based stochastic method was proposed for daily SSC simulation. Here, the multivariate copula function, constructed based on a bivariate copula and two bivariate conditional probability distributions, was used to model the temporal and cross dependence structures in daily SSCs. Then, the daily SSCs were generated by sampling from the multivariate conditional distribution. As a result, synthetic long-term SSCs data beyond the limited observation period can be provided for water resources managers, which plays a critical role in accurately estimating frequency and magnitude of extreme SSCs events. The proposed method was under rigorous examination by applying to a case study at Pingshan station in the Jinsha River Basin, China. Results showed that the generated daily SSC sequences not only had a high degree of accuracy in preserving the statistical characteristics of the daily SSC observations, but also captured both the temporal correlation and the cross-correlation between the daily streamflow and daily SSC. Specifically, the average daily relative error values corresponding to mean, standard deviation, skewness, lag-1 temporal correlation, and cross correlation were 0.87%, 4.24%, 7.52%, 0.51% and 2.02%, respectively. The multivariate copula framework proposed here can accurately and efficiently generate long-term daily SSC data for water resources management such as frequency analysis and risk assessment of extreme SSC events.

54 ENVIRONMENTAL SCIENCES↗

Efficient Interdependent Systems Recovery Modeling with DeepONets

Modeling the recovery of interdependent critical infrastructure is a key component of quantifying and optimizing societal resilience to disruptive events. However, simulating the recovery of large-scale interdependent systems under random disruptive events is computationally expensive. Therefore, we propose the application of Deep Operator Networks (DeepONets) in this paper to accelerate the recovery modeling of interdependent systems. DeepONets are ML architectures which identify mathematical operators from data. The form of governing equations DeepONets identify and the governing equation of interdependent systems recovery model are similar. Therefore, we hypothesize that DeepONets can efficiently model the interdependent systems recovery with little training data. We applied DeepONets to a simple case of four interdependent systems with sixteen states. DeepONets, overall, performed satisfactorily in predicting the recovery of these interdependent systems for out of training sample data when compared to reference results.

97 MATHEMATICS AND COMPUTING↗

Analysis and Monte Carlo simulation of near-terminal aircraft flight paths

The flight paths of arriving and departing aircraft at an airport are stochastically represented. Radar data of the aircraft movements are used to decompose the flight paths into linear and curvilinear segments. Variables which describe the segments are derived, and the best fitting probability distributions of the variables, based on a sample of flight paths, are found. Conversely, given information on the probability distribution of the variables, generation of a random sample of flight paths in a Monte Carlo simulation is discussed. Actual flight paths at Dulles International Airport are analyzed and simulated.

Schiess, J. R.↗

Integrated communication and control systems. II - Design considerations

The ICCS design issues for nonperiodic and stochastic delays are addressed and the framework for alternative design procedures is outlined. The impact of network-induced delays on system stability is investigated and their physical significance is demonstrated using a simulation. The negative effects of vacant sampling and message rejection at the controller are demonstrated.

Ray, Asok↗

Reliable photometric membership (RPM) of galaxies in clusters – I. A machine learning method and its performance in the local universe

ABSTRACT We introduce a new method to determine galaxy cluster membership based solely on photometric properties. We adopt a machine learning approach to recover a cluster membership probability from galaxy photometric parameters and finally derive a membership classification. After testing several machine learning techniques (such as stochastic gradient boosting, model averaged neural network and k-nearest neighbours), we found the support vector machine algorithm to perform better when applied to our data. Our training and validation data are from the Sloan Digital Sky Survey main sample. Hence, to be complete to $M_r^* + 3$, we limit our work to 30 clusters with $z$phot-cl ≤ 0.045. Masses (M200) are larger than $\sim 0.6\times 10^{14} \, \mathrm{M}_{\odot }$ (most above $3\times 10^{14} \, \mathrm{M}_{\odot }$). Our results are derived taking in account all galaxies in the line of sight of each cluster, with no photometric redshift cuts or background corrections. Our method is non-parametric, making no assumptions on the number density or luminosity profiles of galaxies in clusters. Our approach delivers extremely accurate results (completeness, C $\sim 92{\rm{ per\ cent}}$ and purity, P $\sim 87{\rm{ per\ cent}}$) within R200, so that we named our code reliable photometric membership. We discuss possible dependencies on magnitude, colour, and cluster mass. Finally, we present some applications of our method, stressing its impact to galaxy evolution and cosmological studies based on future large-scale surveys, such as eROSITA, EUCLID, and LSST.

Lopes, Paulo A. A.↗

A model and variance reduction method for computing statistical outputs of stochastic elliptic partial differential equations

We present a model and variance reduction method for the fast and reliable computation of statistical outputs of stochastic elliptic partial differential equations. Our method consists of three main ingredients: (1) the hybridizable discontinuous Galerkin (HDG) discretization of elliptic partial differential equations (PDEs), which allows us to obtain high-order accurate solutions of the governing PDE; (2) the reduced basis method for a new HDG discretization of the underlying PDE to enable real-time solution of the parameterized PDE in the presence of stochastic parameters; and (3) a multilevel variance reduction method that exploits the statistical correlation among the different reduced basis approximations and the high-fidelity HDG discretization to accelerate the convergence of the Monte Carlo simulations. The multilevel variance reduction method provides efficient computation of the statistical outputs by shifting most of the computational burden from the high-fidelity HDG approximation to the reduced basis approximations. Furthermore, we develop a posteriori error estimates for our approximations of the statistical outputs. Based on these error estimates, we propose an algorithm for optimally choosing both the dimensions of the reduced basis approximations and the sizes of Monte Carlo samples to achieve a given error tolerance. We provide numerical examples to demonstrate the performance of the proposed method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Final Report, DOE Early Career Award: Predictive modeling of complex physical systems: new tools for statistical inference, uncertainty quantification, and experimental design

Predictive simulation of complex physical systems increasingly rests on the interplay of experimental observations with computational models. Key inputs, parameters, or structural aspects of models may be incomplete or unknown, and must be developed from indirect and limited observations. At the same time, quantified uncertainties are needed to qualify computational predictions in the support of design and decision-making. In this context, Bayesian statistics provides a foundation for inference from noisy and limited data, but at prohibitive computional expense. This project intends to make rigorous predictive modeling *feasible* in complex physical systems, via accelerated and scalable tools for uncertainty quantification, Bayesian inference, and experimental design. Specific objectives are as follows: 1. Develop adaptive posterior approximations and dimensionality reduction approaches for Bayesian inference in high-dimensional nonlinear systems. 2. Extend accelerated Bayesian methodologies to large-scale {\em sequential} data assimilation, fully treating nonlinear models and non-Gaussian state and parameter distributions. 3. Devise efficient surrogate-based methods for Bayesian model selection and the learning of model structure. 4. Develop scalable simulation/optimization approaches to nonlinear Bayesian experimental design, for both parameter inference and model selection. 5. Demonstrate these inferential tools on chemical kinetic models in reacting flow, constructing and refining thermochemical and electrochemical models from limited data. Demonstrate Bayesian filtering on canonical stochastic PDEs and in the dynamic estimation of inhomogeneous subsurface properties and flow fields.

97 MATHEMATICS AND COMPUTING↗

Modified Closures in Monte Carlo Algorithms for Diffusive Binary Stochastic Media Transport Problems

In a stochastic medium, the material properties at a given spatial location are known only statistically. The most common approach to solving particle transport problems involving binary stochastic media (BSM) is to use the atomic mix approximation in which the transport problem is solved using homogenized (volume-averaged) material properties. A common deterministic model developed for solving BSM particle transport problems is the Levermore-Pomraning (LP) model. Zimmerman and Adams proposed a Monte Carlo algorithm (Algorithm A) that solves the LP equations and another Monte Carlo algorithm (Algorithm B) that locally preserves the sampled material realization; we refer to these Monte Carlo algorithms as the LP and LPLRP (LP local realization preserving) algorithms, respectively. One-dimensional (1D) planar geometry benchmark studies have shown that the LPLRP algorithm is often significantly more accurate than the LP algorithm for problems with an incident angular flux as well as for problems with an interior source. The LPLRP algorithm implementation in these benchmark comparisons made explicit use of the one-dimensional nature of the problem. The LP model is derived by assuming an 'upwind' closure in the coupling term relating the two materials in a binary stochastic medium. Su and Pomraning developed a modified form of this closure by considering the small correlation length limit and requiring the modified closure to produce the correct exponential decay for a source-free half-line albedo problem in rod geometry. They concluded that the modified closure is generally not inferior to the LP closure and in some cases is significantly better. Brantley investigated the use of the Su-Pomraning (SP) closure in the Monte Carlo LP algorithm (LP-SP) for the suite of benchmark problems and concluded that 1) the LP-SP algorithm was somewhat more accurate overall than the LP algorithm and somewhat less accurate overall than the LPLRP algorithm for an incident angular flux benchmark suite, and 2) the LP-SP algorithm was generally the least accurate of the algorithms for an interior source benchmark suite. Larsen et al. performed an asymptotic analysis of the transport equation in 1D planar geometry for the situation in which the physical system is 1) a random binary stochastic medium with material macroscopic total cross sections and mean slab width values of O(1) or smaller and 2) optically thick with weak absorption and sources at each spatial point and therefore globally diffusive. The asymptotic analysis demonstrates that, under these assumptions, the transport equation limits to the conventional diffusion equation with atomically-mixed (volume-averaged) material properties. Larsen et al. also demonstrate that, under these same assumptions, the LP equations asymptotically limit to an atomically-mixed diffusion equation with a diffusion coefficient that is too large, leading to an unphysical flattening of the ensemble-averaged scalar flux distribution. Vasques and Yadav recently performed an asymptotic analysis of an adjusted Levermore-Pomraning closure in which the Markovian transition functions are rescaled such that the equations asymptotically limit to the correct atomically-mixed diffusion equation; we refer to this model as LP-VY. In this paper, we demonstrate that the adjusted LP closure proposed by Vasques and Yadav is a special case of the SP closure obtained by assuming that the absorption cross sections in the two materials are equal. We further demonstrate that the Su-Pomraning closure has the correct asymptotic limit for the diffusive physical system and is therefore an appropriate closure for these problems. We also describe how to incorporate these various closures into the Monte Carlo LP and LPLRP particle transport algorithms, and we present numerical results comparing the accuracy of the Monte Carlo LP, LP-VY, LP-SP, LPLRP, and LPLRP-SP algorithms for the set of diffusive benchmark problems. We demonstrated that the SP closure has the correct asymptotic limit for the diffusive physical system under investigation in this paper and that the VY closure is a special case of the SP closure obtained by assuming equal absorption in the two materials. Both of these closures are readily implemented in the Monte Carlo LP and LPLRP algorithms for particle transport in BSM. Through numerical comparisons to a diffusive benchmark suite, we confirmed that 1) the LP and LPLRP algorithms do not limit to the appropriate diffusion solution and 2) the LP-VY, LP-SP, and LPLRP-SP algorithms limit to the appropriate diffusion solution and are of comparable accuracy. The LPLRP algorithm has been shown to be generally more accurate than the LP algorithm for non-diffusive problems. However, the numerical results in this paper indicate that the LPLRP algorithm does not limit to the correct diffusion equation for the asymptotic limit under consideration. The work in this paper demonstrates that the SP algorithm does limit to the correct diffusion equation for the asymptotic limit considered. Future work will investigate the possibility of whether the LPLRP-SP algorithm is accurate for both diffusive and non-diffusive problems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Optimization using pathwise algorithmic derivatives of electromagnetic shower simulations

Among the well-known methods to approximate derivatives of expectancies computed by Monte-Carlo simulations, averages of pathwise derivatives are often the easiest one to apply. Computing them via algorithmic differentiation typically does not require major manual analysis and rewriting of the code, even for very complex programs like simulations of particle-detector interactions in high-energy physics. However, the pathwise derivative estimator can be biased if there are discontinuities in the program, which may diminish its value for applications. This work integrates algorithmic differentiation into the electromagnetic shower simulation code HepEmShow based on G4HepEm, allowing us to study how well pathwise derivatives approximate derivatives of energy depositions in a sampling calorimeter with respect to parameters of the beam and geometry. We found that when multiple scattering is disabled in the simulation, means of pathwise derivatives converge quickly to their expected values, and these are close to the actual derivatives of the energy deposition. Additionally, we demonstrate the applicability of this novel gradient estimator for stochastic gradient-based optimization in a model example.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗