Search NASA⌕ Search

SEARCH · Search NASA

Results for “Stochastic Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Integrated Model, Batch and Domain Parallelism in Training Neural Networks

We propose a new integrated method of exploiting model, batch and domain parallelism for the training of deep neural networks (DNNs) on large distributed-memory computers using minibatch stochastic gradient descent (SGD). Our goal is to find an efficient parallelization strategy for a fixed batch size using $P$ processes. Our method is inspired by the communication-avoiding algorithms in numerical linear algebra. We see $P$ processes as logically divided into a $P_r \times P_c$ grid where the $P_r$ dimension is implicitly responsible for model/domain parallelism and the $P_c$ dimension is implicitly responsible for batch parallelism. In practice, the integrated matrix-based parallel algorithm encapsulates these types of parallelism automatically. We analyze the communication complexity and analytically demonstrate that the lowest communication costs are often achieved neither with pure model nor with pure data parallelism. We also show how the domain parallel approach can help in extending the theoretical scaling limit of the typical batch parallel method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Statistical Learning for Nonlinear Model Reduction from Local Simulations of Stochastic and Particle- and Agent-Based Systems

Stochastic physical systems across the sciences that have very high-dimensional state spaces, with a large number of fast degrees of freedom that force direct simulators to proceed by integration steps that are orders of magnitude smaller than events of interests (e.g., particle collisions). Examples range from molecular motion to dynamics of large populations of cells. A grand challenge in the simulation and understanding of such systems is the systematic construction of accurate, interpretable, reduced models, enabling faster simulations, revealing fundamental properties of the dynamics, and predicting phenomena of interest that the original simulator could not reached with sufficient accuracy or within a given computational budget. In this projected we developed novel statistical estimation/machine learning techniques for analyzing and building empirical reduced models for important families of high-dimensional stochastic systems, in particular: - we developed techniques for estimating interaction kernels in interacting particle- and agent-based systems, which are ubiquitous in Physics, Biology and many other sciences, given observed trajectories of the system; - we developed techniques for nonlinear model reduction for high-dimensional stochastic systems that have a small number of unknown, nonlinear slow variables, and a large number of fast modes, that are possibly of large magnitude, given observed short trajectories of the system in the form of bursts of trajectories from different initial conditions; - we developed novel techniques for estimating linear dynamical systems on graphs when both the dynamics and the underlying graph are unknown, and we have a sparse set of space-time observations; - we considered the problem of estimating an unknown nonlinear observation function of a standard process (e.g. Brownian motion), so that we can recognized if an observed dynamics is "just" a nonlinear version of a known dynamics; we also developed benchmarks for learning algorithms aimed at learning and classifying diffusion processes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

A Low-Rank Multigrid Method for the Stochastic Steady-State Diffusion Problem

We study a multigrid method for solving large linear systems of equations with tensor product structure here. Such systems are obtained from stochastic finite element discretization of stochastic partial differential equations such as the steady-state diffusion problem with random coefficients. When the variance in the problem is not too large, the solution can be well approximated by a low-rank object. In the proposed multigrid algorithm, the matrix iterates are truncated to low rank to reduce memory requirements and computational effort. The technique is proved convergent with an analytic error bound. Numerical experiments show its effectiveness in solving the Galerkin systems compared to the original multigrid solver, especially when the number of degrees of freedom associated with the spatial discretization is large.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

ADAPTIVE ANNEALED IMPORTANCE SAMPLING FOR MULTIMODAL POSTERIOR EXPLORATION AND MODEL SELECTION WITH APPLICATION TO EXTRASOLAR PLANET DETECTION

We describe an algorithm that can adaptively provide mixture summaries of multimodal posterior distributions. The parameter space of the involved posteriors ranges in size from a few dimensions to dozens of dimensions. This work was motivated by an astrophysical problem called extrasolar planet (exoplanet) detection, wherein the computation of stochastic integrals that are required for Bayesian model comparison is challenging. The difficulty comes from the highly nonlinear models that lead to multimodal posterior distributions. We resort to importance sampling (IS) to estimate the integrals, and thus translate the problem to be how to find a parametric approximation of the posterior. To capture the multimodal structure in the posterior, we initialize a mixture proposal distribution and then tailor its parameters elaborately to make it resemble the posterior to the greatest extent possible. We use the effective sample size (ESS) calculated based on the IS draws to measure the degree of approximation. The bigger the ESS is, the better the proposal resembles the posterior. A difficulty within this tailoring operation lies in the adjustment of the number of mixing components in the mixture proposal. Brute force methods just preset it as a large constant, which leads to an increase in the required computational resources. We provide an iterative delete/merge/add process, which works in tandem with an expectation-maximization step to tailor such a number online. The efficiency of our proposed method is tested via both simulation studies and real exoplanet data analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Enhanced mechanical property evaluation using innovative data analytics capability

This report focuses on efforts to improve Multiphysics Object-Oriented Simulation Environ-ment (MOOSE) for mechanical property evaluation using data analytics. These efforts involveimprovements to the stochastic tools module (STM) for stochastic simulations of MOOSE mul-tiphysics model and the development of inverse optimization capabilities. The report gives anoverview of the STM and describes recent updates to its core capabilities and theory on itsreduced-order model (ROM) schemes. Examples are also provided showing the impact of theseupdates and exhibits the usefulness of ROMs. An overview of the gradient based inverse opti-mziation algorithms are given along with examples of their application to source identification.Inverse optimization will provide a new methodology in STM for fitting model parameters toexperimental data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

Adaptive Algebraic Multigrid for Finite Element Elliptic Equations with Random Coefficients

This thesis presents a two-grid algorithm based on Smoothed Aggregation Spectral Element Agglomeration Algebraic Multigrid (SA-ρAMGe) combined with adaptation. The aim is to build an efficient solver for the linear systems arising from discretization of second-order elliptic partial differential equations (PDEs) with stochastic coefficients. Examples include PDEs that model subsurface flow with random permeability field. During a Markov Chain Monte Carlo (MCMC) simulation process, that draws PDE coefficient samples from a certain distribution, the PDE coefficients change, hence the resulting linear systems to be solved change. At every such step the system (discretized PDE) needs to be solved and the computed solution used to evaluate some functional(s) of interest that then determine if the coefficient sample is acceptable or not. The MCMC process is hence computationally intensive and requires the solvers used to be efficient and fast. This fact that at every step of MCMC the resulting linear system changes, makes an already existing solver built for the old problem perhaps not as efficient for the problem corresponding to the new sampled coefficient. This motivates the main goal of our study, namely, to adapt an already existing solver to handle the problem (with changed coefficient) with the objective to achieve this goal to be faster and more efficient than building a completely new solver from scratch. Our approach utilizes the local element matrices (for the problem with changed coefficients) to build local problems associated with constructed by the method agglomerated elements (a set of subdomains that cover the given computational domain). We solve a generalized eigenproblem for each set in a subspace spanned by the previous local coarse space (used for the old solver) and a vector, component of the error, that the old solver cannot handle. A portion of the spectrum of these local eigen-problems (corresponding to eigenvalues close to zero) form the coarse basis used to define the new two-level method of our interest. We illustrate the performance of this adaptive two-level procedure with a large set of numerical experiments that demonstrate its efficiency over building the solvers from scratch.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

H ∞ Control for Energy Dispatch in Autonomous Nanogrid With Communication Delays

This paper proposes an optimal controller and estimator for energy dispatch to balance the power supply and demand considering communication delays. The proposed algorithm involves modeling an autonomous nanogrid (ANG) consisting of distributed energy resources, energy storage systems, loads, an $H$ ∞ controller with a reference power modulation technique, and a state estimator. The ANG was developed to express the dynamic supply-demand energy balance of a nanogird system. Reference power modulation was designed to generate the desired ESS power based on the imbalanced energy. Random communication delays were modeled using a stochastic variable satisfying the Bernoulli random binary distribution. The optimal $H$ ∞ controller and estimator were developed using a linear matrix inequality approach to exponentially stabilize the closed-loop system. Simulations were performed using real daily demand forecasts obtained from the Korea Meteorological Administration to demonstrate the effectiveness of the proposed real-time optimization algorithm.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2018

Global Sensitivity Analysis and Estimation of Model Error, Toward Uncertainty Quantification in Scramjet Computations

The development of scramjet engines is an important research area for advancing hypersonic and orbital flights. Progress toward optimal engine designs requires accurate flow simulations together with uncertainty quantification. However, performing uncertainty quantification for scramjet simulations is challenging due to the large number of uncertain parameters involved and the high computational cost of flow simulations. These difficulties are addressed in this paper by developing practical uncertainty quantification algorithms and computational methods, and deploying them in the current study to large-eddy simulations of a jet in crossflow inside a simplified HIFiRE Direct Connect Rig scramjet combustor. First, global sensitivity analysis is conducted to identify influential uncertain input parameters, which can help reduce the system’s stochastic dimension. Second, because models of different fidelity are used in the overall uncertainty quantification assessment, a framework for quantifying and propagating the uncertainty due to model error is presented. In conclusion, these methods are demonstrated on a nonreacting jet-in-crossflow test problem in a simplified scramjet geometry, with parameter space up to 24 dimensions, using static and dynamic treatments of the turbulence subgrid model, and with two-dimensional and three-dimensional geometries.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

Adaptive experimental design for multi-fidelity surrogate modeling of multi-disciplinary systems

We present an adaptive algorithm for constructing surrogate models of multi-disciplinary systems composed of a set of coupled components. With this goal we introduce “coupling” variables with a priori unknown distributions that allow surrogates of each component to be built independently. Once built, the surrogates of the components are combined to form an integrated-surrogate that can be used to predict system-level quantities of interest at a fraction of the cost of the original model. The error in the integrated-surrogate is greedily minimized using an experimental design procedure that allocates the amount of training data, used to construct each component-surrogate, based on the contribution of those surrogates to the error of the integrated-surrogate. Here, the multi-fidelity procedure presented is a generalization of multi-index stochastic collocation that can leverage ensembles of models of varying cost and accuracy, for one or more components, to reduce the computational cost of constructing the integrated-surrogate. Extensive numerical results demonstrate that, for a fixed computational budget, our algorithm is able to produce surrogates that are orders of magnitude more accurate than methods that treat the integrated system as a black-box.

42 ENGINEERING↗
DOE OSTI2021

High-level coupled-cluster energetics by merging moment expansions with selected configuration interaction

Inspired by our earlier semi-stochastic work aimed at converging high-level coupled-cluster (CC) energetics, we propose a novel form of the CC(P; Q) theory in which the stochastic Quantum Monte Carlo propagations, used to identify dominant higher-than-doubly excited determinants, are replaced by the selected configuration interaction (CI) approach using the perturbative selection made iteratively (CIPSI) algorithm. The advantages of the resulting CIPSI-driven CC(P; Q) methodology are illustrated by a few molecular examples, including the dissociation of F 2 and the automerization of cyclobutadiene, where we recover the electronic energies corresponding to the CC calculations with a full treatment of singles, doubles, and triples based on the information extracted from compact CI wave functions originating from relatively inexpensive Hamiltonian diagonalizations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV Based Random Access IoT Networks With NOMA

In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers. Specifically, IoT devices contend for accessing the shared wireless channel using an adaptive p-persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency. To enable an energy-sustainable capacity-optimal network, we study the joint problem of dynamic multi-UAV altitude control and multi-cell wireless channel access management of IoT devices as a stochastic control problem with multiple energy constraints. We first formulate this problem as a Constrained Markov Decision Process (CMDP), and propose an online model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on Lagrangian primal-dual policy optimization to solve the CMDP. Extensive simulations demonstrate that our proposed algorithm learns a cooperative policy in which the altitude of UAVs and channel access probability of IoT devices are dynamically controlled to attain the maximal long-term network capacity while ensuring energy sustainability of UAVs, outperforming baseline schemes. The proposed CDRL agent can be trained on a small network, yet the learned policy can efficiently manage networks with a massive number of IoT devices and varying initial states, which can amortize the cost of training the CDRL agent.

42 ENGINEERING↗
DOE OSTI2025

AI-powered topic modeling: comparing LDA and BERTopic in analyzing opioid-related cardiovascular risks in women

Topic modeling is a crucial technique in natural language processing (NLP), enabling the extraction of latent themes from large text corpora. Traditional topic modeling, such as Latent Dirichlet Allocation (LDA), faces limitations in capturing the semantic relationships in the text document although it has been widely applied in text mining. BERTopic, created in 2022, leveraged advances in deep learning and can capture the contextual relationships between words. In this work, we integrated Artificial Intelligence (AI) modules to LDA and BERTopic and provided a comprehensive comparison on the analysis of prescription opioid-related cardiovascular risks in women. Opioid use can increase the risk of cardiovascular problems in women such as arrhythmia, hypotension etc. 1,837 abstracts were retrieved and downloaded from PubMed as of April 2024 using three Medical Subject Headings (MeSH) words: “opioid,” “cardiovascular,” and “women.” Machine Learning of Language Toolkit (MALLET) was employed for the implementation of LDA. BioBERT was used for document embedding in BERTopic. Eighteen was selected as the optimal topic number for MALLET and 23 for BERTopic. ChatGPT-4-Turbo was integrated to interpret and compare the results. The short descriptions created by ChatGPT for each topic from LDA and BERTopic were highly correlated, and the performance accuracies of LDA and BERTopic were similar as determined by expert manual reviews of the abstracts grouped by their predominant topics. The results of the t-SNE (t-distributed Stochastic Neighbor Embedding) plots showed that the clusters created from BERTopic were more compact and well-separated, representing improved coherence and distinctiveness between the topics. Our findings indicated that AI algorithms could augment both traditional and contemporary topic modeling techniques. In addition, BERTopic has the connection port for ChatGPT-4-Turbo or other large language models in its algorithm for automatic interpretation, while with LDA interpretation must be manually, and needs special procedures for data pre-processing and stop words exclusion. Therefore, while LDA remains valuable for large-scale text analysis with resource constraints, AI-assisted BERTopic offers significant advantages in providing the enhanced interpretability and the improved semantic coherence for extracting valuable insights from textual data.

Research & Experimental Medicine↗
DOE OSTI2016

3D Representative Volume Element Reconstruction of Fiber Composites via Orientation Tensor and Substructure Features

To provide a seamless integration of manufacturing processing simulation and fiber microstructure modeling, two new stochastic 3D microstructure reconstruction methods are proposed for two types of random fiber composites: random short fiber composites, and Sheet Molding Compounds (SMC) chopped fiber composites. A Random Sequential Adsorption (RSA) algorithm is first developed to embed statistical orientation information into 3D RVE reconstruction of random short fiber composites. For the SMC composites, an optimized Voronoi diagram based approach is developed for capturing the substructure features of SMC chopped fiber composites. The proposed methods are distinguished from other reconstruction works by providing a way of integrating statistical information (fiber orientation tensor) obtained from material processing simulation, as well as capturing the multiscale substructures of the SMC composites.

Li, Yi↗
DOE OSTI2021

Utilizing computer vision and artificial intelligence algorithms to predict and design the mechanical compression response of direct ink write 3D printed foam replacement structures

Additive Manufacturing (AM) of porous polymeric materials, such as foams, recently became a topic of intensive research due their unique combination of low density, impressive mechanical properties, and stress dissipation capabilities. Conventional methods for fabricating foams rely on complex and stochastic processes, making it challenging to achieve precise architectural control of structured porosity. In contrast, AM provides access to a wide range of printable materials, where precise spatial control over structured porosity can be modulated during the fabrication process enabling the production of foam replacement structures (FRS). Current approaches for designing FRS are based on intuitive understanding of their properties or an extensive number of finite element method (FEM) simulations. These approaches, however, are computationally expensive and time consuming. As such, in this work, we present a novel methodology for determining the mechanical compression response of direct ink write (DIW) 3D printed FRS using a simple cross-sectional image. By obtaining measurement data for a relatively small number of samples, an artificial neural network (ANN) was trained, and a computer vision algorithm was used to make inferences about foam compression characteristics from a single cross-sectional image. Finally, a genetic algorithm (GA) was used to solve the inverse design problem, generating the AM printing parameters that an engineer should use to achieve a desired compression response from a DIW printed FRS. The methods developed herein present an avenue for entirely autonomous design and analysis of additively manufactured structures using artificial intelligence.

36 MATERIALS SCIENCE↗
DOE OSTI2016

Cutting planes for the multistage stochastic unit commitment problem

As renewable energy penetration rates continue to increase in power systems worldwide, new challenges arise for system operators in both regulated and deregulated electricity markets to solve the security-constrained coal-fired unit commitment problem with intermittent generation (due to renewables) and uncertain load, in order to ensure system reliability and maintain cost effectiveness. In this paper, we study a security-constrained coal-fired stochastic unit commitment model, which we use to enhance the reliability unit commitment process for day-ahead power system operations. In our approach, we first develop a deterministic equivalent formulation for the problem, which leads to a large-scale mixed-integer linear program. Then, we verify that the turn on/off inequalities provide a convex hull representation of the minimum-up/down time polytope under the stochastic setting. Next, we develop several families of strong valid inequalities mainly through lifting schemes. In particular, by exploring sequence independent lifting and subadditive approximation lifting properties for the lifting schemes, we obtain strong valid inequalities for the ramping and general load balance polytopes. Lastly, branch-and-cut algorithms are developed to employ these valid inequalities as cutting planes to solve the problem. Our computational results verify the effectiveness of the proposed approach.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2015

Computationally-efficient stochastic cluster dynamics method for modeling damage accumulation in irradiated materials

An improved version of a recently developed stochastic cluster dynamics (SCD) method (Marian and Bulatov, 2012) [6] is introduced as an alternative to rate theory (RT) methods for solving coupled ordinary differential equation (ODE) systems for irradiation damage simulations. SCD circumvents by design the curse of dimensionality of the variable space that renders traditional ODE-based RT approaches inefficient when handling complex defect population comprised of multiple (more than two) defect species. Several improvements introduced here enable efficient and accurate simulations of irradiated materials up to realistic (high) damage doses characteristic of next-generation nuclear systems. The first improvement is a procedure for efficiently updating the defect reaction-network and event selection in the context of a dynamically expanding reaction-network. Next is a novel implementation of the τ-leaping method that speeds up SCD simulations by advancing the state of the reaction network in large time increments when appropriate. Lastly, a volume rescaling procedure is introduced to control the computational complexity of the expanding reaction-network through occasional reductions of the defect population while maintaining accurate statistics. The enhanced SCD method is then applied to model defect cluster accumulation in iron thin films subjected to triple ion-beam (Fe{sup 3+}, He{sup +} and H{sup +}) irradiations, for which standard RT or spatially-resolved kinetic Monte Carlo simulations are prohibitively expensive.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2019

Lagrangian relaxation based heuristics for a chance-constrained optimization model of a hybrid solar-battery storage system

Here, we develop a stochastic optimization model for scheduling a hybrid solar-battery storage system. Solar power in excess of the promise can be used to charge the battery, while power short of the promise is met by discharging the battery. We ensure reliable operations by using a joint chance constraint. Models with a few hundred scenarios are relatively tractable; for larger models, we demonstrate how a Lagrangian relaxation scheme provides improved results. To further accelerate the Lagrangian scheme, we embed the progressive hedging algorithm within the subgradient iterations of the Lagrangian relaxation. Lastly, we investigate several enhancements of the progressive hedging algorithm, and find bundling of scenarios results in the best bounds.

14 SOLAR ENERGY↗
DOE OSTI2024

Variance-Reduced Accelerated First-Order Methods: Central Limit Theorems and Confidence Statements

In this paper, we consider a strongly convex stochastic optimization problem and propose three classes of variable sample-size stochastic first-order methods: (i) the standard stochastic gradient descent method, (ii) its accelerated variant, and (iii) the stochastic heavy-ball method. In each scheme, the exact gradients are approximated by averaging across an increasing batch size of sampled gradients. We prove that when the sample size increases at a geometric rate, the generated estimates converge in mean to the optimal solution at an analogous geometric rate for schemes (i)–(iii). Based on this result, we provide central limit statements, whereby it is shown that the rescaled estimation errors converge in distribution to a normal distribution with the associated covariance matrix dependent on the Hessian matrix, the covariance of the gradient noise, and the step length. If the sample size increases at a polynomial rate, we show that the estimation errors decay at a corresponding polynomial rate and establish the associated central limit theorems (CLTs). Under certain conditions, we discuss how both the algorithms and the associated limit theorems may be extended to constrained and nonsmooth regimes. As a result, we provide an avenue to construct confidence regions for the optimal solution based on the established CLTs and test the theoretical findings on a stochastic parameter estimation problem.

Lei, Jinlong↗