Search NASA⌕ Search

SEARCH · Search NASA

Results for “Stochastic optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Surrogate Model Based Optimization for Finding Robust Deep Learning Model Architectures

Deep Learning (DL) models are increasingly used throughout the sciences. However, their performance and usefulness depend greatly on their architecture which is defined by hyperparameters such as the number of nodes, layers, the learning rate, etc. Tuning these hyperparameters is time-consuming because evaluating their performance requires a lengthy training step. Stochastic optimizers used in training lead to performance variability and potentially prediction reliability issues. In this talk, we will describe an automated optimization method based on surrogate models and active learning strategies for tuning DL model architectures. We take into account the prediction variability with the goal to identify architectures that make reliable and robust predictions. We demonstrate our developments on an application arising in particle physics.

deep learning↗
DOE OSTI2019

Fast, Dynamic, and Scalable Algorithms for Large-Scale Constrained Optimization

Scientists and engineers constantly aim to optimize an objective subject to physical, environmental, and/or resource constraints. The technique of using mathematical models to formulate and find real solutions of such problems is known as mathematical optimization, a process that has become invaluable for design and discovery in numerous scientific fields. This project has focused on the design, analysis, and implementation of high-performance computing algorithms for solving cutting-edge optimization problems. These include problems that involve (1) data uncertainties, such as those in the future supply, demand, and capacity of a given power system, (2) extreme numbers of alternatives, such as in the design of electrical power grids to avoid network vulnerabilities, and (3) real-time decisions, such as in the control of chemical reactors. The key features of the new algorithms are that they are fast, dynamic, and scalable to meet the computational requirements of scientists and researchers working to optimize large-scale, complex systems. The algorithms that have been designed, analyzed, and tested as part of the project can be characterized based on the types of optimization problems they have been designed to solve. Much of the work has focused on algorithms for solving large-scale constrained optimization problems. Contributions have also been made in the design of algorithms for unconstrained optimization, with some specialized methods for solving problems that have inherent nonsmoothness. Contributions have also been made in areas that have grown to prominence in the field over the five-year award period. These include the design of algorithms for solving large-scale stochastic optimization problems, such as those that arise in large-scale machine learning, and in complexity analyses for solving nonconvex problems, which has been a focal point of much recent work in continuous optimization. Overall, the project has supported work that has led to almost 30 articles that have been published or accepted to top journals in the field of mathematical optimization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2011

Final Report---Optimization Under Nonconvexity and Uncertainty: Algorithms and Software

the goal of this work was to develop new algorithmic techniques for solving large-scale numerical optimization problems, focusing on problems classes that have proven to be among the most challenging for practitioners: those involving uncertainty and those involving nonconvexity. This research advanced the state-of-the-art in solving mixed integer linear programs containing symmetry, mixed integer nonlinear programs, and stochastic optimization problems. The focus of the work done in the continuation was on Mixed Integer Nonlinear Programs (MINLP)s and Mixed Integer Linear Programs (MILP)s, especially those containing a great deal of symmetry.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Transportation Hub Infrastructure Expansion: Decision Support Under Uncertainty

The Athena project (www.athena-mobility.org) has worked to investigate the relationship between the Dallas-Fort Worth Airport (DFW) and the greater Dallas area in order to better understand and therefore better inform future decision-making regarding the critical infrastructure that influence mobility between the airport and the city. Through this work, infrastructure related to curbside pickup and drop-off, parking, public transit, and the road network congestion were identified as critical to the operation of the DFW transportation hub. The infrastructure analysis and expansion aspect of the Athena project is focused on the restructuring of the CTA curb as a hierarchical curb and the building or repurposing of parking infrastructure as the interplay between these two areas. Many sources of uncertainty exist that may impact future airport and transportation hub operations, such as passenger volume growth, population demographic changes over time, electric vehicle (EV) adoption rates, and autonomous vehicle (AV) adoption rates. Due to these sources of uncertainty, we have selected for our research a modeling framework that can capture various types of uncertainty and hedge against those uncertainties in the optimization process. We analyze road network and curb congestion, the rise of transportation networking companies, trends in parking usage, existing policies around this infrastructure, airport revenue streams, and other contributing factors to enable infrastructure decision making with less uncertainty. To accomplish this wholistic analysis, we have developed a novel multi-stage, multi-period stochastic optimization model which considers the airport's decisions from 2025-2045 under different possible future macro trajectories and day-to-day variations in operational conditions captured as "annual representation of operations" scenarios with respective probabilities. This model has also been designed to leverage the outputs of various efforts under the Athena project to create a combined decision framework for infrastructure decisions. These various efforts include the route optimization model, the ASPIRES simulation, the mode choice model, and the SUMO traffic simulation. Our computational experiments of this system at scale have resulted in a working version of our infrastructure model which enables the explicit representation and consideration of various sources of uncertainty in the decision process to enable robust, flexible decision-making. This model has been effectively run on NREL's HPC system, Eagle, with large numbers of stochastic scenarios and shows promise as a scalable tool for robust consideration of uncertainties in airport planning. We have tested our model using 30,240 operational circumstances in total, resulting in a problem with more 200 million variables. This model was solved in several different configurations, and a workflow to simulate the performance of the infrastructure model results was developed and deployed. In general, our results indicate that a combination of remote parking, remote curb infrastructure, and dynamic pricing can generate revenue, reduce emissions, accommodate emerging technologies such as AVs and EVs, and manage airport passenger growth over time. We note the success of the proposed strategy depends on the data collection and forecasting abilities of DFW. We have also seen that the AV adoption by TNCs might necessitate larger amounts of remote curb. The results of this work inform strategies for airport infrastructure decision making, as well as demonstrate the value of an adaptable model, but also indicate that there are avenues remaining where further research would be of value.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2015

Bootstrap performance profiles in stochastic algorithms assessment

Optimization with stochastic algorithms has become a relevant research field. Due to its stochastic nature, its assessment is not straightforward and involves integrating accuracy and precision. Performance profiles for the mean do not show the trade-off between accuracy and precision, and parametric stochastic profiles require strong distributional assumptions and are limited to the mean performance for a large number of runs. In this work, bootstrap performance profiles are used to compare stochastic algorithms for different statistics. This technique allows the estimation of the sampling distribution of almost any statistic even with small samples. Multiple comparison profiles are presented for more than two algorithms. The advantages and drawbacks of each assessment methodology are discussed.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2019

Higher-moment buffered probability

In stochastic optimization, probabilities naturally arise as cost functionals and chance constraints. Unfortunately, these functions are difficult to handle both theoretically and computationally. The buffered probability of failure and its subsequent extensions were developed as numerically tractable, conservative surrogates for probabilistic computations. In this manuscript, we introduce the higher-moment buffered probability. Whereas the buffered probability is defined using the Conditional Value-at-Risk, the higher-moment buffered probability is defined using Higher-Moment Coherent Risk measures. In this way, the higher-moment buffered probability encodes information about the magnitude of tail moments, not simply the tail average. We prove that the higher-moment buffered probability is closed, monotonic, quasi-convex and can be computed by solving a smooth one-dimensional convex optimization problem. These properties enable smooth reformulations of both higher-moment buffered probability cost functionals and constraints.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Quantum Stochastic Programming [SWR-26-040]

The Quantum Stochastic Programming tool contains quantum computing algorithms for two-stage stochastic optimization, with a focus on the Unit Commitment (UC) problem in power systems. The algorithms combine Discrete Quantum Annealing (DQA) with Quantum Amplitude Estimation (QAE) to compute expected-value objective functions over a probability distribution of wind-power scenarios. Based on: arXiv 2402.15029 - "Quantum algorithms for the two-stage stochastic unit commitment problem"

Maack, Jonathan [National Laboratory of the Rockie↗
DOE OSTI2018

A stochastic downscaling approach for generating high-frequency solar irradiance scenarios

Solar power is increasingly cost viable with solar photovoltaic (PV) installations becoming commonplace. PV planning and operational studies, however, require high-frequency solar irradiance scenarios to understand potential electric grid impacts due to the variability and uncertainty of the underlying solar resource. Existing remote sensing solar data products are often available over large spatial domains, but are limited in temporal resolution. For example, the global horizontal irradiance (GHI) component contained within the National Solar Radiation Database (NSRDB) is available at time resolution of 30 min on an approximately four-kilometer grid. In contrast, substantial solar variability is present at finer time scales and this article describes an algorithm to stochastically generate one-minute GHI from widely available sub-hourly NSRDB. A generalized linear modeling (GLM) framework is proposed, which includes non-Gaussian mixtures, and extends the literature involving the synthesis of GHI data. The model is trained on a set of sample locations around Oregon, USA, and validated across the USA using both the Surface Radiation Budget Network (SURFRAD) dataset and Solar Radiation Monitoring Laboratory (SRML) network. Simulated ensembles show good coverage properties and temporal correlation structure. The resulting downscaled ensembles allow for understanding the unpredictable variability inherent in GHI at locations without direct measurements. Here, future work can leverage the algorithm as part of a stochastic optimization of electric grid operations with high-penetration PV systems.

14 SOLAR ENERGY↗
DOE OSTI2022

Stochastic and a posteriori optimization to mitigate coil manufacturing errors in stellarator design

Abstract It was recently shown in Wechsung et al (2022 Proc. Natl Acad. Sci. USA 119 e2202084119) that there exist electromagnetic coils that generate magnetic fields, which are excellent approximations to quasi-symmetric fields and have very good particle confinement properties. Using a Gaussian process-based model for coil perturbations, we investigate the impact of manufacturing errors on the performance of these coils. We show that even fairly small errors result in noticeable performance degradation. While stochastic optimization yields minor improvements, it is not possible to mitigate these errors significantly. As an alternative to stochastic optimization, we then formulate a new optimization problem for computing optimal adjustments of the coil positions and currents without changing the shapes of the coil. These a-posteriori adjustments are able to reduce the impact of coil errors by an order of magnitude, providing a new perspective for dealing with manufacturing tolerances in stellarator design.

Physics↗
DOE OSTI2017

Parameter optimization for transitions between memory states in small arrays of Josephson junctions

Coupled arrays of Josephson junctions possess multiple stable zero voltage states. Such states can store information and consequently can be utilized for cryogenic memory applications. Basic memory operations can be implemented by sending a pulse to one of the junctions and studying transitions between the states. In order to be suitable for memory operations, such transitions between the states have to be fast and energy efficient. Here in this article we employed simulated annealing, a stochastic optimization algorithm, to study parameter optimization of array parameters which minimizes times and energies of transitions between specifically chosen states that can be utilized for memory operations (Read, Write, and Reset). Simulation results show that such transitions occur with access times on the order of 10–100 ps and access energies on the order of 10 -19 –5×10 -18 J. Numerical simulations are validated with approximate analytical results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Mode connectivity in the loss landscape of parameterized quantum circuits

Variational training of parameterized quantum circuits (PQCs) underpins many workflows employed on near-term noisy intermediate scale quantum (NISQ) devices. It is a hybrid quantum-classical approach that minimizes an associated cost function in order to train a parameterized ansatz. In this work we adapt the qualitative loss landscape characterization for neural networks introduced in Goodfellow et al. (2014); Li et al. (2017) and tests for connectivity used in Draxler et al. (2018) to study the loss landscape features in PQC training. We present results for PQCs trained on a simple regression task, using the bilayer circuit ansatz, which consists of alternating layers of parameterized rotation gates and entangling gates. Multiple circuits are trained with 3 different batch gradient optimizers: stochastic gradient descent, the quantum natural gradient, and Adam. We identify large features in the landscape that can lead to faster convergence in training workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Thermal Energy Storage Model Development within the Integrated Energy Systems Hybrid Repository

This publication details newly created energy storage models developed within the HYBRID Modelica repository as part of the Department of Energy Office of Nuclear Energy (DOE-NE) Integrated Energy Systems (IES) program, led by Idaho National Laboratory (INL). Model development to-date includes creation of dynamic systems-level models of concrete, latent heat, and packed-bed thermocline energy storage technologies for deployment in the IES-based HYBRID repository. Models are developed using the latest publicly available data and incorporate the possibility of control strategy inclusion for use with the existing IES modeling, analysis, and optimization toolset. Simulations showcase the abilities of each technology to cyclically charge and discharge when exposed to time-varying boundary conditions. In addition, low-level surrogate models for some of the thermal energy storage technologies were created using Python. These lower order Python files are much cheaper to run (i.e., computationally faster) and thus operate well when incorporated within the stochastic optimization problems run for the IES program. Moreover, economic data was collected for use within the INL-developed Framework for Optimization of ResourCes and Economics (FORCE). This information is incorporated within the FORCE platform. This work has resulted in creation of systems-level models for concrete, latent heat, and thermocline thermal energy storage systems with associated control systems. Now that these models are available, they can be utilized within different integrated energy park concepts to understand optimal system operation, control, and dispatching. Moreover, given the generic nature of the models, industrial partner technologies (e.g., Storworks Power, EnergyNest) can be quickly added to the repository using the existing models as a basis. Additional dynamic models for thermal energy storage concepts can be developed and added to the HYBRID repository as needed.

25 ENERGY STORAGE↗
DOE OSTI2021

Risk-averse optimization for resilience enhancement of complex engineering systems under uncertainties

With the growth of complexity and extent, large scale interconnected network systems, e.g., transportation networks or infrastructure networks, become more vulnerable to external disturbances. Hence, managing potential disruptive events during the design, operating, and recovery phase of an engineered system and therefore improving the system’s resilience is an important yet challenging task. Here, to ensure system resilience after the occurrence of failure events, this study proposes a mixed-integer linear programming (MILP) based restoration framework using heterogeneous dispatchable agents. The scenario-based stochastic optimization (SO) technique is adopted to deal with the inherent uncertainties imposed on the recovery process from nature. Moreover, different from conventional SO using deterministic equivalent formulations, the CVaR risk measure is implemented for this study because of the temporal sparsity of the decision making in applications such as the recovery from extreme events. The resulting restoration framework involves a large-scale MILP problem and thus an adequate decomposition technique i.e. modified Lagrangian dual decomposition, is also employed to achieve tractable computational complexity. Case study results based on the IEEE 37-bus test feeder demonstrate the benefits of using the proposed framework for resilience improvement as well as the advantages of adopting SO formulations.

42 ENGINEERING↗
DOE OSTI2019

A Stochastic Multi-Commodity Logistic Model for Disaster Preparation in Distribution Systems

In this paper, we propose a stochastic optimization approach for disaster preparation in distribution systems. For an upcoming storm, utilities should have a preparation plan that includes warehousing restoration supplies, securing staging sites (depots), and prepositioning crews and equipment. Pre-storm planning enables faster and more efficient post-disaster deploy ment of crews and equipment resources to damage locations. To assist utilities in making this important preparation, this paper develops a two-stage stochastic mixed integer linear program. The first stage determines the depots, number of crews in each site, and the amount of equipment. The second stage is the recourse action that deals with acquiring new equipment and assigning crews to repair damages in realized scenarios. The objective of the developed model is to minimize the costs of depots, crews, equipment, and penalty costs associated with delays in obtain ing equipment and restoration. We consider the uncertainties of damaged lines, number and type of equipment required, and expected repair times. The model is validated on modified IEEE 123-bus distribution test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Report on the Economic Optimization of a Demonstration Case for a Static N-R HES Configuration using RAVEN

The synthetic time history generation algorithm in RAVEN has been extensively tested and used to perform qualitative assessments of the statistical characteristics of the net demand vs. demand for different values of the penetration of wind generation. Reported results highlight the need to consider the impact of variable renewable on the electricity net demand profile, due to the increase in its volatility with increased wind penetration. This analysis provides a strong foundation for the modeling and simulation needs expressed under the "Nuclear-Renewable Hybrid Energy Systems" (NRHES) project. Following this confirmative analysis the entire stochastic optimization framework has been tested in a demo case. The case chosen is an optimization driven by profitability for a system composed by a nuclear plant and a hydrogen production industrial process. This type of economical analysis is aimed to assess the capability of a hybrid system to penetrate the current energy market. In this sense it differs from the approach suggested under NRHES, where a cost minimization approach is suggested. This analysis is instead aimed to show the flexibility of the developed framework, e.g. for the evaluation of retrofitting projects of already existing plants. The system considered is subject to variable prices of electricity, which are simulated using the algorithm for the production of synthetic time histories. The physical system is considered without inertia so that it can be replaced by dispatching rules based on the highest marginal profit. Two optimization problems have been considered, for the first the size of the nuclear power plant is fixed and the optimization variable is the capacity of the hydrogen production plant. For the second, both plant sizes are optimization parameters while the overall capital expenditure is caped. The economical figures of merits considered are the Internal Rate of Return (IRR) and/or Profitability Index (PI) for the first one (since no cap is placed on the investment size) and Net Present Value (NPV) and/or IRR for the second one where the capital expenditure is constrained. Economic input data concerning capital costs, size-scaling factors, and operational costs for both plants have been collected from literature in order to achieve a simulation as realistic as possible. More work will be needed to obtain even more realistic numbers, possibly from industrial partners, and considering market elasticity for the hydrogen market. In order to perform the optimization of the cases described above, the economical external RAVEN module (CashFlow) for financial analysis has been improved in flexibility and generality. Some improvements have also been made to the optimization algorithm for a more efficient treatment of the boundary constrains. In conclusion three major results have been obtained: a) collection of the economical parameters characterizing the nuclear power plants and hydrogen production plants, b) extension of the financial analysis module (CashFlow) to drive the calculation of all needed cash flow streams and c) the testing of the full framework in a nearly final configuration.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Collaborative Project: Building improved optimized parameter estimation algorithms to improve methane and nitrogen fluxes in a climate model

Soils in natural and managed ecosystems and wetlands are well known sources of methane, nitrous oxides, and reactive nitrogen gases, but the magnitudes of gas flux to the atmosphere are still poorly constrained. Thus, the reasons for the large increases in atmospheric concentrations of methane and nitrous oxide since the preindustrial time period are not well understood. The low atmospheric concentrations of methane and nitrous oxide, despite being more potent greenhouse gases than carbon dioxide, complicate empirical studies to provide explanations. In addition to climate concerns, the emissions of reactive nitrogen gases from soils are important to the changing nitrogen balance in the earth system, subject to human management, and may change substantially in the future. Thus improved modeling of the emission fluxes of these species from the land surface is important. Currently, there are emission modules for methane and some nitrogen species in the Community Earth System Model’s Community Land Model (CLM-ME/N); however, there are large uncertainties and problems in the simulations, resulting in coarse estimates. In this proposal, we seek to improve these emission modules by combining state-of-the-art process modules for emissions, available data, and new optimization methods. In earth science problems, we often have substantial data and knowledge of processes in disparate systems, and thus we need to combine data and a general process level understanding into a model for projections of future climate that are as accurate as possible. The best methodologies for optimization of parameters in earth system models are still being developed. In this proposal we will develop and apply surrogate algorithms that a) were especially developed for computationally expensive simulations like CLM-ME/N models; b) were (in the earlier surrogate optimization Stochastic RBF) demonstrated to perform very well on computationally expensive complex partial differential equations in earth science with limited numbers of simulations; and, c) will be (as part of the proposed research) significantly improved both by adding asynchronous parallelism, early truncation of unsuccessful simulations, and the improvement of both serial and parallel performance by the use of derivative and sensitivity information from global and local surrogate approximations S(x) . The algorithm development and testing will be focused on the CLM-ME/N model application, but the methods are general and are expected to also perform well on optimization for parameter estimation of other climate models and other classes of continuous multimodal optimization problems arising from complex simulation models. In addition, this proposal will compile available datasets of emissions of methane, nitrous oxides and reactive nitrogen species and develop protocols for site level comparisons with the CLM-ME/N. Once the model parameters are optimized against site level data, the model will be simulated at the global level and compared to atmospheric concentration measurements for the current climate, and future emissions will be estimated using climate change as simulated by the CESM. This proposal combines experts in earth system modeling, optimization, computer science, and process level understanding of soil gas emissions in an interdisciplinary team in order to improve the modeling of methane and nitrogen gas emissions. This proposal thus meets the requirements of the SciDAC RFP, by integrating state-of-the-art computer science and earth system to build an improved earth system model.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

A Scalable Solution Framework for Stochastic Transmission and Generation Planning Problems. Draft

Current commercial software tools for transmission and generation investment planning have limited stochastic modeling capabilities. Because of this limitation, electric power utilities generally rely on scenario planning heuristics to identify potentially robust and cost effective investment plans for a broad range of system, economic, and policy conditions. Several research studies have shown that stochastic models perform significantly better than deterministic or heuristic approaches, in terms of overall costs. However, there is a lack of practical solution approaches to solve such models. In this paper we propose a scalable decomposition algorithm to solve stochastic transmission and generation planning problems, respectively considering discrete and continuous decision variables for transmission and generation investments. Given stochasticity restricted to loads and wind, solar, and hydro power output, we develop a simple scenario reduction framework based on a clustering algorithm, to yield a more tractable model. The resulting stochastic optimization model is decomposed on a scenario basis and solved using a variant of the Progressive Hedging (PH) algorithm. We perform numerical experiments using a 240-bus network representation of the Western Electricity Coordinating Council in the US. Although convergence of PH to an optimal solution is not guaranteed for mixed-integer linear optimization models, we find that it is possible to obtain solutions with acceptable optimality gaps for practical applications. Our numerical simulations are performed both on a commodity workstation and on a high-performance cluster. The results indicate that large-scale problems can be solved to a high degree of accuracy in at most two hours of wall clock time.

24 POWER TRANSMISSION AND DISTRIBUTION↗