SEARCH · Search NASA

Results for “Simulation optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44

All sources NASA NTRS DOE OSTI

DOE OSTI2020

P-model v1.0: an optimality-based light use efficiency model for simulating ecosystem gross primary production

Terrestrial photosynthesis is the basis for vegetation growth and drives the land carbon cycle. Accurately simulating gross primary production (GPP, ecosystem-level apparent photosynthesis) is key for satellite monitoring and Earth system model predictions under climate change. While robust models exist for describing leaf-level photosynthesis, predictions diverge due to uncertain photosynthetic traits and parameters which vary on multiple spatial and temporal scales. Here, we describe and evaluate a GPP (photosynthesis per unit ground area) model, the P-model, that combines the Farquhar–von Caemmerer–Berry model for C 3 photosynthesis with an optimality principle for the carbon assimilation–transpiration trade-off, and predicts a multi-day average light use efficiency (LUE) for any climate and C 3 vegetation type. The model builds on the theory developed in Prentice et al. (2014) and Wang et al. (2017a) and is extended to include low temperature effects on the intrinsic quantum yield and an empirical soil moisture stress factor. The model is forced with site-level data of the fraction of absorbed photosynthetically active radiation (fAPAR) and meteorological data and is evaluated against GPP estimates from a globally distributed network of ecosystem flux measurements. Although the P-model requires relatively few inputs, the R 2 for predicted versus observed GPP based on the full model setup is 0.75 (8 d mean, 126 sites) – similar to comparable satellite-data-driven GPP models but without predefined vegetation-type-specific parameters. The R 2 is reduced to 0.70 when not accounting for the reduction in quantum yield at low temperatures and effects of low soil moisture on LUE. The R 2 for the P-model-predicted LUE is 0.32 (means by site) and 0.48 (means by vegetation type). Applying this model for global-scale simulations yields a total global GPP of 106–122 Pg C yr –1 (mean of 2001–2011), depending on the fAPAR forcing data. The P-model provides a simple but powerful method for predicting – rather than prescribing – light use efficiency and simulating terrestrial photosynthesis across a wide range of conditions. The model is available as an R package (rpmodel).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Neutral beams for the burning plasma era: Simulations and modeling of deuterium ion beams for plasma heating (CRADA Final Report)

Together with Realta Fusion, we set up several simulation scenarios and ran simulations using the WarpX code which is developed by LBNL and especially by members of the LBNL team. We investigated beam transport and beam neutralization and ran optimization and benchmark simulations. To achieve this, several new features were added to the warpX code. These code additions will also be beneficial to other users of warpX including several fusion companies and researchers at National Laboratories. Furthermore, we supported Realta Fusion in setting up their own instances of WarpX. We presented our results at the American Physical Society Division of Plasma Physics meeting and are in the process of writing a publication that will be submitted to a peer-reviewed journal to share our results with the broader community.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

NASA NTRS2015

Strategic Planning of Efficient Oceanic Flights

The efficiency of oceanic flights is low due to limited navigational and communication equipment, congestion and airspace restrictions. The availability of Automated Dependent Surveillance-Broadcast (ADS-B) and other improvements provides opportunity for better strategic planning of trajectories. Transatlantic flights between US and Europe constitute one of the busiest oceanic airspace regions in the world. This paper examines the benefits of a wind-optimal trajectory concept with a strategic de-confliction component compared to the current flight planning using the North Atlantic Tracks. The methodology generates a wind-optimal route for each aircraft and a strategic reduction in the potential conflicts between aircraft by a combination of small adjustments to departure times and rerouting. The de-confliction is achieved by optimization techniques involving simulated annealing with local gradient searching. The fuel burn for the tracks in today's Organized Track System are compared with the corresponding quantities for the wind-optimized routes to evaluate the potential benefits of flying wind-optimal routes in North Atlantic Airspace. The analysis is based on air traffic between US and Europe during July 2012. The potential fuel savings depend on existing inefficiencies in current flight plans, atmospheric conditions and location of the city-pairs. The paper provides both aggregate results and detailed examination of some of the most popular city-pairs. Results show that strategic planning can improve the efficiency of flight trajectories by 3 to 5 depending on city-pairs and aircraft type. This translates into a potential fuel savings in the range of (420-970) kg per flight for a Boeing 767-300, the most widely used aircraft between the city-pairs in this study.

oceanic operations↗

DOE OSTI2026

Optimization of an Energy Tuning Assembly for High Explosives Detection

The Portable Isotopic Neutron Spectroscopy (PINS) system, employs neutron-induced gamma-ray spectroscopy and provides a nondestructive method for high explosives detection. In standard operation it uses Californium-252 as a neutron source. Operating PINS with a deuterium-tritium (DT) neutron generator has some advantages over Cf-252, including lifetime and ability to produce high-energy inelastic scattering gamma rays. However, current systems using DT neutron generators suffer from a high environmental background and reduced ability to induce neutron capture, reducing spectral quality and limiting nitrogen sensitivity. Here, this study presents the development of an energy-tuning assembly (ETA) designed to optimize the DT neutron energy spectrum to increase nitrogen reaction rates in a target, thereby improving high explosive detection capabilities. A metaheuristic optimization framework, MultiGNOWEE, coupled with MCNP, was employed to generate two ETA configurations: a single-objective ETA, which maximizes nitrogen capture reactions, and a multi-objective ETA, which balances neutron capture and inelastic scattering. Simulations demonstrated the optimized configurations achieved up to a 10-fold improvement in nitrogen capture rates compared to the bare configuration. Experimental validation was conducted using a DT neutron generator and a high-purity germanium (HPGe) detector. Two prototype ETAs were constructed and assessed on a melamine simulant. Measurements demonstrated improved nitrogen detection for both prototype ETA configurations when compared to the standard system.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2002

Overall Traveling-Wave-Tube Efficiency Improved By Optimized Multistage Depressed Collector Design

Depressed Collector Design The microwave traveling wave tube (TWT) is used widely for space communications and high-power airborne transmitting sources. One of the most important features in designing a TWT is overall efficiency. Yet, overall TWT efficiency is strongly dependent on the efficiency of the electron beam collector, particularly for high values of collector efficiency. For these reasons, the NASA Glenn Research Center developed an optimization algorithm based on simulated annealing to quickly design highly efficient multistage depressed collectors (MDC's). Simulated annealing is a strategy for solving highly nonlinear combinatorial optimization problems. Its major advantage over other methods is its ability to avoid becoming trapped in local minima. Simulated annealing is based on an analogy to statistical thermodynamics, specifically the physical process of annealing: heating a material to a temperature that permits many atomic rearrangements and then cooling it carefully and slowly, until it freezes into a strong, minimum-energy crystalline structure. This minimum energy crystal corresponds to the optimal solution of a mathematical optimization problem. The TWT used as a baseline for optimization was the 32-GHz, 10-W, helical TWT developed for the Cassini mission to Saturn. The method of collector analysis and design used was a 2-1/2-dimensional computational procedure that employs two types of codes, a large signal analysis code and an electron trajectory code. The large signal analysis code produces the spatial, energetic, and temporal distributions of the spent beam entering the MDC. An electron trajectory code uses the resultant data to perform the actual collector analysis. The MDC was optimized for maximum MDC efficiency and minimum final kinetic energy of all collected electrons (to reduce heat transfer). The preceding figure shows the geometric and electrical configuration of an optimized collector with an efficiency of 93.8 percent. The results show the improvement in collector efficiency from 89.7 to 93.8 percent, resulting in an increase of three overall efficiency points. In addition, the time to design a highly efficient MDC was reduced from a month to a few days. All work was done in-house at Glenn for the High Rate Data Delivery Program. Future plans include optimizing the MDC and TWT interaction circuit in tandem to further improve overall TWT efficiency.

Vaden, Karl R.↗

DOE OSTI2020

Co-optimizing water-alternating-carbon dioxide injection projects using a machine learning assisted computational framework

In this report, a robust machine-learning-based computational framework that couples multi-layer neural network (MLNN) proxies and a multi-objective particle swarm optimizer (MOPSO) to design water-alternating-carbon dioxide injection (CO 2 -WAG) projects is presented. The proposed optimization protocol considers various objectives, including oil recovery and CO 2 storage volume. Expert MLNN systems are trained and employed as surrogate models of the high-fidelity compositional simulator in the optimization workflow. When multiple objective functions are considered, two approaches are employed to treat the objectives: the weighted sum method and the Pareto-front-based scheme. A field-scale implementation focusing on tertiary recovery in the Morrow B formation at Farnsworth Unit (FWU) is presented. The developed Pareto-optimal solutions indicate the maximal available oil production can be 1.64 × 10 7 barrels and maximal carbon storage can achieve 2.35 × 10 7 tons. Trade-offs factor is defined to divide the constructed Pareto front into 4 sections with the trade-off factors’ value ranges from 0.35 to 49.9. This work also compares the optimum solution found by the aggregative objective function and the solution repository covered by the Pareto front that considers the physical and operational constraints and reduces uncertainties involved by the multi-objective optimization process. Our comparison indicates multiple solutions exist to satisfy the objective criteria of the WAG design, and these results cannot be found using the traditional weighted sum method. The Pareto front solution can provide more options for project designers, but decisions regarding necessary trade-offs must be made using the solution repository to balance the project economics and CO 2 storage amount.

10 SYNTHETIC FUELS↗

DOE OSTI2021

Structural stability and artificial buckling modes in topology optimization

Abstract This paper demonstrates how a strain energy transition approach can be used to remove artificial buckling modes that often occur in stability constrained topology optimization problems. To simulate the structural response, a nonlinear large deformation hyperelastic simulation is performed, wherein the fundamental load path is traversed using Newton’s method and the critical buckling load levels are estimated by an eigenvalue analysis. The goal of the optimization is to minimize displacement, subject to constraints on the lowest critical buckling loads and maximum volume. The topology optimization problem is regularized via the Helmholtz PDE-filter and the method of moving asymptotes is used to update the design. The stability and sensitivity analyses are outlined in detail. The effectiveness of the energy transition scheme is demonstrated in numerical examples.

Dalklint, Anna (ORCID:0000000346195205)↗

NASA NTRS1971

Optimization of life support systems and their systems reliability

The identification, analysis, and optimization of life support systems and subsystems have been investigated. For each system or subsystem that has been considered, the procedure involves the establishment of a set of system equations (or mathematical model) based on theory and experimental evidences; the analysis and simulation of the model; the optimization of the operation, control, and reliability; analysis of sensitivity of the system based on the model; and, if possible, experimental verification of the theoretical and computational results. Research activities include: (1) modeling of air flow in a confined space; (2) review of several different gas-liquid contactors utilizing centrifugal force: (3) review of carbon dioxide reduction contactors in space vehicles and other enclosed structures: (4) application of modern optimal control theory to environmental control of confined spaces; (5) optimal control of class of nonlinear diffusional distributed parameter systems: (6) optimization of system reliability of life support systems and sub-systems: (7) modeling, simulation and optimal control of the human thermal system: and (8) analysis and optimization of the water-vapor eletrolysis cell.

Fan, L. T.↗

NASA NTRS1987

Performance analysis of an optimal file transfer protocol for integrated mobile satellite services

A file-transfer protocol based on the selective-repeat automatic-retransmission-request (ARQ) scheme has been developed for the Mobile Satellite Experiment (MSAT-X). In this protocol, each file is divided into packets, which are error-detected and acknowledge individually. A file can be transferred in either the open- or closed-end connection. In the open-end connection, the file is transmitted continuously according to the selective-repeat scheme until all packets are positively acknowledged; in the closed-end connection, each retransmission will initiate its own connection request. The relationship between the packet size and the average end-to-end delay in the mobile fading environment is investigated. The communications channel uses trellis-coded modulation with symbol interleaving, packet replication, and error detection. The packet error performance in such a channel is evaluated by simulation, and the optimal packet size for various operating conditions is derived, using the simulated results. It is shown that for a symbol-energy-to-noise ratio of 12 dB, each file should be divided into packets of length between 256 bits and 512 bits (excluding the overhead) to minimize the average end-to-end delay.

Wang, Charles C.↗

DOE OSTI2018

Use of Physics Based Management Optimization for Groundwater Remedy Design - 18671

Numerical modeling coupled with formal mathematical optimization techniques was used to design a groundwater remedy strategy to address a carbon tetrachloride (CCl{sub 4}) plume covering several square miles. The optimization approach, methodology, and project costs are described. The site consists of a large grain elevator in Nebraska. CCl{sub 4} was used as a fumigant and subsequently impacted soils and groundwater. Remedy design focused on the more diffuse plume, which extended approximately 4.5 miles down gradient of the source area. An eleven-layer model was developed and calibrated to observed groundwater conditions. This model was linked with formal optimization techniques to identify the best design capable of meeting client specified requirements and well location limitations. The planned system included requirements on the location of extraction and injection wells, total system flow rate, number of management periods, and specific constraints limiting further plume expansion. A separate trinitrotoluene (TNT) contaminant plume from a third party was located vertically above the leading edge of the CCl{sub 4} plume within the same aquifer. A further constraint imposed was that the design could not affect or cause additional migration of the TNT plume. Optimal system design was determined using HydroGeoLogic's Physics Based Management Optimization (PBMO{sup TM}) system. Over 38,000 candidate solutions consisting of different combinations of pumping and injection wells locations and rates, were evaluated through flow modeling coupled with either particle tracking or fully convective-dispersive transport simulations to determine the optimal design. Planning for installation of the optimal design is underway. Simulations were completed on a distributed computing platform consisting of 3 Quad Core PCs. Flow and particle tracking simulations required less than 2 minutes per simulation. Coupled flow and transport simulation run times were generally between 30-40 minutes per simulation. The total costs for developing the groundwater flow model and completing the subsequent optimal design were approximately $400,000. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2024

Optimal Management of Grid-Interactive Efficient Buildings via Safe Reinforcement Learning

Reinforcement learning (RL)-based methods have achieved significant success in managing grid-interactive efficient buildings (GEBs). However, RL does not carry intrinsic guarantees of constraint satisfaction, which may lead to severe safety consequences. Besides, in GEB control applications, most existing safe RL approaches rely only on the regularisation parameters in neural networks or penalty of rewards, which often encounter challenges with parameter tuning and lead to catastrophic constraint violations. To provide enforced safety guarantees in controlling GEBs, this paper designs a physics-inspired safe RL method whose decision-making is enhanced through safe interaction with the environment. Different energy resources in GEBs are optimally managed to minimize energy costs and maximize customer comfort. The proposed approach can achieve strict constraint guarantees based on prior knowledge of a set of developed hard steady-state rules. Simulations on the optimal management of GEBs, including heating, ventilation, and air conditioning (HVAC), solar photovoltaics, and energy storage systems, demonstrate the effectiveness of the proposed approach.

Huo, Xiang↗

DOE OSTI2025

Clifford Circuit-Based Heuristic Optimization of Fermion-To-Qubit Mappings

Simulation of interacting Fermionic Hamiltonians is one of the most promising applications of quantum computers. However, the feasibility of analyzing Fermionic systems with a quantum computer hinges on the efficiency of Fermion-to-qubit mappings that encode nonlocal Fermionic degrees of freedom in local qubit degrees of freedom. While recent studies have highlighted the importance of designing Fermion-to-qubit mappings that are tailored to specific problem Hamiltonians, the methods proposed so far either are restricted to a narrow class of mappings or they use computationally expensive and unscalable brute-force search algorithms. Here, in this work, we address this challenge by designing a heuristic numerical optimization framework for Fermion-to-qubit mappings. To this end, we first translate the Fermion-to-qubit mapping problem to a Clifford circuit optimization problem and then use simulated annealing to optimize the average Pauli weight of the problem Hamiltonian. For all Fermionic Hamiltonians we have considered, the numerically optimized mappings outperform their conventional counterparts, including ternary-tree-based mappings that are known to be optimal for single creation and annihilation operators. We find that our optimized mappings yield between 15% and 40% improvements on the average Pauli weight when the simulation Hamiltonian has an intermediate level of complexity. Most remarkably, the optimized mappings improve the average Pauli weight for 6 × 6 nearest-neighbor hopping and Hubbard models by more than 40% and 20%, respectively. Surprisingly, we also find specific interaction Hamiltonians for which the optimized mapping outperforms any ternary-tree-based mapping. Our results establish heuristic numerical optimization as an effective method for obtaining mappings tailored for specific Fermionic Hamiltonian.

Hamiltonians↗

NASA NTRSDate not supplied

Computational Optimization of a Rotary Valved Pulse Combustor Concept

A resonant pulse combustor valve concept is introduced that utilizes two slotted, coaxial counterrotating discs for mechanical actuation at the combustor inlet. The intended test article for prototype demonstration is a small, 22 in. long, propane fueled laboratory combustor, flowing approximately 0.006 lbm/s of air. The objective is to develop an externally actuated (i.e., active) valve that yields better performance and longer life than the traditional internally actuated (i.e., passive), reed-type valve found on most pulse combustors. The rotary valve motion is optimized using an axisymmetric, two-dimensional computational fluid dynamic simulation with a domain that includes the valve as a moveable interior wall. Parameters such as slew rate, dwell period in the open position, and total closed period are varied using fuel specific impulse as the figure of merit. Variations in fuel injector location and air fuel ratio are also examined. Additionally, the performance impact of leakage from the rotary valve is quantified since leakage is endemic to the design. The optimized simulation results indicate that the rotary valve concept can deliver the desired performance attributes using disc rotational speeds and stresses that are well within the realm of modern materials. A preliminary mechanical valve design is included in the report.

pressure gain combustion↗

NASA NTRSDate not supplied

Computational Optimization of a Rotary Valved Pulse Combustor Concept

pressure gain combustion↗

DOE OSTI2021

Noise and error analysis and optimization in particle-based kinetic plasma simulations

In this paper we analyze the noise in macro-particle methods used in plasma physics and fluid dynamics, leading to approaches for minimizing the total error, focusing on electrostatic models in one dimension. We begin by describing kernel density estimation for continuous values of the spatial variable x, expressing the kernel in a form in which its shape and width are represented separately. The covariance matrix of the noise in the density is computed, first for uniform true density. The bandwidth of the covariance matrix C(x,y) is related to the width of the kernel. A feature that stands out is the presence of constant negative terms in the elements of the covariance matrix both on and off-diagonal. These negative correlations are related to the fact that the total number of particles is fixed at each time step; they also lead to the property ∫C(x,y)dy = 0. We investigate the effect of these negative correlations on the electric field computed by Gauss's law, finding that the noise in the electric field is related to a process called the Ornstein-Uhlenbeck bridge, leading to a covariance matrix of the electric field with variance significantly reduced relative to that of a Brownian process. For non-constant density, p(x), still with continuous x, we analyze the total error in the density estimation and discuss it in terms of bias-variance optimization (BVO). For some characteristic length l, determined by the density and its second derivative, and kernel width h, having too few particles within h leads to too much variance; for h that is large relative to l, there is too much smoothing of the density. The optimum between these two limits is found by BVO. For kernels of the same width, it is shown that this optimum (minimum) is weakly sensitive to the kernel shape. Next, we repeat the analysis for x discretized on a grid. In this case the charge deposition rule is determined by a particle shape. An important property to be respected in the discrete system is the exact preservation of total charge on the grid; this property is necessary to ensure that the electric field is equal at both ends, consistent with periodic boundary conditions. We find that if the particle shapes satisfy a partition of unity property, the particle charge deposited on the grid is conserved exactly. Further, if the particle shape is expressed as the convolution of a kernel with another kernel that satisfies the partition of unity, then the particle shape obeys the partition of unity. This property holds for kernels of arbitrary width, including widths that are not integer multiples of the grid spacing. Furthermore, we show results relaxing the approximations used to do BVO optimization analytically, by doing numerical computations of the total error as a function of the kernel width, on a grid in x. The comparison between numerical and analytical results shows good agreement over a range of particle shapes. We discuss the practical implications of our results, including the criteria for design and implementation of computationally efficient particle shapes that take advantage of the developed theory.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2022

SNICAR-ADv4: a physically based radiative transfer model to represent the spectral albedo of glacier ice

Abstract. Accurate modeling of cryospheric surface albedo is essential for our understanding of climate change as snow and ice surfaces regulate the global radiative budget and sea-level through their albedo and mass balance. Although significant progress has been made using physical principles to represent the dynamic albedo of snow, models of glacier ice albedo tend to be heavily parameterized and not explicitly connected with physical properties that govern albedo, such as the number and size of air bubbles, specific surface area (SSA), presence of abiotic and biotic light absorbing constituents (LACs), and characteristics of any overlying snow. Here, we introduce SNICAR-ADv4, an extension of the multi-layer two-stream delta-Eddington radiative transfer model with the adding–doubling solver that has been previously applied to represent snow and sea-ice spectral albedo. SNICAR-ADv4 treats spectrally resolved Fresnel reflectance and transmittance between overlying snow and higher-density glacier ice, scattering by air bubbles of varying sizes, and numerous types of LACs. SNICAR-ADv4 simulates a wide range of clean snow and ice broadband albedo (BBA), ranging from 0.88 for (30 µm) fine-grain snow to 0.03 for bare and bubble-free ice under direct light. Our results indicate that representing ice with a density of 650 kg m−3 as snow with no refractive Fresnel layer, as done previously, generally overestimates the BBA by an average of 0.058. However, because most naturally occurring ice surfaces are roughened “white ice”, we recommend modeling a thin snow layer over bare ice simulations. We find optimal agreement with measurements by representing cryospheric media with densities less than 650 kg m−3 as snow and larger-density media as bubbly ice with a Fresnel layer. SNICAR-ADv4 also simulates the non-linear albedo impacts from LACs with changing ice SSA, with peak impact per unit mass of LACs near SSAs of 0.1–0.01 m2 kg−1. For bare, bubble-free ice, LACs actually increase the albedo. SNICAR-ADv4 represents smooth transitions between snow, firn, and ice surfaces and accurately reproduces measured spectral albedos of a variety of glacier surfaces. This work paves the way for adapting SNICAR-ADv4 to be used in land ice model components of Earth system models.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Optimizing Geant4 Hadronic Models

Geant4, the leading detector simulation toolkit used in high energy physics, employs a set of physics models to simulate interactions of particles with matter across a wide range of energies. These models, especially the hadronic ones, rely largely on directly measured cross-sections and inclusive characteristics, and use physically motivated parameters. However, they generally aim to cover a broad range of possible simulation tasks and may not always be optimized for a particular process or a given material. The Geant4 collaboration recently made many parameters of the models accessible via a configuration interface. This opens a possibility to fit simulated distributions to the thin target experimental datasets and extract optimal values of the model parameters and the associated uncertainties. Such efforts are currently undertaken by the Geant4 collaboration with the goal of offering alternative sets of model parameters, also known as "tunes", for certain applications. The effort should subsequently lead to more accurate estimates of the systematic errors in physics measurements given the detector simulation role in performing the physics measurements. Results of the study are presented to illustrate how Geant4 model parameters can be optimized through applying fitting techniques, to improve the agreement between the Geant4 and the experimental data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Microgrid energy scheduling under uncertain extreme weather: Adaptation from parallelized reinforcement learning agents

Microgrids are useful solutions for integrating renewable energy resources and providing seamless green electricity to minimize carbon footprint. In recent years, extreme weather events happened often worldwide and caused significant economic and societal losses. Such events bring uncertainties to the microgrid energy scheduling problems and increase the challenges of microgrid operation. Traditional optimization approaches suffer from the inaccuracy of the uncertain microgrid model and the unseen events. Existing reinforcement learning (RL) - based approaches are also hampered by the limited generalization and the increasing computational burden when stochastic formulations are required to accommodate the uncertainties. This paper proposes a new parallelized reinforcement learning (PRL) method based on the probabilistic events to handle the microgrid energy uncertainties. Specifically, several local learning agents are employed to interact with pertinent microgrid environments in a distributed manner and report outcomes to the global agent, which will optimize microgrid energy resources online during extreme events. The stochastic microgrid energy optimization problem is reformulated to include all possible scenarios with probabilities. The advantage estimate functions of learning agents are designed with a backward sweep to transfer the outcomes to the value function updating process. Two simulation studies, stochastic optimization and online testing, are performed to compare with several existing RL approaches. Results substantiate that the proposed PRL method can achieve up to 20% optimization performance improvement with 4 and 28 times less computation cost than Q-learning with experience replay and multi-agent Q-learning approaches, respectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗