Search NASA⌕ Search

SEARCH · Search NASA

Results for “Policy optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Joint routing of conventional and range-extended electric vehicles in a large metropolitan network

Range-extended electric vehicles combine the higher efficiency and environmental benefits of battery-powered electric motors with the longer mileage and autonomy of conventional internal combustion engines. This combination is particularly advantageous for time-constrained delivery routing in dense urban areas, where battery recharging along routes can be too time-consuming to economically justify the use of all-electric vehicles. However, switching from electric to conventional fossil fuel modes also results in higher costs and emissions and lower efficiency. This paper analyzes this heterogeneous vehicle routing problem and describes two solution methods: an exact branch-price-and-cut algorithm and an iterated tabu search metaheuristic. From a methodological perspective, we find that the exact algorithm consistently obtains tight lower bounds that also serve to certify the metaheuristic solutions as near-optimal. From a policy standpoint, we examine a large-scale real-world case study concerning parcel deliveries in the Chicago metropolitan area and quantify various operational metrics including energy costs and vehicle miles traveled. We find that by deploying roughly 20% of range -extended vehicles with a modest all-electric range of 33 miles, parcel distributors can save energy costs by up to 17% while incurring less than 0.5% increase in vehicle miles traveled. Increasing the range to 60 miles further reduces costs by only 4%, which can alternatively be achieved by decreasing the average service time by 1 minute or increasing driver working time by 1 hour. In conclusion, our study reveals several key areas of improvement on which vehicle manufacturers, distributors, and policy makers can focus their attention.

33 ADVANCED PROPULSION SYSTEMS↗

Optimizing Individualized Treatment Planning for Parkinson’s Disease Using Deep Reinforcement Learning

More than one million people currently live with Parkinson's Disease (PD) in the U.S. alone. Medications, such as levodopa, can help manage PD symptoms. However, medication treatment planning is generally based on patient history and limited interaction between physicians and patients during office visits. This limits the extent of benefit that may be derived from the treatment as disease/patient characteristics are generally non-stationary. Wearable sensors that provide continuous monitoring of various symptoms, such as bradykinesia and dyskinesia, can enhance symptom management. However, using such data to overhaul the current static medication treatment planning approach and prescribe personalized medication timing and dosage that accounts for patient/care-giver/physician feedback/preferences remains an open question. We develop a model to prescribe timing and dosage of medications, given the motor fluctuation data collected using wearable sensors in real-time. We solve the resulting model using deep reinforcement learning (DRL). The prescribed policy determines the optimal treatment plan that minimizes patient's symptoms. Our results show that the model-prescribed policy outperforms the static a priori treatment plan in improving patients' symptoms, providing a proof-of-concept that DRL can augment medical decision making for treatment planning of chronic disease patients.

Watts, Jeremy↗

REopt: A Platform for Energy System Integration and Optimization: Preprint

REopt is NREL's energy planning platform offering concurrent, multi-technology integration and optimization capabilities to help clients meet their cost savings and energy performance goals. The REopt platform provides techno-economic decision-support analysis throughout the energy planning process, from agency-level screening and macro planning to project development to energy asset operation. REopt employs an integrated approach to optimizing a site?s energy costs by considering electricity and thermal consumption, resource availability, complex tariff structures including time-of-use, demand and sell-back rates, incentives, net-metering, and interconnection limits. Formulated as a mixed integer linear program, REopt recommends an optimally-sized mix of conventional and renewable energy, and energy storage technologies; estimates the net present value associated with implementing those technologies; and provides the cost-optimal dispatch strategy for operating them at maximum economic efficiency. The REopt platform can be customized to address a variety of energy optimization scenarios including policy, microgrid, and operational energy applications. This paper presents the REopt techno-economic model along with two examples of recently completed analysis projects.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Multi-scale, Multi-disciplinary, and Multi-agent Explainable AI with Koopman-Undergirded Learning, Prediction, and Analysis (M3EA KULPA) (Project Closeout Report)

The goal of this project was to develop and use domain-aware machine learning formulations, based on the Koopman Operator (KO), for modelling multi-scale, multi-disciplinary (e.g., multi-physics), and/or multi-agent systems. The project developed these formulations for the following cases: • Systems with dynamics at two separate time scales, • Systems with a bi-level hierarchical control structure, • Systems with bi-level hierarchical control and dynamics at two separate time scales (the lower level controls operating at the faster time scale), and • Systems with n separate but interacting agents/disciplines (with/without control, respectively); the controls for each agent could include bi-level hierarchical control and dynamics at two separate time scales as described above. The project then defined a set of dynamical systems consisting of different nonlinear oscillators that could be used to test these different formulations and then subsequently learned the KO models for those systems. With the KO models, we were able to do the following: • Quantify system stability, including both long-term and transient behavior, • Quantify the effects of feedbacks between the different time scales and agents/disciplines in terms of those feedbacks’ effects on system stability, • Replace a standard Proportional-Integral (PI) control in the hierarchical control structure with a KO-based Linear-Quadratic Regular (LQR), a form of optimal control, • Calculate optimal supervisory control policies a) with and without time scale separated dynamics at the lower level control levels and b) with both PI and KO-based LQR lower level control policies, and • Calculate dynamic Nash equilibria for multi-agent systems where each agent makes its own control decisions.

97 MATHEMATICS AND COMPUTING↗

Hamiltonian switching control of noisy bipartite qubit systems

Abstract We develop a Hamiltonian switching ansatz for bipartite control that is inspired by the quantum approximate optimization algorithm, to mitigate environmental noise on qubits. We demonstrate the control for a central spin coupled to bath spins via isotropic Heisenberg interactions, and then make physical applications to the protection of quantum gates performed on superconducting transmon qubits coupling to environmental two-level-systems (TLSs) through dipole-dipole interactions, as well as on such qubits coupled to both TLSs and a Lindblad bath. The control field is classical and acts only on the system qubits. We use reinforcement learning with policy gradient to optimize the Hamiltonian switching control protocols, using a fidelity objective for specific target quantum gates. We use this approach to demonstrate effective suppression of both coherent and dissipative noise, with numerical studies achieving target gate implementations with fidelities over 0.9999 (four nines) in the majority of our test cases and showing improvement beyond this to values of 0.999 999 999 (nine nines) upon a subsequent optimization by GRadient Ascent Pulse Engineering (GRAPE). We analyze how the control depth, total evolution time, number of environmental TLS, and choice of optimization method affect the fidelity achieved by the optimal protocols and reveal some critical behaviors of bipartite control of quantum gates.

Physics↗

Joint optimization of land carbon uptake and albedo can help achieve moderate instantaneous and long-term cooling effects

Abstract Both carbon dioxide uptake and albedo of the land surface affect global climate. However, climate change mitigation by increasing carbon uptake can cause a warming trade-off by decreasing albedo, with most research focusing on afforestation and its interaction with snow. Here, we present carbon uptake and albedo observations from 176 globally distributed flux stations. We demonstrate a gradual decline in maximum achievable annual albedo as carbon uptake increases, even within subgroups of non-forest and snow-free ecosystems. Based on a paired-site permutation approach, we quantify the likely impact of land use on carbon uptake and albedo. Shifting to the maximum attainable carbon uptake at each site would likely cause moderate net global warming for the first approximately 20 years, followed by a strong cooling effect. A balanced policy co-optimizing carbon uptake and albedo is possible that avoids warming on any timescale, but results in a weaker long-term cooling effect.

54 ENVIRONMENTAL SCIENCES↗

A hybrid inventory policy with split delivery under regular and surge demand

This paper proposes a hybrid inventory policy with split delivery under regular demand and surge demand. The combination of regular demand and surge demand can be observed in many areas, such as healthcare inventory and humanitarian supply chain management. The arrival rate for regular demand is typically higher than the arrival rate of surge demand; whereas, the volume of a regular demand is typically lower than the volume of a surge demand. This paper proposes an inventory management model that considers both emergency and regular replenishment corresponding to both demand patterns. The equilibrium equations developed for this model are based on the level crossing theory. This paper finally presents a search-based heuristics that can be used to identify near optimal inventory management policies. Numerical results show that proposed hybrid inventory policy with split delivery outperforms similar hybrid inventory policy without split delivery.

97 MATHEMATICS AND COMPUTING↗

I/O-aware bandwidth allocation for petascale computing systems

In the Big Data era, the gap between the storage performance and an appli- cation's I/O requirement is increasing. I/O congestion caused by concurrent storage accesses from multiple applications is inevitable and severely harms the performance. Conventional approaches either focus on optimizing an ap- plication's access pattern individually or handle I/O requests on a low-level storage layer without any knowledge from the upper-level applications. In this paper, we present a novel I/O-aware bandwidth allocation framework to coordinate ongoing I/O requests on petascale computing systems. The motivation behind this innovation is that the resource management system has a holistic view of both the system state and jobs' activities and can dy- namically control the jobs' status or allocate resource on the y during their execution. We treat a job's I/O requests as periodical subjobs within its lifecycle and transform the I/O congestion issue into a classical scheduling problem. Based on this model, we propose a bandwidth management mech- anism as an extension to the existing scheduling system. We design several bandwidth allocation policies with different optimization objectives either on user-oriented metrics or system performance. We conduct extensive trace- based simulations using real job traces and I/O traces from a production IBM Blue Gene/Q system at Argonne National Laboratory. Experimental results demonstrate that our new design can improve job performance by more than 30%, as well as increasing system performance.

I/O congestion↗

Maintaining environmental quality while expanding biomass production: Sub-regional U.S. policy simulations

This paper evaluates environmental policy effects on ligno-cellulosic biomass production and environ- mental outcomes using an integrated bioeconomic optimization model. The environmental policy integrated climate (EPIC) model is used to simulate crop yields and environmental indicators in current and future potential bioenergy cropping systems based on weather, topographic and soil data. The crop yield and environmental outcome parameters from EPIC are combined with biomass transport costs and economic parameters in a representative farmer profit-maximizing mathematical optimization model. The model is used to predict the impact of alternative policies on biomass production and environmental outcomes. We find that without environmental policy, rising biomass prices initially trigger production of annual crop residues, resulting in increased greenhouse gas emissions, soil erosion, and nutrient losses to surface and ground water. At higher biomass prices, perennial bioenergy crops replace annual crop residues as biomass sources, resulting in lower environmental impacts. Simulations of three environmental policies namely a carbon price, a no-till area subsidy, and a fertilizer tax reveal that only the carbon price policy systematically mitigates environmental impacts. The fertilizer tax is ineffectual and too costly to farmers. The no-till subsidy is effective only at low biomass prices and is too costly to government.

Egbendewe-Mondzozo, Aklesso↗

Toward Accelerating Discovery via Physics-Driven and Interactive Multifidelity Bayesian Optimization

Both computational and experimental material discovery bring forth the challenge of exploring multidimensional and often nondifferentiable parameter spaces, such as phase diagrams of Hamiltonians with multiple interactions, composition spaces of combinatorial libraries, processing spaces, and molecular embedding spaces. Often these systems are expensive or time consuming to evaluate a single instance, and hence classical approaches based on exhaustive grid or random search are too data intensive. This resulted in strong interest toward active learning methods such as Bayesian optimization (BO) where the adaptive exploration occurs based on human learning (discovery) objective. However, classical BO is based on a predefined optimization target, and policies balancing exploration and exploitation are purely data driven. In practical settings, the domain expert can pose prior knowledge of the system in the form of partially known physics laws and exploration policies often vary during the experiment. Here, we propose an interactive workflow building on multifidelity BO (MFBO), starting with classical (data-driven) MFBO, then expand to a proposed structured (physics-driven) structured MFBO (sMFBO), and finally extend it to allow human-in-the-loop interactive interactive MFBO (iMFBO) workflows for adaptive and domain expert aligned exploration. These approaches are demonstrated over highly nonsmooth multifidelity simulation data generated from an Ising model, considering spin–spin interaction as parameter space, lattice sizes as fidelity spaces, and the objective as maximizing heat capacity. Detailed analysis and comparison show the impact of physics knowledge injection and real-time human decisions for improved exploration with increased alignment to ground truth. Here, the associated notebooks allow to reproduce the reported analyses and apply them to other systems.

97 MATHEMATICS AND COMPUTING↗

A TTL-based Approach for Content Placement in Edge Networks

Edge networks are promising to provide better services to users by provisioning computing and storage resources at the edge of networks. However, due to the uncertainty and diversity of user interests, content popularity, distributed network structure, cache sizes, it is challenging to decide where to place the content, and how long it should be cached. In this paper, we study the utility optimization of content placement at edge networks through timer-based (TTL) policies. We propose provably optimal distributed algorithms that operate at each network cache to maximize the overall network utility. Our TTL-based optimization model provides theoretical answers to how long each content must be cached, and where it should be placed in the edge network. Extensive evaluations show that our algorithm significantly outperforms path replication with conventional caching algorithms over some network topologies.

Panigrahy, Nitish K.↗

LandScan mosaic enables high-resolution gridded population estimates with explicit uncertainty

Gridded population datasets represent high-resolution distributions of human occupancy, enabling informed decision-making across a broad range of fields. These data products are valuable for assessing environmental risk, urban development, disaster preparedness and resource allocation—areas where accurate population estimates directly enhance policy effectiveness and optimize resource distribution. Despite the importance of gridded population datasets, traditional population modeling approaches often overlook inherent uncertainties in the estimation process. This limitation can create a false sense of certainty in population estimates, potentially leading to flawed decisions by those who rely on the data. To address this methodological gap, we introduce a probabilistic machine learning modeling framework, LandScan Mosaic, that explicitly incorporates uncertainty into the population modeling process. Our approach systematically quantifies uncertainty in three key modeling parameters of the LandScan HD gridded population dataset: building use types, floor counts, and occupancy rates. By employing Monte Carlo simulations, we propagate these uncertainties through the modeling process, yielding probability distributions of population counts in place of deterministic point estimates. We demonstrate the practical application of this framework in Iloilo City, Philippines, using structured decision-making techniques and our probabilistic estimates to identify and prioritize areas most affected by projected flooding, supporting targeted interventions that address both economic and social risks. In doing so, we propose a population-specific approach for incorporating confidence into structured decision making processes. Through a comparative analysis with conventional deterministic approaches and point estimate approaches, including LandScan HD and WorldPop, we evaluate how the incorporation of machine learning and uncertainty influences decision rankings. This research advances population distribution modeling by offering a robust, quantitative approach that explicitly accounts for uncertainty in the underlying data, along with guidance for how users can apply uncertainty in their decision-making.

Environmental sciences↗

A Probability-based Model for Cost-effective Integration of Renewables into the Electricity Grid (Final Technical Report)

Funded by the U.S. Department of Energy during 2017-2019, the resource adequacy evaluation and production costing tool, called MATPLAN (MATlab and probability-based PLANning), was developed. MATPLAN can take into account the variable nature of renewable energy sources (both solar PV and wind farms), rather than considering deterministic data as in the current practice. It combines both probabilistic and optimization techniques to allow the determination of optimal system expansion policy and perform effective load carrying capacity (ELCC) analysis of a system with high renewable penetration. This is in contrast to the current practice that treats these variable sources as negative loads. This tool expects to benefit system planners by enabling them to consider renewable energy sources as options for their expansion plans. System operators can use MATPLAN to calculate the capacity credit for each solar/wind farm using probabilistic forecasts, thus enabling quantification of the operating reserve requirements to cope with uncertainty and variability of solar/wind output committed in the market.

14 SOLAR ENERGY↗

Systems and methods for quality of service (QoS) based management of bottlenecks and flows in networks

Techniques based on the Theory of Bottleneck Ordering can reveal the bottleneck structure of a network, and the Theory of Flow ordering can take advantage of the revealed bottleneck structure to manage and configure network flows so as to improve the overall network performance. These two techniques provide insights into the inherent topological properties of a network at least in three areas: (1) identification of the regions of influence of each bottleneck; (2) the order in which bottlenecks (and flows traversing them) may converge to their steady state transmission rates in distributed congestion control algorithms; and (3) the design of optimized traffic engineering policies.

97 MATHEMATICS AND COMPUTING↗

Stochastic Optimal Control for Series Hybrid Electric Vehicles

Increasing demand for improving fuel economy and reducing emissions has stimulated significant research and investment in hybrid propulsion systems. In this paper, we address the problem of optimizing online the supervisory control in a series hybrid configuration by modeling its operation as a controlled Markov chain using the average cost criterion. We treat the stochastic optimal control problem as a dual constrained optimization problem. We show that the control policy that yields higher probability distribution to the states with low cost and lower probability distribution to the states with high cost is an optimal control policy, defined as an equilibrium control policy. We demonstrate the effectiveness of the efficiency of the proposed controller in a series hybrid configuration and compare it with a thermostat-type controller.

fuel economy↗

Co-Optimization of Electricity Transmission and Generation Resources for Planning and Policy Analysis: Review of Concepts and Modeling Approaches

The recognition of transmission's interaction with other resources has motivated the development of co-optimization methods to optimize transmission investment while simultaneously considering tradeoffs with investments in electricity supply, demand, and storage resources. For a given set of constraints, co-optimized planning models provide solutions that have lower costs than solutions obtained from decoupled optimization (transmission-only, generation-only, or iterations between them). This paper describes co-optimization and provides an overview of approaches to co-optimizing transmission options, supply-side resources, demand-side resources, and natural gas pipelines. In particular, the paper provides an up-to-date assessment of the present and potential capabilities of existing co-optimization tools, and it discusses needs and challenges for developing advanced co-optimization models.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Multi-Objective Adaptive Mesh Refinement Using Reinforcement Learning

Finite element methods approximate the solution to a partial differential equation (PDE) on a mesh consisting of many elements. In general, using more, smaller elements results in a lower error in the approximation. However, it is often possible to lower the error substantially by only refining, or decreasing the size of, the few elements in the mesh that have the highest error. Adaptive mesh refinement (AMR) is a process that selectively refines regions of a mesh with high error to achieve a desired accuracy in as few degrees of freedom (DOFs) as possible. AMR is favorable compared to uniform refinement, which refines all elements of the mesh equally, because it can often achieve the same accuracy without wasting extra computation time on refinement of elements that already have low error. However, it is difficult to know which elements to refine. In this report we explore ways to choose which elements to refine such that we minimize both the resulting error and the cumulative DOFs used in computation. In particular, we introduce a Pareto-front learning algorithm that trains a policy to give the optimal refinement actions to minimize the cumulative DOFs used to achieve a given target error. Such a policy is useful because it can be deployed on many different problem types where different accuracy levels are desired. Furthermore, training a single policy for a range of target errors allows us to use transfer learning to reduce the required training time.

97 MATHEMATICS AND COMPUTING↗

A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow

With the increasing penetration of distributed energy resources, distributed optimization algorithms have attracted significant attention for power systems applications due to their potential for superior scalability, privacy, and robustness to a single point-of-failure. The Alternating Direction Method of Multipliers (ADMM) is a popular distributed optimization algorithm; however, its convergence performance is highly dependent on the selection of penalty parameters, which are usually chosen heuristically. In this work, we use reinforcement learning (RL) to develop an adaptive penalty parameter selection policy for alternating current optimal power flow (ACOPF) problem solved via ADMM with the goal of minimizing the number of iterations until convergence. We train our RL policy using deep Q-learning and show that this policy can result in significantly accelerated convergence (up to a 59% reduction in the number of iterations compared to existing, curvatureinformed penalty parameter selection methods). Furthermore, we show that our RL policy demonstrates promise for generalizability, performing well under unseen loading schemes as well as under unseen losses of lines and generators (up to a 50% reduction in iterations). This work thus provides a proof-of-concept for using RL for parameter selection in ADMM for power systems applications.

alternating current optimal power flow↗