Search NASA⌕ Search

SEARCH · Search NASA

Results for “Policy optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Justice 40 Tool (J40 Tool) v1.0

The Justice 40 tool provides a quantitative framework to support decision-making around equitable energy interventions at the community level. The tool calculates the optimal portfolio of policy interventions that explicitly mitigates energy insecurity of an eligible population, by reducing its disproportionate energy burden. The place-based analysis assumes a spatial census tract-level resolution and distinguishes different sociodemographic groups within each tract. Instead of focusing on a specific technology, the underlying J40 model captures the combined effect of a set of policy interventions, currently including weatherization, rooftop solar, community solar and community wind. For each tract, the model chooses the optimal (least cost) combination of interventions to address the disproportionate burden, considering the specific population demographics and techno-economic potentials of technologies. Mathematically, this problem is formalized as an optimization model and formulated as a linear program.

Heleno, Miguel↗
DOE OSTI2016

A duality framework for stochastic optimal control of complex systems

In this study, we address the problem of minimizing the long-run expected average cost of a complex system consisting of interactive subsystems. We formulate a multiobjective optimization problem of the one-stage expected costs of the subsystems and provide a duality framework to prove that the control policy yielding the Pareto optimal solution minimizes the average cost criterion of the system. We provide the conditions of existence and a geometric interpretation of the solution. For practical situations having constraints consistent with those studied here, our results imply that the Pareto control policy may be of value when we seek to derive online the optimal control policy in complex systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Learning to Optimize Variational Quantum Circuits to Solve Combinatorial Problems

Quantum computing is a computational paradigm with the potential to outperform classical methods for a variety of problems. Proposed recently, the Quantum Approximate Optimization Algorithm (QAOA) is considered as one of the leading candidates for demonstrating quantum advantage in the near term. QAOA is a variational hybrid quantum-classical algorithm for approximately solving combinatorial optimization problems. The quality of the solution obtained by QAOA for a given problem instance depends on the performance of the classical optimizer used to optimize the variational parameters. In this paper, we formulate the problem of finding optimal QAOA parameters as a learning task in which the knowledge gained from solving training instances can be leveraged to find high-quality solutions for unseen test instances. To this end, we develop two machine-learning-based approaches. Our first approach adopts a reinforcement learning (RL) framework to learn a policy network to optimize QAOA circuits. Our second approach adopts a kernel density estimation (KDE) technique to learn a generative model of optimal QAOA parameters. In both approaches, the training procedure is performed on small-sized problem instances that can be simulated on a classical computer; yet the learned RL policy and the generative model can be used to efficiently solve larger problems. Furthermore, extensive simulations using the IBM Qiskit Aer quantum circuit simulator demonstrate that our proposed RL- and KDE-based approaches reduce the optimality gap by factors up to 30.15 when compared with other commonly used off-the-shelf optimizers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Joint routing of conventional and range-extended electric vehicles in a large metropolitan network

Range-extended electric vehicles combine the higher efficiency and environmental benefits of battery-powered electric motors with the longer mileage and autonomy of conventional internal combustion engines. This combination is particularly advantageous for time-constrained delivery routing in dense urban areas, where battery recharging along routes can be too time-consuming to economically justify the use of all-electric vehicles. However, switching from electric to conventional fossil fuel modes also results in higher costs and emissions and lower efficiency. This paper analyzes this heterogeneous vehicle routing problem and describes two solution methods: an exact branch-price-and-cut algorithm and an iterated tabu search metaheuristic. From a methodological perspective, we find that the exact algorithm consistently obtains tight lower bounds that also serve to certify the metaheuristic solutions as near-optimal. From a policy standpoint, we examine a large-scale real-world case study concerning parcel deliveries in the Chicago metropolitan area and quantify various operational metrics including energy costs and vehicle miles traveled. We find that by deploying roughly 20% of range -extended vehicles with a modest all-electric range of 33 miles, parcel distributors can save energy costs by up to 17% while incurring less than 0.5% increase in vehicle miles traveled. Increasing the range to 60 miles further reduces costs by only 4%, which can alternatively be achieved by decreasing the average service time by 1 minute or increasing driver working time by 1 hour. In conclusion, our study reveals several key areas of improvement on which vehicle manufacturers, distributors, and policy makers can focus their attention.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

Optimizing Individualized Treatment Planning for Parkinson’s Disease Using Deep Reinforcement Learning

More than one million people currently live with Parkinson's Disease (PD) in the U.S. alone. Medications, such as levodopa, can help manage PD symptoms. However, medication treatment planning is generally based on patient history and limited interaction between physicians and patients during office visits. This limits the extent of benefit that may be derived from the treatment as disease/patient characteristics are generally non-stationary. Wearable sensors that provide continuous monitoring of various symptoms, such as bradykinesia and dyskinesia, can enhance symptom management. However, using such data to overhaul the current static medication treatment planning approach and prescribe personalized medication timing and dosage that accounts for patient/care-giver/physician feedback/preferences remains an open question. We develop a model to prescribe timing and dosage of medications, given the motor fluctuation data collected using wearable sensors in real-time. We solve the resulting model using deep reinforcement learning (DRL). The prescribed policy determines the optimal treatment plan that minimizes patient's symptoms. Our results show that the model-prescribed policy outperforms the static a priori treatment plan in improving patients' symptoms, providing a proof-of-concept that DRL can augment medical decision making for treatment planning of chronic disease patients.

Watts, Jeremy↗
DOE OSTI2025

Multi-scale, Multi-disciplinary, and Multi-agent Explainable AI with Koopman-Undergirded Learning, Prediction, and Analysis (M3EA KULPA) (Project Closeout Report)

The goal of this project was to develop and use domain-aware machine learning formulations, based on the Koopman Operator (KO), for modelling multi-scale, multi-disciplinary (e.g., multi-physics), and/or multi-agent systems. The project developed these formulations for the following cases: • Systems with dynamics at two separate time scales, • Systems with a bi-level hierarchical control structure, • Systems with bi-level hierarchical control and dynamics at two separate time scales (the lower level controls operating at the faster time scale), and • Systems with n separate but interacting agents/disciplines (with/without control, respectively); the controls for each agent could include bi-level hierarchical control and dynamics at two separate time scales as described above. The project then defined a set of dynamical systems consisting of different nonlinear oscillators that could be used to test these different formulations and then subsequently learned the KO models for those systems. With the KO models, we were able to do the following: • Quantify system stability, including both long-term and transient behavior, • Quantify the effects of feedbacks between the different time scales and agents/disciplines in terms of those feedbacks’ effects on system stability, • Replace a standard Proportional-Integral (PI) control in the hierarchical control structure with a KO-based Linear-Quadratic Regular (LQR), a form of optimal control, • Calculate optimal supervisory control policies a) with and without time scale separated dynamics at the lower level control levels and b) with both PI and KO-based LQR lower level control policies, and • Calculate dynamic Nash equilibria for multi-agent systems where each agent makes its own control decisions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Hamiltonian switching control of noisy bipartite qubit systems

Abstract We develop a Hamiltonian switching ansatz for bipartite control that is inspired by the quantum approximate optimization algorithm, to mitigate environmental noise on qubits. We demonstrate the control for a central spin coupled to bath spins via isotropic Heisenberg interactions, and then make physical applications to the protection of quantum gates performed on superconducting transmon qubits coupling to environmental two-level-systems (TLSs) through dipole-dipole interactions, as well as on such qubits coupled to both TLSs and a Lindblad bath. The control field is classical and acts only on the system qubits. We use reinforcement learning with policy gradient to optimize the Hamiltonian switching control protocols, using a fidelity objective for specific target quantum gates. We use this approach to demonstrate effective suppression of both coherent and dissipative noise, with numerical studies achieving target gate implementations with fidelities over 0.9999 (four nines) in the majority of our test cases and showing improvement beyond this to values of 0.999 999 999 (nine nines) upon a subsequent optimization by GRadient Ascent Pulse Engineering (GRAPE). We analyze how the control depth, total evolution time, number of environmental TLS, and choice of optimization method affect the fidelity achieved by the optimal protocols and reveal some critical behaviors of bipartite control of quantum gates.

Physics↗
DOE OSTI2023

Joint optimization of land carbon uptake and albedo can help achieve moderate instantaneous and long-term cooling effects

Abstract Both carbon dioxide uptake and albedo of the land surface affect global climate. However, climate change mitigation by increasing carbon uptake can cause a warming trade-off by decreasing albedo, with most research focusing on afforestation and its interaction with snow. Here, we present carbon uptake and albedo observations from 176 globally distributed flux stations. We demonstrate a gradual decline in maximum achievable annual albedo as carbon uptake increases, even within subgroups of non-forest and snow-free ecosystems. Based on a paired-site permutation approach, we quantify the likely impact of land use on carbon uptake and albedo. Shifting to the maximum attainable carbon uptake at each site would likely cause moderate net global warming for the first approximately 20 years, followed by a strong cooling effect. A balanced policy co-optimizing carbon uptake and albedo is possible that avoids warming on any timescale, but results in a weaker long-term cooling effect.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

A hybrid inventory policy with split delivery under regular and surge demand

This paper proposes a hybrid inventory policy with split delivery under regular demand and surge demand. The combination of regular demand and surge demand can be observed in many areas, such as healthcare inventory and humanitarian supply chain management. The arrival rate for regular demand is typically higher than the arrival rate of surge demand; whereas, the volume of a regular demand is typically lower than the volume of a surge demand. This paper proposes an inventory management model that considers both emergency and regular replenishment corresponding to both demand patterns. The equilibrium equations developed for this model are based on the level crossing theory. This paper finally presents a search-based heuristics that can be used to identify near optimal inventory management policies. Numerical results show that proposed hybrid inventory policy with split delivery outperforms similar hybrid inventory policy without split delivery.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

I/O-aware bandwidth allocation for petascale computing systems

In the Big Data era, the gap between the storage performance and an appli- cation's I/O requirement is increasing. I/O congestion caused by concurrent storage accesses from multiple applications is inevitable and severely harms the performance. Conventional approaches either focus on optimizing an ap- plication's access pattern individually or handle I/O requests on a low-level storage layer without any knowledge from the upper-level applications. In this paper, we present a novel I/O-aware bandwidth allocation framework to coordinate ongoing I/O requests on petascale computing systems. The motivation behind this innovation is that the resource management system has a holistic view of both the system state and jobs' activities and can dy- namically control the jobs' status or allocate resource on the y during their execution. We treat a job's I/O requests as periodical subjobs within its lifecycle and transform the I/O congestion issue into a classical scheduling problem. Based on this model, we propose a bandwidth management mech- anism as an extension to the existing scheduling system. We design several bandwidth allocation policies with different optimization objectives either on user-oriented metrics or system performance. We conduct extensive trace- based simulations using real job traces and I/O traces from a production IBM Blue Gene/Q system at Argonne National Laboratory. Experimental results demonstrate that our new design can improve job performance by more than 30%, as well as increasing system performance.

I/O congestion↗
DOE OSTI2024

Toward Accelerating Discovery via Physics-Driven and Interactive Multifidelity Bayesian Optimization

Both computational and experimental material discovery bring forth the challenge of exploring multidimensional and often nondifferentiable parameter spaces, such as phase diagrams of Hamiltonians with multiple interactions, composition spaces of combinatorial libraries, processing spaces, and molecular embedding spaces. Often these systems are expensive or time consuming to evaluate a single instance, and hence classical approaches based on exhaustive grid or random search are too data intensive. This resulted in strong interest toward active learning methods such as Bayesian optimization (BO) where the adaptive exploration occurs based on human learning (discovery) objective. However, classical BO is based on a predefined optimization target, and policies balancing exploration and exploitation are purely data driven. In practical settings, the domain expert can pose prior knowledge of the system in the form of partially known physics laws and exploration policies often vary during the experiment. Here, we propose an interactive workflow building on multifidelity BO (MFBO), starting with classical (data-driven) MFBO, then expand to a proposed structured (physics-driven) structured MFBO (sMFBO), and finally extend it to allow human-in-the-loop interactive interactive MFBO (iMFBO) workflows for adaptive and domain expert aligned exploration. These approaches are demonstrated over highly nonsmooth multifidelity simulation data generated from an Ising model, considering spin–spin interaction as parameter space, lattice sizes as fidelity spaces, and the objective as maximizing heat capacity. Detailed analysis and comparison show the impact of physics knowledge injection and real-time human decisions for improved exploration with increased alignment to ground truth. Here, the associated notebooks allow to reproduce the reported analyses and apply them to other systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A TTL-based Approach for Content Placement in Edge Networks

Edge networks are promising to provide better services to users by provisioning computing and storage resources at the edge of networks. However, due to the uncertainty and diversity of user interests, content popularity, distributed network structure, cache sizes, it is challenging to decide where to place the content, and how long it should be cached. In this paper, we study the utility optimization of content placement at edge networks through timer-based (TTL) policies. We propose provably optimal distributed algorithms that operate at each network cache to maximize the overall network utility. Our TTL-based optimization model provides theoretical answers to how long each content must be cached, and where it should be placed in the edge network. Extensive evaluations show that our algorithm significantly outperforms path replication with conventional caching algorithms over some network topologies.

Panigrahy, Nitish K.↗
DOE OSTI2025

LandScan mosaic enables high-resolution gridded population estimates with explicit uncertainty

Gridded population datasets represent high-resolution distributions of human occupancy, enabling informed decision-making across a broad range of fields. These data products are valuable for assessing environmental risk, urban development, disaster preparedness and resource allocation—areas where accurate population estimates directly enhance policy effectiveness and optimize resource distribution. Despite the importance of gridded population datasets, traditional population modeling approaches often overlook inherent uncertainties in the estimation process. This limitation can create a false sense of certainty in population estimates, potentially leading to flawed decisions by those who rely on the data. To address this methodological gap, we introduce a probabilistic machine learning modeling framework, LandScan Mosaic, that explicitly incorporates uncertainty into the population modeling process. Our approach systematically quantifies uncertainty in three key modeling parameters of the LandScan HD gridded population dataset: building use types, floor counts, and occupancy rates. By employing Monte Carlo simulations, we propagate these uncertainties through the modeling process, yielding probability distributions of population counts in place of deterministic point estimates. We demonstrate the practical application of this framework in Iloilo City, Philippines, using structured decision-making techniques and our probabilistic estimates to identify and prioritize areas most affected by projected flooding, supporting targeted interventions that address both economic and social risks. In doing so, we propose a population-specific approach for incorporating confidence into structured decision making processes. Through a comparative analysis with conventional deterministic approaches and point estimate approaches, including LandScan HD and WorldPop, we evaluate how the incorporation of machine learning and uncertainty influences decision rankings. This research advances population distribution modeling by offering a robust, quantitative approach that explicitly accounts for uncertainty in the underlying data, along with guidance for how users can apply uncertainty in their decision-making.

Environmental sciences↗
DOE OSTI2019

A Probability-based Model for Cost-effective Integration of Renewables into the Electricity Grid (Final Technical Report)

Funded by the U.S. Department of Energy during 2017-2019, the resource adequacy evaluation and production costing tool, called MATPLAN (MATlab and probability-based PLANning), was developed. MATPLAN can take into account the variable nature of renewable energy sources (both solar PV and wind farms), rather than considering deterministic data as in the current practice. It combines both probabilistic and optimization techniques to allow the determination of optimal system expansion policy and perform effective load carrying capacity (ELCC) analysis of a system with high renewable penetration. This is in contrast to the current practice that treats these variable sources as negative loads. This tool expects to benefit system planners by enabling them to consider renewable energy sources as options for their expansion plans. System operators can use MATPLAN to calculate the capacity credit for each solar/wind farm using probabilistic forecasts, thus enabling quantification of the operating reserve requirements to cope with uncertainty and variability of solar/wind output committed in the market.

14 SOLAR ENERGY↗
NASA NTRS1990

Optimal dynamic remapping of data parallel computations

A large class of data parallel computations is characterized by a sequence of phases, with phase changes occurring unpredictably. Dynamic remapping of the workload to processors may be required to maintain good performance. The problem considered, for which the utility of remapping and the future behavior of the workload are uncertain, arises when phases exhibit stable execution requirements during a given phase, but requirements change radically between phases. For these situations, a workload assignment generated for one phase may hinder performance during the next phase. This problem is treated formally for a probabilistic model of computation with at most two phases. The authors address the fundamental problem of balancing the expected remapping performance gain against the delay cost, and they derive the optimal remapping decision policy. The promise of the approach is shown by application to multiprocessor implementations of an adaptive gridding fluid dynamics program and to a battlefield simulation program.

Nicol, David M.↗
DOE OSTI2022

Systems and methods for quality of service (QoS) based management of bottlenecks and flows in networks

Techniques based on the Theory of Bottleneck Ordering can reveal the bottleneck structure of a network, and the Theory of Flow ordering can take advantage of the revealed bottleneck structure to manage and configure network flows so as to improve the overall network performance. These two techniques provide insights into the inherent topological properties of a network at least in three areas: (1) identification of the regions of influence of each bottleneck; (2) the order in which bottlenecks (and flows traversing them) may converge to their steady state transmission rates in distributed congestion control algorithms; and (3) the design of optimized traffic engineering policies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Co-Optimization of Electricity Transmission and Generation Resources for Planning and Policy Analysis: Review of Concepts and Modeling Approaches

The recognition of transmission's interaction with other resources has motivated the development of co-optimization methods to optimize transmission investment while simultaneously considering tradeoffs with investments in electricity supply, demand, and storage resources. For a given set of constraints, co-optimized planning models provide solutions that have lower costs than solutions obtained from decoupled optimization (transmission-only, generation-only, or iterations between them). This paper describes co-optimization and provides an overview of approaches to co-optimizing transmission options, supply-side resources, demand-side resources, and natural gas pipelines. In particular, the paper provides an up-to-date assessment of the present and potential capabilities of existing co-optimization tools, and it discusses needs and challenges for developing advanced co-optimization models.

24 POWER TRANSMISSION AND DISTRIBUTION↗