Search NASA⌕ Search

SEARCH · Search NASA

Results for “policy optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

An Iterative Approach for Solving the SCOPF Problem Applying LP, SOCP, and NLP Subproblems

We propose to develop efficient algorithms and software for the SCOPF problem. We will employ an iterative approach that will: a) use linear subproblems and other active set filtering techniques to identify the most important contingencies and drastically reduce the SCOPF model size; b) solve SOCP relaxations of the reduced SCOPF to converge to the neighborhood of the global optimal solution and establish a lower bound on the solution, and; c) use a non-convex, nonlinear interior-point solver, Artelys Knitro, to converge quickly to the optimal solution. To identify the most effective approach, we will experiment with several techniques to identify the tradeoffs between contingency subproblem complexity and fast solvability.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Optimal Control of SOEC-Based Hydrogen Production Systems for Demand Response Using Deep Reinforcement Learning in Smart Grids

Solid oxide electrolysis cell (SOEC) hydrogen production technology can range in size from small, appliance-size equipment to large-scale, central production facilities that can be tied directly to renewable or non-greenhouse-gas-emitting forms of electricity production, making it an ideal resource for demand response (DR). The SOEC hydrogen production system is a complex integrated system that encompasses fluid dynamics, electrical dynamics, and electrochemical and thermal dynamics, all of which involve non-linearity and non-convexity. Proper control of the SOEC hydrogen production system is crucial to enable its participation in the DR program. Here, to overcome the difficulty of designing an explicit control law for such nonlinear systems with nonconvex optimization features in DR applications, deep reinforcement learning (DRL) is explored to achieve the optimal control of the SOEC system for DR participation. Specifically, a twin delayed deterministic policy gradient (TD3) control framework is applied to achieve optimal response performance during DR events by considering power tracking error and hydrogen production efficiency with a suitable reward function. Two case studies with grid connections for tracking different DR commands were investigated. The first case study involved operating conditions reaching the boundaries, while the second involved operating conditions within the boundaries. The results showed that the proposed DRL-based control for SOEC can track the DR signal in a timely manner while maintaining high energy efficiency.

08 HYDROGEN↗
DOE OSTI2024

Transmission Interface Limits for High-Spatial Resolution Capacity Expansion Modeling

Large-scale capacity expansion models typically rely on estimates of the power transfer limits between modeled zones. Accurate estimation of these interface transfer limits (ITLs) requires modeling the underlying transmission network. Here we expand on a maximum flow optimization method that uses linearized power flow to estimate transfer limits. We apply this method to a data set of the U.S. transmission network to estimate ITLs between U.S. counties. By calculating ITLs using different subsets of the network, we evaluate how the size of the network used in the estimation affects the results. The results show diminishing returns to ITL accuracy after six hops, suggesting that a network subset can reasonably be used to approximate ITLs. The county-level estimates produced in this study will support more spatially resolved capacity expansion modeling and will help inform policy making at local and national levels.

capacity planning↗
DOE OSTI2025

Valued peaks: Sustainable water allocation for small hydropower plants in an era of explicit ecological needs

Optimizing hydropower operations to balance economic profitability and support functioning ecosystem services is integral to river management policy. In this article, we propose a dynamic, constrained optimization framework for small hydropower plants (SHPs) to evaluate trade-offs between economic profitability and socio-ecological requirements. Specifically, we examine the balance between short-term losses in hydropower generation and the potential for compensatory benefits in the form of revenue from recreational ecosystem services, irrespective of the direct beneficiary. Our framework integrates a fish habitat model, a hydropower optimization model, and a recreational ecosystem service estimate to evaluate different environmental flow scenarios. The optimization process gives three outflow release scenarios, informed by previous streamflow realisations (dam inflow), and designed environmental flow constraints. The framework is applied and tested for the river Kuusinkijoki in North-eastern Finland, which is a habitat for migratory brown trout and grayling populations. We show that the revenue loss due to the environmental flow constraints arises through a reduction in revenue per generated energy unit and through a reduction in turbine efficiency. Additionally, the simulation results reveal that all the designed environmental flow constraints cannot be met simultaneously. Under the environmental flow scenario with both minimum flow and flow ramping rate constraints, the annual hydropower revenue decreases by 16.5 %. An annual increase of 8 % in recreational fishing visits offsets the revenue loss. In conclusion, the developed framework provides knowledge of the costs and benefits of hydropower environmental flow constraints and guides the prioritizing process of environmental measures.

13 HYDRO ENERGY↗
DOE OSTI2025

Neural network approaches for parameterized optimal control

Here, we consider numerical approaches for deterministic, finite-dimensional optimal control problems whose dynamics depend on unknown or uncertain parameters. We seek to amortize the solution over a set of relevant parameters in an offline stage to enable rapid decision-making and be able to react to changes in the parameter in the online stage. To tackle the curse of dimensionality arising when the state and/or parameter are high-dimensional, we represent the policy using neural networks. We compare two training paradigms: First, our model-based approach leverages the dynamics and definition of the objective function to learn the value function of the parameterized optimal control problem and obtain the policy using a feedback form. Second, we use actor-critic reinforcement learning to approximate the policy in a data-driven way. Using an example involving a two-dimensional convection-diffusion equation, which features high-dimensional state and parameter spaces, we investigate the accuracy and efficiency of both training paradigms. While both paradigms lead to a reasonable approximation of the policy, the model-based approach is more accurate and considerably reduces the number of PDE solves.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimal Coordination of Electric Vehicles for Grid Services using Deep Reinforcement Learning

Recent research has shown the effectiveness of reinforcement learning (RL) in coordinating electric vehicles (EVs) with vehicle-to-grid capabilities for grid services. However, many of these studies rely on lookup table and deep Q-network techniques, which can be impractical when dealing with continuous states and actions. In addition, existing RL designs inadequately account for battery aging effects, EV user satisfaction, uncertain departure and arrival time, and trip distance, which may compromise effective coordination. This paper aims to bridge these gaps by developing an innovative deep deterministic policy gradient-based RL framework for optimal coordination of EVs. Case studies were carried out using a test system with 100 EVs, and numerical analysis results showed that the proposed RL framework can effectively coordinate EVs to maximize economic benefits and user satisfaction while ensuring the expected battery lifespan.

Das, Avijit↗
DOE OSTI2024

REopt Model Overview and Example Use Cases [Slides]

REopt(R) is a mixed-integer optimization model that minimizes the lifecycle cost of serving energy loads at a site. This work provides and introduction to the model along with its key workflow, techno-economic inputs, key outputs, and key caveats for readers to understand REopt the when, why, how of using this model. This resource also includes helpful links related to REopt model and the data sources it uses during the optimization.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Scalable Control Co-design for Resilient-by-Design Cyber Physical Systems

Critical infrastructure networks, such as power and transportation networks, are often modelled as cyber-physical systems. With ever increasing complexity of these systems, there is a need for newer and more relevant metrics and design tools that will co-optimize the physical system components and control policies to guarantee resilience against cyber and natural threats. To this end, a simulation-based control co-design computational framework that will concurrently determine the system and control parameters of a cyber-physical system to meet pre-specified resilience, operational and economic objectives has been developed. The capabilities of the developed co-design engine are demonstrated by designing the physical components and control parameters of a microgrid system that will meet its resiliency objectives when subjected to various cyber and physical threats.

42 ENGINEERING↗
DOE OSTI2025

Techno-economic assessment of residential PV system tariff policies in Jordan

This study assesses the economic and technical performance of four energy policy scenarios for Jordan's residential photovoltaic (PV) systems: net metering, net billing, zero-export with battery storage, and sell-all-buy-all. With the recent introduction of time-of-use (TOU) tariffs and policies addressing the “duck curve” effect, the research focuses on optimizing PV system sizing across different regulatory frameworks. A detailed techno-economic analysis evaluates these scenarios based on energy production, cost savings, payback periods, and energy self-sufficiency. The findings indicate that net metering and net billing offer the highest cost savings and the shortest payback periods (∼3 years). While the zero-export strategy with battery storage enhances energy self-sufficiency by up to 70%, it requires a higher upfront investment. The sell-all-buy-all scenario supports larger system sizes, achieving a low levelized cost of electricity (0.0696 USD/kWh) and a net present value of 619 USD. Additionally, the study identifies a critical feed-in tariff threshold of 0.055 USD/kWh, at which net billing becomes as financially attractive as net metering. Here, these insights offer valuable recommendations for policymakers to optimize net billing rates and TOU tariffs, promoting the expansion of Jordan's renewable energy sector.

Battery storage↗
DOE OSTI2024

Federated Deep Reinforcement Learning for Decentralized VVO of BTM DERs

The future of grid control requires a hybrid approach combining centralized and decentralized methods to fully utilize the potential of smart edge devices with artificial intelligence (AI) capabilities. This paper aims to develop and evaluate a federated deep reinforcement learning (FDRL) framework for decentralized adaptive volt-var optimization (VVO) of behind-the-meter (BTM) distributed energy resources (DERs). First, this paper models a single deep reinforcement learning (DRL) agent using the Markov Decision Process (MDP) framework for decentralized adaptive VVO of BTM DERs. Two DRL algorithms, soft actor-critic (SAC) and twin-delayed deep deterministic policy gradient (TD3), are compared for their effectiveness in optimizing VVO. Results show that TD3 outperforms SAC, achieving a 71.3% improvement in mean reward. Finally, the DRL agent is deployed within the FDRL framework, using the Flower platform, to enhance learning, provide adaptive control, and ensure data privacy for BTM DERs.

Ravi, Abhijith↗
DOE OSTI2025

Probabilistic Deliverability Assessment of Distributed Energy Resources via Scenario-Based AC Optimal Power Flow

As electric grids decarbonize and distributed energy resources (DERs) become increasingly prevalent, interconnection assessments must evolve to reflect operational variability and control flexibility. This paper highlights key modeling limitations observed in practice and reviews approaches for modeling uncertainty. It then introduces a Probabilistic Deliverability Assessment (PDA) framework designed to complement and extend existing procedures. The framework integrates scenario-based AC optimal power flow (AC OPF), corrective dispatch, and optional multi-temporal constraints. Together, these form a structured methodology for quantifying DER utilization, deliverability, and reliability under uncertainty in load, generation, and topology. Outputs include interpretable metrics with confidence intervals that inform siting decisions and evaluate compliance with reliability thresholds across sampled operating conditions. A case study on Puerto Rico’s publicly available bulk power system model demonstrates the framework’s application using minimal input data, consistent with current interconnection practice. Across staged fossil generation retirements, the PDA identifies high-value DER sites and regions requiring additional reactive power support. Results are presented through mean dispatch signals, reliability metrics, and geospatial visualizations, demonstrating how the framework provides transparent, data-driven siting recommendations. The framework’s modular design supports incremental adoption within existing workflows, encouraging broader use of AC OPF in interconnection and planning contexts.

14 SOLAR ENERGY↗
DOE OSTI2024

Quantifying Energy Justice Goals in the Power Sector: Developing and Using Metrics

New policy goals are explicitly guiding the future grid toward greater energy equity. At the same time, policy goals also guide the grid toward decarbonization and resilience, while maintaining cost, security, and reliability cornerstone requirements. In conclusion, these pressures create a dilemma: moving urgently to address climate change and respond to energy disruptions, but also slowly to engage communities on the climate frontlines in earnest.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Scale-Bridging Optimization Framework for Desalination Integrated Produced Water Networks

In this work, we develop a Pyomo-based non-linear optimization strategy that includes rigorous MVR models. The detailed desalination unit is integrated into the multiperiod produced water network problem using the trust region filter (TRF) method. TRF decomposes the integrated problem into a master problem consisting of the network variables and a simplified surrogate model for the detailed desalination unit. The surrogate is updated using zero and first-order corrections from the optimal solution of the detailed models at every iteration. This framework allows us to co-optimize the design of the desalination units and operating policy for the multiperiod network. A common design is ensured across all periods using global capacity constraints. We validate the solution obtained using the TRF method by solving the full integrated problem for small network instances and show our results on real case studies on produced water networks from the Permian and Appalachian basins. In this work, we describe our TRF formulation, give details on our implementation in Pyomo, and analyze the results obtained by solving the optimization problem using IPOPT. We also present a discussion on the computational efficiency and scaling using the TRF approach against a full-scale integration of the rigorous models within the water network.

Naik, Sakshi↗
DOE OSTI2024

Feedback Optimization of Incentives for Distribution Grid Services

Energy prices and net power injection limitations regulate the operations in distribution grids and typically ensure that operational constraints are met. Nevertheless, unexpected or prolonged abnormal events could undermine the grid's functioning. During contingencies, customers could contribute effectively to sustaining the network by providing services. Herein this paper proposes an incentive mechanism that promotes users' active participation by essentially altering the energy pricing rule. The incentives are modeled via a linear function whose parameters can be computed by the system operator (SO) by solving an optimization problem. Feedback-based optimization algorithms are then proposed to seek optimal incentives by leveraging measurements from the grid, even in the case when the SO does not have a full grid and customer information. Numerical simulations on a standard testbed validate the proposed approach.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Binary Quantum Control Optimization with Uncertain Hamiltonians

Optimizing the controls of quantum systems plays a crucial role in advancing quantum technologies. The time-varying noises in quantum systems and the widespread use of inhomogeneous quantum ensembles raise the need for high-quality quantum controls under uncertainties. In this paper, we consider a stochastic discrete optimization formulation of a discretized binary optimal quantum control problem involving Hamiltonians with predictable uncertainties. We propose a sample-based reformulation that optimizes both risk-neutral and risk-averse measurements of control policies, and solve these with two gradient-based algorithms using sum-up-rounding approaches. Furthermore, we discuss the differentiability of the objective function and prove upper bounds of the gaps between the optimal solutions to binary control problems and their continuous relaxations. We conduct numerical simulations on various sized problem instances based on two applications of quantum pulse optimization; we evaluate different strategies to mitigate the impact of uncertainties in quantum systems. In conclusion, we demonstrate that the controls of our stochastic optimization model achieve significantly higher quality and robustness compared with the controls of a deterministic model.

conditional value-at-risk (CVaR)↗
DOE OSTI2024

Incorporating energy justice and equity objectives in power system models

Ensuring an equitable energy transition requires models and tools that can account for equity and energy justice goals. Power system models (PSMs) are widely used throughout industry, government, and academia to simulate or optimize the operations and planning of current and future electricity systems under different scenarios, parameter assumptions and policy frameworks. These models are important tools that allow users to understand how the power system may evolve under different future conditions, but importantly, they are also used to inform policy implementation and investment decisions across all aspects of the power system. However, existing models seldom include energy justice considerations and therefore energy justice priorities are not reflected in the policies and other decision-making processes that are informed by these models. The purpose of this review is to provide a framework that energy modelers can draw upon to integrate energy justice and equity goals into PSMs. To this end, 99 papers that examine the intersection of energy justice and power system models are summarized and ten core aspects of the power system that can impact energy justice outcomes, and therefore require new modeling approaches, are identified. This review then establishes key current practices, challenges, and opportunities associated with capturing energy justice considerations in power system models across these ten aspects. This review concludes by proposing four key research directions that should be pursued to improve the representation of energy justice and equity in power system modeling. Finally, this review also addresses challenges raised by United Nations Sustainable Development Goal 7, which aims to ensure affordable energy access to everyone and Sustainable Development Goal 13, which aims to take urgent action to address climate change.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

HPC Digital Twins for Evaluating Scheduling Policies, Incentive Structures and their Impact on Power and Cooling

Schedulers are critical for optimal resource utilization in high-performance computing. Traditional methods to evaluate sched- ulers are limited to post-deployment analysis, or simulators, which do not model associated infrastructure. In this work, we present the first-of-its-kind integration of scheduling and digital twins in HPC. This enables what-if studies to understand the impact of parameter configurations and scheduling decisions on the physical assets, even before deployment, or regarching changes not easily realizable in production. We (1) provide the first digital twin framework extended with scheduling capabilities, (2) integrate various top-tier HPC systems given their publicly available datasets, (3) implement extensions to integrate external scheduling simulators. Finally, we show how to (4) implement and evaluate incentive structures, as- well-as (5) evaluate machine learning based scheduling, in such novel digital-twin based meta-framework to prototype scheduling. Our work enables what-if scenarios of HPC systems to evaluate sustainability, and the impact on the simulated system.

Maiterth, Matthias [ORNL] (ORCID:000000018698460X)↗
DOE OSTI2026

Simulating competition in the US bioeconomy to produce hard‐to‐electrify transportation fuels using limited biomass resources

This study presents a novel bioeconomy optimization framework, BiOpt, designed to address critical questions regarding the strategic use of limited US biomass resources for biofuel production. By integrating detailed techno-economic analyses, life cycle assessments, and resource assessment data, BiOpt optimizes resource distributions across competing technologies to maximize economic performance and/or minimize greenhouse gas emissions. Using feedstock scenarios from the 2023 Billion Ton Study, the analysis explores optimal biomass allocations across sustainable aviation fuel, diesel, and marine biofuel conversion pathways given varying production targets and policy incentives. Results demonstrate distinct feedstock preferences and pathway utilizations when prioritizing economic returns vs. emissions reductions. For instance, fats, oils, and greases were highly favored in cost-optimized scenarios, while low-carbon feedstocks such as wet waste dominated greenhouse gas-minimized strategies. The findings underscore the pivotal role of policy incentives and technological advances in shaping biofuel supply chains and provide actionable insights for scaling sustainable biofuel production to decarbonize hard-to-electrify sectors. This framework offers a robust tool for policymakers and stakeholders to evaluate biofuel strategies that balance energy output, economic viability, and environmental impact.

09 BIOMASS FUELS↗