Search NASA⌕ Search

SEARCH · Search NASA

Results for “Policy optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Chance Constraints for Improving the Security of AC Optimal Power Flow

This paper introduces a scalable method for improving the solutions of ac optimal power flow (AC OPF) with respect to deviations in predicted power injections from wind and other uncertain generation resources. The aim of this paper is on providing solutions that are more robust to short-term deviations, and that optimize both the initial operating point and a parametrized response policy for control during fluctuations. We formulate this as a chance-constrained optimization problem. To obtain a tractable representation of the chance constraints, we introduce a number of modeling assumptions and leverage recent theoretical results to reformulate the problem as a convex, second-order cone program, which is efficiently solvable even for large instances. Our experiments demonstrate that the proposed procedure improves the feasibility and cost performance of the OPF solution, while the additional computation time is on the same magnitude as a single deterministic AC OPF calculation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Transmission Interface Limits for High-Spatial Resolution Capacity Expansion Modeling

Large-scale capacity expansion models typically rely on estimates of the power transfer limits between modeled zones. Accurate estimation of these interface transfer limits (ITLs) requires modeling the underlying transmission network. Here we expand on a maximum flow optimization method that uses linearized power flow to estimate transfer limits. We apply this method to a data set of the U.S. transmission network to estimate ITLs between U.S. counties. By calculating ITLs using different subsets of the network, we evaluate how the size of the network used in the estimation affects the results. The results show diminishing returns to ITL accuracy after six hops, suggesting that a network subset can reasonably be used to approximate ITLs. The county-level estimates produced in this study will support more spatially resolved capacity expansion modeling and will help inform policy making at local and national levels.

capacity planning↗
DOE OSTI2017

The International Database of Efficient Appliances (IDEA): A new tool to support appliance energy-efficiency deployment

Appliance energy-efficiency programs are a central component of many countries’ energy-policy portfolios. A major barrier to optimal implementation of these programs is lack of data to determine market baselines, assess the potential for cost-effective energy savings, and track markets over time to evaluate and verify program impacts. To address this gap, we have developed the International Database of Efficient Appliances (IDEA), a suite of software tools that automatically gathers data that is currently dispersed across various online sources and compiles it into a unified repository of information on efficiency, price, and features for a diversity of appliances and devices in markets around the world. In this article we describe the framework and functionality of IDEA, and we demonstrate its power as a resource for research and policy development related to appliance energy efficiency. Using IDEA data for refrigerators in China and India, we assess the potential for cost-effective energy savings within each market by computing robust indicators that can also be easily compared across different appliances and markets. We find that significant cost-effective savings are available on both markets. We discuss implications for the development of future energy-efficiency deployment programs.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Learning and Fast Adaptation for Grid Emergency Control via Deep Meta Reinforcement Learning

As power systems are undergoing a significant transformation with more uncertainties, less inertia and closer to operation limits, there is increasing risk of large outages. Thus, there is an imperative need to enhance grid emergency control to maintain system reliability and security. Towards this end, great progress has been made in developing deep reinforcement learning (DRL) based grid control solutions in recent years. However, existing DRL-based solutions have two main limitations: 1) they cannot handle well with a wide range of grid operation conditions, system parameters, and contingencies; 2) they generally lack the ability to fast adapt to new grid operation conditions, system parameters, and contingencies, limiting their applicability for real-world applications. Here, in this paper, we mitigate these limitations by developing a novel deep meta-reinforcement learning (DMRL) algorithm. The DMRL combines the meta strategy optimization together with DRL, and trains policies modulated by a latent space that can quickly adapt to new scenarios. We test the developed DMRL algorithm on the IEEE 300-bus system. We demonstrate fast adaptation of the meta-trained DRL polices with latent variables to new operating conditions and scenarios using the proposed method, which achieves superior performance compared to the state-of-the-art DRL and model predictive control (MPC) methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Estimation of abatement potentials and costs of air pollution emissions in China

Understanding the air pollution emission abatement potential and associated control cost is a prerequisite to design cost ef?cient control policies. In this study, a linear programming algorithm model, International Control Cost Estimate Tool, was updated with cost data for applications of 56 types of end-of-pipe technologies and ?ve types of renewable energy in 10 major sectors namely power generation, industry combustion, cement pro-duction, iron and steel production, other industry processes, domestic combustion, transportation, solvent use, livestock rearing, and fertilizer use. The updated model was implemented to estimate the abatement potential and marginal cost of multiple pollutants in China. The total maximum abatement potentials of sulfur dioxide (SO2), nitrogen oxides (NOx), primary particulate matter (PM2.5), non-volatile organic compounds (NMVOCs), and ammonia (NH3) in China were estimated to be 19.2, 20.8, 9.1, 17.2 and 8.6 Mt, respectively, which accounted for 89.7%, 89.9%, 94.6%, 74.0%, and 80.2% of their total emissions in 2014, respectively. The associated control cost of such reductions was estimated as 92.5, 469.7, 75.7, 449.0, and 361.8 billion CNY in SO2, NOx, primary PM2.5, NMVOCs and NH3, respectively. Shandong, Jiangsu, Henan, Zhejiang, and Guangdong provinces exhibited large abatement potentials for all pollutants. Provincial disparity analysis shows that high GDP regions tend to have higher reduction potential and total abatement costs. End-of-pipe technologies tended be a cost-ef?cient way to control pollution in industries processes (i.e., cement plants, iron and steel plants, lime production, building ceramic production, glass and brick production), whereas such technologies were less cost- effective in fossil fuel-related sectors (i.e., power plants, industry combustion, domestic combustion, and transportation) compared with renewable energy. The abatement potentials and marginal abatement cost curves developed in this study can further be used as a crucial component in an integrated model to design optimized cost-ef?cient control policies.

Zhang, Fenfen↗
DOE OSTI2025

Valued peaks: Sustainable water allocation for small hydropower plants in an era of explicit ecological needs

Optimizing hydropower operations to balance economic profitability and support functioning ecosystem services is integral to river management policy. In this article, we propose a dynamic, constrained optimization framework for small hydropower plants (SHPs) to evaluate trade-offs between economic profitability and socio-ecological requirements. Specifically, we examine the balance between short-term losses in hydropower generation and the potential for compensatory benefits in the form of revenue from recreational ecosystem services, irrespective of the direct beneficiary. Our framework integrates a fish habitat model, a hydropower optimization model, and a recreational ecosystem service estimate to evaluate different environmental flow scenarios. The optimization process gives three outflow release scenarios, informed by previous streamflow realisations (dam inflow), and designed environmental flow constraints. The framework is applied and tested for the river Kuusinkijoki in North-eastern Finland, which is a habitat for migratory brown trout and grayling populations. We show that the revenue loss due to the environmental flow constraints arises through a reduction in revenue per generated energy unit and through a reduction in turbine efficiency. Additionally, the simulation results reveal that all the designed environmental flow constraints cannot be met simultaneously. Under the environmental flow scenario with both minimum flow and flow ramping rate constraints, the annual hydropower revenue decreases by 16.5 %. An annual increase of 8 % in recreational fishing visits offsets the revenue loss. In conclusion, the developed framework provides knowledge of the costs and benefits of hydropower environmental flow constraints and guides the prioritizing process of environmental measures.

13 HYDRO ENERGY↗
DOE OSTI2019

SimCCS: An open-source tool for optimizing CO 2 capture, transport, and storage infrastructure

Commercial-scale carbon capture and storage (CCS) technology will involve deploying infrastructure on a massive and costly scale. This effort will require careful and comprehensive planning to ensure that capture locations, storage sites, and the dedicated CO 2 distribution pipelines are selected in a robust and cost-effective manner. Presented in 2009, SimCCS is an optimization model for integrated system design that enables researchers, stakeholders, and policy makers to design CCS infrastructure networks. SimCCS 2.0 is a complete, ground-up redesign that is now a portable software package, useable and shareable by the CCS research, industrial, policy, and public communities. SimCCS 2.0 integrates multiple new capabilities including a refined optimization model, novel candidate network generation techniques, and optional integration with high-performance computing platforms. Accessing user-provided CO 2 source, sink, and transportation data, SimCCS 2.0 creates candidate transportation routes and formalizes an optimization problem that determines the most cost-effective CCS system design. This optimization problem is further solved either through a high-performance computing interface, or through third-party software on a local desktop computing platform. Finally, SimCCS 2.0 employs an open-access geographic information system framework to enable analysis and visualization capabilities. SimCCS 2.0 is written in Java and is publicly available via GitHub to encourage collaboration, modification, and community development.

42 ENGINEERING↗
DOE OSTI2023

Explicit model predictive control through robust optimization

A strategy that calculates an explicit state feedback policy to regulate constrained uncertain discrete‐time uncertain linear systems is presented. We consider uncertain processes, affected by box‐bounded multiplicative uncertainty as well as bounded additive uncertainty with linear state and inputs constraints. The proposed method includes (i) the calculation of a terminal set constraint and (ii) the robust reformulation of state constraints in the prediction horizon. These features allow the derivation of the desired policy by solving a single multiparametric quadratic programming problem that guarantees feasible operation in the presence of uncertainty. Additionally, we employ variable and constraint elimination approaches to enhance the computational performance of the strategy. We demonstrate the steps and benefits of these developments with a numerical example and a chemical engineering case study.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Maintenance Engineering Basics & Best Practices

The purpose of this document is to concisely describe the basics of maintenance engineering, summarize maintenance policies at Los Alamos National Laboratory, and also review current industry best-practices through the use of case studies. Maintenance engineering incorporates the study of failure, reliability, policies, and procedures in order to optimize cost and equipment availability. Each of these topics will be covered within this report.

42 ENGINEERING↗
NASA NTRS2005

Cognitive Functioning in Space Exploration Missions: A Human Requirement

Solving cognitive issues in the exploration missions will require implementing results from both Human Behavior and Performance, and Space Human Factors Engineering. Operational and research cognitive requirements need to reflect a coordinated management approach with appropriate oversight and guidance from NASA headquarters. First, this paper will discuss one proposed management method that would combine the resources of Space Medicine and Space Human Factors Engineering at JSC, other NASA agencies, the National Space Biomedical Research Institute, Wyle Labs, and other academic or industrial partners. The proposed management is based on a Human Centered Design that advocates full acceptance of the human as a system equal to other systems. Like other systems, the human is a system with many subsystems, each of which has strengths and limitations. Second, this paper will suggest ways to inform exploration policy about what is needed for optimal cognitive functioning of the astronaut crew, as well as requirements to ensure necessary assessment and intervention strategies for the human system if human limitations are reached. Assessment strategies will include clinical evaluation and fitness-to-perform evaluations. Clinical intervention tools and procedures will be available to the astronaut and space flight physician. Cognitive performance will be supported through systematic function allocation, task design, training, and scheduling. Human factors requirements and guidelines will lead to well-designed information displays and retrieval systems that reduce crew time and errors. Means of capturing process, design, and operational requirements to ensure crew performance will be discussed. Third, this paper will describe the current plan of action, and future challenges to be resolved before a lunar or Mars expedition. The presentation will include a proposed management plan for research, involvement of various organizations, and a timetable of deliverables.

Fiedler, Edan↗
NASA NTRS2013

It's the Heat AND the Humidity -- Assessment of Extreme Heat Scenarios to Enable the Assessment of Climate Impacts on Public Health

In the United States, extreme heat is the most deadly weather-related hazard. In the face of a warming climate and urbanization, which contributes to local-scale urban heat islands, it is very likely that extreme heat events (EHEs) will become more common and more severe in the U.S. In a NASA-funded project supporting the National Climate Assessment, we are providing historical and future measures of extreme heat to enable assessments of the impacts of heat on public health over the coterminous U.S. We use atmospheric temperature and humidity information from meteorological reanalysis and from Global Climate Models (GCMs) to provide data on past and future heat events. The project s emphasis is on providing assessments of the magnitude, frequency and geographic distribution of extreme heat in the U.S. to facilitate public health studies. In our approach, long-term climate change is captured with GCM output, and the temporal and spatial characteristics of short-term extremes are represented by the reanalysis data. Two future time horizons, 2040 and 2090, are the focus of future assessments; these are compared to the recent past period of 1981-2000. We are characterizing regional-scale temperature and humidity conditions using GCM output for two climate change scenarios (A2 and A1B) defined in the Special Report on Emissions Scenarios (SRES). For each future period, 20 years of multi-model GCM output have been analyzed to develop a heat stress climatology based on statistics of extreme heat indicators. Differences between the two future and past periods have been used to define temperature and humidity changes on a monthly time scale and regional spatial scale. These changes, combined with hourly historical meteorological data at a spatial scale (12 km) much finer than that of GCMs, enable us to create future climate realizations, from which we compute the daily heat stress measures and related spatially-specific climatological fields. These include the mean annual number of days above certain thresholds of maximum and minimum air temperatures, heat indices and a new heat stress variable that gives an integrated measure of heat stress (and relief) over the course of a day. Comparisons are made between projected (2040 and 2090) and past (1990) heat stress statistics. All output is being provided at the 12 km spatial scale and will also be aggregated to the county level, which is a popular scale of analysis for public health researchers. County-level statistics will be made available by our collaborators at the Centers for Disease Control and Prevention (CDC) via the Wide-ranging Online Data for Epidemiologic Research (WONDER) system. CDC WONDER makes the information resources of the CDC available to public health professionals and the general public. This addition of heat stress measures to CDC WONDER will allow decision and policy makers to assess the impact of alternative approaches to optimize the public health response to EHEs. It will also allow public health researchers and policy makers to better include such heat stress measures in the context of national health data available in the CDC WONDER system. The users will be able to spatially and temporally query public health and heat-related data sets and create county-level maps and statistical charts of such data across the coterminous U.S.

Crosson, William L↗
DOE OSTI2025

Neural network approaches for parameterized optimal control

Here, we consider numerical approaches for deterministic, finite-dimensional optimal control problems whose dynamics depend on unknown or uncertain parameters. We seek to amortize the solution over a set of relevant parameters in an offline stage to enable rapid decision-making and be able to react to changes in the parameter in the online stage. To tackle the curse of dimensionality arising when the state and/or parameter are high-dimensional, we represent the policy using neural networks. We compare two training paradigms: First, our model-based approach leverages the dynamics and definition of the objective function to learn the value function of the parameterized optimal control problem and obtain the policy using a feedback form. Second, we use actor-critic reinforcement learning to approximate the policy in a data-driven way. Using an example involving a two-dimensional convection-diffusion equation, which features high-dimensional state and parameter spaces, we investigate the accuracy and efficiency of both training paradigms. While both paradigms lead to a reasonable approximation of the policy, the model-based approach is more accurate and considerably reduces the number of PDE solves.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimal Coordination of Electric Vehicles for Grid Services using Deep Reinforcement Learning

Recent research has shown the effectiveness of reinforcement learning (RL) in coordinating electric vehicles (EVs) with vehicle-to-grid capabilities for grid services. However, many of these studies rely on lookup table and deep Q-network techniques, which can be impractical when dealing with continuous states and actions. In addition, existing RL designs inadequately account for battery aging effects, EV user satisfaction, uncertain departure and arrival time, and trip distance, which may compromise effective coordination. This paper aims to bridge these gaps by developing an innovative deep deterministic policy gradient-based RL framework for optimal coordination of EVs. Case studies were carried out using a test system with 100 EVs, and numerical analysis results showed that the proposed RL framework can effectively coordinate EVs to maximize economic benefits and user satisfaction while ensuring the expected battery lifespan.

Das, Avijit↗
DOE OSTI2023

Deep reinforcement learning assisted co-optimization of Volt-VAR grid service in distribution networks

With the increasing penetration of distributed energy resources in distribution networks, Volt-VAR control and optimization (VVC/VVO) have become very important to ensure an acceptable quality of service to all customers. System operators can rely on slow-responding utility devices, including capacitor banks and on-load tap changing transformers, along with fast-responding battery and photovoltaic (PV) inverters for the VVC/VVO implementation. Because of variations in response time of these two classes of devices, and different control actions (discrete versus continuous), coordinated and optimal scheduling and operation have become of utmost importance. Here, this paper develops a look-ahead deep reinforcement learning (DRL)-based multi-objective VVO technique to improve the voltage profile of active distribution networks, decrease network and inverter power loss, and save the operational cost of the grid. It proposes a deep deterministic policy gradient (DDPG)-based approach to schedule the optimal reactive and/or active power set-points of fast-responding inverters, and a deep Q-network (DQN)-based DRL agent to schedule the discrete decisions variables of slow-responding assets. The reactive power output of PV and battery smart inverters are scheduled at 30-minute intervals and the capacitors’ commitment status is scheduled with several hour intervals. The proposed framework is validated on the modified IEEE 34-bus and 123-bus test cases with embedded PV and PV-plus-storage. To validate the efficacy of the proposed VVO, it is compared with several scenarios, including the base case without VVO, localized droop control of DERs, DDPG-only, and twin delayed DDPG (TD3) agent-based DRL techniques. The results justify the superior performance of the proposed method to improve the voltage profile, reduce network power loss, and minimize the look-ahead grid operational cost while minimizing the undesirable power losses in inverters as a result of power factor adjustments.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2015

A Technical and Economic Optimization Approach to Exploring Offshore Renewable Energy Development in Hawaii

This study was conducted with the support of the U.S. Department of Energy’s (DOE’s) Wind and Water Power Technologies Office (WWPTO) as part of ongoing efforts to minimize key risks and reduce the cost and time associated with permitting and deploying ocean renewable energy. The focus of the study was to discuss a possible approach to exploring scenarios for ocean renewable energy development in Hawaii that attempts to optimize future development based on technical, economic, and policy criteria. The goal of the study was not to identify potentially suitable or feasible locations for development, but to discuss how such an approach may be developed for a given offshore area. Hawaii was selected for this case study due to the complex nature of the energy climate there and DOE’s ongoing involvement to support marine spatial planning for the West Coast. Primary objectives of the study included 1) discussing the political and economic context for ocean renewable energy development in Hawaii, especially with respect to how inter-island transmission may affect the future of renewable energy development in Hawaii; 2) applying a Geographic Information System (GIS) approach that has been used to assess the technical suitability of offshore renewable energy technologies in Washington, Oregon, and California, to Hawaii’s offshore environment; and 3) formulate a mathematical model for exploring scenarios for ocean renewable energy development in Hawaii that seeks to optimize technical and economic suitability within the context of Hawaii’s existing energy policy and planning.

58 GEOSCIENCES↗
DOE OSTI2022

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗
DOE OSTI2023

Optimal Power Management of Battery Energy Storage Systems via Ensemble Kalman Inversion

Optimal power management of battery energy storage systems (BESS) is crucial for their safe and efficient operation. Numerical optimization techniques are frequently utilized to solve the optimal power management problems. However, these techniques often fall short of delivering real-time solutions for large-scale BESS due to their computational complexity. To address this issue, this paper proposes a computationally efficient approach. We introduce a new set of decision variables called power-sharing ratios corresponding to each cell, indicating their allocated power share from the output power demand. We then formulate an optimal power management problem to minimize the system-wide power losses while ensuring compliance with safety, balancing, and power supply-demand match constraints. To efficiently solve this problem, a parametrized control policy is designed and leveraged to transform the optimal power management problem into a parameter estimation problem. We then implement the ensemble Kalman inversion to estimate the optimal parameter set. The proposed approach significantly reduces computational requirements due to 1) the much lower dimensionality of the decision parameters and 2) the estimation treatment of the optimal power management problem. Finally, we conduct extensive simulations to validate the effectiveness of the proposed approach. The results show promise in accuracy and computation time compared with explored numerical optimization techniques.

Farakhor, Amir↗
DOE OSTI2024

REopt Model Overview and Example Use Cases [Slides]

REopt(R) is a mixed-integer optimization model that minimizes the lifecycle cost of serving energy loads at a site. This work provides and introduction to the model along with its key workflow, techno-economic inputs, key outputs, and key caveats for readers to understand REopt the when, why, how of using this model. This resource also includes helpful links related to REopt model and the data sources it uses during the optimization.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗