SEARCH · Search NASA

Results for “Policy optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

All sources NASA NTRS DOE OSTI

NASA NTRS2014

Multiagent Flight Control in Dynamic Environments with Cooperative Coevolutionary Algorithms

Dynamic environments in which objectives and environmental features change with respect to time pose a difficult problem with regards to planning optimal paths through these environments. Path planning methods are typically computationally expensive, and are often difficult to implement in real time if system objectives are changed. This computational problem is compounded when multiple agents are present in the system, as the state and action space grows exponentially with the number of agents in the system. In this work, we use cooperative coevolutionary algorithms in order to develop policies which control agent motion in a dynamic multiagent unmanned aerial system environment such that goals and perceptions change, while ensuring safety constraints are not violated. Rather than replanning new paths when the environment changes, we develop a policy which can map the new environmental features to a trajectory for the agent while ensuring safe and reliable operation, while providing 92% of the theoretically optimal performance.

Coevolution↗

DOE OSTI2018

NREL Tools Help Identify and Evaluate Renewable Energy Projects

NREL offers no-cost, publicly available web tools that can be used throughout the development of behind-the-meter renewable energy projects. These tools help gauge initial project potential, optimize system sizing, and refine project economics. This flyer compares four tools that can be used at various stages of project development: Economic Site Analysis (ESA) Tool, PVWatts Calculator, REopt (TM) Lite, and System Advisor Model (SAM).

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Blueprinting Electrified Transit System Implementation

To achieve a more affordable and reliable transportation system, we need to smartly upgrade our power systems and install a large number of charging stations, but conventional planning methods are not up to the task. By applying advanced simulation and optimization tools, we can design a smarter, more cost-effective electric transportation network. The initial focus was on public transit systems, demonstrating how this approach can deliver broader economic, reliability, and air quality benefits nationwide.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2017

Optimal deployment of resources for maximizing impact in spreading processes

Significance Spreading processes play an increasingly important role in marketing, opinion setting, and epidemic modeling. Most existing algorithms for optimal resource allocation in spreading processes are based on topological characteristics of the underlying network and aim to maximize impact at infinite time. Clearly, realistic and efficient real-time allocation policies should consider both network properties and details of the dynamics; additionally, control may be applied only to a restricted set of accessible nodes, and impact should be maximized in a limited time window. We introduce a probabilistic targeting framework that incorporates the dynamics and encompasses previously considered optimization formulations. It is based on a scalable dynamic message-passing approach that allows for the solution of large real-world network instances.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2020

An empirical study of I/O separation for burst buffers in HPC systems

To meet the exascale I/O requirements for the High-Performance Computing (HPC), a new I/O subsystem, Burst Buffer, based on solid state drives (SSD), has been developed. However, the diverse HPC workloads and the bursty I/O pattern cause severe data fragmentation that requires costly garbage collection (GC) and increases the number of bytes written to the SSD. To address this data fragmentation challenge, a new multi-stream feature has been developed for SSDs. In this work, we develop an I/O Separation scheme called BIOS to leverage this multi-stream feature to group the I/O streams based on the user IDs. We propose a stream-aware scheduling policy based on burst buffer pools in the workload manager, and integrate the BIOS with the workload manager to optimize the I/O separation scheme in burst buffer. We evaluate the proposed framework with a burst buffer I/O traces from Cori Supercomputer including a diverse set of applications. Experimental results show that the BIOS could improve the performance by 1.44x on average and reduce the Write Amplification Factor (WAF) by up to 1.20x. Finally, these demonstrate the potential benefits of the I/O separation scheme for solid state storage systems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

IRIS-GNN: Leveraging Graph Neural Networks for Scheduling on Truly Heterogeneous Runtime Systems

The diversity of accelerators in computer systems poses significant challenges for software developers, such as managing vendor-specific compiler toolchains, code fragmentation requiring different kernel implementations, and performance portability issues. To address these, the Intelligent Runtime System (IRIS) was developed. IRIS works across various systems, from smartphones to supercomputers, enabling automatic performance scaling based on available accelerators. It introduces abstract tasks for seamless execution transitions between accelerators while ensuring memory consistency and task dependencies. Although IRIS simplifies system details, optimal dynamic scheduling still requires user input to understand workload structures. To address this, we introduce a new scheduling policy for IRIS, termed IRIS-GNN, which is the first IRIS hybrid policy that operates in conjunction with the dynamic policies. This policy employs a Graph-Neural Network (GNN) to conduct Graph Classification of any task graphs submitted to IRIS. This GNN analyzes the structure and attributes of the task graph, categorizing it as either locality, concurrency, or mixed. This classification subsequently guides the selection of the dynamic policy used by IRIS. We provide a comparison of the performance of IRIS-GNN against the complete spectrum of IRIS’s dynamic policies, assess the overhead introduced by the GNN within this scheduling framework, and ultimately explore its practical application in real-world scenarios.

Johnston, Beau↗

DOE OSTI2023

A Self-Sustained CPS Design for Reliable Wildfire Monitoring

Continuous monitoring of areas nearby the electric grid is critical for preventing and early detection of devastating wildfires. Existing wildfire monitoring systems are intermittent and oblivious to local ambient risk factors, resulting in poor wildfire awareness. Ambient sensor suites deployed near the gridlines can increase the monitoring granularity and detection accuracy. However, these sensors must address two challenging and competing objectives at the same time. First, they must remain powered for years without manual maintenance due to their remote locations. Second, they must provide and transmit reliable information if and when a wildfire starts. The first objective requires aggressive energy savings and ambient energy harvesting, while the second requires continuous operation of a range of sensors. To the best of our knowledge, this paper presents the first self-sustained cyber-physical system that dynamically co-optimizes the wildfire detection accuracy and active time of sensors. The proposed approach employs reinforcement learning to train a policy that controls the sensor operations as a function of the environment (i.e., current sensor readings), harvested energy, and battery level. Here, the proposed cyber-physical system is evaluated extensively using real-life temperature, wind, and solar energy harvesting datasets and an open-source wildfire simulator. In long-term (5 years) evaluations, the proposed framework achieves 89% uptime, which is 46% higher than a carefully tuned heuristic approach. At the same time, it averages a 2-minute initial response time, which is at least 2.5× faster than the same heuristic approach. Furthermore, the policy network consumes 0.6 mJ per day on the TI CC2652R microcontroller using TensorFlow Lite for Micro, which is negligible compared to the daily sensor suite energy consumption.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation

Whittle index policy is a heuristic to the intractable restless multi-armed bandits (RMAB) problem. Although it is provably asymptotically optimal, finding Whittle indices remains difficult. In this paper, we present Neural-Q-Whittle, a Whittle index based Q-learning algorithm for RMAB with neural network function approximation, which is an example of nonlinear two-timescale stochastic approximation with Q-function values updated on a faster timescale and Whittle indices on a slower timescale. Despite the empirical success of deep Q-learning, the non-asymptotic convergence rate of Neural-Q-Whittle, which couples neural networks with two-timescale Q-learning largely remains unclear. This paper provides a finite-time analysis of Neural-Q-Whittle, where data are generated from a Markov chain, and Q-function is approximated by a ReLU neural network. Our analysis leverages a Lyapunov drift approach to capture the evolution of two coupled parameters, and the nonlinearity in value function approximation further requires us to characterize the approximation error. Combing these provide Neural-Q-Whittle with convergence rate, where is the number of iterations.

reinforcement learning, structured learning, conve↗

DOE OSTI2016

Thermal energy storage to minimize cost and improve efficiency of a polygeneration district energy system in a real-time electricity market

District energy systems can produce low-cost utilities for large energy networks, but can also be a resource for the electric grid by their ability to ramp production or to store thermal energy by responding to real-time market signals. In this work, dynamic optimization exploits the flexibility of thermal energy storage by determining optimal times to store and extract excess energy. This concept is applied to a polygeneration distributed energy system with combined heat and power, district heating, district cooling, and chilled water thermal energy storage. The system is a university campus responsible for meeting the energy needs of tens of thousands of people. The objective for the dynamic optimization problem is to minimize cost over a 24-h period while meeting multiple loads in real time. The paper presents a novel algorithm to solve this dynamic optimization problem with energy storage by decomposing the problem into multiple static mixed-integer nonlinear programming (MINLP) problems. Another innovative feature of this work is the study of a large, complex energy network which includes the interrelations of a wide variety of energy technologies. Results indicate that a cost savings of 16.5% is realized when the system can participate in the wholesale electricity market.

25 ENERGY STORAGE↗

DOE OSTI2022

Scalable Energy System Expansion Under Uncertainty Using Multi-Stage Stochastic Optimization

The intermittent nature of renewable energy sources poses challenges for electrical grids. This is due to the variable and uncertain nature of the power output from these resources. These features of renewable generation are more relevant to energy system planning as grids reach higher penetration levels of renewable energy. We present approaches for energy system planning based on scalable computational approaches which enable explicit consideration of operational uncertainties in the planning process. Using multi-stage stochastic programming and the progressive hedging algorithm, we compute energy system expansion decisions on modified versions of the RTS-GMLC test system.

electrical↗

DOE OSTI2020

REopt: Energy Decision Analysis

REopt transforms complex energy project decisions into actionable results for building owners, utilities, and industry. Based on decades of NREL decision-support expertise, REopt guides investment in economic, resilient energy technologies. This presentation provides an overview of how REopt can help building owners, utilities, developers, and industry optimize the economic and resilience benefits of distributed energy resources.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2014

The development of WEEE management and effects of the fund policy for subsidizing WEEE treating in China

Highlights: • The paper systematically introduces recycling system of WEEE management in China. • The paper introduces the new policy about WEEE management in China. • This paper analyzes the results of its implementation. • This paper evaluates the effectiveness to WEEE management in China. - Abstract: The consumption of electrical and electronic equipment is surging, so is the generation of waste electrical and electronic equipment (WEEE). Due to the large quantity, high potential risk and valuable capacity of WEEE, many countries are taking measures to regulate the management of WEEE. The environmental pollution and human health-harming problems caused by irregular treatment of WEEE in China make the government pay more and more attention to its management. This paper reviews the development of WEEE management in China, introduces the new policy which is established for WEEE recycling and especially analyzes the effectiveness of the policy, including huge recovery, formation of new recycling system, strict supervision to related enterprises, and the stimulation to public awareness. Based on the current achievement, some recommendations are given to optimize the WEEE management in China.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2020

REopt Lite Web Tool: Capabilities and Features

REopt Lite transforms complex energy project decisions into actionable results for building owners, utilities, and industry. Based on decades of NREL decision-support expertise, the free, publicly available REopt Lite web tool guides investment in economic, resilient energy technologies. And open source and API access to the tool enables energy analysis at scale. This presentation provides an overview of how REopt Lite can help users optimize the economic and resilience benefits of distributed energy resources.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2023

A game-theoretic approach to nuclear fuel cycle transition analysis under uncertainty

We present a novel methodology for optimizing nuclear fuel cycle transitions that incorporates a game-theoretic approach and captures interactions among multiple decision makers. The methodology is demonstrated using a two-person sequential game with uncertainty, where the two players represent a policy maker and an electric utility company, though the method generalizes to any number and type of individual decision making entities. Coupled with a sophisticated nuclear fuel cycle simulator, rich transition scenarios may be analyzed to identify robust transition strategies. These strategies explicitly treat uncertainties using a stochastic programming approach, devising optimal near-term hedging strategies that simultaneously consider all possible states of the world, maintaining flexibility to allow for intelligent recourse decisions once uncertainties are resolved. In the demonstration game, reactor technology and fuel cycle scheme adopted by the electric utility are shown to depend on both the policy maker’s decisions and the distributions over uncertain technological and economic outcomes.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2016

Mathematical Modelling-Based Energy System Operation Strategy Considering Energy Storage Systems

Renewable energy resources are widely recognized as an alternative to environmentally harmful fossil fuels. More renewable energy technologies will need to penetrate into fossil fuel dominated energy systems to mitigate the globally witnessed climate changes and environmental pollutions. It is necessary to prepare for the potential problems with increased proportions of renewable energy in the energy system, to prevent higher costs and decreased reliability. Motivated by this need, this paper addresses the operation of an energy system with an energy storage system in the context of developing a decision-supporting framework.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Greenhouse Gas Accounting Procedures in Low Carbon Fuel Policies Overlook the Spatial Variability of Miscanthus-Derived Sustainable Aviation Fuel

Low carbon fuel policies such as the U.S. Renewable Fuel Standard (RFS), Canada Clean Fuel Regulations (CFR), and California Low Carbon Fuel Standard (LCFS) as well as the 45Z tax credit are intended to reduce greenhouse gas (GHG) emissions from transportation. Cellulosic feedstocks, optimized biorefineries, and favorable farming locations can significantly reduce biofuel carbon intensity (CI). Despite advances in field-to-fuel GHG monitoring and flexibility in resource allocation within biorefineries (e.g., governing net electricity production), rigid CI accounting procedures in current policies may limit CI responsiveness across candidate sites and processing facilities. Here, this work examines a hypothetical biomass-to-sustainable aviation fuel (SAF) pathway using miscanthus and alcohol-to-jet (i) to demonstrate how GHG accounting requirements drive estimates of biofuel CIs and (ii) to explore potential CI and financial implications of scenario-specific life cycle assessment (LCA). Results demonstrate that GHG accounting using the CFR/LCFS can reasonably account for distinct levels of net electricity production by a biorefinery, but only the CFR yields similar CI sensitivity to spatially explicit factors (feedstock CI, grid electricity CI) as scenario-specific LCA: most GHG accounting frameworks do not capture CI variation across candidate sites in the United States. Ultimately, this work demonstrates the importance of LCA methodological specifications in low carbon fuel policies and tax credits.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2026

Financial-technical co-design for capital-intensive, resource-responsive energy systems

Because of their capital-intensive operation, wind energy systems that are competitive in terms of the cost of the energy that they produce lead to risk-reward trade-offs that make their business cases less favorable than those of conventional energy generation technologies. However, wind energy systems tend to be designed to maximize energy production or minimize cost of energy rather than to maximize their business cases. In this work, we attempt to exploit designs specifically tailored to business cases. We develop a novel framework for analyzing energy systems that ties their design variables to monthly operating incomes using simple models and historical hourly market and resource data. Using this approach, we demonstrate that for a wind site with abundant wind resource in the California Independent System Operator market, we can control the trade-off between mean and 5th percentile monthly returns by choosing the specific power of the turbine at a fixed modeled initial capital cost. Our framework gives a measure of the risk-reward spectrum of energy generation assets that could be built at a given site with respect to the sub-annual resource/market variation.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2023

Convex Q-Learning in Continuous Time with Application to Dispatch of Distributed Energy Resources

Convex Q-learning is a recent approach to reinforcement learning, motivated by the possibility of a firmer theory for convergence, and the possibility of making use of greater a priori knowledge regarding policy or value function structure. This paper explores algorithm design in the continuous time domain, with a finite-horizon optimal control objective. The main contributions are (i) The new Q-ODE: a model-free characterization of the Hamilton-Jacobi-Bellman equation. (ii) A formulation of Convex Q-learning that avoids approximations appearing in prior work. The Bellman error used in the algorithm is defined by filtered measurements, which is necessary in the presence of measurement noise. (iii) Convex Q-learning with linear function approximation is a convex program. It is shown that the constraint region is bounded, subject to an exploration condition on the training input. (iv) The theory is illustrated in application to resource allocation for distributed energy resources, for which the theory is ideally suited.

Lu, Fan↗