Search NASA⌕ Search

SEARCH · Search NASA

Results for “Policy optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Systems Safety Evaluation Based on System Thinking

Every engineer knows well and always considers about 'The local optimization makes the entire worst', and therefore well balanced design is required. Even the total safety has been assured by the safety design based on defense in depth concept, operational problem would occur by the chain of the fallacy of the defense in depth, then safety culture degradation, and finely organizational accident eventually happened. Probabilistic approach is required to the problem. We should answer to the question 'Safety Goal: How safe is safe enough?' for public acceptance. To respond to the operational problem and safety goal question, PRA, Probabilistic Risk Analysis is required. Further, Risk Communication should include comparison and selection among technical systems by risk and benefit analysis. Therefore, systems safety consideration is important based on systems thinking with considering following characteristics: - Goal-oriented- Optimization, Alternative proposals; - Global perspective- Bird's Eye; - Hierarchy- Isomorphism; - Life Cycle: Plan-Design-Construction-Use-Disposal; - Learning and Evolution. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Taking stock of national climate policies to evaluate implementation of the Paris Agreement

Many countries have implemented national climate policies to accomplish pledged Nationally Determined Contributions and to contribute to the temperature objectives of the Paris Agreement on climate change. In 2023, the global stocktake will assess the combined effort of countries. Here, based on a public policy database and a multi-model scenario analysis, we show that implementation of current policies leaves a median emission gap of 22.4 to 28.2 GtCO 2 eq by 2030 with the optimal pathways to implement the well below 2 °C and 1.5 °C Paris goals. If Nationally Determined Contributions would be fully implemented, this gap would be reduced by a third. Interestingly, the countries evaluated were found to not achieve their pledged contributions with implemented policies (implementation gap), or to have an ambition gap with optimal pathways towards well below 2 °C. This shows that all countries would need to accelerate the implementation of policies for renewable technologies, while efficiency improvements are especially important in emerging countries and fossilfuel- dependent countries.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2011

Achieving the Proper Balance Between Crew and Public Safety

A paramount objective of all human-rated launch and reentry vehicle developers is to ensure that the risks to both the crew onboard and the public are minimized within reasonable cost, schedule, and technical constraints. Past experience has shown that proper attention to range safety requirements necessary to ensure public safety must be given early in the design phase to avoid additional operational complexities or threats to the safety of people onboard, and the design engineers must give these requirements the same consideration as crew safety requirements. For human spaceflight, the primary purpose and operational concept for any flight safety system is to protect the public while maximizing the likelihood of crew survival. This paper will outline the policy considerations, technical issues, and operational impacts regarding launch and reentry vehicle failure scenarios where crew and public safety are intertwined and thus addressed optimally in an integrated manner. An overview of existing range and crew safety policy requirements will be presented. Application of these requirements and lessons learned from both the Space Shuttle and Constellation Programs will also be discussed. Using these past programs as examples, the paper will detail operational, design, and analysis approaches to mitigate and balance the risks to people onboard and in the public. Manned vehicle perspectives from the Federal Aviation Administration (FAA) and Air Force organizations that oversee public safety will be summarized as well. Finally, the paper will emphasize the need to factor policy, operational, and analysis considerations into the early design trades of new vehicles to help ensure that both crew and public safety are maximized to the greatest extent possible.

Gowan, John↗
DOE OSTI2016

Dispatchability Maximization for Co-Optimized Energy and Reserve Dispatch With Explicit Reliability Guarantee

In this paper, we consider dispatchability as the set of all admissible nodal wind power injections that will not cause infeasibility in real-time dispatch (RTD). Our work reveals that the dispatchability of the affine policy based RTD (AF-RTD) is a polytope whose coefficients are linear functions of the generation schedule and the gain matrix of affine policy. Two mathematical formulations of the dispatchability maximized energy and reserve dispatch (DM-ERD) are proposed. The first one maximizes the distance from the forecast to the boundaries of the dispatchability polytope subject to the available production cost or reserve cost. Provided the forecast value and variance of wind power, the generalized Gauss inequality (GGI) is adopted to evaluate the probability of infeasible RTD without the exact probability distribution of wind power. Combining the first formulation and the GGI approach, the second one minimizes the total cost subject to a desired reliability level through dispatchability maximization. Efficient convex optimization based algorithms are developed to solve these two models. Different from the conventional robust optimization method, our model does not rely on the specific uncertainty set of wind generation and directly optimizes the uncertainty accommodation capability of the power system. The proposed method is also compared with the affine policy based robust energy and reserve dispatch (AR-ERD). Case studies on the PJM 5-bus system illustrate the proposed concept and method. Experiments on the IEEE 118-bus system demonstrate the applicability of our method on moderate sized systems and its scalability to large dimensional uncertainty.

17 WIND ENERGY↗
NASA NTRS2010

Aircraft Trajectory Optimization and Contrails Avoidance in the Presence of Winds

There are indications that persistent contrails can lead to adverse climate change, although the complete effect on climate forcing is still uncertain. A flight trajectory optimization algorithm with fuel and contrails models, which develops alternative flight paths, provides policy makers the necessary data to make tradeoffs between persistent contrails mitigation and aircraft fuel consumption. This study develops an algorithm that calculates wind-optimal trajectories for cruising aircraft while avoiding the regions of airspace prone to persistent contrails formation. The optimal trajectories are developed by solving a non-linear optimal control problem with path constraints. The regions of airspace favorable to persistent contrails formation are modeled as penalty areas that aircraft should avoid and are adjustable. The tradeoff between persistent contrails formation and additional fuel consumption is investigated, with and without altitude optimization, for 12 city-pairs in the continental United States. Without altitude optimization, the reduction in contrail travel times is gradual with increase in total fuel consumption. When altitude is optimized, a two percent increase in total fuel consumption can reduce the total travel times through contrail regions by more than six times. Allowing further increase in fuel consumption does not seem to result in proportionate decrease in contrail travel times.

Ng, Hok K.↗
NASA NTRS2010

Concepts and Challenges for Environmentally Friendly En Route Operations

A flight trajectory optimization algorithm with fuel and contrails models, which develops alternative flight paths, provides policy makers the necessary data to make tradeoffs between persistent contrails mitigation and aircraft fuel consumption. This study develops an algorithm that calculates wind-optimal trajectories for cruising aircraft while avoiding the regions of airspace prone to persistent contrails formation. The optimal trajectory is derived using Singular Perturbation Method. The regions of airspace favorable to persistent contrails formation are modeled as high-risk areas that aircraft should avoid and are adjustable. The tradeoffs between persistent contrails formation and additional travel time are investigated for wind-optimal trajectories and various contrails-avoidance trajectories at 10 different cruising altitudes for flights departing from Chicago and San Diego to New York. The additional travel times required for avoiding 100% persistent contrails formation at various flight altitudes ranged from approximately 0% to 4.3% for flights from Chicago to New York. For flights between San Diego and New York, additional traveling times vary between 1.3% and 5% depending on the cruise altitude and the percentage of contrail avoidance. Talk will present the results of aircraft fuel consumptions that are proportional to the travel time for cruising aircraft.

Sridhar, Banavar↗
NASA NTRS1996

Decision-Aiding and Optimization for Vertical Navigation of Long-Haul Aircraft

Most decisions made in the cockpit are related to safety, and have therefore been proceduralized in order to reduce risk. There are very few which are made on the basis of a value metric such as economic cost. One which can be shown to be value based, however, is the selection of a flight profile. Fuel consumption and flight time both have a substantial effect on aircraft operating cost, but they cannot be minimized simultaneously. In addition, winds, turbulence, and performance vary widely with altitude and time. These factors make it important and difficult for pilots to (a) evaluate the outcomes associated with a particular trajectory before it is flown and (b) decide among possible trajectories. The two elements of this problem considered here are: (1) determining what constitutes optimality, and (2) finding optimal trajectories. Pilots and dispatchers from major u.s. airlines were surveyed to determine which attributes of the outcome of a flight they considered the most important. Avoiding turbulence-for passenger comfort-topped the list of items which were not safety related. Pilots' decision making about the selection of flight profile on the basis of flight time, fuel burn, and exposure to turbulence was then observed. Of the several behavioral and prescriptive decision models invoked to explain the pilots' choices, utility maximization is shown to best reproduce the pilots' decisions. After considering more traditional methods for optimizing trajectories, a novel method is developed using a genetic algorithm (GA) operating on a discrete representation of the trajectory search space. The representation is a sequence of command altitudes, and was chosen to be compatible with the constraints imposed by Air Traffic Control, and with the training given to pilots. Since trajectory evaluation for the GA is performed holistically, a wide class of objective functions can be optimized easily. Also, using the GA it is possible to compare the costs associated with different airspace design and air traffic management policies. A decision aid is proposed which would combine the pilot's notion of optimality with the GA-based optimization, provide the pilot with a number of alternative pareto-optimal trajectories, and allow him to consider unmodelled attributes and constraints in choosing among them. A solution to the problem of displaying alternatives in a multi-attribute decision space is also presented.

Patrick, Nicholas J. M.↗
NASA NTRS2006

A robust model predictive control algorithm for uncertain nonlinear systems that guarantees resolvability

A robustly stabilizing MPC (model predictive control) algorithm for uncertain nonlinear systems is developed that guarantees resolvability. With resolvability, initial feasibility of the finite-horizon optimal control problem implies future feasibility in a receding-horizon framework. The control consists of two components; (i) feed-forward, and (ii) feedback part. Feed-forward control is obtained by online solution of a finite-horizon optimal control problem for the nominal system dynamics. The feedback control policy is designed off-line based on a bound on the uncertainty in the system model. The entire controller is shown to be robustly stabilizing with a region of attraction composed of initial states for which the finite-horizon optimal control problem is feasible. The controller design for this algorithm is demonstrated on a class of systems with uncertain nonlinear terms that have norm-bounded derivatives and derivatives in polytopes. An illustrative numerical example is also provided.

feed - forward↗
NASA NTRS2005

Small Body GN&C Research Report: A Robust Model Predictive Control Algorithm with Guaranteed Resolvability

A robustly stabilizing MPC (model predictive control) algorithm for uncertain nonlinear systems is developed that guarantees the resolvability of the associated finite-horizon optimal control problem in a receding-horizon implementation. The control consists of two components; (i) feedforward, and (ii) feedback part. Feed-forward control is obtained by online solution of a finite-horizon optimal control problem for the nominal system dynamics. The feedback control policy is designed off-line based on a bound on the uncertainty in the system model. The entire controller is shown to be robustly stabilizing with a region of attraction composed of initial states for which the finite-horizon optimal control problem is feasible. The controller design for this algorithm is demonstrated on a class of systems with uncertain nonlinear terms that have norm-bounded derivatives, and derivatives in polytopes. An illustrative numerical example is also provided.

model predictive control algorithm (MPC)↗
DOE OSTI2020

Multi-task deep reinforcement learning for intelligent multi-zone residential HVAC control

In this short communication, a data-driven deep reinforcement learning (deep RL) method is applied to minimize HVAC users’ energy consumption costs while maintaining users’ comfort. The applied deep RL method's efficiency is enhanced by conducting multi-task learning that can achieve an economic control strategy for a multi-zone residential HVAC system in both cooling and heating scenarios. The applied multi-task deep RL method is compared with a rule-based benchmark case and a single-task deep deterministic policy gradient algorithm to verify its effective and generalized application in optimizing HVAC operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Volt-VAR Optimization in Distribution Networks Using Twin Delayed Deep Reinforcement Learning

Modern distribution grids are undergoing new challenges due to the stochastic nature of distributed energy resources (DERs). High penetration of DERs has a significant impact on Volt-VAR profile and system power losses. This work proposes a deep reinforcement learning (DRL)-based Volt-VAR optimization approach for improving voltage profile and reducing system power loss under high penetration of distributed energy resources, such as battery energy storage and solar photovoltaic units in distribution grids. The twin delayed deep deterministic policy gradient (TD3) method-based DRL agent is proposed to configure optimal set-points of reactive power outputs of fast responding smart inverters. The agent schedules the reactive power of inverters according to their physical capabilities, such as minimum allowed power factor, e.g., 0.9 leading/lagging. The reward function of the proposed DRL scheme is designed carefully to ensure a proper voltage profile of the grids with effective scheduling of reactive power outputs from inverters. The performance of the proposed model is verified on modified IEEE 34- and 123-bus systems and compared with base case with no reactive supply by inverters, and local droop Volt-VAR control approach. The results show that the proposed method performs better than the local droop control and deep deterministic policy gradient (DDPG)-based DRL method for reducing voltage fluctuation and minimizing power loss.

Hossain, Rakib↗
DOE OSTI2024

Feedback Optimization of Incentives for Distribution Grid Services

Energy prices and net power injection limitations regulate the operations in distribution grids and typically ensure that operational constraints are met. Nevertheless, unexpected or prolonged abnormal events could undermine the grid's functioning. During contingencies, customers could contribute effectively to sustaining the network by providing services. Herein this paper proposes an incentive mechanism that promotes users' active participation by essentially altering the energy pricing rule. The incentives are modeled via a linear function whose parameters can be computed by the system operator (SO) by solving an optimization problem. Feedback-based optimization algorithms are then proposed to seek optimal incentives by leveraging measurements from the grid, even in the case when the SO does not have a full grid and customer information. Numerical simulations on a standard testbed validate the proposed approach.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Binary Quantum Control Optimization with Uncertain Hamiltonians

Optimizing the controls of quantum systems plays a crucial role in advancing quantum technologies. The time-varying noises in quantum systems and the widespread use of inhomogeneous quantum ensembles raise the need for high-quality quantum controls under uncertainties. In this paper, we consider a stochastic discrete optimization formulation of a discretized binary optimal quantum control problem involving Hamiltonians with predictable uncertainties. We propose a sample-based reformulation that optimizes both risk-neutral and risk-averse measurements of control policies, and solve these with two gradient-based algorithms using sum-up-rounding approaches. Furthermore, we discuss the differentiability of the objective function and prove upper bounds of the gaps between the optimal solutions to binary control problems and their continuous relaxations. We conduct numerical simulations on various sized problem instances based on two applications of quantum pulse optimization; we evaluate different strategies to mitigate the impact of uncertainties in quantum systems. In conclusion, we demonstrate that the controls of our stochastic optimization model achieve significantly higher quality and robustness compared with the controls of a deterministic model.

conditional value-at-risk (CVaR)↗
DOE OSTI2023

Global-Local Policy Search and its Application in Grid-Interactive Building Control

As the buildings sector represents over 70% of the total U.S. electricity consumption, it offers a great amount of untapped demand-side resources to tackle many critical grid-side problems and improve the overall energy system's efficiency. To help make buildings grid-interactive, this paper proposes a global-local policy search method to train a reinforcement learning (RL) based controller which optimizes building operation during both normal hours and demand response (DR) events. Experiments on a simulated five-zone commercial building demonstrate that by adding a local fine-tuning stage to the evolution strategy policy training process, the control costs can be further reduced by 7.55% in unseen testing scenarios. Baseline comparison also indicates that the learned RL controller outperforms a pragmatic linear model predictive controller (MPC), while not requiring intensive online computation.

demand response↗
NASA NTRS2022

SatNet: A Benchmark for Satellite Scheduling Optimization

Satellites provide essential services such as networking and weather tracking, and the number of near-earth and deep space satellites are expected to grow rapidly in the coming years. Communications with terrestrial ground stations is one of the critical functionalities of any space mission. Satellite scheduling is a problem that has been scientifically investigated since the 1970s. A central aspect of this problem is the need to consider resource contention and satellite visibility constraints as they require line of sight. Due to the combinatorial nature of the problem, prior solutions such as linear programs and evolutionary algorithms require extensive compute capabilities to output a feasible schedule for each scenario. Machine learning based scheduling can provide an alternative solution by training a model with historical data and generating a schedule quickly with model inference. We present SatNet, a benchmark for satellite scheduling optimization based on historical data from the NASA Deep Space Network. We propose formulation of the satellite scheduling problem as a Markov Decision Process and use reinforcement learning (RL) policies to generate schedules. The nature of constraints imposed by SatNet differ from other combinatorial optimization problems such as vehicle routing studied in prior literature. Our initial results indicate that RL is an alternative optimization approach that can generate candidate solutions of comparable quality to existing state-of-the-practice results. However, we also find that RL policies overfit to the training dataset and do not generalize well to new data, thereby necessitating continued research on reusable and generalizable agents.

Wilson, Brian↗
DOE OSTI2019

Energy savings with outdoor temperature-based smart ventilation control strategies in advanced California homes

We simulated the energy performance of smart ventilation controls based on outdoor temperature in homes located in California climate regions, designed to comply with the 2016 Title 24, Part 6 California Energy Code prescriptive requirements. The smart controls shift ventilation rates in time, but ensure an annual occupant pollutant exposure equal to, or less than, what would be experienced in a home with a constant ventilation rate equal to the whole house target airflow in the ASHRAE Standard 62.2-2016. Annual simulations of previously developed prototype homes with validated ventilation models were conducted via co-simulation of EnergyPlus with CONTAM. Controller performance varied substantially by climate zone, airtightness and house prototype. Smart controls were generally ineffective in Climate Zone 1 (Arcata) due to its lack of a cooling season and low diurnal temperature swings. The most successful smart controls used parameters pre-calculated using an optimization routine. In order to better inform state energy policy, we also determined savings weighted by the amount of new home construction in each climate. The best controls averaged about one-third of ventilation-related energy savings that increased to about 48–55% when weighted. Annual average savings were about 650–700 kWh/year for the best controllers. The vast majority of site energy savings were for heating end-uses (>90% of total savings). Whole house ventilation rates increased between 0% and 42%, with typical increases in the 15–20% range. Finally, the best-performing control strategies shifted ventilation rates seasonally and modulated flows within each season. Peak concentrations during low ventilation times were kept below an exposure level of 3, well below the maximum of 5 required to avoid acute exposure issues.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1982

Economic modeling of fault tolerant flight control systems in commercial applications

This paper describes the current development of a comprehensive model which will supply the assessment and analysis capability to investigate the economic viability of Fault Tolerant Flight Control Systems (FTFCS) for commercial aircraft of the 1990's and beyond. An introduction to the unique attributes of fault tolerance and how they will influence aircraft operations and consequent airline costs and benefits is presented. Specific modeling issues and elements necessary for accurate assessment of all costs affected by ownership and operation of FTFCS are delineated. Trade-off factors are presented, aimed at exposing economically optimal realizations of system implementations, resource allocation, and operating policies. A trade-off example is furnished to graphically display some of the analysis capabilities of the comprehensive simulation model now being developed.

Finelli, G. B.↗
NASA NTRS1992

Neurocontrol and neurobiology - New developments and connections

At McDonnell-Douglas, controllers which combine adaptive critic networks with the use of backpropagation in real time have solved difficult control problems crucial to the feasibility of building the National Aerospace Plane (NASP) able to reach earth orbit. As details emerged, parallels to neurobiology have grown stronger and have begun to lead to empirical possibilities of importance to neuroscience. This has led to thoughts of institutional collaboration facilitating what could become a Newtonian revolution in neuroscience, with cognitive implications as well. The authors elaborate on each of these points. The topics discussed are recent progress in neurocontrol; progress in optimization and reinforcement learning; implications for neurobiology and science policy; and a new view of the brain.

Werbos, Paul J.↗