Search NASA⌕ Search

SEARCH · Search NASA

Results for “Policy optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1987

Control-System Design Program

Control-theory design package, Optimal Regulator Algorithms for Control of Linear Systems (ORACLS), developed to aid in design of controllers and optimal filters for systems modeled by linear, time-invariant differential and difference equations. Optimal linear quadratic regulator theory, Linear-Quadratic-Gaussian (LQG) problem, most widely accepted method of determining optimal control policy. Provides for solution to time-in-variant continuous or discrete LQG problems. Attractive to control-system designer providing rigorous tool for dealing with multi-input and multi-output dynamic systems in continuous and discrete form. CDO version written in FORTRAN IV. VAX version written in FORTRAN 77.

Frisch, Harold P.↗
NASA NTRS1988

The Feasibility of the Disturbance Accommodating Controller for Precision Antenna Pointing

The objective of this study is to investigate the feasibility of a pointing (position loop) controller for the NASA-JPL Deep Space Network (DSN) antennas using the Disturbance Accommodating Control (DAC) theory. A model that includes state dependent disturbances was developed, and an example demonstrating the noise estimator is presented as an initial phase in the controller design. The goal is to improve pointing accuracy by the removal of the systematic errors caused by the antenna misalignment as well as sensor noise and random wind and thermal disturbances. Preliminary simulation results show that the DAC technique is successful in both cancelling the imposed errors and maintaining an optimal control policy.

Gresham, L. L.↗
NASA NTRS1989

Analysis of a turning point problem in flight trajectory optimization

The optimal control policy for the aeroglide portion of the minimum fuel, orbital plane change problem for maneuvering entry vehicles is reduced to the solution of a turning point problem for the bank angle control. For this problem a turning point occurs at the minimum altitude of the flight, when the flight path angle equals zero. The turning point separates the bank angle control into two outer solutions that are valid away from the turning point. In a neighborhood of the turning point, where the bank angle changes rapidly, an inner solution is developed and matched with the two outer solutions. An asymptotic analysis of the turning point problem is given, and an analytic example is provided to illustrate the construction of the bank angle control.

Gracey, C.↗
NASA NTRS1994

Damage-Mitigating Control of Space Propulsion Systems for High Performance and Extended Life

A major goal in the control of complex mechanical system such as spacecraft rocket engine's advanced aircraft, and power plants is to achieve high performance with increased reliability, component durability, and maintainability. The current practice of decision and control systems synthesis focuses on improving performance and diagnostic capabilities under constraints that often do not adequately represent the materials degradation. In view of the high performance requirements of the system and availability of improved materials, the lack of appropriate knowledge about the properties of these materials will lead to either less than achievable performance due to overly conservative design, or over-straining of the structure leading to unexpected failures and drastic reduction of the service life. The key idea in this report is that a significant improvement in service life could be achieved by a small reduction in the system dynamic performance. The major task is to characterize the damage generation process, and then utilize this information in a mathematical form to synthesize a control law that would meet the system requirements and simultaneously satisfy the constraints that are imposed by the material and structural properties of the critical components. The concept of damage mitigation is introduced for control of mechanical systems to achieve high performance with a prolonged life span. A model of fatigue damage dynamics is formulated in the continuous-time setting, instead of a cycle-based representation, for direct application to control systems synthesis. An optimal control policy is then formulated via nonlinear programming under specified constraints of the damage rate and accumulated damage. The results of simulation experiments for the transient upthrust of a bipropellant rocket engine are presented to demonstrate efficacy of the damage-mitigating control concept.

Ray, Asok↗
NASA NTRS1995

Damage-mitigating control of a reusable rocket engine for high performance and extended life

The goal of damage mitigating control in reusable rocket engines is to achieve high performance with increased durability of mechanical structures such that functional lives of the critical components are increased. The major benefit is an increase in structural durability with no significant loss of performance. This report investigates the feasibility of damage mitigating control of reusable rocket engines. Phenomenological models of creep and thermo-mechanical fatigue damage have been formulated in the state-variable setting such that these models can be combined with the plant model of a reusable rocket engine, such as the Space Shuttle Main Engine (SSME), for synthesizing an optimal control policy. Specifically, a creep damage model of the main thrust chamber wall is analytically derived based on the theories of sandwich beam and viscoplasticity. This model characterizes progressive bulging-out and incremental thinning of the coolant channel ligament leading to its eventual failure by tensile rupture. The objective is to generate a closed form solution of the wall thin-out phenomenon in real time where the ligament geometry is continuously updated to account for the resulting deformation. The results are in agreement with those obtained from the finite element analyses and experimental observation for both Oxygen Free High Conductivity (OFHC) copper and a copper-zerconium-silver alloy called NARloy-Z. Due to its computational efficiency, this damage model is suitable for on-line applications of life prediction and damage mitigating control, and also permits parametric studies for off-line synthesis of damage mitigating control systems. The results are presented to demonstrate the potential of life extension of reusable rocket engines via damage mitigating control. The control system has also been simulated on a testbed to observe how the damage at different critical points can be traded off without any significant loss of engine performance. The research work reported here is built upon concepts derived from the disciplines of Controls, Thermo-fluids, Structures, and Materials. The concept of damage mitigation, as presented in this report, is not restricted to control of rocket engines. It can be applied to any system where structural durability is an important issue.

Ray, Asok↗
NASA NTRS1995

File-System Workload on a Scientific Multiprocessor

Many scientific applications have intense computational and I/O requirements. Although multiprocessors have permitted astounding increases in computational performance, the formidable I/O needs of these applications cannot be met by current multiprocessors a their I/O subsystems. To prevent I/O subsystems from forever bottlenecking multiprocessors and limiting the range of feasible applications, new I/O subsystems must be designed. The successful design of computer systems (both hardware and software) depends on a thorough understanding of their intended use. A system designer optimizes the policies and mechanisms for the cases expected to most common in the user's workload. In the case of multiprocessor file systems, however, designers have been forced to build file systems based only on speculation about how they would be used, extrapolating from file-system characterizations of general-purpose workloads on uniprocessor and distributed systems or scientific workloads on vector supercomputers (see sidebar on related work). To help these system designers, in June 1993 we began the Charisma Project, so named because the project sought to characterize 1/0 in scientific multiprocessor applications from a variety of production parallel computing platforms and sites. The Charisma project is unique in recording individual read and write requests-in live, multiprogramming, parallel workloads (rather than from selected or nonparallel applications). In this article, we present the first results from the project: a characterization of the file-system workload an iPSC/860 multiprocessor running production, parallel scientific applications at NASA's Ames Research Center.

Kotz, David↗
DOE OSTI2020

Cutting the Deployment Costs of Physics-Based MPC in Buildings by Simulation-Based Imitation Learning

It has been shown that model predictive control (MPC) is a promising solution for energy-efficient building operations. However, the deployment of MPC in a large portion of the building stock has not been possible partially because of high installation costs. Every building is unique and requires a tailored MPC solution. The best performing solutions are often based on physics-based modeling, which is, however, computationally expensive and requires dedicated software. A promising direction that tackles this problem is to train a neural network-based optimal control policy to imitate the behavior of physics-based MPC from the simulation data generated offline. The neural networks give control actions that closely approximate those produced by physics-based MPC, but with a fraction of the computational and memory requirements and without the need for licensed software. The main advantage of the proposed approach stems from simple evaluation at execution time, leading to low computational footprints and easy deployment on embedded HW platforms. In the case study, we present the energy savings potential of physics-based MPC applied to an office building in Belgium. We demonstrate how neural network approximators can be used to cut the implementation and maintenance costs of MPC deployment without compromising performance. We also critically assess the presented approach by pointing out the remaining challenges and open research questions.

Drgona, Jan↗
DOE OSTI2019

Stripping off the implementation complexity of physics-based model predictive control for buildings via deep learning

Over the past decade, model predictive control (MPC) has been considered as the most promising solution for intelligent building operation. Despite extensive effort, transfer of this technology into practice is hampered by the need to obtain an accurate controller model with minimum effort, the need of expert knowledge to set it up, and the need of increased computational power and dedicated software to run it. A promising direction that tackles the last two problems was proposed by approximate explicit MPC where the optimal control policies are learned from MPC data via a suitable function approximator, e.g., a deep learning (DL) model. The main advantage of the proposed approach stems from simple evaluation at execution time leading to low computational footprints and easy deployment on embedded HW platforms. We present the energy savings potential of physics-based (also called 'white-box') MPC applied to an office building in Belgium. Moreover, we demonstrate how deep learning approximators can be used to cut the implementation and maintenance costs of MPC deployment without compromising performance. We also critically assess the presented approach by pointing out the major challenges and remaining open-research questions.

Drgona, Jan↗
DOE OSTI2023

Operation Optimization Using Reinforcement Learning with Integrated Artificial Reasoning Framework

In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.

Kim, Junyung↗
DOE OSTI2022

Risk-Informed Operations and Maintenance Decision Making Using Deep Reinforcement Learning

A challenge for operating nuclear power plants is the significant cost of operations and maintenance, at times consuming up to 66% of annual operating costs. This project aims to build a framework for a risk-informed asset-management tool that integrates inspections, repairs, spare-part inventory, supply chain, and business choices to lower overall O&M costs. Our approach uses a combination of data-driven modeling and deep reinforcement learning to create and implement optimal maintenance policies for the existing nuclear fleet, as well as new advanced reactors. The creation of an asset management tool that uses these advanced methods will give operators new capabilities to help reduce the burden of O&M spending in nuclear power plants.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

A Multiobjective Optimization Framework for Online Stochastic Optimal Control in Hybrid Electric Vehicles

The increasing urgency to extract additional efficiency from hybrid propulsion systems has led to the development of advanced power management control algorithms. In this paper we address the problem of online optimization of the supervisory power management control in parallel hybrid electric vehicles (HEVs). We model HEV operation as a controlled Markov chain and we show that the control policy yielding the Pareto optimal solution minimizes online the long-run expected average cost per unit time criterion. The effectiveness of the proposed solution is validated through simulation and compared to the solution derived with dynamic programming using the average cost criterion. Both solutions achieved the same cumulative fuel consumption demonstrating that the online Pareto control policy is an optimal control policy.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Shifting agricultural practices to produce sustainable, low carbon intensity feedstocks for biofuel production

The carbon intensity (CI) of biofuel's well-to-pump life cycle is calculated by life cycle analysis (LCA) to account for the energy/material inputs of the feedstock production and fuel conversion stages and the associated greenhouse gas (GHG) emissions during these stages. The LCA is used by the California Air Resources Board's Low Carbon Fuel Standard (LCFS) program to calculate CI and monetary credits are issued based on the difference between a given fuel's CI and a reference fuel's CI. Through the Tier 2 certification program under which individual fuel production facilities can submit their own CIs with their facility input data, the LCFS has driven innovative technologies to biofuel conversion facilities, resulting in substantial reductions in GHG emissions as compared to the baseline gasoline or diesel. A similar approach can be taken to allow feedstock petition in the LCFS so that lower-CI feedstock can be rewarded. Here we examined the potential for various agronomic practices to improve the GHG profiles of corn ethanol by performing feedstock-level CI analysis for the Midwestern United States. Our system boundary covers GHG emissions from the cradle-to-farm-gate activities (i.e. farm input manufacturing and feedstock production), along with the potential impacts of soil organic carbon change during feedstock production. We conducted scenario-based CI analysis of ethanol, coupled with regionalized inventory data, for various farming practices to manage corn fields, and identified key parameters affecting cradle-to-farm-gate GHG emissions. The results demonstrate large spatial variations in CI of ethanol due to farm input use and land management practices. In particular, adopting conservation tillage, reducing nitrogen fertilizer use, and implementing cover crops has the potential to reduce GHG emissions per unit corn produced when compared to a baseline scenario of corn–soybean rotation. This work shows a large potential emission offset opportunity by allowing feedstock producers a path to Tier 2 petitions that reward low-CI feedstocks and further reduce biofuels' CI. The prevalence of significant acreage that has not been optimized for CI suggests that policy changes that incentivize optimization of this parameter could provide significant additionality over current trends in farm efficiency and adoption of conservation practice.

09 BIOMASS FUELS↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
DOE OSTI2017

Optimization of Low Level Waste Disposal Policy, Strategies and Techniques - 17201

The mission of low level radioactive waste disposal facilities is to accept and dispose of waste in compliance with all applicable laws, regulations, license requirements, and social expectations. These facilities need to provide a cost effective disposal service to the waste generators, so that the cost savings could be passed down to business, industry, and ultimately, to the general public. To fulfil this mission, waste disposal facilities need to establish and maintain review and optimization mechanisms to respond to changing circumstances that may impact the long-term safe, cost effective operation of their facilities. Maintaining existing disposal resources available as long as possible, managing increased demand for disposal space as a consequence of growing number of nuclear facilities to be decommissioned, and developing capacity for future needs and uncertainties are some of the key challenges that drive optimization of disposal facilities. While the concept of optimization of low level waste disposal facilities is not new, the framework for holistic optimization of waste disposal facilities, enabling them to provide cost effective, sustainable services to the nuclear industry, as well as maintaining optimized safety and radiation protection systems is a notable gap in the current radioactive waste management body of knowledge. A comprehensive optimization framework is required to support the implementation of safe, effective, and sustainable integrated waste management strategies. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS1996

Optimizing Input/Output Using Adaptive File System Policies

Parallel input/output characterization studies and experiments with flexible resource management algorithms indicate that adaptivity is crucial to file system performance. In this paper we propose an automatic technique for selecting and refining file system policies based on application access patterns and execution environment. An automatic classification framework allows the file system to select appropriate caching and pre-fetching policies, while performance sensors provide feedback used to tune policy parameters for specific system environments. To illustrate the potential performance improvements possible using adaptive file system policies, we present results from experiments involving classification-based and performance-based steering.

Madhyastha, Tara M.↗