Search NASA⌕ Search

SEARCH · Search NASA

Results for “policy optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Large scale nonlinear numerical optimal control for finite element models of flexible structures

This paper discusses the development of large scale numerical optimal control algorithms for nonlinear systems and their application to finite element models of structures. This work is based on our expansion of the optimal control algorithm (DDP) in the following steps: improvement of convergence for initial policies in non-convex regions, development of a numerically accurate penalty function method approach for constrained DDP problems, and parallel processing on supercomputers. The expanded constrained DDP algorithm was applied to the control of a four-bay, two dimensional truss with 12 soft members, which generates geometric nonlinearities. Using an explicit finite element model to describe the structural system requires 32 state variables and 10,000 time steps. Our numerical results indicate that for constrained or unconstrained structural problems with nonlinear dynamics, the results obtained by our expanded constrained DDP are significantly better than those obtained using linear-quadratic feedback control.

Shoemaker, Christine A.↗

Product Lifecycle Management and the Quest for Sustainable Space Exploration Solutions

Product Lifecycle Management (PLM) is an outcome of lean thinking to eliminate waste and increase productivity. PLM is inextricably tied to the systems engineering business philosophy, coupled with a methodology by which personnel, processes and practices, and information technology combine to form an architecture platform for product design, development, manufacturing, operations, and decommissioning. In this model, which is being implemented by the Marshall Space Flight Center (MSFC) Engineering Directorate, total lifecycle costs are important variables for critical decision-making. With the ultimate goal to deliver quality products that meet or exceed requirements on time and within budget, PLM is a powerful concept to shape everything from engineering trade studies and testing goals, to integrated vehicle operations and retirement scenarios. This briefing will demonstrate how the MSFC Engineering Directorate is implementing PLM as part of an overall strategy to deliver safe, reliable, and affordable space exploration solutions and how that strategy aligns with the Agency and Center systems engineering policies and processes. Sustainable space exploration solutions demand that all lifecycle phases be optimized, and engineering the next generation space transportation system requires a paradigm shift such that digital tools and knowledge management, which are central elements of PLM, are used consistently to maximum effect. Adopting PLM, which has been used by the aerospace and automotive industry for many years, for spacecraft applications provides a foundation for strong, disciplined systems engineering and accountable return on investment. PLM enables better solutions using fewer resources by making lifecycle considerations in an integrative decision-making process.

Caruso, Pamela W.↗

Controlling Tensegrity Robots through Evolution using Friction based Actuation

Traditional robotic structures have limitations in planetary exploration as their rigid structural joints are prone to damage in new and rough terrains. In contrast, robots based on tensegrity structures, composed of rods and tensile cables, offer a highly robust, lightweight, and energy efficient solution over traditional robots. In addition tensegrity robots can be highly configurable by rearranging their topology of rods, cables and motors. However, these highly configurable tensegrity robots pose a significant challenge for locomotion due to their complexity. This study investigates a control pattern for successful locomotion in tensegrity robots through an evolutionary algorithm. A twelve-rod hardware model is rapidly prototyped to utilize a new actuation method based on friction. A web-based physics simulation is created to model the twelve-rod tensegrity ball structure. Square-waves are used as control policies for the actuators of the tensegrity structure. Monte Carlo trials are run to find the most successful number of amplitudes for the square-wave control policy. From the results, an evolutionary algorithm is implemented to find the most optimized solution for locomotion of the twelve-rod tensegrity structure. The software pattern coupled with the new friction based actuation method can serve as the basis for highly efficient tensegrity robots in space exploration.

Tensegrit↗

Task allocation among multiple intelligent robots

Researchers describe the design of a decentralized mechanism for allocating assembly tasks in a multiple robot assembly workstation. Currently, the approach focuses on distributed allocation to explore its feasibility and its potential for adaptability to changing circumstances, rather than for optimizing throughput. Individual greedy robots make their own local allocation decisions using both dynamic allocation policies which propagate through a network of allocation goals, and local static and dynamic constraints describing which robots are elibible for which assembly tasks. Global coherence is achieved by proper weighting of allocation pressures propagating through the assembly plan. Deadlock avoidance and synchronization is achieved using periodic reassessments of local allocation decisions, ageing of allocation goals, and short-term allocation locks on goals.

Gasser, L.↗

Planning with Continuous Resources in Stochastic Domains

We consider the problem of optimal planning in stochastic domains with metric resource constraints. Our goal is to generate a policy whose expected sum of rewards is maximized for a given initial state. We consider a general formulation motivated by our application domain--planetary exploration--in which the choice of an action at each step may depend on the current resource levels. We adapt the forward search algorithm AO* to handle our continuous state space efficiently.

Mausam, Mausau↗

Fe3 Traffic Simulator Overview

NASA Ames developed a first of its kind air traffic simulation tool known as Flexible engine for Fast time evaluation of Flight environments (Fe3). The patent-pending technology is a robust simulation tool that provides the capability of statistically analyzing the high-density and low-altitude traffic system. With this capability, stakeholders can study the impacts of critical components (such as wind, surveillance, communication, collision, avoidance, traffic rules, energy consumption, etc.) in the low-altitude high-density traffic system, gain insights and help define requirements, policies, and protocols for a safe and efficient traffic system, and assess operational risks and optimize fight schedules.

Flexible engine for Fast time evaluation of Flight↗

Dynamic remapping decisions in multi-phase parallel computations

The effectiveness of any given mapping of workload to processors in a parallel system is dependent on the stochastic behavior of the workload. Program behavior is often characterized by a sequence of phases, with phase changes occurring unpredictably. During a phase, the behavior is fairly stable, but may become quite different during the next phase. Thus a workload assignment generated for one phase may hinder performance during the next phase. We consider the problem of deciding whether to remap a paralled computation in the face of uncertainty in remapping's utility. Fundamentally, it is necessary to balance the expected remapping performance gain against the delay cost of remapping. This paper treats this problem formally by constructing a probabilistic model of a computation with at most two phases. We use stochastic dynamic programming to show that the remapping decision policy which minimizes the expected running time of the computation has an extremely simple structure: the optimal decision at any step is followed by comparing the probability of remapping gain against a threshold. This theoretical result stresses the importance of detecting a phase change, and assessing the possibility of gain from remapping. We also empirically study the sensitivity of optimal performance to imprecise decision threshold. Under a wide range of model parameter values, we find nearly optimal performance if remapping is chosen simply when the gain probability is high. These results strongly suggest that except in extreme cases, the remapping decision problem is essentially that of dynamically determining whether gain can be achieved by remapping after a phase change; precise quantification of the decision model parameters is not necessary.

Nicol, D. M.↗

Fe(exp3) - A Monte Carlo Simulation Capability for Evaluation of New Air Traffic Management Concepts

The concepts of unmanned aircraft system traffic management (UTM) and urban air mobility (UAM) introduce high-density operations in low-altitude airspace and will change the paradigm of the traditional air traffic system. The Flexible engine for Fast-time evaluation of Flight environments (Fe3) provides the capability of statistically analyzing high-density, high-fidelity, and low-altitude traffic system without conducting infeasible and cost-prohibitive flight tests that involve a large volume of aerial vehicles. With this simulation capability, stakeholders can study the impacts of critical factors, define requirements, policies, and protocols needed to support a safe yet efficient traffic system, assess operational risks, and optimize flight schedules. This work provides an introduction to this simulation tool including its architecture and various models involved. Its performance and applications in high density air traffic operations are also presented.

Paralellization↗

NASA Earth Systems Digital Twins (ESDT)

"Similarly to artificial intelligence, which is now revolutionizing many aspects of our daily lives, Earth system digital twin technologies have the potential to revolutionize the way Earth Science research will be conducted in the future, and how results and knowledge from this research will provide information to support decision making and yield impactful societal benefits. An Earth System Digital Twin or ESDT is a dynamic and interactive information system that first provides a digital replica of the past and current states of the Earth or Earth system as accurately and timely as possible; second, allows for computing forecasts of future states under nominal assumptions and based on the current replica; and third, offers the capability to investigate many hypothetical scenarios under varying impact assumptions. In other words, an ESDT provides the integrated What-Now, What-Next, and What-If pictures of the Earth or Earth system, by continuously ingesting newly observed data and by leveraging multiple interconnected models, machine learning as well advanced computing and visualization capabilities. Digital twins have been developed in engineering since 2002, but the interest in digital twins for the Earth domain is more recent and stems from the convergence of several developments: - The huge amount of diverse data that has now been collected continuously for more than 50 years, and that is becoming more and more difficult to access, understand, and utilize. - At the same time, because of climate change and its impacts the information produced by all of this data is becoming of interest to many new non-traditional users for analyzing and predicting various phenomena. - Because of advances in computational and visualization capabilities and the parallel unprecedented development of machine learning (ML), extracting relevant information from these large amounts of data and running complex models faster has become possible. As a result, it is becoming necessary and possible to build intuitive and interactive frameworks that will enable users with various skill levels and/or organizational hierarchy levels to easily access large amounts of targeted information along with the relevant tools and models (Earth system and human activity models), to support them in analyzing and visualizing this information, to help them understand interactions among models, to visualize the potential outcomes of various impacts, and to support decision or policy making. The full power of digital twins is that, through an integrated representation and standardized tools and software technologies, the same digital replica can address the needs of multiple users at various resolutions (spatial and temporal) and for various applications (science, economic, policy, etc.) – “from farmer to scientist”. With all these interests at stake, the challenges of building optimal digital twins are many and complex. The first challenge is to determine if a Digital Twin should be global or local, and multi-domain or thematic. For example, some domains such as Climate or Weather will require a global Digital Twin or Digital Twin capabilities while science areas such as Biodiversity might be more local. We can also envision that multiple thematic ESDTs, e.g., Air Quality, Wildfires, Hydrology could be federated or provide input to other ESDTs, either on a regional level or to a more global ESDT. Overall, we can imagine a future “web” of Digital Twins co-existing in a hierarchy or in a network, and capable of being connected or federated depending on the needs. This last point brings up the very important challenge of interoperability, including standards and protocols that will need to be built into these systems from the beginning. Each individual digital twin would have full flexibility in internal construction but would need standards-based interfaces (input and output) or hooks to make it compatible with others. Another challenge when building digital twins will be to decide how to organize each digital replica. Based on the applications targeted by the DT under implementation, various amounts and types of raw data, Analysis Ready Data (ARD) and information will need to be incorporated. Depending on the required latencies and needs of the users, various solutions can be considered, including Data Cubes, Data Lakes, pointers, or computing information on demand. We envision that each ESDT will choose a solution adapted to its specific objectives. Another important challenge is the type(s) of visualization that will be used, as well as the level of interactivity and refresh rate that will be required. Again, this will depend on the objectives of the ESDT, but also on the various users’ needs. In most cases, several types of visualizations and human interfaces will need to be offered depending on the projected users of that system. In parallel to the challenges highlighted above, there are also many tools and technologies that will need to be developed or improved for all types of digital twins. Among those are improved machine learning technologies, for example providing explainability, but also ML techniques for causality and providing a better integration of physics models. Additionally, reliable uncertainty quantification methods will be needed for all ESDT components, from validating data fusion and assimilation to assessing the accuracy of ML models and weighing the values of decisions supported by those systems. This presentation introduces the ESDT concept, presents several ESDT use cases, and a proposed ESDT architecture framework, as well as various technologies being developed by the Advanced Information Systems Technology (AIST) Program."

Earth Science Remote Sensing; Information Systems↗

Multiagent Flight Control in Dynamic Environments with Cooperative Coevolutionary Algorithms

Dynamic flight environments in which objectives and environmental features change with respect to time pose a difficult problem with regards to planning optimal flight paths. Path planning methods are typically computationally expensive, and are often difficult to implement in real time if system objectives are changed. This computational problem is compounded when multiple agents are present in the system, as the state and action space grows exponentially. In this work, we use cooperative coevolutionary algorithms in order to develop policies which control agent motion in a dynamic multiagent unmanned aerial system environment such that goals and perceptions change, while ensuring safety constraints are not violated. Rather than replanning new paths when the environment changes, we develop a policy which can map the new environmental features to a trajectory for the agent while ensuring safe and reliable operation, while providing 92% of the theoretically optimal performance

Experimentation↗

Fe(3): An Evaluation Tool for Low-Altitude Air Traffic Operations

The concepts of unmanned aircraft system traffic management (UTM) and urban air mobility (UAM) are introducing high-density operations in low altitude airspace in closer proximity to populated areas than conventional high-altitude air traffic. The Flexible engine for Fast-time Evaluation of Flight Environments (Fe (sup 3)) provides the capability of statistically analyzing the high-density, high-fidelity, and low-altitude traffic system under numerous scenarios, such that stake holders can study impacts of factors in the low-altitude high-density traffic system and define requirements, policies, and protocols needed to support a safe yet efficient traffic system, and even assess operational risks and optimize flight schedules without conducting infeasible and cost-prohibitive flight tests that involve a large volume of aerial vehicles. This work provides an introduction to this simulation tool including its architecture and various models involved. Its performance and sample application in UAM and UTM are also presented.

Collision Avoidance↗

Fe3: An Evaluation Tool for Low-Altitude Air Traffic Operations

The concepts of unmanned aircraft system traffic management (UTM) and urban air mobility (UAM) are introducing high-density operations in low altitude airspace in closer proximity to populated areas than conventional high-altitude air traffic. The Flexible engine for Fast-time Evaluation of Flight Environments (Fe (sup 3)) provides the capability of statistically analyzing the high-density, high-fidelity, and low-altitude traffic system under numerous scenarios, such that stake holders can study impacts of factors in the low-altitude high-density traffic system and define requirements, policies, and protocols needed to support a safe yet efficient traffic system, and even assess operational risks and optimize flight schedules without conducting infeasible and cost-prohibitive flight tests that involve a large volume of aerial vehicles. This work provides an introduction to this simulation tool including its architecture and various models involved. Its performance and sample application in UAM and UTM are also presented.

Trajectory Modeling↗

Achieving the Proper Balance Between Crew and Public Safety

A paramount objective of all human-rated launch and reentry vehicle developers is to ensure that the risks to both the crew onboard and the public are minimized within reasonable cost, schedule, and technical constraints. Past experience has shown that proper attention to range safety requirements necessary to ensure public safety must be given early in the design phase to avoid additional operational complexities or threats to the safety of people onboard. This paper will outline the policy considerations, technical issues, and operational impacts regarding launch and reentry vehicle failure scenarios where crew and public safety are intertwined and thus addressed optimally in an integrated manner. Historical examples and lessons learned from both the Space Shuttle and Constellation Programs will be presented. Using these examples as context, the paper will discuss some operational, design, and analysis approaches to mitigate and balance the risks to people onboard and in the public. Manned vehicle perspectives from the FAA and Air Force organizations that oversee public safety will also be summarized. Finally, the paper will emphasize the need to factor policy, operational, and analysis considerations into the early design trades of new vehicles to help ensure that both crew and public safety are maximized to the greatest extent possible.

Gowan, John↗

Multiagent Flight Control in Dynamic Environments with Cooperative Coevolutionary Algorithms

Dynamic environments in which objectives and environmental features change with respect to time pose a difficult problem with regards to planning optimal paths through these environments. Path planning methods are typically computationally expensive, and are often difficult to implement in real time if system objectives are changed. This computational problem is compounded when multiple agents are present in the system, as the state and action space grows exponentially with the number of agents in the system. In this work, we use cooperative coevolutionary algorithms in order to develop policies which control agent motion in a dynamic multiagent unmanned aerial system environment such that goals and perceptions change, while ensuring safety constraints are not violated. Rather than replanning new paths when the environment changes, we develop a policy which can map the new environmental features to a trajectory for the agent while ensuring safe and reliable operation, while providing 92% of the theoretically optimal performance.

Coevolution↗

Climate Hazard Assessment for Stakeholder Adaptation Planning in New York City

This paper describes a time-sensitive approach to climate change projections, developed as part of New York City's climate change adaptation process, that has provided decision support to stakeholders from 40 agencies, regional planning associations, and private companies. The approach optimizes production of projections given constraints faced by decision makers as they incorporate climate change into long-term planning and policy. New York City stakeholders, who are well-versed in risk management, helped pre-select the climate variables most likely to impact urban infrastructure, and requested a projection range rather than a single 'most likely' outcome. The climate projections approach is transferable to other regions and consistent with broader efforts to provide climate services, including impact, vulnerability, and adaptation information. The approach uses 16 Global Climate Models (GCMs) and three emissions scenarios to calculate monthly change factors based on 30-year average future time slices relative to a 30- year model baseline. Projecting these model mean changes onto observed station data for New York City yields dramatic changes in the frequency of extreme events such as coastal flooding and dangerous heat events. Based on these methods, the current 1-in-10 year coastal flood is projected to occur more than once every 3 years by the end of the century, and heat events are projected to approximately triple in frequency. These frequency changes are of sufficient magnitude to merit consideration in long-term adaptation planning, even though the precise changes in extreme event frequency are highly uncertain

Horton, Radley M.↗

Remote monitoring of agricultural systems using NDVI time series and machine learning methods: a tool for an adaptive agricultural policy

This study aims to provide accurate information about changes in agricultural systems (AS) using phenological metrics derived from the NDVI time series. Use of such information could help land managers optimize land use choices and monitor the status of agricultural lands, under a variety of environmental and socioeconomic conditions. For this purpose, the Moderate Resolution Imaging Spectroradiometer (MODIS) NDVI data were used to derive phenological metrics over the Oum Er-Rbia basin (central Morocco). Random forest (RF), support vector machine (SVM), and K-nearest neighbor (KNN) classifiers were explored and compared on their ability to classify AS classes over the study area. Four main AS classes have been considered: (1) irrigated annual crop (IAC), (2) irrigated perennial crop (IPC), (3) rainfed area (RA), and (4) fallow (FA). By comparing the accuracy of the three classifiers, the RF method showed the best performance with an overall accuracy of 0.97 and kappa coefficient of 0.96.The RF method was then chosen to examine time variations in AS over a 16-year period (2000–2016). The AS main variations were detected and evaluated for the four AS classes. These variations have been found to be linked well with other indicators of local agricultural land management, as well as the historical agricultural drought changes over the study area. Overall, the results present a tool for decision makers to improve agricultural management and provide a different perspective in understanding the spatiotemporal dynamics of agricultural systems.

Youssef Lebrini↗

Managing the Risk for Early Onset Osteoporosis in Long-Duration Astronauts Due to Spaceflight

Early Onset Osteoporosis is probably the most recognized but poorly understood long-term health risk due to spaceflight. Osteoporosis management is primarily prophylactic and clinical interventions rely upon the ability to predict fractures which is currently determined by surrogate measures of bone strength. The RMAT for Early Onset Osteoporosis identified some open issues related to the fact that long-duration astronauts compose a unique group of subjects for which clinical approaches for osteoporosis management do not apply. Long-duration astronauts are healthy, young (25 to 55 years of age), predominantly male, and physical fit relative to the typical osteoporosis patient. Moreover, during prolonged space missions (typically 6-month missions) the skeleton not only adapts to weightlessness, but is influenced by numerous risk factors induced by operational constraints, e.g., inability to maintain preflight weight-bearing and aerobic activities, sub-optimal dietary intake (e.g., high sodium content for food stability, lack of fresh fruit and vegetables), suppression of vitamin D metabolism by uv shielding, and remote medicine care. Moreover, adaptation results in novel changes to astronauts bones that cannot be detected by current medically-useful measures. Consequently, a panel of clinicians (recognized leaders and policy-makers in osteoporosis) was convened to review the dataset of bone measures and bone loss risk factors in long-duration astronauts. Driven by the queries in the RMAT, the panel was charged to determine 1) if an intervention is required to prevent this risk, 2) what type and at what time would intervention be optimal, 3) what is the clinical trigger that would require a medical response from flight surgeons and 4) how should research data be used in the clinical care of astronauts. Hence, the RMAT determined that a bone health policy need to be formulated specific for this unique cohort subjected to a novel skeletal condition

Sibonga, Jean D.↗

A preference-ordered discrete-gaming approach to air-combat analysis

An approach to one-on-one air-combat analysis is described which employs discrete gaming of a parameterized model featuring choice between several closed-loop control policies. A preference-ordering formulation due to Falco is applied to rational choice between outcomes: win, loss, mutual capture, purposeful disengagement, draw. Approximate optimization is provided by an active-cell scheme similar to Falco's obtained by a 'backing up' process similar to that of Kopp. The approach is designed primarily for short-duration duels between craft with large-envelope weaponry. Some illustrative computations are presented for an example modeled using constant-speed vehicles and very rough estimation of energy shifts.

Kelley, H. J.↗