SEARCH · Search NASA

Results for “Imitation network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

22 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗

DOE OSTI2024

Learning Constrained Parametric Differentiable Predictive Control Policies With Guarantees

We present differentiable predictive control (DPC), a method for offline learning of constrained neural control policies for nonlinear dynamical systems with performance guarantees. We show that the sensitivities of the parametric optimal control problem can be used to obtain direct policy gradients. Specifically, we employ automatic differentiation (AD) to efficiently compute the sensitivities of the model predictive control (MPC) objective function and constraints penalties. To guarantee safety upon deployment, we derive probabilistic guarantees on closed-loop stability and constraint satisfaction based on indicator functions and Hoeffding’s inequality. We empirically demonstrate that the proposed method can learn neural control policies for various parametric optimal control tasks. In particular, we show that the proposed DPC method can stabilize systems with unstable dynamics, track time-varying references, and satisfy nonlinear state and input constraints. Our DPC method has practical time savings compared to alternative approaches for fast and memory-efficient controller design. Specifically, DPC does not depend on a supervisory controller as opposed to approximate MPC based on imitation learning. We demonstrate that, without losing performance, DPC is scalable with greatly reduced demands on memory and computation compared to implicit and explicit MPC while being more sample efficient than model-free reinforcement learning (RL) algorithms.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2008

Intelligent Integrated Health Management for a System of Systems

An intelligent integrated health management system (IIHMS) incorporates major improvements over prior such systems. The particular IIHMS is implemented for any system defined as a hierarchical distributed network of intelligent elements (HDNIE), comprising primarily: (1) an architecture (Figure 1), (2) intelligent elements, (3) a conceptual framework and taxonomy (Figure 2), and (4) and ontology that defines standards and protocols. Some definitions of terms are prerequisite to a further brief description of this innovation: A system-of-systems (SoS) is an engineering system that comprises multiple subsystems (e.g., a system of multiple possibly interacting flow subsystems that include pumps, valves, tanks, ducts, sensors, and the like); 'Intelligent' is used here in the sense of artificial intelligence. An intelligent element may be physical or virtual, it is network enabled, and it is able to manage data, information, and knowledge (DIaK) focused on determining its condition in the context of the entire SoS; As used here, 'health' signifies the functionality and/or structural integrity of an engineering system, subsystem, or process (leading to determination of the health of components); 'Process' can signify either a physical process in the usual sense of the word or an element into which functionally related sensors are grouped; 'Element' can signify a component (e.g., an actuator, a valve), a process, a controller, an actuator, a subsystem, or a system; The term Integrated System Health Management (ISHM) is used to describe a capability that focuses on determining the condition (health) of every element in a complex system (detect anomalies, diagnose causes, prognosis of future anomalies), and provide data, information, and knowledge (DIaK) not just data to control systems for safe and effective operation. A major novel aspect of the present development is the concept of intelligent integration. The purpose of intelligent integration, as defined and implemented in the present IIHMS, is to enable automated analysis of physical phenomena in imitation of human reasoning, including the use of qualitative methods. Intelligent integration is said to occur in a system in which all elements are intelligent and can acquire, maintain, and share knowledge and information. In the HDNIE of the present IIHMS, an SoS is represented as being operationally organized in a hierarchical-distributed format. The elements of the SoS are considered to be intelligent in that they determine their own conditions within an integrated scheme that involves consideration of data, information, knowledge bases, and methods that reside in all elements of the system. The conceptual framework of the HDNIE and the methodologies of implementing it enable the flow of information and knowledge among the elements so as to make possible the determination of the condition of each element. The necessary information and knowledge is made available to each affected element at the desired time, satisfying a need to prevent information overload while providing context-sensitive information at the proper level of detail. Provision of high-quality data is a central goal in designing this or any IIHMS. In pursuit of this goal, functionally related sensors are logically assigned to groups denoted processes. An aggregate of processes is considered to form a system. Alternatively or in addition to what has been said thus far, the HDNIE of this IIHMS can be regarded as consisting of a framework containing object models that encapsulate all elements of the system, their individual and relational knowledge bases, generic methods and procedures based on models of the applicable physics, and communication processes (Figure 2). The framework enables implementation of a paradigm inspired by how expert operators monitor the health of systems with the help of (1) DIaK from various sources, (2) software tools that assist in rapid visualization of the condition of the system, (3) analical software tools that assist in reasoning about the condition, (4) sharing of information via network communication hardware and software, and (5) software tools that aid in making decisions to remedy unacceptable conditions or improve performance.

Smith, Harvey↗

NASA NTRS1990

EDNA: Expert fault digraph analysis using CLIPS

Traditionally fault models are represented by trees. Recently, digraph models have been proposed (Sack). Digraph models closely imitate the real system dependencies and hence are easy to develop, validate and maintain. However, they can also contain directed cycles and analysis algorithms are hard to find. Available algorithms tend to be complicated and slow. On the other hand, the tree analysis (VGRH, Tayl) is well understood and rooted in vast research effort and analytical techniques. The tree analysis algorithms are sophisticated and orders of magnitude faster. Transformation of a digraph (cyclic) into trees (CLP, LP) is a viable approach to blend the advantages of the representations. Neither the digraphs nor the trees provide the ability to handle heuristic knowledge. An expert system, to capture the engineering knowledge, is essential. We propose an approach here, namely, expert network analysis. We combine the digraph representation and tree algorithms. The models are augmented by probabilistic and heuristic knowledge. CLIPS, an expert system shell from NASA-JSC will be used to develop a tool. The technique provides the ability to handle probabilities and heuristic knowledge. Mixed analysis, some nodes with probabilities, is possible. The tool provides graphics interface for input, query, and update. With the combined approach it is expected to be a valuable tool in the design process as well in the capture of final design knowledge.

Dixit, Vishweshwar V.↗