Search NASA⌕ Search

SEARCH · Search NASA

Results for “Runtime monitoring”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Integrating FRET with Copilot: Automated Translation of Natural Language Requirements to Runtime Monitors

Runtime verification (RV) enables monitoring systems at runtime, to detect property violations early and limit their potential consequences. To provide the level of assurance required for ultra-critical systems, monitor specifications must faithfully reflect the original mission requirements, which are often written in ambiguous natural language. This paper presents an end-to-end framework to capture requirements in structured natural language and generate monitors that capture their semantics faithfully. We leverage NASA’s Formal Requirement Elicitation Tool (FRET), and the RV system Copilot. We extend FRET with mechanisms to capture additional information needed to generate monitors, and introduce OGMA, a new tool to bridge the gap between FRET and Copilot. With this framework, users can write requirements in an intuitive format and obtain real-time C monitors suitable for use in embedded systems. Our tool chain is available as open source.

FRET↗
NASA NTRS2012

Quantified Event Automata: Towards Expressive and Efficient Runtime Monitors

Runtime verification is the process of checking a property on a trace of events produced by the execution of a computational system. Runtime verification techniques have recently focused on parametric specifications where events take data values as parameters. These techniques exist on a spectrum inhabited by both efficient and expressive techniques. These characteristics are usually shown to be conflicting - in state-of-the-art solutions, efficiency is obtained at the cost of loss of expressiveness and vice-versa. To seek a solution to this conflict we explore a new point on the spectrum by defining an alternative runtime verification approach.We introduce a new formalism for concisely capturing expressive specifications with parameters. Our technique is more expressive than the currently most efficient techniques while at the same time allowing for optimizations.

state machines↗
NASA NTRSDate not supplied

Runtime Monitoring for Unmanned Aerospace Systems with Neural Network Components

AI components (e.g., Deep Neural Networks) are increasingly used in unmanned Aerospace systems for safety-relevant applications. Rigorous Verification and Validation methods for such components are still in their infancy and thus, monitoring of the AI's behavior during runtime is essential. In this paper, we will present a runtime-monitoring architecture, which combines the advanced statistical analysis framework SYSAI (System Analysis using Statistical AI) with temporal and probabilistic runtime monitoring carried out by R2U2 (Realizable, Responsive, and Unobtrusive Unit). Learned statistical models of complex systems with AI components are produced by the SYSAI framework and provide detailed information to enable the R2U2 runtime monitor to efficiently perform advanced safety and performance checks in nominal and off-nominal conditions. We will present initial results of our tool set and architecture on a case study, a DNN-based autonomous centerline tracking system (ACT).

Yuning He↗
NASA NTRSDate not supplied

Runtime Monitoring with R2U2 for Aircraft Systems with Neural Networks

R2U2 (Realizable, Responsive, Unobtrusive Unit) is a hardware-supported tool and framework for real-time system monitoring and software health management of cyber-physical systems. During system operation, R2U2 continuously monitors properties about safety, performance, and security of the vehicle and its vital components and can perform diagnostic reasoning. Efficient observers for past-time and future-time Metric Temporal Logic, fast reasoners for Bayesian Networks, and model-based prognostics algorithms are key components of R2U2 and designed for minimal computational footprint. R2U2 has been implemented in software supporting ROS, NASA's cFS/cFE, and Simulink and as an FPGA configuration. The synergistic combination of monitors and observers in R2U2 makes it possible to design powerful models for system runtime monitoring, diagnostics, software health management, prognostics, and security monitoring. In this presentation, I will give a detailed overview of the R2U2 architecture and its features and will discuss the application of R2U2 for safety-monitoring of a neural-network based autonomous centerline tracking system (ACT) for autonomous aircraft.

Runtime Monitoring↗
NASA NTRSDate not supplied

Seeing is Believing: Monitoring Future Time Temporal Logic

Runtime monitors for future-time unbounded temporal logics like RVLTL, LTL 3 and FLTL, have double-exponential (2^2^n) worst-case space complexity bounds in size of the input formula. The semantics of these logics require monitors to perform general satisfiability solving for LTL expressions, a well-studied problem whose computational complexity is NP-hard and PSPACE-complete. This paper introduces an unbounded future-time linear temporal logic defined over a lattice. We call our logic an incremental temporal logic as it can be viewed as incrementally constructing proofs about the trace. On this account, we view online runtime monitoring as a decision procedure for proofs systems about incrementally growing traces. We demonstrate that our incremental temporal logic allows monitor construction to void satisfiability solving while still soundly detecting when the property is violated in an online fashion. This enables asymptotic improvements in space complexity. As proof, we provide a procedure to construct monitors that utilize linear space and time in the size of the input formula, while remaining constant in the size of the input stream and suitable for online monitoring. We further demonstrate, through several examples, that our incremental temporal logic is straightforward to adopt and practical for runtime verification.

temporal logic↗
NASA NTRS2018

Refining the Safety - Liveness Classification of Temporal Properties According to Monitorability

Runtime verification is the topic of analyzing execution traces using formal techniques. It includes monitoring the execution of a system against temporal properties, commonly to detect violations. Not every temporal property is fully monitorable however: in some cases, the correctness of the execution does not depend on any finite prefix. We study the connection between monitorability and Lamport’s classification of properties to safety and liveness and their dual classes.We refine the definition of monitorability and provide algorithms to check which verdicts can be expected, a priori and during runtime verification.

Peled, Doron↗
NASA NTRS2010

Monitoring Distributed Real-Time Systems: A Survey and Future Directions

Runtime monitors have been proposed as a means to increase the reliability of safety-critical systems. In particular, this report addresses runtime monitors for distributed hard real-time systems. This class of systems has had little attention from the monitoring community. The need for monitors is shown by discussing examples of avionic systems failure. We survey related work in the field of runtime monitoring. Several potential monitoring architectures for distributed real-time systems are presented along with a discussion of how they might be used to monitor properties of interest.

Goodloe, Alwyn E.↗
NASA NTRSDate not supplied

Monitoring ROS2: From Requirements to Autonomous Robots

Context/Goals - Capture mission requirements in high-level language. - Monitor requirements for robots. At a Glance - Transform requirements into runtime monitors for ROS2: - Requirements elicitation.* - Transform requirements into Temporal Logic formulas.* - Transform Temporal Logic formulas into runtime monitors.* - Generate hard real-time code for monitors.* - Generate ROS2 application directly from requirements.+ *Steps done in prior work. +Steps extended from prior work.

Runtime verification↗
NASA NTRSDate not supplied

From Requirements to Autonomous Flight: An Overview of the Monitoring ICAROUS Project

The Independent Configurable Architecture for Reliable Operations of Unmanned Systems(ICAROUS) is a software architecture incorporating a set of algorithms to enable autonomous operations of unmanned aircraft applications. This paper provides an overview of Monitoring ICAROUS, a project whose objective is to provide a formal approach to generating runtime monitors for autonomous systems from requirements written in a structured natural language. This approach integrates FRET, a formal requirement elicitation and authoring tool, and Copilot, a runtime verification framework. FRET is used to specify formal requirements in structured natural language. These requirements are translated into temporal logic formulae. Copilot is then used to generate executable runtime monitors from these temporal logic specifications. The generated monitors are directly integrated into ICAROUS to perform runtime verification during flight.

Formal Methods↗
NASA NTRSDate not supplied

Monitoring ICAROUS: From Requirements to Autonomous Flight

The Independent Configurable Architecture for Reliable Operations of Unmanned Systems (ICAROUS) is a software architecture incorporating a set of algorithms to enable autonomous operations of unmanned aircraft applications. This paper provides an overview of Monitoring ICAROUS, a project whose objective is to provide a formal approach to generating runtime monitors for autonomous systems from requirements written in a structured natural language. This approach integrates FRET, a formal requirement elicitation and authoring tool, and Copilot, a runtime verification framework. FRET is used to specify formal requirements in structured natural language. These requirements are translated into temporal logic formulae. Copilot is then used to generate executable runtime monitors from these temporal logic specifications. The generated monitors are directly integrated into ICAROUS to perform runtime verification during flight.

Formal Methods↗
DOE OSTI2016

Lightweight runtime checking of C programs with RTC

The C Programming Language is known for being an efficient language that can be compiled on almost any architecture and operating system. However the absence of dynamic safety checks and a relatively weak type system allows programmer oversights that are hard to spot. In this paper, we present RTC, a runtime monitoring tool that instruments unsafe code and monitors the program execution. RTC is built on top of the ROSE compiler infrastructure. RTC finds memory bugs and arithmetic overflows and underflows, and run-time type violations. Most of the instrumentations are directly added to the source file and only require a minimal runtime system. As a result, the instrumented code remains portable. In tests against known error detection benchmarks, RTC found 98% of all memory related bugs and had zero false positives. Finally, in performance tests conducted with well known algorithms, such as binary search and MD5, we determined that our tool has an average run-time overhead rate of 9.7× and memory overhead rate of 3.5×.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2015

R2U2: Monitoring and Diagnosis of Security Threats for Unmanned Aerial Systems

We present R2U2, a novel framework for runtime monitoring of security properties and diagnosing of security threats on-board Unmanned Aerial Systems (UAS). R2U2, implemented in FPGA hardware, is a real-time, REALIZABLE, RESPONSIVE, UNOBTRUSIVE Unit for security threat detection. R2U2 is designed to continuously monitor inputs from the GPS and the ground control station, sensor readings, actuator outputs, and flight software status. By simultaneously monitoring and performing statistical reasoning, attack patterns and post-attack discrepancies in the UAS behavior can be detected. R2U2 uses runtime observer pairs for linear and metric temporal logics for property monitoring and Bayesian networks for diagnosis of security threats. We discuss the design and implementation that now enables R2U2 to handle security threats and present simulation results of several attack scenarios on the NASA DragonEye UAS.

Formal Methods↗
NASA NTRS2012

Optimized Temporal Monitors for SystemC

SystemC is a modeling language built as an extension of C++. Its growing popularity and the increasing complexity of designs have motivated research efforts aimed at the verification of SystemC models using assertion-based verification (ABV), where the designer asserts properties that capture the design intent in a formal language such as PSL or SVA. The model then can be verified against the properties using runtime or formal verification techniques. In this paper we focus on automated generation of runtime monitors from temporal properties. Our focus is on minimizing runtime overhead, rather than monitor size or monitor-generation time. We identify four issues in monitor generation: state minimization, alphabet representation, alphabet minimization, and monitor encoding. We conduct extensive experimentation and identify a combination of settings that offers the best performance in terms of runtime overhead.

Tabakov, Deian↗
NASA NTRS2019

Software and System Health Management with R2U2

R2U2 (Realizable, Responsive, Unobtrusive Unit) is a hardware-supported tool and framework for the real-time system and software health management of cyber-physical systems. R2U2 continuously monitors properties about safety, performance, and security of the vehicle and can perform diagnostic reasoning. Efficient observers for past-time and future-time Metric Temporal Logic, reasoners for Bayesian Networks, and model-based prognostics algorithms are major components of R2U2. Their combination makes it possible to design powerful models for system runtime monitoring, diagnostics, software health management, prognostics, and security monitoring. The R2U2 monitoring engine is designed for minimal runtime overhead and is available as Simulink block or as a software component for integration into the flight software stack, and enables R2U2 to monitor complex cyber-physical systems without any instrumentation of the flight software. In this presentation, we give an overview of R2U2 architecture and reasoning algorithms, present its features, and give a life demo of the tool.

Schumann, Johann↗
NASA NTRS2017

Traveler Phase 1A Joint Review

The briefing contains the preliminary findings and suggestions for improvement of methods used in development and evaluation of a multi monitor runtime assurance architecture for autonomous flight vehicles. Initial system design, implementation, verification, and flight testing has been conducted. As of yet detailed data review is incomplete, and flight testing has been limited to initial monitor force fights. Detailed monitor flight evaluations have yet to be performed.

autonomy↗
NASA NTRSDate not supplied

Trustworthy Autonomy for Gateway Vehicle System Manager

The Vehicle System Manager (VSM) is the highest-level software control system in the Gateway hierarchical Autonomous System Management Architecture. The VSM provides four function categories: Mission Management and Timeline Execution, Resource Management, Fault Management, Vehicle Control and Operation. VSM provides various levels of automation ranging from fully autonomous operations with no flight crew and minimal ground monitoring to advisory automation when Gateway is crewed and has full ground monitoring. Trustworthiness is achieved via verified specification, comprehensive development verification, and real-time verification using assume-guarantee contracts. Development verification includes semantic verification of the data model via peer review and testing and assume-guarantee contracts implemented using the PlusCal/TLA+ environment. VSM also uses runtime assume-guarantee contracts, implemented in R2U2 via a runtime monitor that feeds the necessary telemetry data to R2U2 and which receives and responds to the R2U2 verdict stream. The full lifecycle verification approach and use of assume-guarantee contracts provides increased trustworthiness to VSM. Preliminary results provide encouragement that VSM can be both autonomous and trustworthy.

Assume-guarantee contracts↗
NASA NTRS2013

Towards Real-time, On-board, Hardware-Supported Sensor and Software Health Management for Unmanned Aerial Systems

Unmanned aerial systems (UASs) can only be deployed if they can effectively complete their missions and respond to failures and uncertain environmental conditions while maintaining safety with respect to other aircraft as well as humans and property on the ground. In this paper, we design a real-time, on-board system health management (SHM) capability to continuously monitor sensors, software, and hardware components for detection and diagnosis of failures and violations of safety or performance rules during the flight of a UAS. Our approach to SHM is three-pronged, providing: (1) real-time monitoring of sensor and/or software signals; (2) signal analysis, preprocessing, and advanced on the- fly temporal and Bayesian probabilistic fault diagnosis; (3) an unobtrusive, lightweight, read-only, low-power realization using Field Programmable Gate Arrays (FPGAs) that avoids overburdening limited computing resources or costly re-certification of flight software due to instrumentation. Our implementation provides a novel approach of combining modular building blocks, integrating responsive runtime monitoring of temporal logic system safety requirements with model-based diagnosis and Bayesian network-based probabilistic analysis. We demonstrate this approach using actual data from the NASA Swift UAS, an experimental all-electric aircraft.

System & Software Health Management↗