SEARCH · Search NASA

Results for “Runtime monitoring”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Embedding Differential Dynamic Logic in PVS

Runtime assurance is a control framework where a complex controller operates under the observation of a monitor. If the monitor detects the controller exhibiting undesirable behavior, control is passed off to a trusted controller until a desirable state is regained. The runtime assurance architecture provides a layer of assurance to the system being controlled, but special care must be taken that the resulting overall system, consisting of the monitors and controllers, is behaving as intended. This talk aims to formally model and reason about runtime assurance-equipped systems as hybrid programs- which are models that consist of both discrete and continuous components. Using the verification tool Plaidypvs, safety properties of some examples involving RTA architectures is shown.

Formal Verification↗

NASA NTRS2017

Runtime Verification Logics A Language Design Perspective

Runtime Verification is a light-weight approach to systems verification, where actual executions of a system are processed and analyzed using rigorous techniques. In this paper we shall narrow the term’s definition to represent the commonly studied variant consisting of verifying that a single system execution conforms to a specification written in a formal specification language. Runtime verification (in this sense) can be used for writing test oracles during testing when the system is too complex for full formal verification, or it can be used during deployment of the system as part of a fault protection strategy, where corrective actions may be taken in case the specification is violated. Specification languages for runtime verification appear to differ from for example temporal logics applied in model checking, in part due to the focus on monitoring of events that carry data, and specifically due to the desire to relate data values existing at different time points, resulting in new challenges in both the complexity of the monitoring approach and the expressiveness of languages. Over the recent years, numerous runtime verification specification languages have emerged, each with its different features and levels of expressiveness and usability. This paper presents an overview and a discussion of this design space.

Reger, Giles↗

DOE OSTI2019

Simulation Planning Using Component Based Cost Model

Successful simulations for scientific discovery on high-performance computing platforms require careful planning, including verification of specific application configuration and runtime parameters, estimation of resource requirements, and steering and monitoring of the simulation. However, simulation planning is an aspect of scientific computing that is extremely sparse in available literature or training. In this paper we focus on the resource management aspect of such planning through formulation of a component-based cost model. We illustrate the methodology and formulation through FLASH, a highly configurable simulation code used in multiple scientific domains.

Dubey, Anshu↗

DOE OSTI2022

Elastic Resource Management for Deep Learning Applications in a Container Cluster

The increasing demand for learning from massive datasets is restructuring our economy. Effective learning, however, involves nontrivial computing resources. Most businesses utilize commercial infrastructure providers (e.g., AWS) to host their computing clusters in the cloud, where various jobs compete for available resources. While cloud resource management is a fruitful research field that has made many advances in production, such as Kubernetes and YARN, few efforts have been invested to further optimize the system performance, especially for deep learning (DL) training jobs in a container cluster. This work introduces FlowCon, a system that is able to monitor the individual evaluation functions of DL jobs at runtime, and thus to make placement decisions on resource allocations elastically. Here, we present a detailed design and implementation of FlowCon and conduct intensive experiments over various DL models. The results demonstrate that FlowCon significantly improves DL job completion time and resource utilization efficiency, compared to default systems. According to the results, FlowCon is able to improve the completion time by up to 68.8% and meanwhile, reduce the makespan by 18.0%, in the presence of various DL job workloads.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2008

Runtime Verification of C Programs

We present in this paper a framework, RMOR, for monitoring the execution of C programs against state machines, expressed in a textual (nongraphical) format in files separate from the program. The state machine language has been inspired by a graphical state machine language RCAT recently developed at the Jet Propulsion Laboratory, as an alternative to using Linear Temporal Logic (LTL) for requirements capture. Transitions between states are labeled with abstract event names and Boolean expressions over such. The abstract events are connected to code fragments using an aspect-oriented pointcut language similar to ASPECTJ's or ASPECTC's pointcut language. The system is implemented in the C analysis and transformation package CIL, and is programmed in OCAML, the implementation language of CIL. The work is closely related to the notion of stateful aspects within aspect-oriented programming, where pointcut languages are extended with temporal assertions over the execution trace.

RMOR↗

NASA NTRS2003

The Mobile Agents Integrated Field Test: Mars Desert Research Station April 2003

The Mobile Agents model-based, distributed architecture, which integrates diverse components in a system for lunar and planetary surface operations, was extensively tested in a two-week field "technology retreat" at the Mars Society s Desert Research Station (MDRS) during April 2003. More than twenty scientists and engineers from three NASA centers and two universities refined and tested the system through a series of incremental scenarios. Agent software, implemented in runtime Brahms, processed GPS, health data, and voice commands-monitoring, controlling and logging science data throughout simulated EVAs with two geologists. Predefined EVA plans, modified on the fly by voice command, enabled the Mobile Agents system to provide navigation and timing advice. Communications were maintained over five wireless nodes distributed over hills and into canyons for 5 km; data, including photographs and status was transmitted automatically to the desktop at mission control in Houston. This paper describes the system configurations, communication protocols, scenarios, and test results.

Clancey, William J.↗

NASA NTRS2001

Java PathExplorer: A Runtime Verification Tool

We describe recent work on designing an environment called Java PathExplorer for monitoring the execution of Java programs. This environment facilitates the testing of execution traces against high level specifications, including temporal logic formulae. In addition, it contains algorithms for detecting classical error patterns in concurrent programs, such as deadlocks and data races. An initial prototype of the tool has been applied to the executive module of the planetary Rover K9, developed at NASA Ames. In this paper we describe the background and motivation for the development of this tool, including comments on how it relates to formal methods tools as well as to traditional testing, and we then present the tool itself.

Havelund, Klaus↗

NASA NTRS2003

Rule-Based Runtime Verification

We present a rule-based framework for defining and implementing finite trace monitoring logics, including future and past time temporal logic, extended regular expressions, real-time logics, interval logics, forms of quantified temporal logics, and so on. Our logic, EAGLE, is implemented as a Java library and involves novel techniques for rule definition, manipulation and execution. Monitoring is done on a state-by-state basis, without storing the execution trace.

Barringer, Howard↗

NASA NTRS2014

Monitoring with Data Automata

We present a form of automaton, referred to as data automata, suited for monitoring sequences of data-carrying events, for example emitted by an executing software system. This form of automata allows states to be parameterized with data, forming named records, which are stored in an efficiently indexed data structure, a form of database. This very explicit approach differs from other automaton-based monitoring approaches. Data automata are also characterized by allowing transition conditions to refer to other parameterized states, and by allowing transitions sequences. The presented automaton concept is inspired by rule-based systems, especially the Rete algorithm, which is one of the well-established algorithms for executing rule-based systems. We present an optimized external DSL for data automata, as well as a comparable unoptimized internal DSL (API) in the Scala programming language, in order to compare the two solutions. An evaluation compares these two solutions to several other monitoring systems.

log analysis↗

DOE OSTI2021

Achieving Runtime State Verification Assurance in Critical Cyber-Physical Infrastructures

Industrial Cyber-Physical Systems (ICPS) are an essential backbone of national critical infrastructures. They help monitor and control crucial cyber-enabled services such as energy generation. Commonly ICPS monitors the physical process through Supervisory Control and Data Acquisition (SCADA) systems. The SCADA ecosystem takes critical real-time and future system operational decisions based on the runtime state behavior of field sensors. Traditional SCADA systems use legacy and insecure communication protocols such as the Modbus protocol that lack adequate security mechanisms to provide robust runtime state behavior assurance of constrained field sensors. Therefore, constrained field sensors are commonly vulnerable to standard semantic attacks that gradually change the behavior state of infected devices. This paper discusses process integrity assurance techniques necessary to enhance the security of behavior-based protocols such as the Modbus protocol. The Runtime State Verification (RSV) protocol proposed in this paper aims to address semantic attacks in the SCADA ecosystem by integrating behavior-based Mandatory Results Automata (MRA) and a Hyperledger Fabric (HLF) network. The RSV protocol provides high process integrity assurance through enhanced behavior-based MRA suitable for the constrained field devices. A proof of concept of the RSV protocol has been evaluated in an emulated water-tube boiler. Preliminary evaluations of the RSV protocol aimed to measure the efficiency of the proposed protocol by monitoring an Combustion Efficiency (CE) process necessary to preserve optimal combustion, thus minimizing costs and future maintenance of water-tube boilers. We analyze the overall network overhead and latency of the proposed RSV protocol by evaluating the HLF network performance and comparing the proposed RSV protocol with the state-ofart BloSPAI protocol. Through the preliminary evaluations of the proposed RSV protocol, this paper demonstrates that the proposed RSV protocol overcomes the shortcomings and network overhead of the BloSPAI protocol by integrating behavior-based authentication through novel MRAs and HLF networks.

Rivera, Abel Gomez↗

DOE OSTI2022

Barriers to Broader Utilization of Fault Detection Technologies for Improving Residential HVAC Equipment Efficiency

Faults in residential heating, ventilating, and air conditioning (HVAC) equipment may occur due to poor installation practices or develop over time, and these faults can negatively impact system efficiency, thermal comfort, and equipment lifespan. Automated fault detection and diagnostic (AFDD) technologies identify energy wasting HVAC faults, such as low indoor airflow and improper refrigerant charge, and guide technicians in improving system efficiency. For residential HVAC, AFDD consists of a range of fault detecting and diagnostic capabilities, sensor configurations, and target applications. AFDD technology can either be permanently installed by the original equipment manufacturer (OEM) using embedded sensors or as an add-on product either during or after installation. Additionally, several advanced installation tools and refrigerant gauge sets include AFDD features for temporary use during equipment installation and tune-ups. Some technologies can detect a fault but have limited diagnostic capabilities. For example, a single-point measurement from the home's thermostat or energy monitor can provide certain fault detection capability by analyzing the equipment runtime or energy consumption. These technologies, though limited at determining the cause of a given fault, may have significant energy savings potential due to their low cost and prevalence in the residential HVAC market. Despite the potential benefits, fault detection technologies face many technical and market barriers preventing broad adoption. Beyond the cost barriers due to the added sensor requirements and technology development, fault detection technologies face many implementation and adoption barriers such as installer training, customer awareness, standardized communication protocols, and methods of test for evaluating accuracy. The purpose of this whitepaper is to characterize market and technical barriers impeding broader utilization of fault detection technology for residential HVAC energy efficiency applications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS2002

An Overview of the Runtime Verification Tool Java PathExplorer

We present an overview of the Java PathExplorer runtime verification tool, in short referred to as JPAX. JPAX can monitor the execution of a Java program and check that it conforms with a set of user provided properties formulated in temporal logic. JPAX can in addition analyze the program for concurrency errors such as deadlocks and data races. The concurrency analysis requires no user provided specification. The tool facilitates automated instrumentation of a program's bytecode, which when executed will emit an event stream, the execution trace, to an observer. The observer dispatches the incoming event stream to a set of observer processes, each performing a specialized analysis, such as the temporal logic verification, the deadlock analysis and the data race analysis. Temporal logic specifications can be formulated by the user in the Maude rewriting logic, where Maude is a high-speed rewriting system for equational logic, but here extended with executable temporal logic. The Maude rewriting engine is then activated as an event driven monitoring process. Alternatively, temporal specifications can be translated into efficient automata, which check the event stream. JPAX can be used during program testing to gain increased information about program executions, and can potentially furthermore be applied during operation to survey safety critical systems.

Havelund, Klaus↗

NASA NTRS2021

Concurrent Runtime Verification of Data Rich Events

This paper presents the open source runtime verification tool MESA (MEssage-based System Analysis), implemented in Scala, which supports concurrent monitors using the Actor model. Furthermore, the tool supports indexing (slicing) on the data values occurring in data-carrying events, for each individual monitor. The tool is generic in the sense that any monitoring system can be used for creating monitors. In this paper, we use the internal Scala DSL Daut for programming such in data parameterized state machines and temporal logic. To illustrate MESA/Daut, we present a case study that monitors flights from live U.S. airspace data streams, verifying that they conform to planned routes. With base in the case study, we then perform an extensive empirical study of the potential benefits from monitoring slices of a single property in concurrently executing actors. Due to the overhead of scheduling “small” actors (one for each slice or a small number of slices), it is not obvious that concurrent execution of such is beneficial. However, as a main result, we demonstrate that concurrent monitoring of slices to handle data-carrying events can provide considerable speed gains.

finite state machines↗

DOE OSTI2017

Traleika Glacier: A hardware-software co-designed approach to exascale computing

The move from current petascale machines to future exascale machines will need both hardware improvements and software changes. Hardware will need to evolve to focus primarily on features that lower energy consumption: near-threshold voltage operation, fine-grained power and clock management and heterogeneity. Software will also need to evolve and be able to express more parallelism, become more dynamic and adaptable in order to be able to operate on a much more variable hardware. Here, we present Traleika Glacier, an effort that seeks to evaluate radical design changes to meet the constraints, both in terms of power and cost, of exascale computing. The salient features of the hardware design presented in the work include a) a use of heterogeneous cores, b) a redesign of the memory system that centers around hierarchical scratchpads and a global address space, c) the hardware acceleration of certain memory and network operations through specialized engines and, d) very fine-grained control and monitoring capabilities. On the software side, we describe a task-based runtime system, the Open Community Runtime (OCR) which aims to express a wide range of higher-level programming models with a very limited set of core concepts: event-driven tasks for computation, events for synchronization and relocatable data-blocks for data management.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2011

Runtime Verification with State Estimation

We introduce the concept of Runtime Verification with State Estimation and show how this concept can be applied to estimate theprobability that a temporal property is satisfied by a run of a program when monitoring overhead is reduced by sampling. In such situations, there may be gaps in the observed program executions, thus making accurate estimation challenging. To deal with the effects of sampling on runtime verification, we view event sequences as observation sequences of a Hidden Markov Model (HMM), use an HMM model of the monitored program to "fill in" sampling-induced gaps in observation sequences, and extend the classic forward algorithm for HMM state estimation (which determines the probability of a state sequence, given an observation sequence) to compute the probability that the property is satisfied by an execution of the program. To validate our approach, we present a case study based on the mission software for a Mars rover. The results of our case study demonstrate high prediction accuracy for the probabilities computed by our algorithm. They also show that our technique is much more accurate than simply evaluating the temporal property on the given observation sequences, ignoring the gaps.

sampling↗

DOE OSTI2016

Reducing Waste in Extreme Scale Systems through Introspective Analysis

Resilience is an important challenge for extreme- scale supercomputers. Today, failures in supercomputers are assumed to be uniformly distributed in time. However, recent studies show that failures in high-performance computing systems are partially correlated in time, generating periods of higher failure density. Our study of the failure logs of multiple supercomputers show that periods of higher failure density occur with up to three times more than the average. We design a monitoring system that listens to hardware events and forwards important events to the runtime to detect those regime changes. We implement a runtime capable of receiving notifications and adapt dynamically. In addition, we build an analytical model to predict the gains that such dynamic approach could achieve. We demonstrate that in some systems, our approach can reduce the wasted time by over 30%.

Bautista-Gomez, Leonardo↗

NASA NTRSDate not supplied

Satellite-assisted Evaluation of Irrigation Application Efficiency for Lettuce in the Salinas Valley

The Salinas Valley region of California’s Central Coast is a majorU.S. vegetable producer. Lettuce is a leading commodity, grown on about 100,000 acres with annual value near $1.4B (2022 Monterey County Ag Report). Vegetables crops in the area aretypically well watered and fertilized in pursuit of commercial yield and quality standards. Information on irrigation application efficiency, meanwhile, can enhance waterresource sustainability and mitigate impacts of nitrate infiltrationon groundwater quality. The Crop Consumptive Use Fraction (CCUF), previously recommended by the California Department of Water Resources, is a metric that expresses evapotranspiration (ET) of applied water relative to total applied water. More recently under Assembly Bill 1668 addressing water conservation and drought planning, the metric can be used to help quantify application efficiency to support preparation of agricultural water management plans in the state. This study will report CCUF evaluations from several commercial lettuce plantings in the Salinas Valley during the 2022 dry season (late spring-early fall). The evaluations were derived from satellite-based observations of cropET from the OpenETcloud application, and applied water monitored by in-line flowmetersand supplemented by grower reports of irrigation system runtime. The study will also compare observed (satellite-based) ET with modeled ET from the CropManage irrigationand nutrient management decision-support system operated by Cooperative Extension. Additional study in cool-season vegetables is ongoing in 2023.

Satellite-assisted↗

NASA NTRSDate not supplied

MESA: Scalable Runtime Verification Tool Using Actors

This work presents our runtime verification approach implemented by the tool MESA (MEssage-based System Analysis) which allows for using concurrent monitors to check for properties specified in linear temporal logic and finite state machines.We employ the actor programing model to implement MESA where monitors are captured by concurrent actors that communicate via messaging. The paper also presents a case study where MESA is used to monitor flights in National Airspace System of United States using live air traffic data stream. The case study which motivated this work in the first place shows that our approach is effective.We also perform empirical study by conducting experiments using monitoring systems with different numbers of concurrent monitors and different layers of indexing.This paper describes our experiments, evaluates our results,and discusses challenges faced during the study. The evaluation shows our approach is scalable.

runtime verification, concurrency, actor programin↗