Search NASA⌕ Search

SEARCH · Search NASA

Results for “runtime monitoring”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Swarm Mentality: Toward Automatic Swarm State Awareness with Runtime Verification

Cyber-Physical Systems (CPSs) already exhibit impressive performance in all areas of human life, and swarms of CPSs promise to increase their capabilities even further. However, to effectively utilize CPS swarms their complexity of operation has to scale sub-linearly with the number of swarm members. Presenting the swarm to an operator as a single entity almost eliminates the additional per-member overhead entirely. To operate a swarm as one entity, and/or to increase the swarm’s autonomy, the operator and the swarm members need to reason and communicate at the same level of abstraction, i.e. the swarm needs a sense of “self.” Therefore, we require the ability to specify whole swarm properties yet monitor them at the member level. We examine one architecture for achieving this awareness by: 1) Defining a taxonomy for comparing techniques that synthesize this belief-state 2) Propose use of the Runtime Verification formal method to fill this role 3) Present preliminary designs for extending and embedding such a system in the Distributed Spacecraft Autonomy architecture to generate per-member monitors from swarm level specification.

Runtime Verification↗
NASA NTRS2015

A Flexible and Non-instrusive Approach for Computing Complex Structural Coverage Metrics

Software analysis tools and techniques often leverage structural code coverage information to reason about the dynamic behavior of software. Existing techniques instrument the code with the required structural obligations and then monitor the execution of the compiled code to report coverage. Instrumentation based approaches often incur considerable runtime overhead for complex structural coverage metrics such as Modified Condition/Decision (MC/DC). Code instrumentation, in general, has to be approached with great care to ensure it does not modify the behavior of the original code. Furthermore, instrumented code cannot be used in conjunction with other analyses that reason about the structure and semantics of the code under test. In this work, we introduce a non-intrusive preprocessing approach for computing structural coverage information. It uses a static partial evaluation of the decisions in the source code and a source-to-bytecode mapping to generate the information necessary to efficiently track structural coverage metrics during execution. Our technique is flexible; the results of the preprocessing can be used by a variety of coverage-driven software analysis tasks, including automated analyses that are not possible for instrumented code. Experimental results in the context of symbolic execution show the efficiency and flexibility of our nonintrusive approach for computing code coverage information

DO-178C↗
NASA NTRS2018

Runtime Verification: From Propositional to First-Order Temporal Logic

Runtime Verification is a branch of formal methods concerned with analysis of execution traces for the purpose of determining the state or general quality of the executing system. The field covers numerous approaches, one of which is specification-based runtime verification, where execution traces are checked against formal specifications. The paper presents syntax, semantics, and monitoring algorithms for respectively propositional and first-order temporal logics. In propositional logics the observed events in the execution trace are represented using atomic propositions, while first-order logic allows universal and existential quantification over data occurring as arguments in events. Monitoring of the first-order case is drastically more challenging than the propositional case, and we present a solution for this problem based on BDDs. We furthermore discuss monitorability of temporal properties by dividing them into different classes representing different degrees of monitorability.

Peled, Doron↗
DOE OSTI2025

Benchmark Tracking System for Performance Monitoring

Benchmarking is essential for high-performance software development, particularly for monitoring performance across code iterations. This project focused on enhancing the benchmarking process for Lamellar, an asynchronous runtime for High-Performance Computing (HPC) systems developed at Pacific Northwest National Laboratory. Prior to this work, benchmark results were difficult to track and compare across code versions, presenting significant challenges in identifying performance regressions and long-term trends. The primary objective was to establish a systematic, reproducible approach for measuring performance and detecting regressions following code commits. Our methodology involved three key components: standardizing benchmark outputs, implementing data versioning, and developing analysis tools. We standardized the benchmark output format to JSON Line records containing specific fields (execution time, hardware specifications, and environmental variables). To address data management challenges, we evaluated several options and eventually chose a git repository dedicated to benchmark data. We developed a suite of Python tools that processed benchmark results, enriched them with metadata, and facilitated search in the repository. The resulting system enables more efficient filtering and comparison of performance metrics across commit histories, hardware configurations, and benchmark variants through a unified query interface. Our implementation reduces computational overhead by first checking for existing results through configuration matching before initiating new benchmark runs, thereby conserving resources. The system has been validated by Lamellar developers. It organizes results by benchmark type and build configurations for efficient retrieval. Future developments include a planned Large Language Model interface for predicting benchmark performance, incorporating the criterion package for statistical analysis, which will enable automated detection of statistically significant performance changes, and integration with continuous integration pipelines. Despite these enhancements being reserved for future work, this project has successfully provided the Lamellar development team with a framework for maintaining consistent performance standards and identifying optimization opportunities across workloads and hardware environments.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2020

Actor-based Runtime Verification with MESA

This work presents a runtime verification approach implemented in the tool MESA (MEssage-based System Analysis) which allows for using concurrent monitors to check for properties specified in data parameterized temporal logic and state machines. The tool is implemented as an internal Scala DSL. We employ the actor programming model to implement MESA where monitors are captured by concurrent actors that communicate via messaging. The paper presents a case study in which MESA is used to effectively monitor a large number of flights from live US airspace data streams. We also perform an empirical study by conducting experiments using monitoring systems with different num- bers of concurrent monitors and different layers of indexing on the data contained in events. The paper describes the experiments, evaluates the results, and discusses challenges faced during the study. The evaluation shows the value of combining concurrency with indexing to handle data rich events.

Mehlitz, Peter↗
NASA NTRS2018

BDDs on the Run

Runtime verification (RV) of first-order temporal logic must handle a potentially large amount of data, accumulated during the monitoring of an execution. The DEJAVU RV system represents data elements and relations using BDDs. This achieves a compact representation, which allows monitoring long executions. However, the potentially unbounded, and frequently very large amounts of data value scan,ultimately, limit the executions that can be monitored. We present an automatic method for “forgetting” data values when they no longer affect the RV verdict on an observed execution.We describe the algorithm and illustrate its operation through an example.

Peled, Doron↗
NASA NTRS2018

The DejaVu Runtime Verification Benchmark

In this paper we present a benchmark for evaluating runtime verification tools. It was originally created in order to compare the DEJAVU runtime verification tool1 with another similar tool. DEJAVU’s logic is first-order past time temporal logic. In order to monitor such properties efficiently, Binary Decision Diagrams (BDDs) [1] are used for representing the data observed in a trace. The details on the logic and its algorithm are described in e.g. [2, 3, 4]. The benchmark consists of six properties, formulated in English, and formalized in DEJAVU’s logic. For each property is provided (normally) three traces, of sizes varying from 10,000 events to one million events. Traces are represented in CSV format.

Ulus, Dogan↗
NASA NTRS2017

R2U2: Tool Overview

R2U2 (Realizable, Responsive, Unobtrusive Unit) is an extensible framework for runtime System HealthManagement (SHM) of cyber-physical systems. R2U2 can be run in hardware (e.g., FPGAs), or software; can monitorhardware, software, or a combination of the two; and can analyze a range of different types of system requirementsduring runtime. An R2U2 requirement is specified utilizing a hierarchical combination of building blocks: temporal formula runtime observers (in LTL or MTL), Bayesian networks, sensor filters, and Boolean testers. Importantly, the framework is extensible; it is designed to enable definitions of new building blocks in combination with the core structure. Originally deployed on Unmanned Aerial Systems (UAS), R2U2 is designed to run on a wide range of embedded platforms, from autonomous systems like rovers, satellites, and robots, to human-assistive ground systems and cockpits. R2U2 is named after the requirements it satisfies; while the exact requirements vary by platform and mission, the ability to formally reason about realizability, responsiveness, and unobtrusiveness is necessary for flight certifiability, safety-critical system assurance, and achievement of technology readiness levels for target systems. Realizability ensures that R2U2 is suficiently expressive to encapsulate meaningful runtime requirements while maintaining adaptability to run on different platforms, transition between different mission stages, and update quickly between missions. Responsiveness entails continuously monitoring the system under test, real-time reasoning, reporting intermediate status, and as-early-as-possible requirements evaluations. Unobtrusiveness ensures compliance with the crucial properties of the target architecture: functionality, certifiability, timing, tolerances, cost, or other constraints.

Rozier, Kristin Y.↗
DOE OSTI2025

A Brief Survey on High Performance Computing Systems Power Management

This paper provides a survey of software-based power management techniques in High Performance Computing (HPC) systems. Seven existing power management and monitoring tools and frameworks are discussed. These are: Variorum, dynamic energy-performance optimizer (DEPO), Powersched, Bull Dynamic Power Optimizer (BDPO), Energy Aware Runtime (EAR), Global Extensible Open Power Manager (GEOPM), and PoLiMEr. Each of these tools is evaluated based on hardware abstraction, optimization methods, usability, and experimental validation. This survey highlights the diversity of approaches in managing energy efficiency, from vendor-neutral APIs to algorithm-driven power capping, and dynamic frequency adjustments. Given that energy requirements for large computational systems is increasing quickly, the importance of integrating these tools into existing HPC environments and the need for further research in this rapidly evolving field is also discussed.

97 - MATHEMATICS AND COMPUTING↗
NASA NTRS2009

Analog Input Data Acquisition Software

DAQ Master Software allows users to easily set up a system to monitor up to five analog input channels and save the data after acquisition. This program was written in LabVIEW 8.0, and requires the LabVIEW runtime engine 8.0 to run the executable.

Arens, Ellen↗
NASA NTRS2017

Unobtrusive Software and System Health Management with R2U2 on a Parallel MIMD Coprocessor

Dynamic monitoring of software and system health of a complex cyber-physical system requires observers that continuously monitor variables of the embedded software in order to detect anomalies and reason about root causes. There exists a variety of techniques for code instrumentation, but instrumentation might change runtime behavior and could require costly software re-certification. In this paper, we present R2U2E, a novel realization of our real-time, Realizable, Responsive, and Unobtrusive Unit (R2U2). The R2U2E observers are executed in parallel on a dedicated 16-core EPIPHANY co-processor, thereby avoiding additional computational overhead to the system under observation. A DMA-based shared memory access architecture allows R2U2E to operate without any code instrumentation or program interference.

Schumann, Johann↗
NASA NTRS2017

MESA: Message-Based System Analysis Using Runtime Verification

In this paper, we present a novel approach and framework for run-time verication of large, safety critical messaging systems. This work was motivated by verifying the System Wide Information Management (SWIM) project of the Federal Aviation Administration (FAA). SWIM provides live air traffic, site and weather data streams for the whole National Airspace System (NAS), which can easily amount to several hundred messages per second. Such safety critical systems cannot be instrumented, therefore, verification and monitoring has to happen using a nonintrusive approach, by connecting to a variety of network interfaces. Due to a large number of potential properties to check, the verification framework needs to support efficient formulation of properties with a suitable Domain Specific Language (DSL). Our approach is to utilize a distributed system that is geared towards connectivity and scalability and interface it at the message queue level to a powerful verification engine. We implemented our approach in the tool called MESA: Message-Based System Analysis, which leverages the open source projects RACE (Runtime for Airspace Concept Evaluation) and TraceContract. RACE is a platform for instantiating and running highly concurrent and distributed systems and enables connectivity to SWIM and scalability. TraceContract is a runtime verication tool that allows for checking traces against properties specified in a powerful DSL. We applied our approach to verify a SWIM service against several requirements.We found errors such as duplicate and out-of-order messages.

Message-based System↗
NASA NTRS2005

BEAM Technology Flight Demonstration

As technologies advance, their growing complexity makes them harder to maintain. Detection methods for isolating and identifying impending problems are needed to balance this complexity. Through comparison of signal pairs from onboard sensors, the Beacon-based Exception Analysis For Multimissions (BEAM) algorithm can identify and help classify deviations in system operation from a data-trained statistical model. The goal of this task is to mature BEAM and validate its performance on a flying test bed. A series of F-18 flight demonstrations with BEAM monitoring engine parameters in real time was used to demonstrate in-the-field readiness. Captured F-18 and simulated F-18 engine data were used in model creation and training. The algorithm was then ported to the embedded system with a data buffering, file writing, and data-time-stamp monitoring shell to reduce the impact of embedded system faults on BEAM'S ability to correctly identify engine faults. Embedded system testing identified hardware related restrictions and contributed to iterative improvements in the code's runtime performance. The system was flown with forced engine flameouts and other pilot induced faults to simulate operation out of the norm. Successful detection of these faults, confirmed through post-flight data analysis, helped BEAM achieve TRL6.

Wang, David↗
NASA NTRS2004

ScyFlow: An Environment for the Visual Specification and Execution of Scientific Workflows

With the advent of grid technologies, scientists and engineers are building more and more complex applications to utilize distributed grid resources. The core grid services provide a path for accessing and utilizing these resources in a secure and seamless fashion. However what the scientists need is an environment that will allow them to specify their application runs at a high organizational level, and then support efficient execution across any given set or sets of resources. We have been designing and implementing ScyFlow, a dual-interface architecture (both GUT and APT) that addresses this problem. The scientist/user specifies the application tasks along with the necessary control and data flow, and monitors and manages the execution of the resulting workflow across the distributed resources. In this paper, we utilize two scenarios to provide the details of the two modules of the project, the visual editor and the runtime workflow engine.

McCann, Karen M.↗
NASA NTRS2024

Components Refurbishment and Chemical Analysis Facility, Hot Spot 1 Solid Waste Management Unit #041 Year 4 Annual Performance Monitoring Report Kennedy Space Center, Florida

This Year 4 Annual Performance Monitoring Report (PMR) presents the operations, maintenance, and monitoring activities for the Hydraulic Containment System (HCS) Interim Measure (IM) at the Components Refurbishment and Chemical Analysis (CRCA) facility located at John F. Kennedy Space Center (KSC), Florida. The primary objective of the HCS is to attain hydraulic control of the dissolved-phase chlorinated volatile organic compound (CVOC) plume, with the secondary objective to reduce concentrations of CVOCs in the high-concentration plume to support transition to monitored natural attenuation (MNA). CRCA has been designated Solid Waste Management Unit 041 under the KSC Resource Conservation and Recovery Act Corrective Action Program. The timeframe for activities documented in this Year 4 PMR extends from November 2022 through September 2023. Baseline sampling activities were completed in June 2019, and full-scale startup of the HCS IM was completed in July-August 2019. The operational runtime of the HCS for the Year 4 reporting period was approximately 94%, with the majority of downtime attributed to associated groundwater sampling events, maintenance, and Hurricane Nicole. Almost five million gallons of groundwater were treated during Year 4 of HCS operations, and concentrations of the site’s contaminants of concern (trans-1,2-dichloroethene and vinyl chloride) have been reduced by over 99%. This PMR describes the activities that were performed during Year 4 to operate and monitor the HCS IM, which includes three extraction wells, seven injection wells, and conveyance piping to a modular structure containing the control panel and an air stripper. Influent and effluent sampling results from the air stripper show that the system is operating as designed and is reducing concentrations of contaminants of concern to below detection limits. In addition to HCS operation, this PMR also discusses performance monitoring that has been implemented to assess progress of the HCS IM and overall plume conditions through scheduled groundwater (quarterly and semi-annual) and sub-slab soil gas (quarterly) sampling and analysis. Two ambient air samples were also collected on a quarterly basis in the vicinity of the modular structure and the paved driveway east of the Solvent Reclamation Area during routine operation and maintenance (O&M) activities to ensure safe breathing zone air quality for on-site personnel. All sub-slab soil gas and ambient air sampling conducted during the Year 4 operational period showed results below applicable regulatory air screening limits. Predictions made during the Year 2 groundwater model updates were in close correlation to post Year 4 plume conditions. A supplemental DPT study conducted in 2022 and 2023. This study indicated that low-concentration plume conditions, where concentrations exceed State of Florida Groundwater Cleanup Target Levels, expanded westward to Kennedy Parkway North and northward to the vicinity of the railroad tracks. Based on these results, recommendations were made to install 14 wells to monitor the downgradient and boundary conditions of the expanded LCP. The contents of this Year 4 PMR were presented during the November 2023 KSC Remediation Team meeting, where Team consensus was reached on several items including continued O&M of the HCS, and continued monitoring of groundwater, ambient air, and sub-slab soil gas. Sampling for per- and polyfluoroalkyl substances at CRCA is ongoing and will be submitted under separate cover.

K. Alex Murphy↗
NASA NTRS2023

Components Refurbishment and Chemical Analysis Facility, Hot Spot 1 SWMU #041 Year 3 Annual Performance Monitoring Report Kennedy Space Center, Florida

This Year 3 Annual Performance Monitoring Report (PMR) presents the operations, maintenance, and monitoring activities for the Hydraulic Containment System (HCS) Interim Measure (IM) at the Components Refurbishment and Chemical Analysis (CRCA) facility located at John F. Kennedy Space Center (KSC), Florida. The primary objective of the HCS is to attain hydraulic control of the dissolved-phase chlorinated volatile organic compound (CVOC) plume, with the secondary objective to reduce concentrations of CVOCs in the high-concentration plume to support transition to monitored natural attenuation (MNA). CRCA has been designated Solid Waste Management Unit 041 under the KSC Resource Conservation and Recovery Act Corrective Action Program. The timeframe for activities documented in this Year 3 PMR extends from September 2021 through October 2022. Baseline sampling activities were completed in June 2019, and full-scale startup of the HCS IM was completed in July-August 2019. The operational runtime of the HCS for the Year 3 reporting period was approximately 91%, with the majority of downtime attributed to system maintenance and repair. This is generally consistent with Year 1 and Year 2 runtimes of 85% and 92%, respectively. To help reduce downtime, an anti-scaling amendment, Redux 390, has been used to reduce scaling and help maintain system design parameters. Over nine million gallons of groundwater were treated during Year 3 of HCS operations, and concentrations of the site’s contaminants of concern (trans-1,2-dichloroethene and vinyl chloride) have been reduced by over 97%. This PMR describes the activities that were performed during Year 3 to operate and monitor the HCS IM, which includes three extraction wells, seven injection wells, and conveyance piping to a modular structure containing the control panel and an air stripper. Influent and effluent sampling results from the air stripper show that the system is operating as designed and is reducing concentrations of contaminants of concern to below detection limits. In addition to HCS operation, this PMR also discusses performance monitoring that has been implemented to assess progress of the HCS IM and overall plume conditions through scheduled groundwater (quarterly and semi-annual) and sub-slab soil gas (quarterly) sampling and analysis. Two ambient air samples were also collected on a quarterly basis in the vicinity of the modular structure and the paved driveway east of the Solvent Reclamation Area during routine operation and maintenance (O&M) activities to ensure safe breathing zone air quality for on-site personnel. All sub-slab soil gas and ambient air sampling conducted during the Year 3 operational period showed results below applicable regulatory air screening limits. Predictions made during the Year 2 groundwater model updates were in close correlation to post Year 3 plume conditions. Therefore, it can be assumed that the projected path remains valid for transition to MNA in one to two years of continuous HCS operation. The contents of this Year 3 PMR were presented during the February 2023 KSC Remediation Team meeting, where Team consensus was reached on several items including continued O&M of the HCS, and continued monitoring of groundwater, ambient air, and sub-slab soil gas. Replacement of MW0019 and VMP04 was also recommended, as well as additional direct-push technology sampling to further delineate the downgradient plume and confirm overall site-wide low-concentration plume boundaries. Sampling for per- and polyfluoroalkyl substances at CRCA is ongoing and will be submitted under separate cover.

K. Alex Murphy↗
NASA NTRS2008

Orchestrator Telemetry Processing Pipeline

Orchestrator is a software application infrastructure for telemetry monitoring, logging, processing, and distribution. The architecture has been applied to support operations of a variety of planetary rovers. Built in Java with the Eclipse Rich Client Platform, Orchestrator can run on most commonly used operating systems. The pipeline supports configurable parallel processing that can significantly reduce the time needed to process a large volume of data products. Processors in the pipeline implement a simple Java interface and declare their required input from upstream processors. Orchestrator is programmatically constructed by specifying a list of Java processor classes that are initiated at runtime to form the pipeline. Input dependencies are checked at runtime. Fault tolerance can be configured to attempt continuation of processing in the event of an error or failed input dependency if possible, or to abort further processing when an error is detected. This innovation also provides support for Java Message Service broadcasts of telemetry objects to clients and provides a file system and relational database logging of telemetry. Orchestrator supports remote monitoring and control of the pipeline using browser-based JMX controls and provides several integration paths for pre-compiled legacy data processors. At the time of this reporting, the Orchestrator architecture has been used by four NASA customers to build telemetry pipelines to support field operations. Example applications include high-volume stereo image capture and processing, simultaneous data monitoring and logging from multiple vehicles. Example telemetry processors used in field test operations support include vehicle position, attitude, articulation, GPS location, power, and stereo images.

Powell, Mark↗