SEARCH · Search NASA

Results for “Runtime Verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

102 records · Page 6

All sources NASA NTRS DOE OSTI

DOE OSTI2026

Benchmark for two-dimensional large scale coherent structures in partially magnetized E × B plasmas—community collaboration & lessons learned

Low-temperature plasmas (LTPs) are essential to both fundamental scientific research and critical industrial applications. As in many areas of science, numerical simulations have become a vital tool for uncovering new physical phenomena and guiding technological development. Code benchmarking remains crucial for verifying implementations and evaluating performance. This work continues the Landmark benchmark initiative, a series specifically designed to support the verification of LTP codes. In this study, seventeen simulation codes from a collaborative community of nineteen international institutions modeled a partially magnetized E × B Penning discharge. The emergence of large scale coherent structures, or rotating plasma spokes, endows this configuration with an enormous range of time scales, making it particularly challenging to simulate. The codes showed excellent agreement on the rotation frequency of the spoke as well as key plasma properties, including time-averaged ion density, plasma potential, and electron temperature profiles. Achieving this level of agreement came with challenges, and we share lessons learned on how to conduct future benchmarking campaigns. Comparing code implementations, computational hardware, and simulation runtimes also revealed interesting trends, which are summarized with the aim of guiding future plasma simulation software development.

benchmarking↗

DOE OSTI2025

STARTR: An Open-Source MARVEL model for the NRIC Virtual Test Bed [Poster]

The National Reactor Innovation Center (NRIC) seeks to improve the understanding of microreactor physics in industry and academia through the development of a Microreactor Applications Research Validation and Evaluation (MARVEL) reactor-based model, published on the Virtual Test Bed (VTB). To achieve this goal, the Sodium-cooled Thermal-spectrum Advanced Research Test Reactor (STARTR) model was built using publicly available MARVEL specifications where possible and approximations where applicable, and was optimized for fast runtimes for researchers to receive rapid simulation feedback. STARTR will fill a gap between stakeholder interest and available models, as the first Sodium-cooled Thermal Reactor (STR) hosted on the VTB with baseline performance sanctioned by INL. This project involved the definition of all materials used in the reactor, geometry and all reactor subcomponents, and assertion of tallies and simulation settings within OpenMC 0.13.3. This poster details a small subset of the overall reactor physics testing: the two-dimensional power peaking factors and the flux energy spectrum, as well as plots of the created geometry. Future work includes code-to-code verification between the OpenMC-based model and a separately designed MCNP 6.2-based model.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗

DOE OSTI2018

ASC CSSE Level 2 Milestone #6362: Resilient Asynchronous Many Task Programming Model

This report is an outcome of the ASC CSSE Level 2 Milestone 6362: Analysis of Re- silient Asynchronous Many-Task (AMT) Programming Model. It comprises a summary and in-depth analysis of resilience schemes adapted to the AMT programming model. Herein, performance trade-offs of a resilient-AMT prograrnming model are assessed through two ap- proaches: (1) an analytical model realized by discrete event simulations and (2) empirical evaluation of benchmark programs representing regular and irregular workloads of explicit partial differential equation solvers. As part of this effort, an AMT execution simulator and a prototype resilient-AMT programming framework have been developed. The former permits us to hypothesize the performance behavior of a resilient-AMT model, and has undergone a verification and validation (V&V) process. The latter allows empirical evaluation of the perfor- mance of resilience schemes under emulated program failures and enabled the aforementioned V&V process. The outcome indicates that (1) resilience techniques implemented within an AMT framework allow efficient and scalable recovery under frequent failures, that (2) the abstraction of task and data instances in the AMT programming model enables readily us- able Application Program Interfaces (APIs) for resilience, and that (3) this abstraction enables predicting the performance of resilient-AMT applications with a simple simulation infrastruc- ture. This outcome will provide guidance for the design of the AMT programming model and runtime systems, user-level resilience support, and application development for ASC's next generation platforms (NGPs).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

A Visual Analytics Framework for the Detection of Anomalous Call Stack Trees in High Performance Computing Applications

Anomalous runtime behavior detection is one of the most important tasks for performance diagnosis in High Performance Computing (HPC). Most of the existing methods find anomalous executions based on the properties of individual functions, such as execution time. However, it is insufficient to identify abnormal behavior without taking into account the context of the executions, such as the invocations of children functions and the communications with other HPC nodes. We improve upon the existing anomaly detection approaches by utilizing the call stack structures of the executions, which record rich temporal and contextual information. With our call stack tree (CSTree) representation of the executions, we formulate the anomaly detection problem as finding anomalous tree structures in a call stack forest. The CSTrees are converted to vector representations using our proposed stack2vec embedding. Structural and temporal visualizations of CSTrees are provided to support users in the identification and verification of the anomalies during an active anomaly detection process. Furthermore, three case studies of real-world HPC applications demonstrate the capabilities of our approach.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2018

Dynamic Power Convertor Development for Radioisotope Power Systems at NASA GRC

The Thermal Energy Conversion Branch at NASA Glenn Research Center (GRC) is supporting the development of high-efficiency power convertors in support of Radioisotope Power System (RPS) development. Significant progress was made towards such a system that utilized Stirling conversion machines during the 2001 to 2015 timeframe. Flight development of the Advanced Stirling Radioisotope Generator (ASRG) was cancelled in 2013 by the Department of Energy (DOE) and NASA Headquarters primarily due to budget constraints, and the Advanced Stirling Convertor (ASC) technology contract was subsequently concluded in 2015. A new chapter of technology development has recently been initiated by the NASA RPS Program. This new effort is considering all dynamic power convertor options, such as Stirling and Brayton cycles. Four convertor development contracts supporting this effort were awarded in 2017. The awarded contracts include two free-piston Stirling, one thermoacoustic Stirling, and one turbo-Brayton designs. The technology development contracts each consist of up to three phases: Design, Fabricate, and Test. As of May 2018, all contracts have completed the Design Phase, and each underwent a design review with an independent review board. Two of the contracts have been awarded the Phase 2 option for fabrication. Convertors manifesting from these development efforts will then undergo independent validation and verification at NASA facilities, which will consist of convertor performance and RPS viability demonstrations. Example tests include launch vibration simulation, performance mapping over the environmental temperature range, and static acceleration exposure. In parallel with this renewed development effort, NASA GRC is still demonstrating free-piston Stirling convertor technology using assets from previous projects. The Stirling Research Laboratory (SRL) is still operating 10 convertors from previous development projects. Four of which are flexure-bearing based, and the other six are gas-bearing based. One of the flexure-bearing convertors has accumulated over 110,000 hours of operation, and holds the current maintenance-free heat-engine run-time record. Another flexure-bearing convertor was recently manually shutdown after 105,620 hours of operation, then disassembled and inspected. This inspection produced a wealth of information about the effects of this amount of runtime on the technology's components. One of the engineering unit flexure-bearing convertors recently underwent launch simulation vibration test, a static acceleration exposure up to 20 g, and was then placed on extended operation. Amongst the gas-bearing convertors, the longest running unit has accumulated over 70,000 hours of operation. Four high-fidelity gas-bearing convertors from the ASRG project are still operating continuously, for which the longest runtime has reached 26,000 hours.

Oriti, Salvatore↗

NASA NTRS2018

Dynamic Power Convertor Development for Radioisotope Power Systems at NASA Glenn Research Center

The Thermal Energy Conversion Branch at NASA Glenn Research Center (GRC) is supporting the development of high-efficiency power convertors for use in Radioisotope Power Systems (RPS). Significant progress was made towards such a system that utilized Stirling conversion during the 2001 to 2015 timeframe. Flight development of the Advanced Stirling Radioisotope Generator (ASRG) was cancelled in 2013 by the Department of Energy (DOE) and NASA Headquarters primarily due to budget constraints, and the Advanced Stirling Convertor (ASC) technology contract was subsequently concluded in 2015. A new chapter of technology development has recently been initiated by the NASA RPS Program. This effort is considering all dynamic power convertor options, such as Stirling and Brayton cycles. Four convertor development contracts supporting this effort were awarded in 2017. The awarded contracts include two free-piston Stirling, one thermoacoustic Stirling, and one turbo-Brayton designs. The technology development contracts each consist of up to three phases: Design, Fabricate, and Test. As of May 2018, all contracts have completed the Design Phase, and each underwent a design review with an independent review board. Three of the contracts are planned to execute the Phase 2 option for fabrication. Convertors manifesting from these development efforts will then undergo independent validation and verification at NASA facilities, which will consist of convertor performance and RPS viability demonstrations. Example tests include launch vibration simulation, performance mapping over the environmental temperature range, and static acceleration exposure. In parallel with this renewed development effort, NASA GRC is still demonstrating free-piston Stirling convertor technology using assets from previous projects. The Stirling Research Laboratory (SRL) is still operating several convertors from previous development projects which have similarities and relevance to current contract designs. Four of which are flexure-bearing based, and another six are gas-bearing based. One of the flexure-bearing convertors has accumulated over 110,000 hours of operation, and holds the current record for maintenance-free heat-engine run-time. Another flexure-bearing convertor was recently manually shutdown after 105,620 hours of operation, then disassembled and inspected. This inspection produced a wealth of information about the effects of this amount of runtime on the technology's components. One of the engineering unit flexure-bearing convertors recently underwent launch simulation vibration test, a static acceleration exposure up to 20 g, and was then placed on extended operation. Amongst the gas-bearing convertors, the longest running unit has accumulated over 70,000 hours of operation. Four high-fidelity gas-bearing convertors from the ASRG project are still operating continuously, for which the longest runtime has reached 28,000 hours.

Oriti, Salvatore M.↗

NASA NTRS2007

Model Checker for Java Programs

Java Pathfinder (JPF) is a verification and testing environment for Java that integrates model checking, program analysis, and testing. JPF consists of a custom-made Java Virtual Machine (JVM) that interprets bytecode, combined with a search interface to allow the complete behavior of a Java program to be analyzed, including interleavings of concurrent programs. JPF is implemented in Java, and its architecture is highly modular to support rapid prototyping of new features. JPF is an explicit-state model checker, because it enumerates all visited states and, therefore, suffers from the state-explosion problem inherent in analyzing large programs. It is suited to analyzing programs less than 10kLOC, but has been successfully applied to finding errors in concurrent programs up to 100kLOC. When an error is found, a trace from the initial state to the error is produced to guide the debugging. JPF works at the bytecode level, meaning that all of Java can be model-checked. By default, the software checks for all runtime errors (uncaught exceptions), assertions violations (supports Java s assert), and deadlocks. JPF uses garbage collection and symmetry reductions of the heap during model checking to reduce state-explosion, as well as dynamic partial order reductions to lower the number of interleavings analyzed. JPF is capable of symbolic execution of Java programs, including symbolic execution of complex data such as linked lists and trees. JPF is extensible as it allows for the creation of listeners that can subscribe to events during searches. The creation of dedicated code to be executed in place of regular classes is supported and allows users to easily handle native calls and to improve the efficiency of the analysis.

Visser, Willem↗

NASA NTRS2024

Discrete Event Simulation-Based Timeline Validation Using R2U2

The Gateway Vehicle Systems Manager (VSM), the top-level software control system in a distributed, hierarchical Autonomous System Management Architecture is, like most modern spacecraft software control systems, heavily data-driven. For example, schedules (timelines) will be developed on the ground and, due to the high degree of autonomy, contain complex procedures involving conditional branching, variable timing, and resource contention resolution. In order to verify that an uploaded timeline will function correctly, it is necessary to explore the feasible set of possible executions. While it is possible to test a timeline using a mission simulation, the complexity of the system and duration of a timeline limits the number of trials and therefore the test coverage. To address this problem, the VSM team is using a discrete event system model that can rapidly generate from a timeline sets of event sequences using Monte Carlo techniques. To achieve rapid and trustworthy checking of the event sequences, we use an offline version of the runtime model checking tool R2U2. This presentation describes the approach the VSM team is using to implement the discrete event simulation and evaluate event sequences using R2U2. The presentation will discuss: 1. Description of the timelines by VSM in the context of VSM operations 2. Expansion of a timeline into a sequence of atomic events 3. Adjustment, in the Monte Carlo environment, of an event sequence to account for uncertainty, external events, and failures 4. Definition of R2U2 input and mission-time linear temporal logic files 5. Generation and use of R2U2 verdict sequences 6. Lessons learned and future work

Verification↗

NASA NTRSDate not supplied

Establishing the Assurance Efficacy of Automated Risk Mitigation Strategies

Verification and validation of increasingly autonomous aviation systems is a major challenge. Traditional techniques for the assurance of high-confidence, safety-critical systems are not equipped to handle the complexity, uncertainty, and lack of predictability inherent in non-deterministic systems. Techniques such as run time monitoring, formal methods, and testing and simulation have been applied to some effect, but it is difficult to properly assess the success of such measures. The authors propose the concept of Assurance Efficacy to address this gap. Assurance Efficacy is seen as a parameter, criteria, or perspective by which to evaluate, identify and explore safety risk mitigation strategies and operational assurance architectures. Validation of the utility of this concept through flight testing is a first step in determining its potential role in assessing the overall safety of complex, increasingly autonomous systems that cannot be fully assured in the design phase.

system safety↗

NASA NTRS2003

Experimental Evaluation of Verification and Validation Tools on Martian Rover Software

To achieve its science objectives in deep space exploration, NASA has a need for science platform vehicles to autonomously make control decisions in a time frame that excludes intervention from Earth-based controllers. Round-trip light-time is one significant factor motivating autonomy capability, another factor is the need to reduce ground support operations cost. An unsolved problem potentially impeding the adoption of autonomy capability is the verification and validation of such software systems, which exhibit far more behaviors (and hence distinct execution paths in the software) than is typical in current deepspace platforms. Hence the need for a study to benchmark advanced Verification and Validation (V&V) tools on representative autonomy software. The objective of the study was to access the maturity of different technologies, to provide data indicative of potential synergies between them, and to identify gaps in the technologies with respect to the challenge of autonomy V&V. The study consisted of two parts: first, a set of relatively independent case studies of different tools on the same autonomy code, second a carefully controlled experiment with human participants on a subset of these technologies. This paper describes the second part of the study. Overall, nearly four hundred hours of data on human use of three different advanced V&V tools were accumulated, with a control group that used conventional testing methods. The experiment simulated four independent V&V teams debugging three successive versions of an executive controller for a Martian Rover. Defects were carefully seeded into the three versions based on a profile of defects from CVS logs that occurred in the actual development of the executive controller. The rest of the document is structured a s follows. In section 2 and 3, we respectively describe the tools used in the study and the rover software that was analyzed. In section 4 the methodology for the experiment is described; this includes the code preparation, seeding of defects, participant training and experimental setup. Next we give a qualitative overview of how the experiment went from the point of view of each technology; model checking (section 5), static analysis (section 6), runtime analysis (section 7) and testing (section 8). The find section gives some preliminary quantitative results on how the tools compared.

Brat, Guillaume↗

NASA NTRS2013

Maintaining the Health of Software Monitors

Software health management (SWHM) techniques complement the rigorous verification and validation processes that are applied to safety-critical systems prior to their deployment. These techniques are used to monitor deployed software in its execution environment, serving as the last line of defense against the effects of a critical fault. SWHM monitors use information from the specification and implementation of the monitored software to detect violations, predict possible failures, and help the system recover from faults. Changes to the monitored software, such as adding new functionality or fixing defects, therefore, have the potential to impact the correctness of both the monitored software and the SWHM monitor. In this work, we describe how the results of a software change impact analysis technique, Directed Incremental Symbolic Execution (DiSE), can be applied to monitored software to identify the potential impact of the changes on the SWHM monitor software. The results of DiSE can then be used by other analysis techniques, e.g., testing, debugging, to help preserve and improve the integrity of the SWHM monitor as the monitored software evolves.

Runtime Monitor↗

DOE OSTI2022

Performance of the Initial Implementation of the Shift Monte Carlo Code in SCALE 6.3

The Shift Monte Carlo code will be introduced in SCALE 6.3 as an alternative to the KENO V.a, KENO-VI, and Monaco codes. Calculations were performed to establish the performance of Shift for criticality safety analyses within the criticality safety analyses sequence (CSAS) based on models in the Verified, Archived Library of Inputs and Data (VALID). This test suite contains over 600 critical experiment models covering a broad range of fissile materials and neutron energy spectra. The comparisons presented include calculated k eff values and runtime performance for serial calculations and a selection of parallel calculations. Comparisons are presented for multigroup (MG) and continuous-energy (CE) calculations for KENO V.a and KENO-VI models. Results generated with a beta version of SCALE 6.3 indicate excellent agreement in k eff values between KENO and Shift. The largest differences in the average k eff value calculated for the 15 categories of KENO V.a models are 0.00020 ± 0.00011 Δ k for MG calculations and 0.00011 ± 0.00005 Δ k for CE calculations. Similar comparisons in three categories using KENO-VI result in the largest differences for MG calculations: as 0.00004 ± 0.00003 Δ k , and -0.00002 ± 0.00003 Δ k for CE. The preliminary results also indicate that Shift is faster than KENO on a per particle basis, especially for fast spectrum systems. The uncertainty per history is also higher, however, so the Monte Carlo figure of merit is higher for KENO for thermal and intermediate spectrum systems. As expected, Shift generally has better speedup than KENO for parallel calculations, regardless of neutron energy spectrum.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗