Search NASA⌕ Search

SEARCH · Search NASA

Results for “test frameworks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS1997

Research in Satellite-Fiber Network Interoperability

This four part report evaluated the performance of high data rate transmission links using the ACTS satellite, and to provide a preparatory test framework for two of the space science applications that have been approved for tests and demonstrations as part of the overall ACTS program. The test plan will provide guidance and information necessary to find the optimal values of the transmission parameters and then apply these parameters to specific applications. The first part will focus on the satellite-to-earth link. The second part is a set of tests to study the performance of ATM on the ACTS channel. The third and fourth parts of the test plan will cover the space science applications, Global Climate Modeling and Keck Telescope Acquisition Modeling and Control.

Edelson, Burt↗
NASA NTRS2010

Architecture-Based Unit Testing of the Flight Software Product Line

This paper presents an analysis of the unit testing approach developed and used by the Core Flight Software (CFS) product line team at the NASA GSFC. The goal of the analysis is to understand, review, and reconunend strategies for improving the existing unit testing infrastructure as well as to capture lessons learned and best practices that can be used by other product line teams for their unit testing. The CFS unit testing framework is designed and implemented as a set of variation points, and thus testing support is built into the product line architecture. The analysis found that the CFS unit testing approach has many practical and good solutions that are worth considering when deciding how to design the testing architecture for a product line, which are documented in this paper along with some suggested innprovennents.

Ganesan, Dharmalingam↗
NASA NTRS2014

Test Driven Development of Scientific Models

Test-Driven Development (TDD), a software development process that promises many advantages for developer productivity and software reliability, has become widely accepted among professional software engineers. As the name suggests, TDD practitioners alternate between writing short automated tests and producing code that passes those tests. Although this overly simplified description will undoubtedly sound prohibitively burdensome to many uninitiated developers, the advent of powerful unit-testing frameworks greatly reduces the effort required to produce and routinely execute suites of tests. By testimony, many developers find TDD to be addicting after only a few days of exposure, and find it unthinkable to return to previous practices.After a brief overview of the TDD process and my experience in applying the methodology for development activities at Goddard, I will delve more deeply into some of the challenges that are posed by numerical and scientific software as well as tools and implementation approaches that should address those challenges.

Overview of TDD Process↗
DOE OSTI2022

Autonomous Closed-Loop Experimental Characterization and Dynamic Model Validation of a Scaled Underwater Kite

This paper presents the closed-loop experimental framework and dynamic model validation for a 1/12-scale underwater kite design. The pool-based tow testing framework described herein, which involves a fully actuated, closed-loop controlled kite and flexible tether, significantly expands upon the capabilities of any previously developed open-source framework for experimental underwater kite characterization. Specifically, the framework has allowed for the validation of three closed-loop flight control strategies, along with a critical comparison between dynamic model predictions and experimental results. In this paper, we provide a detailed presentation of the experimental tow system and kite setup, describe the control algorithms implemented and tested, and quantify the level of agreement between our multi-degree-of-freedom kite dynamic model and experimental data. Finally, we also present a sensitivity analysis that helps to identify the most influential parameters to kite performance and further explain the remaining mismatches between the model and data.

42 ENGINEERING↗
DOE OSTI2020

UnitTesterSG

UnitTesterSG is a unit testing framework that is designed for nested and/or scientific/engineering data structures. It is designed primarily for testing the outputs if a single function or simulation run by storing the expected results file such that comparisons to the stored output can be made with unit tests after the function or software has been edited. This module is designed to be able to unit test arrays, strings, as well as nested and deeply nested objects (including arrays with tuples inside, arrays with strings inside, etc.). The module is compatable with nested objects, and thus has a dependency on the nestedObjectsFunctions module within. The module takes *calculated* results and then compares them to *expected* results. If no expected results are available (or they do not match the calculated results) the module then offers to store the calculated results as expected results for next time.

Savara, Aditya [Oak Ridge National Lab. (ORNL), Oa↗
NASA NTRSDate not supplied

Adventures in cFS Unit Testing: Examining the Past to Explain the Present with an Eye toward the Future

An overview of my experiences writing unit tests for various projects with a specific focus on my work unit testing core Flight System (cFS) applications. I recount some of the direct personal experiences I have had that showed me the utility of having done unit testing for my projects. Many of the tips, tricks and pitfalls encountered during my time writing unit tests for the cFS app, CF, are examined. I also compare and contrast my cFS unit testing development with that of a parallel project, in which I write unit tests using RSpec, a testing framework for the Ruby programming language. I impart my complete methodology behind the CF app unit testing effort and the rationale for why I did it that way. Then I give some ideas for how you can do your own unit testing for cFS applications. You will also learn about my hopes for how unit testing cFS applications can be done going forward from where we are now.

"unit testing"↗
DOE OSTI2022

Development of an MC&A toolbox for liquid-fueled molten salt reactors with online reprocessing (Final Report)

A critical barrier to the deployment of MSRs is the absence of a well-defined nuclear material control and accounting (MC&A) approach, a vital prerequisite to meet NRC licensing requirements as well as facilitating future international exports. Liquid-fueled MSR variants (especially those incorporating online refueling or reprocessing) present a unique set of challenges to traditional nuclear material control and accountancy. Unlike solid-fueled light-water reactors, traditional item-counting methods cannot be applied as an accountancy strategy. Rather, MC&A approaches to MSR variants (including both uranium and thorium fueled designs) are more analogous to bulk material handling facilities (e.g., enrichment and reprocessing); yet further complicating matters is the fact that fuel medium is also highly radioactive. Meanwhile, the space of MSRs covers a broad range of design parameters, including thermal and fast spectra designs; operation in actinide breeder or burner modes, choice of actinide fuel used (e.g., 235 U, 232 Th / 233 U, denatured 233 U, etc.), pool or loop-type configuration, and even different salt chemistry. Each of these design choices introduce significant challenges to MC&A approaches within MSR facilities. We propose to bridge this gap for liquid-fueled MSRs by developing a modular, component-based test framework for evaluating viable process monitoring and MC&A techniques specifically suited to liquid-fueled MSR system variants employing online refueling or reprocessing. This test platform will consist of a toolbox of independent process modules representing discrete physical units (such as the reactor core, off-gas processing, decay tanks, and actinide separation units), each with its own self-contained physics responsive to the input mass flow, along with appropriate measurement models that can be coupled to key flow points. These dynamic physical signatures thus afford the ability to test the viability and efficacy of potential accountancy techniques under the full range of reactor operating conditions. As process modules are connected via mass flows, the result is a reconfigurable, generic MSR mass flow model capable of serving as an MC&A test platform for a broad spectrum of possible MSR configurations. The resulting MSR MC&A toolbox will enable robust assessment of accountancy strategies for this unique facility type, including analysis of physical feedbacks arising both from depletion of the fuel over time as well as from potential off-normal events, including those introduced by equipment failures (e.g., a pump failure) as well as by malicious action (i.e., attempts to divert material). The proposed toolbox both addresses a critical needs area for the MPACT analysis toolkit while leveraging existing MPACT-sponsored tools, especially with respect to simulation of measurement and accountancy techniques for advanced fuel cycle facilities. Beyond enabling new analysis capabilities for MSR systems, the design of this toolbox will be to such to maximize compatibility with existing MPACT tools, such to enhance existing facility MC&A analysis capabilities.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRSDate not supplied

Adaptive Stress Testing: Using Reinforcement Learning to Find Failures in Safety-Critical Systems

Emerging applications in artificial intelligence, such as driverless cars and autonomous aircraft promise to be more efficient, cheaper to operate, and always available. However, ensuring the safety of these systems remains a major challenge to their certification and adoption. These autonomous systems are expected to routinely make safety-critical decisions where failures can have serious consequences including loss of life and property. Testing and validation techniques aim to identify and diagnose potential failures before the system is deployed. However, finding failure scenarios in autonomous systems can be very challenging due to high-dimensional and continuous state spaces, interaction with large environments over many time steps, and the rarity of failures. This talk presents Adaptive Stress Testing (AST), a simulation-based testing framework for finding the most likely path to a failure event of a safety-critical system. The key idea of AST is that stress testing can be formulated as a Partially Observable Markov Decision Process (POMDP), which enables reinforcement learning techniques to be used for finding failure events. Reinforcement learning algorithms can efficiently explore the search space and have been shown to scale to very large systems. We present applications of AST to find failures in various safety-critical systems including the aircraft collision avoidance systems, autonomous cars, and small unmanned aerial vehicles.

autonomous vehicles↗
DOE OSTI2020

Gene expression of functionally-related genes coevolves across fungal species: detecting coevolution of gene expression using phylogenetic comparative methods

Researchers often measure changes in gene expression across conditions to better understand the shared functional roles and regulatory mechanisms of different genes. Analogous to this is comparing gene expression across species, which can improve our understanding of the evolutionary processes shaping the evolution of both individual genes and functional pathways. One area of interest is determining genes showing signals of coevolution, which can also indicate potential functional similarity, analogous to co-expression analysis often performed across conditions for a single species. However, as with any trait, comparing gene expression across species can be confounded by the non-independence of species due to shared ancestry, making standard hypothesis testing inappropriate. We compared RNA-Seq data across 18 fungal species using a multivariate Brownian Motion phylogenetic comparative method (PCM), which allowed us to quantify coevolution between protein pairs while directly accounting for the shared ancestry of the species. Our work indicates proteins which physically-interact show stronger signals of coevolution than randomly-generated pairs. Interactions with stronger empirical and computational evidence also showing stronger signals of coevolution. We examined the effects of number of protein interactions and gene expression levels on coevolution, finding both factors are overall poor predictors of the strength of coevolution between a protein pair. Simulations further demonstrate the potential issues of analyzing gene expression coevolution without accounting for shared ancestry in a standard hypothesis testing framework. Furthermore, our simulations indicate the use of a randomly-generated null distribution as a means of determining statistical significance for detecting coevolving genes with phylogenetically-uncorrected correlations, as has previously been done, is less accurate than PCMs, although is a significant improvement over standard hypothesis testing. These methods are further improved by using a phylogenetically-corrected correlation metric. Our work highlights potential benefits of using PCMs to detect gene expression coevolution from high-throughput omics scale data. This framework can be built upon to investigate other evolutionary hypotheses, such as changes in transcription regulatory mechanisms across species.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRSDate not supplied

AdaStress

This is a tutorial on AdaStress, a tool for finding and analyzing the likeliest failures in a simulated system under test. The presentation outlines the adaptive stress testing framework, provides a demonstration of use, and showcases several examples of failure detection in a complex real-world system.

Reinforcement learning↗
DOE OSTI2025

Portable Software Environment for Ultrahigh-Resolution ELM Development on GPUs

This paper presents our endeavors in developing the large-scale, ultra-high-resolution E3SM Land Model (uELM), specifically designed for exascale computers furnished with accelerators such as Nvidia GPUs. The uELM is a sophisticated code that substantially relies on High-Performance Computing (HPC) environments, necessitating particular machine and software configurations. To facilitate community-based uELM developments employing GPUs, we have created a portable, standalone software environment preconfigured with uELM input datasets, simulation cases, and source code. This environment, utilizing Docker, encompasses all essential code, libraries, and system software for uELM development on GPUs. It also features a functional unit test framework and an offline model testbed for comprehensive numerical experiments. From a technical perspective, the paper discusses GPU-ready container generations, uELM code management, and input data distribution across computational platforms. Lastly, the paper demonstrates the use of environment for functional unit testing, end-to-end simulation on CPUs and GPUs, and collaborative code development.

E3SM Land Model↗
NASA NTRS2021

On the Loading Rate Dependence of Environment-assisted Cracking in Sensitized AA5456-H116 Exposed to Marine Environments

The influence of the applied loading rate (dK/dt) on the environment-assisted cracking (EAC) behavior of sensitized AA5456-H116 in 0.6 M NaCl at applied potentials ranging from −800 to −900 mV_SCE is assessed via a rising-K testing framework. The applied potential strongly affects the dK/dt-dependence of EAC with results suggesting a minimal influence for potentials more positive than −830 mVSCE and a stronger dK/dt-dependence for potentials more negative than −830 mV_SCE. Crack growth rates measured using rising versus static K testing are compared, which demonstrates that rising K methods consistently yield conservative EAC metrics with increased efficiency.

Al-Mg↗
DOE OSTI2025

Dual‐Transformer Deep Learning Framework for Seasonal Forecasting of Great Lakes Water Levels

Abstract The Great Lakes of North America form one of the largest freshwater systems on Earth, and their lake‐wide average water levels (lake levels) can fluctuate by more than 0.5 m on a seasonal scale. These fluctuations pose substantial challenges for coastal resilience, flood risk management, and navigation planning. Accurate seasonal forecasting of lake levels using traditional mechanistic models is challenging due to the complex physical mechanisms and coupled hydroclimatic processes involved. Recently, deep learning has gained prominence in geoscience applications for its ability to recognize intricate patterns within multiphysical data sets. Here, we introduce a novel Dual‐Transformer deep learning framework, tested on the Great Lakes. This architecture integrates two modified Transformer models: the Prophet, which predicts underlying trends, and the Critic, which refines the Prophet's predictions. The final lake level prediction is derived by weighting the outputs of both models through a multi‐layer perceptron, jointly trained with the Prophet and Critic to enhance overall accuracy. Our results demonstrate that the innovative learning framework achieves the highest prediction accuracy compared to established deep learning models when using identical input features. It attains a root mean square error of 4–7 cm in predicting lake levels up to 6 months in advance across the lakes. Additionally, the Dual‐Transformer model runs six orders of magnitude faster than conventional mechanistic models, producing results in less than one second on a typical personal computer. These findings suggest that our deep learning framework has strong potential to advance lake level prediction and carries important implications for water management and disaster mitigation, thereby enhancing the quality of life in coastal regions.

Chen, Yi [Great Lakes Research Center Michigan Tec↗
DOE OSTI2021

Bioverse: A Simulation Framework to Assess the Statistical Power of Future Biosignature Surveys

Next-generation space observatories will conduct the first systematic surveys of terrestrial exoplanet atmospheres and search for evidence of life beyond Earth. While in-depth observations of the nearest habitable worlds may yield enticing results, there are fundamental questions about planetary habitability and evolution that can only be answered through population-level studies of dozens to hundreds of terrestrial planets. To determine the requirements for next-generation observatories to address these questions, we have developed Bioverse. Bioverse combines existing knowledge of exoplanet statistics with a survey simulation and hypothesis testing framework to determine whether proposed space-based direct imaging and transit-spectroscopy surveys will be capable of detecting various hypothetical statistical relationships between the properties of terrestrial exoplanets. Following a description of the code, we apply Bioverse to determine whether an ambitious direct imaging or transit survey would be able to determine the extent of the circumstellar habitable zone and study the evolution of Earth-like planets. Given recent evidence that Earth-sized habitable zone planets are likely much rarer than previously believed, we find that space missions with large search volumes will be necessary to study the population of terrestrial and habitable worlds. Moving forward, Bioverse provides a methodology for performing trade studies of future observatory concepts to maximize their ability to address population-level questions, including and beyond the specific examples explored here.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Julienne v1.0.0

Julienne is a compiler-portable unit-testing framework for Fortran software projects, including those that use the parallel/accelerator-programming features of Fortran 2023. Julienne achieves portability across compilers through minimalism and isolation. The minimal design ensures that Julienne uses only features supported by the majority of Fortran compilers. The isolation through zero dependencies ensures that no other projects block Julienne from building with a particular compiler. Julienne also contains with additional services that support its unit-testing code. These include functions for manipulating strings, command lines and input/output format strings; and a user-defined collective subroutine for verifying that all processes pass a test in parallel testing. Julienne's name derives from the term for vegetables sliced into thin strings: julienne vegetables. Julienne captures the authors' most frequently used thin slice of the Veggies and Sourcery software repositories while avoiding certain compiler limitations of the those two packages.

Rouson, Damian↗
DOE OSTI2025

Enhancing Autonomous Control of Microreactors Using Multi-Agent Reinforcement Learning

In order for microreactors to be economically competitive, operation costs will need to be minimized through some degree of autonomous control. Previous work has demonstrated the effectiveness of reinforcement learning (RL) for load-following control in a drum-controlled microreactor. This study extends that work by exploring the potential of RL to independently control each of the reactor’s drums. We compare a single-agent RL approach with a multi-agent RL (MARL) framework, testing them for generalization across different load-following power profiles and control timescales, and for robustness in cases of randomly disabled control drums. Since the point kinetics simulation environment used in this study cannot resolve spatial effects, we assume that in the absence of spatially localized disturbances, optimal drum movements should be symmetrical. We demonstrate that single-agent RL is able to achieve accurate performance only when symmetric actions are ignored; otherwise, it fails to train a useful controller. Meanwhile, the MARL framework performs symmetric actions by design and trains a robust, accurate agent, as evidenced by mean absolute errors in power matching of 0.41% for the training power profile, 0.68% for a profile with half the drums disabled, and 0.21% for a profile on a realistic load-following time horizon.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Performance portable ice-sheet modeling with MALI

High-resolution simulations of polar ice sheets play a crucial role in the ongoing effort to develop more accurate and reliable Earth system models for probabilistic sea-level projections. These simulations often require a massive amount of memory and computation from large supercomputing clusters to provide sufficient accuracy and resolution; therefore, it has become essential to ensure performance on these platforms. Many of today’s supercomputers contain a diverse set of computing architectures and require specific programming interfaces in order to obtain optimal efficiency. In an effort to avoid architecture-specific programming and maintain productivity across platforms, the ice-sheet modeling code known as MPAS-Albany Land Ice (MALI) uses high-level abstractions to integrate Trilinos libraries and the Kokkos programming model for performance portable code across a variety of different architectures. In this article, we analyze the performance portable features of MALI via a performance analysis on current CPU-based and GPU-based supercomputers. The analysis highlights not only the performance portable improvements made in finite element assembly and multigrid preconditioning within MALI with speedups between 1.26 and 1.82x across CPU and GPU architectures but also identifies the need to further improve performance in software coupling and preconditioning on GPUs. We perform a weak scalability study and show that simulations on GPU-based machines perform 1.24–1.92x faster when utilizing the GPUs. The best performance is found in finite element assembly, which achieved a speedup of up to 8.65x and a weak scaling efficiency of 82.6% with GPUs. We additionally describe an automated performance testing framework developed for this code base using a changepoint detection method. The framework is used to make actionable decisions about performance within MALI. We provide several concrete examples of scenarios in which the framework has identified performance regressions, improvements, and algorithm differences over the course of 2 years of development.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Real-Time Science Decisioning During High Tempo-High Intensity Mission Operations and the Role of Analogs

Introduction: NASA’s VIPER mission presents a unique operational paradigm within the history of robotic spaceflight. The proximity of the Moon to the Earth and the terrain elements (surface characteristics, light/shadow dynamics, communication links) of the lunar South Polar landing site create unprecedented operational conditions between these two planetary bodies. Apollo era lunar science and exploration included humans in situ to operate instruments and assimilate observational inputs in real-time. Previous lunar orbital missions have worked to operational timescales, e.g., decisional timelines and communication exchanges, that were weeks in length. Mars rover missions have worked to operational timescales, e.g., decisional timelines and communication exchanges between Mars and Earth, that were hours, days, and weeks in length. In the case of the VIPER mission, our operational decisioning for rover driving and instrument commanding will be compressed to minute-scale timeframes. These operational conditions directly impact the manner and speed with which the VIPER Science Team (VST) is required to synthesize and analyze data and produce timely science-driven decisions throughout surface mission operations. The VST shall provide mission enhancing scientific input to guide rover traverse planning and drill site confirmation and selection throughout surface operations. Further, the VST input will be of vital importance to the mission’s ability to maximize science return and to meet broader NASA objectives for future lunar in-situ resource utilization (ISRU)and exploration activities. The VST co-located in the Mission Science Center (MSC) will be responsive to the tactical operational cadence of the Mission Operations Center (MOC) and will provide further strategic and Long-Term Planning (LTP) guidance to the mission. The VIPER Science Operations & Integration(SO&I)team has developed an architecture that is focused on the infusion of science-decisioning into the operational framework and execution cadence of VIPER. NASA analog research has played a significant role in the construction of the VIPER science operations systems. As an example, the SO&I team has led analog missions that have focused on bringing together expertise in the sciences (natural, applied and social) and in operations in service of learning how to build and hold together interdisciplinary work environments and what tools are needed to support high tempo, high intensity integrated decisioning. These experiences have provided an essential foundation of knowledge to the VIPER team. Those analogs that specifically influenced the VIPER science operations construct were identified through a process of comparative analysis to prioritize those that offered relevance in whole or in part, and those that did not. The analog research output that provided extensibility to the VIPER science operations architecture included remote teams of humans and robots in cooperation (synchronous and asynchronous) with simulated earthbound systems, engineering and science teams, and the integrated assembly of tools that supported scientific analysis and data synthesis and provided infrastructure for the remote testing framework. Analogs which included real-time data monitoring, synthesis, visualization and access in a democratized and operationalized manner were of particular interest to the development of the VIPER MSC toolset both in terms of the technology and the processes used to develop the supporting infrastructure. We anticipate that each subsequent mission to the lunar south pole, whether with robots or humans, will be able to optimize science and exploration return by evolving strategies to infuse real-time collaborative science-decisioning. Furthermore, these efforts will result in a foundation for science operations development in support of human-robotic exploration of deep space and Mars. NASA analogs can continue to provide the opportunity to prepare, test and iterate on the operational concepts and tools that will support these ever-expanding space exploration efforts. Our presentation will include an overview of the VIPER Science Operations & Integration development process and specifics on what aspects of analog research have had a significant impact on our work systems.

D S S Lim↗