Search NASA⌕ Search

SEARCH · Search NASA

Results for “Test Framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Modernization efforts for the R -Matrix code SAMMY [Abstract]

The R-Matrix code SAMMY is a widely used nuclear data evaluation code focused on the resolved range, which includes corrections for experimental effects. The code is still mostly written in Fortran 77, and uses a memory management system suitable for the time of its initial writing (1984). A modernization effort is under way to bring the code in-line with modern software development practices. A continuous-integration testing framework was added, automating the large existing set of test cases. It is run on every commit. The memory management was updated to current standard practices suitable for modern software analysis tools. The code can be obtained from https://code.ornl.gov/RNSD/SAMMY. The resonance parameters and covariance information are now stored in C++ objects shared by SAMMY and AMPX, the processing code that generates nuclear data libraries for SCALE. This allows for easier maintenance and access to the resonance parameters inside and outside of SAMMY. This feature is already used by accessing and changing parameters in memory in the Bayesian Monte Carlo Evaluation Framework for Cross Sections Nuclear Data and Integral Benchmark Experiments project, Further plans include the switch to the ENDF reading and writing routines in AMPX, as these routines are more robust, easier to maintain, and support more features. Of note here is support for the new GNDS format. Previously it wasn’t easy to share the full covariance matrix for evaluations containing more than one isotope due to limitations on the ENDF format; this is now supported in GNDS. The data are currently available in a binary SAMMY format and can be exported to GNDS to make them more widely available and sharable. The next step will be to use the same resonance processing code at 0K in AMPX and SAMMY as one of the available Reich-Moore R-Matrix formalism. The first step toward this goal is to isolate the reconstruction into a module that takes resonance parameters as its input and does not depend on SAMMY global parameters. This goal has been achieved and it should now be possible to more easily change the resonance formalism and add enhancements as the Phenomenological R-Matrix parameterization of direct, doorway, and compound nuclear reactions discussed elsewhere on this conference. This concerted modernization and enhancement effort provides multiple advantages to the nuclear data community. It will allow parameter optimization using enhanced formalisms, including experimental effects, that better match complex experimental data. Then those evaluated parameters can immediately be passed off to AMPX to be reconstructed with the exact same cross section model and be put into a data library for subsequent testing using SCALE and the Valid Benchmark suite or other suitable benchmark suites.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Development of a Microchannel High Temperature Recuperator for Fuel Cell Systems

This report summarizes the progress made in development of microchannel recuperators for high temperature fuel cell/turbine hybrid systems for generation of clean power at very high efficiencies. Both Solid Oxide Fuel Cell/Turbine (SOFC/T) and Direct FuelCell/Turbine (DFC/T) systems employ an indirectly heated Turbine Generator to supplement fuel cell generated power. The concept extends the high efficiency of the fuel cell by utilizing the fuel cell’s byproduct heat in a Brayton cycle. Features of the SOFC/T and DFC/T systems include: electrical efficiencies of up to 65% on natural gas, minimal emissions, reduced carbon dioxide release to the environment, simplicity in design, and potential cost competitiveness with existing combined cycle power plants. Project work consisted of candidate material selection from FuelCell Energy (FCE) and Pacific Northwest National Laboratory (PNNL) institutional databases as well as from industrial and academic literature. Candidate materials were then downselected and actual samples were tested under representative environmental conditions resulting in further downselection. A microchannel thermal-mechanical model was developed to calculate overall device cost to be later used in developing a final Tier 1 material candidate list. Specifications and operating conditions were developed for both SOFC/T and DFC/T systems. This development included system conceptualization and progression to process flow diagrams (PFD’s) including all major equipment. Material and energy balances were then developed for the two types of systems which were then used for extensive sensitivity studies that used high temperature recuperator (HTR) design parameters (e.g., operating temperature) as inputs and calculated overall system parameters (e.g., system efficiency). The results of the sensitivity studies determined the final HTR design temperatures, pressure drops, and gas compositions. The results also established operating conditions and specifications for all equipment in the SOFC/T and DFC/T systems. Capital cost and Cost of Electricity (COE) sensitivity analyses have been completed for MW-scale SOFC/T and DFC/T systems. Environmental testing consisted of 1000-hour and 2000-hour dry air oxidation testing on leading candidate materials, used to rank order and, in part, develop a final Tier 1 material candidate list. A thermal-mechanical model was subsequently used to provide material and manufacturing cost estimations for microchannel HTR’s to further refine the Tier 1 candidates. A capital cost and 20-year levelized cost of electricity (COE) was developed for a MW-scale version of the SOFC/T system concept as well as for a MW-scale version of the DFC/T system concept. Test frameworks were established for subsequent long-term materials stability testing, including oxidation resistance and mechanical strength. Mechanical strength testing was then carried out by a third-party test laboratory. Technology demonstration vehicles (TDV’s) were designed and fabricated. Several iterations of TDV’s were fabricated, each improved over the previous build as far as fabrication techniques. Two of three fabricated TDV’s were integrated with the TDV Test Facility for hot-testing at simulated operating conditions. The second of these two was successfully hot-tested for over 1000 hours at simulated temperature and pressure. Post-test leakdown assessment showed negligible leakage at benchtop conditions of 30 psig, a considerable improvement over the previous TDV’s.

30 DIRECT ENERGY CONVERSION↗

pFUnit 3.0 Tutorial Advanced

This tutorial will introduce Fortran developers to unit-testing and test-driven development (TDD) using pFUnit. As with other unit-testing frameworks, pFUnit, simplifies the process of writing, collecting, and executing tests while providing clear diagnostic messages for failing tests. pFUnit specifically targets the development of scientific-technical software written in Fortran and includes customized features such as: assertions for multi-dimensional arrays, distributed (MPI) and thread-based (OpenMP) parallellism, and flexible parameterized tests.These sessions will include numerous examples and hands-on exercises that gradually build in complexity. Attendees are expected to have working knowledge of F90, but familiarity with object-oriented syntax in F2003 and MPI will be of benefit for the more advanced examples. By the end of the tutorial the audience should feel comfortable in applying pFUnit within their own development environment.

Test Driven using pFUnit↗

A streamlined build system foundation for developing HPC software

BLT bundles custom CMake macros, unit testing frameworks for C++ and Fortran, and a set of smoke tests for common HPC dependencies. The combination of these three provides a foundation for quickly bootstrapping a CMale-based system for developing HPC softward.

White, Chris↗

Towards Test Driven Development for Computational Science with pFUnit

Developers working in Computational Science & Engineering (CSE)/High Performance Computing (HPC) must contend with constant change due to advances in computing technology and science. Test Driven Development (TDD) is a methodology that mitigates software development risks due to change at the cost of adding comprehensive and continuous testing to the development process. Testing frameworks tailored for CSE/HPC, like pFUnit, can lower the barriers to such testing, yet CSE software faces unique constraints foreign to the broader software engineering community. Effective testing of numerical software requires a comprehensive suite of oracles, i.e., use cases with known answers, as well as robust estimates for the unavoidable numerical errors associated with implementation with finite-precision arithmetic. At first glance these concerns often seem exceedingly challenging or even insurmountable for real-world scientific applications. However, we argue that this common perception is incorrect and driven by (1) a conflation between model validation and software verification and (2) the general tendency in the scientific community to develop relatively coarse-grained, large procedures that compound numerous algorithmic steps.We believe TDD can be applied routinely to numerical software if developers pursue fine-grained implementations that permit testing, neatly side-stepping concerns about needing nontrivial oracles as well as the accumulation of errors. We present an example of a successful, complex legacy CSE/HPC code whose development process shares some aspects with TDD, which we contrast with current and potential capabilities. A mix of our proposed methodology and framework support should enable everyday use of TDD by CSE-expert developers.

pFUnit↗

A Cybersecurity Event Simulation Tool and Platform

Digital Instrumentation and Control Systems (ICSs) have replaced analog control systems in nuclear power plants raising cybersecurity concerns. To study and understand the cybersecurity risks of nuclear power plants both high fidelity models of the plant physics and controllers must be created, and a framework to test and evaluate cyber security events must be established. A testing and evaluation framework of cybersecurity events consists of a method of interfering with control systems, a simulation of the plant network, and a network packet capture and recording tool. Sandia National Labs (SNL) in collaboration with the University of New Mexico’s Institute for Space and Nuclear Power Studies (UNM-ISNPS) is developing such a cybersecurity testing framework.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Pyctos

Pyctos is a concolic testing framework for standard, dynamically-typed Python. Pyctos is able to generate exhaustive test inputs reaching 100% coverage for a subset of pure Python in the absence of type annotations, even where modern fuzzers would fail. Pyctos's underlying reasoning engine is the CVC5 SMT solver, though Z3 is also supported. Pyctos also supports a growing subset of the standard library and some third-party libraries, such as NumPy.

Washbourne, ErickN [Lawrence Livermore National La↗

Research in Satellite-Fiber Network Interoperability

This four part report evaluated the performance of high data rate transmission links using the ACTS satellite, and to provide a preparatory test framework for two of the space science applications that have been approved for tests and demonstrations as part of the overall ACTS program. The test plan will provide guidance and information necessary to find the optimal values of the transmission parameters and then apply these parameters to specific applications. The first part will focus on the satellite-to-earth link. The second part is a set of tests to study the performance of ATM on the ACTS channel. The third and fourth parts of the test plan will cover the space science applications, Global Climate Modeling and Keck Telescope Acquisition Modeling and Control.

Edelson, Burt↗

Architecture-Based Unit Testing of the Flight Software Product Line

This paper presents an analysis of the unit testing approach developed and used by the Core Flight Software (CFS) product line team at the NASA GSFC. The goal of the analysis is to understand, review, and reconunend strategies for improving the existing unit testing infrastructure as well as to capture lessons learned and best practices that can be used by other product line teams for their unit testing. The CFS unit testing framework is designed and implemented as a set of variation points, and thus testing support is built into the product line architecture. The analysis found that the CFS unit testing approach has many practical and good solutions that are worth considering when deciding how to design the testing architecture for a product line, which are documented in this paper along with some suggested innprovennents.

Ganesan, Dharmalingam↗

Test Driven Development of Scientific Models

Test-Driven Development (TDD), a software development process that promises many advantages for developer productivity and software reliability, has become widely accepted among professional software engineers. As the name suggests, TDD practitioners alternate between writing short automated tests and producing code that passes those tests. Although this overly simplified description will undoubtedly sound prohibitively burdensome to many uninitiated developers, the advent of powerful unit-testing frameworks greatly reduces the effort required to produce and routinely execute suites of tests. By testimony, many developers find TDD to be addicting after only a few days of exposure, and find it unthinkable to return to previous practices.After a brief overview of the TDD process and my experience in applying the methodology for development activities at Goddard, I will delve more deeply into some of the challenges that are posed by numerical and scientific software as well as tools and implementation approaches that should address those challenges.

Overview of TDD Process↗

Autonomous Closed-Loop Experimental Characterization and Dynamic Model Validation of a Scaled Underwater Kite

This paper presents the closed-loop experimental framework and dynamic model validation for a 1/12-scale underwater kite design. The pool-based tow testing framework described herein, which involves a fully actuated, closed-loop controlled kite and flexible tether, significantly expands upon the capabilities of any previously developed open-source framework for experimental underwater kite characterization. Specifically, the framework has allowed for the validation of three closed-loop flight control strategies, along with a critical comparison between dynamic model predictions and experimental results. In this paper, we provide a detailed presentation of the experimental tow system and kite setup, describe the control algorithms implemented and tested, and quantify the level of agreement between our multi-degree-of-freedom kite dynamic model and experimental data. Finally, we also present a sensitivity analysis that helps to identify the most influential parameters to kite performance and further explain the remaining mismatches between the model and data.

42 ENGINEERING↗

UnitTesterSG

UnitTesterSG is a unit testing framework that is designed for nested and/or scientific/engineering data structures. It is designed primarily for testing the outputs if a single function or simulation run by storing the expected results file such that comparisons to the stored output can be made with unit tests after the function or software has been edited. This module is designed to be able to unit test arrays, strings, as well as nested and deeply nested objects (including arrays with tuples inside, arrays with strings inside, etc.). The module is compatable with nested objects, and thus has a dependency on the nestedObjectsFunctions module within. The module takes *calculated* results and then compares them to *expected* results. If no expected results are available (or they do not match the calculated results) the module then offers to store the calculated results as expected results for next time.

Savara, Aditya [Oak Ridge National Lab. (ORNL), Oa↗

Adventures in cFS Unit Testing: Examining the Past to Explain the Present with an Eye toward the Future

An overview of my experiences writing unit tests for various projects with a specific focus on my work unit testing core Flight System (cFS) applications. I recount some of the direct personal experiences I have had that showed me the utility of having done unit testing for my projects. Many of the tips, tricks and pitfalls encountered during my time writing unit tests for the cFS app, CF, are examined. I also compare and contrast my cFS unit testing development with that of a parallel project, in which I write unit tests using RSpec, a testing framework for the Ruby programming language. I impart my complete methodology behind the CF app unit testing effort and the rationale for why I did it that way. Then I give some ideas for how you can do your own unit testing for cFS applications. You will also learn about my hopes for how unit testing cFS applications can be done going forward from where we are now.

"unit testing"↗

Framework for a Comparative Accelerated Testing Standard for PV Modules: Preprint

As the photovoltaic industry has grown, the interest in comparative accelerated testing has also grown. Private test labs offer testing services that apply greater stress than the standard qualification tests as tools for differentiating products and for gaining increased confidence in long-term PV investments. While the value of a single international standard for comparative accelerated testing is widely acknowledged, the development of a consensus is difficult. This paper strives to identify a technical basis for a comparative standard.

14 SOLAR ENERGY↗

Development of an MC&A toolbox for liquid-fueled molten salt reactors with online reprocessing (Final Report)

A critical barrier to the deployment of MSRs is the absence of a well-defined nuclear material control and accounting (MC&A) approach, a vital prerequisite to meet NRC licensing requirements as well as facilitating future international exports. Liquid-fueled MSR variants (especially those incorporating online refueling or reprocessing) present a unique set of challenges to traditional nuclear material control and accountancy. Unlike solid-fueled light-water reactors, traditional item-counting methods cannot be applied as an accountancy strategy. Rather, MC&A approaches to MSR variants (including both uranium and thorium fueled designs) are more analogous to bulk material handling facilities (e.g., enrichment and reprocessing); yet further complicating matters is the fact that fuel medium is also highly radioactive. Meanwhile, the space of MSRs covers a broad range of design parameters, including thermal and fast spectra designs; operation in actinide breeder or burner modes, choice of actinide fuel used (e.g., 235 U, 232 Th / 233 U, denatured 233 U, etc.), pool or loop-type configuration, and even different salt chemistry. Each of these design choices introduce significant challenges to MC&A approaches within MSR facilities. We propose to bridge this gap for liquid-fueled MSRs by developing a modular, component-based test framework for evaluating viable process monitoring and MC&A techniques specifically suited to liquid-fueled MSR system variants employing online refueling or reprocessing. This test platform will consist of a toolbox of independent process modules representing discrete physical units (such as the reactor core, off-gas processing, decay tanks, and actinide separation units), each with its own self-contained physics responsive to the input mass flow, along with appropriate measurement models that can be coupled to key flow points. These dynamic physical signatures thus afford the ability to test the viability and efficacy of potential accountancy techniques under the full range of reactor operating conditions. As process modules are connected via mass flows, the result is a reconfigurable, generic MSR mass flow model capable of serving as an MC&A test platform for a broad spectrum of possible MSR configurations. The resulting MSR MC&A toolbox will enable robust assessment of accountancy strategies for this unique facility type, including analysis of physical feedbacks arising both from depletion of the fuel over time as well as from potential off-normal events, including those introduced by equipment failures (e.g., a pump failure) as well as by malicious action (i.e., attempts to divert material). The proposed toolbox both addresses a critical needs area for the MPACT analysis toolkit while leveraging existing MPACT-sponsored tools, especially with respect to simulation of measurement and accountancy techniques for advanced fuel cycle facilities. Beyond enabling new analysis capabilities for MSR systems, the design of this toolbox will be to such to maximize compatibility with existing MPACT tools, such to enhance existing facility MC&A analysis capabilities.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Adaptive Stress Testing: Using Reinforcement Learning to Find Failures in Safety-Critical Systems

Emerging applications in artificial intelligence, such as driverless cars and autonomous aircraft promise to be more efficient, cheaper to operate, and always available. However, ensuring the safety of these systems remains a major challenge to their certification and adoption. These autonomous systems are expected to routinely make safety-critical decisions where failures can have serious consequences including loss of life and property. Testing and validation techniques aim to identify and diagnose potential failures before the system is deployed. However, finding failure scenarios in autonomous systems can be very challenging due to high-dimensional and continuous state spaces, interaction with large environments over many time steps, and the rarity of failures. This talk presents Adaptive Stress Testing (AST), a simulation-based testing framework for finding the most likely path to a failure event of a safety-critical system. The key idea of AST is that stress testing can be formulated as a Partially Observable Markov Decision Process (POMDP), which enables reinforcement learning techniques to be used for finding failure events. Reinforcement learning algorithms can efficiently explore the search space and have been shown to scale to very large systems. We present applications of AST to find failures in various safety-critical systems including the aircraft collision avoidance systems, autonomous cars, and small unmanned aerial vehicles.

autonomous vehicles↗

Gene expression of functionally-related genes coevolves across fungal species: detecting coevolution of gene expression using phylogenetic comparative methods

Researchers often measure changes in gene expression across conditions to better understand the shared functional roles and regulatory mechanisms of different genes. Analogous to this is comparing gene expression across species, which can improve our understanding of the evolutionary processes shaping the evolution of both individual genes and functional pathways. One area of interest is determining genes showing signals of coevolution, which can also indicate potential functional similarity, analogous to co-expression analysis often performed across conditions for a single species. However, as with any trait, comparing gene expression across species can be confounded by the non-independence of species due to shared ancestry, making standard hypothesis testing inappropriate. We compared RNA-Seq data across 18 fungal species using a multivariate Brownian Motion phylogenetic comparative method (PCM), which allowed us to quantify coevolution between protein pairs while directly accounting for the shared ancestry of the species. Our work indicates proteins which physically-interact show stronger signals of coevolution than randomly-generated pairs. Interactions with stronger empirical and computational evidence also showing stronger signals of coevolution. We examined the effects of number of protein interactions and gene expression levels on coevolution, finding both factors are overall poor predictors of the strength of coevolution between a protein pair. Simulations further demonstrate the potential issues of analyzing gene expression coevolution without accounting for shared ancestry in a standard hypothesis testing framework. Furthermore, our simulations indicate the use of a randomly-generated null distribution as a means of determining statistical significance for detecting coevolving genes with phylogenetically-uncorrected correlations, as has previously been done, is less accurate than PCMs, although is a significant improvement over standard hypothesis testing. These methods are further improved by using a phylogenetically-corrected correlation metric. Our work highlights potential benefits of using PCMs to detect gene expression coevolution from high-throughput omics scale data. This framework can be built upon to investigate other evolutionary hypotheses, such as changes in transcription regulatory mechanisms across species.

59 BASIC BIOLOGICAL SCIENCES↗