Search NASA⌕ Search

SEARCH · Search NASA

Results for “test harness”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

OLCF Test Harness

Acceptance and regression testing of a High Performance Computing (HPC) system requires an automated and reproducible framework and tool for running and logging results. Manually running tests across a system is labor intensive and prone to reproducibility errors. The OLCF Test Harness (OTH) provides a framework in which to document required tests for a HPC system. The OTH then provides tools to execute and log results of these tests in an automated fashion.

Dietz, Dan [Oak Ridge National Laboratory (ORNL), ↗
DOE OSTI2015

NCCS Regression Test Harness

The NCCS Regression Test Harness is a software package that provides a framework to perform regression and acceptance testing on NCCS High Performance Computers. The package is written in Python and has only the dependency of a Subversion repository to store the regression tests.

Tharrington, Arnold N.↗
DOE OSTI2018

Test Harness Framework

The code facilitates running tests and then validating the results. The tests are normally attacks on a system and the verification is normally to validate protection software detected the attack, however the framework can be used in any area.

McCarty, MichaelV↗
DOE OSTI2018

asset-score-api-test-harness

This repository contains sample code written in PHP that allows you to access the Asset Score API. The Department of Energy Building Energy Asset Score (Asset Score) is a national standardized tool for evaluating the physical and structural energy efficiency of commercial and multifamily residential buildings. The scoring tool will store user-provided data and generate an asset score and system evaluation for your building envelope and mechanical and electrical systems. The tool will also identify cost-effective upgrade opportunities and help you gain insight into the energy efficiency potential of your building.

Ellis, Edward↗
DOE OSTI2023

SABATH: Surrogate AI Benchmarking Applications' Testing Harness

SABATH provides benchmarking infrastructure for evaluating scientific ML/AI models. It contains support for scientific machine learning surrogates from external repositories. The software dependences are explicitly exposed in the surrogate model definition, which allows the use of advanced optimization, communication, and hardware features.

Luszczek, Piotr [Univ. of Tennessee, Knoxville, TN↗
NASA NTRS1982

High-Density Terminal Box for Testing Wire Harness

Compact terminal box provides access to complex wiring harnesses for testing. Box accommodates more than twice as many wires as previous boxes. Box takes in wires via cable connectors and distributes them to contacts on box face. Instead of separate insulated jacks in metal face panel, box uses pairs of small military-standard metal sockets in precision-drilled plastic panel. Shorting plug provides continuity for wires when not being tested.

Pierce, W. B.↗
NASA NTRS2016

Automation of Flight Software Regression Testing

NASA is developing the Space Launch System (SLS) to be a heavy lift launch vehicle supporting human and scientific exploration beyond earth orbit. SLS will have a common core stage, an upper stage, and different permutations of boosters and fairings to perform various crewed or cargo missions. Marshall Space Flight Center (MSFC) is writing the Flight Software (FSW) that will operate the SLS launch vehicle. The FSW is developed in an incremental manner based on "Agile" software techniques. As the FSW is incrementally developed, testing the functionality of the code needs to be performed continually to ensure that the integrity of the software is maintained. Manually testing the functionality on an ever-growing set of requirements and features is not an efficient solution and therefore needs to be done automatically to ensure testing is comprehensive. To support test automation, a framework for a regression test harness has been developed and used on SLS FSW. The test harness provides a modular design approach that can compile or read in the required information specified by the developer of the test. The modularity provides independence between groups of tests and the ability to add and remove tests without disturbing others. This provides the SLS FSW team a time saving feature that is essential to meeting SLS Program technical and programmatic requirements. During development of SLS FSW, this technique has proved to be a useful tool to ensure all requirements have been tested, and that desired functionality is maintained, as changes occur. It also provides a mechanism for developers to check functionality of the code that they have developed. With this system, automation of regression testing is accomplished through a scheduling tool and/or commit hooks. Key advantages of this test harness capability includes execution support for multiple independent test cases, the ability for developers to specify precisely what they are testing and how, the ability to add automation, and the ability of the harness and cases to be executed continually. This test concept is an approach that can be adapted to support other projects.

Tashakkor, Scott B.↗
DOE OSTI2026

New Report Highlights LAMP's Role as a Robust and Reliable System for Wave Energy Converter Testing

Harnessing the power of waves requires more than just putting devices in the ocean - it takes tools that can refine and accelerate innovation before the first deployment. That's where NLR's largeamplitude motion platform, or LAMP, comes in. LAMP is a six-degree-of-freedom motion system designed to replicate the response of a wave energy converter (WEC) device in realistic ocean conditions. By letting researchers evaluate WEC performance in a safe, controlled lab environment before moving offshore, LAMP helps accelerate innovation while reducing risk. An NLR report, titled Initial Characterization of the NLR LargeAmplitude Motion Platform (https://www.nrel.gov/docs/fy26osti/93733.pdf), provides a deeper understanding of LAMP's capabilities and also lays the groundwork for future advancements in WEC testing methodologies. Researchers studied LAMP's amplitude and frequency limits across different payloads and collected data from 16 test profiles. Testing demonstrated that platform accuracy remained consistent across all payloads evaluated; researchers expect similar accuracy up to LAMP's 10,000-kilogram capacity limit. The measured position limits for single degrees of freedom experienced by WECs, including surge, sway, heave, roll, pitch, and yaw, also exceeded the original manufacturer specifications, providing new insight into the true boundaries of the system.

16 TIDAL AND WAVE POWER↗
DOE OSTI2017

Quinoa, Version 0.2

Quinoa is a set of computational tools that enables research and numerical analysis in fluid dynamics. At this time it remains a test-bed to experiment with various algorithms using fully asynchronous runtime systems. Currently, Quinoa consists of the following tools: (1) Walker, a numerical integrator for systems of stochastic differential equations in time. It is a mathematical tool to analyze and design the behavior of stochastic differential equations. It allows the estimation of arbitrary coupled statistics and probability density functions and is currently used for the design of statistical moment approximations for multiple mixing materials in variable-density turbulence. (2) Inciter, an overdecomposition-aware finite element field solver for partial differential equations using 3D unstructured grids. Inciter is used to research asynchronous mesh-based algorithms and to experiment with coupling asynchronous to bulk-synchronous parallel code. Two planned new features of Inciter, compared to the previous release (LA-CC-16-015), to be implemented in 2017, are (a) a simple Navier-Stokes solver for ideal single-material compressible gases, and (b) solution-adaptive mesh refinement (AMR), which enables dynamically concentrating compute resources to regions with interesting physics. Using the NS-AMR problem we plan to explore how to scale such high-load-imbalance simulations, representative of large production multiphysics codes, to very large problems on very large computers using an asynchronous runtime system. (3) RNGTest, a test harness to subject random number generators to stringent statistical tests enabling quantitative ranking with respect to their quality and computational cost. (4) UnitTest, a unit test harness, running hundreds of tests per second, capable of testing serial, synchronous, and asynchronous functions. (5) MeshConv, a mesh file converter that can be used to convert 3D tetrahedron meshes from and to either of the following formats: Gmsh, http://www.geuz.org/gmsh, Netgen, http://sourceforge.net/apps/mediawiki/netgen-mesher, ExodusII, http://sourceforge.net/projects/exodusii, HyperMesh, http://www.altairhyperworks.com/product/HyperMesh.

Bakosi, Jozsef↗
DOE OSTI2024

FORCE Regression Testing

Via programs including the Light Water Reactor Sustainability and Integrated Energy Systems, the U.S. Department of Energy has invested in the Framework for Optimization of ResourCes and Economics (FORCE) software framework (Idaho National Laboratory 2024a) for the technical and economic analysis of nuclear-integrated energy systems (IES). Nuclear IES expand the use of nuclear from traditional baseload electricity generation to a flexible and adaptive source of combined heat and power. Nuclear heat can be used in the production of a variety of energy currencies such as hydrogen and ammonia as well as other heat applications including water desalination and district heating. FORCE is designed with the intent to provide interconnected analysis tools that enable the accurate technical and economic assessment of specific nuclear IES configurations for individual energy markets. FORCE consists of three main analysis pathways: HYBRID (Idaho National Laboratory 2024b), which contains high-resolution physical models for IES; Holistic Energy Resource Optimization Network (HERON) (Idaho National Laboratory 2024c), which analyzes IES long-term economic viability; and Optimization of Real-time Capacity Allocation (ORCA) (Idaho National Laboratory 2024d), designed for real-time control of IES via digital twins and optimal decision making, including autonomous and remote operation research. Development of the FORCE ecosystem is guided by three pillars: capability, which assures that the computational requirements of IES analysis are met by the software tools; reliability, which provides for consistent code performance and expected behaviors; and accessibility, which lowers the barrier to entry for using the software and accelerates analysis by users beyond the FORCE primary developers. Reliability of the FORCE ecosystem is established according to the American Nuclear Society?s Nuclear Quality Assurance (NQA-1) program [American Society of Mechanical Engineers 1982], with specific levels of software quality assurance (SQA) within NQA-1 applied to each software tool in FORCE. As the tools within FORCE have matured, some integration algorithms to accurately connect the software tools for holistic analysis have been developed and deployed within the FORCE software repository. In accordance with NQA-1 standards, regression tests are required to guarantee the software performs consistently even when new capabilities are added to the software. In this report, we document the deployment of both unit tests, which test the consistent behavior of small pieces of the FORCE code base, as well as integration tests, which test the consistent performance of full use cases for the FORCE integration algorithms. We further document the encapsulation of these tests within a test harness, which collectively checks for each successful test completion on demand. Finally, we document the automation of the test harness using GitHub Actions [GitHub 2024], which require all tests succeed before any new capability or other changes can be added to the FORCE integration software

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2023

A Multi-Sensor, Low Volume, Automated Culture System for Space Biology Experiments

Optical density (absorbance) is commonly used in microbiology to measure cell concentration and other colorimetric indicators. It is the main type of bioscience data returned from the BioSentinel small spacecraft mission, which was recently launched as a secondary payload on Artemis I. The flight unit takes absorbance measurements of yeast cultures at 570 nm, 630 nm, and 850 nm to track cell concentration and oxidation-reduction (redox) state of the metabolic indicator dye alamarBlue over time [1]. Because the data returned from a small spacecraft mission is necessarily limited, BioSentinel uses a specialized ground-based experiment apparatus to map the absorbance data onto several other parameters. This culture apparatus is composed of a polycarbonate base and lid which contain ports for several commercial sensors, including a standard 12 mm dissolved oxygen probe, gaseous CO2, fluid pressure, and two micro-probes for measuring pH and redox potential. It also contains a pressure relief valve, a rubber membrane for culture interaction, and a 12mm port for the custom optical probe designed to match in-flight measurements. Modules in two corners expand the culture volume into narrow columns, enabling solid-state optical density measurement and dielectric spectroscopy. The apparatus can be fit to a plate for magnetic stirring. The apparatus’s optical probe allows mapping between the flight unit’s optical data and the additional parameters measured by the ground-based experiments. Within the probe, a linear actuator pulls a plunger, drawing culture volume into the optical path. The actuator then pushes out the plunger and culture volume, cleaning any stray yeast from the inside surface of the probe. The probe chassis is 3D-printed from biocompatible resin with the optoelectronics cured inside an optically clear conformal coat which functions as a lens. To optimize the probe’s dimensions, including optical path, a test harness was developed to manipulate and measure distance while the LED light sources and photodiode detector are in operation. The test harness was composed of two 3/64” thick polycarbonate plates mounted to a set of calipers. The LEDs and photodiode were mounted on the outside of each plate. A cuvette of sample fluid was then clamped between the plates in line between the LEDs and photodiode. The test harness generated a 78% signal change between water and a dye standard for the green (570 nm) LED, a 90% signal change for the red (630 nm) LED, and a 25% signal change between water and overgrown yeast for the infrared (850 nm) LED. The improvements to the ground experiment apparatus (culture volume reduction, inclusion of an optical probe to measure flight-like optical data, and magnetic stirring for culture homogeneity) allow for more flight-like measurements to be taken and more accurate mapping of these additional parameters to the optical data returned from the flight unit. This enhancement to mission science return will give insight into how deep space radiation may affect human biology for future long-term space exploration.

Multi-Sensor↗
DOE OSTI2019

Harmony: A Harness Monitoring System for the Oak Ridge Leadership Computing Facility

Acceptance of a new system requires extensive testing and is often comprised of hundreds of tests. Summit, the latest flagship super-computer at the Oak Ridge Leadership Computing Facility (OLCF), and the number one system in the November 2018 Top500 list [2], completed its acceptance testing in 2018. To execute acceptance, the acceptance test (AT) team utilizes the OLCF test harness, a tool developed at the OLCF that automates the launch and verification of all acceptance tests. Acceptance requires analysis of test results and classification of all test failures. The sheer number of tests involved makes performing these tasks challenging. To complete these tasks more efficiently, in addition to lessen the personnel burden during acceptance testing, we developed a monitoring system for the OLCF test harness called Harmony.

Kuchta, Cameron↗
DOE OSTI2018

vvtools v. 1.0

Vvtools is a suite of testing tools, with a focus on reproducible verification and validation. They are written in pure Python, and contain a test harness and an automated process management tool. Users of vvtools can develop suites of verification and validation tests and run them on small to large high performance computing resources in an automated and reproducible way. The test harness enables complex processes to be performed in each test and even supports a one-level parent/child dependency between tests. It includes a built in capability to manage workloads requiring multiple processors and platforms that use batch queueing systems.

Drake, RichardR.↗
NASA NTRS2003

Model-driven software verification

In this paper we explore a different approach to software verification. With this approach, a software application can be included, without substantial change, into a verification test-harness and then verified directly, while presearving the ability to apply data abstraction techniques. Only the test-harness is written in the language of the model checker.

software verification↗
NASA NTRS2011

Evaluation of Cable Harness Post-Installation Testing

The Cable Harness Post-Installation Testing Report was written in response to an action issued by the Ares Project Control Board (PCB). The action for the Ares I Avionics & Software Chief Engineer and the Avionics Integration and Vehicle Systems Test Work Breakdown Structure (WBS) Manager in the Vehicle Integration Office was to develop a set of guidelines for electrical cable harnesses. Research showed that post-installation tests have been done since the Apollo era. For Ares I-X, the requirement for post-installation testing was removed to make it consistent with the avionics processes used on the Atlas V expendable launch vehicle. Further research for the report involved surveying government and private sector launch vehicle developers, military and commercial aircraft, spacecraft developers, and harness vendors. Responses indicated crewed launch vehicles and military aircraft perform post-installation tests. Key findings in the report were as follows: Test requirements identify damage, human-rated vehicles should be tested despite the identification of statistically few failures, data does not support the claim that post-installation testing damages the harness insulation system, and proper planning can reduce overhead associated with testing. The primary recommendation of the report is for the Ares projects to retain the practice of post-fabrication and post-installation cable harness testing.

King, M. S.↗
DOE OSTI2020

Towards Acceptance Testing at the Exascale Frontier

At the 2007 Cray User Group meeting, the Oak Ridge Leadership Computing Facility (OLCF) introduced the OLCF Test Harness (OTH), a framework[1] used for acceptance testing of the Jaguar supercomputer[2]. Since then, the OTH framework has evolved to version 2.0 which adds new features and streamlines usability. The OTH is the key piece of software used to orchestrate acceptance testing for all OLCF computational resources before they are deployed for production use, including our leadership class high performance computing (HPC) systems. The OTH framework is written in Python and is publicly available[3].In this paper, we first describe the requirements, design, and structure of the OTH. Then, we present specific improvements developed to support acceptance testing of the OLCF’s Summit system[4]. We will also showcase new OTH features that have been added to streamline the acceptance test process as well as the motivation behind those changes. As part of this work, we also evaluated different workflow tools in order to determine whether these tools could complement the OTH in two key areas: automation and reporting. The advantages and disadvantages identified with each tool will be discussed. Lastly, we summarize the challenges and lessons learned collected from using the OTH for the acceptance of the last three flagship systems at the OLCF. These may be useful for other HPC centers developing their own testing frameworks or those interested in using the OTH.

Melesse Vergara, Veronica↗
DOE OSTI2023

Transient Efficiency, Flexibility, and Reliability Optimization of Coal-Fired Power Plants - Final Report

This program developed an advanced model-based monitoring and model-predictive control algorithms for a coal fired power plant (CFPP), and deployed these algorithms in a real-time platform to demonstrate performance benefits for transient flexibility and plant operation efficiency. More specifically, the objectives were successfully achieved through a combination of (i) developing a high-fidelity transient plant model in Apros, which was used as a high-fidelity plant simulation between $100-50\% TMCR$ where TMCR denotes the turbine maximum continuous rating, i.e., baseload, (ii) developing a very fast physics-based reduced-order model (ROM) of the plant, which ran more than $100\times$ faster than real-time, enabling its use as real-time embedded model for model-based estimation (MBE) and model predictive control (MPC) (iii) implementing a real-time MBE based on ROM using a robust unscented Kalman filter (UKF) to continuously tune the ROM to match the measurements from high-fidelity Apros plant model despite significant plant-model mismatch, and thus, obtain a Digital Twin of the plant (iv) designing and implementing a real-time MPC with dual objectives of transient plant load tracking with high ramp rates and minimizing coal consumption, i.e., improving plant efficiency in the baseload-partload operation range of $100-50\% TMCR$. Each key element above was developed and tested individually, and has been reported in corresponding Topical Reports. Finally, all the individual elements were integrated in an overall closed-loop system, that was successfully tested in desktop Simulink test harness simulations with ROM or high-fidelity model as the plant. Thereafter, the Simulink implementation was used to auto-generate C-code and deploy as real-time Docker microservice containers in Linux, and validate that they can run in real-time in the hardware-in-the loop (HIL) setup and produce the same results as in Simulink. The results of the integrated simulation tests in Simulink as well as the real-time HIL deployment are documented in this final report, showing good load tracking for load ramps at $3-4\%/min$ ramp rates, and achieving up to $5.5\%$ reduction in coal relative to baseline operation at $50\% TMCR$ load. The desktop and HIL simulations show successful performance of the overall model based estimation and control solution and achieve the key objectives of the program for flexible, efficient and reliable operation of subcritical coal fired power plants.

20 FOSSIL-FUELED POWER PLANTS↗