Search NASA⌕ Search

SEARCH · Search NASA

Results for “test frameworks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Application-level regression testing framework using Jenkins

Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. The goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We also describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integrate with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Sudoku Online Testing Framework v.0.2

The Sudoku Online Testing Framework provides a configurable basis for testing user interactions to solve visualization-related problems with sudoku puzzles and games. It consists of a server, a client, and test code. The python server implements sudoku puzzle logic and board selection functions. The JavaScript client implements a graphical user interface (GUI) front-end to display the puzzles and facilitate interactions with users. Within the GUI, logical rules may be selected. Cells may be selected to “pivot” on to make a board for each possible value in the cell. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-6409 O

Leger, Michelle↗
DOE OSTI2023

A Testing Framework for Grid-Forming Resources

This paper presents a testing framework for grid-forming (GFM) resources. First, it analytically derives in the frequency domain the active and reactive power response of an ideal voltage source with a reactor. These transfer functions are then used to quantify the voltage source behavior expected from a GFM inverter within subtransient timescales (5-15 cycles) following a grid disturbance. The paper also shows that the testing of a GFM inverter might require a reactor of an appropriate size between the inverter and the grid simulator used for the inverter testing. Finally, the paper presents a systematic approach for developing specifications for GFM resources using active and reactive power response frequency scans. The testing framework and results presented in this paper are demonstrated using EMT-PSCAD simulations of a 1-MW GFM inverter.

frequency scans↗
DOE OSTI2023

A Testing Framework for Grid-Forming Resources: Preprint

This paper presents a testing framework for grid-forming (GFM) resources. It first analytically derives in frequency domain the active and reactive power response of an ideal voltage source with a reactor. These transfer functions are then used to quantify the voltage source behavior expected from a GFM inverter within subtransient time scales (5-10 cycles) following a grid disturbance. The paper also shows that the testing of a GFM inverter might require a reactor of an appropriate size between the inverter and the grid simulator used for the inverter testing. Finally, the paper presents a systematic approach for developing specifications for GFM resources using active and reactive power response frequency scans. The testing framework and results presented in this paper are demonstrated using EMT simulations of a 1-MW GFM inverter.

frequency scans↗
DOE OSTI2016

ADC evaluation boards design and test framework for LCLS-II precision receiver

In the low-level RF control field, ADC acquisition accuracy and noise set the boundary of our control ability, making it important to develop low-noise acquisition systems. From the design to test stage, all the noise terms should be understood and characterized. The specific need addressed here is the precision acquisition system for the second Linac Coherent Light Source (LCLS-II), led by SLAC National Accelerator Laboratory. Test circuit boards for the LTC2174 and AD9268 ADCs are designed and fabricated by LBNL. An ADC test framework based on FPGA evaluation board to assess performance has been developed. The ADC test framework includes both DSP (Digital Signal Processing) firmware and processing software. It is useful for low-level RF control and other synchronization projects. Investigating the clock jitter between two channels give us an understanding of that noise source. Working with the test framework, the raw ADC data are transferred to a computer through a Gigabit Ethernet interface. Then short-term error signal can be calculated based on a sine wave fit. By changing low-pass filter bandwidth, relative long-term performance can also be obtained. Amplitude jitter and differential phase jitter are the key issues for ADCs. This paper will report the test results for LTC2174 and AD9268 chips. The integral amplitude jitter is smaller than 0.003 %, and the integral phase noise is smaller than 0.0015° (measured at 47 MHz RF, 100 MHz clock, bandwidth 1 Hz to 100 kHz) for both ADC chips.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2017

A Unit Testing Framework for Scientific Legacy Code

Large-scale scientific applications play important roles in supporting research. However, it is often very expensive and time-consuming to make changes to, maintain and evolve the scientific code due to its complexity and poor programming skills of researchers. Therefore, in order to visualize scientific code architecture to optimize software design, understand undocumented source code, and analyze software flow and functionality, we first introduce a unit testing framework (UTF). Then, because such infrastructure's performance is very crucial in practical use since the scientific legacy applications simulate instances in a long period of time, we improve the UTF by applying Message Passing based Parallelization and parallel I/O operations. Furthermore, due to the scientific code has enormous state data and the I/O capacity on the server is limited, we apply in situ data analysis method to encounter fewer resource limitations, and adopt signal processing to greatly reduce data transfer. Last, we demonstrated the correctness and high-efficiency of our framework for legacy Earth model on Titan supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

SPEL: Software tool for Porting E3SM Land Model with OpenACC in a Function Unit Test Framework

Most high-end computers adopt hybrid architecture, porting a large-scale scientific code onto accelerators is necessary. The paper presents a generic method for porting large-scale scientific code onto accelerators using compiler directives within a modularized function unit test platform. We have implemented the method and designed a software tool (SPEL) to port the E3SM Land Model (ELM) onto the GPUs in the Summit computer. SPEL automatically generates GPU-ready test modules for all ELM functions, such as CanopyFlux, SoilTemperature, and EcosystemDynamics. SPEL breaks the ELM into a collection of standalone unit test programs for easy code verification and further performance improvement. We further optimize several ELM test modules with advanced techniques, including memory reduction, reconstructed parallel loops, and asynchronous GPU kernel launch. We hope our study will inspire new toolkit developments that expedite large-scale scientific code porting with compiler directives.

Schwartz, Peter↗
DOE OSTI2018

So ware-Defined Network Solutions for Science Scenarios: Performance Testing Framework and Measurements

High-performance scientific work flows utilize supercomputers, scientific instruments, and large storage systems. Their executions require fast setup of a small number of dedicated network connections across the geographically distributed facility sites. We present Software-Defined Network (SDN) solutions consisting of site daemons that use dpctl, Floodlight, ONOS, or OpenDaylight controllers to set up these connections. The development of these SDN solutions could be quite disruptive to the infrastructure, while requiring a close coordination among multiple sites; in addition, the large number of possible controller and device combinations to investigate could make the infrastructure unavailable to regular users for extended periods of time. In response, we develop a Virtual Science Network Environment (VSNE) using virtual machines, Mininet, and custom scripts that support the development, testing, and evaluation of SDN solutions, without the constraints and expenses of multi-site physical infrastructures; furthermore, the chosen solutions can be directly transferred to production deployments. By complementing VSNE with a physical testbed, we conduct targeted performance tests of various SDN solutions to help choose the best candidates. In addition, we propose a switching response method to assess the setup times and throughput performances of different SDN solutions, and present experimental results that show their advantages and limitations.

Settlemyer, Bradley↗
DOE OSTI2024

MUPPET: An automated OpenMP mutation testing framework for performance optimization

MUPPET is a tool for OpenMP programs that identifies program modifications, called mutations, aimed at improving program performance. Existing performance optimization techniques, including profiling-based and auto-tuning techniques, fail to indicate program modifications at the source level thus preventing their portability across compilers. MUPPET aims to help HPC developers reason about performance defects and missed opportunities to improve performance at the source code level.

Parasyris, Konstantinos↗