Search NASA⌕ Search

SEARCH · Search NASA

Results for “software testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Vitis CI/CD & Containerization

Modern development flows that use tooling for automated building, testing, and deployment of software are becoming the norm for large scale software and hardware projects. These flows offer quite a few advantages that make them desirable, but when attempting to implement them for projects that use FPGAs, some complications can arise when attempting to integrate them with traditional FPGA toolchains and deployment workflows. In this poster, we present early our early efforts integrating the Vitis FPGA toolchain into a modern, Kuberentes and Github actions based CI/CD pipeline.

Hawks, Benjamin G. [Fermilab]↗
DOE OSTI2024

Assessment of Envelope- and Machine Learning-Based Electrical Fault Type Detection Algorithms for Electrical Distribution Grids

This study introduces envelope- and machine learning (ML)-based electrical fault type detection algorithms for electrical distribution grids, advancing beyond traditional logic-based methods. The proposed detection model involves three stages: anomaly area detection, ML-based fault presence detection, and ML-based fault type detection. Initially, an envelope-based detector identifying the anomaly region was improved to handle noisier power grid signals from meters. The second stage acts as a switch, detecting the presence of a fault among four classes: normal, motor, switching, and fault. Finally, if a fault is detected, the third stage identifies specific fault types. This study explored various feature extraction methods and evaluated different ML algorithms to maximize prediction accuracy. The performance of the proposed algorithms is tested in an emulated software–hardware electrical grid testbed using different sample rate meters/relays, such as SEL735, SEL421, SEL734, SEL700GT, and SEL351S near and far from an inverter-based photovoltaic array farm. The performance outcomes demonstrate the proposed model’s robustness and accuracy under realistic conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

IoT Firmware Emulation and Its Security Application in Fuzzing: A Critical Revisit

As IoT devices with microcontroller (MCU)-based firmware become more common in our lives, memory corruption vulnerabilities in their firmware are increasingly targeted by adversaries. Fuzzing is a powerful method for detecting these vulnerabilities, but it poses unique challenges when applied to IoT devices. Direct fuzzing on these devices is inefficient, and recent efforts have shifted towards creating emulation environments for dynamic firmware testing. However, unlike traditional software, firmware interactions with peripherals that are significantly more diverse presents new challenges for achieving scalable full-system emulation and effective fuzzing. This paper reviews 27 state-of-the-art works in MCU-based firmware emulation and its applications in fuzzing. Instead of classifying existing techniques based on their capabilities and features, we first identify the fundamental challenges faced by firmware emulation and fuzzing. We then revisit recent studies, organizing them according to the specific challenges they address, and discussing how each specific challenge is addressed. We compare the emulation fidelity and bug detection capabilities of various techniques to clearly demonstrate their strengths and weaknesses, aiding users in selecting or combining tools to meet their needs. Finally, we highlight the remaining technical gaps and point out important future research directions in firmware emulation and fuzzing.

Zhou, Wei (ORCID:0000000178340839)↗
DOE OSTI2025

The ICEBERG Test Stand for DUNE Cold Electronics Development

ICEBERG is a liquid argon time projection chamber at Fermilab for the purpose of testing detector components and software for the Deep Underground Neutrino Experiment (DUNE). The detector features a 1.15m x 1m anode plane following the specifications of the DUNE horizontal drift far detector and a newly installed X-ARAPUCA photodetector. The status of ICEBERG is reported along with analysis of noise, pulser, and cosmic ray data from the ninth run beginning May 2024 with the goal of advising the DUNE collaboration on the optimal wire readout electronics configuration. In addition, development of an absolute energy scale calibration method is currently underway using known sources such as cosmic ray muon Michel electrons at the ~10 MeV scale and $^{39}$Ar decay electrons at the ~100keV scale. Research into AI-based identification of such events at the data acquisition level is introduced.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Ghidra Galore

SAND2025-00232O Ghidra Galore is a software tool that provides a test harness for running an analysis script in multiple versions of Ghidra, an open-source binary disassembler, using Docker. It allows users to understand how different versions of Ghidra evolve over time by analyzing a corpus of input files. The software includes a Python script that wraps the Docker functionality, simplifying the process of running the analysis script across different Ghidra versions. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Crussell, Jonathan [Sandia National Lab. (SNL-CA),↗
DOE OSTI2024

Software Quality Assurance Plan ANSYS LSDYNA Version 2023R1

ANSYS Inc. develops and markets engineering simulation software and services used in the aerospace, automotive, manufacturing, electronics, biomedical, energy, defense, and many other industries. ANSYS is dedicated to engineering simulation and is the world’s leading software provider. ANSYS was founded in 1970 and is headquartered in Canonsburg, Pennsylvania. ANSYS provides an engineering analysis tool combining structural, thermal, computational fluid dynamics, acoustic and electromagnetic simulation capabilities. ANSYS LS-DYNA is the most used explicit simulation program capable of simulating the response of materials to short periods of severe loading. Its many elements, contact formulations, material models, and other controls can be used to simulate complex models with control over all the details of the problem. ANSYS LS-DYNA has a vast array of capabilities to simulate extreme deformation problems using its explicit solver. Engineers can tackle simulations involving material failure and look at how the failure progresses through a part or through a system. Models with large amounts of parts or surfaces interacting with each other are also easily handled, and the interactions and load passing between complex behaviors are modeled accurately. Using computers with higher numbers of CPU cores can drastically reduce solution times. In addition, many consulting firms and hundreds of universities use ANSYS for analysis, research, and educational purposes. ANSYS is recognized worldwide as one of the most widely used and capable programs of its type. ANSYS has successfully passed over 100 customer quality system audits against American Society of Mechanical Engineers (ASME) NQA-1 and 10 CFR Part 50, Appendix B, since the company was founded, over 60 of which have been since 1997. ANSYS has successfully passed over 100 International Organization for Standardization (ISO) 9001 assessments. ANSYS design analysis software is the first created within a quality system with ISO 9001 certification, which is the internationally accepted quality standard. Product development, testing, maintenance, and support processes also meet the US Nuclear Regulatory Commission’s (NRC’s) quality requirements, as they have for nearly four decades. ANSYS staff perform more than 60,000 software verification tests before releasing each new product. ASME NQA-1-2012 (Subpart 2.7 is specific to software) is the industry- and NRC-accepted approach (consensus standard) for meeting 10 CFR Part 50, Appendix B, requirements.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

MaPSA Quality Control and AI-Enhanced Grading For the CMS Phase-II Tracker Upgrade

The Compact Muon Solenoid (CMS) experiment will undergo changes as part of the Large Hadron Collider upgrade. The CMS tracker will be upgraded to cope with the new radiation environment and to provide tracking at the first level trigger. This upgrade features a new type of silicon module called PS Module, which combines a Pixel sensor and a Strip sensor in the same module. The pixel portion of the PS module has a sensor bump bonded to 16 Macro Pixel ASICs (MPA) to form a Macro Pixel Sub Assembly (MaPSA). At Fermilab, MaPSAs are tested for quality control before being assembled with the strip sensors, readout and service electronics to form a PS Module. All of this test data is stored in a centralized database, and is used to grade the final module to determine if it will be installed in the detector. The Phase II Outer Tracker Analyzer of Test Outputs (POTATO) is the software that processes this data and determines the module grades. Using recent technologies, an AI agent is being im plemented into POTATO in order to allow users to more efficiently sort through the large amounts of analysis data and ensure that only the user specified data is being considered. This poster will display the process of testing a MaPSA, how that test data is relevant to module assembly and grading, and how the POTATO grading tool is being improved with the use of an embedded AI agent.

Gzamouranis, Olivia [Purdue U.]↗
DOE OSTI2024

Verification of the EvaluateFlux Utility Program

The EvaluateFlux program is a post-processing utility program for DIF3D, specifically DIF3D-VARIANT which handles Cartesian and hexagonal geometries. The EvaluateFlux program was developed to allow users to obtain flux and power traverses through the geometry domain, and its initial purpose was to facilitate foil analysis by evaluating the flux solution from DIF3D-VARIANT and combining it with foil cross section data. The EvaluateFlux program can calculate the neutron flux, as well as the reaction rates, at any user provided evaluation point. It does this by identifying the spatial mesh associated with the evaluation point and then evaluates the polynomial based neutron flux moments stored in the NHFLUX file at that point. The output of EvaluateFlux varies depending on the input setup. The maximum output includes the neutron flux and microscopic and macroscopic reaction rates at each evaluation point. The purpose of this work is to verify the outputs of EvaluateFlux. Simple models that have hand calculatable results are first defined and used to verify the EvaluateFlux outputs. More complex cases are then added where a duplicate program of EvaluateFlux that uses PrintTables outputs of the binary files is used to verify the EvaluateFlux outputs. In those complex cases, hand calculations of selected evaluation points were also displayed to confirm the software verification. For all the tests done, the hand calculations agreed well with those calculated by EvaluateFlux. For the larger complex problems, the duplicate program that can process hundreds of evaluation points was able to identify that zero points within some meshes have large errors. This aspect was attributed to the truncation error on the input provided to the duplicate program and is not a concern for the accuracy of the EvaluateFlux software.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Maximizing Efficiency and Quality: Leveraging Automated Testing for Laboratory Commissioning

The traditional commissioning process uses sampling to select equipment for functional acceptance testing when large quantities of equipment are present. Although this approach is generally effective in identifying wide-spread issues, it has several shortcomings: it fails to evaluate equipment not included in the sample, provides only a one-time validation of equipment operation, and the standard documentation is a simple checklist of pass/fail questions. During the construction and commissioning process of the new Research and Innovation Laboratory (RAIL) in Golden, CO, the National Renewable Energy Laboratory team engaged Group14 Engineering to implement a Connected Commissioning process using fault detection and diagnostic software for automated functional acceptance testing. This presentation highlights the advantages offered by automated functional testing in this critical laboratory setting: (1) sampling 100% of BAS-connected equipment during functional testing, (2) testing results backed by data beyond the traditional pass/fail checklist, and (3) an automated test process that can be regularly executed by the building management team for ongoing commissioning throughout the life of the building. The presentation will also cover technical challenges associated with Connected Commissioning and the important conversations with key stakeholders that need to occur well before functional acceptance testing in order to successfully implement the automated testing processes.

automated testing↗
DOE OSTI2025

An Open-Source Framework for Rapid Validation of Scientific ASICs

Spacely is an open-source framework for the post-silicon validation of analog, digital, and mixed-signal ASICs (Application-Specific Integrated Circuits) which maximizes the reuse of hardware and software, reducing the time taken to achieve meaningful test results. Spacely specifically addresses the needs of small, flexible ASIC design teams commonly found in academia or research institutions which benefit most from sharing the overhead of test stand creation between many unique ASIC designs. Spacely is a set of software, firmware, and design practices. It targets two primary hardware platforms (NI-PXI and Caribou) as well as offering extensible support for bench instruments. Spacely provides a high-level Python interface to all test hardware for accessibility, while also giving more sophisticated teams the opportunity to integrate custom test firmware. The design principles of Spacely are presented in brief. Current documentation is available at https://github.com/SpacelyProject/spacely-docs.

Quinn, Adam [Fermilab]↗
DOE OSTI2026

Investigating Temperature Uniformity and Accuracy in PV Module Lamination: A Verification Study

This study investigates the temperature uniformity and accuracy of a photovoltaic (PV) module lamination process by addressing inconsistencies identified in 2017 data where irregular temperature changes were observed across setpoints. The 2017 data showed a notable drop in temperature upon bladder initiation, except for the 145 degrees Celsius profile. This inconsistency indicated potential inaccuracies in manual data recording methods. To address this concern, a verification experiment was conducted to evaluate temperature uniformity across the 2014 Bent River SPL2828 laminator platen and within test samples. Thermocouples, paired with Omega data acquisition software, were deployed to measure temperatures at multiple platen locations and within test samples. The experiment compared lamination temperatures of polyethylene-co-vinyl acetate (EVA) encapsulant when paired with solite glass or TPE backsheets. The methodology included verifying temperature uniformity directly on the platen and by using a large glass/EVA/glass sample using multiple thermocouples. Smaller samples were built with glass/EVA/glass and glass/EVA/backsheet configurations with one centered thermocouple to verify and compare sample temperatures. This verification aims to refine lamination temperature profiles, enhance data accuracy and provide insights into optimal process control for uniform module lamination. Ensuring consistent and uniform lamination may improve the accuracy and reliability of research outcomes.

14 SOLAR ENERGY↗
DOE OSTI2025

ON THE EFFECTIVENESS OF LLMS IN UNIT TEST GENERATION FOR STRUCTURED TEXT PROGRAMS

The reliability of industrial automation systems heavily depends on the correctness of Programmable Logic Controller (PLC) programs, which are often written in Structured Text (ST). While Large Language Models (LLMs) have shown promise in automating test generation for mainstream programming languages, their effectiveness for the syntactically strict ST language remains underexplored. This thesis presents a systematic empirical evaluation of three state-of-the-art LLMs—GPT-4o, Gemini 2.5 Pro, and Claude Sonnet 4.5—for generating ST unit tests. We examine three prompting strategies: Natural Language (NL), Code Language (CL), and Chain-of-Thought (CoT), across a curated set of 11 ST function blocks. The quality of the generated tests is assessed using Compilation Success Rate (CSR), Statement Coverage (SC), and Branch Coverage (BC). In the zero-shot setting, Claude Sonnet 4.5 achieves the highest CSR, while Gemini 2.5 Pro consistently delivers the best statement and branch coverage, particularly under CL prompts. By incorporating a one-shot CL prompt, all models exhibit substantial improvements—most notably GPT-4o, whose CSR increases from 45.45% to 90.91%, with substantial gains in both SC and BC. To further contextualize these findings, we compare GPT-4o’s one-shot results with PLCAutoTester, a state-ofthe- art ST unit test generation tool, on an additional benchmark dataset. While LLMgenerated tests approach competitive coverage levels, PLCAutoTester maintains significantly higher and more stable coverage across programs. This study provides the first comprehensive benchmark of modern LLMs for ST unit testing, highlighting their strengths, limitations, and improvements through one-shot prompting, and positioning their performance relative to specialized automated testing tools in industrial automation.

42 ENGINEERING↗
DOE OSTI2025

Exploring the Nature of Neutrinos with the Deep Underground Neutrino Experiment (DUNE)

The Deep Underground Neutrino Experiment (DUNE) is a next-generation experimental program designed to study the behaviour of neutrino oscillation. DUNE will utilize a neutrino beam originating at Fermilab, near Chicago, and will leverage a detector at Fermilab (Near Detector) and a detector 1300 km away in South Dakota (Far Detector), south of Saskatchewan. In the first phase of DUNE, its Far Detector will comprise of two 10,000 ton (fiducial) liquid argon (LAr) time-projection chamber (TPC) modules – powerful tracking calorimeter detectors – placed nearly a mile underground. With this large, sensitive, underground detector, DUNE aims to collect a high statistics and pure sample of neutrinos at the Far Detector. This setup also offers the potential to study non-beam physical processes via e.g. neutrinos produced in the atmosphere, supernova neutrino bursts, and/or solar neutrinos, etc. A second phase will aim to add more detector mass and expand the program. The Near Detector will consist of a LAr TPC module as well: critical to constraining systematic uncertainties in the oscillation analysis. However, this LAr TPC will have a novel design using a pixel-based readout instead of the traditional wire-based readout. This and the segmentation of the LAr TPC into multiple units are crucial in mitigating the high multiplicity of neutrino interactions expected in any readout window given its proximity to the beam. The Near Detector will feature additional components and capability beyond the LAr TPC, allowing one to deeply characterize the neutrino flux. Due to the complexity of this experimental program, several smaller-scale prototype detectors have been operating to test, validate, and improve both the technical designs and software for processing and analyzing events. By operating in charged particle test beams or neutrino beams, several of the prototypes are also capable of producing valuable results. Canadian institutions are involved in the realization of the DUNE through efforts with both the Near and Far Detectors and prototypes. DUNE is anticipated to begin operating near the end of this decade/the beginning of the next. This talk will focus on the overall DUNE program, for example its ultimate plans, status, and the efforts with prototypes.

Howard, Bruce [York U., Canada; Fermilab]↗
DOE OSTI2024

DEReliction: A Cybersecurity Vulnerability Assessment Methodology for Distributed Energy Resources

With the increasing integration of Distributed Energy Resources (DER) into the electric grid, maintaining grid reliability and resilience requires that these devices remain secure. This paper discusses a cybersecurity vulnerability assessment methodology that incorporates best practices from Sandia National Laboratories, SANS Institute, OWASP Foundation, and other web and Internet of Things (IoT) penetration testing (“pen testing”) programs, courses, and frameworks for assessing the security posture of devices. The methodology involves five sequential steps: (1) Collect Public Information, (2) Extract Hardware Details, (3) Inventory Software Components, (4) Identify Vulnerabilities, and (5) Test Vulnerabilities. Each step uncovers potential weaknesses in both hardware and software components of DER devices, considering adversary tactics, techniques, and procedures (TTPs), and potential attack vectors along the way. The results from the execution of this method on multiple residential- and small commercial-scale photovoltaic (PV) inverters reveled hardware and software vulnerabilities, which highlight the benefit of taking a methodical approach to discover vulnerabilities. While the specific vulnerability details are not shared here, a generalized overview of findings underscore the importance of robust security assessments for DER devices. Adoption of an assessment framework of this kind will identify and mitigate cybersecurity threats and bolster the resilience of DER-integrated electric grids.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Throughput Estimation of Data Transport Networks From Digital Twin Measurements

Digital twins of networked infrastructures, known as Virtual Infrastructure Twins (VITs), are increasingly used for software development, pre-deployment testing, and design space exploration. While VITs avoid the costs and potential disruptions associated with experiments on operational networks, their throughput measurements are typically not sufficiently accurate for performance profiling of wide-area networks that they emulate. Here, machine learning (ML) methods are developed to transform these inaccurate VIT network throughput measurements to closely match in peak and overall profile of those from a physical testbed or production network. First, a micro kernel network reflecting a physical network is utilized to collect one-time measurements on a host to support this ML transformation. Then, a generic multi-modal ML method is developed to learn a map that transforms measurements from subsequent VITs on the same host to match past, current and follow-on testbed and cloud networks. ML generalization equations are derived to establish its correctness and probabilistically guarantee its generalization accuracy. Experimental results are presented for a variety of VIT hosts with target testbed and cloud networks; they include a case study of a four-site science ecosystem wherein inaccurate convex VIT measurement profiles are transformed into accurate concave profiles of target networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Software-Defined Data Center Network Architecture using VXLAN-based BGP EVPN for Dynamic Workflows in a Supercomputing Environment (VXLAN-based BGP EVPN Fabric for HPC) v1

This software repository automates the deployment of a multi-vendor VXLAN-based BGP EVPN architecture, leveraging Containerlab to instantiate a stretched CLOS topology. It integrates Linux, Nokia SR Linux, and Arista cEOS, using BGP for underlay, overlay, and topology extension. The software enables rapid prototyping and testing of advanced network configurations. Its key advantage lies in providing a dynamic, programmable environment for research and development of critical technologies supporting dynamic workflows within supercomputing environments, surpassing the limitations of static, vendor-locked alternatives by fostering interoperability and agility.

Kumar, Ronal [Lawrence Berkeley National Laborator↗
DOE OSTI2024

Wholesale Electricity Analysis via Simulation & Learning Experiments (WEASLE): Platform Development and Pilot Competition

This document reports the development of the Wholesale Electricity Analysis via Simulation and Learning Experiments (WEASLE) platform and the pilot competition that was conducted to test the platform. Due to the increasing reliance on variable renewable energy resources for bulk power, the pilot competition, called the Energy Storage Participation Algorithm Competition (ESPA-Comp), was used to test the effect of various market designs on storage utilization and market efficiency. Basic details of the platform are provided, including an overview of the market clearing engine, the battery dispatch and degradation models, electric grid topology and resource mix, and software architecture. Two market designs were tested: a two-settlement market analogous to typical ISO design today, and a multi-settlement market that allows additional forward-trading periods during the real-time market. Results from the pilot competition show that the storage bidding problem is nontrivial and is well suited for future challenges. We find that: 1) all four teams utilized different approaches to the bidding problem, 2) different methodological approaches led to substantially different offer behaviors, 3) resource profits are clustered by team and methodological approach, 4) simulated offers reduced market surplus by about 0.5%, 5) substantially different prices between two-settlement and multi-settlement markets albeit minimal difference in overall market surplus.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Fortran Compiler Test Suite v0.1.0

There is no currently available, open-source, easily adapted test suite for checking a compiler's conformance to the Fortran Standard. The main innovation of this software is in the framework to make the test suite apply to a new compiler. Making it open-source will enable community contributions of the test cases. Single organization construction of a comprehensive test suite is something that would be financially infeasible.

Richardson, Bradley↗