Search NASASearch

SEARCH · Search NASA

Results for “regression testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Multiversion software reliability through fault-avoidance and fault-tolerance

In this project we have proposed to investigate a number of experimental and theoretical issues associated with the practical use of multi-version software in providing dependable software through fault-avoidance and fault-elimination, as well as run-time tolerance of software faults. In the period reported here we have working on the following: We have continued collection of data on the relationships between software faults and reliability, and the coverage provided by the testing process as measured by different metrics (including data flow metrics). We continued work on software reliability estimation methods based on non-random sampling, and the relationship between software reliability and code coverage provided through testing. We have continued studying back-to-back testing as an efficient mechanism for removal of uncorrelated faults, and common-cause faults of variable span. We have also been studying back-to-back testing as a tool for improvement of the software change process, including regression testing. We continued investigating existing, and worked on formulation of new fault-tolerance models. In particular, we have partly finished evaluation of Consensus Voting in the presence of correlated failures, and are in the process of finishing evaluation of Consensus Recovery Block (CRB) under failure correlation. We find both approaches far superior to commonly employed fixed agreement number voting (usually majority voting). We have also finished a cost analysis of the CRB approach.

Vouk, Mladen A.

A Vehicle Management End-to-End Testing and Analysis Platform for Validation of Mission and Fault Management Algorithms to Reduce Risk for NASAs Space Launch System

The engineering development of the National Aeronautics and Space Administration's (NASA) new Space Launch System (SLS) requires cross discipline teams with extensive knowledge of launch vehicle subsystems, information theory, and autonomous algorithms dealing with all operations from pre-launch through on orbit operations. The nominal and off-nominal characteristics of SLS's elements and subsystems must be understood and matched with the autonomous algorithm monitoring and mitigation capabilities for accurate control and response to abnormal conditions throughout all vehicle mission flight phases, including precipitating safing actions and crew aborts. This presents a large and complex systems engineering challenge, which is being addressed in part by focusing on the specific subsystems involved in the handling of off-nominal mission and fault tolerance with response management. Using traditional model-based system and software engineering design principles from the Unified Modeling Language (UML) and Systems Modeling Language (SysML), the Mission and Fault Management (M&FM) algorithms for the vehicle are crafted and vetted in Integrated Development Teams (IDTs) composed of multiple development disciplines such as Systems Engineering (SE), Flight Software (FSW), Safety and Mission Assurance (S&MA) and the major subsystems and vehicle elements such as Main Propulsion Systems (MPS), boosters, avionics, Guidance, Navigation, and Control (GNC), Thrust Vector Control (TVC), and liquid engines. These model-based algorithms and their development lifecycle from inception through FSW certification are an important focus of SLS's development effort to further ensure reliable detection and response to off-nominal vehicle states during all phases of vehicle operation from pre-launch through end of flight. To test and validate these M&FM algorithms a dedicated test-bed was developed for full Vehicle Management End-to-End Testing (VMET). For addressing fault management (FM) early in the development lifecycle for the SLS program, NASA formed the M&FM team as part of the Integrated Systems Health Management and Automation Branch under the Spacecraft Vehicle Systems Department at the Marshall Space Flight Center (MSFC). To support the development of the FM algorithms, the VMET developed by the M&FM team provides the ability to integrate the algorithms, perform test cases, and integrate vendor-supplied physics-based launch vehicle (LV) subsystem models. Additionally, the team has developed processes for implementing and validating the M&FM algorithms for concept validation and risk reduction. The flexibility of the VMET capabilities enables thorough testing of the M&FM algorithms by providing configurable suites of both nominal and off-nominal test cases to validate the developed algorithms utilizing actual subsystem models such as MPS, GNC, and others. One of the principal functions of VMET is to validate the M&FM algorithms and substantiate them with performance baselines for each of the target vehicle subsystems in an independent platform exterior to the flight software test and validation processes. In any software development process there is inherent risk in the interpretation and implementation of concepts from requirements and test cases into flight software compounded with potential human errors throughout the development and regression testing lifecycle. Risk reduction is addressed by the M&FM group but in particular by the Analysis Team working with other organizations such as S&MA, Structures and Environments, GNC, Orion, Crew Office, Flight Operations, and Ground Operations by assessing performance of the M&FM algorithms in terms of their ability to reduce Loss of Mission (LOM) and Loss of Crew (LOC) probabilities. In addition, through state machine and diagnostic modeling, analysis efforts investigate a broader suite of failure effects and associated detection and responses to be tested in VMET to ensure reliable failure detection, and confirm responses do not create additional risks or cause undesired states through interactive dynamic effects with other algorithms and systems. VMET further contributes to risk reduction by prototyping and exercising the M&FM algorithms early in their implementation and without any inherent hindrances such as meeting FSW processor scheduling constraints due to their target platform - the ARINC 6535-partitioned Operating System, resource limitations, and other factors related to integration with other subsystems not directly involved with M&FM such as telemetry packing and processing. The baseline plan for use of VMET encompasses testing the original M&FM algorithms coded in the same C++ language and state machine architectural concepts as that used by FSW. This enables the development of performance standards and test cases to characterize the M&FM algorithms and sets a benchmark from which to measure their effectiveness and performance in the exterior FSW development and test processes. This paper is outlined in a systematic fashion analogous to a lifecycle process flow for engineering development of algorithms into software and testing. Section I describes the NASA SLS M&FM context, presenting the current infrastructure, leading principles, methods, and participants. Section II defines the testing philosophy of the M&FM algorithms as related to VMET followed by section III, which presents the modeling methods of the algorithms to be tested and validated in VMET. Its details are then further presented in section IV followed by Section V presenting integration, test status, and state analysis. Finally, section VI addresses the summary and forward directions followed by the appendices presenting relevant information on terminology and documentation.

Trevino, Luis

Multi deep learning-based stochastic microstructure reconstruction and high-fidelity micromechanics simulation of time-dependent ceramic matrix composite response

A multi deep learning-based framework is developed for efficient, automated microstructure reconstruction and generation of stochastic representative volume elements (SRVEs) with periodic boundary conditions (PBCs) for accurate modeling of ceramic matrix composite (CMC) response. The methodology comprises a convolutional neural network coupled with regression layers to act as a vanilla regression network for semantic segmentation of the microstructure, allowing accurate characterization of the phases and their distributions at the microscale. Scanning electron microscope and confocal microscope are used to obtain C/SiNC and SiC/SiNC CMCs micrographs for vanilla regression testing. Microstructure variability in terms of fiber volume fraction and porosity are quantified through the output regression layer, ensuring accurate representation of material variability in SRVE construction. Generative adversarial network (GAN) and its variants are designed to produce high-fidelity SRVE, spanning CMCs microstructure variability space. A circular padding algorithm is developed to generate SRVEs with PBCs during training of GANs. The accuracy of the generated SRVEs is established through micromechanics simulations, where an efficient formulation of the high-fidelity generalized methods of cells (HFGMC) approach is used to compute the effective mechanical properties. Furthermore, an iterative algorithm is implemented in the HFGMC solver to simulate time-dependent deformation of SiC/SiNC subjected to creep loading conditions.

36 MATERIALS SCIENCE

Application of software technology to automatic test data analysis

The verification process for a major software subsystem was partially automated as part of a feasibility demonstration. The methods employed are generally useful and applicable to other types of subsystems. The effort resulted in substantial savings in test engineer analysis time and offers a method for inclusion of automatic verification as a part of regression testing.

Stagner, J. R.

Optimal Estimation-Based Algorithm to Retrieve Aerosol Optical Properties for GEMS Measurements Over Asia

The Geostationary Environment Monitoring Spectrometer (GEMS) is scheduled to be in orbit in 2019 onboard the GEO-KOMPSAT 2B satellite and will continuously monitor air quality over Asia. The GEMS will make measurements in the UV spectrum (300-500 nm) with 0.6 nm resolution. In this study, an algorithm is developed to retrieve aerosol optical properties from UV-visible measurements for the future satellite instrument and is tested using 3 years of existing OMI L1B data. This algorithm provides aerosol optical depth (AOD), single scattering albedo (SSA) and aerosol layer height (ALH) using an optimized estimation method. The retrieved AOD shows good correlation with Aerosol Robotic Network (AERONET) AOD with correlation coefficients of 0.83, 0.73 and 0.80 for heavy-absorbing fine (HAF) particles, dust and non-absorbing (NA) particles, respectively. However, regression tests indicate underestimation and overestimation of HAF and NA AOD, respectively. In comparison with AOD from the OMI/Aura Near-UV Aerosol Optical Depth and Single Scattering Albedo 1-orbit L2 Swath 13 km x 24 km V003 (OMAERUV) algorithm, the retrieved AOD has a correlation coefficient of 0.86 and linear regression equation, AOD(sub GEMS) = 1.18AOD(sub OMAERUV) + 0.09. An uncertainty test based on a reference method, which estimates retrieval error by applying the algorithm to simulated radiance data, revealed that assumptions in the spectral dependency of aerosol absorptivity in the UV cause significant errors in aerosol property retrieval, particularly the SSA retrieval. Consequently, retrieved SSAs did not show good correlation with AERONET values. The ALH results were qualitatively compared with the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) products and were found to be well correlated for highly absorbing aerosols. The difference between the attenuated-backscatter-weighted height from CALIOP and retrieved ALH were mostly closed to zero when the retrieved AOD is higher than 0.8 and SSA is lower than 0.93. Although retrieval accuracy was not significantly improved, the simultaneous consistent retrieval of AOD, SSA and ALH alone demonstrates the value of this stand-alone algorithm, given their nature for error using other methods. The use of these properties as input parameters for the air mass factor calculation is expected to improve the retrieval of other trace gases over Asia.

Geostationary Environment Monitoring Spectrometer

Testing-Based Compiler Validation for Synchronous Languages

In this paper we present a novel lightweight approach to validate compilers for synchronous languages. Instead of verifying a compiler for all input programs or providing a fixed suite of regression tests, we extend the compiler to generate a test-suite with high behavioral coverage and geared towards discovery of faults for every compiled artifact. We have implemented and evaluated our approach using a compiler from Lustre to C.

Compiler Validation

Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning

Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates.As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.

Verification and Validation

A Change Impact Analysis to Characterize Evolving Program Behaviors

Change impact analysis techniques estimate the potential effects of changes made to software. Directed Incremental Symbolic Execution (DiSE) is an intraprocedural technique for characterizing the impact of software changes on program behaviors. DiSE first estimates the impact of the changes on the source code using program slicing techniques, and then uses the impact sets to guide symbolic execution to generate path conditions that characterize impacted program behaviors. DiSE, however, cannot reason about the flow of impact between methods and will fail to generate path conditions for certain impacted program behaviors. In this work, we present iDiSE, an extension to DiSE that performs an interprocedural analysis. iDiSE combines static and dynamic calling context information to efficiently generate impacted program behaviors across calling contexts. Information about impacted program behaviors is useful for testing, verification, and debugging of evolving programs. We present a case-study of our implementation of the iDiSE algorithm to demonstrate its efficiency at computing impacted program behaviors. Traditional notions of coverage are insufficient for characterizing the testing efforts used to validate evolving program behaviors because they do not take into account the impact of changes to the code. In this work we present novel definitions of impacted coverage metrics that are useful for evaluating the testing effort required to test evolving programs. We then describe how the notions of impacted coverage can be used to configure techniques such as DiSE and iDiSE in order to support regression testing related tasks. We also discuss how DiSE and iDiSE can be configured for debugging finding the root cause of errors introduced by changes made to the code. In our empirical evaluation we demonstrate that the configurations of DiSE and iDiSE can be used to support various software maintenance tasks

Rungta, Neha Shyam

Code Coverage Status of the ARC Code DIF3D

The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. DIF3D plays a pivotal role in the ARC system as the primary homogenized assembly neutronic calculation methodology for fast reactor problems. Over its 40 years history, ARC software usage with DIF3D has been applied to numerous fast and thermal spectrum reactor analysis projects with good to excellent comparison against experiments. With continued improvement of computation resources, many of the geometry modeling capabilities in DIF3D that were primarily used in low order schemes are not really needed anymore. Today, the diffusion and transport capabilities of DIF3D-VARIANT are primarily used in the reactor design process with some scattered usage of DIF3D-FD and DIF3D-Nodal. In recent work, the DIF3D software verification was completed for DIF3D-FD and DIF3D-VARIANT on the geometry options used in the Versatile Test Reactor project. While we can be confident that these capabilities of DIF3D are well used and thus trusted, it does not demonstrate that all possible input options of DIF3D are actually working, but just those that were tested as part of VTR are and that they are correct. Thus, the purpose of the present work is to identify a set of test problems for DIF3D and assess the code coverage of DIF3D for those test problems. The goal is to document what parts of the existing DIF3D code are touched by the set of test problems and which are not. Because the verification work done on DIF3D-VARIANT and DIF3D-FD was focused on the most common uses of DIF3D for fast reactor analysis, the code coverage assessment of those capabilities is the highest priority. This will ensure that nothing is being missed by the existing verification test problems that DIF3D relies upon. The DIF3D-Nodal capability will also be inspected for code coverage as part of this work to further ensure that regular regression testing of DIF3D will trap any likely errors the end user might experience with the DIF3D software. The code coverage analysis of DIF3D was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of DIF3D. The detailed coverage tables are given for each submodule of DIF3D separately, and for the submodules which are primarily developed for DIF3D, most of the source files could be at least partially touched. Most of the uncovered parts/files could be easily ignored, because they are either for error message and debugging output or obviously not needed by DIF3D. Out of the entire source codes of DIF3D, only a few uncovered modules deserve further investigation.

22 GENERAL STUDIES OF NUCLEAR REACTORS

Functional Equivalence Acceptance Testing of FUN3D for Entry Descent and Landing Applications

The functional equivalence of the unstructured grid code FUN3D to the the structured grid code LAURA (Langley Aerothermodynamic Upwind Relaxation Algorithm) is documented for applications of interest to the Entry, Descent, and Landing (EDL) community. Examples from an existing suite of regression tests are used to demonstrate the functional equivalence, encompassing various thermochemical models and vehicle configurations. Algorithm modifications required for the node-based unstructured grid code (FUN3D) to reproduce functionality of the cell-centered structured code (LAURA) are also documented. Challenges associated with computation on tetrahedral grids versus computation on structured-grid derived hexahedral systems are discussed.

Gnoffo, Peter A.

Differential Adaptive Stress Testing of Airborne Collision Avoidance Systems

The next-generation Airborne Collision Avoidance System (ACAS X) is currently being developed and tested to replace the Traffic Alert and Collision Avoidance System (TCAS) as the next international standard for collision avoidance. To validate the safety of the system, stress testing in simulation is one of several approaches for analyzing near mid-air collisions (NMACs). Understanding how NMACs can occur is important for characterizing risk and informingdevelopment of the system. Recently, adaptive stress testing (AST) has been proposed as a way to find the most likely path to a failure event. The simulation-based approach accelerates search by formulating stress testing as a sequential decision process then optimizing it using reinforcement learning. The approach has been successfully applied to stress test a prototype of ACAS Xin various simulated aircraft encounters. In some applications, we are not as interestedin the system's absolute performance as its performance relative to another system. Such situations arise, for example, during regression testing or when deciding whether a new system should replace an existing system. In our collision avoidance application, we are interested in finding cases where ACAS X fails but TCAS succeeds in resolving a conflict. Existing approaches do not provide an efficient means to perform this type of analysis. This paper extends the AST approach to differential analysis by searching two simulators simultaneously and maximizing the difference between their outcomes. We call this approach differential adaptive stress testing (DAST). We apply DAST to compare a prototype of ACAS X against TCAS and show examples of encounters found by the algorithm.

Lee, Ritchie

Fiscal Year 2025 Software Quality Assurance Activities for the ARC Software

The continued goal of the ARC SQA project in the Advanced Reactor Technologies program of DOE is to resolve the QA gaps for the ARC software that limit, or prevent, commercialization of the software for industry users. This project started in earnest in fiscal year 2023 which saw the entire code system moved from a SVN repository to a GitLab repository and an associated software quality assurance plan (SQAP) developed and ratified. Most of the QA gaps in the ARC software were identified in collaboration with industry partners and work begin in fiscal year 2023 and continued through 2024 and 2025. The continuous integration testing was extended to RCT, DASSH, and SE2ANL. Minor changes were required to the original continuous integration methodology to make this happen. When full confidence in the methodology is complete, a report will be created to detail the automated regression testing methodology and minor reports will be created to detail the tolerance settings that have been applied to the output for each ARC code. The primary documentation that is missing includes user manuals, user guides, software verification reports, and code coverage assessments. The DASSH, SE2ANL, and SE2RCT manuals were completed this fiscal year. A review of the SE2ANL software identified that it is unrealistic to include updated correlations or different geometry models and it was scheduled for deprecation in favor of DASSH. The SE2ANL manual is essential for SE2RCT as they are similar but quite different in purpose. The only piece of software missing a manual consistent with the source code is NUBOW-3D which is a focus of the coming year. The code coverage report for DIF3D was updated and code coverage reports were created for REBUS, RCT, PERSENT, GAMSRC, and DASSH. Minor coverage issues were identified for all of these pieces of software which did not prevent the work done to transition them to the OneAPI compiler. Because SE2ANL was scheduled for deprecation, it was not transitioned, but it was successfully tested with the OneAPI compiler. This leaves SE2RCT and NUBOW-3D as the only pieces of software not transitioned to OneAPI and further work is required to get SE2RCT to work properly. The SE2RCT software transition will begin early next year while the NUBOW-3D software requires a manual before it can begin. Software verification work has been completed for DIF3D, REBUS, GAMSOR, GAMSRC, VARPOW, EvaluateFlux, and SUMMAR. The PERSENT software verification work was completed this year which was somewhat delayed because of unexpected bugs in the software. The PERSENT manual was updated to detail some of the issues and discuss the bowing reactivity worth feature added in the previous fiscal year. The RCT, DASSH, SE2RCT, and NUBOW-3D software are the only maintained pieces of software without verification reports. The software verification work for DASSH will be a focus in the upcoming fiscal year and it is hoped that some of the test cases created can serve as verification tests for SE2RCT. The NUBOW-3D work will begin when the manual and requirements report are completed. Only minor industry partner software development funds were provided this year. The DASSH software was updated to handle general axial geometry for each assembly and the NUBOW-3D software was updated to incorporate a new input format and better output. Overall progress on resolving the QA gaps has been good this year.

22 GENERAL STUDIES OF NUCLEAR REACTORS

Fiscal Year 2025 Software Quality Assurance Activities for the ARC Software

The continued goal of the ARC SQA project in the Advanced Reactor Technologies program of DOE is to resolve the QA gaps for the ARC software that limit, or prevent, commercialization of the software for industry users. This project started in earnest in fiscal year 2023 which saw the entire code system moved from a SVN repository to a GitLab repository and an associated software quality assurance plan (SQAP) developed and ratified. Most of the QA gaps in the ARC software were identified in collaboration with industry partners and work begin in fiscal year 2023 and continued through 2024 and 2025. The continuous integration testing was extended to RCT, DASSH, and SE2ANL. Minor changes were required to the original continuous integration methodology to make this happen. When full confidence in the methodology is complete, a report will be created to detail the automated regression testing methodology and minor reports will be created to detail the tolerance settings that have been applied to the output for each ARC code. The primary documentation that is missing includes user manuals, user guides, software verification reports, and code coverage assessments. The DASSH, SE2ANL, and SE2RCT manuals were completed this fiscal year. A review of the SE2ANL software identified that it is unrealistic to include updated correlations or different geometry models and it was scheduled for deprecation in favor of DASSH. The SE2ANL manual is essential for SE2RCT as they are similar but quite different in purpose. The only piece of software missing a manual consistent with the source code is NUBOW-3D which is a focus of the coming year. The code coverage report for DIF3D was updated and code coverage reports were created for REBUS, RCT, PERSENT, GAMSRC, and DASSH. Minor coverage issues were identified for all of these pieces of software which did not prevent the work done to transition them to the OneAPI compiler. Because SE2ANL was scheduled for deprecation, it was not transitioned, but it was successfully tested with the OneAPI compiler. This leaves SE2RCT and NUBOW-3D as the only pieces of software not transitioned to OneAPI and further work is required to get SE2RCT to work properly. The SE2RCT software transition will begin early next year while the NUBOW-3D software requires a manual before it can begin. Software verification work has been completed for DIF3D, REBUS, GAMSOR, GAMSRC, VARPOW, EvaluateFlux, and SUMMAR. The PERSENT software verification work was completed this year which was somewhat delayed because of unexpected bugs in the software. The PERSENT manual was updated to detail some of the issues and discuss the bowing reactivity worth feature added in the previous fiscal year. The RCT, DASSH, SE2RCT, and NUBOW-3D software are the only maintained pieces of software without verification reports. The software verification work for DASSH will be a focus in the upcoming fiscal year and it is hoped that some of the test cases created can serve as verification tests for SE2RCT. The NUBOW-3D work will begin when the manual and requirements report are completed. Only minor industry partner software development funds were provided this year. The DASSH software was updated to handle general axial geometry for each assembly and the NUBOW-3D software was updated to incorporate a new input format and better output. Overall progress on resolving the QA gaps has been good this year.

97 MATHEMATICS AND COMPUTING

Table-driven configuration and formatting of telemetry data in the Deep Space Network

With a restructured software architecture for telemetry system control and data processing, the NASA/Deep Space Network (DSN) has substantially improved its ability to accommodate a wide variety of spacecraft in an era of 'better, faster, cheaper'. In the new architecture, the permanent software implements all capabilities needed by any system user, and text tables specify how these capabilities are to be used for each spacecraft. Most changes can now be made rapidly, outside of the traditional software development cycle. The system can be updated to support a new spacecraft through table changes rather than software changes, reducing the implementation, test, and delivery cycle for such a change from three months to three weeks. The mechanical separation of the text table files from the program software, with tables only loaded into memory when that mission is being supported, dramatically reduces the level of regression testing required. The format of each table is a different compromise between ease of human interpretation, efficiency of computer interpretation, and flexibility.

Manning, Evan

ncompare: A Python Package for Comparing netCDF Structures

Earth science researchers and data engineers have a common problem: they often need to compare data files to see what is different between them. A lot of time is spent developing code to test differences. When it comes to comparing multidimensional data file formats like netCDFs (Network Common Data Form), this is particularly challenging and time-consuming, since there is frequently a need to evaluate the differences between dimension sizes, variable structures, and variable attributes, especially for regression testing. Since netCDFs are widely used in Earth science — with climate models, oceanographic or atmospheric reanalyses, and observational data — improved means of evaluating netCDF files can help enable a wide range of applications. We have developed a reusable open source approach through `ncompare`, which is a Python package for comparing netCDF structures [[https://github.com/nasa/ncompare]]. The `ncompare` tool compares the structure of two Network Common Data Form (NetCDF) files at the command line. It facilitates rapid comparisons by generating a formatted display of the matching and non-matching groups, variables, and associated metadata between two NetCDF datasets. The user has the option to colorize the terminal output for ease of viewing, and `ncompare` can optionally save comparison reports in text, comma-separated value (CSV), and/or Microsoft Excel formats. Despite the availability of tools (such as ncmpidiff or nccmp) that compare the values of variables, there was not previously a readily available, Python-based tool for rapid visual comparisons of group and variable structures, attributes, and chunking. `ncompare` was developed at NASA’s Atmospheric Science Data Center (ASDC) and is a collaboration with NASA Openscapes [[https://nasa-openscapes.github.io]] mentors across 11 of NASA’s data centers. Openscapes’ overarching vision is to support scientific researchers using NASA Earthdata as they migrate their workflows to the cloud. Relevant links: - https://github.com/nasa/ncompare - https://github.com/pyOpenSci/software-submission/issues/146 - https://nasa-openscapes.github.io

Daniel Kaufman

Software Testing and Verification in Climate Model Development

Over the past 30 years most climate models have grown from relatively simple representations of a few atmospheric processes to a complex multi-disciplinary system. Computer infrastructure over that period has gone from punch card mainframes to modem parallel clusters. Model implementations have become complex, brittle, and increasingly difficult to extend and maintain. Existing verification processes for model implementations rely almost exclusively upon some combination of detailed analysis of output from full climate simulations and system-level regression tests. In additional to being quite costly in terms of developer time and computing resources, these testing methodologies are limited in terms of the types of defects that can be detected, isolated and diagnosed. Mitigating these weaknesses of coarse-grained testing with finer-grained "unit" tests has been perceived as cumbersome and counter-productive. In the commercial software sector, recent advances in tools and methodology have led to a renaissance for systematic fine-grained testing. We discuss the availability of analogous tools for scientific software and examine benefits that similar testing methodologies could bring to climate modeling software. We describe the unique challenges faced when testing complex numerical algorithms and suggest techniques to minimize and/or eliminate the difficulties.

Clune, Thomas L.

MFC 5.0: An exascale many-physics flow solver

Many problems of interest in engineering, medicine, and the fundamental sciences rely on high-fidelity flow simulation, making performant computational fluid dynamics solvers a mainstay of the open-source software community. Previous work MFC 3.0 was made a published, documented, and open-source solver via Bryngelson et al. Comp. Phys. Comm. (2021) with numerous physical features, numerical methods, and scalable infrastructure. MFC 5.0 is a significant update to MFC 3.0, featuring a broad set of well-established and novel physical models and numerical methods, as well as the introduction of GPU and APU (or superchip) acceleration. Here, we exhibit state-of-the-art performance and ideal scaling on the first two exascale supercomputers, OLCF Frontier and LLNL El Capitan. Combined with MFC’s single-accelerator performance, MFC achieves exascale computation in practice, and achieved the largest-to-date public CFD simulation at 200 trillion grid points as a 2025 ACM Gordon Bell Prize finalist. New physical features include the immersed boundary method, N-fluid phase change, Euler–Euler and Euler–Lagrange sub-grid bubble models, fluid-structure interaction, hypo- and hyper-elastic materials, chemically reacting flow, two-material surface tension, magnetohydrodynamics (MHD), and more. Numerical techniques now represent the current state-of-the-art, including general relaxation characteristic boundary conditions, WENO variants, Strang splitting for stiff sub-grid flow features, and low Mach number treatments. Weak scaling to tens of thousands of GPUs on OLCF Summit and Frontier and LLNL El Capitan achieves efficiencies within 5% of ideal to over 90% of their respective system sizes. Strong scaling results for a 16-times increase in device count show parallel efficiencies over 90% on OLCF Frontier. MFC’s software stack has undergone further improvements, including continuous integration, which ensures code resilience and correctness through over 300 regression tests; metaprogramming, which reduces code length while maintaining performance portability; and code generation for computing chemical reactions

Computational fluid dynamics

NEML2: An efficient and modular multiphysics constitutive modeling library for hybrid computing environments

This paper presents NEML2, an open-source, high-performance library developed for constitutive material modeling, designed to support the flexible and modular development of models for complex material behavior. Building on the foundational structure of its predecessor, NEML, the NEML2 library introduces significant improvements, including enhanced vectorization, automatic differentiation, and seamless integration with PyTorch, facilitating the application of machine learning techniques in material simulations. NEML2 provides a C++ backend with Python bindings, enabling users to create custom material models that can be executed efficiently on both CPU and GPU platforms. The library also supports coupling with Multiphysics simulation frameworks like MOOSE, making it suitable for realistic simulations involving coupled physical processes. Rigorous quality assurance through unit and regression testing ensures the reliability of results, while the extensible, user-friendly design encourages collaboration and reproducibility across the scientific community. This paper provides an overview of NEML2’s architecture, core features, and applications, highlighting its impact on accelerating material qualification and advancing computational methods in materials science.

GPU