Search NASA⌕ Search

SEARCH · Search NASA

Results for “software validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

PeakQC: A Software Tool for Omics-Agnostic Automated Quality Control of Mass Spectrometry Data

Mass spectrometry is broadly employed to study complex molecular mechanisms in various biological and environmental fields, enabling 'omics' research such as proteomics, metabolomics, and lipidomics. As study cohorts grow larger and more complex with dozens to hundreds of samples, the need for robust quality control (QC) measures through automated software tools becomes paramount to ensure the integrity, high quality, and validity of scientific conclusions from downstream analyses and minimize the waste of resources. Since existing QC tools are mostly dedicated to proteomics, automated solutions supporting metabolomics are needed. To address this need, we developed the software PeakQC, a tool for automated QC of MS data that is independent of omics molecular types (i.e., omics-agnostic). It allows automated extraction and inspection of peak metrics of precursor ions (e.g., errors in mass, retention time, arrival time) and supports various instrumentations and acquisition types, from infusion experiments or using liquid chromatography and/or ion mobility spectrometry front-end separations and with/without fragmentation spectra from data-dependent or independent acquisition analyses. Diagnostic plots for fragmentation spectra are also generated. Here, in this paper, we describe and illustrate PeakQC’s functionalities using different representative data sets, demonstrating its utility as a valuable tool for enhancing the quality and reliability of omics mass spectrometry analyses.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Using a Large Language Model as a Building Block to Generate Usable Validation and Verification Suite for OpenMP

In the HPC area, both hardware and software move quickly. Often new hardware is developed and deployed, the corresponding software stack, including compilers and other tools, are under active development while leading edge software developers are working to port and tune their applications, all at the same time. While the software ecosystem is in flux, one of the key challenges for users is obtaining insight into the state of implementation of key features in the programming languages and models their applications are using – whether they have been implemented, and whether the implementation conforms to the specification, especially for newly implemented features (less tested by widespread use). OpenMP is one of the most prominent shared memory programming models used for on-node programming in HPC. With the shift towards accelerators (such as GPUs and FPGAs) and heterogeneous programming OpenMP features are getting more complex. It is natural to ask whether generative AI approaches, and large language models (LLMs) in particular, can help in producing validation and verification test suites to allow users better and faster insights into the availability and correctness of OpenMP features of interest. In this work, we explore the use of ChatGPT-4 to generate a suite of tests for OpenMP features. We have chosen a set of directives and clauses, a total of 78 combinations, which first appeared in OpenMP 3.0 (released in May 2008) but are also relevant for accelerators. We prompted ChatGPT to generate tests in the C and Fortran languages, for both host (CPU) and device (accelerator). On the Summit super-computer using the GNU implementation, we found that, of the 78 generated tests 67 C tests and 43 Fortran tests compiled successfully and fewer than those executed to completion. On further analysis we show that not all generated tests are valid. We document the process, results, and provide detailed analysis regarding the quality of tests generated. With the aim of providing input to a production quality validation and verification suite, we manually implement the corrections required to make the tests valid according to the current OpenMP specification. We quantify this effort as small, medium, or large, and record the lines of code changed to correct the invalid tests. With the corrected tests we validate recent implementations from HPE, AMD, and GNU on the Frontier supercomputer. Our experiment and subsequent analysis show that although LLMs are capable of producing HPC specific codes, they are limited by their understanding of the deeper semantics and restrictions of programming models such as OpenMP. Unsurprisingly more commonly used features have better support, while some OpenMP 3.0 directives such as sections and tasking are not universally supported on accelerators. We demonstrate that successful compilation and execution to completion are inadequate metrics for evaluating generated code and that, at this time, commodity LLMs require expert intervention for code verification. This points to gaps in the training data that is currently available for HPC. We demonstrate that with "small" effort 37% of generated invalid C tests and 63% of generated invalid Fortran tests could be corrected. This improves productivity of test generation as we circumvent writing from scratch and the common programming errors associated with it.

Pophale, Swaroop [ORNL] (ORCID:0000000185446367)↗
DOE OSTI2026

Report of the 2026 Workshop on Next-Generation Ecosystems for Scientific Computing: Harnessing Community, Software, and AI for Cross-Disciplinary Team Science

Scientific computing is undergoing rapid transformation as advances in artificial intelligence, heterogeneous computing, automation, and data-intensive research reshape not only computational tools but also the institutions, workforce models, and collaborative practices that support scientific discovery. This report synthesizes insights from the 2026 Workshop on Next-Generation Ecosystems for Scientific Computing, the second in a three-year series focused on strengthening scientific computing ecosystems through socio-technical co-design. Workshop discussions identified four interdependent strategic themes: software ecosystems for AI-enabled scientific discovery; trust, validation, and traceability; human-AI teaming and paradigm shifts; and workforce, pedagogy, and governance. The report translates these themes into eight priorities for community action spanning shared research infrastructure, trust and traceability, user experience, human-AI teaming, workforce development, cross-sector coordination, stewardship and sustainability, and evaluation of scientific value. Together, these priorities outline directions for building scientific computing ecosystems that remain trustworthy, sustainable, innovative, and resilient as AI assumes a growing role in scientific work.

AI↗
DOE OSTI2024

Transient Stability Analysis of Direct Droop-Controlled Grid-Forming Inverter with Current Clipping as Current Limiting Control

This paper investigates the transient stability of the direct droop-controlled grid-forming (GFM) inverter which employs the current clipping-based approach for overcurrent limitation. An equivalent circuit model of the direct-droop GFM inverter with current clipping is developed for this purpose. It is shown that during overcurrent conditions, the GFM inverter with current clipping acts as a voltage source with a virtual resistance behind the inverter filters. To compute the parameters of this virtual resistance an analytical expression is given in the paper. Then utilizing the developed circuit, a criteria to exit current limiting during post-fault conditions is given. The developed analytical approaches are then validated through electromagnetic transient (EMT) simulation results in PSCAD software and finally concluding remarks are drawn.

Mohiuddin, Sheik Mohammad↗
DOE OSTI2026

Multi-Artifact Analysis of Self-Admitted Technical Debt in Scientific Software

Context: Self-admitted technical debt (SATD) occurs when developers acknowledge shortcuts in code. In scientific software (SSW), such debt poses unique risks to the validity and reproducibility of results. Objective: This study aims to identify, categorize, and evaluate scientific debt, a specialized form of SATD in SSW, and assess the extent to which traditional SATD categories capture these domain-specific issues. Method: We conduct a multi-artifact analysis across code comments, commit messages, pull requests, and issue trackers from 23 open-source SSW projects. We construct and validate a curated dataset of scientific debt, develop a multi-source SATD classifier to guide SATD management, and conduct a practitioner validation to assess the practical relevance of scientific debt. Results: Our classifier performs strongly across 900,358 artifacts from 23 SSW projects. SATD is most prevalent in pull requests and issue trackers, underscoring the value of multi-artifact analysis. Models trained on traditional SATD often miss scientific debt, emphasizing the need for its explicit detection in SSW. Practitioner validation confirmed that scientific debt is both recognizable and useful in practice. Conclusions: Scientific debt represents a unique form of SATD in SSW that that is not adequately captured by traditional categories and requires specialized identification and management. Our dataset, classification analysis, and practitioner validation results provide the first formal multi-artifact perspective on scientific debt, highlighting the need for tailored SATD detection approaches in SSW.

Melin, Eric [Boise State University]↗
DOE OSTI2025

pnnl-predictive-phenomics/SElon-GEM

The software was developed to solve the problem of continuous validation for genome-scale model of S. elongatus. The origin of the work was due to the difficulty of curating genome-scale metabolic models, and memote was invented to solve this problem.

Pino, James [Pacific Northwest National Laboratory↗
DOE OSTI2024

A Simulation and Optimization Framework for Managing Wind-Driven Loading on PV Systems

As PV modules continue to trend toward larger, thinner, and more flexible forms they grow more susceptible to damage from dynamic wind loading. As a result, understanding the impact of wind on PV systems, particularly when mounted on solar-tracking hardware, and identifying robust, stable array layouts and stow strategies is becoming increasingly important for the PV community. We are developing an open-source software package, PVade (PV aerodynamic design engineering), to simulate the cascading fluid-structure interaction that occurs within solar-tracking arrays to enable researchers to test hardware, layout, and tracker control changes, leading to enhanced stability and a reduction in wind-driven damage. We will give an overview of the PVade software and present the latest outcomes from our ongoing validation campaign in which we compare statistical structural responses with field data. From there, we will present simulated results from a larger, multi-row array and highlight the effect of varying tracker angles on stability as measured by both acceleration and deformation, focusing on the stability differences between positive and negative tilt angles.

fluid structure interaction↗
DOE OSTI2024

Simulating Wind-Driven Loading on PV Systems

As PV modules continue to trend toward larger, thinner, and more flexible forms they grow more susceptible to damage from dynamic wind loading. As a result, understanding the impact of wind on PV systems, particularly when mounted on compliant solar-tracking hardware, and identifying robust, stable array layouts and stow strategies is becoming increasingly important for the PV community. We are developing an open-source software package, PVade (PV aerodynamic design engineering), to simulate the cascading fluid-structure interaction that occurs within single-axis, solar-tracking arrays to enable researchers to test hardware, layout, and tracker control changes, leading to enhanced stability and a reduction in wind-driven damage. We will give an overview of the PVade software and present the latest outcomes from our ongoing validation campaign in which we compare time series and statistical structural responses with field data. From there, we will present simulated results from a larger, multi-row array and highlight the effect of varying tracker angles on stability and the differences between positive and negative tilt angles.

fluid↗
DOE OSTI2025

Learning-Based Quantum Compilation: Translating QASM to QIR with CodeBERT

We propose a learning-based approach to quantum compilation by translating OpenQASM to Quantum Intermediate Representation (QIR) using a fine-tuned CodeBERT model. Trained on 10,000 synthetic QASM-QIR pairs, the model captures code semantics while addressing QIR verbosity and the 512-token limit via a custom token compression scheme. Finetuning was performed on the Frontier supercomputer, with results showing syntactic correctness and stable validation loss reduction. Our method moves toward enabling flexible, language-modeldriven quantum software tools. It also introduces syntax error handling and the possibility of incorporating classical control constructs, addressing limitations in existing rule-based compilers like qBraid-QIR. While the current model has been validated on quantum-only circuits, we propose future evaluations on hybrid quantum-classical examples. This poster will provide architecture insights, compression examples, training loss plots, and QIR outputs. Our work highlights the potential for scalable, adaptable compilation in future quantum toolchains.

Afrose, Sharmin [ORNL]↗
DOE OSTI2026

Coupling Microstructural Evolution Simulations to Material Property Degradation Predictions for Plasma-Facing Materials

Reliable material performance is required for plasma-facing material (PFM) candidates. Previous research has shown that plasma and neutron radiation exposure induces microstructural changes in PFMs; changes in thermal and electrical conductivities and in material hardening and embrittlement were also observed after neutron irradiation. These material property changes will negatively impact the performance of the PFMs in a fusion reactor. Despite the well-known connection between material microstructure, properties, and performance, there is a need for validated modeling capabilities connecting PFM property degradation with microstructural evolution under fusion-relevant conditions. We are developing a simulation capability to couple plasma-induced microstructural evolution to material property degradation. Our approach relies on deliberate mapping between individual simulation models and experimental characterization for validation. The open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) software was used for this simulation capability development. A MOOSE phase-field model was coupled with the cluster dynamics code, Xolotl, to predict microstructural evolution. Microstructure characterization techniques, including scanning electron microscopy (SEM), transmission electron microscopy (TEM), and laser scanning confocal microscopy (LSCM) are used to validate these microstructural evolution simulations. Calculation of thermal and electrical conductivities with first principles simulations was performed for bulk material and for grain boundaries; these results are used within MOOSE models to calculate effective thermal and electrical conductivities as a function of grain characteristics. Thermoreflectance and four-probe techniques were employed to measure the thermal and electrical conductivities, respectively. A MOOSE crystal plasticity model was adapted to predict microstructure-sensitive deformation behavior, and X-ray diffraction (XRD) was used to collect bulk dislocation density data for validation. After individual simulation validation, these models are coupled to predict material property changes resulting from plasma exposure. We focused here on an experimental design to emphasize the separate effects of moderate thermal loads and plasma exposure using tungsten. Annealing of tungsten was performed under a protective environment for temperatures ranging from 500 C to 1500 C. The plasma exposure was completed in the Tritium Plasma Experiment at Idaho National Laboratory under a deuterium flux of 1e22 D/m^2-s. This incremental approach is employed to build confidence in the modeling capability: separate-effects tests ensure that the models capture key mechanisms from single environmental conditions before predicting PFM property degradation under combined loads. We will show our early results from coupling these simulation models to predict PFM property changes from microstructural evolution. Comparisons of the simulation results with preliminary validation data will be discussed.

36 - MATERIALS SCIENCE↗
DOE OSTI2026

Connect microstructure evolution to property degradation with validated simulation

Reliable material performance is required for plasma-facing material (PFM) candidates. Previous research has shown that plasma and neutron radiation exposure induces microstructural changes in PFMs; changes in thermal and electrical conductivities and in material hardening and embrittlement were also observed after neutron irradiation. These material property changes will negatively impact the performance of the PFMs in a fusion reactor. Despite the well-known connection between material microstructure, properties, and performance, there is a need for validated modeling capabilities connecting PFM property degradation with microstructural evolution under fusion-relevant conditions. We are developing a simulation capability to couple plasma-induced microstructural evolution to material property degradation. Our approach relies on deliberate mapping between individual simulation models and experimental characterization for validation. The open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) software was used for this simulation capability development. A MOOSE phase-field model was coupled with the cluster dynamics code, Xolotl, to predict microstructural evolution. Microstructure characterization techniques, including scanning electron microscopy (SEM), transmission electron microscopy (TEM), and laser scanning confocal microscopy (LSCM) are used to validate these microstructural evolution simulations. Calculation of thermal and electrical conductivities with first principles simulations was performed for bulk material and for grain boundaries; these results are used within MOOSE models to calculate effective thermal and electrical conductivities as a function of grain characteristics. Thermoreflectance and four-probe techniques were employed to measure the thermal and electrical conductivities, respectively. A MOOSE crystal plasticity model was adapted to predict microstructure-sensitive deformation behavior, and X-ray diffraction (XRD) was used to collect bulk dislocation density data for validation. After individual simulation validation, these models are coupled to predict material property changes resulting from plasma exposure. We focused here on an experimental design to emphasize the separate effects of moderate thermal loads and plasma exposure using tungsten. Annealing of tungsten was performed under a protective environment for temperatures ranging from 500$^o$C to 1500$^o$C. The plasma exposure was completed in the Tritium Plasma Experiment at Idaho National Laboratory under a deuterium flux of 1e22 $\frac{D}{m^2s}$. This incremental approach is employed to build confidence in the modeling capability: separate-effects tests ensure that the models capture key mechanisms from single environmental conditions before predicting PFM property degradation under combined loads. We will show our early results from coupling these simulation models to predict PFM property changes from microstructural evolution. Comparisons of the simulation results with preliminary validation data will be discussed.

36 - MATERIALS SCIENCE↗
DOE OSTI2026

PV Operations Software Transparency: A PVMAC Industry Snapshot

The rapid growth of photovoltaic (PV) deployment has increased reliance on software platforms for monitoring, workflow automation, diagnostics, and performance analytics. As these tools play a central role in asset management and operations and maintenance (O&M), greater transparency in methodologies, data handling, and validation practices benefits the broader PV ecosystem. To better understand current practices and identify opportunities for improved clarity and interoperability, 24 software providers contributed detailed responses through the PV O&M Analytics Collaborative (PVMAC) initiative, the first structured questionnaire of its kind in the industry, covering onboarding, interoperability, data quality, diagnostics, AI/ML, and other operational categories. These providers represent over 1.1 TW of solar assets under management. The analysis shows broad adoption of digital twins, AI/ML, and API integrations, but also highlights challenges in onboarding processes, inconsistent definitions and methodologies, variability in key performance indicator (KPI) calculations, and limited independent validation. Greater standardization, clearer documentation, and stronger validation frameworks could improve transparency, comparability, and trust across PV operations software platforms.

14 SOLAR ENERGY↗
DOE OSTI2024

Evaluating a Commercial Dynamic Line Rating Software with the National PMU Dataset

To accelerate the development of data-driven applications for power systems, the Department of Energy (DOE) supported the collection and curation of a synchrophasor dataset spanning two years of observations from transmission utilities across the US. This National PMU Dataset (NPDS) was anonymized and distributed to awardees of a DOE research grant under nondisclosure agreements (NDAs) but has also been retained at PNNL to enable further research. Agreements with data contributors prevent the data from being shared outside the organization. However, establishing a blind research validation methodology is envisioned to maximize the value proposition of the NPDS. In this validation strategy, researchers may share algorithms/software (potentially as executables to protect intellectual property) with PNNL, and PNNL will share feedback about the software’s performance on subsets of the NPDS. Such a blind methodology ensures that sensitive information about critical infrastructure remains protected, but the value of the NPDS can be extended to research beyond PNNL. Through iterative feedback, the algorithms may be tweaked to address real-world artifacts. As the NPDS data is temporally and geographically diverse, it may capture features absent in smaller datasets used during the development of the algorithm under test. This report presents lessons learned from applying the blind validation methodology to LineID™, a synchrophasor-based dynamic line rating software developed by Topolonet Corporation. Improvements made to the software through iterative feedback, limitations of the validation methodology, as well as how the limitations of the NPDS affected the evaluation process are discussed. Observations indicate that the proposed validation methodology can be valuable for evaluating other tools in the future.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

SLIA Reference Architecture Models

The SLIA Reference Architecture Models project, sponsored by the DOE CESER Energy CyberSense Program (Oct 2024–Sep 2025), advanced LLNL’s PySCES simulation tool to better support CyTRICS Prioritization and Initial Risk Assessment (PIRA) reference architectures. Key achievements include enhancements to the PySCES transmission substation facility model, expanded asset coverage, and enhancements to the PySCES code base. Software improvements reduced code complexity, migrated PySCES to Python version 3.11, introduced an object-oriented design, and added a schema database for easier updates and validation. New features support device criticality assessments and a more precise parametric simulation mode. Remaining gaps include model validation, workflow limitations, Monte Carlo convergence issues, full device criticality metric implementation, model fidelity, and general software improvements. Continued development is recommended to address these gaps and fully align PySCES with CyTRICS PIRA requirements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Development, validation, and verification of multi-pass thermo-mechanical welding simulations using the open-source MOOSE framework: NeT TG4 benchmark weldment

This study develops and validates a sequentially coupled thermo-mechanical welding simulation for the three-pass 316L stainless steel NeT TG4 benchmark weldment using the open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) and the Nuclear Engineering Material model Library (NEML). A diffused ellipsoidal heat source was calibrated against thermocouple data and weld macrographs to accurately model the fusion zone geometry and transient thermal fields. Material hardening is represented using the Lemaitre-Chaboche mixed isotropic-kinematic hardening model, while four annealing models - no annealing, single-stage at 1050 °C and 1300 °C, and two-stage at 800 °C/1300 °C - were implemented to assess the impact of annealing models on the accuracy of the predicted welding-induced plasticity, distortions, and residual stresses. The predictions were validated against experimental measurements and benchmarked against results from commercial software, demonstrating that thermo-mechanical MOOSE welding simulations achieve comparable accuracy with enhanced computational efficiency. This work highlights the potential of using open-source finite element frameworks like MOOSE for advanced manufacturing simulations.

Ji, Wendy [Australian Nuclear Science and Technolo↗
DOE OSTI2024

LPBF Processability of NiTiHf Alloys: Systematic Modeling and Single-Track Studies

Research into the processability of NiTiHf high-temperature shape memory alloys (HTSMAs) via laser powder bed fusion (LPBF) is limited; nevertheless, these alloys show promise for applications in extreme environments. This study aims to address this limitation by investigating the printability of four NiTiHf alloys with varying Hf content (1, 2, 15, and 20 at. %) to assess their suitability for LPBF applications. Solidification cracking is one of the main limiting factors in LPBF processes, which occurs during the final stage of solidification. To investigate the effect of alloy composition on printability, this study focuses on this defect via a combination of computational modeling and experimental validation. To this end, solidification cracking susceptibility is calculated as Kou’s index and Scheil–Gulliver model, implemented in Thermo-Calc/2022a software. An innovative powder-free experimental method through laser remelting was conducted on bare NiTiHf ingots to validate the parameter impacts of the LPBF process. The result is the processability window with no cracking likelihood under diverse LPBF conditions, including laser power and scan speed. This comprehensive investigation enhances our understanding of the processability challenges and opportunities for NiTiHf HTSMAs in advanced engineering applications.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Best practices in software development for robust and reproducible geoscientific models based on insights from the Global Carbon Budget's dynamic vegetation models

Computational models play an increasingly vital role in scientific research by enabling the numerical simulation of complex processes. Such models are also fundamental in geosciences. For instance, they offer critical insights into the impacts of global change on the Earth system today and in the future. Beyond their value as research tools, models are also software products and should therefore adhere to certain established software engineering standards. However, scientists are rarely trained as software developers, which can lead to potential deficiencies in software quality like unreadable, inefficient, or erroneous code. The complexity of models, coupled with their integration into broader workflows, also often makes it challenging to reproduce results, evaluate processes, and build upon them. In this paper, we review the state and current practices of the development processes of the state-of-the-art land surface models used by the Global Carbon Budget. We combine the experience of modelers from the respective research groups with the expertise of software engineers from tech companies to outline key principles and tools for improving software quality in research. We explore four main areas: (1) model testing and validation, (2) scientific, technical, and user documentation, (3) version control, continuous integration, and code review, and (4) the portability and reproducibility of workflows. Our review reveals that while modeling communities are incorporating many best practices, significant room for improvement remains in areas such as automated testing, automated documentation, and reproducibility. Therefore, we here identify and promote essential software engineering practices, including numerous examples of practices from within the community that can serve as guidelines for other models and could help streamline processes across the entire community. We conclude with an open-source example implementation of these principles, demonstrating portable and reproducible data flows, a continuous integration setup, and web-based visualizations. This example may serve as a practical resource for model developers, users, and all scientists engaged in scientific programming.

Gregor, Konstantin [Technical Univ. of Munich (Ger↗
DOE OSTI2025

Establishing model credibility for process-microstructure-property relationships in additive manufacturing using exascale computing

Additive Manufacturing (AM) of alloys holds significant promise as a disruptive technology in various industries, yet its adoption is often hindered by challenges in achieving consistent part quality. These issues are primarily due to the complex process-microstructure-property (PSP) relationships inherent to AM. Computational models can greatly aid in understanding these relationships, but their widespread impact and adoption has been limited by a lack of validated, open-source, and computationally efficient PSP modeling frameworks and hardware limitations. Here, this study leverages the ExaAM software suite and data from the AMBench-2018 series of laser powder bed fusion (LPBF) benchmark experiments to perform a comprehensive model assessment, including verification, validation, sensitivity analysis, and uncertainty quantification. The RADICAL-EnTK workflow manager was used to perform an ensemble of heat transport, solidification, and mechanical response simulations on the exascale computer Frontier, considering uncertainties in critical model inputs such as laser spot size and nucleation parameters, and consisting of 125 explicit grain structure simulations and 7875 crystal plasticity simulations. For a selected location within the Inconel 625 AMBench-2018 test artifact, sensitivity analysis and uncertainty quantification were performed using the predicted distributions of grain structure and mechanical properties. Qualitative agreement was found between the predicted grain size and texture and the observed AMBench-2018 microstructure, the mean predicted yield stress was within 5% of the experimental measurement mean, and the mean predicted engineering stress at 5% strain was within 10% of the experimental measurement mean. The insights gained from development and validation of the ExaAM PSP modeling framework will help guide future directions for enhancing the credibility and reliability of PSP models in AM, thereby accelerating the adoption of AM technologies in various industries.

Additive manufacturing↗