SEARCH · Search NASA

Results for “model checking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2026

Minimizing the Electromechanical Stresses in Poloidal Field Coils by Optimizing their Numbers and Locations using FREDA Framework

Poloidal field (PF) and central solenoid (CS) coils play a crucial role in sustaining the equilibrium and preserving the shape of highly confined tokamak plasmas. Ensuring that PF coil current and mechanical stress stay within superconducting and structural limitations is an important check in the design assessment. Minimizing the PF coil currents and mechanical stresses influences reliability, cost, and performance. A free-boundary MHD equilibrium code—FreeGS is employed within the fusion reactor design and assessment (FREDA) whole facility modeling (WFM) framework to construct the plasma equilibrium based on the configuration and currents in the PF coils. Here, we present the capability of the FreeGS code to minimize the currents, forces, and electromagnetic stresses on the PF coils by optimizing their number, sizes, structures, and locations while maintaining an MHD stable plasma configuration with a large confinement factor. The workflow is initialized with a configuration of plasma parameters and coils’ locations from the 0-D tokamak build systems code in the FREDA framework. Then, FreeGS is called to calculate the initial equilibrium at the minimum total current in PF coils. Thereafter, FreeGS’s internal optimizer minimizes the currents and hoop and central forces on the PF coils while maintaining the reference equilibrium. Finally, the input configuration is updated with the optimized parameters for equilibria over the ramp-up phase of a burning-plasma operation. FREDA’s whole facility optimization capability, which includes all magnetic field coil systems, blanket, vacuum vessel (VV), first wall, divertor, etc., is under development and out of the scope for this study.

Hassan, Ehab [ORNL] (ORCID:0000000181060301)↗

DOE OSTI2025

Rosenbluth-like separation of the $J/ψ$ near-threshold photoproduction: An access to the gluon gravitational form factors at high t

Here, we perform analysis of the near-threshold $J/\psi $ photoproduction data off the proton based on two theoretical approaches, GPD \cite{Guo3} and holographic \cite{Zahed2}, that represent the differential cross sections as powers of the skewness parameter with coefficients that depend only on the momentum transfer $t$. This allows to separate kinematically the corresponding coefficient functions, in much the same way as this is done for the electric and magnetic form factors using the Rosenbluth separation. We examine the independence of the extracted functions with the photon beam energy. These functions, under additional assumptions, are related to the proton's gluon Gravitational Form Factors (gGFFs). We compare the extracted functions with lattice calculations of the gGFFs in the region of $0.5<|t|<2$~GeV$^{2}$, where they overlap. Such analysis demonstrates the possibility of extracting some combinations of the gGFFs from the data at high $t$, complementary to the lattice calculations available in the low $t$ region. However, higher statistics are needed to more accurately check the predicted scaling behavior of the data and compare with the lattice results, thus testing and comparing the theoretical assumptions used in the GPD and holographic models.

Pentchev, Lubomir [Thomas Jefferson National Accel↗

DOE OSTI2026

From models to reality: a systematic review on simulated and measured residential heat pump energy savings

High-performance HVAC solutions are central to residential energy management. A substantial share of these are electric, reversible-cycle systems, with heat pumps representing the largest portion of current and near-term adoption. This review synthesizes peer-reviewed and grey literature on residential space heating and cooling heat pumps. The academic literature is dominated by modeling (73.8%), with limited field measurement (13.1%). Grey literature from United States serve as a supplemental resource providing measured savings. Conversions from electric-resistance heating consistently show the largest site energy reductions, while oil/propane baselines yield moderate savings, and gas baseline scenario often deliver small and region-dependent savings. This study cross-checks the grey literature measured data with simulation data filtered from the ResStock dataset. The comparison indicates a discrepancy between simulations and measured data: simulated site EUIs are typically lower than measured EUIs, but percentage energy savings fall in similar ranges, implying simulations capture directional effects while underestimating energy use. Factors associated with variability and model–measurement differences include system characterization and control representation (e.g., backup heat engagement, thermostat/setpoint strategies, commissioning/installation quality), occupant behavior, weather normalization, metering scope, and envelope characterization. This paper also outlines the proposed methodology for comparing simulation and measured data for heat pumps. It emphasizes the metrics used for comparison and units harmonization, building characteristics matching, and compact metadata are needed for simulations to match measured data. The proposed methodology is expected to improve the credibility of simulated savings as measured evidence grows.

Yu, Lili↗

DOE OSTI2025

Improved Weld Residual Stress Modeling System in BlackBear

This report presents enhancements to the MOOSE-based BlackBear application aimed at improving its capability to simulate welding and other thermo-mechanical manufacturing processes. Two primary avenues of improvement are pursued. First, to enhance user accessibility, we introduce a centralized default block restriction mechanism that ensures coverage checks are performed within user-specified default blocks. This default setting is applied consistently to all block-describable objects, such as variables, kernels, and more. In addition, we develop a modular action for moving heat source simulations, which integrates path file parsing, subdomain modification, and heat source kernel enforcement into a single, streamlined configuration. Second, to improve solver robustness, we implement an alternative method for assigning initial conditions to the updated active domain during the simulation, thereby enhancing convergence behavior. To validate the framework, we design and conduct several benchmark simulations, including heat conduction with progressive material addition, linear elasticity with time-dependent material deposition, and viscoplasticity model with isotropic hardening under similar conditions. Finally, we demonstrate the effectiveness of the proposed framework through large-scale thermo-mechanical welding simulations in both two and three dimensions.

42 ENGINEERING↗

DOE OSTI2025

Explainable discrepancy checker and diagnosis for digital Twin-based supervisory control system

By virtually representing a physical object and process, a digital twin (DT) enables optimal autonomous operations by combining classical and novel frameworks in sensors, state predictions, and multi-input/multi-output systems. A DT’s values depend on how well models estimate quantities of interest and on how uncertainty is handled. Moreover, DTs often combine physics-based and data-driven models with mixed fidelities, where classical uncertainty quantification (UQ) struggles with many sources of uncertainty and real-time constraints. Here, this work presents a UQ-based discrepancy checking and diagnosis tool for a DT-based supervisory control system. The tool is developed using metadata from an automated DT development process to learn correlations between sources of uncertainties and outcomes. During operation, it compares predictions with measurements, attributes discrepancies to dominant sources, and recommends parameter and configuration updates. We verify the workflow on a synthetic temperature-control problem and deploy it on a virtual Thermal Energy Delivery System, reducing mismatch and improving control robustness.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2025

Physical Interpretation of Early Battery Life Prediction Models

Early battery life prediction models are most useful for R&D if they help us understand the early changes in battery electrochemical response that correspond with long-term degradation and failure. Linear regression models such as Fused lasso and Partial Least Squares can fit coefficients directly to high-dimensional electrochemical data like capacity-voltage and ΔV–state-of-charge, i.e., Q(V) and ΔV(SOC) curves, learning coefficients that can be physically interpreted. We leverage the ISU-ILCC battery aging data set to learn high-dimensional coefficients for early battery life prediction from traditional slow-rate capacity check data, demonstrating learning on Q(V), d Q· d V −1 , and ΔV(SOC) curves. A thorough study on the dependence of coefficient values on train/test size and data preprocessing methods is made, demonstrating the reliability of high-dimensional regression approaches unless very small amounts of data are used for model training. For this data set, coefficients from Q(V) and d Q· d V −1 models highlight changes in electrode stoichiometry due to lithium loss, while ΔV(SOC) coefficients highlight changes in positive electrode diffusivity due to particle cracking as well as electrode stoichiometry shifts. By directly interpreting the coefficients of a regression model, we make physical insights into battery degradation mechanisms without requiring the assumptions of traditional battery data analysis methods.

25 ENERGY STORAGE↗

DOE OSTI2025

Equation of State for the Thermodynamic Properties of Trans-1,2-dichloroethene [R-1130(E)]

We present an empirical equation of state in terms of the Helmholtz energy for trans-1,2-dichloroethene [R-1130(E)]. The range of validity is from the triple-point temperature, 223.31 K to 525 K with pressures up to 30 MPa. It may be used to calculate all thermodynamic properties in the fluid phase, including liquid, gas, and supercritical regions. Comparisons are given with existing literature data and estimated uncertainties are provided. In addition, checks were made for correct extrapolation behavior so that the equation behaves in a physically realistic manner when used outside of its range of validity, enabling its use in mixture models. The estimated uncertainties (at a k = 2 or 95 % level of confidence) are based on comparisons with critically assessed data and are 0.25 % for vapor pressure for temperatures in the range 300 K < T < 454 K, rising to 1.5 % as the temperature decreases from 300 K to 265 K. For density in the liquid phase the estimated uncertainty is 0.14 % for temperatures 270 K < T < 410 K and for pressures up to 30 MPa. For the vapor phase the estimated uncertainty in density is 3 %. The uncertainty for liquid-phase heat capacity is 1 % at atmospheric pressure over the temperature range 268 K < T < 309 K, and the uncertainty for the speed of sound in the liquid phase is 0.25 % for temperatures 230 K < T < 420 K and for pressures up to 30 MPa. The uncertainties are larger outside of these specified ranges and in the critical region.

1,2-Dichloroethene↗

DOE OSTI2026

Design and Characterization of the 162.5 MHz RF Component Layout for Fermilab's PIP-II Reference Line

The Proton Improvement Plan II (PIP-II) Reference Line at Fermi National Accelerator Laboratory distributes phase-stable radio-frequency (RF) signals throughout the accelerator. PIP-II requires stable timing and phase reference signals so its accelerating cavities transfer energy to the particle beam at the correct point in each RF cycle. The full system includes 162.5, 325, and 650 MHz sections corresponding to the frequency sections of the PIP-II Linac, with this project focusing on the 162.5 MHz section. The Reference Line must provide a phase stable source signal while responding to phase changes caused by environmental conditions or system drift. Its RF components will be mounted on aluminum heat plates inside a temperature controlled enclosure to further limit temperature-driven phase changes. To prepare the system for manufacture, the project reviewed component functions and dimensions, developed a computer-aided design (CAD) model, and arranged the hardware to support short cable paths, grounding, fastener access, and maintenance. Several full scale, three dimensional printed prototypes allowed the available components to be mounted and inspected. These fit checks revealed mechanical conflicts and guided revisions to component placement, countersink geometry, labeling, and plate thickness. Electrical characterization was also performed on selected RF hardware to compare its measured behavior with the performance metrics that were set for our design. Overall, the project produced a manufacturable 162.5 MHz layout, physical fit-check prototypes, and documented electrical measurements that support review before metal fabrication. The 325 and 650 MHz layouts remain future work because they require additional minor mechanical changes.

Subedi, Harsheet [Unlisted, US, CA] (ORCID:0009000↗

DOE OSTI2026

Updated constraints from electric dipole moments in the MSSM with R-parity violation

We revisit the electric dipole moments (EDMs) of quarks and leptons in the Minimal Supersymmetric Standard Model (MSSM) with trilinear R-parity violation (RPV). In this framework, EDMs are induced at the two-loop level via RPV interactions. We perform a comprehensive recalculation of several classes of Barr-Zee type diagrams in a general Rξ gauge. While we find general agreement with previous analytic results in the literature, our work provides a valuable independent cross-check of the complicated calculations. We also point out some subtleties in the intermediate steps and in the choice of the flavor basis for the numerical evaluation of the expressions. By confronting the theoretical predictions with the latest experimental limits on EDMs, we derive updated constraints on combinations of RPV couplings. We highlight an approximate, testable correlation between the proton and neutron EDM that emerges within the considered class of RPV models, offering a distinctive signature for future EDM experiments.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2026

BSEC flux towers: CSAT3B and TRH

The data were collected as part of the BSEC project, during the period from June 2025 to May 2026. Directory "broadway" contains data collected on a multi-level flux tower (US-BWf) in the Broadway East neighborhood (1808 North Patterson Park Ave., Baltimore City, MD 21213; LAT: 39o18'40.31'' N; LONG: 76o35'12.43'' W). At each of the four measurement heights (8.5 m, 11.1 m, 13.4 m, 15.9 m), a Campbell Scientific CSAT3B sonic anemometer was operated at 50 Hz to measure virtual temperature (tc) and three velocity components (u: 270 degrees; v: 180 degrees; w: vertical), and a RM Young temperature sensor (model 41382VC) was operated at 1 Hz inside a compact aspirated radiation shield (model 43502) to measure absolute temperature (T) and relative humidity (RH). Inside directory "broadway", directory "netcdf" contains data collected each day in 5-minute chunks that have been converted to NetCDF format (before quality checking), while "4hr" contains data arranged into 4-hour chunks (also in NetCDF format) that have been through basic quality checking steps (treating data points with nonzero diagnostic codes as missing data; fixing six or fewer consecutive missing data points using linear interpolation). Users are recommended to start with data in directory "4hr", while data in directory "netcdf" can be used for reference purposes.

Baltimore↗

DOE OSTI2025

Sequence modeling of higher-order wave modes of quasi-circular, spinning, non-precessing binary black hole mergers

Higher-order gravitational wave modes from quasi-circular, spinning, non-precessing binary-black-hole (BBH) mergers encode rich information about the nonlinear dynamics of strong-field gravity. We present a transformer-based sequence-completion surrogate that, given an early-inspiral segment, forecasts the subsequent late inspiral, merger, and ringdown. The intended applications are (i) patching or completing expensive or interrupted numerical-relativity (NR) simulations and (ii) providing late-time cross-checks and rapid hybridization studies. The training set is built from the NRHybSur3dq8 surrogate, which provides spherical-harmonic modes up to $\ell$ ≤ 4 (excluding (4, 0) and (4,±1), and including (5, 5)) for mass ratios q ≤ 8, dimensionless spin components s$^{z}_{1,2}$ ϵ[–0.8, 0.8], and inclination angles θ ϵ [0, π]. Waveforms are supplied on the interval t ϵ [–5000M, –100 M) and the model autoregressively generates the plus and cross polarizations (h + , h x ) on t ϵ [–100 M, 130M]. Training on the Delta supercomputer with 16 NVIDIA A100 GPUs required ~15 h on more than 14 million hybrid waveforms. Evaluation on a held-out test set of 840,000 samples yields mean and median overlaps of 0.996 and 0.997, respectively, with respect to the surrogate ground truth.

black-hole merger↗

DOE OSTI2025

GRUMDN: A Multi-Task Model for Predicting Human Patterns-of-Life from Stay Transition Data

Understanding human patterns-of-life (PoL) is essential towards ensuring safe and secure indoor facility environment as well as outdoor urban environment. Prediction of human movement in between places of interest is vital in understanding human PoL. Movement between spaces maybe represented and detected in one of the two forms: 1) trajectories: locations measured at regular time intervals by mobile sensors, bluetooth or GPS sensors; or 2) stay transitions: semantic PoI (points of interest) and stay duration data measurable by eventbased sensors that collect data when a check-in or check-out event is detected. Stay transition data provides a more compressed data format compared to trajectories data, especially in situations with longer stay durations, while preserving the information necessary for PoL analysis. Now as introduced briefly in the paper, our deployed end application (Digital Twin of a facility with non-player characters, besides the interactive user in virtual reality) needed a well-performing and validated AI/ML model for simulating high quality stay transitions behavior. In this study we thus primarily present our findings with developing and validating that model, which is a multi-task neural network for stay transition prediction. The neural network consists of two heads, for corresponding two tasks of stay category prediction and stay duration prediction. We evaluated gated recurrent units and multi-layer perceptrons of varying network sizes for stay category prediction; while mixture density networks, noisy generator-only networks, and generative adversarial networks of varying network sizes for stay duration prediction. We have then evaluated four multi-task models, constructed by combining these specialized models, on their ability to predict stay transition data. We tested our models on datasets from two different cases: 1) a simulation-generated dataset of indoor movement within the HFIR (high flux isotope reactor) nuclear reactor facility at Oak Ridge National Laboratory (ORNL); and 2) the GeoLife human mobility dataset of outdoor urban movement available in literature. Our results indicate that GRUMDN, which combines gated recurrent units (GRU) for stay category prediction task, and mixture density networks (MDN) for stay duration prediction task, did overall outperform other multitask models and the current state-of-the-art.

Gunaratne, Chathika [ORNL] (ORCID:0000000225088745↗

DOE OSTI2026

Data for Quantifying the Impact of Surfactants on Cloud Condensation Nuclei Activity Using a Particle-Resolved Model

This dataset contains simulation results from PartMC-MOSAIC and WRF-PartMC that used in the journal article: Quantifying the Impact of Surfactants on Cloud Condensation Nuclei Activity Using a Particle-Resolved Model. Two compressed folder are uploaded here, one is for the data that used in this article, the other folder is the python scripts to process the data. For more details of the uploaded files, please check the README file.

CCN↗

DOE OSTI2025

Measurement of $\nu_\mu$ CC Interactions With Two-Proton Final State in MINERvA

This dissertation presents a measurement of charged–current (CC) muon–neutrino interactions with exactly two protons and no pions in the final state (CC~$2p\,0\pi$), using data collected by the MINERvA detector in the NuMI medium–energy beam at Fermilab. Such two–proton topologies are a sensitive probe of nuclear dynamics in the few–GeV regime, including multi–nucleon correlations (npnh, notably $2p2h$) and intranuclear final–state interactions (FSI) such as pion absorption and nucleon rescattering. A precise experimental characterization of these processes is essential both for neutrino–interaction theory and for reducing systematic uncertainties in oscillation experiments that rely on accurate modeling of neutrino–nucleus interactions. Events are selected by requiring a $\nu_\mu$ CC interaction with a reconstructed $\mu^-$ and two proton tracks originating from a common vertex in MINERvA’s finely segmented scintillator tracker, with no reconstructed mesons. Muon charge and momentum are constrained by matching to the MINOS Near Detector, while proton identification exploits energy–loss profiles and stopping–proton features. Backgrounds from pion–producing channels that enter the signal region through FSI or reconstruction effects are constrained with data–driven sidebands (Michel–electron and isolated–cluster “blob” samples) and tuned via a simultaneous fit across signal and sideband regions. To correct detector resolution and acceptance effects, the analysis employs iterative Bayesian unfolding with extensive validation: statistical pseudo–experiments, and robustness checks against generator systematic “universes” and additional strong shape warps. Single–differential cross sections are reported for three observables tailored to the two–proton final state: the opening–angle cosine $\cos\!\left(\theta_{pp}\right)$, the leading–proton momentum, and the subleading–proton momentum. Systematic uncertainties include contributions from neutrino flux, interaction modeling (e.g., npnh and resonance parameters, pion FSI), and detector response (calibration, reconstruction efficiencies). The resulting distributions provide targeted constraints on the interplay of multi–nucleon dynamics and FSI that shape CC~$2p\,0\pi$ final states on hydrocarbon. Comparisons to modern GENIE–based simulations highlight kinematic regions where model components require refinement. These measurements thus inform generator tuning and improve the reliability of neutrino–energy reconstruction strategies for current and future long–baseline oscillation programs.

Syrotenko, Vladyslav S. [Tufts U.]↗

DOE OSTI2024

Comparative Pore Structure and Dynamics for Bacterial Microcompartment Shell Protein Assemblies in Sheets or Shells

Bacterial microcompartments (BMCs) are protein-bound organelles found in some bacteria that encapsulate enzymes for enhanced catalytic activity. These compartments spatially sequester enzymes within semipermeable shell proteins, analogous to many membrane-bound organelles. The shell proteins assemble into multimeric tiles; hexamers, trimers, and pentamers, and these tiles self-assemble into larger assemblies with icosahedral symmetry. While icosahedral shells are the predominant form in vivo , the tiles can also form nanoscale cylinders or sheets. The individual multimeric tiles feature central pores that are key to regulating transport across the protein shell. Our primary interest is to quantify pore shape changes in response to alternative component morphologies at the nanoscale. We used molecular modeling tools to develop atomically detailed models for both planar sheets of tiles and curved structures representative of the complete shells found in vivo . Subsequently, these models were animated using classical molecular dynamics simulations. From the resulting trajectories, we analyzed the overall structural stability, water accessibility to individual residues, water residence time, and pore geometry for the hexameric and trimeric protein tiles from the Haliangium ochraceu m model BMC shell. These exhaustive analyses suggest no substantial variation in pore structure or solvent accessibility between the flat and curved shell geometries. We additionally compare our analysis to hydroxyl radical footprinting data to serve as a check against our simulation results, highlighting specific residues where water molecules are bound for a long time. Although with little variation in morphology or water interaction, we propose that the planar and capsular morphology can be used interchangeably when studying permeability through BMC pores.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

The DEHVILS in the details: Type Ia supernova Hubble residual comparisons and mass step analysis in the near-infrared

Measurements of type Ia supernovae (SNe Ia) in the near-infrared (NIR) have been used both as an alternate path to cosmology compared to optical measurements and as a method of constraining key systematics for the larger optical studies. With the DEHVILS sample, the largest published NIR sample with consistent NIR coverage of maximum light across three NIR bands ( Y, J , and H ), we check three key systematics: (i) the reduction in Hubble residual scatter as compared to the optical, (ii) the measurement of a “mass step” or lack thereof and its implications, and (iii) the ability to distinguish between various dust models by analyzing slopes and correlations between Hubble residuals in the NIR and optical. We produce SN Ia simulations of the DEHVILS sample and find that it is harder to differentiate between various dust models than previously understood. Additionally, we find that fitting with the current SALT3-NIR model does not yield accurate wavelength-dependent stretch-luminosity correlations, and we propose a limited solution for this problem. From the data, we see that (i) the standard deviation of Hubble residual values from NIR bands treated as standard candles are 0.007–0.042 mag smaller than those in the optical, (ii) the NIR mass step is not constrainable with the current sample size of 47 SNe Ia from DEHVILS, and (iii) Hubble residuals in the NIR and optical are correlated in the data. We test a few variations on the number and combinations of filters and data samples, and we observe that none of our findings or conclusions are significantly impacted by these modifications.

Astronomy & Astrophysics↗

DOE OSTI2025

Benchmark Tracking System for Performance Monitoring

Benchmarking is essential for high-performance software development, particularly for monitoring performance across code iterations. This project focused on enhancing the benchmarking process for Lamellar, an asynchronous runtime for High-Performance Computing (HPC) systems developed at Pacific Northwest National Laboratory. Prior to this work, benchmark results were difficult to track and compare across code versions, presenting significant challenges in identifying performance regressions and long-term trends. The primary objective was to establish a systematic, reproducible approach for measuring performance and detecting regressions following code commits. Our methodology involved three key components: standardizing benchmark outputs, implementing data versioning, and developing analysis tools. We standardized the benchmark output format to JSON Line records containing specific fields (execution time, hardware specifications, and environmental variables). To address data management challenges, we evaluated several options and eventually chose a git repository dedicated to benchmark data. We developed a suite of Python tools that processed benchmark results, enriched them with metadata, and facilitated search in the repository. The resulting system enables more efficient filtering and comparison of performance metrics across commit histories, hardware configurations, and benchmark variants through a unified query interface. Our implementation reduces computational overhead by first checking for existing results through configuration matching before initiating new benchmark runs, thereby conserving resources. The system has been validated by Lamellar developers. It organizes results by benchmark type and build configurations for efficient retrieval. Future developments include a planned Large Language Model interface for predicting benchmark performance, incorporating the criterion package for statistical analysis, which will enable automated detection of statistically significant performance changes, and integration with continuous integration pipelines. Despite these enhancements being reserved for future work, this project has successfully provided the Lamellar development team with a framework for maintaining consistent performance standards and identifying optimization opportunities across workloads and hardware environments.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Report on the AAPM grand challenge on deep generative modeling for learning medical image statistics

Abstract Background The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. Purpose The goal of this challenge was to promote the development of deep generative models for medical imaging and to emphasize the need for their domain‐relevant assessments via the analysis of relevant image statistics. Methods As part of this Grand Challenge, a common training dataset and an evaluation procedure was developed for benchmarking deep generative models for medical image synthesis. To create the training dataset, an established 3D virtual breast phantom was adapted. The resulting dataset comprised about 108 000 images of size 512 512. For the evaluation of submissions to the Challenge, an ensemble of 10 000 DGM‐generated images from each submission was employed. The evaluation procedure consisted of two stages. In the first stage, a preliminary check for memorization and image quality (via the Fréchet Inception Distance [FID]) was performed. Submissions that passed the first stage were then evaluated for the reproducibility of image statistics corresponding to several feature families including texture, morphology, image moments, fractal statistics, and skeleton statistics. A summary measure in this feature space was employed to rank the submissions. Additional analyses of submissions was performed to assess DGM performance specific to individual feature families, the four classes in the training data, and also to identify various artifacts. Results Fifty‐eight submissions from 12 unique users were received for this Challenge. Out of these 12 submissions, 9 submissions passed the first stage of evaluation and were eligible for ranking. The top‐ranked submission employed a conditional latent diffusion model, whereas the joint runners‐up employed a generative adversarial network, followed by another network for image superresolution. In general, we observed that the overall ranking of the top 9 submissions according to our evaluation method (i) did not match the FID‐based ranking, and (ii) differed with respect to individual feature families. Another important finding from our additional analyses was that different DGMs demonstrated similar kinds of artifacts. Conclusions This Grand Challenge highlighted the need for domain‐specific evaluation to further DGM design as well as deployment. It also demonstrated that the specification of a DGM may differ depending on its intended use.

Radiology, Nuclear Medicine & Medical Imaging↗