Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

SU-D-BRB-02: Patient-Specific Rectal Toxicity Predictor Based Plan Quality Control for Prostate Stereotactic Body Radiation Therapy (SBRT)

Purpose: To develop a patient-specific rectal toxicity predictor guided plan quality control tool for prostate SBRT plans. Methods: For prostate SBRT cases, four segments of rectal walls including peri-prostatic anterior rectal wall, peri-prostatic lateral rectal walls, peri-prostatic posterior rectal wall and rectum superior to prostate are identified as organs at risk and the circumference of rectal wall receiving more than 39 Gy (CRW39) and 24 Gy (CRW24) are rectal toxicity predictors. In this new geometry-dosimetry model, a patient geometry descriptor, differential circumference of rectal wall (dCRW) is used as model input geometry parameters and plan dosimetric endpoints CRW39 and CRW24 are output dosimetric parameters. Linear models are built to correlate dCRW to both CRW39 and CRW24 and established with both a linear regression method and a modified bagging ensemble machine learning method. 27 SBRT prostate cases are retrospectively studied from a dose-escalated clinical trial research. 20 prescribed 50 Gy SBRT cases are recruited to train the model and the other rescaled 7 cases are used to evaluated model feasibility and accuracy. Results: Each solved linear coefficient sequence related to CRW39 or CRW24 is a one-dimensional decreasing function of the distance from the PTV boundary, indicating that the different locations of each rectal circumference have different contributions to each particular dosimetric endpoint. The fitting errors for those trained 20 prostate SBRT cases are small with mean values of 2.39%, 2.45% relative to the endpoint values for SBRT rectal toxicity predictor CRW39 and CRW24 respectively. 1 out of 7 evaluation plans is identified as poor quality plan. After re-planning, the CRW39 and CRW24 can be reduced by 3.34% and 3%, without sacrificing PTV coverage. Conclusion: The proposed patient geometry-plan toxicity predictor model for SBRT plans can be successfully applied to plan quality control for prostate SBRT cases.

60 APPLIED LIFE SCIENCES↗

Incorporating the Acclimation of Photosynthesis and Leaf Respiration in the Noah‐MP Land Surface Model: Model Development and Evaluation

Abstract Realistic simulation of leaf photosynthetic and respiratory processes is needed for accurate prediction of the global carbon cycle. These two processes systematically acclimate to long‐term environmental changes by adjusting photosynthetic and respiratory traits (e.g., the maximum photosynthetic capacity at 25°C ( V cmax,25 ) and the leaf respiration rate at 25°C ( R 25 )) following increasingly well‐understood principles. While some land surface models (LSMs) now account for thermal acclimation, they do so by assigning empirical parameterizations for individual plant functional types (PFTs). Here, we have implemented an Eco‐Evolutionary Optimality (EEO)‐based scheme to represent the universal acclimation of photosynthesis and leaf respiration to multiple environmental effects, and that therefore requires no PFT‐specific parameterizations, in a standard version of the widely used LSM, Noah MP. We evaluated model performance with plant trait data from a 5‐year experiment and extensive global field measurements, and carbon flux measurements from FLUXNET2015. We show that observed R 25 and V cmax,25 vary substantially both temporally and spatially within the same PFT ( C.V. >20%). Our EEO‐based scheme captures 62% of the temporal and 70% of the spatial variations in V cmax,25 (73% and 54% of the variations in R 25 ). The standard scheme underestimates gross primary production by 10% versus 2% for the EEO‐based scheme and generates a larger spread in r (correlation coefficient) across flux sites (0.79 ± 0.16 vs. 0.84 ± 0.1, mean ± S.D.). The standard scheme greatly overestimates canopy respiration (bias: ∼200% vs. 8% for the EEO scheme), resulting in less CO 2 uptake by terrestrial ecosystems. Our approach thus simulates climate‐carbon coupling more realistically, with fewer parameters.

Ren, Yanghang [Department of Earth System Science ↗

Normal Tissue Complication Probability Modeling of Radiation-Induced Hypothyroidism After Head-and-Neck Radiation Therapy

Purpose: To determine the dose-response relationship of the thyroid for radiation-induced hypothyroidism in head-and-neck radiation therapy, according to 6 normal tissue complication probability models, and to find the best-fit parameters of the models. Methods and Materials: Sixty-five patients treated with primary or postoperative radiation therapy for various cancers in the head-and-neck region were prospectively evaluated. Patient serum samples (tri-iodothyronine, thyroxine, thyroid-stimulating hormone [TSH], free tri-iodothyronine, and free thyroxine) were measured before and at regular time intervals until 1 year after the completion of radiation therapy. Dose-volume histograms (DVHs) of the patients' thyroid gland were derived from their computed tomography (CT)-based treatment planning data. Hypothyroidism was defined as increased TSH (subclinical hypothyroidism) or increased TSH in combination with decreased free thyroxine and thyroxine (clinical hypothyroidism). Thyroid DVHs were converted to 2 Gy/fraction equivalent doses using the linear-quadratic formula with {alpha}/{beta} = 3 Gy. The evaluated models included the following: Lyman with the DVH reduced to the equivalent uniform dose (EUD), known as LEUD; Logit-EUD; mean dose; relative seriality; individual critical volume; and population critical volume models. The parameters of the models were obtained by fitting the patients' data using a maximum likelihood analysis method. The goodness of fit of the models was determined by the 2-sample Kolmogorov-Smirnov test. Ranking of the models was made according to Akaike's information criterion. Results: Twenty-nine patients (44.6%) experienced hypothyroidism. None of the models was rejected according to the evaluation of the goodness of fit. The mean dose model was ranked as the best model on the basis of its Akaike's information criterion value. The D{sub 50} estimated from the models was approximately 44 Gy. Conclusions: The implemented normal tissue complication probability models showed a parallel architecture for the thyroid. The mean dose model can be used as the best model to describe the dose-response relationship for hypothyroidism complication.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Structure of ducted particle-laden turbulent jets

New and existing measurements have been used to develop and evaluate models of particle-laden jets and sprays, yielding encouraging results for a stochastic separated flow (SSF) model which treats effects of interphase slip and turbulence on the basis of random walk computations of particle or drop motion. An evaluation of the SSF model using the measurements of Modarress et al. (1982, 1983) yielded results comparable to earlier work, suggesting that data from coflowing jets in ducts should be accompanied by static pressure measurements, since even small pressure gradients appreciably influence jet properties.

Zhang, Q.-F.↗

Model and Analytic Processes for Export License Assessments

This paper represents the Department of Energy Office of Nonproliferation Research and Development (NA-22) Simulations, Algorithms and Modeling (SAM) Program's first effort to identify and frame analytical methods and tools to aid export control professionals in effectively predicting proliferation intent; a complex, multi-step and multi-agency process. The report focuses on analytical modeling methodologies that alone, or combined, may improve the proliferation export control license approval process. It is a follow-up to an earlier paper describing information sources and environments related to international nuclear technology transfer. This report describes the decision criteria used to evaluate modeling techniques and tools to determine which approaches will be investigated during the final 2 years of the project. The report also details the motivation for why new modeling techniques and tools are needed. The analytical modeling methodologies will enable analysts to evaluate the information environment for relevance to detecting proliferation intent, with specific focus on assessing risks associated with transferring dual-use technologies. Dual-use technologies can be used in both weapons and commercial enterprises. A decision-framework was developed to evaluate which of the different analytical modeling methodologies would be most appropriate conditional on the uniqueness of the approach, data availability, laboratory capabilities, relevance to NA-22 and Office of Arms Control and Nonproliferation (NA-24) research needs and the impact if successful. Modeling methodologies were divided into whether they could help micro-level assessments (e.g., help improve individual license assessments) or macro-level assessment. Macro-level assessment focuses on suppliers, technology, consumers, economies, and proliferation context. Macro-level assessment technologies scored higher in the area of uniqueness because less work has been done at the macro level. An approach to developing testable hypotheses for the macro-level assessment methodologies is provided. The outcome of this works suggests that we should develop a Bayes Net for micro-level analysis and continue to focus on Bayes Net, System Dynamics and Economic Input/Output models for assessing macro-level problems. Simultaneously, we need to develop metrics for assessing intent in export control, including the risks and consequences associated with all aspects of export control.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

PDRD (SR13046) Tritium Production Final Report

Utilizing the results of Texas A&M University (TAMU) senior design projects on tritium production in four different small modular reactors (SMR), the Savannah River National Laboratory’s (SRNL) developed an optimization model evaluating tritium production versus uranium utilization under a FY2013 plant directed research development (PDRD) project. The model is a tool that can evaluate varying scenarios and various reactor designs to maximize the production of tritium per unit of unobligated United States (US) origin uranium that is in limited supply.

07 ISOTOPE AND RADIATION SOURCES↗

Software enhancements to the IVSEM model of the CTBTO IMS.

Sandia National Laboratories (SNL) developed the Integrated Verification System Evaluation Model (IVSEM) to estimate the performance of the International Monitoring System (IMS) operated by the Comprehensive Nuclear Test Ban Treaty Organization (CTBTO). IVSEM was developed in several phases between 1995 and 2000. The model was developed in FORTRAN with an IDL-based user interface and was compiled for Windows and UNIX operating systems. Continuing interest in this analysis capability, coupled with numerous advances in desktop computer hardware and software since IVSEM was written, enabled significant improvements to IVSEM run-time performance and data analysis capabilities. These improvements were implemented externally without modifying the FORTRAN executables, which had been previously verified. This paper describes the parallelization approach developed to significantly reduce IVSEM run-times and the new test setup and analysis tools developed to facilitate better IVSEM operation.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Moving beyond the total sea ice extent in gauging model biases

Here, reproducing characteristics of observed sea ice extent remains an important climate modeling challenge. This study describes several approaches to improve how model biases in total sea ice distribution are quantified, and applies them to historically forced simulations contributed to phase 5 of the Coupled Model Intercomparison Project (CMIP5). The quantity of hemispheric total sea ice area, or some measure of its equatorward extent, is often used to evaluate model performance. A new approach is introduced that investigates additional details about the structure of model errors, with an aim to reduce the potential impact of compensating errors when gauging differences between simulated and observed sea ice. Using multiple observational datasets, several new methods are applied to evaluate the climatological spatial distribution and the annual cycle of sea ice cover in 41 CMIP5 models. It is shown that in some models, error compensation can be substantial, for example resulting from too much sea ice in one region and too little in another. Error compensation tends to be larger in models that agree more closely with the observed total sea ice area, which may result from model tuning. The results herein suggest that consideration of only the total hemispheric sea ice area or extent can be misleading when quantitatively comparing how well models agree with observations. Further work is needed to fully develop robust methods to holistically evaluate the ability of models to capture the finescale structure of sea ice characteristics; however, the “sector scale” metric used here aids in reducing the impact of compensating errors in hemispheric integrals.

54 ENVIRONMENTAL SCIENCES↗

A critical evaluation of the upper ocean heat budget in the Climate Forecast System Reanalysis data for the south central equatorial Pacific

Coupled ocean-atmospheric models suffer from the common bias of a spurious rain belt south of the central equatorial Pacific throughout the year. Observational constraints on key processes responsible for this bias are scarce. The recently available reanalysis from a coupled model system for the National Centers for Environmental Prediction (NCEP) Climate Forecast System Reanalysis (CFSR) data is a potential benchmark for climate models in this region. Its suitability for model evaluation and validation, however, needs to be established. This paper examines the mixed layer heat budget and the ocean surface currents - key factors for the sea surface temperature control in the double Inter-Tropical Convergence Zone in the central Pacific - from 5{sup o}S to 10{sup o}S and 170{sup o}E to 150{sup o}W. Two independent approaches are used. The first approach is through comparison of CFSR data with collocated station observations from field experiments; the second is through the residual analysis of the heat budget of the mixed layer. We show that the CFSR overestimates the net surface flux in this region by 23 W m{sup -2}. The overestimated net surface flux is mainly due to an even larger overestimation of shortwave radiation by 44 W m{sup -2}, which is compensated by a surface latent heat flux overestimated by 14 W m{sup -2}. However, the quality of surface currents and the associated oceanic heat transport in CFSR are not compromised by the surface flux biases, and they agree with the best available estimates. The uncertainties of the observational data from field experiments are also briefly discussed in the present study.

54 ENVIRONMENTAL SCIENCES↗

Reliability estimation of an advanced nuclear fuel using coupled active learning, multifidelity modeling, and subset simulation

Tristructural isotropic (TRISO)-coated particle fuel is a robust nuclear fuel and determining its reliability is critical for the success of advanced nuclear technologies. However, TRISO failure probabilities are small and the associated computational models are expensive. Here we used coupled active learning, multifidelity modeling, and subset simulation to estimate the failure probabilities of TRISO fuels using several 1D and 2D models. With multifidelity modeling, we replaced expensive high-fidelity (HF) model evaluations with information fusion from two low-fidelity (LF) models. For the 1D TRISO models, we considered three multifidelity modeling strategies: only Kriging, Kriging LF prediction plus Kriging correction, and deep neural network (DNN) LF prediction plus Kriging correction. While the results across these multifidelity modeling strategies compared satisfactorily, strategies employing information fusion from two LF models called the HF model least often. Next, for the 2D TRISO model, we considered two multifidelity modeling strategies: DNN LF prediction plus Kriging correction (data-driven) and 1D TRISO LF prediction plus Kriging correction (physics-based). The physics-based strategy, as expected, consistently required the fewest calls to the HF model. However, the data-driven strategy had a lower overall simulation time since the DNN predictions are instantaneous, and the 1D TRISO model requires a non-negligible simulation time.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

The Monolithic Heat Pipe Microreactor Reference Plant Model Updates

This work presents the latest improvements to, and investigations performed with, the generic monolithic heat-pipe-cooled microreactor reference plant model for the United States Nuclear Regulatory Commission. This model serves as the foundation for the future detailed design evaluation models based on license applications. This model has been developed with the Comprehensive Reactor Analysis Bundle (BlueCRAB) and its specifications are based on open literature publications for the eVinci™ design . BlueCRAB is the U.S. Nuclear Regulatory Commission non-light-water reactor analysis system based on MOOSE, the Multiphysics Object-Oriented Simulation Environment framework, which can couple the Griffin, BISON, and Sockeye applications to resolve the various physics that are essential for the safety analysis of this type of reactor system. The core specifications include tristructural isotropic fuel, graphite monolith, graphite reflectors, and drums composed of graphite and B 4 C.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Optimal control of a low wing-loading STOL aircraft

Linear optimal quadratic control theory is applied to a low wing-loading STOL aircraft for ride quality and flight path following. Design criteria include minimum rms response to wind turbulence and desired transient response characteristics. Design techniques include proper choosing of design versus evaluation models, choosing appropriate performance index responses, and use of classical evaluation techniques. Results are obtained through a combination of frequency response shaping and gust observation. Effects of control rate and authority saturation are examined with a new rapid calculation of random input describing functions. Parameter sensitivity is also evaluated using a Liapunov type matrix equation.

Cunningham, T. B.↗

Classification of Clouds and Deep Convection from GEOS-5 Using Satellite Observations

With the increased resolution of global atmospheric models and the push toward global cloud resolving models, the resemblance of model output to satellite observations has become strikingly similar. As we progress with our adaptation of the Goddard Earth Observing System Model, Version 5 (GEOS-5) as a high resolution cloud system resolving model, evaluation of cloud properties and deep convection require in-depth analysis beyond a visual comparison. Outgoing long-wave radiation (OLR) provides a sufficient comparison with infrared (IR) satellite imagery to isolate areas of deep convection. We have adopted a binning technique to generate a series of histograms for OLR which classify the presence and fraction of clear sky versus deep convection in the tropics that can be compared with a similar analyses of IR imagery from composite Geostationary Operational Environmental Satellite (GOES) observations. We will present initial results that have been used to evaluate the amount of deep convective parameterization required within the model as we move toward cloud system resolving resolutions of 10- to 1-km globally.

Putman, William↗

MUlti-SpEctral, MUlti-SpEcies, MUlti-SatEllite (MUSES) Retrieval Algorithm: Towards Extending Multi-Decadal NASA EOS Atmospheric Composition Data Records

Multi-Spectra, Multi-Species, Multi-Sensors (MUSES): Builds off of heritage from the Tropospheric Emission Spectrometer (TES) optimal estimation (OE) algorithm to combine a priori and satellite data, including rigorous error analysis diagnostics and observation operators needed for trend analysis, climate model evaluation, and data assimilation; has generic design to incorporate forward model radiances from hyperspectral measurements from multiple sensors into the joint retrieval algorithm.

Fu, Dejian↗

Numerical Reanalyses as a Gateway to Arctic Synthesis

Reanalyses are regularly gridded, retrospective depictions of the physical earth system, which are produced through the correction of a short-term forecast to available observations. In the Arctic, reanalyses are particularly well suited to marshal the sparse observing network to provide a plausible, multivariate representation of conditions. Atmospheric reanalyses such as MERRA-2 (NASA Modern-Era Retrospective analysis for Research and Applications, version 2) and ocean reanalyses such as SODA3 (Univ. Maryland Simple Ocean Data Assimilation version 3) are widely used in Arctic research for diagnostic studies of circulation, model evaluation, and as boundary conditions for a variety of process models. Here, we provide examples that illustrate the utility of reanalyses for providing information on the spatial and temporal scales of recent, rapid changes in the Arctic. Recent trends in Arctic surface temperatures, surface melt over Greenland and Arctic glaciers, and evolving freshwater conditions in the Arctic Ocean are examples where reanalyses can provide information that cannot easily be obtained via other means. These examples provide information on the scale, magnitude, and the uncertainty of recent Arctic change and provide a context for future scenarios. We further quantify uncertainties in key reanalyses variables and approaches for addressing these issues.

Arctic↗

Enhancing segmentation fairness through curriculum learning and progressive loss: a centralized and federated perspective on radiograph analysis

Bias in medical image segmentation can lead to unequal performance across demographic subgroups, raising concerns about fairness and reliability in clinical AI systems. While deep learning models have achieved high segmentation accuracy, ensuring equitable performance across race and gender remains a significant challenge, particularly in privacy-sensitive healthcare environments. This study investigates fairness-aware medical image segmentation for hip and knee radiographs using deep learning models evaluated in both centralized and Federated Learning (FL) settings. We introduce Curriculum Learning (CL) strategies and Progressive Loss (PL) functions to regulate sample difficulty during training. In addition, we propose two novel fairness-oriented federated learning algorithms, Federated Intersection over Union (FedIoU) and Federated Intersection over Union with Outlier Analysis (FedIoUoutlier). Experiments are conducted using multiple segmentation backbones and simulated multi-site data partitions derived from the Osteoarthritis Initiative dataset. Model performance is evaluated using Intersection over Union (IoU), IoU standard deviation, Skewed Error Ratio (SER), and Min-Max Disparity across race and gender subgroups. Statistical significance was verified using paired t-tests to compare per-sample IoU performance against baseline configurations. Across both hip and knee segmentation tasks, curriculum learning and progressive loss strategies consistently improved segmentation accuracy and reduced demographic performance disparities in centralized training. In federated settings, fairness-aware aggregation further enhanced performance. Notably, FedIoUoutlier combined with balanced curriculum learning and tiered progressive loss achieved the highest mean IoU while yielding the lowest SER and Min-Max Disparity, indicating improved fairness without sacrificing accuracy. In several configurations, federated models matched or exceeded the performance of optimized centralized models, with statistically significant improvements in per-sample IoU over baseline configurations. The results demonstrate that structured training strategies and fairness-aware federated aggregation can jointly improve accuracy, stability, and demographic fairness in medical image segmentation. By integrating curriculum learning, progressive loss, and novel FL algorithms, this work provides a practical pathway toward equitable and privacy-preserving AI systems for medical imaging.

97 MATHEMATICS AND COMPUTING↗

General Multifidelity Surrogate Models: Framework and Active-Learning Strategies for Efficient Rare Event Simulation

Estimating the probability of failure for complex real-world systems using high-fidelity computational models is often prohibitively expensive, especially when the probability is small. Exploiting low-fidelity models can make this process more feasible, but merging information from multiple low-fidelity and high-fidelity models poses several challenges. Here, this paper presents a robust multi-fidelity surrogate modeling strategy in which the multi-fidelity surrogate is assembled using an active learning strategy using an on-the-fly model adequacy assessment set within a subset simulation framework for efficient reliability analysis. The multi-fidelity surrogate is assembled by first applying a Gaussian process correction to each low-fidelity model and assigning a model probability based on the model's local predictive accuracy and cost. Three strategies are proposed to fuse these individual surrogates into an overall surrogate model based on model averaging and deterministic/stochastic model selection. The strategies also dictate which model evaluations are necessary. No assumptions are made about the relationships between low-fidelity models, while the high-fidelity model is assumed to be the most accurate and most computationally expensive model. Through two analytical and two numerical case studies, including a case study evaluating the failure probability of Tristructural isotropic-coated (TRISO) nuclear fuels, the algorithm is shown to be highly accurate while drastically reducing the number of high-fidelity model calls (and hence computational cost).

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Transport model comparison studies of intermediate-energy heavy-ion collisions

Transport models are the main method to obtain physics information on the nuclear equation of state and in-medium properties of particles from low to relativistic-energy heavy-ion collisions. The Transport Model Evaluation Project (TMEP) has been pursued to test the robustness of transport model predictions in reaching consistent conclusions from the same type of physical model. To this end, calculations under controlled conditions of physical input and set-up were performed with various participating codes. These included both calculations of nuclear matter in a box with periodic boundary conditions, which test separately selected ingredients of a transport code, and more realistic calculations of heavy-ion collisions. Over the years, six studies have been performed within this project. In this intermediate review, we summarize and discuss the present status of the project. We also provide condensed descriptions of the 26 participating codes, which contributed to some part of the project. These include the major codes in use today. After a compact description of the underlying transport approaches, we review the main results of the studies completed so far. They show, that in box calculations the differences between the codes can be well understood and a convergence of the results can be reached. These studies also highlight the systematic differences between the two families of transport codes, known under the names of Boltzmann–Uehling–Uhlenbeck (BUU) and Quantum Molecular Dynamics (QMD) type codes. However, when the codes were compared in full heavy-ion collisions using different physical models, as recently for pion production, they still yielded substantially different results. This calls for further comparisons of heavy-ion collisions with controlled models and of box comparisons of important ingredients, like momentum-dependent fields, which are currently underway. Furthermore, our evaluation studies often indicate improved strategies in performing transport simulations and thus can provide guidance to code developers. Results of transport simulations of heavy-ion collisions from a given code will have more significance if the code can be validated against benchmark calculations such as the ones summarized in this review.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗