Search NASA⌕ Search

SEARCH · Search NASA

Results for “model credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Data Fusion for the Development of a Multimodal Freight Transload Facilities Dataset in the U.S.

To withstand the growing demand of commodity volume and its strain on the transportation infrastructure, it is necessary to identify the flow of commodities by route and mode. However, a national multimodal freight routing model does not exist for the U.S. The development of such model requires multiple building blocks, such as virtual representations of roadway, railway, and waterway networks, transload facilities (TFs), and access/egress links. Most of these blocks have a robust database in the U.S., except for the TFs. Here, this paper presents the fusion of dispersed and heterogeneous representations of multimodal TFs into a single, comprehensive, geospatial freight TF dataset. The TF dataset is derived from several sources, including the U.S. Army Corps of Engineers Master Docks Plus, the National Transportation Atlas Database, the Intermodal Association of North America, industry publications, and other public information. First, individual datasets were queried and reconciled. A geocoding/reverse geocoding process was applied to get the best street address and latitude/longitude location for each terminal. Then, duplicate terminals were identified by a fuzzy match algorithm based on terminal name and location, and removed. Validation was performed by visual inspection of random facilities. The main contributions of this work are: a publicly available version of the TF dataset, including facility location and multimodal transfer capability of 9,003 facilities, and an enterprise-version with the same facilities but including commodity handling capabilities. The main purpose of developing the TF dataset is to inform multimodal routing algorithms. The proposed TF dataset allows for credibly modeling the multimodal transfer of commodities within shipment routes.

Commodity Routing↗
DOE OSTI2026

Community-Engaged Modeling of Urban Flood Adaptation Pathways

Climate change is intensifying the hydrologic cycle, leading to more frequent and severe rainfall-driven (pluvial) flooding in urban areas. In the mid-Atlantic US cities, aging and under-designed stormwater infrastructure is increasingly strained by these events, resulting in recurring damage to property and disruptions to transportation networks. In this study, we combine community engagement with hydrologic modeling to develop and evaluate potential urban flood adaptation strategies. Over a three-year period, local technical experts and community representatives met regularly to discuss flooding concerns, identify priorities, and co-develop adaptation strategies. These discussions informed the development of an urban flooding model (EPA Storm Water Management Model) for the Baltimore Harbor watershed, the focus location of this study. The flooding model integrates complex surface and subsurface stormwater infrastructure data, local expert knowledge, and community insights. We simulate stakeholder-prioritized adaptations, such as green and gray infrastructure strategies. Model results demonstrate that enhanced infrastructure maintenance is the most effective adaptation for reducing flood depths, but has varied effects across the watershed, and can increase flooding in some locations. Spatially concentrated greening provides limited benefit to the watershed as a whole, but moderate benefit in community priority areas. Together, these adaptations have the potential to reduce flood depths by as much as 58% in some locations, greatly reducing property damage and mobility impacts, primary concerns of stakeholders. Future work will implement robust optimization tools to search for adaptations which meet stakeholder objectives and perform highly under varied future climate conditions. This work contributes to the expanding literature on collaborative modeling and demonstrates that community-engaged approaches can enhance model credibility and generate more actionable insights for communities seeking to strengthen climate resilience.

Spangler, Ava [Pennsylvania State University] (ORC↗
DOE OSTI2026

Community-Informed Urban Flood Modeling for Impact Mitigation

Climate change is intensifying the hydrologic cycle, leading to more frequent and severe rainfall-driven (pluvial) flooding in urban areas. In the mid-Atlantic US cities, aging and under-designed stormwater infrastructure is increasingly strained by these events, resulting in recurring damage to property and disruptions to transportation networks. In this study, we combine community engagement with hydrologic modeling to develop and evaluate potential urban flood adaptation strategies. Over a three-year period, local technical experts and community representatives met regularly to discuss flooding concerns, identify priorities, and co-develop adaptation strategies. These discussions informed the development of an urban flooding model (EPA Storm Water Management Model) for the Baltimore Harbor watershed, the focus location of this study. The flooding model integrates complex surface and subsurface stormwater infrastructure data, local expert knowledge, and community insights. We simulate stakeholder-prioritized adaptations, such as green and gray infrastructure strategies. Model results demonstrate that enhanced infrastructure maintenance is the most effective adaptation for reducing flood depths, but has varied effects across the watershed, and can increase flooding in some locations. Spatially concentrated greening provides limited benefit to the watershed as a whole, but moderate benefit in community priority areas. Together, these adaptations have the potential to reduce flood depths by as much as 58% in some locations, greatly reducing property damage and mobility impacts, primary concerns of stakeholders. Future work will implement robust optimization tools to search for adaptations which meet stakeholder objectives and perform highly under varied future climate conditions. This work contributes to the expanding literature on collaborative modeling and demonstrates that community-engaged approaches can enhance model credibility and generate more actionable insights for communities seeking to strengthen climate resilience.

Baltimore MD↗
DOE OSTI2021

Comprehensive Material Characterization and Simultaneous Model Calibration for Improved Computational Simulation Credibility

Computational simulation is increasingly relied upon for high-consequence engineering decisions, and a foundational element to solid mechanics simulations is a credible material model. Our ultimate vision is to interlace material characterization and model calibration in a real-time feedback loop, where the current model calibration results will drive the experiment to load regimes that add the most useful information to reduce parameter uncertainty. The current work investigated one key step to this Interlaced Characterization and Calibration (ICC) paradigm, using a finite load-path tree to incorporate history/path dependency of nonlinear material models into a network of surrogate models that replace computationally-expensive finite-element analyses. Our reference simulation was an elastoplastic material point subject to biaxial deformation with a Hill anisotropic yield criterion. Training data was generated using either a space-filling or adaptive sampling method, and surrogates were built using either Gaussian process or polynomial chaos expansion methods. Surrogate error was evaluated to be on the order of 10 ⁻5 and 10 ⁻3 percent for the space-filling and adaptive sampling training data, respectively. Direct Bayesian inference was performed with the surrogate network and with the reference material point simulator, and results agreed to within 3 significant figures for the mean parameter values, with a reduction in computational cost over 5 orders of magnitude. These results bought down risk regarding the surrogate network and facilitated a successful FY22-24 full LDRD proposal to research and develop the complete ICC paradigm.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A framework for strategic discovery of credible neural network surrogate models under uncertainty

The widespread integration of deep neural networks in developing data-driven surrogate models for high-fidelity simulations of complex physical systems highlights the critical necessity for robust uncertainty quantification techniques and credibility assessment methodologies, ensuring the reliable deployment of surrogate models in consequential decision-making. Here, this study presents the Occam Plausibility Algorithm for surrogate models (OPAL-surrogate), providing a systematic framework to uncover predictive neural network-based surrogate models within the large space of potential models, including various neural network classes and choices of architecture and hyperparameters. The framework is grounded in hierarchical Bayesian inferences and employs model validation tests to evaluate the credibility and prediction reliability of the surrogate models under uncertainty. Leveraging these principles, OPAL-surrogate introduces a systematic and efficient strategy for balancing the trade-off between model complexity, accuracy, and prediction uncertainty. The effectiveness of OPAL-surrogate is demonstrated through two modeling problems, including the deformation of porous materials for building insulation and turbulent combustion flow for ablation of solid fuels within hybrid rocket motors.

42 ENGINEERING↗
DOE OSTI2021

Evaluation of the Large-Scale and Regional Climatic Response Across North Africa to Natural Variability in Oceanic Modes and Terrestrial Vegetation Among the CMIP5 Models

Hydrologic variability is a serious threat to the poverty-stricken regions of North Africa. Meanwhile, the scientific community struggles to attribute these extreme climatic episodes to specific oceanic and land drivers. Prior modeling studies have not assessed simulated feedbacks over North Africa against an established observational benchmark, so their results are considered as untested, model-specific findings. The Coupled Model Intercomparison Project Phase Five (CMIP5) archive represents the state-of-the-art in climate projections, with most CMIP5 models now containing interactive vegetation phenology; however, there have been no studies to date of their simulated vegetation feedbacks. The capability of CMIP5 models at accurately simulating the forcing of both sea-surface temperature (SST) and regional leaf area index (LAI) anomalies on North African climate needs to be a key consideration in assessing the models’ overall credibility and determining appropriate model weighting for developing climate projections. The Generalized Equilibrium Feedback Assessment (GEFA) is a promising statistical method, which can be applied to either model output or observations, for isolating the local and remote impacts of individual oceanic or terrestrial forcings on regional climate. We performed a combined observational and modeling assessment of land-ocean-atmosphere interactions across the distinct ecological and moisture gradients of North Africa. We evaluated and demonstrated the reliability of the GEFA statistical method over North Africa using the Community Earth System Model (CESM), applied GEFA to observational data to quantify the observed forcing of ocean basin SST anomalies and regional LAI anomalies on North African climate, evaluated the CMIP5 models’ performance in terms of representing these key observed feedbacks, and formulated CMIP5 feedback performance metrics for weighting North African climate projections. Our study represented the first attempt to separate the observed roles of oceanic and vegetation feedbacks across North Africa, the first systematic assessment and intercomparison of land-ocean-atmosphere feedbacks in CMIP5, and the first exploration of vegetation feedbacks among CMIP5 models. The following work was accomplished. (1) The team generated 14 scientific publications and 30 presentations based on the DOE-funded research. (2) A stepwise version of the GEFA statistical method was developed in which unimportant forcings were dropped to increase the reliability of the results. (3) SGEFA was successfully validated through experiments with CESM in terms of its ability to isolate the atmospheric responses to individual oceanic or land forcings. (4) Observational evidence was revealed for the Sahel’s positive vegetation-rainfall feedback on the seasonal to interannual time scale, and it was attributed to a moisture recycling mechanism rather than an albedo mechanism. (5) An approach with SGEFA was developed in which the individual contributions of soil moisture versus vegetation forcings could be separated, revealing that the former forcing outweighed the latter for sub-Saharan Africa. (6) Tropical ocean temperatures were found to be key regulators of pan-tropical vegetation variability, especially for arid and semi-arid regions, including sub-Saharan Africa. (7) The CMIP5 models largely underestimated the importance of land feedbacks across the Sahel. (8) The general consensus among CMIP5 models indicates, for the late 21st century, a diminished seasonal predictability of sub-Saharan African regional climate and an elevated role of the land surface compared to oceanic drivers in regulating regional climate variability. (9) Land-ocean-atmosphere interactions were demonstrated to be key contributors to the seasonal predictability of African wildfire activity. (10) The El Djouf was determined to be of greater importance than the Bodélé depression in terms of providing trans-Atlantic dust transport to the Americas.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Modeling Interfaces to Support Low-Level Waste Disposal System Performance Assessments - 20366

In low-level waste (LLW) disposal sites, interfaces between cementitious materials, used as waste forms and/or engineered barriers, and the surrounding soil or backfill material are often encountered. Reactions across these interfaces may lead to chemical and structural alteration of the cementitious and backfill materials that may ultimately affect long-term performance. For example, the ingress of carbon dioxide from soil gas into a waste tank concrete shell at the Hanford Site or Savannah River Site (SRS) or a low-level waste disposal vault at SRS leads to carbonation of cement hydration products (e.g. portlandite, calcium-silicate hydrate or C-S-H, and ettringite). The result of carbonation is a decrease in pH in the cement paste portion of the tank wall or vault concrete structure that could ultimately lead to de-passivation of the embedded structural steel and, eventually, to cracking. Cracking can lead to increased ingress of water into the structure, corresponding increased release of constituents of concern, and to subsequent increased transport of these constituents to the surrounding environment. The mobility of trace constituents (e.g., radionuclides of concern) in waste forms may increase in response to changes in pH, pore structure, and mineralogical gradients. Depletion of portlandite results in subsequent decalcification of C-S-H that leads to changes in the cement strength and may lead to structural failure. At the boundary between waste forms and concrete vaults, the migration of sulfate ions from the salt waste form (e.g., the saltstone waste form used for LLW disposal at the Savannah River Site in Saltstone Disposal Units, or SDUs) into the barrier or vault concrete has been predicted to result in expansive mineral phase formation (or 'sulfate attack'). Expansive mineral formation could result in cracking and potential loss of structural integrity in the vault concrete. In the performance assessment (PA) for the SRS SDUs, prediction of both the carbonation and sulfate attack fronts are critical to assessing the long-term performance of these LLW disposal vaults. A general purpose geochemical reactive transport model has been developed in LeachXS/ORCHESTRA to evaluate typical interfaces for LLW disposal environments, including SDUs at SRS. The model may be used to predict the long-term performance of interfaces between cementitious materials and saltstone or backfill materials with respect to primary phases by considering mass transport, cement chemistry, geochemical speciation, and multi-ionic diffusion over interfaces between different materials. The present study compares the performance and interactions between vault concrete and saltstone waste form for a representative SDU scenario. The materials were characterized using pH-dependence leaching test (EPA 1313) and semi-dynamic transport tests (EPA 1315) with test results used to parameterize the interface model. The development of a credible inter model provides an important basis for supporting future PAs where carbonation and sulfate attack are important aging and degradation mechanisms. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Viscous flow properties and hydrodynamic diameter of phenothiazine-based redox-active molecules in different supporting salt environments

We report viscous flow properties of a redox-active organic molecule, N-(2-(2-methoxyethoxy)ethyl)phenothiazine (MEEPT), a candidate for non-aqueous redox flow batteries, and two of its radical cation salts. A microfluidic viscometer enabled the use of small sample volumes in determining viscosity as a function of shear rate and concentration in the non-aqueous solvent, acetonitrile, both with and without supporting salts. All solutions tested show Newtonian behavior over shear rates of up to 30 000 s−1, which was rationalized by scaling arguments for the diffusion-based relaxation time of a single MEEPT molecule without aggregation. Neat MEEPT is flowable but with a large viscosity (412 mPa⋅s at room temperature), which is ∼1000 times larger than that of acetonitrile. MEEPT solutions in acetonitrile have low viscosities; at concentrations up to 0.5 M, the viscosity increases by less than a factor of two. From concentration-dependent viscosity measurements, molecular information was inferred from intrinsic viscosity (hydrodynamic diameter) and the Huggins coefficient (interactions). Model fit credibility was assessed using the Bayesian Information Criterion. It is found that the MEEPT and its charged cations are “flowable” and do not flocculate at concentrations up to 0.5 M. MEEPT has a hydrodynamic diameter of around 8.5 Å, which is almost insensitive to supporting salt and state of charge. This size is comparable to molecular dimensions of single molecules obtained from optimized structures using density functional theory calculations. The results suggest that MEEPT is a promising candidate for redox flow batteries in terms of its viscous flow properties.

Wang, Yilin (ORCID:0000000316102995)↗
DOE OSTI2023

Incorporating Full Elastodynamic Effects and Dipping Fault Geometries in Community Code Verification Exercises for Simulations of Earthquake Sequences and Aseismic Slip (SEAS)

ABSTRACT Numerical modeling of earthquake dynamics and derived insight for seismic hazard relies on credible, reproducible model results. The sequences of earthquakes and aseismic slip (SEAS) initiative has set out to facilitate community code comparisons, and verify and advance the next generation of physics-based earthquake models that reproduce all phases of the seismic cycle. With the goal of advancing SEAS models to robustly incorporate physical and geometrical complexities, here we present code comparison results from two new benchmark problems: BP1-FD considers full elastodynamic effects, and BP3-QD considers dipping fault geometries. Seven and eight modeling groups participated in BP1-FD and BP3-QD, respectively, allowing us to explore these physical ingredients across multiple codes and better understand associated numerical considerations. With new comparison metrics, we find that numerical resolution and computational domain size are critical parameters to obtain matching results. Codes for BP1-FD implement different criteria for switching between quasi-static and dynamic solvers, which require tuning to obtain matching results. In BP3-QD, proper remote boundary conditions consistent with specified rigid body translation are required to obtain matching surface displacements. With these numerical and mathematical issues resolved, we obtain excellent quantitative agreements among codes in earthquake interevent times, event moments, and coseismic slip, with reasonable agreements made in peak slip rates and rupture arrival time. We find that including full inertial effects generates events with larger slip rates and rupture speeds compared to the quasi-dynamic counterpart. For BP3-QD, both dip angle and sense of motion (thrust versus normal faulting) alter ground motion on the hanging and foot walls, and influence event patterns, with some sequences exhibiting similar-size characteristic earthquakes, and others exhibiting different-size events. These findings underscore the importance of considering full elastodynamics and nonvertical dip angles in SEAS models, as both influence short- and long-term earthquake behavior and are relevant to seismic hazard.

Geochemistry & Geophysics↗
DOE OSTI2024

Interlaced Characterization and Calibration (ICC) for Improved Computational Simulation Credibility

Accurate material characterization and model calibration are pivotal for simulations used for high-consequence engineering decisions. Current characterization and calibration methods (1) use simplified test specimen geometries and global data, (2) cannot guarantee that sufficient characterization data is collected for a specific model of interest, (3) provide only mean parameter values with no uncertainty quantification, and (4) are sequential, inflexible, and time-consuming. This work developed a new paradigm—coined Interlaced Characterization and Calibration (ICC)—which drives forward the state-of-the-art in model calibration by bringing together recent advancements into one improved workflow. The ICC paradigm (1) employs tools to efficiently use full-field data to calibrate high-fidelity material models, (2) aligns the data needed with the data collected by adopting an optimal experimental design protocol, (3) provides uncertainty metrics on the calibrated model parameters, and (4) incorporates these advances into a quasi real-time feedback loop. The ICC framework was validated synthetically with both low-fidelity and high-fidelity simulations paired with several different elastoplastic material models, and was also demonstrated experimentally with an aluminum 6061 cruciform exemplar specimen. Results showed that the ICC framework—in which Bayesian optimal experimental design actively guided the experiment— resulted in calibrations with similar or better accuracy than predetermined experiments based on subject matter expertise. Moreover, the ICC framework produced a complete model calibration— with quantified uncertainties on model parameters—in 1 week, a 5 - 10× increase in efficiency over traditional approaches. Thus, the ICC paradigm improves both the calibration process and quality, by (1) improving efficiency, which increases agility of solid mechanics modeling and enables utilization of computational simulation (CompSim) at earlier stages of the design cycle and (2) providing quantified, and in some cases reduced, parameter uncertainties, which increases confidence in model predictions and supports credible decision making.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Latent Pitfalls in Microstructure-Based Modeling for Thermally Aged 9Cr-1Mo-V Steel (Grade 91)

A case study was conducted on a mechanistic model development that predicted tensile strength deterioration with thermal aging of 9Cr-1Mo-V steel in supporting the 60-year design life expected for advanced nuclear reactors. For property prediction beyond practical testing times, mechanistic modeling is highly desired, as it taps into the physics of structure–property relationships and therefore can generate reliable results for extrapolation. Meanwhile, as mechanistic models are often complicated, reflecting the intricacy of microstructure and strengthening mechanisms, pitfalls that are difficult to detect often exist. Here, this paper discusses latent pitfalls that are common in mechanistic modeling or specific in this 9Cr-1Mo-V case development through using the American Society of Mechanical Engineers verification and validation in computational solid mechanics (ASME V&V 10) standard for evaluating credibility of modeling in materials engineering. Suggestions are also made for enhancing reliability of microstructure-based modeling.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Comparative Assessment of Data-driven Process Models in Health Information Technology

Process mining for conformance analysis consists of comparing a reference process model against a data-driven process model generated via log files from information technology systems. However, in the absence of a complete reference process model, we found no suggested approaches in the literature to address the need for evaluating process conformance among different healthcare facilities to assess standardization of care. Our goal is to find similarities and dissimilarities in data-driven process models among US Veterans Health Administration (VHA) facilities that can be indicative of patient safety issues. Our hypothesis was that the analysis would not produce statistically significant differences in outcome. We present a unique implementation of conformance analysis in process mining that consists of combining process mining, process mapping and statistical metrics. We illustrate our approach by applying it to the analysis of two clinical radiology order process models generated from healthcare data provided by two similar facilities in the VHA. The comparative assessment showed that about 70% of the orders completed successfully and 30% were not completed due to policy and duplications. Our analysis found a good statistical correlation between both facilities, as the Spearman’s correlation coefficient between facilities for the frequency of cases per total hours was 0.87879, for the frequency of cases by state transition was 0.79702 and for the throughput time per state transition was 0.63582. Additional statistical analyses using the Mann-Whitney U test and the root mean square error both produced values that were not significant. The foregoing approach validated our hypothesis by demonstrating a good statistical correlation of data describing the flow of clinical radiology orders absent a credible reference model. Finding good agreement between both facilities was important in confirming that the clinical orders flow in a similar manner, suggesting standardization of care.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

The development of a high-resolution Eulerian radiation-hydrodynamics simulation capability for laser-driven Hohlraums

Hohlraums are hollow cylindrical cavities with high-Z material walls used to convert laser energy into uniform x-ray radiation drives for inertial confinement fusion capsule implosions and high energy density physics experiments. Credible computational modeling of hohlraums requires detailed modeling and coupling of laser physics, hydrodynamics, radiation transport, heat transport, and atomic physics. We report on improvements to Los Alamos National Laboratory's xRAGE radiation-hydrodynamics code in order to enable hohlraum modeling. xRAGE's Eulerian hydrodynamics and adaptive mesh refinement make it uniquely well suited to study the impacts of multiscale features in hohlraums. In order to provide confidence in this new modeling capability, we demonstrate xRAGE's ability to produce reasonable agreement with data from several benchmark hohlraum experiments. We also use xRAGE to perform integrated simulations of a recent layered high density carbon capsule implosion on the National Ignition Facility in order to evaluate the potential impacts of the capsule support tent, mixed cell conductivity methodologies, plasma transport, and cross-beam energy transfer (XBT). We find that XBT, seeded by plasma flows in the laser entrance hole (LEH), causes a slight decrease in energy coupling to the capsule and that all of these impact the symmetry of the x-ray drive such that they have an appreciable impact on the capsule implosion shape.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Structural uncertainty assessment for fire-engulfed objects in crosswind: Establishing credibility for a multiphysics wall-modeled large-eddy simulation paradigm

A structural uncertainty validation study for a large-scale, fire-engulfed, elevated object subjected to crosswind is presented to establish the credibility of a high-fidelity, low-Mach, turbulent reacting flow wall-modeled large-eddy simulation (WMLES) approach that includes multiphysics coupling to participating media radiation and conjugate heat transfer. To establish that WMLES can accurately predict surface quantities including drag and pressure coefficient in the low-Mach crosswind regime, a foundational elevated isothermal cylinder validation case is presented at a similar gap-to-diameter ratio of 0.25, spanning the subcritical to supercritical drag regime (Re 𝐷 = 1.1 × 10 5 and 4.3 × 10 5 , respectively). Here, this study exercised both static and dynamic coefficient LES (Smagorinsky and 𝑘 sgs ) with both local and exchange-based velocity sampling. Results showcase that the drag crisis (or the sudden drop in drag coefficient at increased Re 𝐷 ) is well captured when using an exchange-based dynamic coefficient WMLES methodology, while noting lack of mesh convergence and overall drag and pressure coefficient predictively when using a static coefficient, local velocity sampling WMLES. For the 𝒪⁡(10) m JP-8 liquid pool fire crosswind validation study presented, two experimental crosswind configurations (2 m/s and 9.5 m/s) are showcased for a fire-engulfed mock fuselage roughly 4 m in diameter. Using the best model-form practices identified in the isothermal study, dynamic coefficient 𝑘 sgs exchange-based WMLES fire validation findings demonstrate accurate peak irradiation and skin temperature predictions as a function of crosswind magnitude. Excessive yaw in the low-crosswind fuselage configuration, consistent with experimental findings, captured a significant predicted asymmetry in flame attachment and heat flux toward the downwind cylindrical cap—indicative of axial vortex structures transporting the flame along the upper and lower fuselage leeward surface. All fire mesh resolution simulations captured the experimental finding that as crosswind increased, predicted flame shape and peak irradiation magnitude onto the fuselage transitioned from a windward to a leeward cylinder location due to the migration of the upper- to lower-shear fuel/air mixing layer thereby demonstrating the novelty, significance, and credibility of this high-fidelity WMLES reacting flow framework.

Domino, Stefan Paul [Sandia National Laboratories ↗
DOE OSTI2025

Recommendations for Comprehensive and Independent Evaluation of Machine Learning‐Based Earth System Models

Abstract Machine learning (ML) is a revolutionary technology with demonstrable applications across multiple disciplines. Within the Earth science community, ML has been most visible for weather forecasting, producing forecasts that rival modern physics‐based models. Given the importance of deepening our understanding and improving predictions of the Earth system on all time scales, efforts are now underway to develop Earth‐system models (ESMs) capable of representing all components of the coupled Earth system (or their aggregated behavior) and their response to external changes over long timescales. Building trust in ESMs is a much more difficult problem than for weather forecast models, not least because the model must represent the alternate (e.g., future or paleoclimatic) coupled states of the system for which there are no direct observations. Given that the physical principles that enable predictions about the response of the Earth system are often not explicitly coded in these ML‐based models, demonstrating the credibility of ML‐based ESMs thus requires us to build evidence of their consistency with the physical system. To this end, this paper puts forward five recommendations to enhance comprehensive, standardized, and independent evaluation of ML‐based ESMs to strengthen their credibility and promote their wider use.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

HOLISTIC CREDIBILITY ASSESSMENT OF A MACHINE LEARNING-BASED SURROGATE MODEL IN AN AERODYNAMICS APPLICATION

Elements of ASME V&V 20 credibility assessment methodology are used in tandem with credibility assessment tools from the machine learning community to assess the credibility of a deep neural network based surrogate model. This surrogate model is trained, tested, validated, and employed in the context of aerodynamic coefficient prediction for a NACA 0012 airfoil in subsonic and transonic flow. The parameter space is defined by angle of attack, Reynolds number, and Mach number.

Kirsch, Jared Roelof [Sandia National Laboratories↗
DOE OSTI2023

Future precipitation increase constrained by climatological pattern of cloud effect

The fractional increase in global mean precipitation (?P ¯/P ¯) is a first-order measure of the hydrological cycle intensification under anthropogenic warming. However, ?P ¯/P ¯ varies by a factor of more than three among model projections, hindering credible assessments of the associated climate impacts. The uncertainty in ?P ¯/P ¯ stems from uncertainty in both hydrological sensitivity (global mean precipitation increase per unit warming) and climate sensitivity (global mean temperature increase per forcing). Here, by investigating hydrological and climate sensitivities in a unified surface-energy-balance perspective, we find that both sensitivities are significantly correlated with surface shortwave cloud feedback, which is further linked to the climatological pattern of cloud shortwave effect. The observed pattern of cloud effect thus constrains both sensitivities and consequently constrains ?P ¯/P ¯. The 5%-95% uncertainty range of ?P ¯/P ¯ from 1979-2005 to 2080-2100 under the high-emission (moderate-emission) scenario is constrained from 6.34±3.53% (4.19±2.28%) in the raw ensemble-model projection to 7.03±2.59% (4.63±1.71%). The constraint thus suggests a higher most-likely ?P ¯/P ¯ and reduces the uncertainty by ~25%, providing valuable information for impact assessments.

Zhou, Wenyu↗