Search NASA⌕ Search

SEARCH · Search NASA

Results for “model performance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

A High-Pressure Shock Tube Study of Hydrogen and Ammonia Addition to Natural Gas for Reduced Carbon Emissions in Power Generation Gas Turbines

Ignition delay times from undiluted mixtures of natural gas (NG)/H2/Air and NG/NH3/Air were measured using a high-pressure shock tube at the University of Central Florida. The combustion temperatures were experimentally tested between 1000-1500 K near a constant pressure of 25 bar. Mixtures were kept undiluted to replicate the same chemistry pathways seen in gas turbine combustion chambers. Recorded combustion pressures exceeded 200 bar due to the large energy release, hence why these were performed at the high-pressure shock tube facility. The data is compared to the predictions of the NUIGMech 1.1 mechanism for chemical kinetic model validation and refinement. An exceptional agreement was shown for stoichiometric conditions in all cases but strayed at lean and rich equivalence ratios, especially in the lower temperature regime of H2 addition and all temperature ranges of the baseline NG mixture. Hydrogen addition also decreased ignition delay times by nearly 90%, while NH3 fuel addition made no noticeable difference in ignition delay time. NG/NH3 exhibited similar chemistry to pure NG under the same conditions, which is shown in a sensitivity analysis, demonstrating hydrogen chemistry to be dominant in NG/H2 mixtures and hydrocarbon chemistry to be dominant in NG/NH3 mixtures. The reaction CH3 + O2 = CH3O + O is identified and suggested as a possible modification target to improve model performance. Increasing the robustness of chemical kinetic models via experimental validation will directly aid in designing next-generation combustion chambers for use in gas turbines, which in turn will greatly lower global emissions and reduce greenhouse effects.

Pierro, Michael↗

Uncertain role of clouds in shaping summertime atmosphere-sea ice connections in reanalyses and CMIP6 models

Downwelling longwave radiation (DLR) driven by the atmospheric and cloud conditions in the troposphere is suggested to be a dominant factor to determine the summertime net surface energy budget over the Arctic Ocean and thus plays a key role to shape the September sea ice. We use reanalyses and the self-organizing map (SOM) method to distinguish CMIP6 model performance in replicating the observed strong atmosphere-DLR connection. We find all models can reasonably simulate the linkage between key atmosphere variables and the clear sky DLR but behave differently in replicating the atmosphere-DLR connection due to cloud forcing. In ERA5 and strongly coupled models, tropospheric high pressure is associated with decreased clouds in the mid- and high-levels and increased clouds near the surface. This out-of-phase structure indicates that DLR cloud forcing is nearly neutral, making the clear sky DLR more important to bridge JJA circulation to late-summer sea ice. In MERRA-2 and weakly coupled models, tropospheric clouds display a vertically homogeneous reduction; the cloud DLR is thus strongly reduced due to the cooling effect, which partially cancels out the clear sky DLR and makes the total DLR less efficient to translate circulation forcing to sea ice. The differences of cloud vertical distribution in CMIP6 appear to be differentiated by circulation related relative humidity. Therefore, a better understanding of the discrepancy of different reanalyses and remote sensing products is critical to comprehensively evaluate simulated interactions among circulation, clouds, sea ice and energy budget at the surface in summer.

54 ENVIRONMENTAL SCIENCES↗

Dissipation Scaled Internal Wave Drag in a Global Heterogeneously Coupled Internal/External Mode Total Water Level Model

This study showcases a global, heterogeneously coupled total water level system wherein salinity and temperature outputs from a coarser-resolution (~12 km) ocean general circulation model are used to calculate density-driven terms within a global, higher-resolution (~2.5 km) depth-averaged total water level model. We demonstrate that the inclusion of baroclinic forcing in the barotropic model requires modification of the internal wave drag term to prevent excess degradation of tidal results compared to the barotropic model. By scaling the internal tide dissipation by an easy to calculate dissipation ratio, the resulting heterogeneously coupled model has complex root mean square errors (RMSE) of 2.27 cm in the deep ocean and 12.16 cm in shallow waters for the M 2 tidal constituent. While this represents a 10%–20% deterioration as compared to the barotropic model, the improvements in total water level prediction more than offset this degradation. Global median RMSE compared to observations of total water levels, 30-day sea levels, and non-tidal residuals improve by 1.86 (18.5%), 2.55 (42.5%), and 0.36 (5.3%) cm respectively. The drastic improvement in model performance highlights the importance of including density-driven effects within global hydrodynamic models and will help to improve the results of both hindcasts and forecasts in modeling extreme and nuisance flooding. With only an 11% increase in model run time compared to the fully barotropic total water level model, this approach paves the way for high resolution coastal water level and flood models to be used alongside climate models, improving operational forecasting of total water levels.

Blakely, Coleman Peter [University of Notre Dame, ↗

Systematic Benchmarking of Climate Models: Methodologies, Applications, and New Directions

As climate models become increasingly complex, there is a growing need to comprehensively and systematically assess model performance with respect to observations. Given the increasing number and diversity of climate model simulations in use, the community has moved beyond simple model intercomparison and toward developing methods capable of benchmarking a large number of simulations against a suite of climate metrics. Here, we present a detailed review of evaluation and benchmarking methods and approaches developed in the last decade, focusing primarily on scientific implications for Coupled Model Intercomparison Project (CMIP) simulations and CMIP6 results that contributed to the Intergovernmental Panel on Climate Change (IPCC) Sixth Assessment Report (AR6). Based on this review, we explain the resulting contemporary philosophy of model benchmarking, and provide clear distinctions and definitions of the terms model verification, process validation, evaluation, and benchmarking. While significant progress has been made in model development based on systematic evaluation and benchmarking efforts, some climate system biases still remain. The development of open‐source community software packages has played a fundamental role in identifying areas of significant model improvement and bias reduction. We review the key features of several software packages that have been commonly used over the past decade to evaluate and benchmark global and regional climate models. Additionally, we discuss best practices for the selection of evaluation and benchmarking metrics and for interpreting the obtained results, the importance of selecting suitable sources of reference data and accurate uncertainty quantification.

Environmental sciences↗

Transfer learning for analysis of collective and non-collective Thomson scattering spectra

Thomson scattering (TS) diagnostics provide reliable, minimally perturbative measurements of fundamental plasma parameters, such as electron density (⁠n e ) and electron temperature (⁠T e ⁠). Deep neural networks can provide accurate estimates of ⁠n e and T e when conventional fitting algorithms may fail, such as when TS spectra are dominated by noise, or when fast analysis is required for real-time operation. Although deep neural networks typically require large training sets, transfer learning can improve model performance on a target task with limited data by leveraging pre-trained models from related source tasks, where select hidden layers are further trained using target data. We present five architecturally diverse deep neural networks, pre-trained on synthetic TS data and adapted for experimentally measured TS data, to evaluate the efficacy of transfer learning in estimating n e and T e in both the collective and non-collective scattering regimes. We evaluate errors in n e and T e estimates as a function of training set size for models trained with and without transfer learning, and we observe decreases in model error from transfer learning when the training set contains ≲ 200 experimentally measured spectra.

Artificial neural networks↗

Understanding water and energy fluxes in the Amazonia: Lessons from an observation‐model intercomparison

Abstract Tropical forests are an important part of global water and energy cycles, but the mechanisms that drive seasonality of their land‐atmosphere exchanges have proven challenging to capture in models. Here, we (1) report the seasonality of fluxes of latent heat (LE), sensible heat ( H ), and outgoing short and longwave radiation at four diverse tropical forest sites across Amazonia—along the equator from the Caxiuanã and Tapajós National Forests in the eastern Amazon to a forest near Manaus, and from the equatorial zone to the southern forest in Reserva Jaru; (2) investigate how vegetation and climate influence these fluxes; and (3) evaluate land surface model performance by comparing simulations to observations. We found that previously identified failure of models to capture observed dry‐season increases in evapotranspiration (ET) was associated with model overestimations of (1) magnitude and seasonality of Bowen ratios (relative to aseasonal observations in which sensible was only 20%–30% of the latent heat flux) indicating model exaggerated water limitation, (2) canopy emissivity and reflectance (albedo was only 10%–15% of incoming solar radiation, compared to 0.15%–0.22% simulated), and (3) vegetation temperatures (due to underestimation of dry‐season ET and associated cooling). These partially compensating model‐observation discrepancies (e.g., higher temperatures expected from excess Bowen ratios were partially ameliorated by brighter leaves and more interception/evaporation) significantly biased seasonal model estimates of net radiation ( R n ), the key driver of water and energy fluxes (LE ~ 0.6 R n and H ~ 0.15 R n ), though these biases varied among sites and models. A better representation of energy‐related parameters associated with dynamic phenology (e.g., leaf optical properties, canopy interception, and skin temperature) could improve simulations and benchmarking of current vegetation–atmosphere exchange and reduce uncertainty of regional and global biogeochemical models.

Restrepo‐Coupe, Natalia↗

Summary Report on the Refined User Requirements for U.S. Fusion Prototypic Neutron Source

The Fusion Prototypic Neutron Source (FPNS) is proposed as an urgent near-term facility necessary for understanding D-T fusion neutron degradation processes in materials and for the development of high-performance radiation tolerant materials for fusion power reactors. This facility is central to our goal of predicting the behavior of materials in the harsh D-T neutron environment and calibrating and verifying the materials performance models necessary to support design of next-generation fusion reactors. Key FPNS performance metrics were previously defined for the cost-efficient facility necessary to adequately and expediently resolve key materials science knowledge gaps required to move to next-step fusion devices. This brief report is intended to expand upon this previous metrics discussion as an aid to selecting the most desirable FPNS concept and enabling a facility that meets the program needs.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Defect Thermodynamics and Transport Properties of Proton Conducting Perovskite Electrode and Electrolyte Materials Evaluated Based on Density Functional Theory Modeling

Both electron-rich and electron-poor perovskite oxides have been used in solid oxide cell applications as electrode and electrolyte materials. The rich oxygen defect chemistry and its coupling to temperature, hydrogen-steam or oxygen-steam gas pressure, or to the applied potentials creates enormous complexities for modeling performance and degradation of the materials. Herein, density functional theory-based thermodynamic modeling was carried out to describe the defect chemistry and transport properties of the proton-conducting electrolyte BaZr1-xYxO3-δ (x≤0.1) and of the triple-conducting perovskite (La,Ba)(Fe,M)O3-δ (M=Y and Zr). The defect thermodynamics of intrinsic point defects and the hydrogen-related defect reactions were solved in integrated defect models and further used to predict the Brouwer diagram and the transport properties of the functional perovskites. For the electron-poor electrolytes BaZr0.9Y0.1O3-δ, the developed model has been used to describe the experimental transport properties in the SOC operating conditions. Specifically, the roles played by the acceptor-bound holes and the intrinsic and hydrogen point defects upon the conductivities of holes, protons, and oxygen vacancies under the hydrogen-rich and oxygen-rich conditions at various humidity levels were demonstrated. A defect modeling tool was also developed for the triple-conducting perovskite (La,Ba)(Fe,M)O3-δ (M=Y and Zr) to examine magnetic effects and hydride defects in defect equilibria.

defect thermodynamics↗

Technoeconomic Opportunity Analysis for Local Power Generation in Falls City, Nebraska

Falls City is a small community in Nebraska interested in understanding how energy from local energy systems could support the community's economic development planning. To address the current community needs and address the future energy demand technical assistance conducted through the Communities Local Energy Action Program (Communities LEAP) assessed the technical and economic opportunities of adding energy technologies to Falls City's municipally owned and operated electric utility system. The modeling performed considered the technical and economic feasibility of technologies using the System Advisor Model (SAM). The modeling explored three technology configurations using multiple years of historical weather and wholesale cost data (2015-2022 & a typical meteorological year), and two different wholesale escalation rates (0.3% and 2.5%). Wholesale energy prices were based on the Southwest Power Pool's (SPP) real-time energy market and the annual escalation rates of these rates based on historical SPP wholesale and national retail electricity price trends. Results from the modeling showed that at current CAPEX costs and SPP wholesale electricity costs no technology combination averaged across the scenarios run provide a positive net present value (NPV). External financial support, changes in market conditions, and additional revenue streams would help create more economically favorable projects. As conditions change re-evaluation may be necessary.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Full-core high-burnup BWR LOCA fuel performance analysis and FFRD susceptibility

The susceptibility of the boiling water reactor (BWR) Limerick Unit 1 to fuel fragmentation, relocation, and dispersal during a postulated large-break loss-of-coolant accident (LBLOCA) was calculated using a multiphysics framework. The simulations include full-core, rod-resolved neutronic, thermal hydraulic, and fuel performance models using the VERA, TRACE, and BISON codes. This work focused on the transient BISON simulations, which include both the normal operation and LBLOCA periods in the same simulations. Cladding integrity was assessed using two correlations that are included with BISON. make page break Several new BWR-specific features were recently added to BISON. This work represents the first time these features have been included in a core-scale set of simulations. This study hence evaluates the performance of these new models for an operating reactor with realistic operating conditions. Simulation results showed that cladding integrity was maintained (i.e., no rods burst). Finally, future work to improve BWR and PWR predictions using this framework is suggested.

BISON↗

Utilizing ensemble learning for performance and power modeling and improvement of parallel cancer deep learning CANDLE benchmarks

Abstract Machine learning (ML) continues to grow in importance across nearly all domains in modeling to learn from data. Often a tradeoff exists between a model's ability to minimize bias and variance. In this article, we utilize ensemble learning to combine linear, nonlinear, and tree‐/rule‐based ML methods to cope with the bias‐variance tradeoff and result in more accurate models. We use the datasets collected for two parallel cancer deep learning CANDLE benchmarks, NT3 and P1B2, to build performance and power models based on hardware performance counters using single‐object and multiple‐objects ensemble learning to identify the most important counters for improvement on the Cray XC40 Theta at Argonne National Laboratory. Based on the insights from these models, we improve the performance and energy of P1B2 and NT3 by optimizing the deep learning environments TensorFlow, Keras, Horovod, and Python under the huge page size of 8 MB. Experimental results show that ensemble learning not only produces more accurate models but also provides more robust performance counter ranking. We achieve up to 61.15% performance improvement and up to 62.58% energy saving for P1B2 and up to 55.81% performance improvement and up to 52.60% energy saving for NT3 on up to 24,576 cores.

Wu, Xingfu↗

Verification of Reactor Activation Modelling with Physical Characterization Data and the Impact on Long Term Assessments - 20303

Whiteshell Laboratories, located in Manitoba, Canada, provided research facilities for the Canadian nuclear industry since the early 1960's. The facility was centered on Whiteshell Reactor 1 (WR-1), an organically cooled, heavy water moderated nuclear reactor. WR-1 was safely shut down in 1985, defueled and drained, and has been in a safe Storage with Surveillance (Deferred Decommissioning) state until present day. In support of an Environmental Assessment for proposed in-situ disposal of WR-1, detailed characterization information regarding the reactor core components (calandria, calandria tubes, and pressure tubes) was necessary to provide confidence that the long-term impacts of the facility can be assessed adequately. A simplified estimate of the total activation of core components was completed in 1992 using conservative assumptions, expected reactor configuration and computer models built with WIMS-CRNL, ONEDANT and ORIGEN-S. Due to the simplification, and age of the work, there was considerable doubt that the modelled inventory estimate of the core components was sufficient to support long-term performance modeling of the planned in-situ decommissioning. Additional data would be required to provide this confidence. Several options were considered for characterizing the WR-1 core components, but it was decided to collect only a limited number of samples in an effort to validate the 1992 simplified estimate as bounding. The limited number of samples reduced risks to workers, both radiological and non-radiological, and significantly reduced the costs required to confidently define and bound the radiological inventory of activation products in the core. The results of the sampling were compared to the simplified activation estimation. There was good agreement between the modelled and sampled results, with all major contributors to total activity showing similar relative quantities. The samples were generally lower than predicted by the models, with few exceptions. Notable exceptions included Nb-94, which were higher in the sample results than the model. This was determined to be caused by the use of Niobium containing steel alloys in the stainless steel pressure tube that were not accounted for in the models. The comparison of results provided confidence that the original inventory estimates produced through the simplified model are conservative and bounding, and therefore suitable for use in the long-term assessment of in situ disposal of the WR-1 reactor. This paper provides details of the simplified computer model results, their comparison to results of the sampling, and the changes adopted in the assessment of the in-situ decommissioning of WR-1 as a result. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Leveraging Large Language Models for Real-World Data Evidence: A Framework for Automated Treatment Extraction and Data Harmonization

Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.

artificial intelligence↗

An Energy Calculator for Simple Commercial Buildings

According to the EIA, simple commercial buildings account for 97% of total commercial building stock. However, most simple commercial buildings for example small- to mid-sized offices, retail, schools and warehouses do not benefit from the data-driven decision-making capabilities of whole-building energy modeling. The high cost of custom modeling limits the use of energy modeling of simple buildings for new construction or retrofit measures. Lack of tools providing helpful information on interactive savings estimates creates difficulties in meeting aggressive decarbonization and energy efficiency goals for simple building designers and utility program managers. This paper reviews a beta phase Simple Building Calculator with the ability to generate relatively accurate and interactive modeling results based on a limited but robust set of inputs. It can evaluate whole-building or single measure savings in new or existing buildings, compare measure package choices, or provide simplified performance modeling for energy codes and utility incentives. The tool combines physical (annual whole building prototype simulation) and statistical modeling techniques to predict annual energy performance. It supports a variety of building characteristics for envelope, HVAC, and lighting with parameters ranging from vintage to max tech configurations, as well as support for single-zone and simple multi-zone HVAC systems. The Simple Building Calculator was designed to provide immediate feedback for otherwise computationally intensive tasks like measure comparison, development of multiple measure package combinations, or verification that measures meet efficiency targets—all with the goal of providing a tool for quick annual energy simulation of simple commercial buildings.

Hart, Reid↗

Classifying metal‐binding sites with neural networks

Abstract To advance our ability to predict impacts of the protein scaffold on catalysis, robust classification schemes to define features of proteins that will influence reactivity are needed. One of these features is a protein's metal‐binding ability, as metals are critical to catalytic conversion by metalloenzymes. As a step toward realizing this goal, we used convolutional neural networks (CNNs) to enable the classification of a metal cofactor binding pocket within a protein scaffold. CNNs enable images to be classified based on multiple levels of detail in the image, from edges and corners to entire objects, and can provide rapid classification. First, six CNN models were fine‐tuned to classify the 20 standard amino acids to choose a performant model for amino acid classification. This model was then trained in two parallel efforts: to classify a 2D image of the environment within a given radius of the central metal binding site, either an Fe ion or a [2Fe‐2S] cofactor, with the metal visible (effort 1) or the metal hidden (effort 2). We further used two sub‐classifications of the [2Fe‐2S] cofactor: (1) a standard [2Fe‐2S] cofactor and (2) a Rieske [2Fe‐2S] cofactor. The accuracy for the model correctly identifying all three defined features was >95%, despite our perception of the increased challenge of the metalloenzyme identification. This demonstrates that machine learning methodology to classify and distinguish similar metal‐binding sites, even in the absence of a visible cofactor, is indeed possible and offers an additional tool for metal‐binding site identification in proteins.

59 BASIC BIOLOGICAL SCIENCES↗

Interpretable Models for Workflow Differentiation in High-Performance Scientific Networks

Scientific workflows in high-performance networks spawn hundreds of interdependent flows that must be managed collectively—yet existing network classifiers treat each flow in isolation, leading to fragmented QoS decisions and missed interflow patterns. We present a novel traffic classification solution that operates at the workflow level, distinguishing entire filetransfer operations from streaming analytics by capturing how concurrent flows interact and burst together. We introduce a workflow identification window (WIW) that ingests raw packet headers from parallel flows into unified tensors, preserving the spatial-temporal patterns that differentiate scientific workflows. This approach achieves 98.7% accuracy using CNN, LSTM, and hybrid architectures, while maintaining 84% accuracy on production traffic collected a week later—demonstrating robustness to temporal drift. By integrating SHAP and GradCAM explainability, we reveal that early-packet timing patterns and cross-flow correlations drive classification decisions, providing operators with interpretable insights. Our system enables coherent workflow-level QoS enforcement and dynamic bandwidth allocation in scientific networks, eliminating manual per-flow configuration while maintaining classification latency at millisecond level.

Giannakou, Anna [LBL, Berkeley]↗

Understanding HPC Benchmark Performance on Intel Broadwell and Cascade Lake Processors

Hardware platforms in high performance computing are constantly getting more complex to handle even when considering multicore CPUs alone. Numerous features and configuration options in the hardware and the software environment that are relevant for performance are not even known to most application users or developers. Microbenchmarks, i.e., simple codes that fathom a particular aspect of the hardware, can help to shed light on such issues, but only if they are well understood and if the results can be reconciled with known facts or performance models. The insight gained from microbenchmarks may then be applied to real applications for performance analysis or optimization. In this paper we investigate two modern Intel x86 server CPU architectures in depth: Broadwell EP and Cascade Lake SP. We highlight relevant hardware configuration settings that can have a decisive impact on code performance and show how to properly measure on-chip and off-chip data transfer bandwidths. The new victim L3 cache of Cascade Lake and its advanced replacement policy receive due attention. Finally we use DGEMM, sparse matrix-vector multiplication, and the HPCG benchmark to make a connection to relevant application scenarios.

97 MATHEMATICS AND COMPUTING↗

Evaluation of a preliminary regional Earth model through comparison of synthetic and observed waveform data

In this report, we document the process related to developing a regional geologic model of a 605 x 1334 km area centered around Utah and encompassing surrounding states. This model is developed to test the effect that composition of a model has on the generation of synthetic data with the intent of using this information to improve upon full waveform moment tensor inversions. We compare observed data from three seismic events and five stations to the synthetic data generated by a preliminary model derived from a geologic framework model (GFM) developed by the USGS. The synthetic data and observed data comparisons indicate that our preliminary model performs well at smaller offset distances in the northern and central sections of the model. However, the southern stations consistently display synthetic data P- and S-wave arrival times that do not match the observed data arrival times, indicating that the velocity structure of the southern part of the model especially is inaccurate.

58 GEOSCIENCES↗