Search NASA⌕ Search

SEARCH · Search NASA

Results for “Performance Model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

Modelling evapotranspiration during precipitation deficits: Identifying critical processes in a land surface model

Surface fluxes from land surface models (LSMs) have traditionally been evaluated against monthly, seasonal or annual mean states. The limited ability of LSMs to reproduce observed evaporative fluxes under water-stressed conditions has been previously noted, but very few studies have systematically evaluated these models during rainfall deficits. We evaluated latent heat fluxes simulated by the Community Atmosphere Biosphere Land Exchange (CABLE) LSM across 20 flux tower sites at sub-annual to inter-annual timescales, in particular focusing on model performance during seasonal-scale rainfall deficits. The importance of key model processes in capturing the latent heat flux was explored by employing alternative representations of hydrology, leaf area index, soil properties and stomatal conductance. We found that the representation of hydrological processes was critical for capturing observed declines in latent heat during rainfall deficits. By contrast, the effects of soil properties, LAI and stomatal conductance were highly site-specific. Whilst the standard model performs reasonably well at annual scales as measured by common metrics, it grossly underestimates latent heat during rainfall deficits. A new version of CABLE, with a more physically consistent representation of hydrology, captures the variation in the latent heat flux during seasonal-scale rainfall deficits better than earlier versions, but remaining biases point to future research needs. Lastly, our results highlight the importance of evaluating LSMs under water-stressed conditions and across multiple plant functional types and climate regimes.

54 ENVIRONMENTAL SCIENCES↗

Tracking the Performance Evolution of Blue Gene Systems

— IBM’s Blue Gene supercomputer has evolved through three generations from the original Blue Gene/L to P to Q. A higher level of integration has enabled greater single-core performance, and a larger concurrency per compute node. Although these changes have brought with them a higher overall system peak-performance, no study has examined in detail the evolution of perfor-mance across system generations. In this work we make two significant contri-butions – that of providing a comparative performance analysis across Blue Gene generations using a consistent set of tests, and also in providing a validat-ed performance model of the NEK-Bone proxy application. The combination of empirical analysis and the predictive performance model enable us to not only directly compare measured performance but also allow for a comparison of sys-tem configurations that cannot currently be measured. We provide insights into how the changing characteristics of Blue Gene have impacted on the application performance, as well as what future systems may be able to achieve.

Kerbyson, Darren J.↗

Earthquake catalog-based machine learning identification of laboratory fault states and the effects of magnitude of completeness

Here, machine learning regression can predict macroscopic fault properties such as shear stress, friction, and time to failure using continuous records of fault zone acoustic emissions. Here we show that a similar approach is successful using event catalogs derived from the continuous data. Our methods are applicable to catalogs of arbitrary scale and magnitude of completeness. We investigate how machine learning regression from an event catalog of laboratory earthquakes performs as a function of the catalog magnitude of completeness. We find that strong model performance requires a sufficiently low magnitude of completeness, and below this magnitude of completeness, model performance saturates.

58 GEOSCIENCES↗

Forte: An Interactive Visual Analytic Tool for Trust-Augmented Net-Load Forecasting

Accurate net-load forecasting is vital for energy planning, aiding decisions on trade and load distribution. However, assessing the performance of forecasting models across diverse input variables, like temperature and humidity, remains challenging, particularly for eliciting a high degree of trust in the model outcomes. In this context, there is a growing need for data-driven technological interventions to aid scientists in comprehending how models react to both noisy and clean input variables, thus shedding light on complex behaviors and fostering confidence in the outcomes. In this paper, we present Forte, a visual analytics-based application to explore deep probabilistic net-load forecasting models across various input variables and understand the error rates for different scenarios. With carefully designed visual interventions, this web-based interface empowers scientists to derive insights about model performance by simulating diverse scenarios, facilitating an informed decision-making process. We discuss observations made using Forte and demonstrate the effectiveness of visualization techniques to provide valuable insights into the correlation between weather inputs and net-load forecasts, ultimately advancing grid capabilities by improving trust in forecasting models.

Bhattacharjee, Kaustav↗

Quantifying Uncertainty in PV Energy Estimates Final Report

Uncertainty in PV energy estimates is "one of the most critical areas of lack of understanding" according to independent engineers, financiers, PV model developers, and other industry stakeholders. The primary problem is a lack of rigorous, transparent, widely accepted methods for quantifying uncertainty in energy production estimates. Uncertainty in energy production estimates arises from variability of the solar resource, inexact PV performance models and their parameters, and system reliability considerations. Uncertainty in annual energy production is frequently calculated for larger projects in order to quantify financial risk. Key statistics for energy, such as the P-values "P50" and "P90" (the annual energy values that are exceeded in future years with 50\% and 90\% probability, respectively) are used by financing institutions to calculate the repayment risk for the project. The current methods to estimate these statistics are typically proprietary, specialized, and involve significant post-processing of commercial performance model results. This black-box approach leads to inconsistent P-value estimates from different parties, which reduces investors' confidence in the results. Since the financial community bases its risk assessment on these estimates, reduced confidence increases perceived project risk, and consequently financing costs. The goal of this project was to establish a set of best practices for quantifying uncertainty in energy production estimates, including identifying what sources of uncertainty must be considered with clear definitions and metrics, determining which sources are the biggest drivers of uncertainty, and providing a computationally efficient framework for combining different sources of uncertainty that is flexible enough to accommodate substitutions of data or methods when better information is available. We engaged a wide set of stakeholders to ensure industry endorsement and adoption, and leveraged complementary projects investigating individual sources of uncertainty in great detail, as well as others' work that started down this path.

ENERGY PLANNING, POLICY, AND ECONOMY,SOLAR ENERGY↗

Causality guided machine learning model on wetland CH 4 emissions across global wetlands

Wetland CH 4 emissions are among the most uncertain components of the global CH 4 budget. The complex nature of wetland CH 4 processes makes it challenging to identify causal relationships for improving our understanding and predictability of CH 4 emissions. In this study, we used the flux measurements of CH 4 from eddy covariance towers (30 sites from 4 wetlands types: bog, fen, marsh, and wet tundra) to construct a causality-constrained machine learning (ML) framework to explain the regulative factors and to capture CH 4 emissions at sub-seasonal scale. We found that soil temperature is the dominant factor for CH 4 emissions in all studied wetland types. Ecosystem respiration (CO 2 ) and gross primary productivity exert controls at bog, fen, and marsh sites with lagged responses of days to weeks. Integrating these asynchronous environmental and biological causal relationships in predictive models significantly improved model performance. More importantly, modeled CH 4 emissions differed by up to a factor of 4 under a +1°C warming scenario when causality constraints were considered. These results highlight the significant role of causality in modeling wetland CH 4 emissions especially under future warming conditions, while traditional data-driven ML models may reproduce observations for the wrong reasons. Our proposed causality-guided model could benefit predictive modeling, large-scale upscaling, data gap-filling, and surrogate modeling of wetland CH 4 emissions within earth system land models.

54 ENVIRONMENTAL SCIENCES↗

Comparative investigations of multi-fidelity modeling on performance of electrostatically-actuated cracked micro-beams

Silicon is a commonly used material for the fabrication of beams for use in micro-electrical-mechanical systems (MEMS). Although silicon is a brittle material, it has been shown to accumulate fatigue damage at the micro-scale. Understanding the effect this has on the overall device performance is critical to the design of reliable devices. Analytical methods for modeling damage provide expedient results but are limited by broad modeling assumptions. Numerical models account for more detailed physical phenomena but can be computationally intensive. In this work, two different crack scenarios are modeled using both analytical techniques and 3D computational simulations. First, the effects of a single surface crack on the static deflection and natural frequency of an electrostatically actuated micro-beam are formulated and compared. Then, a new method for approximating damage associated with realistic distributed crack networks is formulated for use in an analytical model and numerical simulations. A method for utilizing experimentally derived crack statistics to inform the analytical and numerical distributed crack models is developed. Good agreement between the analytical and numerical models is obtained for both crack scenarios. Altogether, these models can be used to effectively simulate a variety of damage and fatigue behaviors in silicon-based MEMS devices.

42 ENGINEERING↗

Intensified Soil Moisture Extremes Decrease Soil Organic Carbon Decomposition: A Mechanistic Modeling Analysis

Earth system models have predicted that there will be more frequent and severe precipitation and drought events in terrestrial ecosystems. Microbially mediated decomposition of soil organic carbon (SOC) tends to increase as soils wet and decrease as soils dry. However, the long-term SOC change under intensified moisture extremes remains poorly known as it depends on the frequency and intensity of soil drying and wetting. In this study, we explored long-term SOC dynamics under scenarios of alternating drying-wetting cycles using the Microbial-ENzyme Decomposition model, a mechanistic microbial model. The model was parameterized with 11 years of observations from a temperate deciduous broadleaf forest site, showing satisfactory model performance in both model calibration (R 2 = 0.67) and validation (R 2 = 0.69) against heterotrophic respiration. We then used the model to simulate the long-term SOC dynamics under five scenarios of alternating drying-wetting cycles with different frequencies and severities over a period of 100 years. Results showed that the changes in active microbial biomass C and the corresponding turnover rates of SOC pools were more sensitive to soil drying than soil wetting. As a result, the cumulative soil carbon emission from microbial respiration decreased by 433.7 g C m -2 after the 100-year simulation in the highest frequency and intensity moisture scenario, but was not significantly affected by the lowest frequency and intensity scenario. This study emphasizes the nonlinear response of SOC decomposition to soil moisture changes, which causes decreased decomposition by microbes under drying that is, not compensated by increased decomposition under wetting conditions.

58 GEOSCIENCES↗

Image-based novel fault detection with deep learning classifiers using hierarchical labels

One important characteristic of modern fault classification systems is the ability to flag the system when faced with previously unseen fault types. This work considers the unknown fault detection capabilities of deep neural network-based fault classifiers. Specifically, we propose a methodology on how, when available, labels regarding the fault taxonomy can be used to increase unknown fault detection performance without sacrificing model performance. To achieve this, we propose to utilize soft label techniques to improve the state-of-the-art deep novel fault detection techniques during the training process and novel hierarchically consistent detection statistics for online novel fault detection. Lastly, we demonstrated increased detection performance on novel fault detection in inspection images from the hot steel rolling process, with results well replicated across multiple scenarios and baseline detection methods.

42 ENGINEERING↗

TX$^2$: Transformer eXplainability and eXploration

The Transformer eXplainability and eXploration (Martindale & Stewart, 2021), or TX 2 software package, is a library designed for artificial intelligence researchers to better understand the performance of transformer models (Vaswani et al., 2017) used for sequence classification. The tool is capable of integrating with a trained transformer model and a dataset split into training and testing populations to produce an ipywidget (Project Jupyter Contributors, 2021) dashboard with a number of visualizations to understand model performance with an emphasis on explainability and interpretability. The TX 2 package is primarily intended to integrate into a workflow centered around Jupyter Notebooks (Kluyver et al., 2016), and currently assumes the use of PyTorch (Paszke et al., 2019) and Hugging Face transformers library (Wolf et al., 2020). The dashboard includes visualization and data exploration features to aid researchers, including an interactive UMAP embedding graph (McInnes et al., 2018) to understand classification clusters, a word salience map that can be updated as researchers alter textual entries in near real time, a set of tools to understand word frequency and importance based on the clusters in the UMAP embedding graph, and a set of traditional confusion matrix analysis tools.

97 MATHEMATICS AND COMPUTING↗

Modeling the temporal dynamics of nonstructural carbohydrate pools in forest trees

Trees store carbohydrates, in the form of sugars and starch, as reserves to be used to power both future growth as well as to support day-to-day metabolic functions. These reserves are particularly important in the context of how trees cope with disturbance and stress—for example, as related to pest outbreaks, wind or ice damage, and extreme climate events. In this project, we measured the size of carbon reserves in forest trees, and determined how quickly these reserves are used and replaced—i.e., their “turnover time”. Our work was conducted at Harvard Forest, a temperate deciduous forest in central Massachusetts. Through field sampling, laboratory-based chemical analyses, and allometric modeling, we scaled these measurements up to whole-tree NSC budgets. We used these data to test and improve computer simulation models of carbon flow through forest ecosystems. Our modeling focused on the mathematical representation of these stored carbon reserves, and we examined the sensitivity of model performance to different model structures. This project contributes to DOE’s goal to improve next-generation models of the earth system, and to understand the impacts of climate change on terrestrial ecosystems.

59 BASIC BIOLOGICAL SCIENCES↗

FY17 end of year MARMOT materials model development update

This document summarizes the lower length scale progress made in FY17 in developing materials models for fuel performance modeling of oxide fuels and their implementation in the MARMOT code. The progress covers the development of a phase field recrystallization model for high-burnup structure formation in oxide fuels, a phase field model for studying stage-3 fission gas release in UO 2 , a phase-field model for irradiation-enhanced densification in UO 2 , a quantitative model for hydride morphology in Zirconium based cladding, and the calculation of hydrogen diffusivity under stress in various Zirconium based alloys as cladding candidates. Further development in the MARMOT code has been carried out as well to improve the multi-phase, multi-component phase field model, the representation of interfacial energy in the Kim-Kim-Suzuki model, and the description of interfacial stress. For most of the above, separated reports have been generated with detailed description. Here a brief summary is provided, with references pointing to the detailed reports to guide interested readers.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Leveraging design of experiments to build chemometric models for the quantification of uranium (VI) and HNO3 by Raman spectroscopy

Partial least squares regression (PLSR) and support vector regression (SVR) models were optimized for the quantification of U(VI) (10–320 g L −1 ) and HNO 3 (0.6–6 M) by Raman spectroscopy with optimized calibration sets chosen by optimal design of experiments. The designed approach effectively minimized the number of samples in the calibration set for PLSR and SVR by selecting sample concentrations with a quadratic process model, despite complex confounding and covarying spectral features in the spectra. The top PLS2 model resulted in percent root mean square errors of prediction for U(VI), HNO 3 , and NO 3 − of 3.7%, 3.6%, and 2.9%, respectively. PLS1 models performed similarly despite modeling an analyte with a majority linear response (i.e., uranyl symmetric stretch) and another with more covarying vibrational modes (i.e., HNO 3 ). Partial least squares (PLS) model loadings and regression coefficients were evaluated to better understand the relationship between weaker Raman bands and covarying spectral features. Support vector machine models outperformed PLS1 models, resulting in percent root mean square error of prediction values for U(VI) and HNO 3 of 1.5% and 3.1%, respectively. The optimal nonlinear SVR model was trained using a similar number of samples (11) compared with the PLSR model, even though PLS is a linear modeling approach. The generic D-optimal design presented in this work provides a robust statistical framework for selecting training set samples in disparate two-factor systems. This approach reinforces Raman spectroscopy for the quantification of species relevant to the nuclear fuel cycle and provides a robust chemometric modeling approach to bolster online monitoring in challenging process environments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Active Learning for Rapid Targeted Synthesis of Compositionally Complex Alloys

The next generation of advanced materials is tending toward increasingly complex compositions. Synthesizing precise composition is time-consuming and becomes exponentially demanding with increasing compositional complexity. An experienced human operator does significantly better than a novice but still struggles to consistently achieve precision when synthesis parameters are coupled. The time to optimize synthesis becomes a barrier to exploring scientifically and technologically exciting compositionally complex materials. This investigation demonstrates an active learning (AL) approach for optimizing physical vapor deposition synthesis of thin-film alloys with up to five principal elements. We compared AL-based on Gaussian process (GP) and random forest (RF) models. The best performing models were able to discover synthesis parameters for a target quinary alloy in 14 iterations. We also demonstrate the capability of these models to be used in transfer learning tasks. RF and GP models trained on lower dimensional systems (i.e., ternary, quarternary) show an immediate improvement in prediction accuracy compared to models trained only on quinary samples. Furthermore, samples that only share a few elements in common with the target composition can be used for model pre-training. We believe that such AL approaches can be widely adapted to significantly accelerate the exploration of compositionally complex materials.

Chemistry↗

BAYESIAN TECHNIQUES FOR COMPARING TIME-DEPENDENT GRMHD SIMULATIONS TO VARIABLE EVENT HORIZON TELESCOPE OBSERVATIONS

The Event Horizon Telescope (EHT) is a millimeter-wavelength, very-long-baseline interferometry (VLBI) experiment that is capable of observing black holes with horizon-scale resolution. Early observations have revealed variable horizon-scale emission in the Galactic Center black hole, Sagittarius A* (Sgr A*). Comparing such observations to time-dependent general relativistic magnetohydrodynamic (GRMHD) simulations requires statistical tools that explicitly consider the variability in both the data and the models. We develop here a Bayesian method to compare time-resolved simulation images to variable VLBI data, in order to infer model parameters and perform model comparisons. We use mock EHT data based on GRMHD simulations to explore the robustness of this Bayesian method and contrast it to approaches that do not consider the effects of variability. We find that time-independent models lead to offset values of the inferred parameters with artificially reduced uncertainties. Moreover, neglecting the variability in the data and the models often leads to erroneous model selections. We finally apply our method to the early EHT data on Sgr A*.

79 ASTRONOMY AND ASTROPHYSICS↗

Development of a Performance and Processing Property Acceptance Region for Cementitious Low-Level Waste Forms at Savannah River Site - 13174

The Saltstone Production and Disposal Facilities (SPF and SDF) at the Savannah River Site (SRS) have been treating decontaminated salt solution, a low-level aqueous waste stream (LLW) since facility commissioning in 1990. In 2012, the Saltstone Facilities implemented a new Performance Assessment (PA) that incorporates an alternate design for the disposal facility to ensure that the performance objectives of DOE Order 435.1 and the National Defense Authorization Act (NDAA) of Fiscal Year 2005 Section 3116 are met. The PA performs long term modeling of the waste form, disposal facility, and disposal site hydrogeology to determine the transport history of radionuclides disposed in the LLW. Saltstone has been successfully used to dispose of LLW in a grout waste form for 15 years. Numerous waste form property assumptions directly impact the fate and transport modeling performed in the PA. The extent of process variability and consequence on performance properties are critical to meeting the assumptions of the PA. The SPF has ensured performance property acceptability by way of implementing control strategies that ensure the process operates within the analyzed limits of variability, but efforts continue to improve the understanding of facility performance in relation to the PA analysis. A similar understanding of the impact of variability on processing parameters is important from the standpoint of the operability of the production facility. The fresh grout slurry properties (particularly slurry rheology and the rate of hydration and structure formation) of the waste form directly impact the pressure and flow rates that can be reliably processed. It is thus equally important to quantify the impact of variability on processing parameters to ensure that the design basis assumptions for the production facility are maintained. Savannah River Remediation (SRR) has been pursuing a process that will ultimately establish a property acceptance region (PAR) to incorporate elements important to both processability and long-term performance properties. This process involves characterization of both emplaced product samples from the disposal facility and laboratory-simulated samples to demonstrate the effectiveness of the lab simulation. With that basis confirmed, a comprehensive variability study using non-radioactive simulants will define the acceptable PAR, or 'operating window' for Saltstone production and disposal. This same process will be used in the future to evaluate new waste streams for disposal or changes to the existing process flowsheet. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

A Benchmarking Framework for Evaluating Large Language Model Capabilities in Nuclear Reactor Safety Applications

Large language models (LLMs) are increasingly capable of answering technical questions, synthesizing domain knowledge, and supporting engineering workflows. For nuclear science and engineering, these capabilities require careful, domain-specific evaluation before they can be credibly incorporated into safety-related activities, regulatory review, or technical decision support. This paper presents preliminary results from benchmarking framework for evaluating LLM capabilities in nuclear contexts. The framework is organized into three evaluation categories: nuclear fundamentals, general dual-use knowledge, and plant specific knowledge. These categories are intended to distinguish general nuclear engineering competence from broader technical reasoning and more context-dependent nuclear knowledge. Initial evaluations focus on nuclear fundamentals using questions representative of the knowledge expected of a nuclear professional engineer. Results indicate that contemporary frontier models perform at a high level and substantially exceed the performance of older model generations, with some models approaching saturation of the current benchmark. These findings suggest both the rapid improvement of LLM capabilities in specialized technical domains and the need for more discriminating evaluation methods. The paper presents the benchmark structure, preliminary model-comparison results, and ongoing work. This work supports development of verifiable, responsible, and safety-conscious methods for assessing AI systems in nuclear engineering applications.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗

Natural Gas Variability In California: Environmental Impacts And Device Performance Combustion Modeling of Pollutant Emissions From a Residential Cooking Range

As part of a larger study of liquefied natural gas impacts on device performance and pollutant emissions for existing equipment in California, this report describes a computer modeling study of a partially premixed flame issuing from a single cooktop burner port. The model consisted of a reactive computational fluid dynamics three-dimensional spatial grid and a 71-species chemical mechanism with propane combustion capability. Simulations were conducted with a simplified fuel mixture containing methane, ethane, and propane in proportions that yield properties similar to fuels distributed throughout much of California now and in recent years (baseline fuel), as well as with two variations of simulated liquefied natural gas blends. A variety of simulations were conducted with baseline fuel to explore the effect of several key parameters on pollutant formation and other flame characteristics. Simulations started with fuel and air issuing through the burner port, igniting, and continuing until the flame was steady with time. Conditions at this point were analyzed to understand fuel, secondary air and reaction product flows, regions of pollutant formation, and exhaust concentrations of carbon monoxide, nitric oxide and formaldehyde. A sensitivity study was conducted, varying the inflow parameters of this baseline gs about real-world operating conditions. Flame properties responded as expected from reactive flow theory. In the simulation, carbon monoxide levels were influenced more by the mixture's inflow velocity than by the gas-to-air ratio in the mixture issuing from the inflow port. Additional simulations were executed at two inflow conditions - high heat release and medium heat release - to examine the impact of replacing the baseline gas with two mixtures representative of liquefied natural gas. Flame properties and pollutant generation rates were very similar among the three fuel mixtures.

03 NATURAL GAS↗