Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

Covariance-Free Bifidelity Control Variates Importance Sampling for Rare Event Reliability Analysis

Multifidelity modeling has been steadily gaining attention as a tool to address the problem of exorbitant model evaluation costs that makes the estimation of failure probabilities a significant computational challenge for complex real-world problems, particularly when failure is a rare event. To implement multifidelity modeling, estimators that efficiently combine information from multiple models/sources are necessary. In past works, the variance reduction techniques of control variates (CV) and importance sampling (IS) have been leveraged for this task. In this paper, we present the CVIS framework—a creative take on a coupled CV and IS estimator for bifidelity reliability analysis. The framework addresses some of the practical challenges of the CV method by using an estimator for the control variate mean and sidestepping the need to estimate the covariance between the original estimator and the control variate through a clever choice for the tuning constant. Furthermore, the task of selecting an efficient IS distribution is also considered, with a view towards maximally leveraging the bifidelity structure and maintaining expressivity. Additionally, a diagnostic is provided that indicates both the efficiency of the algorithm as well as the relative predictive quality of the models utilized. Finally, the behavior and performance of the framework is explored through analytical and numerical examples.

Markov chain Monte Carlo↗

elm-diagnostics

elm-diagnostics is a Python package for computing diagnostic analyses and visualizations for the E3SM Land Model (ELM) component and is meant to support new feature development in ELM. The tool reads model history files and performs quantitative analyses including budget-closure checking, variable transformations, temporal aggregations, and statistical summaries to support model evaluation, validation, and scientific interpretation. The framework is designed for extensibility, with modular architecture enabling straightforward addition of new diagnostic methods, derived variables, analysis types, visualization approaches, and model-specific adaptations

Hoffman, Matt [Los Alamos National Laboratory]↗

The GGCMI Phase 2 Experiment: Global Gridded Crop Model Simulations Under Uniform Changes in CO2, Temperature, Water, and Nitrogen Levels (Protocol Version 1.0)

Concerns about food security under climate change motivate efforts to better understand future changes in crop yields. Process-based crop models, which represent plant physiological and soil processes, are necessary tools for this purpose since they allow representing future climate and management conditions not sampled in the historical record and new locations to which cultivation may shift. However, process-based crop models differ in many critical details, and their responses to different interacting factors remain only poorly understood. The Global Gridded Crop Model Intercomparison (GGCMI) Phase 2 experiment, an activity of the Agricultural Model Intercomparison and Improvement Project (AgMIP), is designed to provide a systematic parameter sweep focused on climate change factors and their interaction with overall soil fertility, to allow both evaluating model behavior and emulating model responses in impact assessment tools. In this paper we describe the GGCMI Phase 2 experimental protocol and its simulation data archive. A total of 12 crop models simulate five crops with systematic uniform perturbations of historical climate, varying CO2, temperature, water supply, and applied nitrogen (“CTWN”) for rainfed and irrigated agriculture, and a second set of simulations represents a type of adaptation by allowing the adjustment of growing season length. We present some crop yield results to illustrate general characteristics of the simulations and potential uses of the GGCMI Phase 2 archive. For example, in cases without adaptation, modeled yields show robust decreases to warmer temperatures in almost all regions, with a nonlinear dependence that means yields in warmer baseline locations have greater temperature sensitivity. Inter-model uncertainty is qualitatively similar across all the four input dimensions but is largest in high-latitude regions where crops may be grown in the future.

GGCMI↗

Collaborative Project: The problem of bias in defining uncertainty in computationally enabled strategies for data-driven climate model development. Final Technical Report.

The objective of the project is to develop strategies for better representing scientific sensibilities within statistical measures of model skill that then can be used within a Bayesian statistical framework for data-driven climate model development and improved measures of model scientific uncertainty. One of the thorny issues in model evaluation is quantifying the effect of biases on climate projections. While any bias is not desirable, only those biases that affect feedbacks affect scatter in climate projections. The effort at the University of Texas is to analyze previously calculated ensembles of CAM3.1 with perturbed parameters to discover how biases affect projections of global warming. The hypothesis is that compensating errors in the control model can be identified by their effect on a combination of processes and that developing metrics that are sensitive to dependencies among state variables would provide a way to select version of climate models that may reduce scatter in climate projections. Gabriel Huerta at the University of New Mexico is responsible for developing statistical methods for evaluating these field dependencies. The UT effort will incorporate these developments into MECS, which is a set of python scripts being developed at the University of Texas for managing the workflow associated with data-driven climate model development over HPC resources. This report reflects the main activities at the University of New Mexico where the PI (Huerta) and the Postdocs (Nosedal, Hattab and Karki) worked on the project.

58 GEOSCIENCES↗

Machine learning for the redox potential prediction of molecules in organic redox flow battery

Here, organic redox flow batteries (ORFB) are recognized as an innovative technology for the large-scale storage of renewable energy. The redox potential of organic redox-active molecules plays a vital role in their performance. Advanced screening techniques like high-throughput experiment and machine learning (ML) have significantly enhanced organic material performance and transformed the field of ORFB. However, the scarcity of experimental data poses a considerable challenge for ML model development in this domain. In our study, we developed lightweight graph-based Gaussian process regression (GPR) models with GPU-accelerated marginalized graph kernel and hybrid kernel to predict the redox potentials of organic redox-active molecules for ORFBs, specifically focusing on small datasets. To evaluate model accuracy, we created a new experimental database of organic redox-active molecules by the data from hundreds of published papers and assembled previous computational datasets. We also considered some key parameters, such as pH conditions and solvent type, to assess their impact on redox potential prediction. Our GPR model predicted redox potentials with high accuracy across all datasets using minimal training data. The study provides powerful tools for molecule screening and design and delivers valuable guidance on designing training datasets for costly experiments.

25 ENERGY STORAGE↗

Quantifying the Occurrence of Record Hot Years Through Normalized Warming Trends

Surface air temperature trends and extreme events are of global concern and they are related. Here we show that the occurrence of record hot years over different latitudes from 1960-2019 are more strongly correlated with the observational annual mean temperature trends normalized by internal variability. Compared with the raw trends showing Arctic amplification, the normalized trends show a tropical amplification over land. Two hot spots with more frequent occurrence of record hot years are identified: northern hemisphere ocean (versus land) and southern hemisphere tropical land (versus mid- and high-latitude lands). Ensemble mean results from 32 Earth system models agree with observations better than individual models, but they do not reproduce observed large differences in correlations across latitudes between normalized trends and record-breaking events over land versus ocean. Furthermore, our results enable the quantification of record hot year occurrence through normalized warming trends and provide new metrics for model evaluation and improvement.

58 GEOSCIENCES↗

Mississippi State University Center for Air Sea Technology FY95 Research Program

The Mississippi State University (MSU) Center for Air Sea Technology (CAST) evolved from the Institute for Naval Oceanography's (INO) Experimental Center for Mesoscale Ocean Prediction (ECMOP) which was started in 1989. MSU CAST subsequently began operation on 1 October 1992 under an Office of Naval Research (ONR) two-year grant which ended on 30 September 1994. In FY95 MSU CAST was successful in obtaining five additional research grants from ONR, as well as several other research contracts from the Naval Oceanographic Office via NASA, the Naval Research Laboratory, the Army Corps of Engineers, and private industry. In the past, MSU CAST technical research and development has produced tools, systems, techniques, and procedures that improve efficiency and overcome deficiency for both the operational and research communities residing with the Department of Defense, private industry, and university ocean modeling community. We continued this effort with the following thrust areas: to develop advanced methodologies and tools for model evaluation, validation and visualization, both oceanographic and atmospheric; to develop a system-level capability for conducting temporally and ; spatially scaled ocean simulations driven by or are responsive to ocean models, and take into consideration coupling to atmospheric models; to continue the existing oceanographic/atmospheric data management task with emphasis on distributed databases in a network environment, with database optimization and standardization, including use of Mosaic and World Wide Web (WWW) access; and to implement a high performance parallel computing technology for CAST ocean models

Yeske, Lanny↗

The Research of MOX Fuels in Japan

In Japan, uranium and plutonium mixed oxide (MOX) fuels has been developed for sodium-cooled fast reactors. The developing MOX fuels come in variety of O/M ratio, Pu content, minor actinide (MA) content and density. We have studied a science based fuel technology to evaluate fuel behaviors in fabrication process and irradiation condition of such various fuels. The technologies which are constructed based on experimental database can apply to mechanistic evaluation of fuel behaviors. To develop the science based fuel technology, many different varieties of basic properties have been investigated, and experimental database was constructed. And a mechanistic physical property model has been studied. The models contribute to describe various behaviors in fuel fabrication process and irradiation condition. In Japan, the researches on MOX fuel technologies have been carried out for sodium cooled fast reactor development. It is important to develop the fuel technologies on fuel fabrication process and irradiation performance model evaluation. The researches will be carried out by experimental approach and expanded in international collaboration frame works such as the CNWG and The Generation IV International Forum (GIF). (author)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Scalable Risk Assessment of Rare Events in Power Systems With Uncertain Wind Generation and Loads

Risk assessment of rare events has become increasingly important in power system planning and operation with the increasing integration of renewable energy and the presence of system uncertainties. However, quantifying the risk posed by rare events via the traditional method, i.e., Monte Carlo sampling (MCS), incurs substantial computational expense stemming from the vast ensemble of power flow simulations. To accelerate the assessment, this paper proposes a Deep Neural Network (DNN)-kernelized vector-valued Gaussian Process (VVGP) approach with excellent computational efficiency while maintaining high accuracy. Consequently, serving as a surrogate model for the power flow solver, the DNN-kernelized VVGP enables significantly faster but accurate risk assessment compared to the power flow solver. The developed surrogate model evaluates low-order N - k events that contain more than 90% instances by adeptly capturing the topological features while the high-order N - k events are assessed via a power flow solver, thereby striking a balance between computational efficiency and uncertainty quantification accuracy. Moreover, the model incorporates a Support Vector Machine (SVM) classifier to resample concerning low-probability tail events to counteract the biases potentially introduced during the DNN-kernelized VVGP evaluations. Simulations conducted on the modified IEEE 24-bus, 118-bus, and European 1354-bus systems demonstrate that the proposed method maintains the accuracy benchmark set by MCS while significantly reducing computational demands in large-scale power systems as compared to other state-of-the-art methods.

17 WIND ENERGY↗

A Physics-based Smart Persistence model for Intra-hour forecasting of solar radiation (PSPI) using GHI measurements and a cloud retrieval technique

Short-term solar forecasting models based solely on global horizontal irradiance (GHI) measurements are often unable to discriminate the forecasting of the factors affecting GHI from those that can be precisely computed by atmospheric models. Here, we introduce a Physics-based Smart Persistence model for Intra-hour forecasting of solar radiation (PSPI) that decomposes the forecasting of GHI into the computation of extraterrestrial solar radiation and solar zenith angle and the forecasting of cloud albedo and cloud fraction. The extraterrestrial solar radiation and solar zenith angle are accurately computed by the Solar Position Algorithm (SPA) developed at the National Renewable Energy Laboratory (NREL). A cloud retrieval technique is used to estimate cloud albedo and cloud fraction from surface-based observations of GHI. With the assumption of persistent cloud structures, the cloud albedo and cloud fraction are predicted for future time steps using a two-stream approximation and a 5-min exponential weighted moving average, respectively. Our model evaluation using the long-term observations of GHI at NREL's Solar Radiation Research Laboratory (SRRL) shows that the PSPI has a better performance than the persistence and smart persistence models in all forecast time horizons between 5 and 60 min, which is more significant in cloudy-sky conditions. Finally, compared to the persistence and smart persistence models, the PSPI does not require additional observations of various atmospheric parameters but is customizable in that additional observations, if available, can be ingested to further improve the GHI forecast. An advanced technology of cloud forecast is also expected to improve the future performance of the PSPI.

14 SOLAR ENERGY↗

Model Validation Database for Fires Involving Fuels at Liquefied Natural Gas Facilities

This document provides a description of the model evaluation protocol (MEP) database for fires involving liquefied natural gas (LNG) and processing fuels at LNG facilities. The purpose of the MEP is to provide procedures regarding the assessment of a model's suitability to predict thermal exclusion zones resulting from a fire. The database includes measurements from pool fire, jet fire, and fireball experiments which are provided in a spreadsheet. Users are to enter model results into the spreadsheet which automatically generates statistical performance measures and graphical comparisons with the experimental data. The intent of this document is to provide a description of the experiments and of the procedure required to carry out the validation portion of the MEP. In addition, the statistical performance measures, measurements for comparisons, and parameter variation are provided.

03 NATURAL GAS↗

Model Validation Database for Fires Involving Fuels at Liquefied Natural Gas Facilities (Version 2)

This document provides a description of the model evaluation protocol (MEP) database for fires involving liquefied natural gas (LNG) and processing fuels at LNG facilities. The purpose of the MEP is to provide procedures regarding the assessment of a model’s suitability to predict thermal exclusion zones resulting from a fire. The database includes measurements from pool fire, jet fire, and fireball experiments which are provided in a spreadsheet. Users are to enter model results into the spreadsheet which automatically generates statistical performance measures and graphical comparisons with the experimental data. The intent of this document is to provide a description of the experiments and of the procedure required to carry out the validation portion of the MEP. In addition, the statistical performance measures, measurements for comparisons, and parameter variation are provided.

03 NATURAL GAS↗

New Particle Formation Event Dataset at the Southern Great Plains (SGP) Observatory from 2018 to 2023

This data set contains observations of new particle formation (NPF) events collected at the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) observatory from 2018 to 2023. Measurements include particle number size distributions, radiance measurements, and associated meteorological variables from onsite instrumentation relevant for identifying and characterizing NPF events. Events were identified using standardized criteria and documented to support investigations of aerosol nucleation, growth dynamics, and their interactions with local atmospheric conditions. The data set provides a multi-year record that enables evaluation of seasonal and interannual variability in NPF occurrence and intensity at a mid-continental site. These data are intended to support studies of aerosol-cloud-climate interactions and model evaluation within both ARM and the broader atmospheric science community. More information can be found within the README_NPF_SGP_2018_2023.docx file.

event↗

Features of a SINDA/FLUINT model of a liquid oxygen supply line

The modeling features used in a steady-state heat transfer problem using SINDA/FLUINT are described. The problem modeled is a 125 feet long, 3 inch diameter pipe, filled with liquid oxygen flow driven by a given pressure gradient. The pipe is fully insulated in five sections. Three sections of 1 inch thick spray-on foam and two sections of vacuum jacket. The model evaluates friction, turns losses and convection heat transfer between the fluid and the pipe wall. There is conduction through the foam insulation with temperature dependent thermal conductivity. The vacuum space is modeled with radiation and gas molecular conduction, if present, in the annular gap. Heat is transferred between the outer surface and surrounding ambient by natural convection and radiation; and, by axial conduction along the pipe and through the vacuum jacket spacers and welded seal flanges. The model makes extensive use of SINDA/FLUINT basic capabilities such as the GEN option for nodes and conductors (to generate groups of nodes or conductors), the SIV option (to generate single, temperature varying conductors), the SIM option (for multiple, temperature varying conductors) and the M HX macros for fluids (to generate strings of lumps, paths, and ties representing a diabatic duct). It calls subroutine CONTRN (returns the relative location in the G-array of a network conductor, given an actual conductor number) enabling an extensive manipulation of conductor (calculates an assignment of their values) with DO loops. Models like this illustrate to the new and even to the old SINDA/FLUINT user, features of the program that are not so obvious or known, and that are extremely handy when trying to take advantage of both, the automation of the DATA headers and make surgical modifications to specific parameters of the thermal or fluid elements in the OPERATIONS portion of the model.

Simmonds, Boris G.↗

Business Case Analysis of the Towed Gilder Air Launched System (TGALS)

The Aerospace Corporation developed an integrated Business Case Analysis (BCA) model on behalf of the NASA Armstrong Flight Research Center (AFRC). This model evaluated the potential profitability of the Towed Glider Air Launched System (TGALS) concept, under development at AFRC, identifying potential technical, programmatic, and business decisions that could improve its business viability. The model addressed system performance metrics; development, production and operation cost estimates; market size and product service positioning; pricing alternatives; and market share.

TGALS↗

Evaluation of HEU-Beryllium Benchmark Experiments to Improve Computational Analysis of Space Reactors

An assessment was previously performed to evaluate modeling capabilities and quantify preliminary biases and uncertainties associated with the modeling methods and data utilized in designing a nuclear reactor such as a beryllium-reflected, highly-enriched-uranium (HEU)-O 2 fission surface power (FSP) system for space nuclear power. The conclusion of the previous study was that current capabilities could preclude the necessity of a cold critical test of the FSP; however, additional testing would reduce uncertainties in the beryllium and uranium cross-section data and the overall uncertainty in the computational models. A series of critical experiments using HEU metal were performed in the 1960s and 1970s in support of criticality safety operations at the Y-12 Plant. Of the hundreds of experiments, three were identified as fast-fission configurations reflected by beryllium metal. These experiments have been evaluated as benchmarks for inclusion in the International Handbook of Evaluated Criticality Safety Benchmark Experiments (IHECSBE). Further evaluation of the benchmark experiments was performed using the sensitivity and uncertainty analysis capabilities of SCALE 6. The data adjustment methods of SCALE 6 have been employed in the validation of an example FSP design model to reduce the uncertainty due to the beryllium cross section data.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Evaluating penalized logistic regression models to predict Heat-Related Electric grid stress days

Understanding the conditions associated with stress on the electricity grid is important in the development of contingency plans for maintaining reliability during periods when the grid is stressed. In this paper, heat-related grid stress and the relationship with weather conditions is examined using data from the eastern United States. Penalized logistic regression models were developed and applied to predict stress on the electric grid using weather data. The inclusion of other weather variables, such as precipitation, in addition to temperature improved model performance. Several candidate models and datasets were examined. A penalized logistic regression model fit at the operation-zone level was found to provide predictive value and interpretability. Additionally, the importance of different weather variables observed at different time scales were examined. Maximum temperature and precipitation were identified as important across all zones while the importance of other weather variables was zone specific. The methods presented in this work are extensible to other regions and can be used to aid in planning and development of the electrical grid.

Bramer, L. M.↗

Towards a flexible framework for community-wide ensemble forecasting tailored for major space environment impacts

To build continuously improving space weather predictive capabilities based on science and enable assessments and rapid implementations of advances in research into source-to-impact modelling systems we need: to assemble parts of the puzzle by solving problems focused on specific physical domains; to identify essential space environment quantities (ESEQs) passed between domains and linked to impacts; evaluate modeling capabilities for each ESEQ; connect all validated solutions from space weather origins on the sun to impacts on humans and critical infrastructure; to design displays for ensemble predictions tailored for major space weather user groups; to build a collaborative environment for efficient sharing of information and capabilities (models/data/expertise) and collaborative development. The presentation will overview existing community-wide space weather forecasting frameworks and research-to-operations pipelines and discuss opportunities to build a collaborative plug-and-play platform for interconnecting predictive capabilities developed under different space weather programs.

space weather↗