Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

SRS Composite Analysis Monitoring Plan (FY2019 Model Validation)

Using a projected end-state date of 2065 (SRNS 2015b), the Savannah River Site (SRS) Composite Analysis (CA) modeling for each facility and waste site began on the inventory year assigned to it so that source depletion and radionuclide transport out of the system could be appropriately captured. Some SRS waste sites that have already achieved their end states (i.e. end-state inventories and end-state configuration) are currently contributing to the potential off-site public dose through source release, groundwater transport, discharge to on-site surface streams, and stream transport to the CA point of assessments (POAs). The inventory year assigned to these waste sites is 2002 or before. This means that SRS CA results from 2002 and beyond are a reasonable representation for these waste sites that have already achieved their end states and are currently contributing to the potential off-site public dose. The SRS Annual Environmental Report (AER) monitoring can differentiate and separate liquid pathway data allowing the data representing only waste sites at their end state to be produced. Because the SRS CA has projected reasonable end-state impacts from 2002 and beyond, and the AER monitoring can differentiate and separate operating and end-state contributions to annual liquid pathway release, an opportunity exists to use the AER monitoring data to validate the SRS CA model. The CA model validation program uses a graded and systematic approach for taking corrective action, starting with an SRS established administrative dose limit of 15 mrem/yr, below which no action is required. Based on the location of the 2010 SRS CA POAs, the only potential exposure pathway for the public is through surface water. The completion of the FY2019 CA model validation indicates that the SRS CA projected dose, while generally conservative, provides a reasonable representation of the maximum annual doses. These doses are well below the administrative limit; therefore, no additional action is required.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

Hydrogen and Ammonia Blending With Natural Gas: Ignition Delay Times and Chemical Kinetic Model Validation at Gas Turbine Relevant Conditions

Abstract Ignition delay times from undiluted mixtures of natural gas (NG)/H2/Air and NG/NH3/Air were measured using a high-pressure shock tube at the University of Central Florida. The combustion temperatures were experimentally tested between 1000–1500 K near a constant pressure of 25 bar. As mentioned, mixtures were kept undiluted to replicate the same chemistry pathways seen in gas turbine combustion chambers. Recorded combustion pressures exceeded 200 bar due to the large energy release, hence why these were performed at the high-pressure shock tube facility. The data is compared to the predictions of the NUIGMech 1.1 mechanism for chemical kinetic model validation and refinement. An exceptional agreement was shown for stoichiometric conditions in all cases but strayed at lean and rich equivalence ratios, especially in the lower temperature regime of H2 addition and all temperature ranges of the baseline NG mixture. Hydrogen addition also decreased ignition delay times by nearly 90%, while NH3 fuel addition made no noticeable difference in ignition time. NG/NH3 exhibited similar chemistry to pure NG under the same conditions, which is shown in a sensitivity analysis. The reaction CH3 + O2 = CH3O + O is identified and suggested as a possible modification target to improve model performance. Increasing the robustness of chemical kinetic models via experimental validation will directly aid in designing next-generation combustion chambers for use in gas turbines, which in turn will greatly lower global emissions and reduce greenhouse effects.

Pierro, Michael↗
DOE OSTI2022

Overview of the TJ-II stellarator research programme towards model validation in fusion plasmas

TJ-II stellarator results on modelling and validation of plasma flow asymmetries due to on-surface potential variations, plasma fuelling physics, Alfvén eigenmodes (AEs) control and stability, the interplay between turbulence and neoclassical (NC) mechanisms and liquid metals are reported. Regarding the validation of the neoclassically predicted potential asymmetries, its impact on the radial electric field along the flux surface has been successfully validated against Doppler reflectometry measurements. Research on the physics and modelling of plasma core fuelling with pellets and tracer encapsulated solid pellet injection has shown that, although post-injection particle radial redistributions can be understood qualitatively from NC mechanisms, turbulence and fluctuations are strongly affected during the ablation process. Advanced analysis tools based on transfer entropy have shown that radial electric fields do not only affect the radial turbulence correlation length but are also capable of reducing the propagation of turbulence from the edge into the scrape-off layer. Direct experimental observation of long range correlated structures show that zonal flow structures are ubiquitous in the whole plasma cross-section in the TJ-II stellarator. Alfvénic activity control strategies using ECRH and ECCD as well as the relation between zonal structures and AEs are reported. Finally, the behaviour of liquid metals exposed to hot and cold plasmas in a capillary porous system container was investigated.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

MAPPER – A NOVEL CAPABILITY TO SUPPORT NUCLEAR MODEL VALIDATION AND MAPPING OF BIASES AND UNCERTAINTIES

This paper overviews the initial results of a new project at the Oak Ridge National Laboratory -- supported via an internal seed funding program -- focused on the development of a novel computational capability, called MAPPER, to support model validation. The MAPPER development aims to eliminate the need for empirical criteria, such as similarity indices, often employed to identify applicable experiments for given application conditions. To achieve that, MAPPER employs an information-theoretic-based approach, based on the Kullback-Leibler (KL) divergence principle, to combine responses of available or to-be-built experiments with application responses of interest using a training set of samples generated using randomized execution of the experiments and the application high fidelity analysis models. These samples are condensed using reduced order modeling techniques in the form of a joint probability distribution function (PDF) connecting each application response of interest with a new effective experimental response. The initial focus of the MAPPER capability will be to support the confirmatory analysis required for criticality safety analysis of storage facilities which require k-eff biases to be known for safe operation. This paper reports some of the initial results obtained with MAPPER as applied to a set of critical experiments for which existing similarity-based methods have been shown to provide inaccurate estimates of the biases.

Mertyurek, Ugur↗
DOE OSTI2021

ACCRUE—An Integral Index for Measuring Experimental Relevance in Support of Neutronic Model Validation

A key challenge for the introduction of any design changes, e.g., advanced fuel concepts, first-of-a-kind nuclear reactor designs, etc., is the cost of the associated experiments, which are required by law to validate the use of computer models for the various stages, starting from conceptual design, to deployment, licensing, operation, and safety. To achieve that, a criterion is needed to decide on whether a given experiment, past or planned, is relevant to the application of interest. This allows the analyst to select the best experiments for the given application leading to the highest measures of confidence for the computer model predictions. The state-of-the-art methods rely on the concept of similarity or representativity, which is a linear Gaussian-based inner-product metric measuring the angle—as weighted by a prior model parameters covariance matrix—between two gradients, one representing the application and the other a single validation experiment. This manuscript emphasizes the concept of experimental relevance which extends the basic similarity index to account for the value accrued from past experiments and the associated experimental uncertainties, both currently missing from the extant similarity methods. Accounting for multiple experiments is key to the overall experimental cost reduction by prescreening for redundant information from multiple equally-relevant experiments as measured by the basic similarity index. Accounting for experimental uncertainties is also important as it allows one to select between two different experimental setups, thus providing for a quantitative basis for sensor selection and optimization. The proposed metric is denoted by ACCRUE, short for Accumulative Correlation Coefficient for Relevance of Uncertainties in Experimental validation. Using a number of criticality experiments for highly enriched fast metal systems and low enriched thermal compound systems with accident tolerant fuel concept, the manuscript will compare the performance of the ACCRUE and basic similarity indices for prioritizing the relevance of a group of experiments to the given application.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Adaptive Protection and Validated Models to Enable Deployment of High Penetrations of Solar PV (PV-MOD)

The availability and validation of various PV models in commercial tools differ, with some models not yet thoroughly validated for advanced inverter functionalities and reliable performance under weak system conditions. Many existing models do not fully incorporate new inverter control functions, which can affect system stability. The increasing deployment of solar PV and other inverter-based resources (IBRs), including distributed energy resources (DERs), is influencing the reliable operation of protection schemes in distribution systems and microgrids. Emerging adaptive protection schemes (APS) offer new opportunities for protecting these systems during varying configurations and DER operating conditions, though their demonstration and validation remain limited. Adaptive protection schemes face similar challenges, as they are typically designed for specific configurations. There is a growing need for tools and methodologies to streamline the deployment of adaptive protection for safe and reliable DER integration. The project main objective was to develop and validate high-fidelity generic models of solar PV facilities for stability, protection, EMT, and QSTS analyses. This objective was achieved, and these models can now be integrated into commercial software tools, enabling utilities, vendors, and developers to study high-penetration PV systems more confidently. The project also demonstrated advanced applications of these models, including the design and deployment of adaptive protection schemes in high-penetration field applications and microgrids, supporting grid safety and reliability. Several milestones were reached by the end of the project. A sophisticated inverter test plan was developed, and inverters representative of the North American marketplace were selected. EPRI and NREL tested various inverters, conforming to IEEE standards. Improvements were made to existing generic models of IBR units, IBR plants, and aggregated feeders for various analyses. The first generic electromagnetic transient (EMT) model for a solar PV plant was developed, conforming to IEEE Std 2800™-2022 and validated against laboratory measurements of a 2.2 MVA large-scale battery energy storage system (BESS) inverter. That model was then used to produce reference responses illustrating examples of validated and verified IBR plant models that pass or fail tests for technical minimum capability and performance as specified in the IEEE standard. The developed, tested, and validated generic models can be used for transmission planning, stability assessments, expansion planning, and evaluating potential future IBR interconnection requirements. They can also support interconnection screens and conformity assessments of IBR plants, including solar PV. The project significantly contributed to the ongoing standardization and model-based representation and verification of IBR responses. The project further addressed challenges of common distribution protection schemes with increasing deployment of DER by developing, validating, and demonstrating adaptive protection schemes (APS) that can improve the reliable and safe integration of DER into distribution systems. New APS were designed using improved DER models for three common distribution systems: a radial feeder, a meshed network, and a microgrid. Modeling and hardware-in-the-loop (HIL) testing of the APS were conducted, successfully showing their effectiveness and selectivity. Proof-of-concept field demonstration was achieved for two APS, i.e., one on a radial feeder and another one in a microgrid. Field demonstration could not be achieved for the APS on a meshed network, primarily due apprehension of one utility partner and also due to limited access to the protective algorithms in the network protectors. Guidelines developed from the lessons learned in the project lay out the general process followed in the design, installation, and commissioning of APS for various distribution systems. Distribution utility partners’ apprehension about field demonstration of the new APS were addressed—with varying success—by taking a stepped risk-management approach of modeling of a wide range of sensitivities first, performing in-depth proof-of-concept testing in the laboratory including HIL next, and finally deliberately implementing and commissioning the actual protection equipment and algorithms into parts of—or in parallel operation to—the three real distribution systems. Future work should include pilot projects that further show the acceptable performance of the developed APS before these schemes be rolled out more widely. Inclusion of both utility and original equipment manufacturers (OEMs) in future projects could increase chances of successful field demonstration. Despite challenges in achieving the field demonstration goal of the project for all three APS, the research significantly contributed to the innovation of adaptive protection solutions for scalable and reliable DER integration into distribution systems. This project significantly enhances the understanding of the impact of using appropriate inverter models on distribution and transmission (T&D) systems. By addressing the limitations of existing generic models, the project introduces high-fidelity models for stability, protection, electromagnetic transient (EMT), and quasi-static time series (QSTS) analyses. These models, integrated into commercial software tools, enable utilities, vendors, and developers to confidently study high-penetration PV systems. The project also demonstrates advanced applications, including adaptive protection schemes (APS) for distribution systems and microgrids, ensuring grid safety and reliability. The technical effectiveness and economic feasibility of the methods are evident through the development and validation of sophisticated inverter test plans and the selection of representative inverters. Testing by EPRI and NREL on retail, commercial, and utility-scale inverters, conforming to IEEE standards, underscores the robustness of the models. Improvements to existing generic models for various analyses further enhance their validity and applicability. The project also identifies gaps in common distribution protection schemes and designed new APS using improved DER models, demonstrating their effectiveness through modeling and hardware-in-the-loop (HIL) testing. The project’s benefits to the public are manifold. By advancing the standardization and model-based representation of IBR response, it supports transmission planning, stability assessments, and future IBR interconnection requirements. The generic models can facilitate better communication between transmission planners and developers, supporting expected IBR plant capability and performance. Additionally, the development of APS for radial feeders, meshed networks, and microgrids supports the integration of distributed energy resources (DERs) into distribution systems, enhancing grid reliability and safety. The project’s emphasis on thorough testing and simplicity in design ensures practical and scalable solutions for DER integration.

14 SOLAR ENERGY↗
DOE OSTI2023

Physics model validation of propane and methane for Hydrogen Plus Other Alternative Fuels Risk Assessment Models (HyRAM+)

HyRAM+ is a toolkit that includes fast-running models for the unconstrained (i.e., no wall interactions) dispersion and flames for non-premixed fuels. The models were developed for use with hydrogen, but the toolkit was expanded to include propane and methane in a recent release. Here, in this work we validate the dispersion and flame models for these additional fuels, based on reported literature data. The validation efforts spanned a range of release conditions, from subsonic to underexpanded jets and flames for a range of mass flow rates. In general, the dispersion model works well for both propane and methane although the width of the jet/plume is predicted to be wider than observed in some cases. The flame model tends to over-predict the induced buoyancy for low-momentum flames, while the radiative heat flux agrees with the experimental data reasonably well, for both fuels. The models could be improved but give acceptable predictions for propane and methane behavior for the purposes of risk assessment.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2019

Benchmarks for Model Validation based on LiDAR Wake Measurements

Technology development and design decisions in wind energy are often based on results from simulations performed for individual wind turbines or entire wind plants. It is therefore critical to ensure that the models being used for research and industry applications in wind energy be thoroughly validated against measurements. A full-system validation of wind plant simulations must consider the atmospheric inflow, the response of the wind turbines, and their wakes. This task is complicated by the lack of freely available, quality-controlled, high-quality measurements. Here, such measurements are used to offer a validation exercise that can be used to assess the accuracy of models of any fidelity level. When it comes to real-world measurements, the dataset considered herein is simple in terms of terrain but exhibits pronounced diurnal cycles. Instead of a full-scale wind plant, we consider an individual research-scale, utility wind turbine instrumented for power and loads measurements. Three benchmarks are defined, with increasing levels of complexity: near neutral, slightly unstable, and very stable atmospheric stratification. Through comparisons between observations and simulations, the benchmarks provide complementary information about the model performance and its ability to reproduce mean and dynamic wake characteristics. This article describes the measurements and methodology used to define these benchmarks and provides the information required to perform simulations and conduct the model-measurement comparison. The objective is to provide a robust wake model validation exercise open to anyone, which will serve to minimize uncertainty in model validation practices related to varying methodologies across simulation tools and users.

17 WIND ENERGY↗
DOE OSTI2019

Comprehensive experimental investigation on biomass-glass beads binary fluidization: A data set for CFD model validation

Fluidization behavior of biomass and glass beads binary mixtures in a bubbling fluidized bed was experimentally investigated. Mixtures containing different mass fraction of Loblolly Pine white wood and glass beads were fluidized at different fluidization velocities. The particle properties were characterized in a QICPIC that uses a dynamic image processing method to measure both particle size and sphericity. The minimum fluidization velocity was determined using the pressure drop method. An image processing method was developed to capture the dynamic expanded bed height at a very high frequency. The effect of biomass mass fraction and inlet gas velocity on mixing and segregation behavior was studied and analyzed through pressure drop measurements. Pressure drop fluctuations and expanded bed height fluctuations via fast Fourier transform were analyzed and compared. Here, the complete and accurate experimental data reported in this study could provide a benchmark data set for various computational fluid dynamics models validation, calibration, and identification.

42 ENGINEERING↗
DOE OSTI2023

Development of a wind turbine model and simulation platform using an acausal approach: Multiphysics modeling, validation, and control

This article presents the development of the Control-oriented, Reconfigurable, and Acausal Floating Turbine Simulator (CRAFTS). CRAFTS has a modular, hierarchical model architecture that enables rapid and accurate simulation of wind turbines. The architecture facilitates the incorporation of model variants, and its system reconfiguration features help simulate multiple design variants. CRAFTS also supports the integration of models developed on a causality-free platform (e.g., Modelica®) with existing causal models. This article focuses on the validation of a land-based wind turbine models against OpenFAST, an industry-standard platform, for several test cases. Closed-loop scenarios are also tested using the standard ROSCO controller and compared against OpenFAST. In addition, a nonlinear controller developed in our prior work is also evaluated. The test cases demonstrate the user-friendly and computationally efficient capabilities of CRAFTS to facilitate control co-design, assist in incorporating multiphysics models, be adaptable to design variants, and allow for rapid simulations to validate models and evaluate controllers.

17 WIND ENERGY↗
DOE OSTI2021

Multiscale Modeling of a Direct Nonoxidative Methane Dehydroaromatization Reactor with a Validated Model for Catalyst Deactivation

Due to the recent boom in shale gas production, aromatics production using direct nonoxidative methane dehydroaromatization (DHA) is being investigated extensively. However, due to rapid coke formation, catalysts in the nonoxidative methane DHA reactors get deactivated, which is one of the critical issues for the commercial success of the methane DHA process. In this paper, a model for catalyst deactivation is developed. Rate models for other DHA reactions are developed by considering the decrease in the catalyst activity with time. Due to the very fast coke formation rate on the fresh catalyst, there is coke formation immediately upon the introduction of the feed. Therefore, an algorithm is developed for estimation of the initial state of the reactor and the kinetic parameters by coupling an iterative direct substitution approach with an optimization approach. Transient experimental data from an in-house reactor are first reconciled and then used for developing the kinetic model including the coke formation model. Using the rate model, a dynamic, heterogeneous, multiscale reactor model with embedded heating is developed. Here, the model couples the catalyst pellet level model with a reactor level model. Impacts of temperature, L/D ratio, and scheduling of reactors on variability in conversion and yield with time are studied.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Coarse Woody Debris Decomposition Assessment Tool: Model validation and application

Coarse woody debris (CWD) is a significant component of the forest biomass pool; hence a model is warranted to predict CWD decomposition and its role in forest carbon (C) and nutrient cycling under varying management and climatic conditions. A process-based model, CWDDAT (Coarse Woody Debris Decomposition Assessment Tool) was calibrated and validated using data from the FACE (Free Air Carbon Dioxide Enrichment) Wood Decomposition Experiment utilizing pine ( Pinus taeda ), aspen ( Populous tremuloides ) and birch ( Betula papyrifera ) on nine Experimental Forests (EF) covering a range of climate, hydrology, and soil conditions across the continental USA. The model predictions were evaluated against measured FACE log mass loss over 6 years. Four widely applied metrics of model performance demonstrated that the CWDDAT model can accurately predict CWD decomposition. The R 2 (squared Pearson’s correlation coefficient) between the simulation and measurement was 0.80 for the model calibration and 0.82 for the model validation ( P <0.01). The predicted mean mass loss from all logs was 5.4% lower than the measured mass loss and 1.4% lower than the calculated loss. The model was also used to assess the decomposition of mixed pine-hardwood CWD produced by Hurricane Hugo in 1989 on the Santee Experimental Forest in South Carolina, USA. The simulation reflected rapid CWD decomposition of the forest in this subtropical setting. The predicted dissolved organic carbon (DOC) derived from the CWD decomposition and incorporated into the mineral soil averaged 1.01 g C m -2 y -1 over the 30 years. The main agents for CWD mass loss were fungi (72.0%) and termites (24.5%), the remainder was attributed to a mix of other wood decomposers. These findings demonstrate the applicability of CWDDAT for large-scale assessments of CWD dynamics, and fine-scale considerations regarding the fate of CWD carbon.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

TEAMER - AquaHarmonics High Fidelity WEC Sim PTO and Control Model Validation, Test Logs and Results

Collaborative effort between AquaHarmonics, Sandia National Laboratories (SNL), and the National Renewable Energy Laboratory (NREL) to revise and validate Aquaharmonics' full wave to wire model, allowing for reduced uncertainty and increased understanding of design requirements of a utility scale wave energy converter (WEC). SNL and NREL in collaboration with AquaHarmonics, will set up and run WEC Simulator (WEC-Sim) models of the AquaHarmonics WEC, building off past model developments for inclusion of custom PTO (power take-off) dynamics. The intent is to review, update, and verify or validate a new WEC-Sim model against wave tank experimental data. Furthermore, the WEC-Sim model will be coupled to an energy storage system model to better understand the wave-to-wire functionality. This data set is described in the "Test Log" excel file. Please refer to that document for details on each specific test date/time, constraint parameters and model hardware setup details. Sim model can be found in the associated MHKDR link below.

16 TIDAL AND WAVE POWER↗
DOE OSTI2020

A Numerical Study of Hypoxia in Chesapeake Bay Using an Unstructured Grid Model: Validation and Sensitivity to Bathymetry Representation

A three-dimensional unstructured-grid hydrodynamic and water quality model (Semi-implicit Cross-scale Hydroscience Intergrated System Model-Integrated Compartment Model) is applied successfully for Chesapeake Bay. The model is validated with observations of salinity, chlorophyll-a, dissolved oxygen, nutrients, and phytoplankton productions from the year 1991 to 1995 for the mainstem and some major tributaries, based on multiple model skill scores. Model experiments are conducted to test the importance of having (1) an accurate representation of bathymetry to correctly predict hypoxia and other processes and (2) a high-resolution model grid for tributaries to correctly simulate water quality variables. Further, comparison with the model experiment results with bathymetry smoothing indicates that bathymetry smoothing, as commonly used for many systems, changes the stratification and lateral circulation pattern, resulting in more salt intrusion into shallow water regions, and an increase in the freshwater age. Consequently, a model with bathymetry smoothing can lead to an unrealistic prediction of the distribution of hypoxia and phytoplankton production. Local grid refinement shows significant improvement of model simulations on local stratification and water quality variables. Overall, the use of high-resolution unstructured grid model leads to a faithful representation of the complex geometry, and thus a seamless cross-scale capability for simulating water quality processes in the Bay including tributaries and tidal creeks.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Selection and Ranking of Experiments from the Halden Database in support of Multiscale Model Validation

Validating fuel performance codes, such as BISON, requires an extensive amount of experiments covering a wide range of operating conditions and fuel types. Within the light-water reactor (LWR) space, there have been several international experimental programs that have contributed to the wealth of available experimental data available for use. One of those international programs, the Halden Reactor Project (HRP), began in 1958 and utilized the Halden Boiling Water Reactor to conduct many highly instrumented experiments until the reactor closed in 2018. Idaho National Laboratory, through the U.S. Department of Energy, has utilized several Halden experiments to perform the initial validation of the BISON code based upon their inclusion in international modeling and simulation benchmarks. Recently, the HRP has provided member organizations a complete copy of all available data, reports, and presentations since the HRP began. This report provides an initial exploration of the data available in the database for use in validating the multiscale models under development in the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program for LWR applications. Ranking tables that identify potential validation cases are provided for the high priority models of interest. It was found that some of the recommended high priority experiments correspond to additional rods in existing assemblies already available in the BISON validation suite. It is expected that several of these cases will be incorporated into future NEAMS milestones in the fuels technical area for increased validation of BISON.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Model validation and error attribution for a drifting qubit

Qubit performance is often reported in terms of a variety of single-value metrics, each providing a facet of the underlying noise mechanism limiting performance. However, the value of these metrics may drift over long timescales, and reporting a single number for qubit performance fails to account for the low-frequency noise processes that give rise to this drift. Here, in this work, we demonstrate how we can use the distribution of these values to validate or invalidate candidate noise models. We focus on the case of randomized benchmarking (RB), where typically a single error rate is reported but this error rate can drift over time when multiple passes of RB are performed. We show that using a statistical test as simple as the Kolmogorov-Smirnov statistic on the distribution of RB error rates can be used to rule out noise models, assuming the experiment is performed over a long enough time interval to capture relevant low frequency noise. With confidence in a noise model, we show how care must be exercised when performing error attribution using the distribution of drifting RB error rate.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2019

LIVVkit 2.1: automated and extensible ice sheet model validation

A collection of scientific analyses, metrics, and visualizations for robustvalidation of ice sheet models is presented using the Land Ice Verificationand Validation toolkit (LIVVkit), version 2.1, and the LIVVkit Extensionsrepository (LEX), version 0.1. This software collection targets stand-aloneice sheet or coupled Earth system models, and handles datasets and analysesthat require high-performance computing and storage. LIVVkit aims to enableefficient and fully reproducible workflows for postprocessing, analysis, andvisualization of observational and model-derived datasets in a shareableformat, whereby all data, methodologies, and output are distributed to usersfor evaluation. Extending from the initial LIVVkit software framework, wedemonstrate Greenland ice sheet simulation validation metrics using thecoupled Community Earth System Model (CESM) as well as an idealizedstand-alone high-resolution Community Ice Sheet Model, version 2 (CISM2),coupled to the Albany/FELIX velocity solver (CISM-Albany or CISM-A). As oneexample of the capability, LIVVkit analyzes the degree to which modelscapture the surface mass balance (SMB) and identifies potential sources ofbias, using recently available in situ and remotely sensed data ascomparison. Related fields within atmosphere and land surface models, e.g.,surface temperature, radiation, and cloud cover, are also diagnosed. Appliedto the CESM1.0, LIVVkit identifies a positive SMB bias that is focusedlargely around Greenland's southwest region that is due to insufficientablation.

58 GEOSCIENCES↗