Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Utilization of Model Predictive Control to Balance Power Absorption Against Load Accumulation

Wave energy converter (WEC) control strategies have been primarily focused on maximizing power absorption. The use of model predictive control strategies allows for a finite-horizon, multiterm objective function to be solved. This work utilizes a multiterm objective function to maximize power absorption while minimizing the structural loads on the WEC system. Furthermore, a Kalman filter and autoregressive model were used to estimate and forecast the wave exciting force and predict the future dynamics of the WEC. The WEC's power-take-off time-averaged power and structural loads under a perfect forecast assumption in irregular waves were compared against results obtained from the Kalman filter and autoregressive model to evaluate model predictive control performance.

16 TIDAL AND WAVE POWER↗
NASA NTRS1976

Dynamic modeling for evaluation of solar collector performance

Dynamic modeling is being developed to provide a tool for systematic analysis of performance characteristics in energy systems where time-dependent behavior is important. A dynamic model program designed to aid in the understanding of solar collector behavior over the full range of operating conditions was constructed and tested in the evaluation of a conceptual collector design.

Hamilton, C. L.↗
DOE OSTI2023

A seamless approach for evaluating climate models across spatial scales

In regions of the world where topography varies significantly with distance, most global climate models (GCMs) have spatial resolutions that are too coarse to accurately simulate key meteorological variables that are influenced by topography, such as clouds, precipitation, and surface temperatures. One approach to tackle this challenge is to run climate models of sufficiently high resolution in those topographically complex regions such as the North American Regionally Refined Model (NARRM) subset of the Department of Energy’s (DOE) Energy Exascale Earth System Model version 2 (E3SM v2). Although high-resolution simulations are expected to provide unprecedented details of atmospheric processes, running models at such high resolutions remains computationally expensive compared to lower-resolution models such as the E3SM Low Resolution (LR). Moreover, because regionally refined and high-resolution GCMs are relatively new, there are a limited number of observational datasets and frameworks available for evaluating climate models with regionally varying spatial resolutions. As such, we developed a new framework to quantify the added value of high spatial resolution in simulating precipitation over the contiguous United States (CONUS). To determine its viability, we applied the framework to two model simulations and an observational dataset. We first remapped all the data into Hierarchical Equal-Area Iso-Latitude Pixelization (HEALPix) pixels. HEALPix offers several mathematical properties that enable seamless evaluation of climate models across different spatial resolutions including its equal-area and partitioning properties. The remapped HEALPix-based data are used to show how the spatial variability of both observed and simulated precipitation changes with resolution increases. This study provides valuable insights into the requirements for achieving accurate simulations of precipitation patterns over the CONUS. It highlights the importance of allocating sufficient computational resources to run climate models at higher temporal and spatial resolutions to capture spatial patterns effectively. Furthermore, the study demonstrates the effectiveness of the HEALPix framework in evaluating precipitation simulations across different spatial resolutions. This framework offers a viable approach for comparing observed and simulated data when dealing with datasets of varying spatial resolutions. By employing this framework, researchers can extend its usage to other climate variables, datasets, and disciplines that require comparing datasets with different spatial resolutions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Land Model Testbed: Accelerating Development, Benchmarking and Analysis of Land Surface Models

A Land Model Testbed (LMT), designed to provide a computational framework for systematically assessing model fidelity and supporting rapid development of complex multiscale models, offers a general-purpose workflow for conducting large ensemble simulations of multiple land surface models, post-processing large volumes of model output, and evaluating model results. It leverages existing tools for launching model simulations and the International Land Model Benchmarking (ILAMB) package for assessing model fidelity through comparison with best-available observational datasets. Increased complexity and proliferation of uncertain parameters in process representations in land surface models has driven the need for frequent and intensive testing and evaluating of models to quantify uncertainties and optimize parameters such that results are consistent with observations. The LMT described here meets these needs by providing tools to run thousands of ensemble simulations simultaneously and post-process their output files, by automating execution of an enhanced version of ILAMB with site-specific benchmarks and multivariate functional relationships, and by offering ensemble diagnostics and a customizable dashboard for displaying model performance metrics and associated graphics. We envision the LMT capabilities will serve as a foundational computational resource for a proposed user facility focused on terrestrial multiscale model--data integration.

Sreepathi, Sarat↗
DOE OSTI2015

External Peer Review Team Report for Corrective Action Unit 97: Yucca Flat/Climax Mine, Nevada National Security Site, Nye County, Nevada, Revision 0

The peer review team commends the Navarro-Intera, LLC (N-I), team for its efforts in using limited data to model the fate of radionuclides in groundwater at Yucca Flat. Recognizing the key uncertainties and related recommendations discussed in Section 6.0 of this report, the peer review team has concluded that U.S. Department of Energy (DOE) is ready for a transition to model evaluation studies in the corrective action decision document (CADD)/corrective action plan (CAP) stage. The DOE, National Nuclear Security Administration Nevada Field Office (NNSA/NFO) clarified the charge to the peer review team in a letter dated October 9, 2014, from Bill R. Wilborn, NNSA/NFO Underground Test Area (UGTA) Activity Lead, to Sam J. Marutzky, N-I UGTA Project Manager: “The model and supporting information should be sufficiently complete that the key uncertainties can be adequately identified such that they can be addressed by appropriate model evaluation studies. The model evaluation studies may include data collection and model refinements conducted during the CADD/CAP stage. One major input to identifying ‘key uncertainties’ is the detailed peer review provided by independent qualified peers.” The key uncertainties that the peer review team recognized and potential concerns associated with each are outlined in Section 6.0, along with recommendations corresponding to each uncertainty. The uncertainties, concerns, and recommendations are summarized in Table ES-1. The number associated with each concern refers to the section in this report where the concern is discussed in detail.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Evaluating mesoscale model predictions of diurnal speedup events in the Altamont Pass Wind Resource Area of California

Mesoscale model predictions of wind, turbulence, and wind energy capacity factors are evaluated in the Altamont Pass Wind Resource Area of California (APWRA), where the diurnal regional sea breeze and associated terrain-driven speedup flows drive wind energy production during the summer months. Results from the Weather Research and Forecasting model version 4.4 using a novel three-dimensional planetary boundary layer (3D PBL) scheme, which treats both vertical and horizontal turbulent mixing, are compared to those using a well-established one-dimensional (1D) scheme that treats only vertical turbulent mixing. Each configuration is evaluated over a nearly 3-month-long period during the Hill Flow Study, and due to the recurring nature of the observed speedup flows, diurnal composite averaging is used to capture robust trends in model performance. Both model configurations showed similar overall skill. The general timing and direction of the speedup flows is captured, but their magnitude is overestimated within a typical wind turbine rotor layer. Both also fail to capture a persistent observed near-surface jet-like flow, likely due to the limited grid resolution that is typical of mesoscale models. However, the 3D PBL configuration shows several minor improvements over the 1D PBL configuration, including improved wind speed and turbulence kinetic energy profiles during the accelerating phase of the speedup events, as well as reduced positive wind speed bias at surface stations across the APWRA region. Using a mesoscale wind farm parameterization, modeled capacity factors are also compared to monthly data reported to the US Energy Information Administration (EIA) during the study period. Although the monthly trend in the data is captured, both model configurations overestimate capacity factors by roughly 7 %–11 %. Through model evaluation, this study provides confidence in the 3D PBL scheme for wind energy applications in complex terrain and provides guidance for future testing.

17 WIND ENERGY↗
DOE OSTI2018

DWT-Based Aggregated Load Modeling and Evaluation for Quasi-Static Time-Series Simulation on Distribution Feeders

This paper presents an extension of a previously reported discrete wavelet transform (DWT) based load modeling methodology which targets at modeling time-series load profiles to enable the more effective and realistic quasi-static time-series simulation of distribution feeders. The time-series load model is composed of two major parts: 1) The low-resolution field-measured load data which usually have a resolution of 30 or 15 min., 2) The high-resolution load variability model data extracted from the established variability database which can have resolution up to 1 sec. A load aggregation methodology is developed to aggregate the load profiles so that load profiles at different transformer ratings can be effectively modeled. Validation, evaluation, and analysis of the developed load modeling approach has been performed on the IEEE-123 feeder and an actual utility feeder from California. The analysis completed on the two feeders have demonstrated the effectiveness and revealed the value of the developed model for distribution feeder quasi-static time-series simulation at high temporal resolution.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Trustworthiness modeling and evaluation for a nearly autonomous management and control system

The Nearly Autonomous Management and Control (NAMAC) system supports the advanced reactor operation by recommending control actions to operators based on real-time measurements and digital twins (DTs) learning from the knowledge base. To enable the safe and reliable use of autonomous technologies, NAMAC and its recommendations should be trustworthy to operators and regulators at both the design and operation stages. This study proposes a NAMAC trustworthiness modeling and evaluation framework supported by trustworthiness ontologies and evidence-based approaches. The development-time and run-time ontologies are separately constructed and then converted to Bayesian networks to quantitatively evaluate the NAMAC trustworthiness. This evaluation is demonstrated by collecting and characterizing evidence from NAMAC practices, such as the development and assessment of the NAMAC system, data coverage assessment, and the training and optimizations of neural-network-based DTs. Our proposed approach can aggregate various trustworthiness attributes of complex artificial-intelligence-supported systems for safety-critical applications. It also considers the interaction between different DTs and extends beyond the trustworthiness evaluation of a single DT. In conclusion, the evidence-based method enhances the transparency of the trustworthiness modeling and evaluation processes and helps identify uncertainties and subjectivity involved in the processes.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2012

Beyond the buildingcentric approach: A vision for an integrated evaluation of sustainable buildings

The available sustainable building evaluation systems have produced a new environmental design paradigm. However, there is an increasing need to overcome the buildingcentric approach of these systems, in order to further exploit their innovate potential for sustainable building practices. The paper takes this challenge by developing a cross-scale evaluation approach focusing on the reliability of sustainable building design solutions for the context in which the building is situated. An integrated building-urban evaluation model is proposed based on the urban matrix, which is a conceptualisation of the built environment as a social-ecological system. The model aims at evaluating the sustainability of a building considering it as an active entity contributing to the resilience of the urban matrix. Few holistic performance indicators are used for evaluating such contribution, so expressing the building reliability. The discussion on the efficacy of the model shows that it works as a heuristic tool, supporting the acquisition of a better insight into the complexity which characterises the relationships between the building and the built environment sustainability. Shading new lights on the meaning of sustainable buildings, the model can play a positive role in innovating sustainable building design practices, thus complementing current evaluation systems. - Highlights: Black-Right-Pointing-Pointer We model an integrated building-urban evaluation approach. Black-Right-Pointing-Pointer The urban matrix represents the social-ecological functioning of the urban context. Black-Right-Pointing-Pointer We introduce the concept of reliability to evaluate sustainable buildings. Black-Right-Pointing-Pointer Holistic indicators express the building reliability. Black-Right-Pointing-Pointer The evaluation model works as heuristic tool and complements other tools.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

DWT-Based Aggregated Load Modeling and Evaluation for Quasi-Static Time-Series Simulation on Distribution Feeders: Preprint

This paper presents an extension of a previously reported discrete wavelet transform (DWT) based load modeling methodology which targets at modeling time-series load profiles to enable the more effective and realistic quasi-static time-series simulation of distribution feeders. The time-series load model is composed of two major parts: 1) The low-resolution field-measured load data which usually have a resolution of 30 or 15 min., 2) The high-resolution load variability model data extracted from the established variability database which can have resolution up to 1 sec. A load aggregation methodology is developed to aggregate the load profiles so that load profiles at different transformer ratings can be effectively modeled. Validation, evaluation, and analysis of the developed load modeling approach has been performed on the IEEE-123 feeder and an actual utility feeder from California. The analysis completed on the two feeders have demonstrated the effectiveness and revealed the value of the developed model for distribution feeder quasi-static time-series simulation at high temporal resolution.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2009

Probabilistic Multi-Factor Interaction Model for Evaluating Continuous Smooth Curves

The Multi-Factor Interaction Model (MFIM) is used to evaluate the divot weight (foam weight ejected) from launch external tanks. The MFIM has sufficient degrees of freedom to evaluate a large number of factors that may contribute to the divot ejection. It also accommodates all interactions by its product form. Each factor has an exponent that satisfies only two points the initial and final points. They are selected so that the MFIM will generate a common curve to fit the individual point data. The results show that the approach used generates the continuous curve.

Chamis, Christos C.↗
DOE OSTI2017

A VISCOELASTIC MODEL FOR EVALUATING EXTRUSION-BASED PRINT CONDITIONS

Extrusion-based printing systems have improved significantly over the past several years, allowing for higher throughput, higher temperatures, and larger components. At the same time, advanced materials are being introduced on the market that can provide improved performance over a range of operating conditions. Often these materials incorporate fiber reinforcements, reactive resins, and additives to control reaction kinetics, flow rheology, or thermal stability. This study presents a general framework for evaluating the printability of various candidate materials based on a basic viscoelastic model. The model addresses fundamental requirements for extrusion-based printing, including pressure-driven flow, bead formation, bead functionality, and component-level functionality. The effectiveness of this model for evaluating the impact of compositional variations and identifying appropriate processing conditions has been demonstrated for specific materials on direct write, fused filament fabrication, and large-scale extrusion platforms.

Duty, Chad↗
DOE OSTI2015

A Modified Rabbit Ulna Defect Model for Evaluating Periosteal Substitutes in Bone Engineering: A Pilot Study

The present work defines a modified critical size rabbit ulna defect model for bone regeneration in which a non-resorbable barrier membrane was used to separate the radius from the ulna to create a valid model for evaluation of tissue-engineered periosteal substitutes. Eight rabbits divided into two groups were used. Critical defects (15 mm) were made in the ulna completely eliminating periosteum. For group I, defects were filled with a nanohydroxyapatite poly(ester urethane) scaffold soaked in PBS and left as such (group Ia) or wrapped with a tissue-engineered periosteal substitute (group Ib). For group II, an expanded-polytetrafluoroethylene (e-PTFE) (GORE-TEX{sup ®}) membrane was inserted around the radius then the defects received either scaffold alone (group IIa) or scaffold wrapped with periosteal substitute (group IIb). Animals were euthanized after 12–16 weeks, and bone regeneration was evaluated by radiography, computed microtomography (μCT), and histology. In the first group, we observed formation of radio-ulnar synostosis irrespective of the treatment. This was completely eliminated upon placement of the e-PTFE (GORE-TEX{sup ®}) membrane in the second group of animals. In conclusion, modification of the model using a non-resorbable e-PTFE membrane to isolate the ulna from the radius was a valuable addition allowing for objective evaluation of the tissue-engineered periosteal substitute.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2017

Evaluating a Priori Ozone Profile Information Used in TEMPO (Tropospheric Emissions: Monitoring of Pollution) Tropospheric Ozone Retrievals

A primary objective for TOLNet is the evaluation and validation of space-based tropospheric O3 retrievals from future systems such as the Tropospheric Emissions: Monitoring of Pollution (TEMPO) satellite. This study is designed to evaluate the tropopause-based O3 climatology (TB-Clim) dataset which will be used as the a priori profile information in TEMPO O3 retrievals. This study also evaluates model simulated O3 profiles, which could potentially serve as a priori O3 profile information in TEMPO retrievals, from near-real-time (NRT) data assimilation model products (NASA Global Modeling and Assimilation Office (GMAO) Goddard Earth Observing System (GEOS-5) Forward Processing (FP) and Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA2)) and full chemical transport model (CTM), GEOS-Chem, simulations. The TB-Clim dataset and model products are evaluated with surface (0-2 km) and tropospheric (0-10 km) TOLNet observations to demonstrate the accuracy of the suggested a priori dataset and information which could potentially be used in TEMPO O3 algorithms. This study also presents the impact of individual a priori profile sources on the accuracy of theoretical TEMPO O3 retrievals in the troposphere and at the surface. Preliminary results indicate that while the TB-Clim climatological dataset can replicate seasonally-averaged tropospheric O3 profiles observed by TOLNet, model-simulated profiles from a full CTM (GEOS-Chem is used as a proxy for CTM O3 predictions) resulted in more accurate tropospheric and surface-level O3 retrievals from TEMPO when compared to hourly (diurnal cycle evaluation) and daily-averaged (daily variability evaluation) TOLNet observations. Furthermore, it was determined that when large daily-averaged surface O3 mixing ratios are observed (65 ppb), which are important for air quality purposes, TEMPO retrieval values at the surface display higher correlations and less bias when applying CTM a priori profile information compared to all other data products. The primary reason for this is that CTM predictions better capture the spatio-temporal variability of the vertical profiles of observed tropospheric O3 compared to the TB-Clim dataset and other NRT data assimilation models evaluated during this study.

Pollution↗
DOE OSTI2020

APPLICATION OF MATURITY MODELS FOR EVALUATING CYBERSECURITY PROGRAMS AT NUCLEAR AND RADIOLOGICAL FACILITIES

Maturity models can be used to provide government agencies, industry associations, and organizations operating nuclear facilities with the ability to quickly evaluate the maturity of their cybersecurity programs and identify areas to prioritize for improvement. The Cybersecurity Capability Maturity Model (C2M2) was developed by the U.S. Department of Energy to allow organizations in the energy sector to evaluate the programmatic capabilities of their cybersecurity programs in a consistent manner, communicate programmatic maturity information, prioritize cybersecurity investments in targeted areas of concern, and track how the maturity of their cybersecurity program evolves over time. The C2M2 is designed for use by any critical infrastructure organization regardless of ownership, structure, or size. The C2M2 can be easily fine-tuned to access the maturity of nuclear cybersecurity programs and their application at individual facilities. Built on a foundation of existing cybersecurity standards, frameworks, programs, and initiatives, the model features 10 security domains. Performance in each security domain is characterized using a structured set of cybersecurity practices that represent activities an organization can perform to improve cybersecurity in their domain. Each practice can be quickly evaluated as being either fully, largely, partially, or not implemented. Once practices are evaluated for each security domain, the model defines four maturity indicator levels that apply independently to each domain in the model. To earn a maturity level in a given domain, an organization must adequately perform all the practices for that maturity level and its predecessor level(s). A small assessment team can conduct a C2M2 assessment in a single day. A screening version of the C2M2 allows an initial look at the maturity of nuclear cybersecurity programs that could be completed in under an hour.

Cyber security, Nuclear security, maturity model↗
DOE OSTI2020

Evaluating integrated surface/subsurface permafrost thermal hydrology models in ATS (v0.88) against observations from a polygonal tundra site

Numerical simulations are essential tools for understanding the complex hydrologic response of Arctic regions to a warming climate. However, strong coupling among thermal and hydrological processes on the surface and in the subsurface and the significant role that subtle variations in surface topography have in regulating flow direction and surface storage lead to significant uncertainties. Careful model evaluation against field observations is thus important to build confidence.We evaluate the integrated surface/subsurface permafrost thermal hydrology models in the Advanced Terrestrial Simulator (ATS) against field observations from polygonal tundra at the Barrow Environmental Observatory. ATS couples a multiphase, 3D representation of subsurface thermal hydrology with representations of overland nonisothermal flows, snow processes, and surface energy balance. We simulated thermal hydrology of a 3D ice-wedge polygon with geometry that is abstracted but broadly consistent with the surface microtopography at our study site. The simulations were forced by meteorological data and observed water table elevations in ice-wedge polygon troughs. With limited calibration of parameters appearing in the soil evaporation model, the 3-year simulations agreed reasonably well with snow depth, summer water table elevations in the polygon center, and high-frequency soil temperature measurements at several depths in the trough, rim, and center of the polygon. Upscaled evaporation is in good agreement with flux tower observations. The simulations were found to be sensitive to parameters in the bare soil evaporation model, snowpack, and the lateral saturated hydraulic conductivity.Timing of fall freeze-up was found to be sensitive to initial snow density, illustrating the importance of including snow aging effects. The study provides new support for an emerging class of integrated surface/subsurface permafrost simulators.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2013

Evaluation of Model Microphysics within Precipitation Bands of Extratropical Cyclones

Recent studies evaluating the bulk microphysical schemes (BMPs) within cloud resolving models (CRMs) have indicated large uncertainties and errors in the amount and size distributions of snow and cloud ice aloft. The snow prediction is sensitive to the snow densities, habits, and degree of riming within the BMPs. Improving these BMPs is a crucial step toward improving both weather forecasting and climate predictions. Several microphysical schemes in the Weather Research and Forecasting (WRF) model down to 1.33‐km grid spacing are evaluated using aircraft, radar, and ground in situ data from the Global Precipitation Mission Coldseason Precipitation Experiment (GCPEx) experiment, as well as a few years (15 winter storms) of surface measurements of riming, crystal habit, snow density, and radar measurements at Stony Brook, NY (SBNY on north shore of Long Island) during the 2009-2012 winter seasons. Surface microphysical measurements at SBNY were taken every 15 to 30 minutes using a stereo microscope and camera, and snow depth and snow density were also recorded. During these storms, a vertically-pointing Ku-band radar was used to observe the vertical evolution of reflectivity and Doppler vertical velocities. A Particle Size and Velocity (PARSIVEL) disdrometer was also used to measure the surface size distribution and fall speeds of snow at SBNY. For the 15 cases at SBNY, the WSM6, Morrison (MORR), Thompson (THOM2), and Stony Brook (SBU-YLIN) BMPs were validated. A non-spherical snow assumption (THOM2 and SBU-YLIN) simulated a more realistic distribution of reflectivity than spherical snow assumptions in the WSM6 and MORR schemes. The MORR, WSM6, and SBU-YLIN schemes are comparable to the observed velocity distribution in light and moderate riming periods. The THOM2 is approx 0.25 m/s too slow with its velocity distribution in these periods. In heavier riming, the vertical Doppler velocities in the WSM6, THOM2, and MORR schemes were approx 0.25 m/s too slow, while the SBU-YLIN was 0.25 to 0.5 m/s too fast. Overall, the BMPs simulate a size distribution close to the observed for D < 4 mm in the dendritic, plates, and mixed habit periods. The model BMPs underestimate the size distribution when large aggregates were observed. For D > 6 mm in the dendrites, side planes, and mixed habit periods, the BMPs are likely not simulating enough aggregation to create a larger size distribution, although the MORR (double moment) scheme seemed to perform best. These SBNY results will be compared with some results from GCPEx for a warm frontal snow band observed at 18 February 2012.

Colle, Brian A.↗