Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44

Probabilistic system identification in the time domain

The objective of system identification is to determine reliable dynamical models of a structure by systematically using its measured excitation and response. It brings together in an integrated fashion, experimental, analytical, and computational techniques in structural dynamics. Areas of application for system identification include the following: (1) Model Evaluation--assessing assumptions (linearity and equivalent viscous damping) and techniques (finite-element modeling) used to construct theoretical models of a structure; (2) Model Improvement--updating of a theoretical model to enable more accurate response predictions for possible future loads on the structure, or for control of the structure; (3) Empirical Modelling--developing empirical relationships (nonlinear models) or empirical parameter values (modal damping) because the present state of the art does not provide theoretical results; and (4) Damage Detection and Assessment--continual or episodic updating of a structural model through vibration monitoring to detect and locate any structural damage. It can be argued that since the construction or modification of models using test data is subject to inherent uncertainties, the above problems should be properly treated within a Bayesian probabilistic framework. Such a methodology is presented which allows the precision of the estimates of the model parameters to be computed. It also leads to a guiding principle in applications. Namely, when selecting a single model from a given class of models, one should take the most probable model in the class based on the experimental data. Practical applications of this principle are given which are based on the utilization of measured seismic motions in large civil structures. Examples include the application of a computer program MODE-ID to identify modal properties directly from seismic excitation and response time histories from a nine-story steel-frame building at JPL and from a freeway overpass bridge.

Beck, James L.↗

Tropical impacts of SST forcing: A case study for 1987 versus 1988

The response of the NASA/Goddard Institute for Space Studies general circulation model (GCM) to large tropical sea surface temperature (SST) anomalies is investigated by evaluating model simulations of the particularly contrasting summer monsoon seasons 1987 and 1988. These years are representative of the warm and cold phases, respectively, of a recent El Nino-Southern Oscillation (ENSO) event. An ensemble averaging the results of three simulations was considered for each season, using monthly mean observed SST anomalies for June-August 1987 and 1988 as lower boundary forcing. Consistent with the European Center for Medium Weather Forecast (ECMWF)-analyzed winds, the simulations based on 1988 as compared to 1987 SST exhibit stronger upper-tropospheric irrotational circulation between the monsoon regions and the Southern Hemispheric sub-tropical anticyclones, a stronger Pacific Walker cell and a weaker subtropical westerly jet over the South Pacific. In the same vein, the modeled precipitation, indicating a more northerly position of the Pacific Inter-Tropical Convergence Zone (ITCZ) in 1988 compared with 1987, is supported by satellite observations of outgoing longwave radiation and highly reflective clouds.

Druyan, Leonard M.↗

Preliminary Results of a U.S. Deep South Modeling Experiment Using NASA SPoRT Initialization Datasets for Operational National Weather Service Local Model Runs

A joint collaborative modeling effort among the NWS offices in Mobile, AL, and Houston, TX, and NASA Short-term Prediction Research and Transition (SPoRT) Center began during the 2011-2012 cold season, and continued into the 2012 warm season. The focus was on two frequent U.S. Deep South forecast challenges: the initiation of deep convection during the warm season; and heavy precipitation during the cold season. We wanted to examine the impact of certain NASA produced products on the Weather Research and Forecasting Environmental Modeling System in improving the model representation of mesoscale boundaries such as the local sea-, bay- and land-breezes (which often leads to warm season convective initiation); and improving the model representation of slow moving, or quasi-stationary frontal boundaries (which focus cold season storm cell training and heavy precipitation). The NASA products were: the 4-km Land Information System, a 1-km sea surface temperature analysis, and a 4-km greenness vegetation fraction analysis. Similar domains were established over the southeast Texas and Alabama coastlines, each with an outer grid with a 9 km spacing and an inner nest with a 3 km grid spacing. The model was run at each NWS office once per day out to 24 hours from 0600 UTC, using the NCEP Global Forecast System for initial and boundary conditions. Control runs without the NASA products were made at the NASA SPoRT Center. The NCAR Model Evaluation Tools verification package was used to evaluate both the positive and negative impacts of the NASA products on the model forecasts. Select case studies will be presented to highlight the influence of the products.

Wood, Lance↗

Cost Contributors to Geothermal Power Production

The US Department of Energy Geothermal Technologies Office (DOE-GTO) has developed the tool Geothermal Electricity Technologies Evaluation Model (GETEM) to assess the levelized cost of electricity (LCOE) of power produced from geothermal resources. Recently modifications to GETEM allow the DOE-GTO to better assess how different factors impact the generation costs, including initial project risk, time required to complete a development, and development size. The model characterizes the costs associated with project risk by including the costs to evaluate and drill those sites that are considered but not developed for commercial power generation, as well as to assign higher costs to finance those activities having more risk. This paper discusses how the important parameters impact the magnitude project costs for different project scenarios. The cost distributions presented include capital cost recovery for the exploration, confirmation, well field completion and power plant construction, as well as the operation and maintenance (O&M) costs. The paper will present these cost distributions for both EGS and hydrothermal resources.

15 GEOTHERMAL ENERGY↗

RISMC Industry Application no. 1 (ECCS/LOCA) Core characterization automation: Reference PWR designs for IA no. 1

The U.S. NRC is currently proposing rulemaking '10 CFR 50.46c' to revise the loss-of-coolant-accident (LOCA) and emergency core cooling system (ECCS) acceptance criteria to include the effects of higher burn-up on cladding performance. The key implications of this proposition are that both cladding and ECCS performance need to be considered in a coupled manner. This implies that LOCA methodologies will need to be updated for regulatory purposes. The Risk-Informed Safety Margin Characterization (RISMC) Pathway, as part of the DOE Light Water Reactor Sustainability (LWRS) research and development program initialed several programs at the Idaho National Laboratory (INL) to support the industry by offering potential solutions to this new LOCA/ECCS analysis. One of these programs is the Industry Application no. 1 (IA1). It's goal is to develop an integrated evaluation model called LOCA Toolkit for the US (LOTUS) to revisit how risks are evaluated in a LOCA under 10 CFR 50.46c. The focus of LOTUS is to establish the automation interfaces among the physics disciplines involved, namely the core design automation, the fuels/clad performance, the system analysis, the risk assessment and the core design optimization. The risk-informed margins management approach for IA1 will provide a means of quantifying the impact on the key LOCA analysis figures-of-merit like peak cladding temperature (PCT), equivalent cladding reacted (ECR), etc. of a change in LOCA analysis inputs. The information obtained can then be used for decision-making and margin management. The modular approach for LOTUS will enable plant owners/vendors to further customize the LOTUS framework for use within their established codes and methods. One of the LOTUS modules deals with the core design automation (CD-A). The purpose of this module is to supply the LOCA analysis with initial conditions, i.e. assembly/pin power histories, power shapes, etc. to be employed by the Fuels/Clad Performance (EP) and System Analysis (SA) modules. In the CD-A module, the plant owner/operator will characterize his core design with the tools he has. For the current LOTUS demonstration, the tools used are the already coupled codes PHISICS and RELAP5-3D2,3. Cross section generation will be done using the Studsvik lattice code HELIOS-2.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Land surface Verification Toolkit (LVT)

LVT is a framework developed to provide an automated, consolidated environment for systematic land surface model evaluation Includes support for a range of in-situ, remote-sensing and other model and reanalysis products. Supports the analysis of outputs from various LIS subsystems, including LIS-DA, LIS-OPT, LIS-UE. Note: The Land Information System Verification Toolkit (LVT) is a NASA software tool designed to enable the evaluation, analysis and comparison of outputs generated by the Land Information System (LIS). The LVT software is released under the terms and conditions of the NASA Open Source Agreement (NOSA) Version 1.1 or later. Land Information System Verification Toolkit (LVT) NOSA.

surface model↗

Summary of Summer Project: Evaluating New Double Perovskite Halide Scintillators for Radiation Detection Applications

Project Outline: • Experimental validation of machine learning based prediction models; • Evaluating double perovskite halides (chlorides, some bromides) – A 2 bb'X 6 :Ce; A = Br + , Cs + ; B = Na + , K + ; B' = RE 3+ ; X = Cl - , Br - ; • Performing single crystal growth and characterization (physical, optical, scintillation properties).

61 RADIATION PROTECTION AND DOSIMETRY↗

Irreversible Electroporation Adjacent to the Rectum: Evaluation of Pathological Effects in a Pig Model

To evaluate the effects of irreversible electroporation (IRE) on the rectum wall after IRE applied adjacent to the rectum. CT-guided IRE adjacent to the rectum wall was performed in 11 pigs; a total of 44 lesions were created. In five pigs, ablations were performed without a water-filled endorectal coil (group A); in six pigs, ablation was performed with the coil to avoid displacement of the rectum wall (group B). The pigs were killed after 7-15 days and the rectums were harvested for pathological evaluation. There was no evidence of perforation on gross postmortem examination. Perirectal muscle lesions were observed in 18 of 20 ablations in group A and in 21 of 24 ablations in group B. Inflammation and fibrosis of the muscularis propria was observed in ten of 18 lesions in group A and in ten of 21 lesions in group B. In group A, findings were limited to the external layer of the muscularis propria except for one lesion; in group B, findings were transmural in all cases. Transmural necrosis with marked suppurative mucosal inflammation was observed in seven of 21 lesions in group B and in no lesion in group A. IRE-ablation adjacent to the rectum may be uneventful if the rectum wall is mobile and able to contract. IRE-ablation of the rectum may be harmful if the rectum wall is fixed adjacent to the IRE-probe.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Fully digital data acquisition system for the neutron time-of-flight spectrometer TOFOR at JET

A prototype of a fully digital data acquisition system based on 1 Gsps 12 bit digitizers for the TOFOR fusion neutron spectrometer at JET is assessed. The prototype system enables the use of geometry-based background discrimination techniques, which are modeled, evaluated, and compared to experimental data. The experimental results are in line with the models and show a significant improvement in signal-to-background ratio in measured time-of-flight spectrum compared to the existing data acquisition system.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

California annual grass phenology and allometry influence ecosystem dynamics and fire regime in a vegetation demography model

Grass-dominated ecosystems cover wide areas of the land surface yet have received far less attention from the Earth System Model (ESM) community. This limits model projections of ecosystem dynamics in response to global change and coupled vegetation–climate dynamics. We used the Functionally Assembled Terrestrial Ecosystem Simulator (FATES), a dynamic vegetation demography model, to determine ecosystem sensitivity to alternate, observed grass allometries and biophysical traits, and evaluated model performance in capturing California C 3 annual grasslands structure and fire regimes. Grass allometry, leaf physiology, plant phenology, and plant mortality all drove the seasonal variation in matter and energy exchange and fire dynamics in California annual grasslands. Allometry influenced grassland structure and function mainly through canopy architecture-mediated space and light competition instead of through carbon partitioning strategy. Regional variation in grassland annual burned area was driven by variation in ecosystem productivity. Our study advances the modeling of grassy ecosystems in ESMs by establishing the importance of grass allometry and plant phenology and mortality in driving C 3 annual grassland seasonal dynamics and fire regime. The calibrated annual grass allometry and biophysical traits presented can be applied in future studies to project climate–vegetation–fire feedbacks in annual grass-dominant ecosystems under global change.

54 ENVIRONMENTAL SCIENCES↗

Spatial Variability of Winds and HRRR–NCEP Model Error Statistics at Three Doppler-Lidar Sites in the Wind-Energy Generation Region of the Columbia River Basin

Annually and seasonally averaged wind profiles from three Doppler lidars were gathered from sites in the Columbia River basin of east-central Oregon and Washington, a major region of wind-energy production, for the Second Wind Forecast Improvement Project (WFIP2) experiment. The profile data are used to quantify the spatial variability of wind flows in this area of complex terrain, to assess the HRRR-NCEP model's ability to capture spatial and temporal variability of wind profiles, and to evaluate model errors. Annually averaged measured wind speed differences over the 70-km extent of the lidar measurements reached 1 m s -1 within the wind-turbine rotor layer, and 2 m s -1 for 200-500 m AGL. Stronger wind speeds in the lowest 500 m occurred at sites higher in elevation, farther from the river, and farther west - closer to the Cascade Mountain barrier. Validating against the lidar data, the HRRR model underestimated strong wind speeds (>12 m s -1 ) and, consequently, their frequency of occurrence, especially at the two lowest-elevation sites, producing annual low biases in rotor-layer wind speed of 0.5 m s -1 . The RMSE between measured and modeled winds at all sites was about 3 m s -1 and did not degrade significantly with forecast lead time. The nature of the model errors was different for different seasons. Furthermore, although the three sites were located in the same basin terrain, the nature of the model errors was different at each site. Thus, if only one of the sites had been instrumented, different conclusions would have been drawn as to the major sources of model error, depending on where the measurements were made.

17 WIND ENERGY↗

CondiDiag1.0: a flexible online diagnostic tool for conditional sampling and budget analysis in the E3SM atmosphere model (EAM)

Abstract. Numerical models used in weather and climate prediction take into account a comprehensive set of atmospheric processes (i.e., phenomena) such as the resolved and unresolved fluid dynamics, radiative transfer, cloud and aerosol life cycles, and mass or energy exchanges with the Earth's surface. In order to identify model deficiencies and improve predictive skills, it is important to obtain process-level understanding of the interactions between different processes. Conditional sampling and budget analysis are powerful tools for process-oriented model evaluation, but they often require tedious ad hoc coding and large amounts of instantaneous model output, resulting in inefficient use of human and computing resources. This paper presents an online diagnostic tool that addresses this challenge by monitoring model variables in a generic manner as they evolve within the time integration cycle. The tool is convenient to use. It allows users to select sampling conditions and specify monitored variables at run time. Both the evolving values of the model variables and their increments caused by different atmospheric processes can be monitored and archived. Online calculation of vertical integrals is also supported. Multiple sampling conditions can be monitored in a single simulation in combination with unconditional sampling. The paper explains in detail the design and implementation of the tool in the Energy Exascale Earth System Model (E3SM) version 1. The usage is demonstrated through three examples: a global budget analysis of dust aerosol mass concentration, a composite analysis of sea salt emission and its dependency on surface wind speed, and a conditionally sampled relative humidity budget. The tool is expected to be easily portable to closely related atmospheric models that use the same or similar data structures and time integration methods.

58 GEOSCIENCES↗

Evaluation of cloud shading effects on the generation and modification of mesoscale circulations

This study presents scale analysis and numerical model evaluations of the impact of cloud shading on (1) the development of sea breeze and thermally induced upslope flows and (2) the generation of mesoscale circulations between cloudy areas adjacent to clear areas. Based on the assumption of modifications of solar and longwave radiation which are typical for some overcast conditions, it was found that the reduction in the first type of circulation is significant and most noticeable in the vertical velocities. In the second case, thermally induced circulations in favorable conditions appear to approach the typical intensity of the sea breeze.

Segal, M.↗

Studies in Software Cost Model Behavior: Do We Really Understand Cost Model Performance?

While there exists extensive literature on software cost estimation techniques, industry practice continues to rely upon standard regression-based algorithms. These software effort models are typically calibrated or tuned to local conditions using local data. This paper cautions that current approaches to model calibration often produce sub-optimal models because of the large variance problem inherent in cost data and by including far more effort multipliers than the data supports. Building optimal models requires that a wider range of models be considered while correctly calibrating these models requires rejection rules that prune variables and records and use multiple criteria for evaluating model performance. The main contribution of this paper is to document a standard method that integrates formal model identification, estimation, and validation. It also documents what we call the large variance problem that is a leading cause of cost model brittleness or instability.

data mining↗

On the Performance of Global Magnetohydrodynamic Models in the Earth's Magnetosphere

We study the performance of four magnetohydrodynamic models (BATS-R-US, GUMICS, LFM, Open GGCM) in the Earths magnetosphere. Using the Community Coordinated Modeling Centers Run-on-Request system, we compare model predictions with magnetic field measurements of the Cluster, Geotail and Wind spacecraft during a multiple substorm event. We also compare model cross polar cap potential results to those obtained from the Super Dual Auroral Radar Network(SuperDARN) and the model magnetopause standoff distances to an empirical magnetopause model. The correlation coefficient (CC) and prediction efficiency (PE) metrics are used to objectively evaluate model performance quantitatively. For all four models, the best performance outside geosynchronous orbit is found on the dayside. Generally, the performance of models decreases steadily down stream from the Earth. On the dayside most CCs are above 0.5 with CCs for Bx and Bz close to 0.9 for three out of four models. In the magnetotail at a distance of about -130 Earth radii from Earth, the prediction efficiency of all models is below that of using an average value for the prediction with the exception of Bz. Bx is most often best predicted and correlated both on the dayside and the nightside close to the Earth whereas in the far tail the CC and PE for Bz are substantially higher than other components in all models. We also find that increasing the resolution or coupling an additional physics module does notautomatically increase the model performance in the magnetosphere.

geomagnetic tail↗

Evaluating Global Aerosol Models and Aerosol and Water Vapor Properties Near Clouds

The 'Evaluating Global Aerosol Models and Aerosol and Water Vapor Properties Near Clouds' project focused extensively on the analysis and utilization of water vapor and aerosol profiles derived from the ARM Raman lidar at the Southern Great Plains ARM site. A wide range of different tasks were performed during this project, all of which improved quality of the data products derived from the lidar or advanced the understanding of atmospheric processes over the site. These activities included: upgrading the Raman lidar to improve its sensitivity; participating in field experiments to validate the lidar aerosol and water vapor retrievals; using the lidar aerosol profiles to evaluate the accuracy of the vertical distribution of aerosols in global aerosol model simulations; examining the correlation between relative humidity and aerosol extinction, and how these change, due to horizontal distance away from cumulus clouds; inferring boundary layer turbulence structure in convective boundary layers from the high-time-resolution lidar water vapor measurements; retrieving cumulus entrainment rates in boundary layer cumulus clouds; and participating in a field experiment that provided data to help validate both the entrainment rate retrievals and the turbulent profiles derived from lidar observations.

54 ENVIRONMENTAL SCIENCES↗

SciDAC's Earth System Grid Center for Enabling Technologies Semi-Annual Progress Report for the Period October 1, 2009 through March 31, 2010

This report summarizes work carried out by the ESG-CET during the period October 1, 2009 through March 31, 2009. It includes discussion of highlights, overall progress, period goals, collaborations, papers, and presentations. To learn more about our project, and to find previous reports, please visit the Earth System Grid Center for Enabling Technologies (ESG-CET) website. This report will be forwarded to the DOE SciDAC program management, the Office of Biological and Environmental Research (OBER) program management, national and international collaborators and stakeholders (e.g., the Community Climate System Model (CCSM), the Intergovernmental Panel on Climate Change (IPCC) 5th Assessment Report (AR5), the Climate Science Computational End Station (CCES), the SciDAC II: A Scalable and Extensible Earth System Model for Climate Change Science, the North American Regional Climate Change Assessment Program (NARCCAP), and other wide-ranging climate model evaluation activities).

99 GENERAL AND MISCELLANEOUS↗

Development of a Human Motor Model for the Evaluation of an Integrated Alerting and Notification Flight Deck System

A human motor model was developed on the basis of performance data that was collected in a flight simulator. The motor model is under consideration as one component of a virtual pilot model for the evaluation of NextGen crew alerting and notification systems in flight decks. This model may be used in a digital Monte Carlo simulation to compare flight deck layout design alternatives. The virtual pilot model is being developed as part of a NASA project to evaluate multiple crews alerting and notification flight deck configurations. Model parameters were derived from empirical distributions of pilot data collected in a flight simulator experiment. The goal of this model is to simulate pilot motor performance in the approach-to-landing task. The unique challenges associated with modeling the complex dynamics of humans interacting with the cockpit environment are discussed, along with the current state and future direction of the model.

Daiker, Ron↗