Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

CoCoMET v1.0: a unified open-source toolkit for atmospheric object tracking and analysis

Advances in performance and analysis capabilities have accelerated the development of object tracking algorithms for atmospheric research. This has resulted in a growing number of studies using Lagrangian tracking techniques to analyze the evolution of atmospheric phenomena and the underlying processes. However, the increasing complexity and variety of tracking algorithms present a steep learning curve for new users and make it difficult for existing users to compare algorithm performance. We introduce CoCoMET (Community Cloud Model Evaluation Toolkit), an open-source toolkit that addresses these issues. CoCoMET simplifies the process of running multiple tracking algorithms simultaneously and analyzing objects in both model and observational datasets by specifying parameters in a single configuration file. It standardizes input data from different sources into a consistent format and unifies the tracking output across algorithms. CoCoMET enhances the functionality of existing tracking methods by calculating additional properties such as cell growth and dissipation rates, perimeter, surface area, convexity, and irregularity. In addition, CoCoMET includes a novel method for identifying mergers and splits in 2D and 3D tracks and supports the integration of Eulerian/stationary datasets external to the tracking data for process studies. Its potential utility is demonstrated through examples of model intercomparison, model evaluation against observations, and comparisons between tracking algorithms. Designed for open-source environments, CoCoMET will continue to expand with future releases, incorporating more input data types and tracking algorithms.

54 ENVIRONMENTAL SCIENCES↗

Evaluation and modeling of autonomous attitude thrust control for the Geostation Operational Environmental Satellite (GOES)-8 orbit determination

The Geostationary Operational Environmental Satellite (GOES)-8 experienced a series of orbital perturbations from autonomous attitude control thrusting before perigee raising maneuvers. These perturbations influenced differential correction orbital state solutions determined by the Goddard Space Flight Center (GSFC) Goddard Trajectory Determination System (GTDS). The maneuvers induced significant variations in the converged state vector for solutions using increasingly longer tracking data spans. These solutions were used for planning perigee maneuvers as well as initial estimates for orbit solutions used to evaluate the effectiveness of the perigee raising maneuvers. This paper discusses models for the incorporation of attitude thrust effects into the orbit determination process. Results from definitive attitude solutions are modeled as impulsive thrusts in orbit determination solutions created for GOES-8 mission support. Due to the attitude orientation of GOES-8, analysis results are presented that attempt to absorb the effects of attitude thrusting by including a solution for the coefficient of reflectivity, C(R). Models to represent the attitude maneuvers are tested against orbit determination solutions generated during real-time support of the GOES-8 mission. The modeling techniques discussed in this investigation offer benefits to the remaining missions in the GOES NEXT series. Similar missions with large autonomous attitude control thrusting, such as the Solar and Heliospheric Observatory (SOHO) spacecraft and the INTELSAT series, may also benefit from these results.

Forcey, W.↗

Assessment of Microreactor Safety Analysis Challenges and Recommendations for Utilization of the Comprehensive Reactor Analysis Bundle

To enable the broad deployment of microreactors in fundamentally new application regimes (i.e., mobile and autonomous operations), their safety must be indisputable in terms of possessing inherent resistance to severe offsite dose consequences. Therefore, mechanistic beyond-design-basis event source term calculations that demonstrate a sufficiently large margin of safety will be required to accommodate these new application regimes, which have no history of commercial regulation. Even for traditional reactor operation configurations, safety analysis expertise and familiarity with accident phenomena and conditions in microreactors—specifically those with heat pipe primary cooling arrangements—are lacking compared with other advanced reactor concepts and small modular reactors. Recently, modeling and simulation tools to account for unique heat pipe design aspects have been developed by Sandia National Laboratories with MELCOR and by the US Department of Energy’s (DOE’s) Office of Nuclear Energy Advanced Modeling and Simulation Program with BlueCRAB. However, further demonstration and assessment of potential knowledge gaps are needed to support these codes’ broad usage by the microreactor community. Through the DOE Microreactor Program, an initial assessment of these two tools and guidance on how an evaluation model could be constructed was performed and is reported herein. Moreover, a proposed approach for demonstrating an evaluation model using these two tools is outlined.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Characterizing the Relative Importance Assigned to Physical Variables by Climate Scientists when Assessing Atmospheric Climate Model Fidelity

Evaluating a climate model’s fidelity (ability to simulate observed climate) is a critical step in establishing confidence in the model’s suitability for future climate projections, and in tuning climate model parameters. Model developers use their judgement in determining which trade-offs between different aspects of model fidelity are acceptable. However, little is known about the degree of consensus in these evaluations, and whether experts use the same criteria when different scientific objectives are defined. Here, we report on results from a broad community survey studying expert assessments of the relative importance of different output variables when evaluating a global atmospheric model’s mean climate. We find that experts adjust their ratings of variable importance in response to the scientific objective, for instance, scientists rate surface wind stress as significantly more important for Southern Ocean climate than for the water cycle in the Asian watershed. There is greater consensus on the importance of certain variables (e.g., shortwave cloud forcing) than others (e.g., aerosol optical depth). We find few differences in expert consensus between respondents with greater or less climate modeling experience, and no statistically significant differences between the responses of climate model developers and users. In conclusion, the concise variable lists and community ratings reported here provide baseline descriptive data on current expert understanding of certain aspects of model evaluation, and can serve as a starting point for further investigation, as well as developing more sophisticated evaluation and scoring criteria with respect to specific scientific objectives.

54 ENVIRONMENTAL SCIENCES↗

Analysing Scales of Precipitation

The Analysing Scales of Precipitation (ASoP) package is being interfaced with the DOE Coordinated Model Evaluation Capabilities (CMEC) project. Users will be able to use the cmec-driver software to easily run ASoP and generate precipitation metrics as part of climate model evaluation workflows. These metrics provide information about the spatial and temporal patterns of precipitation at different timescales.

Ordonez, AnaC.↗

Evaluation of Modeled and Measured Energy Savings in Existing All Electric Public Housing in the Pacific Northwest

This project analyzes the cost effectiveness of energy savings measures installed by a large public housing authority in Salishan, a community in Tacoma Washington. Research focuses on the modeled and measured energy usage of the first six phases of construction, and compares the energy usage of those phases to phase 7. Market-ready energy solutions were also evaluated to improve the efficiency of affordable housing for new and existing (built since 2001) affordable housing in the marine climate of Washington State.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

ENSO Simulated by Intermediate Coupled Models and Evaluated with Observations Over 1970-1996. Part II: Sensitivity to the Eastward Penetration of the Equatorial Westerlies and to the Off-Equatorial O

Introducing new parameterizations of subsurface temperature and atmospheric convection in the Cane and Zebiak's model allows to reproduce oscillations with a period close to 4 years, equatorial wind anomalies that are located close to the dateline and realistic amplitudes of SST, wind and thermocline anomalies in the equatorial Pacific both in warm and cold phases.

westerlies SST equatorial pacific thermocline anom↗

Evaluation of Modeled and Measured Energy Savings in Existing All Electric Public Housing in the Pacific Northwest

This project analyzes the cost effectiveness of energy savings measures installed by a large public housing authority in Salishan, a community in Tacoma Washington. Research focuses on the modeled and measured energy usage of the first six phases of construction, and compares the energy usage of those phases to phase 7. Market-ready energy solutions were also evaluated to improve the efficiency of affordable housing for new and existing (built since 2001) affordable housing in the marine climate of Washington State.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Model-based evaluation of school- and non-school-related measures to control the COVID-19 pandemic

The role of school-based contacts in the epidemiology of SARS-CoV-2 is incompletely understood. We use an age-structured transmission model fitted to age-specific seroprevalence and hospital admission data to assess the effects of school-based measures at different time points during the COVID-19 pandemic in the Netherlands. Our analyses suggest that the impact of measures reducing school-based contacts depends on the remaining opportunities to reduce non-school-based contacts. If opportunities to reduce the effective reproduction number (R e ) with non-school-based measures are exhausted or undesired and R e is still close to 1, the additional benefit of school-based measures may be considerable, particularly among older school children. As two examples, we demonstrate that keeping schools closed after the summer holidays in 2020, in the absence of other measures, would not have prevented the second pandemic wave in autumn 2020 but closing schools in November 2020 could have reduced R e below 1, with unchanged non-school-based contacts.

60 APPLIED LIFE SCIENCES↗

Thermo-Mechanical Modeling and Evaluation of the Cracking Response of Additively Manufactured Monolithic SiC Lattice Structures Subjected to Laser Heating

Increasing operating temperatures of solar receivers is paramount to the efficiency of concentrated solar thermal (CST) and solar power (CSP) systems. Owing to its high temperature stability combined with excellent thermal and optical properties, SiC has been the material of choice for application in high-temperature solar receivers. The state-of-the-art SiC volumetric concentrating solar air receivers such as honeycomb design have been demonstrated in field tests to achieve exit air temperatures approaching 800oC. However, successful application of CST systems for decarbonization requires significant increase in temperature capability of SiC receiver technology. Supported by an award from the Solar Technology Office (SETO), US Department of Energy (DOE), GE Research in collaboration with Heliogen Holdings Inc, is engaged in the development of ultra-High Operating Temperature SiC-matrix Solar Thermal Air Receiver (HOTSSTAR) enabled by additive manufacturing. The program objective is to design and to demonstrate a techno-economically viable SiC air receiver technology to achieve exit air temperature of 1100oC for CST applications. This report summarizes the learnings of a computational study on additively manufactured SiC lattice structures subjected to laser heating. The results from this study aim to help guide the design of a SiC receiver through a better understanding of lattice structure geometric parameters and their implications on the cracking response of the structure. In the study, a heat flux was applied to the surface of a 2”-diameter cylindrical lattice structure to simulate a 4kW CO2 laser. The resulting temperature distribution was applied to a structural model to approximate the stress distribution within the lattice structure and a Weibull analysis was performed to gain insight into the probability of failure and to evaluate the cracking response of the structure. With this approach, the influence of lattice density on temperature, stress, and probability of failure was explored for two different lattice beam spacings. A discussion on modeling assumptions, a comparison with experimental results, and an evaluation of the lattice cracking response is provided.

14 SOLAR ENERGY↗

Comparing Model Representations of Physiological Limits on Transpiration at a Semi-arid Ponderosa Pine Site

Mechanistic representations of biogeochemical processes in ecosystem models are rapidly advancing, requiring advancements in model evaluation approaches. Here we quantify multiple aspects of model functional performance to evaluate improved process representations in ecosystem models. We compare semi-empirical stomatal models with hydraulic constraints against more mechanistic representations of stomatal and hydraulic functioning at a semi-arid pine site using a suite of metrics and analytical tools. We find that models generally perform similarly under unstressed conditions, but performance diverges under atmospheric and soil drought. The more empirical models better capture synergistic information flows between soil water potential and vapor pressure deficit to transpiration, while the more mechanistic models are overly deterministic. Although models can be parameterized to yield similar functional performance, alternate parameterizations could not overcome structural model constraints that underestimate the unique information contained in soil water potential about transpiration. Additionally, both multilayer canopy and big-leaf models were unable to capture the magnitude of canopy temperature divergence from air temperature, and we demonstrate that errors in leaf temperature can propagate to considerable error in simulated transpiration. This study demonstrates the value of merging underutilized observational data streams with emerging analytical tools to characterize ecosystem function and discriminate among model process representations.

54 ENVIRONMENTAL SCIENCES↗

Advancing Models and Evaluation of Cumulus, Climate and Aerosol Interactions

This project was successfully able to meet its’ goals, but faced some serious challenges due to personnel issues. Nonetheless, it was largely successful. The Project Objectives were as follows: 1. Develop a unified representation of stratifom and cumulus cloud microphysics for NCAR/DOE global community models. 2. Examine the effects of aerosols on clouds and their impact on precipitation in stratiform and cumulus clouds. We will also explore the effects of clouds and precipitation on aerosols. 3. Test these new formulations using advanced evaluation techniques and observations and release

54 ENVIRONMENTAL SCIENCES↗

Burst pressure models and validations for thick-walled pipelines containing corrosion defects

Corrosion is one major threat to pipeline integrity. Over the past decades, many corrosion models have been developed for determining the remaining strength of corroded pipelines, including ASME B31.G, Modified B31.G, LPC, PCORRC and their modified models. All these corrosion models are applicable only to large diameter, thin-walled pipelines with a diameter to wall thickness ratio D/t ≥ 20. In practice, many pipelines have a small diameter and thick wall with a D/t ratio < 20, and thus an adequate corrosion model is needed for assessing remaining strength for corroded thick-walled pipelines. This paper briefly reviews the theoretical burst pressure models for defect-free thin and thick-walled pipelines and four representative corrosion assessment models for thin-walled corroded pipelines. On this basis, two modified corrosion models are proposed to thick-walled pipelines in terms of the average shear stress yield theory. To verify the proposed corrosion models, comprehensive validations are performed. Numerical validations include the elastic-plastic finite element analysis to determine burst pressure for pipelines without and with corrosion defects and the model evaluation using a large dataset of available FEA results of burst pressure for machined defects. Experimental validations include a set of burst pressure tests for defect-free thick-walled pipes with different thicknesses and the model evaluation using one large burst dataset for machined defects with flat bottoms and another large dataset for real corrosion defects with curved river bottom profiles. Both numerical and experimental validations show that the proposed corrosion models can more accurately predict the remaining strength for corroded thin and thick-walled pipelines.

Pipeline↗

An adaptive Gaussian process-based method for efficient Bayesian experimental design in groundwater contaminant source identification problems: ADAPTIVE GAUSSIAN PROCESS-BASED INVERSION

Surrogate models are commonly used in Bayesian approaches such as Markov Chain Monte Carlo (MCMC) to avoid repetitive CPU-demanding model evaluations. However, the approximation error of a surrogate may lead to biased estimations of the posterior distribution. This bias can be corrected by constructing a very accurate surrogate or implementing MCMC in a two-stage manner. Since the two-stage MCMC requires extra original model evaluations, the computational cost is still high. If the information of measurement is incorporated, a locally accurate approximation of the original model can be adaptively constructed with low computational cost. Based on this idea, we propose a Gaussian process (GP) surrogate-based Bayesian experimental design and parameter estimation approach for groundwater contaminant source identification problems. A major advantage of the GP surrogate is that it provides a convenient estimation of the approximation error, which can be incorporated in the Bayesian formula to avoid over-confident estimation of the posterior distribution. The proposed approach is tested with a numerical case study. Without sacrificing the estimation accuracy, the new approach achieves about 200 times of speed-up compared to our previous work using two-stage MCMC.

Zhang, Jiangjiang↗

Cloud Influence on ERA5 and AMPS Surface Downwelling Longwave Radiation Biases in West Antarctica

The surface downwelling longwave radiation component (LW?) is crucial for the determination of the surface energy budget and has significant implications for the resilience of ice surfaces in the polar regions. Accurate model evaluation of this radiation component requires knowledge about the phase, vertical distribution, and associated temperature of water in the atmosphere, all of which control the LW? signal measured at the surface. In this study, we examine the LW? model errors found in the Antarctic Mesoscale Prediction System (AMPS) operational forecast model and the ERA5 reanalysis model relative to observations from the AWARE campaign at McMurdo Station and the West Antarctic Ice Sheet (WAIS) Divide. The errors are calculated separately for observed clear sky conditions, ice cloud occurrences, and liquid-bearing cloud layer (LBCL) occurrences. The analysis results show a tendency (in both models at each site) to underestimate the LW? during clear sky conditions, high error variability (standard deviations > 20 W/m2) during any type of cloud occurrence, and negative LW? biases when LBCLs are observed (bias magnitudes > 15 W/m2 in tenuous LBCL cases; > 43 W/m2 in optically thick/opaque LBCLs instances). We suggest that a generally dry and liquid water-deficient atmosphere responsible for the identified LW? biases in both models is the result of excessive ice formation and growth, which could stem from model initial and lateral boundary conditions, microphysics scheme, aerosol representation, and/or limited vertical resolution.

Silber, Israel↗

An Enhanced Model for Evaluating Failure Propagation in Launch Vehicle Engine Sections

An important component in the assessment of launch vehicle safety is the estimation of the likelihood of large-scale explosions given the existence of failures that manifest as a localized energy release. A failure propagation model is presented for simulating cascading failures of energetic components in proximity where the primary modes of energy transfer are fragments/shrapnel and blast overpressure. The model is an extension of the model developed by Mathias and Motiwala (2015) [1] with enhancements to include the effects of fragment ricochet, fragment drag, fragment ballistic limit equation thresholds, and blast impulse thresholds. A series of sensitivity studies were conducted for a generic launch vehicle engine section and results are presented that illustrate the effects of the model modifications.

Susie Go↗

An Enhanced Model for Evaluating Failure Propagation in Launch Vehicle Engine Sections

An important component in the assessment of launch vehicle safety is the estimation of the likelihood of large-scale explosions given the existence of failures that manifest as a localized energy release. A failure propagation model is presented for simulating cascading failures of energetic components in proximity where the primary modes of energy transfer are fragments/shrapnel and blast overpressure. The model is an extension of the model developed by Mathias and Motiwala (2015) [1] with enhancements to include the effects of fragment ricochet, fragment drag, fragment ballistic limit equation thresholds, and blast impulse thresholds. A series of sensitivity studies were conducted for a generic launch vehicle engine section and results are presented that illustrate the effects of the model modifications.

Susie Go↗