Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40

Tropical and Subtropical Cloud Transitions in Weather and Climate Prediction Models: The GCSS/WGNE Pacific Cross-Section Intercomparison (GPCI)

A model evaluation approach is proposed in which weather and climate prediction models are analyzed along a Pacific Ocean cross section, from the stratocumulus regions off the coast of California, across the shallow convection dominated trade winds, to the deep convection regions of the ITCZ the Global Energy and Water Cycle Experiment Cloud System Study/Working Group on Numerical Experimentation (GCSS/ WGNE) Pacific Cross-Section Intercomparison (GPCI). The main goal of GPCI is to evaluate and help understand and improve the representation of tropical and subtropical cloud processes in weather and climate prediction models. In this paper, a detailed analysis of cloud regime transitions along the cross section from the subtropics to the tropics for the season June July August of 1998 is presented. This GPCI study confirms many of the typical weather and climate prediction model problems in the representation of clouds: underestimation of clouds in the stratocumulus regime by most models with the corresponding consequences in terms of shortwave radiation biases; overestimation of clouds by the 40-yr ECMWF Re-Analysis (ERA-40) in the deep tropics (in particular) with the corresponding impact in the outgoing longwave radiation; large spread between the different models in terms of cloud cover, liquid water path and shortwave radiation; significant differences between the models in terms of vertical cross sections of cloud properties (in particular), vertical velocity, and relative humidity. An alternative analysis of cloud cover mean statistics is proposed where sharp gradients in cloud cover along the GPCI transect are taken into account. This analysis shows that the negative cloud bias of some models and ERA-40 in the stratocumulus regions [as compared to the first International Satellite Cloud Climatology Project (ISCCP)] is associated not only with lower values of cloud cover in these regimes, but also with a stratocumulus-to-cumulus transition that occurs too early along the trade wind Lagrangian trajectory. Histograms of cloud cover along the cross section differ significantly between models. Some models exhibit a quasi-bimodal structure with cloud cover being either very large (close to 100%) or very small, while other models show a more continuous transition. The ISCCP observations suggest that reality is in-between these two extreme examples. These different patterns reflect the diverse nature of the cloud, boundary layer, and convection parameterizations in the participating weather and climate prediction models.

Teixeira, J.↗

Developing an Empirical Model for Jet-Surface Interaction Noise

The process of developing an empirical model for jet-surface interaction noise is described and the resulting model evaluated. Jet-surface interaction noise is generated when the high-speed engine exhaust from modern tightly integrated or conventional high-bypass ratio engine aircraft strikes or flows over the airframe surfaces. An empirical model based on an existing experimental database is developed for use in preliminary design system level studies where computation speed and range of configurations is valued over absolute accuracy to select the most promising (or eliminate the worst) possible designs. The model developed assumes that the jet-surface interaction noise spectra can be separated from the jet mixing noise and described as a parabolic function with three coefficients: peak amplitude, spectral width, and peak frequency. These coefficients are fit to functions of surface length and distance from the jet lipline to form a characteristic spectra which is then adjusted for changes in jet velocity and/or observer angle using scaling laws from published theoretical and experimental work. The resulting model is then evaluated for its ability to reproduce the characteristic spectra and then for reproducing spectra measured at other jet velocities and observer angles; successes and limitations are discussed considering the complexity of the jet-surface interaction noise versus the desire for a model that is simple to implement and quick to execute.

Brown, Clifford A.↗

Developing an Empirical Model for Jet-Surface Interaction Noise

The process of developing an empirical model for jet-surface interaction noise is described and the resulting model evaluated. Jet-surface interaction noise is generated when the high-speed engine exhaust from modern tightly integrated or conventional high-bypass ratio engine aircraft strikes or flows over the airframe surfaces. An empirical model based on an existing experimental database is developed for use in preliminary design system level studies where computation speed and range of configurations is valued over absolute accuracy to select the most promising (or eliminate the worst) possible designs. The model developed assumes that the jet-surface interaction noise spectra can be separated from the jet mixing noise and described as a parabolic function with three coefficients: peak amplitude, spectral width, and peak frequency. These coefficients are t to functions of surface length and distance from the jet lipline to form a characteristic spectra which is then adjusted for changes in jet velocity and/or observer angle using scaling laws from published theoretical and experimental work. The resulting model is then evaluated for its ability to reproduce the characteristic spectra and then for reproducing spectra measured at other jet velocities and observer angles; successes and limitations are discussed considering the complexity of the jet-surface interaction noise versus the desire for a model that is simple to implement and quick to execute.

Brown, Clif↗

Assessment and Simulation of Particulate Transport for Delivery of Solid Amendments into the Subsurface: FY25 Status Report

For particulate-based amendments to be viable for field-scale remediation at the Hanford Site (e.g., the 200 DV-1 Operable Unit), amendment particles need to be delivered a reasonable distance away from an injection well to provide cost effective in situ treatment. Field-scale particle transport models can estimate spatial deposition of amendment particles in the subsurface, which is critical for developing an overall remediation strategy. However, field-scale particle simulations are currently limited due to insufficient simulation capability and a lack of experimental data to validate and parameterize particle transport models. During this fiscal year, the following progress has been made toward a field-scale particle transport modeling evaluation: (1) in addition to the two particle transport models implemented last FY, four additional particle transport models have been implemented within PFLOTRAN; (2) a Python-based pre-screening tool was finalized, enabling users to quickly estimate the particle radius of influence (ROI) for any given particle-amendment system; and (3) an initial compatibility assessment was completed using both particle transport simulations deployed through the pre-screening tool, in conjunction with general guidelines to (a) identify the most suitable amendment particle sizes for various Hanford sediments and (b) evaluate amendment-delivery fluid compatibility. The preliminary compatibility assessment revealed that for amendment delivery success to the various Hanford target formations, amendment particle sizes will likely need to be smaller than the amendment sizes tested in the DV-1 treatability study. It is recommended that amendment particles be decreased in size, or alternative smaller size amendments be obtained from the manufacture, prior to any further experimental testing. Also, preliminary testing suggests that xanthan gum may be the most broadly compatible delivery fluid. Planned laboratory experiments will be instrumental in validating and refining the PFLOTRAN particulate transport model formulations, ultimately enabling predictive capabilities to facilitate the design of field-scale amendment delivery systems. This work consists of acquiring new theoretical or experimental knowledge. The information associated with this report should not be used as design input or operating parameters without additional qualification.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Crash Risks Evaluation of Urban Expressways: A Case Study in Shanghai

We report that proactive traffic safety management systems can reduce crashes by identifying crash precursors, evaluating real-time crash risks, and implementing suitable interventions. The basic prerequisite for developing such a system is to propose a reliable crash risk evaluation model that takes real-time traffic flow data as input. Previous studies have primarily focused on real-time crash prediction using some statistical or machine-learning methods. However, further quantitative evaluation and classification of crash risks have been ignored. In this study, we conduct a systematic crash risk evaluation workflow, including crash risk prediction, crash risk quantification, and crash risk classification. Specifically, the crash risk prediction using an extended logit model is proposed, from which CAS, CSD, UAS, DAS, DTV are identified to be contributing factors of crash risks. Then a crash risk quantification model based on the parameter evaluation of the extended logit model is developed. The crash risks of urban expressways and their spatial-temporal evolution trends are quantified. Finally, the crash risks are classified into high crash risk level, moderate crash risk level, and low crash risk level by the k-means cluster algorithm. Then the threshold boundaries of different crash risk levels are determined. The research results provide a proactive guidance for traffic safety management of urban expressways.

33 ADVANCED PROPULSION SYSTEMS↗

The coastal streamflow flux in the Regional Arctic System Model

The coastal streamflow flux from the Arctic drainage basin is an important driver of dynamics in the coupled ice-ocean system. Comprising more than one third of the total freshwater flux into the Arctic Ocean, streamflow is a key component of the regional and global freshwater cycle. To better represent the coupling of the streamflow flux to the ocean, we have developed and applied the RVIC streamflow routing model within the Regional Arctic System Model (RASM). The RASM is a high-resolution regional Earth System Model whose domain includes all of the Arctic drainage basin. Here in this paper, we introduce the RVIC streamflow routing model, detailing its application within RASM and its advancements in terms of representing high-resolution streamflow processes. We evaluate model simulated streamflow relative to in situ observations and demonstrate a method for improving model performance using a simple optimization procedure. We also present a new, spatially and temporally consistent, high-resolution data set of coastal freshwater fluxes for the Arctic drainage basin and surrounding areas that is based on a fully coupled RASM simulation and intended for use in Arctic Ocean modeling applications. This data set is evaluated relative to other coastal streamflow data sets commonly used by the ocean modeling community. We demonstrate that the RASM-simulated streamflow flux better represents the annual cycle than existing data sets, especially in ungauged areas. Finally, we assess the impact that streamflow has on the coupled ice-ocean system, finding that the presence of streamflow leads to reduced sea surface salinity, increased sea surface temperatures, and decreased sea ice thickness.

54 ENVIRONMENTAL SCIENCES↗

Development of a Reference Model for Molten-Salt-Cooled Pebble-Bed Reactor Using SAM

To support the development and utilization of the SAM code for fluoride-salt-cooled high-temperature pebble-bed reactor (PB-FHR) safety analysis and licensing, an effort was devoted to developing reference models for PB-FHR primary loop and the reactor cavity cooling system (RCCS). A reference standard problem of a prototypical reactor design is foundational to NRC and industry to verify the adequacy of computer codes and evaluation models for a specific reactor type. The SAM code was enhanced for the reference PB-FHR model development, including a 1D-3D flow coupling scheme and conjugate heat transfer between porous media and solid structures. The reference FHR primary loop model was developed based on the UC Berkeley Mk1 FHR design with additional design features from the Kairos Power’s KP-FHR core. The PB-FHR core is modeled by a two-dimensional porous medium while the rest of the primary loop is represented by a SAM one-dimensional model. A SAM one-dimensional primary loop model was also developed for comparison using a single channel approach for the reactor core. Furthermore, the water-based NSTF cavity was modeled using SAM multi-dimensional flow module, to assist in evaluating SAM capabilities of modeling the emergency heat removal systems relying on RCCS concepts.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Multi-Task Learning Software Suite

Software suite with multitask learning models. The pipeline includes functionality to process data, train, evaluate models, and generate reports.

Goncalves, Andre↗

Bayesian Multitask with Structure Learning

Software including a Bayesian multitask learning model. The pipeline includes functionality to process data, train, evaluate models, and generate reports.

Soper, BradenC.↗

The Monolithic Heat Pipe Microreactor Reference Plant Model

This work introduces a reference plant model for a generic monolithic heat-pipe-cooled microreactor. The model will serve as a springboard to develop future evaluation models in the licensing process of similar microreactor designs at the U.S. Nuclear Regulatory Commission. This model has been developed with the Comprehensive Reactor Analysis Bundle and its specifications are based on open literature publications for the eVinci TM design. BlueCRAB is the U.S. Nu- clear Regulatory Commission non-light-water reactor analysis system based on the Multiphysics Object-Oriented Simulation Environment framework, which can couple the Griffin, BISON, and Sockeye applications to resolve the various physics that are essential for the safety analysis of this type of reactor system. The core specifications includes tristructural isotropic fuel, graphite monolith, graphite reflectors, and drums composed of graphite and B 4 C. No moderator or burnable poison pins are used in the design. The fuel enrichment is reduced to control excess reactivity in the core. This core design is not optimized and only serves for testing purposes, since the primary objective of this work is to exercise the multiphysics coupling for this type of reactor system. A three dimensional (3D) core heterogeneous Griffin discrete ordinates (SN) transport model allows the precise calculation of the flux distribution and pin powers. Griffin transfers the power density distribution and obtains a temperature distribution to and from BISON. The BISON model com- putes the 3D core temperature distribution and is coupled to 876 Sockeye subapplications running a heat pipe model. This 3D conduction model is coupled to the various heat pipes via heat flux boundary conditions. The model includes a small gap between the heat pipe and the monolith. Convective heat transfer boundaries with either ambient temperature or condenser temperature as heat sinks are imposed at the model boundaries. The 2D Sockeye heat pipe model uses a vapor- only methodology, which provides the needed resolution for transient calculations and allows the determination of various heat pipe limits. This approach is superior to the superconductor model traditionally used in steady-state calculations. BlueCRAB computes steady-state power and temperature distributions that serve as the initial condition for a loss-of-heat-sink transient simulation. The steady-state results show significant peaking due to the position of the control drum, but this is a characteristic of the particular design used, which is not optimized at this stage. The transient results show the reactor power slowly stabilizing towards a 3% power level after the partial loss of secondary heat removal. Several recriticalities are observed due to cooling through the secondary system but the reactor is self-stabilizing and behaves as expected.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Compiled records of carbon isotopes in atmospheric CO 2 for historical simulations in CMIP6

Abstract. The isotopic composition of carbon (Δ 14 C and δ 13 C) in atmospheric CO 2 and in oceanic and terrestrial carbon reservoirs is influenced by anthropogenic emissions and by natural carbon exchanges, which can respond to and drive changes in climate. Simulations of 14 C and 13 C in the ocean and terrestrial components of Earth system models (ESMs) present opportunities for model evaluation and for investigation of carbon cycling, including anthropogenic CO 2 emissions and uptake. The use of carbon isotopes in novel evaluation of the ESMs' component ocean and terrestrial biosphere models and in new analyses of historical changes may improve predictions of future changes in the carbon cycle and climate system. We compile existing data to produce records of Δ 14 C and δ 13 C in atmospheric CO 2 for the historical period 1850–2015. The primary motivation for this compilation is to provide the atmospheric boundary condition for historical simulations in the Coupled Model Intercomparison Project 6 (CMIP6) for models simulating carbon isotopes in the ocean or terrestrial biosphere. The data may also be useful for other carbon cycle modelling activities.

54 ENVIRONMENTAL SCIENCES↗

Topological data analysis and machine learning for recognizing atmospheric river patterns in large climate datasets

Identifying weather patterns that frequently lead to extreme weather events is a crucial first step in understanding how they may vary under different climate change scenarios. Here, we propose an automated method for recognizing atmospheric rivers (ARs) in climate data using topological data analysis and machine learning. The method provides useful information about topological features (shape characteristics) and statistics of ARs. We illustrate this method by applying it to outputs of version 5.1 of the Community Atmosphere Model version 5.1 (CAM5.1) and the reanalysis product of the second Modern-Era Retrospective Analysis for Research and Applications (MERRA-2). An advantage of the proposed method is that it is threshold-free – there is no need to determine any threshold criteria for the detection method – when the spatial resolution of the climate model changes. Hence, this method may be useful in evaluating model biases in calculating AR statistics. Further, the method can be applied to different climate scenarios without tuning since it does not rely on threshold conditions. We show that the method is suitable for rapidly analyzing large amounts of climate model and reanalysis output data.

54 ENVIRONMENTAL SCIENCES↗

Flow interference in a variable porosity trisonic wind tunnel.

Pressure data from a 20-degree cone-cylinder in a variable porosity wind tunnel for the Mach range 0.2 to 5.0 are compared to an interference free standard in order to determine wall interference effects. Four 20-degree cone-cylinder models representing an approximate range of percent blockage from one to six were compared to curve-fits of the interference free standard at each Mach number and errors determined at each pressure tap location. The average of the absolute values of the percent error over the length of the model was determined and used as the criterion for evaluating model blockage interference effects. The results are presented in the form of the percent error as a function of model blockage and Mach number.

Davis, J. W.↗

Management systems research study

The development of a Monte Carlo simulation of procurement activities at the NASA Ames Research Center is described. Data cover: simulation of the procurement cycle, construction of a performance evaluation model, examination of employee development, procedures and review of evaluation criteria for divisional and individual performance evaluation. Determination of the influences and apparent impact of contract type and structure and development of a management control system for planning and controlling manpower requirements.

Bruno, A. V.↗

Techniques for the estimation of leaf area index using spectral data

Based on the radiative transport theory of a homogeneous canopy, a new approach for obtaining transformations of spectral data used to estimate leaf area index (LAI), is developed. The transformations which are obtained without any ground knowledge of LAI show low sensitivity to soil variability, and are linearly related to LAI with relationships which are predictable from leaf reflectance, transmittance properties, and canopy reflectance models. Evaluation of the SAIL (scattering by arbitrarily inclined leaves) model is considered. Using only nadir view data, results obtained on winter and spring wheat and corn crops are presented.

Badhwar, G. D.↗

Real-time automated failure analysis for on-orbit operations

A system which is to provide real-time failure analysis support to controllers at the NASA Johnson Space Center Control Center Complex (CCC) for both Space Station and Space Shuttle on-orbit operations is described. The system employs monitored systems' models of failure behavior and model evaluation algorithms which are domain-independent. These failure models are viewed as a stepping stone to more robust algorithms operating over models of intended function. The described system is designed to meet two sets of requirements. It must provide a useful failure analysis capability enhancement to the mission controller. It must satisfy CCC operational environment constraints such as cost, computer resource requirements, verification, and validation. The underlying technology and how it may be used to support operations is also discussed.

Kirby, Sarah↗

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗