Search NASA⌕ Search

SEARCH · Search NASA

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Spectroscopic Determination of the Physical Conditions in Hot, Optically Thin Sources

This annual report covers the period from Oct 1, 2000 to Sep 30, 2001. The Astrophysics Plasma Emission Code and Database (APEC/APED), developed in part under this grant, have been upgraded to Version 1.1 and are now beginning to be used outside our research group in applications to X-ray spectral data from Chandra and XMM-Newton. These models represent the best theoretical data currently available and are reasonably complete below about 30 angstroms. Stellar coronae are being used to benchmark the atomic data in APED as part of the Emission Line Project. Initial results suggest that the models for most of the strongest lines are in good agreement with the observations, including the H-like and He-like emission from dominant elements and the Fe L shell emission, both near 1 keV and in the extreme ultraviolet near 100 eV. At this point in time, we define 'good agreement' at the level of accuracy expected from the atomic physics, approximately 20-30%. In order to benchmark the spectral models beyond the canonical theoretical accuracy, we are working closely with the Chandra gratings calibration group to ensure that we are using the optimal calibration, primarily effective areas and line response functions.

Brickhouse, N.↗

Evaluation of the Snow Simulations from the Community Land Model, Version 4 (CLM4)

This paper evaluates the simulation of snow by the Community Land Model, version 4 (CLM4), the land model component of the Community Earth System Model, version 1.0.4 (CESM1.0.4). CLM4 was run in an offline mode forced with the corrected land-only replay of the Modern-Era Retrospective Analysis for Research and Applications (MERRA-Land) and the output was evaluated for the period from January 2001 to January 2011 over the Northern Hemisphere poleward of 30 deg N. Simulated snow-cover fraction (SCF), snow depth, and snow water equivalent (SWE) were compared against a set of observations including the Moderate Resolution Imaging Spectroradiometer (MODIS) SCF, the Interactive Multisensor Snow and Ice Mapping System (IMS) snow cover, the Canadian Meteorological Centre (CMC) daily snow analysis products, snow depth from the National Weather Service Cooperative Observer (COOP) program, and Snowpack Telemetry (SNOTEL) SWE observations. CLM4 SCF was converted into snow-cover extent (SCE) to compare with MODIS SCE. It showed good agreement, with a correlation coefficient of 0.91 and an average bias of -1.54 x 10(exp 2) sq km. Overall, CLM4 agreed well with IMS snow cover, with the percentage of correctly modeled snow-no snow being 94%. CLM4 snow depth and SWE agreed reasonably well with the CMC product, with the average bias (RMSE) of snow depth and SWE being 0.044m (0.19 m) and -0.010m (0.04 m), respectively. CLM4 underestimated SNOTEL SWE and COOP snow depth. This study demonstrates the need to improve the CLM4 snow estimates and constitutes a benchmark against which improvement of the model through data assimilation can be measured.

northern hemisphere↗

Comment on 'Modeling of Convective-Stratiform Precipitation Processes: Sensitivity to Partitioning Methods' by Matthias Steiner

Despite the obvious notion that the presence of hail or graupel is a good indication of convection, the model results show this does not provide an objective benchmark partly due to the unrealistic presence of small amounts of hail or graupel throughout the anvil in the model but mainly because of the significant amounts of hail or graupel, especially in the tropical TOGA COARE simulation, in the transition zone. Without use of a "transition" category, it is open to debate as how this region should best be defined, as stratiform or as convective. So, the presence of significant hail or graupel contents in this zone significantly degrades its use an objective benchmark for convection. The separation algorithm comparison was done in the context of a cloud-resolving model. These models are widely used and serve a variety of purposes especially with regard to retrieving information that cannot be directly measured by providing synthetic data sets that are consistent and complete. Separation algorithms are regularly applied in these models. However, as with any modeling system, these types 'of models are constantly being improved to overcome any known deficiencies and make them more accurate representations of observed systems. The presence of hail and graupel in the anvil and the bias towards heavy rainfall rates are two such examples of areas that need improvement. Since, both of these can effect the perceived performance of the separation algorithms, the Lang et al. (2003) study did not want to overstate the relative performance of any specific algorithms.

Lang, Steve↗

Turbulence Effects on Shaped Booms: Finite Impulse Response Filter Development

Numerical simulations of propagation through turbulent atmospheres can quantify effects on ground waveforms, but such simulations are computationally expensive. To enable quick turnaround analyses as required by NASA’s Quesst Mission, updating the N-wave filtering approach developed by researchers at The Pennsylvania State University to include shaped booms is proposed as an alternative method for estimating turbulence effects on acoustic metrics more quickly. Beginning with a nearfield pressure cylinder modeled after the on-design X-59 configuration, a database of propagation results at 45 turbulence conditions was compiled using nonlinear turbulence propagation modeling code (KZKFourier) and used as input to a process for generating finite impulse response (FIR) filters. Ground waveforms distorted by turbulence were selected to represent mean and mean±standard deviation levels for six metrics, and corresponding FIR filters were generated through a matrix deconvolution process. In order to evaluate how well the FIR filters perform, additional KZKFourier verification cases were devised with different input conditions, and results used as a benchmark. Convolution of shaped boom waveforms modeled using nonturbulent propagation simulations with the new FIR filters showed better agreement on average with KZKFourier statistical results than the N-wave-based FIR filters.

turbulence↗

SatNet: A Benchmark for Satellite Scheduling Optimization

Satellites provide essential services such as networking and weather tracking, and the number of near-earth and deep space satellites are expected to grow rapidly in the coming years. Communications with terrestrial ground stations is one of the critical functionalities of any space mission. Satellite scheduling is a problem that has been scientifically investigated since the 1970s. A central aspect of this problem is the need to consider resource contention and satellite visibility constraints as they require line of sight. Due to the combinatorial nature of the problem, prior solutions such as linear programs and evolutionary algorithms require extensive compute capabilities to output a feasible schedule for each scenario. Machine learning based scheduling can provide an alternative solution by training a model with historical data and generating a schedule quickly with model inference. We present SatNet, a benchmark for satellite scheduling optimization based on historical data from the NASA Deep Space Network. We propose formulation of the satellite scheduling problem as a Markov Decision Process and use reinforcement learning (RL) policies to generate schedules. The nature of constraints imposed by SatNet differ from other combinatorial optimization problems such as vehicle routing studied in prior literature. Our initial results indicate that RL is an alternative optimization approach that can generate candidate solutions of comparable quality to existing state-of-the-practice results. However, we also find that RL policies overfit to the training dataset and do not generalize well to new data, thereby necessitating continued research on reusable and generalizable agents.

Wilson, Brian↗

Invasive Species Forecasting System: A Decision Support Tool for the U.S. Geological Survey: FY 2005 Benchmarking Report v.1.6

The National Institute of Invasive Species Science (NIISS), through collaboration with NASA's Goddard Space Flight Center (GSFC), recently began incorporating NASA observations and predictive modeling tools to fulfill its mission. These enhancements, labeled collectively as the Invasive Species Forecasting System (ISFS), are now in place in the NIISS in their initial state (V1.0). The ISFS is the primary decision support tool of the NIISS for the management and control of invasive species on Department of Interior and adjacent lands. The ISFS is the backbone for a unique information services line-of-business for the NIISS, and it provides the means for delivering advanced decision support capabilities to a wide range of management applications. This report describes the operational characteristics of the ISFS, a decision support tool of the United States Geological Survey (USGS). Recent enhancements to the performance of the ISFS, attained through the integration of observations, models, and systems engineering from the NASA are benchmarked; i.e., described quantitatively and evaluated in relation to the performance of the USGS system before incorporation of the NASA enhancements. This report benchmarks Version 1.0 of the ISFS.

Stohlgren, Tom↗

Analyses of Cometary Silicate Crystals: DDA Spectral Modeling of Forsterite

Comets are the Solar System's deep freezers of gases, ices, and particulates that were present in the outer protoplanetary disk. Where comet nuclei accreted was so cold that CO ice (approximately 50K) and other supervolatile ices like ethane (C2H2) were preserved. However, comets also accreted high temperature minerals: silicate crystals that either condensed (greater than or equal to 1400 K) or that were annealed from amorphous (glassy) silicates (greater than 850-1000 K). By their rarity in the interstellar medium, cometary crystalline silicates are thought to be grains that formed in the inner disk and were then radially transported out to the cold and ice-rich regimes near Neptune. The questions that comets can potentially address are: How fast, how far, and over what duration were crystals that formed in the inner disk transported out to the comet-forming region(s)? In comets, the mass fractions of silicates that are crystalline, f_cryst, translate to benchmarks for protoplanetary disk radial transport models. The infamous comet Hale-Bopp has crystalline fractions of over 55%. The values for cometary crystalline mass fractions, however, are derived assuming that the mineralogy assessed for the submicron to micron-sized portion of the size distribution represents the compositional makeup of all larger grains in the coma. Models for fitting cometary SEDs make this assumption because models can only fit the observed features with submicron to micron-sized discrete crystals. On the other hand, larger (0.1-100 micrometer radii) porous grains composed of amorphous silicates and amorphous carbon can be easily computed with mixed medium theory wherein vacuum mixed into a spherical particle mimics a porous aggregate. If crystalline silicates are mixed in, the models completely fail to match the observations. Moreover, models for a size distribution of discrete crystalline forsterite grains commonly employs the CDE computational method for ellipsoidal platelets (c:a:b=8.14x8.14xl in shape with geometrical factors of x:y:z=1:1:10, Fabian et al. 2001; Harker et al. 2007). Alternatively, models for forsterite employ statistical methods like the Distribution of Hollow Spheres (Min et al. 2008; Oliveira et al. 2011) or Gaussian Random Spheres (GRS) or RGF (Gielen et al. 200S). Pancakes, hollow spheres, or GRS shapes similar to wheat sheaf crystal habit (e.g., Volten et al. 2001; Veihelmann et al. 2006), however, do not have the sharp edges, flat faces, and vertices seen in images of cometary crystals in interplanetary dust particles (IDPs) or in Stardust samples. Cometary forsterite crystals often have equant or tabular crystal habit (J. Bradley). To simulate cometary crystals, we have computed absorption efficiencies of forsterite using the Discrete Dipole Approximation (DDA) DDSCAT code on NAS supercomputers. We compute thermal models that employ a size distribution of discrete irregularly shaped forsterite crystals (nonspherical shapes with faces and vertices) to explore how crystal shape affects the shape and wavelength positions of the forsterite spectral features and to explore whether cometary crystal shapes support either condensation or annealing scenarios (Lindsay et al. 2012a, b). We find forsterite crystal shapes that best-fit comet Hale-Bopp are tetrahedron, bricks or brick platelets, essentially equant or tabular (Lindsay et al. 2012a,b), commensurate with high temperature condensation experiments (Kobatake et al. 2008). We also have computed porous aggregates with crystal monomers and find that the crystal resonances are amplified. i.e., the crystalline fraction is lower in the aggregate than is derived by fitting a linear mix of spectral features from discrete subcomponents, and the crystal resonances 'appear' to be from larger crystals (Wooden et al. 2012). These results may indicate that the crystalline mass fraction in comets with comae dominated by aggregates may be lower than deduced by popular methods that only emoy ensembles of discrete crystals.

Wooden, Diane↗

Advances and trends in the development of computational models for tires

Status and some recent developments of computational models for tires are summarized. Discussion focuses on a number of aspects of tire modeling and analysis including: tire materials and their characterization; evolution of tire models; characteristics of effective finite element models for analyzing tires; analysis needs for tires; and impact of the advances made in finite element technology, computational algorithms, and new computing systems on tire modeling and analysis. An initial set of benchmark problems has been proposed in concert with the U.S. tire industry. Extensive sets of experimental data will be collected for these problems and used for evaluating and validating different tire models. Also, the new Aircraft Landing Dynamics Facility (ALDF) at NASA Langley Research Center is described.

Noor, A. K.↗

Numerical Investigation of Flow Over Periodic Hill

Hybrid turbulence modeling strategies that blend between large-eddy simulation and Reynolds-averaged Navier-Stokes promise to make optimal use of the available grid resolution. A delayed detached eddy simulation hybrid turbulence modeling strategy, based on the shear-stress transport turbulence model, was implemented into a finite-volume compressible Navier-Stokes code. Then the flow through a two-dimensional periodic hill channel was simulated. This geometry is a widely accepted benchmark test case for validating hybrid turbulence modeling strategies. The computational grid for the simulation was downloaded from a hybrid turbulence model validation website. The present hybrid turbulence model simulation predicts the size of the separated flow region as well as velocity and Reynolds stress profiles in good agreement with large-eddy simulation reference data.

Computational Fluid Dynamics↗

The Sensitivity of WRF Daily Summertime Simulations over West Africa to Alternative Parameterizations. Part 1: African Wave Circulation

The performance of the NCAR Weather Research and Forecasting Model (WRF) as a West African regional-atmospheric model is evaluated. The study tests the sensitivity of WRF-simulated vorticity maxima associated with African easterly waves to 64 combinations of alternative parameterizations in a series of simulations in September. In all, 104 simulations of 12-day duration during 11 consecutive years are examined. The 64 combinations combine WRF parameterizations of cumulus convection, radiation transfer, surface hydrology, and PBL physics. Simulated daily and mean circulation results are validated against NASA's Modern-Era Retrospective Analysis for Research and Applications (MERRA) and NCEP/Department of Energy Global Reanalysis 2. Precipitation is considered in a second part of this two-part paper. A wide range of 700-hPa vorticity validation scores demonstrates the influence of alternative parameterizations. The best WRF performers achieve correlations against reanalysis of 0.40-0.60 and realistic amplitudes of spatiotemporal variability for the 2006 focus year while a parallel-benchmark simulation by the NASA Regional Model-3 (RM3) achieves higher correlations, but less realistic spatiotemporal variability. The largest favorable impact on WRF-vorticity validation is achieved by selecting the Grell-Devenyi cumulus convection scheme, resulting in higher correlations against reanalysis than simulations using the Kain-Fritch convection. Other parameterizations have less-obvious impact, although WRF configurations incorporating one surface model and PBL scheme consistently performed poorly. A comparison of reanalysis circulation against two NASA radiosonde stations confirms that both reanalyses represent observations well enough to validate the WRF results. Validation statistics for optimized WRF configurations simulating the parallel period during 10 additional years are less favorable than for 2006.

variability↗

Stagnation point nonequilibrium radiative heating and the influence of energy exchange models

A nonequilibrium radiative heating prediction method has been used to evaluate several energy exchange models used in nonequilibrium computational fluid dynamics methods. The radiative heating measurements from the FIRE II flight experiment supply an experimental benchmark against which different formulations for these exchange models can be judged. The models which predict the lowest radiative heating are found to give the best agreement with the flight data. Examination of the spectral distribution of radiation indicates that despite close agreement of the total radiation, many of the models examined predict excessive molecular radiation. It is suggested that a study of the nonequilibrium chemical kinetics may lead to a correction for this problem.

Hartung, Lin C.↗

The Evaluation of the Regional Atmospheric Modeling System in the Eastern Range Dispersion Assessment System

The Applied Meteorology Unit (AMU) evaluated the Regional Atmospheric Modeling System (RAMS) contained within the Eastern Range Dispersion Assessment System (ERDAS). ERDAS provides emergency response guidance for Cape Canaveral Air Force Station and Kennedy Space Center operations in the event of an accidental hazardous material release or aborted vehicle launch. The RAMS prognostic data are available to ERDAS for display and are used to initialize the 45th Space Wing/Range Safety dispersion model. Thus, the accuracy of the dispersion predictions is dependent upon the accuracy of RAMS forecasts. The RAMS evaluation consisted of an objective and subjective component for the 1999 and 2000 Florida warm seasons, and the 1999-2000 cool season. In the objective evaluation, the AMU generated model error statistics at surface and upper-level observational sites, compared RAMS errors to a coarser RAMS grid configuration, and benchmarked RAMS against the nationally-used Eta model. In the subjective evaluation, the AMU compared forecast cold fronts, low-level temperature inversions, and precipitation to observations during the 1999-2000 cool season, verified the development of the RAMS forecast east coast sea breeze during both warm seasons, and examined the RAMS daily thunderstorm initiation and precipitation patterns during the 2000 warm season. This report summarizes the objective and subjective verification for all three seasons.

Case, Jonathan↗

Stagnation Point Nonequilibrium Radiative Heating and the Influence of Energy Exchange Models

A nonequilibrium radiative heating prediction method has been used to evaluate several energy exchange models used in nonequilibrium computational fluid dynamics methods. The radiative heating measurements from the FIRE II flight experiment supply an experimental benchmark against which different formulations for these exchange models can be judged. The models which predict the lowest radiative heating are found to give the best agreement with the flight data. Examination of the spectral distribution of radiation indicates that despite close agreement of the total radiation, many of the models examined predict excessive molecular radiation. It is suggested that a study of the nonequilibrium chemical kinetics may lead to a correction for this problem.

Hartung, Lin C.↗

Construction of Polarimetric Radar-Based Reference Rain Maps for the Iowa Flood Studies Campaign

The Global Precipitation Measurement (GPM) Mission Iowa Flood Studies (IFloodS) campaign was conducted in central and northeastern Iowa during the months of April-June, 2013. Specific science objectives for IFloodS included quantification of uncertainties in satellite and ground-based estimates of precipitation, 4-D characterization of precipitation physical processes and associated parameters (e.g., size distributions, water contents, types, structure etc.), assessment of the impact of precipitation estimation uncertainty and physical processes on hydrologic predictive skill, and refinement of field observations and data analysis approaches as they pertain to future GPM integrated hydrologic validation and related field studies. In addition to field campaign archival of raw and processed satellite data (including precipitation products), key ground-based platforms such as the NASA NPOL S-band and D3R Ka/Ku-band dual-polarimetric radars, University of Iowa X-band dual-polarimetric radars, a large network of paired rain gauge platforms, and a large network of 2D Video and Parsivel disdrometers were deployed. In something of a canonical approach, the radar (NPOL in particular), gauge and disdrometer observational assets were deployed to create a consistent high-quality distributed (time and space sampling) radar-based ground "reference" rainfall dataset, with known uncertainties, that could be used for assessing the satellite-based precipitation products at a range of space/time scales. Subsequently, the impact of uncertainties in the satellite products could be evaluated relative to the ground-benchmark in coupled weather, land-surface and distributed hydrologic modeling frameworks as related to flood prediction. Relative to establishing the ground-based "benchmark", numerous avenues were pursued in the making and verification of IFloodS "reference" dual-polarimetric radar-based rain maps, and this study documents the process and results as they pertain specifically to efforts using the NPOL radar dataset. The initial portions of the "process" involved dual-polarimetric quality control procedures which employed standard phase and correlation-based approaches to removal of clutter and non-meteorological echo. Calculation of a scale-adaptive KDP was accomplished using the method of Wang and Chandrasekar (2009; J. Atmos. Oceanic Tech.). A dual-polarimetric blockage algorithm based on Lang et al. (2009; J. Atmos. Oceanic Tech.) was then implemented to correct radar reflectivity and differential reflectivity at low elevation angles. Next, hydrometeor identification algorithms were run to identify liquid and ice hydrometeors. After the quality control and data preparation steps were completed several different dual-polarimetric rain estimation algorithms were employed to estimate rainfall rates using rainfall scans collected approximately every two to three minutes throughout the campaign. These algorithms included a polarimetrically-tuned Z-R algorithm that adjusts for drop oscillations (via Bringi et al., 2004, J. Atmos. Oceanic Tech.), and several different hybrid polarimetric variable approaches, including one that made use of parameters tuned to IFloodS 2D Video Disdrometer measurements. Finally, a hybrid scan algorithm was designed to merge the rain rate estimates from multiple low level elevation angle scans (where blockages could not be appropriately corrected) in order to create individual low-level rain maps. Individual rain maps at each time step were subsequently accumulated over multiple time scales for comparison to gauge network data. The comparison results and overall error character depended strongly on rain event type, polarimetric estimator applied, and range from the radar. We will present the outcome of these comparisons and their impact on constructing composited "reference" rainfall maps at select time and space scales.

Radar↗

Experimental Evaluation of a Planning Language Suitable for Formal Verification

The marriage of model checking and planning faces two seemingly diverging alternatives: the need for a planning language expressive enough to capture the complexity of real-life applications, as opposed to a language simple, yet robust enough to be amenable to exhaustive verification and validation techniques. In an attempt to reconcile these differences, we have designed an abstract plan description language, ANMLite, inspired from the Action Notation Modeling Language (ANML) [17]. We present the basic concepts of the ANMLite language as well as an automatic translator from ANMLite to the model checker SAL (Symbolic Analysis Laboratory) [7]. We discuss various aspects of specifying a plan in terms of constraints and explore the implications of choosing a robust logic behind the specification of constraints, rather than simply propose a new planning language. Additionally, we provide an initial assessment of the efficiency of model checking to search for solutions of planning problems. To this end, we design a basic test benchmark and study the scalability of the generated SAL models in terms of plan complexity.

Butler, Rick W.↗

Reduced Complexity Model Intercomparison Project Phase 2: Synthesizing Earth System Knowledge for Probabilistic Climate Projections

Over the last decades, climate science has evolved rapidly across multiple expert domains. Our best tools to capture state-of-the-art knowledge in an internally self-consistent modelling framework are the increasingly complex fully coupled Earth System Models (ESMs). However, computational limitations and the structural rigidity of ESMs mean that the full range of uncertainties across multiple domains are difficult to capture with ESMs alone. The tools of choice are instead more computationally efficient reduced complexity models (RCMs), which are structurally flexible and can span the response dynamics across a range of domain-specific models and ESM experiments. Here we present Phase 2 of the Reduced Complexity Model Intercomparison Project (RCMIP Phase 2), the first comprehensive intercomparison of RCMs that are probabilistically calibrated with key benchmark ranges from specialized research communities. Unsurprisingly, but crucially, we find that models which have been constrained to reflect the key benchmarks better reflect the key benchmarks. Under the low-emissions SSP1-1.9 scenario, across the RCMs, median peak warming projections range from 1.3 to 1.7°C (relative to 1850-1900, using an observationally-based historical warming estimate of 0.8°C between 1850-1900 and 1995-2014). Further developing methodologies to constrain these projection uncertainties seems paramount given the international community's goal to contain warming to below 1.5°C above pre-industrial in the long-term. Our findings suggest that users of RCMs should carefully evaluate their RCM, specifically its skill against key benchmarks and consider the need to include projections benchmarks either from ESM results or other assessments to reduce divergence in future projections.

Climate↗

Evaluation of Inelastic Constitutive Models for Nonlinear Structural Analysis

The influence of inelastic material models on computed stress-strain states, and therefore predicted lives, was studied for thermomechanically loaded structures. Nonlinear structural analyses were performed on a fatigue specimen which was subjected to thermal cycling in fluidized beds and on a mechanically load cycled benchmark notch specimen. Four incremental plasticity creep models (isotropic, kinematic, combined isotropic-kinematic, combined plus transient creep) were exercised. Of the plasticity models, kinematic hardening gave results most consistent with experimental observations. Life predictions using the computed strain histories at the critical location with a Strainrange Partitioning approach considerably overpredicted the crack initiation life of the thermal fatigue specimen.

Kaufman, A.↗

NASTRAN DMAP Fuzzy Structures Analysis: Summary of Research

The main proposed tasks of Cooperative Agreement NCC1-382 were: (1) developing MSC/NASTRAN DMAP language scripts to implement the Soize fuzzy structures approach for modeling the dynamics of complex structures; (2) benchmarking the results of the new code to those for a cantilevered beam in the literature; and (3) testing and validating the new code by comparing the fuzzy structures results to NASA Langley experimental and conventional finite element results for two model test structures representative of aircraft fuselage sidewall construction: (A) a small aluminum test panel (SLP, single longeron panel) with a single longitudinal stringer attached with bolts; and (B) a 47 by 72 inch flat aluminum fuselage panel (AFP, aluminum fuselage panel) including six longitudinal stringers and four frame stiffeners attached with rivets.

Sparrow, Victor W.↗