Search NASA⌕ Search

SEARCH · Search NASA

Results for “model bias”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

DESI mock challenge: Halo and galaxy catalogues with the bias assignment method

We present a novel approach to the construction of mock galaxy catalogues for large-scale structure analysis based on the distribution of dark matter halos obtained with effective bias models at the field level. We aim to produce mock galaxy catalogues capable of generating accurate covariance matrices for a number of cosmological probes that are expected to be measured in current and forthcoming galaxy redshift surveys (e.g. two- and three-point statistics). The construction of the catalogues shown in this paper is part of a mock-comparison project within the Dark Energy Spectroscopic Instrument (DESI) collaboration. We use the bias assignment method ( BAM ) to model the statistics of halo distribution through a learning algorithm using a few detailed N-body simulations, and approximated gravity solvers based on Lagrangian perturbation theory. We introduce cosmic-web-dependent corrections to modelling redshift-space distortions at the N-body level – both in the halo and galaxy distributions –, as well as a multi-scale approach for accurate assignment of halo properties. Using specific models of halo occupation distributions to populate halos, we generate galaxy mocks with the expected number density and central-satellite fraction of emission-line galaxies, which are a key target of the DESI experiment. BAM generates mock catalogues with per cent accuracy in a number of summary statistics, such as the abundance, the two- and three-point statistics of halo distributions, both in real and redshift space. In particular, the mock galaxy catalogues display ~3%-10% accuracy in the multipoles of the power spectrum up to scales of k ~ 0.4 h -1 Mpc. We show that covariance matrices of two- and three-point statistics obtained with BAM display a similar structure to the reference simulation. BAM offers an efficient way to produce mock halo catalogues with accurate two- and three-point statistics and is able to generate a variety of multi-tracer catalogues with precise covariance matrices of several cosmological probes. We discuss future developments of the algorithm towards mock production in DESI and other galaxy-redshift surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Dark energy survey year 3 results: High-precision measurement and modeling of galaxy-galaxy lensing

We present and characterize the galaxy-galaxy lensing signal measured using the first three years of data from the Dark Energy Survey (DES Y3) covering 4132 deg 2 . These galaxy-galaxy measurements are used in the DES Y3 3 × 2 pt cosmological analysis, which combines weak lensing and galaxy clustering information. We use two lens samples: a magnitude-limited sample and the redmagic sample, which span the redshift range ∼ 0.2 – 1 with 10.7 and 2.6 M galaxies, respectively. For the source catalog, we use the metacalibration shape sample, consisting of ≃ 100 M galaxies separated into four tomographic bins. Our galaxy-galaxy lensing estimator is the mean tangential shear, for which we obtain a total SNR of ∼ 148 for maglim ( ∼ 120 for redmagic), and ∼ 67 ( ∼ 55 ) after applying the scale cuts of 6 Mpc / h . Thus we reach percent-level statistical precision, which requires that our modeling and systematic-error control be of comparable accuracy. The tangential shear model used in the 3 × 2 pt cosmological analysis includes lens magnification, a five-parameter intrinsic alignment model, marginalization over a point mass to remove information from small scales and a linear galaxy bias model validated with higher-order terms. We explore the impact of these choices on the tangential shear observable and study the significance of effects not included in our model, such as reduced shear, source magnification, and source clustering. We also test the robustness of our measurements to various observational and systematics effects, such as the impact of observing conditions, lens-source clustering, random-point subtraction, scale-dependent metacalibration responses, point spread function residuals, and B modes.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

RADA

The code for regularized adversarial domain adaption is provided here. The RADA methodology is developed for correcting climate model biases using an adversarial learning strategy. The code here covers the data processing, the algorithm, the baseline models, and the evaluation.

Anderson, GemmaJ.↗
DOE OSTI2023

Impact of Global Warming on U.S. Summertime Mesoscale Convective Systems: A Simple Lagrangian Parcel Model Perspective

Abstract Mesoscale convective systems (MCSs) are the dominant rainfall producer over the United States during the warm season, causing natural disasters and severe weather every year. Global climate models have large uncertainty in projecting precipitation changes in the future climate. Here, a simple Lagrangian parcel model is used to investigate the impact of global warming on MCS initiation and growth. The single-column parcel model projects a mean precipitation decrease over the central United States and an increase to its east, in agreement with the CMIP5 model projection. It also highlights the crucial role of current climate mean-state model bias in exaggerating the change in future mean precipitation projection by 25%. As for convective population, the model captures the decreased occurrence frequency of weak to moderate convection and increased frequency of strong convection due to the increased CAPE and CIN, in agreement with convection-permitting regional simulations. Novel parameterizations of gust-front propagation speed and subsidence strength are developed as guided by cloud-resolving simulations. The multicolumn parcel model employing those parameterizations captures readily the cold pool–induced upscale growth feature. It simulates smaller mesoscale clusters over the central United States under global warming due to gust-front slowdown and subsidence strength enhancement, which are further attributed to land aridity–induced weakening of initial accumulated precipitation and strengthening of updraft speed, respectively. That said, mesoscale clusters could become bigger under more favorable conditions in future climate, including boundary layer moistening, convection lifetime lengthening, and cold pool mechanical-lifting enhancement, which require further investigations to improve mechanistic understanding of future MCS changes.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2002

Stratospheric Analysis and Forecasting in the Northern Winter of 1999/2000: The NASA DAO's GEOS-3 System

An evaluation is presented of the performance in the northern winter 1999/2000 of the GEOS-3 troposphere-stratosphere data assimilation system (DAS). The impacts of the two main input data types are assessed: upper-air soundings (sondes) provide wind and temperature information and satellite-based (Tiros Operational Vertical Sounders: TOVS) give estimates of the thermal structure. It is shown that in the low stratosphere (300-70hPa) the analyses are generally slightly warmer than the sonde data, but colder than the TOVS data; this relationship reverses between 70 and 10 hPa. There are geographical biases, related to the spatial and temporal coverage of the observation types and to the statistical weights assigned to them in the DAS. Forecasts show a tendency to reduce zonal asymmetries in the atmospheric flow and to suppress stratospheric temperature minima. In the DAS, the analysis increments compensate for this, but it leads to important biases in the multi-day forecasts. The analysis increments are as large as the diabatic forcing in the lower polar stratosphere, indicating a substantial model bias. The results provide important insights into the roles of different data types and the circulation model in producing accurate analyses for studies of polar chemistry and physical processes.

Pawson, Steven↗
DOE OSTI2021

Evaluating Climate Models with the CLIVAR 2020 ENSO Metrics Package

El Niño–Southern Oscillation (ENSO) is the dominant mode of interannual climate variability on the planet, with far-reaching global impacts. It is therefore key to evaluate ENSO simulations in state-of-the-art numerical models used to study past, present, and future climate. Recently, the Pacific Region Panel of the International Climate and Ocean: Variability, Predictability and Change (CLIVAR) Project, as a part of the World Climate Research Programme (WCRP), led a community-wide effort to evaluate the simulation of ENSO variability, teleconnections, and processes in climate models. The new CLIVAR 2020 ENSO metrics package enables model diagnosis, comparison, and evaluation to 1) highlight aspects that need improvement; 2) monitor progress across model generations; 3) help in selecting models that are well suited for particular analyses; 4) reveal links between various model biases, illuminating the impacts of those biases on ENSO and its sensitivity to climate change; and to 5) advance ENSO literacy. By interfacing with existing model evaluation tools, the ENSO metrics package enables rapid analysis of multipetabyte databases of simulations, such as those generated by the Coupled Model Intercomparison Project phases 5 (CMIP5) and 6 (CMIP6). The CMIP6 models are found to significantly outperform those from CMIP5 for 8 out of 24 ENSO-relevant metrics, with most CMIP6 models showing improved tropical Pacific seasonality and ENSO teleconnections. Only one ENSO metric is significantly degraded in CMIP6, namely, the coupling between the ocean surface and subsurface temperature anomalies, while the majority of metrics remain unchanged.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2011

Lessons Learned from Assimilating Altimeter Data into a Coupled General Circulation Model with the GMAO Augmented Ensemble Kalman Filter

Satellite altimetry measurements have provided global, evenly distributed observations of the ocean surface since 1993. However, the difficulties introduced by the presence of model biases and the requirement that data assimilation systems extrapolate the sea surface height (SSH) information to the subsurface in order to estimate the temperature, salinity and currents make it difficult to optimally exploit these measurements. This talk investigates the potential of the altimetry data assimilation once the biases are accounted for with an ad hoc bias estimation scheme. Either steady-state or state-dependent multivariate background-error covariances from an ensemble of model integrations are used to address the problem of extrapolating the information to the sub-surface. The GMAO ocean data assimilation system applied to an ensemble of coupled model instances using the GEOS-5 AGCM coupled to MOM4 is used in the investigation. To model the background error covariances, the system relies on a hybrid ensemble approach in which a small number of dynamically evolved model trajectories is augmented on the one hand with past instances of the state vector along each trajectory and, on the other, with a steady state ensemble of error estimates from a time series of short-term model forecasts. A state-dependent adaptive error-covariance localization and inflation algorithm controls how the SSH information is extrapolated to the sub-surface. A two-step predictor corrector approach is used to assimilate future information. Independent (not-assimilated) temperature and salinity observations from Argo floats are used to validate the assimilation. A two-step projection method in which the system first calculates a SSH increment and then projects this increment vertically onto the temperature, salt and current fields is found to be most effective in reconstructing the sub-surface information. The performance of the system in reconstructing the sub-surface fields is particularly impressive for temperature, but not as satisfactory for salt.

Keppenne, Christian↗
NASA NTRS2012

Application of the CALIOP Layer Product to Evaluate the Vertical Distribution of Aerosols Estimated by Global Models: AeroCom Phase I Results

The CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) layer product is used for a multimodel evaluation of the vertical distribution of aerosols. Annual and seasonal aerosol extinction profiles are analyzed over 13 sub-continental regions representative of industrial, dust, and biomass burning pollution, from CALIOP 2007-2009 observations and from AeroCom (Aerosol Comparisons between Observations and Models) 2000 simulations. An extinction mean height diagnostic (Z-alpha) is defined to quantitatively assess the models' performance. It is calculated over the 0-6 km and 0-10 km altitude ranges by weighting the altitude of each 100 m altitude layer by its aerosol extinction coefficient. The mean extinction profiles derived from CALIOP layer products provide consistent regional and seasonal specificities and a low inter-annual variability. While the outputs from most models are significantly correlated with the observed Z-alpha climatologies, some do better than others, and 2 of the 12 models perform particularly well in all seasons. Over industrial and maritime regions, most models show higher Z-alpha than observed by CALIOP, whereas over the African and Chinese dust source regions, Z-alpha is underestimated during Northern Hemisphere Spring and Summer. The positive model bias in Z-alpha is mainly due to an overestimate of the extinction above 6 km. Potential CALIOP and model limitations, and methodological factors that might contribute to the differences are discussed.

Aerosols and particles↗
DOE OSTI2021

The Fall and Rise of the Global Climate Model

Abstract Global models are an essential tool for climate projections, but conventional coarse‐resolution atmospheric general circulation models suffer from errors both in their parameterized cloud physics and in their representation of climatically important circulation features. A notable recent study by Terai et al. (2020, https://doi.org/10.1029/2020ms002274) documents a global model capable of reproducing the regime‐based effect of aerosols on cloud liquid water path expected from observational evidence. This may represent a significant advance in cloud process fidelity in global models. Such models can be expected to give a better estimate of the effective radiative forcing of the climate. If this advance in cloud process representation can be matched by advances in the representation of circulation features such as monsoons, then such models may also be able to navigate the complex tangle between spatially heterogeneous aerosol–cloud interactions and regional circulation patterns. This tight link between aerosol and circulation results in anthropogenic perturbations of climate variables of societal importance, such as regional rainfall distributions. Upcoming global models with km‐scale resolution may improve the regional circulation and be able to take advantage of the Terai et al. (2020, https://doi.org/10.1029/2020ms002274) improvement in cloud physics. If so, an era of significantly improved regional climate projection capabilities may soon dawn. If not, then the improvement in cloud physics might spur intensified efforts on problems in model dynamics. Either way, based on the rapid changes in aerosol emissions in the near future, learning to make reliable projections based on biased models is a skill that will not go out of style.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

A semiempirical transparency model for dual energy cargo radiography applications

Cargo containers passing through ports are scanned by non-intrusive inspection systems to search for concealed illicit materials. By using two photon beams with different energy spectra, dual energy inspection systems are sensitive to both the area density and the atomic number of cargo contents. Most literature on the subject assumes a simple exponential attenuation model for photon intensity in which only free streaming photons are detected. However, this approximation neglects second order effects such as scattering, leading to a biased model and thus incorrect material predictions. This work studies the accuracy of the free streaming model by comparing it to simulation outputs, finding that the model shows poor atomic number reconstruction accuracy at high- Z and suffers significantly if the source energy spectra and detector response function are not known exactly. To address these challenges, this work introduces a semiempirical transparency model which modifies the free streaming model by rescaling different components of the mass attenuation coefficient, allowing the model to capture secondary effects ignored by the free streaming model. The semiempirical model displays improvement agreement with simulated results at high- Z and shows excellent extrapolation to materials and thicknesses which were not included during the calibration step. Furthermore, this work demonstrates that the semiempirical model yields accurate atomic number predictions even when the source spectra and detector response are not known exactly. In conclusion, using the semiempirical model, manufacturers can perform a simple calibration to enable more precise Z reconstruction capabilities, which has the potential to significantly improve the performance of existing dual energy radiographic systems.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Moving beyond post hoc explainable artificial intelligence: a perspective paper on lessons learned from dynamical climate modeling

AI models are criticized as being black boxes, potentially subjecting climate science to greater uncertainty. Explainable artificial intelligence (XAI) has been proposed to probe AI models and increase trust. In this review and perspective paper, we suggest that, in addition to using XAI methods, AI researchers in climate science can learn from past successes in the development of physics-based dynamical climate models. Dynamical models are complex but have gained trust because their successes and failures can sometimes be attributed to specific components or sub-models, such as when model bias is explained by pointing to a particular parameterization. We propose three types of understanding as a basis to evaluate trust in dynamical and AI models alike: (1) instrumental understanding, which is obtained when a model has passed a functional test; (2) statistical understanding, obtained when researchers can make sense of the modeling results using statistical techniques to identify input–output relationships; and (3) component-level understanding, which refers to modelers' ability to point to specific model components or parts in the model architecture as the culprit for erratic model behaviors or as the crucial reason why the model functions well. We demonstrate how component-level understanding has been sought and achieved via climate model intercomparison projects over the past several decades. Such component-level understanding routinely leads to model improvements and may also serve as a template for thinking about AI-driven climate science. Currently, XAI methods can help explain the behaviors of AI models by focusing on the mapping between input and output, thereby increasing the statistical understanding of AI models. Yet, to further increase our understanding of AI models, we will have to build AI models that have interpretable components amenable to component-level understanding. We give recent examples from the AI climate science literature to highlight some recent, albeit limited, successes in achieving component-level understanding and thereby explaining model behavior. The merit of such interpretable AI models is that they serve as a stronger basis for trust in climate modeling and, by extension, downstream uses of climate model data.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2019

Comprehensive Assessment of Models and Events Using Library Tools (CAMEL) Framework: Time Series Comparisons

The Comprehensive Assessment of Models and Events using Library Tools (CAMEL) framework leverages existing Community Coordinated Modeling Center services: Run on Request post processing tools that generate model time series outputs and the new Community Coordinated Modeling Center Metadata Registry that describes simulation runs using Space Physics Archive Search and Extract metadata. The new CAMEL visualization tool compares the modeled time series with observational data and computes a suite of skill scores such as Prediction Efficiency, Root Mean Square Error, and Symmetric Signed Percentage Bias. Model data pairs used for skill calculations are obtained considering a user selected maximum difference between the time of observation and the nearest model output. The system renders available data for all locations and time periods selected using interactive visualizations that allow the user to zoom, pan, and pick data values along traces. Skill scores are reported for each selected event or aggregated over all events for all participating model runs. Separately, scores are reported for all locations (satellites or stations) and for each location individually. We are building on past experiences with model data comparisons of magnetosphere and ionosphere model outputs from GEM2008, GEMCEDAR Electrodynamics Thermosphere Ionosphere, and the SWPC Operational Space Weather Model challenges. The CAMEL visualization tool is demonstrated using three validation studies: (a) Wang Sheeley Arge heliosphere simulations compared against OMNI solar wind data, (b) ground magnetic perturbations from several magnetosphere and ionosphere electrodynamics models as observed by magnetometers, and (c) electron fluxes from several ring current simulations compared to Radiation Belt Storm Probes Helium Oxygen Proton Electron instrument measurements, integrated over different energy ranges.

Rastätter, Lutz↗
DOE OSTI2021

A spatial evaluation of Arctic sea ice and regional limitations in CMIP6 historical simulations

The Arctic sea ice response to a warming climate is assessed in a subset of models participating in Phase 6 of the Coupled Model Intercomparison Project (CMIP6), using several metrics in comparison with satellite observations and results from the Pan-Arctic Ice Ocean Modeling and Assimilation System and the Regional Arctic System Model. Our study examines the historical representation of sea ice extent, volume, and thickness using spatial analysis metrics, such as the integrated ice-edge error, Brier score, and Spatial Probability Score. We find that the CMIP6 multi-model mean captures the mean annual cycle and 1979-2014 sea ice trends remarkably well. However, individual models experience a wide range of uncertainty in the spatial distribution of sea ice when compared against satellite measurements and reanalysis data. Our metrics expose common and individual regional model biases, which sea ice temporal analyses alone do not capture. We identify large ice edge and ice thickness errors in Arctic sub-regions, implying possible model specific limitations in or lack of representation of some key physical processes. We postulate that many of them could be related to the oceanic forcing, especially in the marginal and shelf seas, where seasonal sea ice changes are not adequately simulated. We therefore conclude that an individual model’s ability to represent the observed/reanalysis spatial distribution still remains a challenge. We propose the spatial analysis metrics as useful tools to diagnose model limitations, narrow down possible processes affecting them, and guide future model improvements critical to the representation and projections of Arctic climate change.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

The Tropical Basin Interaction Model Intercomparison Project (TBIMIP)

Abstract. Large-scale interaction between the three tropical ocean basins is an area of intense research that is often conducted through experimentation with numerical models. A common problem is that modeling groups use different experimental setups, which makes it difficult to compare results and delineate the role of model biases from differences in experimental setups. To address this issue, an experimental protocol for examining interaction between the tropical basins is introduced. The Tropical Basin Interaction Model Intercomparison Project (TBIMIP) consists of experiments in which sea surface temperatures (SSTs) are prescribed to follow observed values in selected basins. There are two types of experiments. One type, called standard pacemaker, consists of simulations in which SSTs are restored to observations in selected basins during a historical simulation. The other type, called pacemaker hindcast, consists of seasonal hindcast simulations in which SSTs are restored to observations during 12-month forecast periods. TBIMIP is coordinated by the Climate and Ocean – Variability, Predictability, and Change (CLIVAR) Research Focus on Tropical Basin Interaction. The datasets from the model simulations will be made available to the community to facilitate and stimulate research on tropical basin interaction and its role in seasonal-to-decadal variability and climate change.

Richter, Ingo↗
DOE OSTI2024

Using Satellite and ARM Observations to Evaluate Cold Air Outbreak Cloud Transitions in E3SM Global Storm‐Resolving Simulations

Abstract This study examines marine boundary layer cloud regime transition during a cold air outbreak (CAO) over the Norwegian Sea, simulated by a global storm‐resolving model (GSRM) known as the Simple Cloud‐Resolving Energy Exascale Earth System Model Atmosphere Model (SCREAM). By selecting observational references based on a combination of large‐scale conditions rather than strict time‐matched comparisons, this study finds that SCREAM qualitatively captures the CAO cloud transition, including boundary layer growth, cloud mesoscale structure, and phase partitioning. SCREAM also accurately locates the greatest ice and liquid in the mesoscale updrafts, however, underestimates supercooled liquid water in cumulus clouds. The model evaluation approach adopted by this study takes advantages of the existing computational‐expensive global simulations of GSRM and the available observations to understand model performance and can be applied to assessments of other cloud regimes in different regions. Such practice provides valuable guidance on the future effort to correct and improve biased model behaviors.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Evaluation of the Rapid Refresh Numerical Weather Prediction Model over Arctic Alaska

Abstract Despite a need for accurate weather forecasts for societal and economic interests in the U.S. Arctic, thorough evaluations of operational numerical weather prediction in the region have been limited. In particular, the Rapid Refresh Model (RAP), which plays a key role in short-term forecasting and decision-making, has seen very limited assessment in northern Alaska, with most evaluation efforts focused on lower latitudes. In the present study, we verify forecasts from version 4 of the RAP against radiosonde, surface meteorological, and radiative flux observations from two Arctic sites on the northern Alaskan coastline, with a focus on boundary layer thermodynamic and dynamic biases, model representation of surface inversions, and cloud characteristics. We find persistent seasonal thermodynamic biases near the surface that vary with wind direction, and may be related to the RAP’s handling of sea ice and ocean interactions. These biases seem to have diminished in the latest version of the RAP (version 5), which includes refined handling of sea ice, among other improvements. In addition, we find that despite capturing boundary layer temperature profiles well overall, the RAP struggles to consistently represent strong, shallow surface inversions. Further, while the RAP seems to forecast the presence of clouds accurately in most cases, there are errors in the simulated characteristics of these clouds, which we hypothesize may be related to the RAP’s treatment of mixed-phase clouds.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗