Search NASA⌕ Search

SEARCH · Search NASA

Results for “ensemble data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

Observability under recurrent loss of data

An account is given of the concept of extended observability in finite-dimensional linear time-invariant systems under recurrent loss of data, where the state vector has to be reconstructed from an ensemble of sensor data at nonconsecutive samples. An at once necessary and sufficient condition for extended observability that can be expressed via a recursive relation is presented, together with such conditions for this as may be related to the characteristic polynomial of the state transition matrix in a discrete-time setting, or of the system matrix in a continuous-time setting.

Luck, Rogelio↗
NASA NTRS2015

Simultaneous Radar and Satellite Data Storm-Scale Assimilation Using an Ensemble Kalman Filter Approach for 24 May 2011

Assimilating high-resolution radar reflectivity and radial velocity into convection-permitting numerical weather prediction models has proven to be an important tool for improving forecast skill of convection. The use of satellite data for the application is much less well understood, only recently receiving significant attention. Since both radar and satellite data provide independent information, combing these two sources of data in a robust manner potentially represents the future of high-resolution data assimilation. This research combines Geostationary Operational Environmental Satellite 13 (GOES-13) cloud water path (CWP) retrievals with Weather Surveillance Radar-1988 Doppler (WSR-88D) reflectivity and radial velocity to examine the impacts of assimilating each for a severe weather event occurring in Oklahoma on 24 May 2011. Data are assimilated into a 3-km model using an ensemble adjustment Kalman filter approach with 36 members over a 2-h assimilation window between 1800 and 2000 UTC. Forecasts are then generated for 90 min at 5-min intervals starting at 1930 and 2000 UTC. Results show that both satellite and radar data are able to initiate convection, but that assimilating both spins up a storm much faster. Assimilating CWP also performs well at suppressing spurious precipitation and cloud cover in the model as well as capturing the anvil characteristics of developed storms. Radar data are most effective at resolving the 3D characteristics of the core convection. Assimilating both satellite and radar data generally resulted in the best model analysis and most skillful forecast for this event.

Jones, Thomas A.↗
DOE OSTI2016

The Decadal Climate Prediction Project (DCPP) contribution to CMIP6

The Decadal Climate Prediction Project (DCPP) is a coordinated multi-model investigation into decadal climate prediction, predictability, and variability. The DCPP makes use of past experience in simulating and predicting decadal variability and forced climate change gained from the fifth Coupled Model Intercomparison Project (CMIP5) and elsewhere. It builds on recent improvements in models, in the reanalysis of climate data, in methods of initialization and ensemble generation, and in data treatment and analysis to propose an extended comprehensive decadal prediction investigation as a contribution to CMIP6 (Eyring et al., 2016) and to the WCRP Grand Challenge on Near Term Climate Prediction (Kushnir et al., 2016). The DCPP consists of three components. Component A comprises the production and analysis of an extensive archive of retrospective forecasts to be used to assess and understand historical decadal prediction skill, as a basis for improvements in all aspects of end-to-end decadal prediction, and as a basis for forecasting on annual to decadal timescales. Component B undertakes ongoing production, analysis and dissemination of experimental quasi-real-time multi-model forecasts as a basis for potential operational forecast production. Component C involves the organization and coordination of case studies of particular climate shifts and variations, both natural and naturally forced (e.g. the “hiatus”, volcanoes), including the study of the mechanisms that determine these behaviours. Furthermore, groups are invited to participate in as many or as few of the components of the DCPP, each of which are separately prioritized, as are of interest to them. The Decadal Climate Prediction Project addresses a range of scientific issues involving the ability of the climate system to be predicted on annual to decadal timescales, the skill that is currently and potentially available, the mechanisms involved in long timescale variability, and the production of forecasts of benefit to both science and society.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2022

Kamodo’s Model-Agnostic Satellite Flythrough: Lowering the Utilization Barrier for Heliophysics Model Outputs

Heliophysics model outputs are increasingly accessible, but typically are not usable by the majority of the community unless directly collaborating with the relevant model developers. Prohibitive factors include complex file output formats, cryptic metadata, unspecified and often customized coordinate systems, and non-linear coordinate grids. Some pockets of progress exist, giving interfaces to various simulation outputs, but only for a small set of outputs and typically not with open-source, freely available packages. Additionally, the increasing array of tools built upon these sporadic interfaces are typically model-specific. We present Kamodo’s model-agnostic satellite flythrough capabilities as the solution to the utilization barrier for heliophysics model outputs. Developed at the Community Coordinated Modeling Center, these flythrough capabilities are built in Python upon a network of model-agnostic interfaces developed in collaboration with model developers, providing interpolation results the community can trust. Kamodo’s flythrough capabilities present the user with a growing variety of flythrough tools based upon a rapidly expanding library of heliophysics model outputs in several domains, currently including a variety of Ionosphere-Thermosphere-Mesosphere and global magnetosphere model outputs. Each capability is designed to be easily accessible via simplistic model-agnostic syntax, with the entire package freely available in the cloud on Github. Here, we describe the tools developed, include several sample applications for common science questions, demonstrate interoperability with selected packages, and summarize ongoing developments.

Software↗
DOE OSTI2018

Cluster Sampling Filters for Non-Gaussian Data Assimilation

This paper presents a fully non-Gaussian filter for sequential data assimilation. The filter is named the “cluster sampling filter”, and works by directly sampling the posterior distribution following a Markov Chain Monte-Carlo (MCMC) approach, while the prior distribution is approximated using a Gaussian Mixture Model (GMM). Specifically, a clustering step is introduced after the forecast phase of the filter, and the prior density function is estimated by fitting a GMM to the prior ensemble. Using the data likelihood function, the posterior density is then formulated as a mixture density, and is sampled following an MCMC approach. Four versions of the proposed filter, namely C ℓ MCMC , C ℓ HMC , MC- C ℓ HMC , and MC- C ℓ HMC are presented. C ℓ MCMC uses a Gaussian proposal density to sample the posterior, and C ℓ HMC is an extension to the Hamiltonian Monte-Carlo (HMC) sampling filter. MC- C ℓ MCMC and MC- C ℓ HMC are multi-chain versions of the cluster sampling filters C ℓ MCMC and C ℓ HMC respectively. The multi-chain versions are proposed to guarantee that samples are taken from the vicinities of all probability modes of the formulated posterior. The new methodologies are tested using a simple one-dimensional example, and a quasi-geostrophic (QG) model with double-gyre wind forcing and bi-harmonic friction. Here, numerical results demonstrate the usefulness of using GMMs to relax the Gaussian prior assumption especially in the HMC filtering paradigm.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2023

Comment on “Advanced Testing of Low, Medium, and High ECS CMIP6 GCM Simulations Versus ERA5-T2m” by N. Scafetta (2022)

Scafetta (2022, https://doi.org/10.1029/2022gl097716) purports to test Coupled Model Intercomparison Project Phase 6 (CMIP6) climate models through a comparison of temperature changes over three decades. Unfortunately, the paper contains numerous conceptual and statistical errors that undermine all of the conclusions. First, no uncertainty is given for the observational temperature difference, making it impossible to assess compatibility with any model result. Second, the CMIP6 data are the ensemble means for each model, but the metric being tested is sensitive to the internal variability and so the full ensemble for each model must be used. When this is corrected, the conclusion that “all models with ECS > 3.0°C overestimate the observed global surface warming” is not sustained. Third, the statistical test in Section 2 would reject all models even in a perfect model setup given sufficient ensemble members, thus the second conclusion “that spatial t-statistics rejects the data-model agreement” is also not sustainable.

CMIP6↗
DOE OSTI2023

Multifidelity validation of digital surrogates using variable-density turbulent mixing models

In this study, ensembles of experimental data are presented and utilized to compare and validate two models used in the simulation of variable-density (Atwood = 0.22), compressible turbulent mixing. Though models of this kind (Reynolds averaged NavierStokes and large-eddy simulations) have been validated extensively with more canonical flows in previous studies, here the present approach offers novelty in the complexity of the geometry, the ensemble-based validation, and the uniformity of the computational framework on which the models are tested. Moreover, all experimental and computational tasks were completed by the authors which has led to a tightly coupled experimental configuration with its “digital twin.” The experimental divergent-shock-tube facility and its data acquisition methods are described and replicated in simulation space. A 2D Euler model which neglects the turbulent mixing at the interface is optimized to experimental data using a Gaussian process. This model then serves as the basis for both the 2D RANS and 3D LES studies that make comparisons to the mixing-layer data from the experiment. A relatively simple RANS model is shown to produce good agreement with experimental data only at late flow development times. The LES ensembles generally show good agreement with experimental data but display sensitivity to the characterization of initial conditions. Resolution-dependent behavior is also observed for certain higher-order statistics of interest. Overall, the LES model successfully captures the effects of divergent geometry, compressibility, and combined nonlinear instabilities inherent to the problem. The successful prediction of mixing width and its growth rate highlight the existence of three distinct regimes in the development of the instability, each with similarities to previously studied instabilities.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2013

An Ensemble Recentering Kalman Filter with an Application to Argo Temperature Data Assimilation into the NASA GEOS-5 Coupled Model

A two-step ensemble recentering Kalman filter (ERKF) analysis scheme is introduced. The algorithm consists of a recentering step followed by an ensemble Kalman filter (EnKF) analysis step. The recentering step is formulated such as to adjust the prior distribution of an ensemble of model states so that the deviations of individual samples from the sample mean are unchanged but the original sample mean is shifted to the prior position of the most likely particle, where the likelihood of each particle is measured in terms of closeness to a chosen subset of the observations. The computational cost of the ERKF is essentially the same as that of a same size EnKF. The ERKF is applied to the assimilation of Argo temperature profiles into the OGCM component of an ensemble of NASA GEOS-5 coupled models. Unassimilated Argo salt data are used for validation. A surprisingly small number (16) of model trajectories is sufficient to significantly improve model estimates of salinity over estimates from an ensemble run without assimilation. The two-step algorithm also performs better than the EnKF although its performance is degraded in poorly observed regions.

Particle Filter↗
DOE OSTI2016

A JOINT CHANDRA AND SWIFT VIEW OF THE 2015 X-RAY DUST-SCATTERING ECHO OF V404 CYGNI

We present a combined analysis of the Chandra and Swift observations of the 2015 X-ray echo of V404 Cygni. Using a stacking analysis, we identify eight separate rings in the echo. We reconstruct the soft X-ray light curve of the 2015 June outburst using the high-resolution Chandra images and cross-correlations of the radial intensity profiles, indicating that about 70% of the outburst fluence occurred during the bright flare at the end of the outburst on MJD 57199.8. By deconvolving the intensity profiles with the reconstructed outburst light curve, we show that the rings correspond to eight separate dust concentrations with precise distance determinations. We further show that the column density of the clouds varies significantly across the field of view, with the centroid of most of the clouds shifted toward the Galactic plane, relative to the position of V404 Cyg, invalidating the assumption of uniform cloud column typically made in attempts to constrain dust properties from light echoes. We present a new XSPEC spectral dust-scattering model that calculates the differential dust-scattering cross section for a range of commonly used dust distributions and compositions and use it to jointly fit the entire set of Swift echo data. We find that a standard Mathis–Rumpl–Nordsieck model provides an adequate fit to the ensemble of echo data. The fit is improved by allowing steeper dust distributions, and models with simple silicate and graphite grains are preferred over models with more complex composition.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Penalized ensemble Kalman filters for high dimensional non-linear systems

The ensemble Kalman filter (EnKF) is a data assimilation technique that uses an ensemble of models, updated with data, to track the time evolution of a usually non-linear system. It does so by using an empirical approximation to the well-known Kalman filter. However, its performance can suffer when the ensemble size is smaller than the state space, as is often necessary for computationally burdensome models. This scenario means that the empirical estimate of the state covariance is not full rank and possibly quite noisy. To solve this problem in this high dimensional regime, we propose a computationally fast and easy to implement algorithm called the penalized ensemble Kalman filter (PEnKF). Under certain conditions, it can be theoretically proven that the PEnKF will be accurate (the estimation error will converge to zero) despite having fewer ensemble members than state dimensions. Further, as contrasted to localization methods, the proposed approach learns the covariance structure associated with the dynamical system. These theoretical results are supported with simulations of several non-linear and high dimensional systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Native glycosylated HIV-1 Env in silico ensemble

The computationally modeled datasets generated in this study, namely the natively glycosylated SOSIP Env and the uniform mannose-9 glycosylated SOSIP Env ensembles will be submitted as Mendeley dataset server as required by the journal publishers, for public availability. These include two sets of Protein Data Bank (PDB)ensembles of 1000 structures each. The files are in the standardized PDB format. Each of the constituent ensemble structures are separated by the ‘END’ tag within the files. Snapshots from the beginning and end of each data set are given here. The column descriptions are as follows: Col1 –ATOM tag; Col 2 –atom number; COL 3 –atom name; COL4 –residue name; COL 5 –chain identifier; COL 6 –residue number; COL. 7,8,9 –X,Y,Z coordinates of atom in 3Dspace; COL 10 –Occupancy Factor; COL 11 -Temperature factor; COL 12 –segment identifier; COL 13 –element symbol.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

Improving Wind Predictions in the Marine Atmospheric Boundary Layer through Parameter Estimation in a Single-Column Model

A current barrier to greater deployment of offshore wind turbines is the poor quality of numerical weather prediction model wind and turbulence forecasts over open ocean. The bulk of development for atmospheric boundary layer (ABL) parameterization schemes has focused on land, partly due to a scarcity of observations over ocean. The 100-m FINO1 tower in the North Sea is one of the few sources worldwide of atmospheric profile observations from the sea surface to turbine hub height. These observations are crucial to developing a better understanding and modeling of physical processes in the marine ABL. In this study, we use the WRF single column model (SCM), coupled with an ensemble Kalman filter from the Data Assimilation Research Testbed (DART), to create 100-member ensembles at the FINO1 location. The goal of this study is to determine the extent to which model parameter estimation can improve offshore wind forecasts.

17 WIND ENERGY↗
NASA NTRS1994

Wavelet compression techniques for hyperspectral data

Hyperspectral sensors are electro-optic sensors which typically operate in visible and near infrared bands. Their characteristic property is the ability to resolve a relatively large number (i.e., tens to hundreds) of contiguous spectral bands to produce a detailed profile of the electromagnetic spectrum. In contrast, multispectral sensors measure relatively few non-contiguous spectral bands. Like multispectral sensors, hyperspectral sensors are often also imaging sensors, measuring spectra over an array of spatial resolution cells. The data produced may thus be viewed as a three dimensional array of samples in which two dimensions correspond to spatial position and the third to wavelength. Because they multiply the already large storage/transmission bandwidth requirements of conventional digital images, hyperspectral sensors generate formidable torrents of data. Their fine spectral resolution typically results in high redundancy in the spectral dimension, so that hyperspectral data sets are excellent candidates for compression. Although there have been a number of studies of compression algorithms for multispectral data, we are not aware of any published results for hyperspectral data. Three algorithms for hyperspectral data compression are compared. They were selected as representatives of three major approaches for extending conventional lossy image compression techniques to hyperspectral data. The simplest approach treats the data as an ensemble of images and compresses each image independently, ignoring the correlation between spectral bands. The second approach transforms the data to decorrelate the spectral bands, and then compresses the transformed data as a set of independent images. The third approach directly generalizes two-dimensional transform coding by applying a three-dimensional transform as part of the usual transform-quantize-entropy code procedure. The algorithms studied all use the discrete wavelet transform. In the first two cases, a wavelet transform coder was used for the two-dimensional compression. The third case used a three dimensional extension of this same algorithm.

Evans, Bruce↗
DOE OSTI2025

Boosting efficiency and reducing graph reliance: Basis adaptation integration in Bayesian multi-fidelity networks

The computational cost of high-fidelity numerical models makes outer-loop analysis, which requires repeated interrogation of the model such as uncertainty quantification, computationally demanding. Multi-fidelity methods, which construct a surrogate model using data from an ensemble of models of varying cost and accuracy, can substantially reduce the cost of outer-loop analysis. However, these methods can be difficult to apply when the model ensemble does not admit a clear hierarchy a priori and the correlations between models are low. Consequently, in this paper, we present a multi-fidelity method that leverages dimension reduction to enhance the correlation between models, thereby reducing the amount of data needed to train a surrogate from an unordered ensemble of models. Our method utilizes basis adaptation to build low-dimensional polynomial chaos expansions of each model and employs Multi-fidelity Networks to encode the relationships among models. We show that the resulting method exhibit two notable advantages over its counterpart: (1) enhanced accuracy (both reduced bias and variance); and (2) reduced dependency on the graph structure encoding relationships among models. We demonstrate the approach on an analytical test problem and a challenging finite element model for a spent nuclear fuel. Our method produces a surrogate model that is significantly more accurate than either a single-fidelity surrogate or a multi-fidelity surrogate constructed without basis adaptation.

42 ENGINEERING↗
DOE OSTI2023

Using Ultrasound Image Augmentation and Ensemble Predictions to Prevent Machine-Learning Model Overfitting

Deep learning predictive models have the potential to simplify and automate medical imaging diagnostics by lowering the skill threshold for image interpretation. However, this requires predictive models that are generalized to handle subject variability as seen clinically. Here, we highlight methods to improve test accuracy of an image classifier model for shrapnel identification using tissue phantom image sets. Using a previously developed image classifier neural network—termed ShrapML—blind test accuracy was less than 70% and was variable depending on the training/test data setup, as determined by a leave one subject out (LOSO) holdout methodology. Introduction of affine transformations for image augmentation or MixUp methodologies to generate additional training sets improved model performance and overall accuracy improved to 75%. Further improvements were made by aggregating predictions across five LOSO holdouts. This was done by bagging confidences or predictions from all LOSOs or the top-3 LOSO confidence models for each image prediction. Top-3 LOSO confidence bagging performed best, with test accuracy improved to greater than 85% accuracy for two different blind tissue phantoms. This was confirmed by gradient-weighted class activation mapping to highlight that the image classifier was tracking shrapnel in the image sets. Overall, data augmentation and ensemble prediction approaches were suitable for creating more generalized predictive models for ultrasound image analysis, a critical step for real-time diagnostic deployment.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Communicating the Impacts of Projected Climate Change on Heavy Rainfall Using a Weighted Ensemble Approach

Urban flood risks are often determined by the frequency analysis of observed rainfall data, communicated using isohyetal maps showing rainfall totals for a range of durations and recurrence intervals. However, to assess future changes in heavy rainfall, it is necessary to study the future projected rainfall time series. Impacts of climate change are typically assessed using climate projections based on global climate model (GCM) outputs and downscaled to finer temporal and spatial scales. The projected data, however, are not generated in a format that urban planners and engineers can easily use to design for future conditions. This paper presents a method to analyze and express climate data in a format that can be readily used in hydrologic models to assess the effects of future extreme rainfall events. Future conditions’ climate data were analyzed using a weighted ensemble approach, which resulted in projected rainfall frequency estimates and their confidence limits. Two multimodel data sets were selected to illustrate this approach in Cook County, Illinois, which belongs to the Chicago metropolitan area. The first data set included statistical downscaling data based on the Intergovernmental Panel for Climate Change’s (IPCC) Coupled Model Intercomparison Project Phase 3 (CMIP3) data. The weighted ensemble analysis applied to this data set produced results that indicated significant increases in projected heavy rainfall. For example, for CMIP3 Scenario A2, for the late 21st century, the 100-year, 24-h rainfall in the northern parts of the county was 29% larger than the model-generated rainfall for the present time. Finally, for the same time horizon and scenario, the confidence interval based on projected data was 87% wider compared with that of the published source (NOAA Atlas 14), calculated using the past observed data. Also, equal-weight delta-corrected IPCC CMIP5-based dynamical downscaling data were applied to the same region for the mid-21st century, producing increases in heavy rainfall fairly similar to those of CMIP3.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Optical–SZE scaling relations for DES optically selected clusters within the SPT-SZ Survey

We study the Sunyaev-Zel'dovich effect (SZE) signature in South Pole Telescope (SPT) data for an ensemble of 719 optically identified galaxy clusters selected from 124.6 deg$^2$ of the Dark Energy Survey (DES) science verification data, detecting a stacked SZE signal down to richness $\lambda\sim20$. The SZE signature is measured using matched-filtered maps of the 2500 deg$^2$ SPT-SZ survey at the positions of the DES clusters, and the degeneracy between SZE observable and matched-filter size is broken by adopting as priors SZE and optical mass-observable relations that are either calibrated using SPT selected clusters or through the Arnaud et al. (2010, A10) X-ray analysis. We measure the SPT signal to noise $\zeta$-$\lambda$, relation and two integrated Compton-$y$ $Y_\textrm{500}$-$\lambda$ relations for the DES-selected clusters and compare these to model expectations accounting for the SZE-optical center offset distribution. For clusters with $\lambda > 80$, the two SPT calibrated scaling relations are consistent with the measurements, while for the A10-calibrated relation the measured SZE signal is smaller by a factor of $0.61 \pm 0.12$ compared to the prediction. For clusters at $20 < \lambda < 80$, the measured SZE signal is smaller by a factor of $\sim$0.20-0.80 (between 2.3 and 10~$\sigma$ significance) compared to the prediction, with the SPT calibrated scaling relations and larger $\lambda$ clusters showing generally better agreement. We quantify the required corrections to achieve consistency, showing that there is a richness dependent bias that can be explained by some combination of contamination of the observables and biases in the estimated masses. We discuss possible physical effects, as contamination from line-of-sight projections or from point sources, larger offsets in the SZE-optical centering or larger scatter in the $\lambda$-mass relation at lower richnesses.

79 ASTRONOMY AND ASTROPHYSICS↗