Search NASA⌕ Search

SEARCH · Search NASA

Results for “ensemble data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS2007

Evaluating Model Performance of an Ensemble-based Chemical Data Assimilation System During INTEX-B Field Mission

We present a global chemical data assimilation system using a global atmosphere model, the Community Atmosphere Model (CAM3) with simplified chemistry and the Data Assimilation Research Testbed (DART) assimilation package. DART is a community software facility for assimilation studies using the ensemble Kalman filter approach. Here, we apply the assimilation system to constrain global tropospheric carbon monoxide (CO) by assimilating meteorological observations of temperature and horizontal wind velocity and satellite CO retrievals from the Measurement of Pollution in the Troposphere (MOPITT) satellite instrument. We verify the system performance using independent CO observations taken on board the NSFINCAR C-130 and NASA DC-8 aircrafts during the April 2006 part of the Intercontinental Chemical Transport Experiment (INTEX-B). Our evaluations show that MOPITT data assimilation provides significant improvements in terms of capturing the observed CO variability relative to no MOPITT assimilation (i.e. the correlation improves from 0.62 to 0.71, significant at 99% confidence). The assimilation provides evidence of median CO loading of about 150 ppbv at 700 hPa over the NE Pacific during April 2006. This is marginally higher than the modeled CO with no MOPITT assimilation (-140 ppbv). Our ensemble-based estimates of model uncertainty also show model overprediction over the source region (i.e. China) and underprediction over the NE Pacific, suggesting model errors that cannot be readily explained by emissions alone. These results have important implications for improving regional chemical forecasts and for inverse modeling of CO sources and further demonstrate the utility of the assimilation system in comparing non-coincident measurements, e.g. comparing satellite retrievals of CO with in-situ aircraft measurements. The work described above also brought to light several short-comings of the data assimilation approach for CO profiles. Because of the limited vertical resolution of the measurement, the retrievals at different altitudes are correlated which can lead to problems with numerical error and overall efficiency. This has resulted in a manuscript that is about to be submitted to JGR:

Arellano, A. F., Jr.↗
NASA NTRS2010

Aerosol Observability and Predictability: From Research to Operations for Chemical Weather Forecasting. Lagrangian Displacement Ensembles for Aerosol Data Assimilation

A challenge common to many constituent data assimilation applications is the fact that one observes a much smaller fraction of the phase space that one wishes to estimate. For example, remotely sensed estimates of the column average concentrations are available, while one is faced with the problem of estimating 3D concentrations for initializing a prognostic model. This problem is exacerbated in the case of aerosols because the observable Aerosol Optical Depth (AOD) is not only a column integrated quantity, but it also sums over a large number of species (dust, sea-salt, carbonaceous and sulfate aerosols. An aerosol transport model when driven by high-resolution, state-of-the-art analysis of meteorological fields and realistic emissions can produce skillful forecasts even when no aerosol data is assimilated. The main task of aerosol data assimilation is to address the bias arising from inaccurate emissions, and Lagrangian misplacement of plumes induced by errors in the driving meteorological fields. As long as one decouples the meteorological and aerosol assimilation as we do here, the classic baroclinic growth of error is no longer the main order of business. We will describe an aerosol data assimilation scheme in which the analysis update step is conducted in observation space, using an adaptive maximum-likelihood scheme for estimating background errors in AOD space. This scheme includes e explicit sequential bias estimation as in Dee and da Silva. Unlikely existing aerosol data assimilation schemes we do not obtain analysis increments of the 3D concentrations by scaling the background profiles. Instead we explore the Lagrangian characteristics of the problem for generating local displacement ensembles. These high-resolution state-dependent ensembles are then used to parameterize the background errors and generate 3D aerosol increments. The algorithm has computational complexity running at a resolution of 1/4 degree, globally. We will present the result of assimilating AOD retrievals from MODIS (on both Aqua and TERRA satellites) from AERONET for validation. The impact on the GEOS-5 Aerosol Forecasting will be fully documented.

da Silva, Arlindo↗
DOE OSTI2024

Approximate CFTs and random tensor models

Abstract A key issue in both the field of quantum chaos and quantum gravity is an effective description of chaotic conformal field theories (CFTs), that is CFTs that have a quantum ergodic limit. We develop a framework incorporating the constraints of conformal symmetry and locality, allowing the definition of ensembles of ‘CFT data’. These ensembles take on the same role as the ensembles of random Hamiltonians in more conventional quantum ergodic phases of many-body quantum systems. To describe individual members of the ensembles, we introduce the notion of approximate CFT, defined as a collection of ‘CFT data’ satisfying the usual CFT constraints approximately, i.e. up to small deviations. We show that they generically exist by providing concrete examples. Ensembles of approximate CFTs are very natural in holography, as every member of the ensemble is indistinguishable from a true CFT for low-energy probes that only have access to information from semi-classical gravity. To specify these ensembles, we impose successively higher moments of the CFT constraints. Lastly, we propose a theory of pure gravity in AdS 3 as a random matrix/tensor model implementing approximate CFT constraints. This tensor model is the maximum ignorance ensemble compatible with conformal symmetry, crossing invariance, and a primary gap to the black-hole threshold. The resulting theory is a random matrix/tensor model governed by the Virasoro 6j-symbol.

Physics↗
DOE OSTI2018

Improved Reanalysis and Prediction of Atmospheric Fields Over the Southern Ocean Using Campaign‐Based Radiosonde Observations

Abstract This study investigated the impact of radiosonde observations from the Southern Ocean obtained by the Australian R/V Aurora Australis on the ALERA2 experimental ensemble reanalysis data set and ensemble forecast experiment. An observing system experiment (OSE) that included additional ship‐launched radiosonde data captured the atmospheric structure over the Southern Ocean. ALERA2 without additional radiosondes had positive temperature biases exceeding 7 °C in the upper troposphere when low‐pressure cyclonic systems passed over the ship. The spread in the upper level was reduced by 15% in the OSE, which propagated downstream from the ship's position because of the sparse observing network over southern high latitudes. Comparison of two 63‐member ensemble forecast experiments initialized by ALERA2 and the OSE revealed that prediction of midlatitude cyclone tracks was improved by the realistic representation of upper‐level troughs in the OSE forecast. This confirms that additional radiosondes over the Southern Ocean reduce uncertainty and error in midlatitude cyclone forecasts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Real-time ensemble microalgae growth forecasting with data assimilation

Accurate short-range (e.g., 7-day) microalgae growth forecasts will be beneficial for both production and harvesting of microalgae. This study developed an operational microalgae growth forecasting system with ensemble data assimilation (DA). The forecasting system was validated against observed Monoraphidium minutum 26B-AM growth in two outdoor pond cultures located in Mesa, Arizona, U.S. We first examined the relative roles of uncertainty in the meteorological forecast and initial conditions (i.e., algal concentration at the time of forecast) in the microalgae 7-day forecast and found initial conditions dominated the microalgae forecasting skill, suggesting the importance of implementing DA to improve initial condition characterization. To correct the systematic bias in biomass simulations, we developed a particle filter with bias estimation (PFBE) DA method to estimate biases and correct the model forecast. We found the DA forecasting system could improve the 7-day microalgae forecasting skill by about 85% on average compared to model forecasts without DA. These results suggest the potential accuracy of biomass growth forecasts may be sufficient to inform real-time operational decisions, such as harvesting planning, for commercial-scale microalgae production.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2014

Hybrid Data Assimilation without Ensemble Filtering

The Global Modeling and Assimilation Office is preparing to upgrade its three-dimensional variational system to a hybrid approach in which the ensemble is generated using a square-root ensemble Kalman filter (EnKF) and the variational problem is solved using the Grid-point Statistical Interpolation system. As in most EnKF applications, we found it necessary to employ a combination of multiplicative and additive inflations, to compensate for sampling and modeling errors, respectively and, to maintain the small-member ensemble solution close to the variational solution; we also found it necessary to re-center the members of the ensemble about the variational analysis. During tuning of the filter we have found re-centering and additive inflation to play a considerably larger role than expected, particularly in a dual-resolution context when the variational analysis is ran at larger resolution than the ensemble. This led us to consider a hybrid strategy in which the members of the ensemble are generated by simply converting the variational analysis to the resolution of the ensemble and applying additive inflation, thus bypassing the EnKF. Comparisons of this, so-called, filter-free hybrid procedure with an EnKF-based hybrid procedure and a control non-hybrid, traditional, scheme show both hybrid strategies to provide equally significant improvement over the control; more interestingly, the filter-free procedure was found to give qualitatively similar results to the EnKF-based procedure.

Kalman Filter↗
DOE OSTI2023

Data Science Infrastructure SOFTWARE

LANL science workflows generate complex data sets and ensembles of data requiring significant compute and storage resources. The Data Science Infrastructure (DSI) project focuses on data-driven approaches to make data more readily available to LANL projects. DSI workflows leverage metadata stored in data-agnostic databases, supported by an abstraction layer API to simplify searching and accessing data across simulation runs, experimental runs, filesystems and environments. The abstraction layer API allows the user to query a range of data types: raw output, processed data, configuration data, machine learning models, performance data, etc. In addition to the abstraction backend API, the DSI project is developing client-driven query APIs and UIs to support specific user workflows.

Turton, Terece↗
DOE OSTI2026

Challenges and alternatives to empirical orthogonal functions for earth system data

Empirical orthogonal functions (EOFs) applied to gridded Earth system data enables users to diagnose modes of variability with relative ease. Yet, many challenges to interpretation exist such that they must be used with awareness and intention when applied to gridded climate data, especially with large ensembles. Utilizing data from two different Earth system modelling large ensemble frameworks, the Energy Exoscale Earth System Model and the Community Earth System Model, as well as reanalysis data, common EOF pitfalls are summarized and discussed. Challenges include erroneous mode swapping, sign flipping, and the temporal variability of the centers of action. For modes of variability with similar contribution to variance, mode swapping is not uncommon. Sign flipping can occur with almost any mode where the pattern is correct, but the sign is arbitrary. Although the variability of the center of action is not necessarily problematic, it potentially complicates interpretation over multi-century timescales. A wide variety of alternative methods to EOFs exist, but fitness-for-purpose must be evaluated. Additionally, illustrations of alternative methods and examples of proper use are provided. Alternative methods fit into three categories: EOF variants, linear methods, and multilinear methods.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Finite-temperature many-body perturbation theory in the canonical ensemble

Benchmark data are presented for the zeroth- through third-order many-body perturbation corrections to the electronic Helmholtz energy, internal energy, and entropy in the canonical ensemble in a wide range of temperature. They are determined as numerical λ-derivatives of the respective quantities computed by thermal full configuration interaction with a perturbation-scaled Hamiltonian, $\hat{H} = \hat{H}_0 + λ\hat{V}$. Sum-over-states analytical formulas for up to the third-order corrections to these properties are also derived as analytical λ-derivatives. These formulas, which are verified by exact numerical agreement with the benchmark data, are given in terms of the Hirschfelder–Certain degenerate perturbation energies and should be valid for both degenerate and nondegenerate reference states at any temperature down to zero. Further, the results in the canonical ensemble are compared with the same in the grand canonical ensemble.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2023

Estimating Ocean Observation Impacts on Coupled Atmosphere‐Ocean Models Using Ensemble Forecast Sensitivity to Observation (EFSO)

Ensemble Forecast Sensitivity to Observation (EFSO) is a technique that can efficiently identify the beneficial/detrimental impacts of every observation in ensemble-based data assimilation (DA). While EFSO has been successfully employed on atmospheric DA, it has never been applied to ocean or coupled DA due to the lack of a suitable error norm for oceanic variables. This study introduces a new density-based error norm incorporating sea temperature and salinity forecast errors, making EFSO applicable to ocean DA for the first time. We implemented the oceanic EFSO on the CFSv2-LETKF and investigated the impact of ocean observations under a weakly coupled DA framework. By removing the detrimental ocean observations detected by EFSO, the CFSv2 forecasts were significantly improved, showing the validation of impact estimation and the great potential of EFSO to be extended as a data selection criterion.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2008

Application of an Ensemble Smoother to Precipitation Assimilation

Assimilation of precipitation in a global modeling system poses a special challenge in that the observation operators for precipitation processes are highly nonlinear. In the variational approach, substantial development work and model simplifications are required to include precipitation-related physical processes in the tangent linear model and its adjoint. An ensemble based data assimilation algorithm "Maximum Likelihood Ensemble Smoother (MLES)" has been developed to explore the ensemble representation of the precipitation observation operator with nonlinear convection and large-scale moist physics. An ensemble assimilation system based on the NASA GEOS-5 GCM has been constructed to assimilate satellite precipitation data within the MLES framework. The configuration of the smoother takes the time dimension into account for the relationship between state variables and observable rainfall. The full nonlinear forward model ensembles are used to represent components involving the observation operator and its transpose. Several assimilation experiments using satellite precipitation observations have been carried out to investigate the effectiveness of the ensemble representation of the nonlinear observation operator and the data impact of assimilating rain retrievals from the TMI and SSM/I sensors. Preliminary results show that this ensemble assimilation approach is capable of extracting information from nonlinear observations to improve the analysis and forecast if ensemble size is adequate, and a suitable localization scheme is applied. In addition to a dynamically consistent precipitation analysis, the assimilation system produces a statistical estimate of the analysis uncertainty.

Zhang, Sara↗
NASA NTRS1989

Geology of Southern Guinevere Planitia, Venus, based on analyses of Goldstone radar data

The ensemble of 41 backscatter images of Venus acquired by the S Band (12.6 cm) Goldstone radar system covers approx. 35 million km and includes the equatorial portion of Guinevere Planitia, Navka Planitia, Heng-O Chasma, and Tinatin Planitia, and parts of Devana Chasma and Phoebe Regio. The images and associated altimetry data combine relatively high spatial resolution (1 to 10 km) with small incidence angles (less than 10 deg) for regions not covered by either Venera Orbiter or Arecibo radar data. Systematic analyses of the Goldstone data show that: (1) Volcanic plains dominate, including groups of small volcanic constructs, radar bright flows on a NW-SE arm of Phoebe Regio and on Ushas Mons and circular volcano-tectonic depressions; (2) Some of the regions imaged by Goldstone have high radar cross sections, including the flows on Ushas Mons and the NW-SE arm of Phoebe Regio, and several other unnamed hills, ridged terrains, and plains areas; (3) A 1000 km diameter multiringed structure is observed and appears to have a morphology not observed in Venera data (The northern section corresponds to Heng-O Chasma); (4) A 150 km wide, 2 km deep, 1400 km long rift valley with upturned flanks is located on the western flank of Phoebe Regio and extends into Devana Chasma; (5) A number of structures can be discerned in the Goldstone data, mainly trending NW-SE and NE-SW, directions similar to those discerned in Pioneer-Venus topography throughout the equatorial region; and (6) The abundance of circular and impact features is similar to the plains global average defined from Venera and Arecibo data, implying that the terrain imaged by Goldstone has typical crater retention ages, measured in hundreds of millions of years. The rate of resurfacing is less than or equal to 4 km/Ga.

Arvidson, R. E.↗
NASA NTRS1999

Dimensionality Reduction Through Classifier Ensembles

In data mining, one often needs to analyze datasets with a very large number of attributes. Performing machine learning directly on such data sets is often impractical because of extensive run times, excessive complexity of the fitted model (often leading to overfitting), and the well-known "curse of dimensionality." In practice, to avoid such problems, feature selection and/or extraction are often used to reduce data dimensionality prior to the learning step. However, existing feature selection/extraction algorithms either evaluate features by their effectiveness across the entire data set or simply disregard class information altogether (e.g., principal component analysis). Furthermore, feature extraction algorithms such as principal components analysis create new features that are often meaningless to human users. In this article, we present input decimation, a method that provides "feature subsets" that are selected for their ability to discriminate among the classes. These features are subsequently used in ensembles of classifiers, yielding results superior to single classifiers, ensembles that use the full set of features, and ensembles based on principal component analysis on both real and synthetic datasets.

Oza, Nikunj C.↗
DOE OSTI2025

Using feature importance as an exploratory data analysis tool on Earth system models

Abstract. Machine learning (ML) models are commonly used to generate predictions, but these models can also support the discovery of new science. Generating accurate predictions necessitates that a model captures the structure of the underlying data. If the structure is properly extracted, ML could be a useful exploratory and evidential tool. In this paper, we present a case study that demonstrates the use of ML for exploratory data analysis (EDA) in the climate space. We apply the ML explainability method of spatiotemporal zeroed feature importance (stZFI) to understand how climate-variable associations evolve over space and time. Our analyses focus on data from ensembles of Earth system models (ESMs) which provide data on different climate states and conditions. We elect to work with ESM ensembles since they allow us to compare feature importance across alternative scenarios not available with observed data. The ensembles also account for natural variability so that we can distinguish between signal and noise due to natural climate variability when computing feature importance. The use of perturbed initial condition ensembles introduces variability mimicking the natural variability in the atmosphere; thus the signals emerging using feature importance (FI) can be evaluated against the natural variability in the climate system. For our analyses, we consider the 1991 volcanic eruption of Mount Pinatubo, which was a large stratospheric aerosol injection. We explore the climate pathway associated with the eruption from aerosols to radiation to temperature at both the near-surface and stratospheric levels. In addition to applying the method to data generated from two different ESMs, we apply stZFI to reanalysis data to compare the associations identified by stZFI. We show how stZFI tracks the importance of aerosol optical depth over time on forecasting temperatures. This case study illustrates usefulness of an ML tool (stZFI) for EDA on a well-studied climate exemplar.

Ries, Daniel (ORCID:0000000250294647)↗
DOE OSTI2016

An ensemble constrained variational analysis of atmospheric forcing data and its application to evaluate clouds in CAM5: Ensemble 3DCVA and Its Application

Large-scale atmospheric forcing data can greatly impact the simulations of atmospheric process models including Large Eddy Simulations (LES), Cloud Resolving Models (CRMs) and Single-Column Models (SCMs), and impact the development of physical parameterizations in global climate models. This study describes the development of an ensemble variationally constrained objective analysis of atmospheric large-scale forcing data and its application to evaluate the cloud biases in the Community Atmospheric Model (CAM5). Sensitivities of the variational objective analysis to background data, error covariance matrix and constraint variables are described and used to quantify the uncertainties in the large-scale forcing data. Application of the ensemble forcing in the CAM5 SCM during March 2000 intensive operational period (IOP) at the Southern Great Plains (SGP) of the Atmospheric Radiation Measurement (ARM) program shows systematic biases in the model simulations that cannot be explained by the uncertainty of large-scale forcing data, which points to the deficiencies of physical parameterizations. The SCM is shown to overestimate high clouds and underestimate low clouds. These biases are found to also exist in the global simulation of CAM5 when it is compared with satellite data.

58 GEOSCIENCES↗
DOE OSTI2022

Autodifferentiable Ensemble Kalman Filters

Data assimilation is concerned with sequentially estimating a temporally evolving state. This task, which arises in a wide range of scientific and engineering applications, is particularly challenging when the state is high-dimensional and the state-space dynamics are unknown. This paper introduces a machine learning framework for learning dynamical systems in data assimilation. Here, our auto-differentiable ensemble Kalman filters (AD-EnKFs) blend ensemble Kalman filters for state recovery with machine learning tools for learning the dynamics. In doing so, AD-EnKFs leverage the ability of ensemble Kalman filters to scale to high-dimensional states and the power of automatic differentiation to train high-dimensional surrogate models for the dynamics. Numerical results using the Lorenz-96 model show that AD-EnKFs outperform existing methods that use expectation-maximization or particle filters to merge data assimilation and machine learning. In addition, AD-EnKFs are easy to implement and require minimal tuning.

autodifferentiation↗
NASA NTRS2021

Large-Scale High-Resolution Coastal Mangrove Forests Mapping Across West Africa With Machine Learning Ensemble and Satellite Big Data

Coastal mangrove forests provide important ecosystem goods and services, including carbon sequestration, biodiversity conservation, and hazard mitigation. However, they are being destroyed at an alarming rate by human activities. To characterize mangrove forest changes, evaluate their impacts, and support relevant protection and restoration decision making, accurate and up-to-date mangrove extent mapping at large spatial scales is essential. Available large-scale mangrove extent data products use a single machine learning method commonly with 30 m Landsat imagery, and significant inconsistencies remain among these data products. With huge amounts of satellite data involved and the heterogeneity of land surface characteristics across large geographic areas, finding the most suitable method for large-scale high-resolution mangrove mapping is a challenge. The objective of this study is to evaluate the performance of a machine learning ensemble for mangrove forest mapping at 20 m spatial resolution across West Africa using Sentinel-2 (optical) and Sentinel-1 (radar) imagery. The machine learning ensemble integrates three commonly used machine learning methods in land cover and land use mapping, including Random Forest (RF), Gradient Boosting Machine (GBM), and Neural Network (NN). The cloud-based big geospatial data processing platform Google Earth Engine (GEE) was used for pre-processing Sentinel-2 and Sentinel-1 data. Extensive validation has demonstrated that the machine learning ensemble can generate mangrove extent maps at high accuracies for all study regions in West Africa (92%–99% Producer’s Accuracy, 98%–100% User’s Accuracy, 95%–99% Overall Accuracy). This is the first-time that mangrove extent has been mapped at a 20 m spatial resolution across West Africa. The machine learning ensemble has the potential to be applied to other regions of the world and is therefore capable of producing high-resolution mangrove extent maps at global scales periodically.

coastal environment↗