Search NASA⌕ Search

SEARCH · Search NASA

Results for “Conditional distribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Distributionally Robust Decision Making Leveraging Conditional Distributions

Distributionally robust optimization (DRO) is a powerful tool for decision making under uncertainty. It is particularly appealing because of its ability to leverage existing data. However, many practical problems call for decision-making with some auxiliary information, and DRO in the context of conditional distributions is not straightforward. We propose a conditional kernel distributionally robust optimization (CKDRO) method that enables robust decision making under conditional distributions through kernel DRO and the conditional mean operator in the reproducing kernel Hilbert space (RKHS). In particular, we consider problems where there is a correlation between the unknown variable y and an auxiliary observable variable x. Given past data of the two variables and a queried auxiliary variable, CKDRO represents the conditional distribution P(y|x) as the conditional mean operator in the RKHS space and quantifies the ambiguity set in the RKHS as well, which depends on the size of the dataset as well as the query point. To justify the use of RKHS, we demonstrate that the ambiguity set defined in RKHS can be viewed as a ball under a metric that is similar to the Wasserstein metric. The DRO is then dualized and solved via a finite dimensional convex program. The proposed CKDRO approach is applied to a generation scheduling problem and shows that the result of CKDRO is superior to common benchmarks in terms of quality and robustness.

Chen, Yuxiao↗
DOE OSTI2021

Adversarial sampling of unknown and high-dimensional conditional distributions

Many engineering problems require the prediction of realization-to-realization variability or a refined description of modeled quantities. In that case, it is necessary to sample elements from unknown high-dimensional spaces with possibly millions of degrees of freedom. While there exist methods able to sample elements from probability density functions (PDF) with known shapes, several approximations need to be made when the distribution is unknown. In this paper the sampling method, as well as the inference of the underlying distribution, are both handled with a data-driven method known as generative adversarial networks (GAN), which trains two competing neural networks to produce a network that can effectively generate samples from the training set distribution. In practice, it is often necessary to draw samples from conditional distributions. When the conditional variables are continuous, only one (if any) data point corresponding to a particular value of a conditioning variable may be available, which is not sufficient to estimate the conditional distribution. This work handles this problem using an a priori estimation of the conditional moments of a PDF. Herein, two approaches, stochastic estimation, and an external neural network are compared for computing these moments; however, any preferred method can be used. The algorithm is demonstrated in the case of the deconvolution of a filtered turbulent flow field. It is shown that all the versions of the proposed algorithm effectively sample the target conditional distribution with minimal impact on the quality of the samples compared to state-of-the-art methods. Additionally, the procedure can be used as a metric for the diversity of samples generated by a conditional GAN (cGAN) conditioned with continuous variables.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Distributed Conditions for Small-signal Stability of Power Grids and Local Control Design

Operating modern power grids with stability guarantees is markedly important. Typical methods for analyzing and certifying power grid stability are largely centralized relying on the ability of the system operator to gather network-wide information and accurately compute the system's eigenvalues. These methods are oftentimes not privacy-preserving and computationally burdensome. They are therefore, not well-suited to modern power grids where small-signal stability has to be evaluated timely, efficiently and in a privacy-preserving fashion. Herein, we introduce a distributed methodology for certifying small-signal stability of power grids and designing the local controllers. First, we analytically derive distributed conditions for network-wide stability that bus agents can inspect using local information. By leveraging these conditions, we then introduce a distributed control design algorithm (DCDA) that can guide the local control design so that stability of the interconnected system is guaranteed. The agents that adopt the proposed distributed algorithm are responsible for tuning their local controllers, producing their local control commands and ensuring that their local stability condition is met. The system operator is only responsible for verifying network-wide stability upon receiving affirmative responses from all agents and, announcing, that the overall system is stable. The proposed DCDA algorithm is numerically validated via simulations using the IEEE 39-bus system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Conditional distribution estimation of building characteristics with diffusion models for urban energy modeling

Understanding current energy consumption behavior in communities is critical for informing future energy use decisions and enabling efficient energy management. Urban energy models, which are used to simulate these energy use patterns, require large datasets with detailed building characteristics for accurate outcomes. However, such detailed characteristics at the individual building level are often unknown and costly to acquire, or unavailable. Through this work, we propose using a generative modeling approach to generate realistic building attributes to fill in the data gaps and finally provide complete characteristics as inputs to energy models. Our model learns complex, building-level patterns from training on a large-scale residential building stock model containing 2.2 million buildings. We employ a tabular diffusion-based framework that is designed to handle heterogeneous (discrete and continuous) features in tabular building data, such as occupancy, floor area, heating, cooling, and other equipment details. We develop a capability for conditional diffusion, enabling the imputation of missing building characteristics conditioned on known attributes. We conduct a comprehensive validation of our conditional diffusion model, firstly by comparing the generated conditional distributions against the underlying data distribution, and secondly, by performing a case study for a Baltimore residential region, showing the practical utility of our approach. Our work is one of the first to demonstrate the potential of generative modeling to accelerate building energy modeling workflows.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

A multistage distributionally robust optimization approach to water allocation under climate uncertainty

This paper investigates a Multistage Distributionally Robust Optimization (MDRO) approach to water allocation under climate uncertainty. The MDRO is formed by creating sets of conditional distributions (called conditional ambiguity sets) on a finite scenario tree. The distributions in the conditional ambiguity sets remain close to a nominal conditional distribution according a ø-divergence (e.g., Kullback-Leibler divergence, Hellinger distance, Burg entropy, etc.). Here, the paper discusses a decomposition algorithm to solve the resulting MDRO with ø-divergences, which uses the dual formulation and solves only linear subproblems instead of convex ones. Some properties of the algorithm such as generating feasible policies and valid upper/lower bounds are established. The paper then applies the modeling and solution techniques to allocate water in a rapidly-developing area of Tucson, Arizona. Tucson, like many arid and semi-arid regions around the world, faces considerable uncertainty in its ability to provide water for its citizens in the future. The primary sources of uncertainty in the Tucson region include (1) unpredictable population growth, (2) the availability of water from the Colorado River, and (3) the effects of climate variability on water consumption. This paper integrates forecasts for all these sources of uncertainty into a single optimization model for robust and sustainable water allocation. Then, it uses this model to analyze the value of constructing additional treatment facilities to reduce future water shortages. The results indicate that the MDRO approach can be very valuable for water managers by providing insights to minimize their risks and help them plan for the future.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Local structure graph models with higher-order dependence

Local structure graph models (LSGMs) describe random graphs and networks as a Markov random field (MRF)—each graph edge has a specified conditional distribution dependent on explicit neighbourhoods of other graph edges. Centered parameterizations of LSGMs allow for direct control and interpretation of parameters for large- and small-scale structures (e.g., marginal means vs. dependence). Here, we extend this parameterization to account for triples of dependent edges and illustrate the importance of centered parameterizations for incorporating covariates and interpreting parameters. Using a MRF framework, common exponential random graph models are also shown to induce conditional distributions without centered parameterizations and thereby have undesirable features. This work attempts to advance graph models through conditional model specifications with modern parameterizations, covariates and higher-order dependencies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

SIDDA: SInkhorn Dynamic Domain Adaptation for image classification with equivariant neural networks

Modern neural networks (NNs) often do not generalize well in the presence of a ‘covariate shift’; that is, in situations where the training and test data distributions differ, but the conditional distribution of classification labels given the data remains unchanged. In such cases, NN generalization can be reduced to a problem of learning more robust, domain-invariant features. Domain adaptation (DA) methods include a broad range of techniques aimed at achieving this; however, these methods have struggled with the need for extensive hyperparameter tuning, which then incurs significant computational costs. In this work, we introduce SInkhorn Dynamic Domain Adaptation (SIDDA), an out-of-the-box DA training algorithm built upon the Sinkhorn divergence, that can achieve effective domain alignment with minimal hyperparameter tuning and computational overhead. We demonstrate the efficacy of our method on multiple simulated and real datasets of varying complexity, including simple shapes, handwritten digits, real astronomical observations, and remote sensing data. These datasets exhibit covariate shifts due to noise, blurring, differences between telescopes, and variations in imaging wavelengths. SIDDA is compatible with a variety of NN architectures, and it works particularly well in improving classification accuracy and model calibration when paired with symmetry-aware equivariant NNs (ENNs). We find that SIDDA consistently enhances the generalization capabilities of NNs, achieving up to a ${\approx}40\%$ improvement in classification accuracy on unlabeled target data, while also providing a more modest performance gain of $\lesssim 1\%$ on labeled source data. We also study the efficacy of DA on ENNs with respect to the varying group orders of the dihedral group DN, and find that the model performance improves as the degree of equivariance increases. Finally, if SIDDA achieves proper domain alignment, it also enhances model calibration on both source and target data, with the most significant gains in the unlabeled target domain—achieving over an order of magnitude improvement in the expected calibration error and Brier score. SIDDA’s versatility across various NN models and datasets, combined with its automated approach to domain alignment, has the potential to significantly advance multi-dataset studies by enabling the development of highly generalizable models.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

SIDDA: SInkhorn Dynamic Domain Adaptation

Modern neural networks (NNs) often do not generalize well in the presence of a "covariate shift"; that is, in situations where the training and test data distributions differ, but the conditional distribution of classification labels remains unchanged. In such cases, NN generalization can be reduced to a problem of learning more domain-invariant features. Domain adaptation (DA) methods include a range of techniques aimed at achieving this; however, these methods have struggled with the need for extensive hyperparameter tuning, which then incurs significant computational costs. In this work, we introduce SIDDA, an out-of-the-box DA training algorithm built upon the Sinkhorn divergence, that can achieve effective domain alignment with minimal hyperparameter tuning and computational overhead. We demonstrate the efficacy of our method on multiple simulated and real datasets of varying complexity, including simple shapes, handwritten digits, and real astronomical observations. SIDDA is compatible with a variety of NN architectures, and it works particularly well in improving classification accuracy and model calibration when paired with equivariant neural networks (ENNs). We find that SIDDA enhances the generalization capabilities of NNs, achieving up to a ≈40% improvement in classification accuracy on unlabeled target data. We also study the efficacy of DA on ENNs with respect to the varying group orders of the dihedral group DN, and find that the model performance improves as the degree of equivariance increases. Finally, we find that SIDDA enhances model calibration on both source and target data--achieving over an order of magnitude improvement in the ECE and Brier score. SIDDA's versatility, combined with its automated approach to domain alignment, has the potential to advance multi-dataset studies by enabling the development of highly generalizable models.

Pandya, Sneh [Northeastern U.]↗
NASA NTRS1973

The specification of distributed boundary conditions in numerical simulation of the ionosphere

The approach described makes it possible to use boundary conditions which do not have to be associated with any particular point of the solution interval. The approach was applied in a program which solves simultaneously the coupled differential equations describing the concentrations of the O(+) ions, their temperature, the electron temperature and the concentrations of O2(+), NO(+) and N2(+) ions. Several versions of the proposed scheme are described for various kinds of boundary conditions. Attention is given to a given columnar ionic content, a given incoming or outgoing flux, or a given height of the peak concentration.

Waldman, H.↗
DOE OSTI2024

Bounds on galaxy stochasticity from halo occupation distribution modeling

The joint probability distribution of matter overdensity and galaxy counts in cells is a powerful probe of cosmology, and the extent to which variance in galaxy counts at fixed matter density deviates from Poisson shot noise is not fully understood. The lack of informed bounds on this stochasticity is currently the limiting factor in constraining cosmology with the galaxy–matter probability distribution function (PDF). We investigate stochasticity in the conditional distribution of galaxy counts along lines of sight with fixed matter density, and we present a halo occupation distribution (HOD)-based approach for obtaining plausible ranges for stochasticity parameters. To probe the high-dimensional space of possible galaxy–matter connections, we derive a set of HODs that conserve the galaxies’ linear bias and number density to produce RED M A G I C-like galaxy catalogs within the A BACUS S UMMIT suite of N -body simulations. We study the impact of individual HOD parameters and cosmology on stochasticity and perform a Monte Carlo search in HOD parameter space subject to the constraints on bias and density. In mock catalogs generated by the selected HODs, shot noise in galaxy counts spans both sub-Poisson and super-Poisson values, ranging from 80% to 133% of Poisson variance for cells with mean matter density. Nearly all of the derived HODs show a positive relationship between local matter density and stochasticity. For galaxy catalogs with higher stochasticity, modeling galaxy bias to second order is required for an accurate description of the conditional PDF of galaxy counts at fixed matter density. The presence of galaxy assembly bias also substantially extends the range of stochasticity in the super-Poisson direction. This HOD-based approach leverages degrees of freedom in the galaxy–halo connection to obtain informed bounds on nuisance model parameters and can be adapted to study other parametrizations of shot noise in galaxy counts, in particular to motivate prior ranges on stochasticity for cosmological analyses.

Britt, Dylan (ORCID:000000019905601X)↗
DOE OSTI2026

Cluster infall for mass calibration in the stage-IV era

The outskirts of galaxy clusters present a promising avenue for constraining cluster masses in a way that is robust to the impact of baryonic physics. We assess the accuracy to which the cluster infall regions can be used for cluster mass calibration. Building on previous work, we parametrize the velocity distribution 𝑃⁡(𝑣r,𝑣tan|𝑟,𝑀) of dark matter halos on scales 𝑟 ≥ 5⁢ℎ −1 Mpc as the product of the marginalized distribution 𝑃⁡(𝑣 r |𝑟,𝑀) and the conditional distribution 𝑃⁡(𝑣 tan |𝑣 r ,𝑟,𝑀), calibrating the radial and mass dependence of these distributions in numerical simulations. We then project our model along the line of sight to obtain accurate predictions for the distributions of line-of-sight velocities at a given projected radius and cluster mass 𝑃⁡(𝑣 LOS |𝑅,𝑀), which we can observe with spectroscopic survey data. Furthermore, with our model, we forecast that spectra from the Dark Energy Spectroscopic Instrument can constrain cluster masses with subpercent-level precision, comparable to that of stage-IV weak lensing surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Elliptically-Contoured Tensor-variate Distributions with Application to Image Learning

Statistical analysis of tensor-valued data has largely used the tensor-variate normal (TVN) distribution that may be inadequate for data arising from distributions with heavier or lighter tails. We study a general family of elliptically contoured (EC) TV distributions and derive its characterizations, moments, marginal, and conditional distributions. We describe procedures for maximum likelihood estimation from data that are (1) uncorrelated draws from an EC distribution, (2) from a scale mixture of the TVN distribution, and (3) from an underlying but unknown EC distribution, for which we extend Tyler’s robust estimator. A detailed simulation study highlights the benefits of choosing an EC distribution over the TVN for heavier-tailed data. We develop TV classification rules using discriminant analysis and EC errors and show that they better predict cats and dogs from images in the Animal Faces-HQ dataset than the TVN-based rules. A novel tensor-on-tensor regression and TV analysis of variance (TANOVA) framework under EC errors is also demonstrated to better characterize gender, age, and ethnic origin than the usual TVN-based TANOVA in the celebrated labeled faces of the wild dataset.

97 MATHEMATICS AND COMPUTING↗