Search NASA⌕ Search

SEARCH · Search NASA

Results for “Curse of dimensionality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

65 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1991

Supercomputer optimizations for stochastic optimal control applications

Supercomputer optimizations for a computational method of solving stochastic, multibody, dynamic programming problems are presented. The computational method is valid for a general class of optimal control problems that are nonlinear, multibody dynamical systems, perturbed by general Markov noise in continuous time, i.e., nonsmooth Gaussian as well as jump Poisson random white noise. Optimization techniques for vector multiprocessors or vectorizing supercomputers include advanced data structures, loop restructuring, loop collapsing, blocking, and compiler directives. These advanced computing techniques and superconducting hardware help alleviate Bellman's curse of dimensionality in dynamic programming computations, by permitting the solution of large multibody problems. Possible applications include lumped flight dynamics models for uncertain environments, such as large scale and background random aerospace fluctuations.

Chung, Siu-Leung↗
NASA NTRS2002

Spectral Data Reduction via Wavelet Decomposition

The greatest advantage gained from hyperspectral imagery is that narrow spectral features can be used to give more information about materials than was previously possible with broad-band multispectral imagery. For many applications, the new larger data volumes from such hyperspectral sensors, however, present a challenge for traditional processing techniques. For example, the actual identification of each ground surface pixel by its corresponding reflecting spectral signature is still one of the most difficult challenges in the exploitation of this advanced technology, because of the immense volume of data collected. Therefore, conventional classification methods require a preprocessing step of dimension reduction to conquer the so-called "curse of dimensionality." Spectral data reduction using wavelet decomposition could be useful, as it does not only reduce the data volume, but also preserves the distinctions between spectral signatures. This characteristic is related to the intrinsic property of wavelet transforms that preserves high- and low-frequency features during the signal decomposition, therefore preserving peaks and valleys found in typical spectra. When comparing to the most widespread dimension reduction technique, the Principal Component Analysis (PCA), and looking at the same level of compression rate, we show that Wavelet Reduction yields better classification accuracy, for hyperspectral data processed with a conventional supervised classification such as a maximum likelihood method.

Kaewpijit, S.↗
NASA NTRS2011

Model Adaptation for Prognostics in a Particle Filtering Framework

One of the key motivating factors for using particle filters for prognostics is the ability to include model parameters as part of the state vector to be estimated. This performs model adaptation in conjunction with state tracking, and thus, produces a tuned model that can used for long term predictions. This feature of particle filters works in most part due to the fact that they are not subject to the "curse of dimensionality", i.e. the exponential growth of computational complexity with state dimension. However, in practice, this property holds for "well-designed" particle filters only as dimensionality increases. This paper explores the notion of wellness of design in the context of predicting remaining useful life for individual discharge cycles of Li-ion batteries. Prognostic metrics are used to analyze the tradeoff between different model designs and prediction performance. Results demonstrate how sensitivity analysis may be used to arrive at a well-designed prognostic model that can take advantage of the model adaptation properties of a particle filter.

Saha, Bhaskar↗
NASA NTRS2014

Neural Network Machine Learning and Dimension Reduction for Data Visualization

Neural network machine learning in computer science is a continuously developing field of study. Although neural network models have been developed which can accurately predict a numeric value or nominal classification, a general purpose method for constructing neural network architecture has yet to be developed. Computer scientists are often forced to rely on a trial-and-error process of developing and improving accurate neural network models. In many cases, models are constructed from a large number of input parameters. Understanding which input parameters have the greatest impact on the prediction of the model is often difficult to surmise, especially when the number of input variables is very high. This challenge is often labeled the "curse of dimensionality" in scientific fields. However, techniques exist for reducing the dimensionality of problems to just two dimensions. Once a problem's dimensions have been mapped to two dimensions, it can be easily plotted and understood by humans. The ability to visualize a multi-dimensional dataset can provide a means of identifying which input variables have the highest effect on determining a nominal or numeric output. Identifying these variables can provide a better means of training neural network models; models can be more easily and quickly trained using only input variables which appear to affect the outcome variable. The purpose of this project is to explore varying means of training neural networks and to utilize dimensional reduction for visualizing and understanding complex datasets.

Liles, Charles A.↗
NASA NTRS2017

Reachability Subspace Exploration Using Continuation Methods

Reachability manifold computation suffers from the curse of dimensionality and for large state spaces is computationally intractable. This paper examines the use of continuation methods to address this issue by formulating the reachability sub-space manifold calculation into a number of initial valued problems. As a result of computing the reachability manifold for a subspace of interest, an exponential improvement in computational cost occurs. This concept is applied to a position subspace reachability problem of a spacecraft in a Keplerian orbit under maximum thrust constraints. Future work includes a comparison of the proposed method with computing reachability manifolds using viscosity solutions of the Hamilton Jacobi Bellman partial differential equation.

orbital mechanics↗
NASA NTRS2018

An Active Subspace Method for Accelerating Convergence in Delaunay-Based Optimization via Dimension Reduction

Delaunay-based derivative-free optimization, ∆DOGS, is an efficient and provably-convergent global optimization method for the problems which has computationally expensive objection function and the analytical expression for the objective function is not available. ∆-DOGS is a novel optimization scheme in the family of response surface methods (RSMs); however, it suffers from the curse of dimensionality since the computational cost increases dramatically as the number of design parameters increases. As a result, the number of design parameters in ∆-DOGS algorithm is relatively low (n.10). To avoid such problems, this paper proposes a combination of derivative-free optimization, seeking the global minimizer of an expensive and nonconvex objective function f(x) and active subspace method, detecting the directions of the most variability using evaluations of the gradient. The contribution of other directions to the objective function is bounded by a sufficiently small constant. This new algorithm iteratively applied Delaunay-based derivative-free optimization to seek the minimizer on the d-dimensional active subspace that has most function variation. Inverse mapping is needed to project data from active subspace to full-model for evaluating function values. This task is overcome by solving an inequality constrained problem that curves the response surface of the objective function. The test results show that this strategy is effective on a handful of optimization problems.

Bewley, Thomas R.↗
DOE OSTI2026

Universal Optimization and Tighter Fidelity Bounds for Approximate Quantum Error Correction

Approximate quantum error correction (AQEC) not only dictates the performance of discrete- and continuous-variable quantum error correction codes but also serves as a unifying framework across various physical disciplines. Identifying the optimal recovery channel to maximize the entanglement fidelity via standard semidefinite programming is computationally bottlenecked by the exponentially growing number of Kraus operators with system size, rendering large-scale optimization prohibitive. While analytical near-optimal maps exist, they typically work only when the Knill-Laflamme conditions are nearly satisfied. In this Letter, we establish an efficient framework by leveraging the duality between recovery and environment decoupling. This framework yields a tighter analytical lower bound on entanglement fidelity than the conventional limit set by the transpose channel. Furthermore, by exploiting the decayed weights of noise Kraus operators, we introduce a framework based on principal component analysis to reduce the dimension. In thermal loss channels where the weights decay exponentially, our approach achieves a 33-fold computational speedup while maintaining rigorous accuracy. Our approach enables high-precision optimization for AQEC codes that were previously intractable due to the curse of dimensionality.

Wu, Jing [Fermilab] (ORCID:0000000249460732)↗
NASA NTRS2013

Selection of Hyperspectral Narrowbands (HNBs) and Composition of Hyperspectral Twoband Vegetation Indices (HVIs) for Biophysical Characterization and Discrimination of Crop Types Using Field Reflectance and Hyperion-EO-1 Data

The overarching goal of this study was to establish optimal hyperspectral vegetation indices (HVIs) and hyperspectral narrowbands (HNBs) that best characterize, classify, model, and map the world's main agricultural crops. The primary objectives were: (1) crop biophysical modeling through HNBs and HVIs, (2) accuracy assessment of crop type discrimination using Wilks' Lambda through a discriminant model, and (3) meta-analysis to select optimal HNBs and HVIs for applications related to agriculture. The study was conducted using two Earth Observing One (EO-1) Hyperion scenes and other surface hyperspectral data for the eight leading worldwide crops (wheat, corn, rice, barley, soybeans, pulses, cotton, and alfalfa) that occupy approx. 70% of all cropland areas globally. This study integrated data collected from multiple study areas in various agroecosystems of Africa, the Middle East, Central Asia, and India. Data were collected for the eight crop types in six distinct growth stages. These included (a) field spectroradiometer measurements (350-2500 nm) sampled at 1-nm discrete bandwidths, and (b) field biophysical variables (e.g., biomass, leaf area index) acquired to correspond with spectroradiometer measurements. The eight crops were described and classified using approx. 20 HNBs. The accuracy of classifying these 8 crops using HNBs was around 95%, which was approx. 25% better than the multi-spectral results possible from Landsat-7's Enhanced Thematic Mapper+ or EO-1's Advanced Land Imager. Further, based on this research and meta-analysis involving over 100 papers, the study established 33 optimal HNBs and an equal number of specific two-band normalized difference HVIs to best model and study specific biophysical and biochemical quantities of major agricultural crops of the world. Redundant bands identified in this study will help overcome the Hughes Phenomenon (or "the curse of high dimensionality") in hyperspectral data for a particular application (e.g., biophysical characterization of crops). The findings of this study will make a significant contribution to future hyperspectral missions such as NASA's HyspIRI. Index Terms-Hyperion, field reflectance, imaging spectroscopy, HyspIRI, biophysical parameters, hyperspectral vegetation indices, hyperspectral narrowbands, broadbands.

Vegetation↗
DOE OSTI2024

Hutchinson Trace Estimation for high-dimensional and high-order Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have proven effective in solving partial differential equations (PDEs), especially when some data are available by seamlessly blending data and physics. However, extending PINNs to high-dimensional and even high-order PDEs encounters significant challenges due to the computational cost associated with automatic differentiation in the residual loss function calculation. Herein, we address the limitations of PINNs in handling high-dimensional and high-order PDEs by introducing the Hutchinson Trace Estimation (HTE) method. Starting with the second-order high-dimensional PDEs, which are ubiquitous in scientific computing, HTE is applied to transform the calculation of the entire Hessian matrix into a Hessian vector product (HVP). This approach not only alleviates the computational bottleneck via Taylor-mode automatic differentiation but also significantly reduces memory consumption from the Hessian matrix to an HVP’s scalar output. We further showcase HTE’s convergence to the original PINN loss and its unbiased behavior under specific conditions. Comparisons with the Stochastic Dimension Gradient Descent (SDGD) highlight the distinct advantages of HTE, particularly in scenarios with significant variability and variance among dimensions. We further extend the application of HTE to higher-order and higher-dimensional PDEs, specifically addressing the biharmonic equation. By employing tensor-vector products (TVP), HTE efficiently computes the colossal tensor associated with the fourth-order high-dimensional biharmonic equation, saving memory and enabling rapid computation. The effectiveness of HTE is illustrated through experimental setups, demonstrating comparable convergence rates with SDGD under memory and speed constraints. Additionally, HTE proves valuable in accelerating the Gradient-Enhanced PINN (gPINN) version as well as the Biharmonic equation. Overall, HTE opens up a new capability in scientific machine learning for tackling high-order and high-dimensional PDEs.

Curse of dimensionality↗
DOE OSTI2025

Bias-Variance Trade-Off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs

Physics-Informed Neural Networks (PINNs) have triggered a paradigm shift in scientific computing, leveraging mesh-free properties and robust approximation capabilities. While proving effective for low-dimensional partial differential equations (PDEs), the computational cost of PINNs remains a hurdle in high-dimensional scenarios. This is particularly pronounced when computing high-order and high-dimensional derivatives in the physics-informed loss. Randomized Smoothing PINN (RS-PINN) introduces Gaussian noise for stochastic smoothing of the original neural net model, enabling the use of Monte Carlo methods for derivative approximation, which eliminates the need for costly automatic differentiation. Despite its computational efficiency, especially in the approximation of high-dimensional derivatives, RS-PINN introduces biases in both loss and gradients, negatively impacting convergence, especially when coupled with stochastic gradient descent (SGD) algorithms. We present a comprehensive analysis of biases in RS-PINN, attributing them to the nonlinearity of the Mean Squared Error (MSE) loss as well as the intrinsic nonlinearity of the PDE itself. We propose tailored bias correction techniques, delineating their application based on the order of PDE nonlinearity. The derivation of an unbiased RS-PINN allows for a detailed examination of its advantages and disadvantages compared to the biased version. Specifically, the biased version has a lower variance and runs faster than the unbiased version, but it is less accurate due to the bias. To optimize the bias-variance trade-off, we combine the two approaches in a hybrid method that balances the rapid convergence of the biased version with the high accuracy of the unbiased version. In addition to methodological contributions, we present an enhanced implementation of RS-PINN. Extensive experiments on diverse high-dimensional PDEs, including Fokker-Planck, Hamilton-Jacobi-Bellman (HJB), viscous Burgers’, Allen-Cahn, and Sine-Gordon equations, illustrate the bias-variance trade-off and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are provided for selecting biased, unbiased, or hybrid versions, depending on the dimensionality and nonlinearity of the specific PDE problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Diffusion-Model-Assisted Supervised Learning of Generative Models for Density Estimation

Here, we present a supervised learning framework of training generative models for density estimation. Generative models, including generative adversarial networks (GANs), normalizing flows, and variational auto-encoders (VAEs), are usually considered as unsupervised learning models, because labeled data are usually unavailable for training. Despite the success of the generative models, there are several issues with the unsupervised training, e.g., requirement of reversible architectures, vanishing gradients, and training instability. To enable supervised learning in generative models, we utilize the score-based diffusion model to generate labeled data. Unlike existing diffusion models that train neural networks to learn the score function, we develop a training-free score estimation method. This approach uses mini-batch-based Monte Carlo estimators to directly approximate the score function at any spatial-temporal location in solving an ordinary differential equation (ODE), corresponding to the reverse-time stochastic differential equation (SDE). This approach can offer both high accuracy and substantial time savings in neural network training. Once the labeled data are generated, we can train a simple, fully connected neural network to learn the generative model in the supervised manner. Compared with existing normalizing flow models, our method does not require the use of reversible neural networks and avoids the computation of the Jacobian matrix. Compared with existing diffusion models, our method does not need to solve the reverse-time SDE to generate new samples. As a result, the sampling efficiency is significantly improved. We demonstrate the performance of our method by applying it to a set of 2D datasets as well as real data from the University of California Irvine (UCI) repository.

97 MATHEMATICS AND COMPUTING↗