Search NASA⌕ Search

SEARCH · Search NASA

Results for “High-Dimensional”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
NASA NTRS1996

Communications and information research: Improved space link performance via concatenated forward error correction coding

With the development of new advanced instruments for remote sensing applications, sensor data will be generated at a rate that not only requires increased onboard processing and storage capability, but imposes demands on the space to ground communication link and ground data management-communication system. Data compression and error control codes provide viable means to alleviate these demands. Two types of data compression have been studied by many researchers in the area of information theory: a lossless technique that guarantees full reconstruction of the data, and a lossy technique which generally gives higher data compaction ratio but incurs some distortion in the reconstructed data. To satisfy the many science disciplines which NASA supports, lossless data compression becomes a primary focus for the technology development. While transmitting the data obtained by any lossless data compression, it is very important to use some error-control code. For a long time, convolutional codes have been widely used in satellite telecommunications. To more efficiently transform the data obtained by the Rice algorithm, it is required to meet the a posteriori probability (APP) for each decoded bit. A relevant algorithm for this purpose has been proposed which minimizes the bit error probability in the decoding linear block and convolutional codes and meets the APP for each decoded bit. However, recent results on iterative decoding of 'Turbo codes', turn conventional wisdom on its head and suggest fundamentally new techniques. During the past several months of this research, the following approaches have been developed: (1) a new lossless data compression algorithm, which is much better than the extended Rice algorithm for various types of sensor data, (2) a new approach to determine the generalized Hamming weights of the algebraic-geometric codes defined by a large class of curves in high-dimensional spaces, (3) some efficient improved geometric Goppa codes for disk memory systems and high-speed mass memory systems, and (4) a tree based approach for data compression using dynamic programming.

Rao, T. R. N.↗
NASA NTRS1998

Support Vector Machines for Hyperspectral Remote Sensing Classification

The Support Vector Machine provides a new way to design classification algorithms which learn from examples (supervised learning) and generalize when applied to new data. We demonstrate its success on a difficult classification problem from hyperspectral remote sensing, where we obtain performances of 96%, and 87% correct for a 4 class problem, and a 16 class problem respectively. These results are somewhat better than other recent results on the same data. A key feature of this classifier is its ability to use high-dimensional data without the usual recourse to a feature selection step to reduce the dimensionality of the data. For this application, this is important, as hyperspectral data consists of several hundred contiguous spectral channels for each exemplar. We provide an introduction to this new approach, and demonstrate its application to classification of an agriculture scene.

Gualtieri, J. Anthony↗
NASA NTRS2002

The Statistical Mechanics of Ideal Homogeneous Turbulence

Plasmas, such as those found in the space environment or in plasma confinement devices, are often modeled as electrically conducting fluids. When fluids and plasmas are energetically stirred, regions of highly nonlinear, chaotic behavior known as turbulence arise. Understanding the fundamental nature of turbulence is a long-standing theoretical challenge. The present work describes a statistical theory concerning a certain class of nonlinear, finite dimensional, dynamical models of turbulence. These models arise when the partial differential equations describing incompressible, ideal (i.e., nondissipative) homogeneous fluid and magnetofluid (i.e., plasma) turbulence are Fourier transformed into a very large set of ordinary differential equations. These equations define a divergenceless flow in a high-dimensional phase space, which allows for the existence of a Liouville theorem, guaranteeing a distribution function based on constants of the motion (integral invariants). The novelty of these particular dynamical systems is that there are integral invariants other than the energy, and that some of these invariants behave like pseudoscalars under two of the discrete symmetry transformations of physics, parity, and charge conjugation. In this work the 'rugged invariants' of ideal homogeneous turbulence are shown to be the only significant scalar and pseudoscalar invariants. The discovery that pseudoscalar invariants cause symmetries of the original equations to be dynamically broken and induce a nonergodic structure on the associated phase space is the primary result presented here. Applicability of this result to dissipative turbulence is also discussed.

Shebalin, John V.↗
NASA NTRS1998

Demonstrations at Supercomputing 1998

Contents of the papers include: Visualization techniques- The next generation, Small-machine visualizing very big data sets, Public display of Lunar Prospector data, Steering molecules via Feel, High-dimensional data browser, Multi-source visualization, Interactive surface flow visualization, and the Virtual Windtunnel.

Bryson, Steve↗
NASA NTRS2003

Mining Distance Based Outliers in Near Linear Time with Randomization and a Simple Pruning Rule

Defining outliers by their distance to neighboring examples is a popular approach to finding unusual examples in a data set. Recently, much work has been conducted with the goal of finding fast algorithms for this task. We show that a simple nested loop algorithm that in the worst case is quadratic can give near linear time performance when the data is in random order and a simple pruning rule is used. We test our algorithm on real high-dimensional data sets with millions of examples and show that the near linear scaling holds over several orders of magnitude. Our average case analysis suggests that much of the efficiency is because the time to process non-outliers, which are the majority of examples, does not depend on the size of the data set.

Bay, Stephen D.↗
NASA NTRS2003

Gene Expression Dynamics Inspector (GEDI): for integrative analysis of expression profiles

Genome-wide expression profiles contain global patterns that evade visual detection in current gene clustering analysis. Here, a Gene Expression Dynamics Inspector (GEDI) is described that uses self-organizing maps to translate high-dimensional expression profiles of time courses or sample classes into animated, coherent and robust mosaics images. GEDI facilitates identification of interesting patterns of molecular activity simultaneously across gene, time and sample space without prior assumption of any structure in the data, and then permits the user to retrieve genes of interest. Important changes in genome-wide activities may be quickly identified based on 'Gestalt' recognition and hence, GEDI may be especially useful for non-specialist end users, such as physicians. AVAILABILITY: GEDI v1.0 is written in Matlab, and binary Matlab.dll files which require Matlab to run can be downloaded for free by academic institutions at http://www.chip.org/~ge/gedihome.html Supplementary information: http://www.chip.org/~ge/gedihome.html.

Database Management Systems↗
NASA NTRS2005

Discovering System Health Anomalies Using Data Mining Techniques

We present a data mining framework for the analysis and discovery of anomalies in high-dimensional time series of sensor measurements that would be found in an Integrated System Health Monitoring system. We specifically treat the problem of discovering anomalous features in the time series that may be indicative of a system anomaly, or in the case of a manned system, an anomaly due to the human. Identification of these anomalies is crucial to building stable, reusable, and cost-efficient systems. The framework consists of an analysis platform and new algorithms that can scale to thousands of sensor streams to discovers temporal anomalies. We discuss the mathematical framework that underlies the system and also describe in detail how this framework is general enough to encompass both discrete and continuous sensor measurements. We also describe a new set of data mining algorithms based on kernel methods and hidden Markov models that allow for the rapid assimilation, analysis, and discovery of system anomalies. We then describe the performance of the system on a real-world problem in the aircraft domain where we analyze the cockpit data from aircraft as well as data from the aircraft propulsion, control, and guidance systems. These data are discrete and continuous sensor measurements and are dealt with seamlessly in order to discover anomalous flights. We conclude with recommendations that describe the tradeoffs in building an integrated scalable platform for robust anomaly detection in ISHM applications.

Sriastava, Ashok, N.↗
DOE OSTI2024

DeepUQ: Assessing the Aleatoric Uncertainties from two Deep Learning Methods

Assessing the quality of aleatoric uncertainty estimates from uncertainty quantification (UQ) deep learning methods is important in scientific contexts, where uncertainty is physically meaningful and important to characterize and interpret exactly. We systematically compare aleatoric uncertainty measured by two UQ techniques, Deep Ensembles (DE) and Deep Evidential Regression (DER). Our method focuses on both zero-dimensional (0D) and two-dimensional (2D) data, to explore how the UQ methods function for different data dimensionalities. We investigate uncertainty injected on the input and output variables and include a method to propagate uncertainty in the case of input uncertainty so that we can compare the predicted aleatoric uncertainty to the known values. We experiment with three levels of noise. The aleatoric uncertainty predicted across all models and experiments scales with the injected noise level. However, the predicted uncertainty is miscalibrated to $\rm{std}(\sigma_{\rm al})$ with the true uncertainty for half of the DE experiments and almost all of the DER experiments. The predicted uncertainty is the least accurate for both UQ methods for the 2D input uncertainty experiment and the high-noise level. While these results do not apply to more complex data, they highlight that further research on post-facto calibration for these methods would be beneficial, particularly for high-noise and high-dimensional settings.

Nevin, Rebecca↗
DOE OSTI2025

Using machine learning to jointly harness the strength of microscopic, fundamental-science driven and macroscopic, application-driven experiments

The PARADIGM project aims at accelerating progress in science by quantitatively answering the following question: What is the optimal combination of fundamental-science and application driven experiments to maximally reduce pertinent data uncertainties? Hence, we are bridging between microscopic experiments and data, and macroscopic simulations and experiments. Answering this question entails solving a high-dimensional and complex optimization problem which we solve with machine learning techniques.

LANSCE↗
DOE OSTI2026

ZEUS: An Efficient GPU Optimization Method Integrating PSO, BFGS, and Automatic Differentiation

We introduce a novel, efficient computational method, ZEUS, for numerical optimization, and provide an open-source implementation. It has four key ingredients: (1) particle swarm optimization (PSO), (2) the use of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, (3) automatic differentiation (AD), and (4) GPUs. Our approach addresses the computational challenges inherent in high-dimensional, non-convex optimization problems. In the first phase of the algorithm, we get a potentially good set of starting points using PSO. Thereafter, we run BFGS independently in parallel from these starting points. BFGS is one of the best-performing algorithms for numerical optimization. However, it requires the gradient of the function being optimized. ZEUS integrates automatic differentiation into BFGS thus avoiding the need for the user to calculate derivatives explicitly. The use of GPUs allows ZEUS to speed up the calculations substantially. We carry out systematic studies to explore the trade-offs between the number of PSO iterations taken, starting points, and BFGS iteration depth. We show that a handful of iterations of PSO can improve global convergence when combined with BFGS. We also present performance studies using common test functions. The source code can be found at https://github.com/fnal-numerics/global-optimizer-gpu.

Soos, Dominik [Old Dominion U.]↗
DOE OSTI2026

HDSense: An efficient method for ranking observable sensitivity

Identifying which observables most effectively constrain model parameters can be computationally prohibitive when considering full likelihoods of many correlated observables. This is especially important for, e.g., hadronization models, where high precision is required to interpret the results of collider experiments. We introduce the High-Dimensional Sensitivity (HDSense) score, a computationally efficient metric for ranking observable sets using only one-dimensional histograms. Derived by profiling over unknown correlations in the Fisher information framework, the score balances total information content against redundancy between observables. We apply HDSense to rank a set observables in terms of their constraining power with respect to five parameters of the Lund string model of hadronization implemented in Pythia using simulated leptonic collider events at the $Z$ pole. Validation against machine-learning--based full-likelihood approximations demonstrates that HDSense successfully identifies near-optimal observable subsets. The framework naturally handles data from multiple experiments with different acceptances and incorporates detector effects. While demonstrated on hadronization models, the methodology applies broadly to generic parameter estimation problems where correlations are unknown or difficult to model.

Assi, Benoît [Cincinnati U.] (ORCID:00000003092433↗
DOE OSTI2026

Generative models on phase space

Deep generative models such as diffusion and flow matching are powerful machine learning tools capable of learning and sampling from high-dimensional distributions. They are particularly useful when the training data appears to be concentrated on a submanifold of the data embedding space. For high-energy physics data, consisting of collections of relativistic energy-momentum 4-vectors, this submanifold can enforce extremely strong physically-motivated priors, such as energy and momentum conservation. If these constraints are learned only approximately, rather than exactly, this can inhibit the interpretability and reliability of such generative models. To remedy this deficiency, we introduce generative models which are, by construction, confined at every step of their sampling trajectory to the manifold of massless N-particle Lorentz-invariant phase space in the center-of-momentum frame. In the case of diffusion models, the "pure noise" forward process endpoint corresponds to the uniform distribution on phase space, which provides a clear starting point from which to identify how correlations among the particles emerge during the reverse (de-noising) process. We demonstrate that our models are able to learn both few-particle and many-particle distributions with various singularity structures, paving the way for future interpretability studies using generative models trained on simulated jet data.

Bogorad, Zachary [Fermilab]↗
DOE OSTI2026

CaloTrilogy: Toward a Breakthrough in One-Step, End-to-End, Physics-Guided Shower Generation for Modern Calorimeters

High-precision calorimeter simulation at current and future colliders imposes rapidly growing computational demands, motivating the development of machine-learning surrogates for traditional Monte Carlo tools such as Geant4. Flow matching and diffusion-based generative models have become leading approaches for high-dimensional fast simulation because of their sample quality, but typically require ${\cal O}(100)$ function evaluations at inference and often rely on auxiliary networks to constrain global observables, compromising streamlined end-to-end generation. We introduce a unified framework that improves the balance between speed, shower quality, and physics fidelity. The method combines: (i) an average velocity field integrator that enables sampling in one or a few evaluations; (ii) a learned generative prior in shower space, constructed from data rather than random noise; and (iii) physics-guided loss terms that impose inductive biases on key observables during training. These elements are training time regularizers, preserving end-to-end inference with no additional cost. With only one or a few evaluation steps, the model achieves shower quality competitive with state-of-the-art flow and diffusion approaches, tested on several public high granularity calorimeter datasets. The results demonstrate inter-layer shower structure consistent with the underlying physics, providing a strong candidate for future fast simulation workflows.

Jiang, Cheng [Edinburgh U.]↗
DOE OSTI2026

Affine Transformations to Enable Machine Learning for Semi-Quantitative EDS Analysis

Energy Dispersive X-ray Spectroscopy (EDS) is an essential technique for determining elemental concentrations and distributions within microstructures, critical for materials discovery, optimization, and qualification. However, most published EDS data is qualitative because current quantitative EDS analysis methods require extensive calibration and post-processing, limiting their practicality and widespread adoption. This work seeks to establish a framework for accelerated EDS characterization and spectrum analysis that can leverage ML to analyze correlations between various elemental compositions and resulting EDS spectra. The complex physics and data result in a high-dimensional problem that grows exponentially with the number of elements in the system and the complexity of the spectrum analysis. ML provides a way to compute and optimize the results of this highly dimensional problem in a flexible way to tailor it to the user’s specific needs and material system. However, the framework emphasizes transparency through a strictly mathematical affine transformation, so the analysis remains understandable and reviewable to facilitate adoption by the scientific community. While currently implemented methods are simplistic and unvalidated, further development and demonstration of this framework could enable high-throughput, accurate, and accessible EDS characterization.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

Affine Transformations to Correlate Experimental and Simulated EDS Spectra for Multi-Element Systems

Energy Dispersive X-ray Spectroscopy (EDS) is an essential technique for determining elemental concentrations and distributions within microstructures, critical for materials discovery, optimization, and qualification. However, most published EDS data is qualitative because current quantitative EDS analysis methods require extensive calibration and post-processing, limiting their practicality and widespread adoption. This work seeks to establish a framework for accelerated EDS characterization and spectrum analysis that can leverage ML to analyze correlations between various elemental compositions and resulting EDS spectra. The complex physics and data result in a high-dimensional problem that grows exponentially with the number of elements in the system and the complexity of the spectrum analysis. ML provides a way to compute and optimize the results of this highly dimensional problem in a flexible way to tailor it to the user’s specific needs and material system. However, the framework emphasizes transparency through a strictly mathematical affine transformation, so the analysis remains understandable and reviewable to facilitate adoption by the scientific community. While currently implemented methods are simplistic and unvalidated, further development and demonstration of this framework could enable high-throughput, accurate, and accessible EDS characterization.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

Strong Lensing Cosmology with Population-level Calibrated Neural Ratio Estimation

Strong gravitational lensing contains key information about cosmic acceleration. Modern and next-generation galaxy imaging surveys are expected to provide high-quality data on $\mathcal{O}(10^5)$ galaxy-galaxy lensing systems. The plethora and complexity of the data are likely to present computational challenges for parameter inference methods for fitting high-dimensional likelihoods, which are often analytically intractable. Neural Ratio Estimation (NRE) efficiently computes individual likelihood ratios that can be combined into population-level posteriors. We use simulations to study the capacity of NRE to jointly predict the dark energy equation-of-state parameter $w$ and the total matter density $Ω_{m}$ from lensing images and companion spectroscopic information. We also introduce a post hoc posterior coverage calibration procedure that mitigates the model overconfidence that is typically found in neural density estimation applications. Our experiments show that the errors on both parameters decrease with increasing inference population sizes. In particular, for 100 lenses in a standard $Λ$CDM Universe, our calibrated NRE model achieves median fractional uncertainty of $22.8\%$ in $w$ and $2.9\%$ in $Ω_{m}$. This proof of concept demonstrates a potentially scalable approach for efficient cosmological parameter inference with large populations of galaxy-scale lenses observed in future surveys.

Jarugula, Sreevani [Fermilab] (ORCID:0000000253867↗