Search NASA⌕ Search

SEARCH · Search NASA

Results for “high dimensional data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Visualizing Temporal Topic Embeddings with a Compass

—Dynamic topic modeling is useful at discovering the development and change in latent topics over time. However, present methodology relies on algorithms that separate document and word representations. This prevents the creation of a meaningful embedding space where changes in word usage and documents can be directly analyzed in a temporal context. This paper proposes an expansion of the compass-aligned temporal Word2Vec methodology into dynamic topic modeling. Such a method allows for the direct comparison of word and document embeddings across time in dynamic topics. This enables the creation of visualizations that incorporate temporal word embeddings within the context of documents into topic visualizations. In experiments against the current state-of-the-art, our proposed method demonstrates overall competitive performance in topic relevancy and diversity across temporal datasets of varying size. Simultaneously, it provides insightful visualizations focused on temporal word embeddings while maintaining the insights provided by global topic evolution, advancing our understanding of how topics evolve over time.

Cluster analysis↗
DOE OSTI2025

Quantum Transfer Learning to Boost Dementia Detection

Dementia is a devastating condition with profound implications for individuals, families, and healthcare systems. Early and accurate detection of dementia is critical for timely intervention and improved patient outcomes. While classical machine learning and deep learning approaches have been explored extensively for dementia prediction, these solutions often struggle with high-dimensional biomedical data and large-scale datasets, quickly reaching computational and performance limitations. To address this challenge, quantum machine learning (QML) has emerged as a promising paradigm, offering faster training and advanced pattern recognition capabilities. This work aims to demonstrate the potential of quantum transfer learning (QTL) to enhance the performance of a weak classical deep learning model applied to a binary classification task for dementia detection. Besides, we show the effect of noise on the QTL-based approach, investigating the reliability and robustness of this method. Using the OASIS 2 dataset, we show how quantum techniques can transform a suboptimal classical model into a more effective solution for biomedical image classification, highlighting their potential impact on advancing healthcare technology.

Bhowmik, Sounak [University of Tennessee, Knoxvill↗
DOE OSTI2025

Divertor Plasma Detachment Control Neural Network

DivControlNN is a state-of-the-art software tool that leverages advanced machine learning techniques to predict and control divertor plasma behavior in fusion reactors. Plasma, a highly energetic and electrically charged gas, requires meticulous management to protect reactor components and maintain optimal energy production. Conventional simulation methods, although extremely detailed, typically demand extensive computational time-making them unsuitable for real-time control scenarios. DivControlNN addresses this challenge by learning from tens of thousands of high-fidelity simulations, thereby creating a rapid surrogate model that can deliver near-instantaneous predictions. At the core of its functionality is a sophisticated technique known as latent space mapping, which condenses complex, high-dimensional plasma data into a compact, lower-dimensional representation. This streamlined representation enables the system to quickly forecast essential plasma properties and determine the precise conditions required for effective detachment. Detachment is a crucial process in which the plasma is cooled before reaching the divertor plates, thereby reducing heat loads and mitigating material erosion. In recent experiments conducted on the KSTAR tokamak in South Korea, DivControlNN successfully guided the detachment process without any fine-tuning-even when applied to a new tungsten divertor configuration. By achieving a computational speed-up of over one hundred million times compared to traditional simulation methods while maintaining low prediction errors, DivControlNN stands to significantly enhance real-time control and diagnostic capabilities in future fusion reactors. This breakthrough paves the way for safer, more reliable reactor operation and represents a major advancement toward realizing fusion energy as a practical, sustainable, and clean power source.

Xu, Xueqiao [Lawrence Livermore National Laborator↗
DOE OSTI2024

Reviews and syntheses: Opportunities for robust use of peak intensities from high-resolution mass spectrometry in organic matter studies

Abstract. Earth's biogeochemical cycles are intimately tied to the biotic and abiotic processing of organic matter (OM). Spatial and temporal variations in OM chemistry are often studied using direct infusion, high-resolution Fourier transform mass spectrometry (FTMS). An increasingly common approach is to use ecological metrics (e.g., within-sample diversity) to summarize high-dimensional FTMS data, notably Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR MS). However, problems can arise when FTMS peak-intensity data are used in a way that is analogous to abundances in ecological analyses (e.g., species abundance distributions). Using peak-intensity data in this way requires the assumption that intensities act as direct proxies for concentrations. Here, we show that comparisons of the same peak across samples (within-peak) may carry information regarding variations in relative concentration, but comparing different peaks (between-peak) within or between samples does not. We further developed a simulation model to study the quantitative implications of using peak intensities to compute ecological metrics (e.g., intensity-weighted mean properties and diversity) that rely on information about both within-peak and between-peak shifts in relative abundance. We found that, despite analytical limitations in linking concentration to intensity, ecological metrics often perform well in terms of providing robust qualitative inferences and sometimes quantitatively accurate estimates of diversity and mean molecular characteristics. We conclude with recommendations for the robust use of peak intensities for natural organic matter studies. A primary recommendation is the use and extension of the simulation model to provide objective guidance on the degree to which conceptual and quantitative inferences can be made for a given analysis of a given dataset. Broad use of this approach can help ensure rigorous scientific outcomes from the use of FTMS peak intensities in environmental applications.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1992

High dimensional reflectance analysis of soil organic matter

Recent breakthroughs in remote-sensing technology have led to the development of high spectral resolution imaging sensors for observation of earth surface features. This research was conducted to evaluate the effects of organic matter content and composition on narrowband soil reflectance across the visible and reflective infrared spectral ranges. Organic matter from four Indiana agricultural soils, ranging in organic C content from 0.99 to 1.72 percent, was extracted, fractionated, and purified. Six components of each soil were isolated and prepared for spectral analysis. Reflectance was measured in 210 narrow bands in the 400- to 2500-nm wavelength range. Statistical analysis of reflectance values indicated the potential of high dimensional reflectance data in specific visible, near-infrared, and middle-infrared bands to provide information about soil organic C content, but not organic matter composition. These bands also responded significantly to Fe- and Mn-oxide content.

Henderson, T. L.↗
NASA NTRS1993

A comparison of spectral mixture analysis an NDVI for ascertaining ecological variables

In this study, we compare the performance of spectral mixture analysis to the Normalized Difference Vegetation Index (NDVI) in detecting change in a grassland across topographically-induced nutrient gradients and different management schemes. The Konza Prairie Research Natural Area, Kansas, is a relatively homogeneous tallgrass prairie in which change in vegetation productivity occurs with respect to topographic positions in each watershed. The area is the site of long-term studies of the influence of fire and grazing on tallgrass production and was the site of the First ISLSCP (International Satellite Land Surface Climatology Project) Field Experiment (FIFE) from 1987 to 1989. Vegetation indices such as NDVI are commonly used with imagery collected in few (less than 10) spectral bands. However, the use of only two bands (e.g. NDVI) does not adequately account for the complex of signals making up most surface reflectance. Influences from background spectral variation and spatial heterogeneity may confound the direct relationship with biological or biophysical variables. High dimensional multispectral data allows for the application position of techniques such as derivative analysis and spectral curve fitting, thereby increasing the probability of successfully modeling the reflectance from mixed surfaces. The higher number of bands permits unmixing of a greater number of surface components, separating the vegetation signal for further analyses relevant to biological variables.

Wessman, Carol A.↗
NASA NTRSDate not supplied

Transcriptomics-based Machine Learning (ML) Analysis Predicts Space-Exposed Murine Livers

NASA has employed high-throughput molecular assays to identify sub-cellular changes impacting human physiology during spaceflight. Machine learning (ML) methods hold the promise to improve our ability to identify important signals within highly dimensional molecular data. However, the inherent limitation of study subject numbers within a spaceflight mission minimizes the utility of ML approaches. To overcome the sample power limitations, data from multiple spaceflight missions must be aggregated while appropriately addressing intra- and inter-study variabilities. Here we describe an approach to log transform, scale and normalize data from six heterogeneous, mouse liver derived transcriptomics datasets (ntotal=137) which enabled ML-methods to perform well (AUC ≥ 0.87) in classifying spaceflown vs ground control animals rather than mission-of-origin. Concordance was found between liver-specific biological processes identified from harmonized ML-based analysis and study-by-study classical omics analysis. This work demonstrates the feasibility of applying ML methods on integrated, heterogeneous datasets of small sample size.

Machine Learning↗
DOE OSTI2024

Quantum Annealing for Real-World Machine Learning Applications

Optimizing the training of a machine learning pipeline is important for reducing training costs and improving model performance. One such optimizing strategy is quantum annealing, which is an emerging computing paradigm that has shown potential in optimizing the training of a machine learning model. The implementation of a physical quantum annealer has been realized by D-Wave systems and is available to the research community for experiments. Recent experimental results on a variety of machine learning applications have shown interesting results especially under the conditions where the performance of classical machine learning techniques are limited such as limited training data and high dimensional features. This chapter explores the application of D-Wave’s quantum annealer for optimizing machine learning pipelines for real-world classification problems. We review the application domains on which a physical quantum annealer has been used to train machine learning classifiers. We discuss and analyze the experiments performed on the D-Wave quantum annealer for applications such as image recognition, remote sensing imagery, security, computational biology, biomedical sciences, and physics. We discuss the possible advantages and the problems for which quantum annealing is likely to be advantageous over classical computation.

Kumar nath, Rajdeep↗
DOE OSTI2026

BoBa

BoBa is a C++ software library for working with large matrices, tensors, and tensor decompositions. The library provides tools for dense matrix and tensor operations, tensor decompositions, and tensor decomposition methods that support modern CPU and GPU architectures. It includes portable abstractions for linear algebra, tensor algebra, and multidimensional computation. BoBa is intended for scientific computing applications that involve large multidimensional data sets or high dimensional mathematical models. Its capabilities support tasks such as data compression, linear algebra, efficient numerical computation, and the development of scalable algorithms for heterogeneous hardware. Tutorials, tests, and example applications are included to help users learn and apply the library.

Yao, Jin [Lawrence Livermore National Laboratory (↗
NASA NTRS1994

High dimensional feature reduction via projection pursuit

The recent development of more sophisticated remote sensing systems enables the measurement of radiation in many more spectral intervals than previously possible. An example of that technology is the AVIRIS system, which collects image data in 220 bands. As a result of this, new algorithms must be developed in order to analyze the more complex data effectively. Data in a high dimensional space presents a substantial challenge, since intuitive concepts valid in a 2-3 dimensional space to not necessarily apply in higher dimensional spaces. For example, high dimensional space is mostly empty. This results from the concentration of data in the corners of hypercubes. Other examples may be cited. Such observations suggest the need to project data to a subspace of a much lower dimension on a problem specific basis in such a manner that information is not lost. Projection Pursuit is a technique that will accomplish such a goal. Since it processes data in lower dimensions, it should avoid many of the difficulties of high dimensional spaces. In this paper, we begin the investigation of some of the properties of Projection Pursuit for this purpose.

Jimenez, Luis↗
DOE OSTI2024

Physics-coupled data-driven design of high-temperature alloys

We present a materials design loop, which streamlines physics-coupled machine learning (ML) surrogate models to discover new alloy chemistries with improved properties. The efficacy is demonstrated by discovering a high-temperature alumina-forming austenitic (AFA) stainless steel with enhanced creep, followed by experimental validation. The ML models have been trained using a well-curated, highly consistent experimental dataset augmented with synthetic microstructural features from a computational thermodynamic approach. We have populated a large number of hypothetical AFA alloys to explore the high-dimensional composition space and have predicted their creep properties by providing the same synthetic input features obtained from the trained ML models. Uncertainties from the ML training were taken as thresholds for truncating predicted results to identify alloys with improved or deteriorated creep. Individual elemental compositions have been determined via probability density distribution analysis from the group of alloys at the top and bottom of the predicted creep values for further virtual and experimental validations. In conclusion, we anticipate that this workflow can be applied to screen desired conditions, such as chemistry and processing parameters, in high-dimensional space through physics-guided data analytics.

Alloy design↗
NASA NTRS1997

High Performance Input/Output Systems for High Performance Computing and Four-Dimensional Data Assimilation

The approach of this task was to apply leading parallel computing research to a number of existing techniques for assimilation, and extract parameters indicating where and how input/output limits computational performance. The following was used for detailed knowledge of the application problems: 1. Developing a parallel input/output system specifically for this application 2. Extracting the important input/output characteristics of data assimilation problems; and 3. Building these characteristics s parameters into our runtime library (Fortran D/High Performance Fortran) for parallel input/output support.

Fox, Geoffrey C.↗
NASA NTRSDate not supplied

Onboard Hyperspectral Image Classification via Transfer Learning for Communication-Limited Spacecraft

Employing deep-learning and artificial-intelligence (AI) techniques onboard spacecraft can dramatically improve priority data selection to ensure more effective use of the available downlink. However, deployment of effective deep-learning models requires significant training on the ground, which may not be feasible, due to limited data available in an unexplored environment. Therefore, this research explores building robust classification models for onboard data processing where training data is highly limited using transfer-learning techniques. In this paper, we focus on the use case of hyperspectral imaging for remote sensing, a domain where the high dimensionality of the data from the sensor can rapidly saturate the downlink bandwidth. With this bottleneck, there is an impending need to autonomously and robustly classify data onboard to optimize downlink of high-impact measurements, thus maximizing the scientific utility per bit transmitted to the ground. This paper examines the use of deep neural networks onboard for hyperspectral image classification in a communication-limited scenario to analyze how the models perform with limited training data. The use of transfer learning can ameliorate the issue of poor generalization by transferring features learned from training on a large source dataset for one classification task to the target classification task with limited training data. For two deep-learning models from literature, we compare the accuracy of the models trained using transfer learning to models trained from scratch using a random weight initialization with varying amounts of training data. We demonstrate the feasibility and performance of running inference of the deep-learning models on representative flight-like hardware.

Advanced Avionics, Machine Learning, Data Processi↗
NASA NTRS2016

A Spatiotemporal Indexing Approach for Efficient Processing of Big Array-Based Climate Data with MapReduce

Climate observations and model simulations are producing vast amounts of array-based spatiotemporal data. Efficient processing of these data is essential for assessing global challenges such as climate change, natural disasters, and diseases. This is challenging not only because of the large data volume, but also because of the intrinsic high-dimensional nature of geoscience data. To tackle this challenge, we propose a spatiotemporal indexing approach to efficiently manage and process big climate data with MapReduce in a highly scalable environment. Using this approach, big climate data are directly stored in a Hadoop Distributed File System in its original, native file format. A spatiotemporal index is built to bridge the logical array-based data model and the physical data layout, which enables fast data retrieval when performing spatiotemporal queries. Based on the index, a data-partitioning algorithm is applied to enable MapReduce to achieve high data locality, as well as balancing the workload. The proposed indexing approach is evaluated using the National Aeronautics and Space Administration (NASA) Modern-Era Retrospective Analysis for Research and Applications (MERRA) climate reanalysis dataset. The experimental results show that the index can significantly accelerate querying and processing (10 speedup compared to the baseline test using the same computing cluster), while keeping the index-to-data ratio small (0.0328). The applicability of the indexing approach is demonstrated by a climate anomaly detection deployed on a NASA Hadoop cluster. This approach is also able to support efficient processing of general array-based spatiotemporal data in various geoscience domains without special configuration on a Hadoop cluster.

big data↗
NASA NTRS1987

Surface pressure measurements on the blade of an operating Mod-2 wind turbine with and without vortex generators

Pressure measurements covering a range of wind velocities were made at one span location on the surface of an operating Mod-2, 2500 kW, wind turbine blade. The data, which were taken with and without vortex generators installed on the leading edge, show the existence of higher pressure coefficients than would be expected from two-dimensional wind tunnel data. These high pressure ratios may be the result of three-dimensional flow over the blade, which delays flow separation. Data are presented showing the repetitiveness of abrupt changes in the pressure distribution that occur as the blade rotates. Calculated values of suction and flap coefficients are also presented.

Nyland, Ted W.↗
DOE OSTI2024

Data-Efficient Dimensionality Reduction and Surrogate Modeling of High-Dimensional Stress Fields

Tensor datatypes representing field variables like stress, displacement, velocity, etc., have increasingly become a common occurrence in data-driven modeling and analysis of simulations. Numerous methods [such as convolutional neural networks (CNNs)] exist to address the meta-modeling of field data from simulations. As the complexity of the simulation increases, so does the cost of acquisition, leading to limited data scenarios. Modeling of tensor datatypes under limited data scenarios remains a hindrance for engineering applications. Here, in this article, we introduce a direct image-to-image modeling framework of convolutional autoencoders enhanced by information bottleneck loss function to tackle the tensor data types with limited data. The information bottleneck method penalizes the nuisance information in the latent space while maximizing relevant information making it robust for limited data scenarios. The entire neural network framework is further combined with robust hyperparameter optimization. We perform numerical studies to compare the predictive performance of the proposed method with a dimensionality reduction-based surrogate modeling framework on a representative linear elastic ellipsoidal void problem with uniaxial loading. The data structure focuses on the low-data regime (fewer than 100 data points) and includes the parameterized geometry of the ellipsoidal void as the input and the predicted stress field as the output. The results of the numerical studies show that the information bottleneck approach yields improved overall accuracy and more precise prediction of the extremes of the stress field. Additionally, an in-depth analysis is carried out to elucidate the information compression behavior of the proposed framework.

artificial intelligence↗
DOE OSTI2024

Physics-Informed Active Learning With Simultaneous Weak-Form Latent Space Dynamics Identification

The parametric greedy latent space dynamics identification (gLaSDI) framework has demonstrated promising potential for accurate and efficient modeling of high-dimensional nonlinear physical systems. However, it remains challenging to handle noisy data. Here, to enhance robustness against noise, we incorporate the weak-form estimation of nonlinear dynamics (WENDy) into gLaSDI. In the proposed weak-form gLaSDI (WgLaSDI) framework, an autoencoder and WENDy are trained simultaneously to discover intrinsic nonlinear latent-space dynamics of high-dimensional data. Compared with the standard sparse identification of nonlinear dynamics (SINDy) employed in gLaSDI, WENDy enables variance reduction and robust latent space discovery, therefore leading to more accurate and efficient reduced-order modeling. Furthermore, the greedy physics-informed active learning in WgLaSDI enables adaptive sampling of optimal training data on the fly for enhanced modeling accuracy. The effectiveness of the proposed framework is demonstrated by modeling various nonlinear dynamical problems, including viscous and inviscid Burgers' equations, time-dependent radial advection, and the Vlasov equation for plasma physics. With data that contains 5%–10% Gaussian white noise, WgLaSDI outperforms gLaSDI by orders of magnitude, achieving 1%–7% relative errors. Compared with the high-fidelity models, WgLaSDI achieves 121 to 1779x speed-up.

97 MATHEMATICS AND COMPUTING↗