Search NASA⌕ Search

SEARCH · Search NASA

Results for “high dimensional data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

High-dimensional data analytics in civil engineering: A review on matrix and tensor decomposition

Recent developments in sensing and monitoring techniques have led to the generation of high-dimensional data in the field of civil engineering. High-dimensional data analytics methods have thus been developed to interpret such complex data. Among the different high-dimensional data analytics techniques, matrix and tensor decomposition methods have acquired a notable interest in the civil engineering community over the past decade. Due to their unique ability to deal with highly redundant and correlated data, these methods are establishing themselves as promising and efficient tools to analyze high-dimensional data in the civil engineering arena. In this paper, high-dimensional data is referred to as a data set in which the number of features is comparable or larger than the number of observations. This review paper aims to summarize the applications of matrix and tensor decomposition methods in civil engineering over the last decade. The survey begins with a general overview of matrix and tensor decomposition followed by highlighting their significance in the field. Afterward, various applications of these high-dimensional data analytics methods in civil engineering are presented, while the advantages offered by these methods are discussed. Lastly, challenges and potential research avenues for employing matrix and tensor decomposition and future emerging trends for their novel use are highlighted.

42 ENGINEERING↗
DOE OSTI2022

Grassmannian Diffusion Maps--Based Dimension Reduction and Classification for High-Dimensional Data

This work introduces the Grassmannian diffusion maps (GDMaps), a novel nonlinear dimensionality reduction technique that defines the affinity between points through their representation as low-dimensional subspaces corresponding to points on the Grassmann manifold. Here, the method is designed for applications, such as image recognition and data-based classification of constrained high-dimensional data where each data point itself is a high-dimensional object (i.e., a large matrix) that can be compactly represented in a lower-dimensional subspace. The GDMaps is composed of two stages. The first is a pointwise linear dimensionality reduction wherein each high-dimensional object is mapped onto the Grassmann manifold representing the low-dimensional subspace on which it resides. The second stage is a multipoint nonlinear kernel-based dimension reduction using diffusion maps to identify the subspace structure of the points on the Grassmann manifold. To this end, an appropriate Grassmannian kernel is used to construct the transition matrix of a random walk on a graph connecting points on the Grassmann manifold. Spectral analysis of the transition matrix yields low-dimensional Grassmannian diffusion coordinates embedding the data into a low-dimensional reproducing kernel Hilbert space. Further, a novel data classification/recognition technique is developed based on the construction of an overcomplete dictionary of reduced dimension whose atoms are given by the Grassmannian diffusion coordinates. Three examples are considered. First, a "toy" example shows that the GDMaps can identify an appropriate parametrization of structured points on the unit sphere. The second example demonstrates the ability of the GDMaps to revealing the intrinsic subspace structure of high-dimensional random field data. In the last ex- ample, a face recognition problem is solved considering face images subject to varying illumination conditions, changes in face expressions, and occurrence of occlusions. The technique presented high recognition rates (i.e., 95% in the best case) using a fraction of the data required by conventional methods.

42 ENGINEERING↗
DOE OSTI2021

An integrated manifold learning approach for high-dimensional data feature extractions and its applications to online process monitoring of additive manufacturing

As an effective dimension reduction and feature extraction technique, manifold learning has been successfully applied to high-dimensional data analysis. With the rapid development of sensor technology, a large amount of high-dimensional data such as image streams can be easily available. Thus, a promising application of manifold learning is in the field of sensor signal analysis, particular for the applications of online process monitoring and control using high-dimensional data. The objective of this study is to develop a manifold learning-based feature extraction method for process monitoring of Additive Manufacturing (AM) using online sensor data. Due to the non-parametric nature of most existing manifold learning methods, their performance in terms of computational efficiency, as well as noise resistance has yet to be improved. To address this issue, this study proposes an integrated manifold learning approach termed multi-kernel metric learning embedded isometric feature mapping (MKML-ISOMAP) for dimension reduction and feature extraction of online high-dimensional sensor data such as images. Based on the extracted features with the utilization of supervised classification and regression methods, an online process monitoring methodology for AM is implemented to identify the actual process quality status. Finally, in the numerical simulation and real-world case studies, the proposed method demonstrates excellent performance in both prediction accuracy and computational efficiency.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Robust and optimal alignment of high-dimensional data using maximum likelihood estimation through a random sample consensus framework

Abstract Correcting spatial orientations of groups of high-dimensional data sets such that they are all in a consistent coordinate system is often a time-consuming and error-prone process. Automation of this process can be accomplished by using Generalized Procrustes Analysis to estimate the relative orientations among a population of high-dimensional data sets. A least squares Procrustes solution is applied through a maximum likelihood estimation and random sample consensus framework for robustness. The likelihood model is comprised of a mixture distribution where inliers are modeled using t -distribution and outliers from a uniform distribution. Applications will focus on a synthetic data set that emulates triaxial acceleration data and also real shock data from a population of triaxial accelerometers. Outliers represent either non-rigid body responses, environmental noise, and/or sensor and data acquisition issues. The intended application for the methodology is to robustly automate the rotation of populations of experimentally collected triaxial accelerometer data sets to a single global coordinate system.

LOSAC↗
DOE OSTI2021

Machine learning of high dimensional data on a noisy quantum processor

Abstract Quantum kernel methods show promise for accelerating data analysis by efficiently learning relationships between input data points that have been encoded into an exponentially large Hilbert space. While this technique has been used successfully in small-scale experiments on synthetic datasets, the practical challenges of scaling to large circuits on noisy hardware have not been thoroughly addressed. Here, we present our findings from experimentally implementing a quantum kernel classifier on real high-dimensional data taken from the domain of cosmology using Google’s universal quantum processor, Sycamore. We construct a circuit ansatz that preserves kernel magnitudes that typically otherwise vanish due to an exponentially growing Hilbert space, and implement error mitigation specific to the task of computing quantum kernels on near-term hardware. Our experiment utilizes 17 qubits to classify uncompressed 67 dimensional data resulting in classification accuracy on a test set that is comparable to noiseless simulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Real-Time Health Monitoring for Gas Turbine Components Using Online Learning and High-Dimensional Data

Capital-intensive turbomachinery, such as gas turbines and combined cycle plants, are constantly being monitored for performance anomalies, faults, and physical degradation. Although these power-generating assets are equipped with hundreds of sensors, existing monitoring tools can only handle moderate-sized data. As a result, only a handful of aggregate metrics are used to monitor machine health. At the same time, developing advanced tools suitable for large datasets have been restricted by the lack of appropriate data. The objective of this proposal was to demonstrate a Big Data analytics framework for fault detection and diagnosis in gas turbine applications. We develop a predictive analytics framework methodology guided by these experimental data, industrial data from our collaborators, and physics-based models with engineering domain knowledge. Our analytics framework consists of four key components (1) a data curation process that addresses data storage, data quality assessments, and integrity checks, (2) a feature engineering component that utilizes statistical methods and transformation algorithms guided by physics-based models to extract high-fidelity fault features that can be leveraged for fault detection and classifying fault severities, (3) a Machine Learning-based fault detection and diagnostics algorithms for detecting operational and hardware faults in the combustion and the turbines section. We utilize two industry-class gas turbine component test rigs to generate first of its kind data for critical gas turbine faults with varying severity levels. Advanced gas turbine test facilities will be interrogated using state-of-the-art instrumentation techniques to build fault signatures and data trends for key combustor and turbine faults. Data generated from a combustor test rig (Georgia Tech) and a turbine test rig (Penn State) during both normal operation and with seeded faults serve as the basis for the Big Data sets. The test conditions in the two test facilities include common, critical events that occur in the operation. Utilizing the combustor test rig, we examine two common combustor faults: lean blowout and centerbody degradation. For the turbine section we develop analytic models for monitoring cooling faults in the gas turbine

03 NATURAL GAS↗
DOE OSTI2021

Real-Time Health Monitoring for Gas Turbine Components Using Online Learning and High-Dimensional Data (Final Report)

Capital-intensive turbomachinery, such as gas turbines and combined cycle plants, are constantly being monitored for performance anomalies, faults, and physical degradation. Although these power-generating assets are equipped with hundreds of sensors, existing monitoring tools can only handle moderate-sized data. As a result, only a handful of aggregate metrics are used to monitor machine health. At the same time, developing advanced tools suitable for large datasets have been restricted by the lack of appropriate data. The objective of this proposal was to demonstrate a Big Data analytics framework for fault detection and diagnosis in gas turbine applications. We develop a predictive analytics framework methodology guided by these experimental data, industrial data from our collaborators, and physics-based models with engineering domain knowledge. Our analytics framework consists of four key components: (1) a data curation process that addresses data storage, data quality assessments, and integrity checks, (2) a feature engineering component that utilizes statistical methods and transformation algorithms guided by physics-based models to extract high-fidelity fault features that can be leveraged for fault detection and classifying fault severities, (3) a Machine Learning-based fault detection and diagnostics algorithms for detecting operational and hardware faults in the combustion and the turbines section. We utilize two industry-class gas turbine component test rigs to generate first of its kind data for critical gas turbine faults with varying severity levels. Advanced gas turbine test facilities will be interrogated using state-of-the-art instrumentation techniques to build fault signatures and data trends for key combustor and turbine faults. Data generated from a combustor test rig (Georgia Tech) and a turbine test rig (Penn State) during both normal operation and with seeded faults serve as the basis for the Big Data sets. The test conditions in the two test facilities include common, critical events that occur in the operation. Utilizing the combustor test rig, we examine two common combustor faults: lean blowout and centerbody degradation. For the turbine section we develop analytic models for monitoring cooling faults in the gas turbine.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2025

MIC-DP: A Scalable Correlation-Aware Differential Privacy Framework for High-Dimensional Data

Conventional differential privacy (DP) assumes record independence, limiting effectiveness on real-world datasets with temporal, spatial, or structural correlations. These dependencies undermine privacy guarantees and degrade utility in domains like healthcare, IoT, and smart city analytics. We propose Maximum Information Correlated Differential Privacy (MIC-DP), a novel framework that dynamically calibrates noise based on statistical dependencies. MIC-DP uses the Maximum Information Coefficient (MIC) to capture both linear and nonlinear correlations without explicit modeling, enabling adaptive sensitivity adjustment and improved privacy–utility trade-offs. Evaluations on healthcare (MIMIC), demographic (ACI), and synthetic datasets show that MIC-DP reduces mean absolute error (MAE) by up to 5.2% under strict privacy budgets (ϵ≤1), with aggregate utility improvements reaching 18% across datasets and evaluation metrics. MIC-DP provides formal (ϵ,δ)-privacy guarantees, scales efficiently with feature count, and supports deployment in moderate-scale, privacy-sensitive applications. Its tunable performance and runtime efficiency make MIC-DP suitable for privacy-sensitive applications where low-latency analytics and strong privacy guarantees must coexist. These results demonstrate MIC-DP’s effectiveness as a correlation-aware solution for practical DP.

Yang, Wenjun [Univ. of Washington, Tacoma, WA (Uni↗
DOE OSTI2022

Data Mining and Visualization of High-Dimensional ICME Data for Additive Manufacturing

Integrated computational materials engineering (ICME) methods combining CALPHAD with process-based simulations can produce rich, high-dimensional data for alloy and process design. In ICME methods for metallurgical applications, the visualization and interpretation of such high-dimensional data has previously been through heat maps represented in 2 or 3 dimensions. While such an approach is ideal when one variable is varied at a time, in the case of high-dimensional data with multiple variables varied simultaneously, as is the case in additive manufacturing, interpreting the trends through two- or three-dimensional heat maps becomes challenging. Here, we propose a strategy of mixed visual data mining and quantitative analysis for high-dimensional metallurgical and process data using high-throughput thermodynamic calculations. Two case studies show the application of the proposed approach. The first case study investigated the effects of feedstock chemistry on the δ ferrite formation in 316L stainless steel powders used for binder jet additive manufacturing. The second case study linked Scheil–Gulliver calculations to a process model for dissimilar joining of aluminum alloys 5356 and 6111 during laser hot-wire additive manufacturing. Both cases contained thousands of calculated data points, showcasing the utility of visual data analysis through parallel coordinate plotting, Pearson correlation coefficient matrices, and scatter matrices compared to traditional process maps. These visualization techniques can be extended to many additive manufacturing problems to capture process–structure–property relationships for additively manufactured components.

36 MATERIALS SCIENCE↗
DOE OSTI2021

DNTTD (Distributed Non-Negative Tensor Train Decomposition)

The era of exascale computing opens new venues for innovations and discoveries in many scientific, engineering, and commercial fields. However, with the exa flops also come the extra-large high-dimensional data generated by high performance computing. High-dimensional data is presented as multidimensional arrays, aka tensors. The presence of latent (not directly observable) structures in the tensor allows a unique representation and compression of the data by classical tensor factorization techniques. However, the classical tensor methods are not always stable or they can be exponential in their memory requirements, which makes them not suitable for high-dimensional tensors. Tensor train (TT) is a state-of-the-art tensor network introduced for factorization of high-dimensional tensors. TT transforms the initial high-dimensional tensor in a network of three-dimensional tensors that requires only a linear storage. Many real-world data, such as, density, temperature, population, probability, etc., are non-negative and for an easy interpretation, the algorithms preserving non-negativity are preferred. Here, we introduce a distributed non-negative tensor-train and demonstrate its scalability and the compression on synthetic and real world big datasets.

Bhattarai, Manish↗
DOE OSTI2023

Clustering High-dimensional Toxicogenomics Data with Rare Signals

Toxicogenomics studies the gene and protein activities to drug treatments or toxic exposures. As the drugs and genes are numerous, toxicogenomics data are naturally high dimensional, with dimension sizes up to millions. In addition, the distribution of toxicogenomics data is oftentimes skewed, and they contain rare but important signals representing a cell or organism’s response to toxicity. The combination of high dimension and extremely skewed distribution of toxicogenomics data makes clustering analysis extremely challenging.We present our study of clustering toxicogenomics data using classical approaches such as principal component analysis as well as deep learning approaches such as auto-encoders. Our experiments show that these approaches fail to preserve rare signals and produce high-quality clusters. We then explore augmenting matrix factorization with deep learning techniques such as attention mechanism to produce latent representations for clustering. Our technique is able to better preserve rare signals after dimensionality reduction than prior approaches. Furthermore, we combine our augmented matrix factorization with a mechanism similar to autoencoder to balance separable clusters and low regeneration errors. Our experiments demonstrate better clustering with our proposed approach.

Cong, Guojing↗
DOE OSTI2025

Visualization of Noisy and Less Noisy Computational Basis States in Quantum Computing

Quantum computing technology holds substantial promise as a reliable computational paradigm. However, current noisy intermediate scale quantum (NISQ) systems, are significantly impacted by noise originating from hardware inconsistencies. This noise causes errors and lowers output fidelity. So we must find which basis states cause errors. However, there are two main challenges in analyzing noise corresponding to basis states. First, the noise distribution data is high dimensional in nature, thereby making its analysis challenging. Second, although functional box plots have been used in the state of the art research to understand such a high dimensional data, they suffer from clutter and occlusion issues because of overplotting. In this study, we introduce an innovative visualization pipeline to address the aforementioned challenges to provide a clear depiction of noisy and less-noisy basis states. Specifically, our proposed visualization pipeline comprises three stages namely, low dimensional embedding, clustering, and violin plot visualization, to reduce visual clutter and effectively analyze high-dimensional noise distribution data. Our analysis uses quantum machine learning (QML) circuits as case study for drawing a distinction between noisy and less noisy basis states.

Senapati, Priyabrata [Kent State University]↗
DOE OSTI2021

Data-driven building energy modeling with feature selection and active learning for data predictive control

Three gaps impede the development of cost-effective and accurate data-driven building energy modeling/models (DBEM) for energy forecasting and predictive control strategies. Gap 1: data bias is common in building operation data, but this topic is hardly studied in DBEM; Gap 2: high data dimensionality is common in DBEM, but a systematic and scalable methodology is lacking to solve the problem; Gap 3: the interactions between data bias and high dimensionality have not been systematically studied for DBEM and predictive control in buildings. In this work, to address the three gaps mentioned above, we develop a framework that integrates active learning and feature selection for DBEM used for whole building data predictive control (or DPC, which is a branch of model predictive control). The framework provides a systematic methodology and automatic workflow that starts with raw data from building automation systems to the establishment of data-driven energy models for DPC controllers. The developed strategies and framework are evaluated in a virtual testbed based on EnergyPlus and BCVTB. Improved performance and reduced computational complexity are observed from the DBEM built with the developed framework, as well as the DPC controller based on that DBEM, indicating the effectiveness of the developed framework.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data With Spatial Information

Efficient processing of large-scale time series data is an intricate problem in machine learning. Conventional sensor signal processing pipelines with hand engineered feature extraction often involve huge computational cost with high dimensional data. Deep recurrent neural networks have shown promise in automated feature learning for improved time-series processing. However, generic deep recurrent models grow in scale and depth with increased complexity of the data. This is particularly challenging in presence of high dimensional data with temporal and spatial characteristics. Consequently, this work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to efficiently process complex multi-dimensional time series data with spatial information. Here, the cellular recurrent architecture in the proposed model allows for location-aware synchronous processing of time series data from spatially distributed sensor signal sources. Extensive trainable parameter sharing due to cellularity in the proposed architecture ensures efficiency in the use of recurrent processing units with high-dimensional inputs. This study also investigates the versatility of the proposed DCRNN model for classification of multi-class time series data from different application domains. Consequently, the proposed DCRNN architecture is evaluated using two time-series datasets: a multichannel scalp EEG dataset for seizure detection, and a machine fault detection dataset obtained in-house. The results suggest that the proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Real-time Data Analytics for Condition Monitoring of Complex Industrial Systems

Modern industrial systems are now fitted with several sensors for condition monitoring. This is advantageous because these sensors can provide mass amounts of data that have the potential for aiding in tasks such as fault detection, diagnosis, and prognostics. However, the information valuable for performing these tasks is often clouded in noise and must be mined from high-dimensional data structures. Therefore, this dissertation presents a data analytics framework for performing these condition monitoring tasks using high-dimensional data. Demonstrations of this framework are detailed for challenges related to power generation systems in automobiles, power plants, and aircraft engines. These implementations leverage data collected from state-of-the-art, industry class test-rigs. Results indicate the ability of this framework to develop effective methodologies for condition monitoring of complex systems.

Peters, Benjamin↗