SEARCH · Search NASA

Results for “data representations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Unsupervised Machine Learning for Exploratory Data Analysis of Exoplanet Transmission Spectra

Abstract Transit spectroscopy is a powerful tool for decoding the chemical compositions of the atmospheres of extrasolar planets. In this paper, we focus on unsupervised techniques for analyzing spectral data from transiting exoplanets. After cleaning and validating the data, we demonstrate methods for: (i) initial exploratory data analysis, based on summary statistics (estimates of location and variability); (ii) exploring and quantifying the existing correlations in the data; (iii) preprocessing and linearly transforming the data to its principal components; (iv) dimensionality reduction and manifold learning; (v) clustering and anomaly detection; and (vi) visualization and interpretation of the data. To illustrate the proposed unsupervised methodology, we use a well-known public benchmark data set of synthetic transit spectra. We show that there is a high degree of correlation in the spectral data, which calls for appropriate low-dimensional representations. We explore a number of different techniques for such dimensionality reduction and identify several suitable options in terms of summary statistics, principal components, etc. We uncover interesting structures in the principal component basis, namely well-defined branches corresponding to different chemical regimes of the underlying atmospheres. We demonstrate that those branches can be successfully recovered with a K-means clustering algorithm in a fully unsupervised fashion. We advocate for lower-dimensional representations of the spectroscopic data in terms of the main principal components, in order to reveal the existing structure in the data and quickly characterize the chemical class of a planet.

Matchev, Konstantin T. (ORCID:0000000341829096)↗

NASA NTRS1980

An Economical Semi-Analytical Orbit Theory for Retarded Satellite Motion About an Oblate Planet

Brouwer and Brouwer-Lyddanes' use of the Von Zeipel-Delaunay method is employed to develop an efficient analytical orbit theory suitable for microcomputers. A succinctly simple pseudo-phenomenologically conceptualized algorithm is introduced which accurately and economically synthesizes modeling of drag effects. The method epitomizes and manifests effortless efficient computer mechanization. Simulated trajectory data is employed to illustrate the theory's ability to accurately accommodate oblateness and drag effects for microcomputer ground based or onboard predicted orbital representation. Real tracking data is used to demonstrate that the theory's orbit determination and orbit prediction capabilities are favorably adaptable to and are comparable with results obtained utilizing complex definitive Cowell method solutions on satellites experiencing significant drag effects.

Gordon, R. A.↗

NASA NTRS1998

Empirical Mode Decomposition and Hilbert Spectral Analysis

The difficult facing data analysis is the lack of method to handle nonlinear and nonstationary time series. Traditional Fourier-based analyses simply could not be applied here. A new method for analyzing nonlinear and nonstationary data has been developed. The key part is the Empirical Mode Decomposition (EMD) method with which any complicated data set can be decomposed into a finite and often small number of Intrinsic Mode Functions (IMF) that serve as the basis of the representation of the data. This decomposition method is adaptive, and, therefore, highly efficient. The IMFs admit well-behaved Hilbert transforms, and yield instantaneous energy and frequency as functions of time that give sharp identifications of imbedded structures. The final presentation of the results is an energy-frequency-time distribution, designated as the Hilbert Spectrum. Among the main conceptual innovations is the introduction of the instantaneous frequencies for complicated data sets, which eliminate the need of spurious harmonics to represent nonlinear and nonstationary signals. Examples from the numerical results of the classical nonlinear equation systems and data representing natural phenomena are given to demonstrate the power of this new method. The classical nonlinear system data are especially interesting, for they serve to illustrate the roles played by the nonlinear and nonstationary effects in the energy-frequency-time distribution.

Huang, Norden E.↗

NASA NTRS1999

A Critical Examination of Wind-Wave Spectral Functional Form

Traditionally, data from random ocean waves are presented in spectral functions. The spectra are the result of Fourier analysis. Fourier spectral analysis has dominated data analysis for, at least, the last hundred years. It has been the standard method for is examining the global amplitude-frequency distributions. Although Fourier transform valid under extremely general conditions, there are some crucial restrictions for the Fourier spectral analysis. The system must be linear, and the data must be stationary- otherwise, the resulting spectrum will make little physical sense. The stationarity requirement is also a common required criterion for most of other available data analysis methods. Nevertheless, few, if any, natural phenomena are linear and stationary. To compound these complications is the imperfection of our probes or numerical schemes the interactions of the imperfect probes even with a perfect linear system can make the final data nonlinear. Furthermore, all the available data are usually of finite duration. Under these conditions, Fourier analysis is of limited use, For lack of alternatives, however, Fourier analysis is still used to process such data. The loose application of Fourier analysis and the insouciant adoption of the stationary and linear assumptions may lead to misleading conclusions. Ocean waves are know to be nonlinear, and the wind system generating the wave field are seldom stationary- As a result, the traditional examination of the spectral form hardly made physical sense. A new method for analyzing nonlinear and nonstationary data has been developed. The key part is the Empirical Mode Decomposition (EMD) method with which any complicated data set can be decomposed into a finite and often small number of Intrinsic Mode Functions (IMF) that serve as the basis of the representation of the data, This decomposition method is adaptive, and, therefore, highly efficient. The IMFs admit well-behaved Hilbert transforms, and yield instantaneous energy and frequency as functions of time that give sharp identifications of imbedded structures. The final presentation of the results is an energy-frequency-time distribution, designated as the Hilbert Spectrum. Among the main conceptual innovations is the introduction of the instantaneous frequencies for complicated data sets, which eliminate the need of spurious harmonics to represent nonlinear and nonstationary signals. The spectral forms of the ocean waves are very different. This new method offers the first physical meaningful view of the wave spectrum. Data from laboratory and field will be presented to illustrate the differences.

Huang, Norden E.↗

NASA NTRS1975

Engineering studies of vectorcardiographs in blood pressure measuring systems, appendix 3

The following subjects were covered: (1) ASM80 manual, (2) signal preprocessing as an aid to on-line EKG analysis, and (3) high speed evaluation of magnetic tape recordings of electrocardiograms. A description of the ASM80 symbolic assembly program for the INTEL 8080 microprocessor and a user's manual were presented. The capability of three redundancy reduction algorithms to produce adequate representations of electrocardiographic data was examined. A hardware device was constructed which carried out zero order interpolation on a signal. Examination of the zero order interpolators reconstructed signal indicated that this representation was adequate for analysis of rhythm. A system to analyze magnetic tapes of electrocardiograms recorded over 24 hour intervals was designed. The recordings are sampled 200 times per second using a Nova computer and a special interface system. This system was tested on several recordings of clinical data, containing over 75 premature ventricular contractions, each one of which was flagged.

Mark, R. G.↗

DOE OSTI2015

Coupling extended magnetohydrodynamic fluid codes with radiofrequency ray tracing codes for fusion modeling

Neoclassical tearing modes are macroscopic ($L$ ~ 1 m) instabilities in magnetic fusion experiments; if unchecked, these modes degrade plasma performance and may catastrophically destroy plasma confinement by inducing a disruption. Fortunately, the use of properly tuned and directed radiofrequency waves ($λ$ ~ 1 mm) can eliminate these modes. Numerical modeling of this difficult multiscale problem requires the integration of separate mathematical models for each length and time scale [T. G. Jenkins, S. E. Kruger, Phys. Plasmas 19 (2012) 122508]; the extended MHD model captures macroscopic plasma evolution while the RF model tracks the flow and deposition of injected RF power through the evolving plasma profiles. The scale separation enables use of the eikonal (ray-tracing) approximation to model the RF wave propagation. In this work we demonstrate a technique, based on methods of computational geometry, for mapping the ensuing RF data (associated with discrete ray trajectories) onto the finite-element/pseudospectral grid that is used to model the extended MHD physics. In the new representation, the RF data can then be used to construct source terms in the equations of the extended MHD model, enabling quantitative modeling of RF-induced tearing mode stabilization. Though our specific implementation uses the NIMROD extended MHD [C. R. Sovinec et al., J. Comp. Phys. 195 (2004) 355] and GENRAY RF [A. P. Smirnov, R. W. Harvey, K. Kupfer, Bull. Amer. Phys. Soc. 39 (1994) 1626] codes, the approach presented can be applied more generally to any code coupling requiring the mapping of ray tracing data onto Eulerian grids.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

NASA NTRS1988

Digital and optical shape representation and pattern recognition; Proceedings of the Meeting, Orlando, FL, Apr. 4-6, 1988

The present conference discusses topics in pattern-recognition correlator architectures, digital stereo systems, geometric image transformations and their applications, topics in pattern recognition, filter algorithms, object detection and classification, shape representation techniques, and model-based object recognition methods. Attention is given to edge-enhancement preprocessing using liquid crystal TVs, massively-parallel optical data base management, three-dimensional sensing with polar exponential sensor arrays, the optical processing of imaging spectrometer data, hybrid associative memories and metric data models, the representation of shape primitives in neural networks, and the Monte Carlo estimation of moment invariants for pattern recognition.

Juday, Richard D.↗

DOE OSTI2018

Change detection and change monitoring of natural and man-made features in multispectral and hyperspectral satellite imagery

An approach for land cover classification, seasonal and yearly change detection and monitoring, and identification of changes in man-made features may use a clustering of sparse approximations (CoSA) on sparse representations in learned dictionaries. A Hebbian learning rule may be used to build multispectral or hyperspectral, multiresolution dictionaries that are adapted to regional satellite image data. Sparse image representations of pixel patches over the learned dictionaries may be used to perform unsupervised k-means clustering into land cover categories. The clustering process behaves as a classifier in detecting real variability. This approach may combine spectral and spatial textural characteristics to detect geologic, vegetative, hydrologic, and man-made features, as well as changes in these features over time.

47 OTHER INSTRUMENTATION↗

DOE OSTI2021

Outlier classification using autoencoders: application for fluctuation driven flows in fusion plasmas

Understanding the statistics of fluctuation driven flows in the boundary layer of magnetically confined plasmas is desired to accurately model the lifetime of the vacuum vessel components. Mirror Langmuir probes (MLPs) are a novel diagnostic that uniquely allow us to sample the plasma parameters on a time scale shorter than the characteristic time scale of their fluctuations. Sudden large-amplitude fluctuations in the plasma degrade the precision and accuracy of the plasma parameters reported by MLPs for cases in which the probe bias range is of insufficient amplitude. While some data samples can readily be classified as valid and invalid, we find that such a classification may be ambiguous for up to 40% of data sampled for the plasma parameters and bias voltages considered in this study. In this contribution, we employ an autoencoder (AE) to learn a low-dimensional representation of valid data samples. By definition, the coordinates in this space are the features that mostly characterize valid data. Ambiguous data samples are classified in this space using standard classifiers for vectorial data. In this way, we avoid defining complicated threshold rules to identify outliers, which require strong assumptions and introduce biases in the analysis. By removing the outliers that are identified in the latent low-dimensional space of the AE, we find that the average conductive and convective radial heat fluxes are between approximately 5% and 15% lower as when removing outliers identified by threshold values. For contributions to the radial heat flux due to triple correlations, the difference is up to 40%.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

NASA NTRS1992

Some practicable applications of quadtree data structures/representation in astronomy

Development of quadtree as hierarchical data structuring technique for representing spatial data (like points, regions, surfaces, lines, curves, volumes, etc.) has been motivated to a large extent by storage requirements of images, maps, and other multidimensional (spatially structured) data. For many spatial algorithms, time-efficiency of quadtrees in terms of execution may be as important as their space-efficiency concerning storage conditions. Briefly, the quadtree is a class of hierarchical data structures which is based on the recursive partition of a square region into quadrants and sub-quadrants until a predefined limit. Beyond the wide applicability of quadtrees in image processing, spatial information analysis, and building digital databases (processes becoming ordinary for the astronomical community), there may be numerous further applications in astronomy. Some of these practicable applications based on quadtree representation of astronomical data are presented and suggested for further considerations. Examples are shown for use of point as well as region quadtrees. Statistics of different leaf and non-leaf nodes (homogeneous and heterogeneous sub-quadrants respectively) at different levels may provide useful information on spatial structure of astronomical data in question. By altering the principle guiding the decomposition process, different types of spatial data may be focused on. Finally, a sampling method based on quadtree representation of an image is proposed which may prove to be efficient in the elaboration of sampling strategy in a region where observations were carried out previously either with different resolution or/and in different bands.

Pasztor, L.↗

NASA NTRS1997

Wavelet Analyses of F/A-18 Aeroelastic and Aeroservoelastic Flight Test Data

Time-frequency signal representations combined with subspace identification methods were used to analyze aeroelastic flight data from the F/A-18 Systems Research Aircraft (SRA) and aeroservoelastic data from the F/A-18 High Alpha Research Vehicle (HARV). The F/A-18 SRA data were produced from a wingtip excitation system that generated linear frequency chirps and logarithmic sweeps. HARV data were acquired from digital Schroeder-phased and sinc pulse excitation signals to actuator commands. Nondilated continuous Morlet wavelets implemented as a filter bank were chosen for the time-frequency analysis to eliminate phase distortion as it occurs with sliding window discrete Fourier transform techniques. Wavelet coefficients were filtered to reduce effects of noise and nonlinear distortions identically in all inputs and outputs. Cleaned reconstructed time domain signals were used to compute improved transfer functions. Time and frequency domain subspace identification methods were applied to enhanced reconstructed time domain data and improved transfer functions, respectively. Time domain subspace performed poorly, even with the enhanced data, compared with frequency domain techniques. A frequency domain subspace method is shown to produce better results with the data processed using the Morlet time-frequency technique.

Martin J Brenner↗

DOE OSTI2018

Classification of multispectral or hyperspectral satellite imagery using clustering of sparse approximations on sparse representations in learned dictionaries obtained using efficient convolutional sparse coding

An approach for land cover classification, seasonal and yearly change detection and monitoring, and identification of changes in man-made features may use a clustering of sparse approximations (CoSA) on sparse representations in learned dictionaries. The learned dictionaries may be derived using efficient convolutional sparse coding to build multispectral or hyperspectral, multiresolution dictionaries that are adapted to regional satellite image data. Sparse image representations of images over the learned dictionaries may be used to perform unsupervised k-means clustering into land cover categories. The clustering process behaves as a classifier in detecting real variability. This approach may combine spectral and spatial textural characteristics to detect geologic, vegetative, hydrologic, and man-made features, as well as changes in these features over time.

47 OTHER INSTRUMENTATION↗

NASA NTRSDate not supplied

Improving the Representation of Land Surface Processes using the Data Assimilation Research Testbed (DART)

The land surface is a critical part of the earth system as processes related to water, carbon, energy and nitrogen cycling have important implications for climate forcing, air quality, water availability and seasonal atmospheric forecasting. Despite advances in land surface modeling, land surface model performance is often limited because of errors related to initial and boundary conditions, model structure, and parameters. Data assimilation (DA) techniques combined with an expanding network of earth system observations present an opportunity to reduce these errors and improve simulations. Here we apply an Ensemble Kalman Filter DA system as part of the Data Assimilation Research Testbed to a variety of land surface simulations. First, we describe the use of remotely sensed biomass observations to provide improved simulations of plant phenology, carbon and water cycling for regions highly sensitive to climate change (Western US, China, and Arctic). We discuss approaches to account for systemic biases between models and observations, including the use of spatially-varying adaptive ensemble inflation as an alternative approach to re-scaling soil moisture observations. Finally, we discuss a strategy to incorporate complementary observations (snow water equivalent, solar-induced fluorescence) to better constrain the representation of carbon and water cycling across complex terrain.

DART↗

DOE OSTI2024

Refining Planetary Boundary Layer Height Retrievals From Micropulse‐Lidar at Multiple ARM Sites Around the World

Abstract Knowledge of the planetary boundary layer height (PBLH) is crucial for various applications in atmospheric and environmental sciences. Lidar measurements are frequently used to monitor the evolution of the PBLH, providing more frequent observations than traditional radiosonde‐based methods. However, lidar‐derived PBLH estimates have substantial uncertainties, contingent upon the retrieval algorithm used. In addressing this, we applied the Different Thermo‐Dynamic Stabilities (DTDS) algorithm to establish a PBLH data set at five separate Department of Energy's Atmospheric Radiation Measurement sites across the globe. Both the PBLH methodology and the products are subject to rigorous assessments in terms of their uncertainties and constraints, juxtaposing them with other products. The DTDS‐derived product consistently aligns with radiosonde PBLH estimates, with correlation coefficients exceeding 0.77 across all sites. This study delves into a detailed examination of the strengths and limitations of PBLH data sets with respect to both radiosonde‐derived and other lidar‐based estimates of the PBLH by exploring their respective errors and uncertainties. It is found that varying techniques and definitions can lead to diverse PBLH retrievals due to the inherent intricacy and variability of the boundary layer. Our DTDS‐derived PBLH data set outperforms existing products derived from ceilometer data, offering a more precise representation of the PBLH. This extensive data set paves the way for advanced studies and an improved understanding of boundary‐layer dynamics, with valuable applications in weather forecasting, climate modeling, and environmental studies.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2019

Outlier classification using autoencoders: Application for fluctuation driven flows in fusion plasmas

Understanding the statistics of fluctuation driven flows in the boundary layer of magnetically confined plasmas is desired to accurately model the lifetime of the vacuum vessel components. Mirror Langmuir probes (MLPs) are a novel diagnostic that uniquely allow us to sample the plasma parameters on a time scale shorter than the characteristic time scale of their fluctuations. Sudden large-amplitude fluctuations in the plasma degrade the precision and accuracy of the plasma parameters reported by MLPs for cases in which the probe bias range is of insufficient amplitude. While some data samples can readily be classified as valid and invalid, we find that such a classification may be ambiguous for up to 40% of data sampled for the plasma parameters and bias voltages considered in this study. In this contribution to this work, we employ an autoencoder (AE) to learn a low-dimensional representation of valid data samples. By definition, the coordinates in this space are the features that mostly characterize valid data. Ambiguous data samples are classified in this space using standard classifiers for vectorial data. In this way, we avoid defining complicated threshold rules to identify outliers, which require strong assumptions and introduce biases in the analysis. By removing the outliers that are identified in the latent low-dimensional space of the AE, we find that the average conductive and convective radial heat fluxes are between approximately 5% and 15% lower as when removing outliers identified by threshold values. For contributions to the radial heat flux due to triple correlations, the difference is up to 40%

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2025

powersqueeze

powersqueeze (psqz) is a truncated power iteration library intended for high-performance computing platforms. psqz efficiently produces low-dimensional, linear measurements of graph matrix spectra by combining classical power iteration with sparse Johnson-Lindenstrauss transforms. psqz is intended to produce high-quality, fast, data-oblivious low-dimensional representations of high-dimensional sparse data such as graphs and term-document matrices. psqz is intended to replace similar workflows that depend on directly approximating a truncated eigendecomposition (e.g., the first step of spectral clustering), which is a much more expensive operation.

Priest, BenjaminW [Lawrence Livermore National Lab↗

DOE OSTI2016

Predictive Big Data Analytics: A Study of Parkinson’s Disease Using Large, Complex, Heterogeneous, Incongruent, Multi-Source and Incomplete Observations

Background A unique archive of Big Data on Parkinson’s Disease is collected, managed and disseminated by the Parkinson’s Progression Markers Initiative (PPMI). The integration of such complex and heterogeneous Big Data from multiple sources offers unparalleled opportunities to study the early stages of prevalent neurodegenerative processes, track their progression and quickly identify the efficacies of alternative treatments. Many previous human and animal studies have examined the relationship of Parkinson’s disease (PD) risk to trauma, genetics, environment, co-morbidities, or life style. The defining characteristics of Big Data–large size, incongruency, incompleteness, complexity, multiplicity of scales, and heterogeneity of information-generating sources–all pose challenges to the classical techniques for data management, processing, visualization and interpretation. We propose, implement, test and validate complementary model-based and model-free approaches for PD classification and prediction. To explore PD risk using Big Data methodology, we jointly processed complex PPMI imaging, genetics, clinical and demographic data. Methods and Findings Collective representation of the multi-source data facilitates the aggregation and harmonization of complex data elements. This enables joint modeling of the complete data, leading to the development of Big Data analytics, predictive synthesis, and statistical validation. Using heterogeneous PPMI data, we developed a comprehensive protocol for end-to-end data characterization, manipulation, processing, cleaning, analysis and validation. Specifically, we (i) introduce methods for rebalancing imbalanced cohorts, (ii) utilize a wide spectrum of classification methods to generate consistent and powerful phenotypic predictions, and (iii) generate reproducible machine-learning based classification that enables the reporting of model parameters and diagnostic forecasting based on new data. We evaluated several complementary model-based predictive approaches, which failed to generate accurate and reliable diagnostic predictions. However, the results of several machine-learning based classification methods indicated significant power to predict Parkinson’s disease in the PPMI subjects (consistent accuracy, sensitivity, and specificity exceeding 96%, confirmed using statistical n-fold cross-validation). Clinical (e.g., Unified Parkinson's Disease Rating Scale (UPDRS) scores), demographic (e.g., age), genetics (e.g., rs34637584, chr12), and derived neuroimaging biomarker (e.g., cerebellum shape index) data all contributed to the predictive analytics and diagnostic forecasting. Conclusions Model-free Big Data machine learning-based classification methods (e.g., adaptive boosting, support vector machines) can outperform model-based techniques in terms of predictive precision and reliability (e.g., forecasting patient diagnosis). We observed that statistical rebalancing of cohort sizes yields better discrimination of group differences, specifically for predictive analytics based on heterogeneous and incomplete PPMI data. UPDRS scores play a critical role in predicting diagnosis, which is expected based on the clinical definition of Parkinson’s disease. Even without longitudinal UPDRS data, however, the accuracy of model-free machine learning based classification is over 80%. The methods, software and protocols developed here are openly shared and can be employed to study other neurodegenerative disorders (e.g., Alzheimer’s, Huntington’s, amyotrophic lateral sclerosis), as well as for other predictive Big Data analytics applications.

Science & Technology - Other Topics↗

DOE OSTI2023

MLMOD: Machine Learning Methods for Data-Driven Modeling in LAMMPS

MLMOD is a software package for incorporating machine learning approaches and models into simulations of microscale mechanics and molecular dynamics in LAMMPS. Recent machine learning approaches provide promising data-driven approaches for learning representations for system behaviors from experimental data and high fidelity simulations. The package facilitates learning and using data-driven models for (i) dynamics of the system at larger spatial-temporal scales (ii) interactions between system components, (iii) features yielding coarser degrees of freedom, and (iv) features for new quantities of interest characterizing system behaviors. MLMOD provides hooks in LAMMPS for (i) modeling dynamics and time-step integration, (ii) modeling interactions, and (iii) computing quantities of interest characterizing system states. The package allows for use of machine learning methods with general model classes including Neural Networks, Gaussian Process Regression, Kernel Models, and other approaches. Here we discuss our prototype C++/Python package, aims, and example usage. For related papers, examples, updates, and additional information see https://github.com/atzberg/mlmod and http://atzberger.org/.

97 MATHEMATICS AND COMPUTING↗