Search NASA⌕ Search

SEARCH · Search NASA

Results for “high dimensional data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
NASA NTRS1990

Experimental evidence of a dimensional crossover in Y1Ba2Cu3O(7-delta)

High-resolution torque-magnetometry data have been obtained on an untwinned single crystal of Y1Ba2Cu3O(7-delta) in the temperature range 63-88 K (Tc = 90.5 K). At T = 80 K and above, the data are fitted extremely well with the accepted three-dimensional phenomenological theory, but below this temperature an anomalous torque develops when the magnetic field lies close to the Cu-O planes. The qualitative and quantitative features of these results provide strong evidence that breakdown of the three-dimensional description below 80 K is associated with a crossover to two-dimensional superconducting behavior.

Farrell, D. E.↗
NASA NTRS1988

Image processing software for imaging spectrometry

The paper presents a software system, Spectral Analysis Manager (SPAM), which has been specifically designed and implemented to provide the exploratory analysis tools necessary for imaging spectrometer data, using only modest computational resources. The basic design objectives are described as well as the major algorithms designed or adapted for high-dimensional images. Included in a discussion of system implementation are interactive data display, statistical analysis, image segmentation and spectral matching, and mixture analysis.

Mazer, Alan S.↗
DOE OSTI2025

Landscaper v1

Understanding the inner workings of machine learning models through their loss landscapes offers crucial insights into model properties, optimization dynamics, and generalizability. However, accessing these insights has traditionally required specialized mathematical expertise, limiting broader adoption. Landscaper is an open-source Python package designed to bridge this gap. Landscaper seamlessly integrates a suite of multi-dimensional loss landscape analyses with cutting-edge topological data analysis (TDA) methods. This powerful combination makes both fundamental loss landscape analysis and advanced TDA techniques accessible to the broader scientific ML community, without requiring deep pre-existing mathematical knowledge. Landscaper offers three key functionalities: * Construction: Builds detailed loss landscape representations through versatile low and high-dimensional sampling techniques. * Quantification: Applies advanced metrics, including a novel topological data analysis (TDA) based smoothness metric, enabling new perspectives on model behavior. * Visualization: Offers intuitive tools to visualize and interpret loss landscapes, providing actionable insights beyond traditional performance metrics.

Weber, Gunther [Lawrence Berkeley National Laborat↗
DOE OSTI2025

CLPNets: Coupled Lie–Poisson neural networks for multi-part Hamiltonian systems with symmetries

To accurately compute data-based prediction of Hamiltonian systems, it is essential to utilize methods that preserve the structure of the equations over time. We consider a particularly challenging case of systems with interacting parts that do not reduce to pure momentum evolution. Such systems are essential in scientific computations, such as discretization of a continuum elastic rod, which can be viewed as the group of rotations and translations $SE(3)$. The evolution involves not only the momenta but also the relative positions and orientations of the particles. The presence of Lie group-valued elements, such as relative positions and orientations, poses a problem for applying previously derived methods for data-based computing. We develop a novel method of data-based computation and complete phase space learning of such systems. We follow the original framework of SympNets (Jin et al., 2020) and LPNets (Eldred et al., 2024), building the neural network from phase space mappings that preserve the Lie–Poisson structure. We derive a novel system of mappings that are built into neural networks describing the evolution of such systems. We call such networks Coupled Lie–Poisson Neural Networks, or CLPNets. We consider increasingly complex examples for the applications of CLPNets, starting with the rotation of two rigid bodies about a common axis, progressing to the free rotation of two rigid bodies, and finally to the evolution of two connected and interacting $SE(3)$ components, describing the discretization of an elastic rod into two elements. Our method preserves all Casimir invariants to machine precision, preserves energy to high accuracy, and shows good resistance to the curse of dimensionality, requiring only a few thousand data points for all cases studied (three to eighteen dimensions). Additionally, the method is highly economical in memory requirements, requiring only about 200 parameters for the most complex case considered.

Data-based modeling↗
DOE OSTI2025

Linear Discriminant Analysis-Based Machine Learning and All-Atom Molecular Dynamics Simulations for Probing Electro-Osmotic Transport in Cationic-Polyelectrolyte-Brush-Grafted Nanochannels

Deciphering the correct mechanisms governing certain phenomena in polyelectrolyte (PE) brush grafted systems, revealed through atomistic simulations, is an extremely challenging problem. In a recent study, our all-atom molecular dynamics (MD) simulations revealed a non-linearly large electroosmotic (EOS) flow (in the presence of an applied electric field) in nanochannels grafted with PMETAC [Poly(2-(methacryloyloxy)ethyl trimethylammonium chloride] brushes. Given the lack of any formal procedure that would have directed us to identify the correct factors responsible for such an occurrence, we needed to spend several months and devote significant analyses to unravel the involved mechanisms. In this paper, we propose a Linear Discriminant Analysis (LDA) based Machine Learning (ML) approach to address this gap. At first, we obtain data on certain basic features from the all-atom MD data. These basic features represent the number of atoms of certain species around one atom of another (or same) species. Here, we obtain such data on basic features for a reference case (case of an EOS flow in PMETAC-brush-grafted nanochannels with a smaller electric field) and a perturbed case (case of an EOS flow in PMETAC-brush-grafted nanochannels with a larger electric field) in bins in which the nanochannel half height has been divided into. These datasets are high-dimensional dataset, to which the LDA is applied. This leads to the projection of the data (between the reference and the perturbed states) in a highly separated form on a 1D line. From such LDA calculations, we are able to identify the relative importance of the different basic features in ensuring this separation of the data (between the reference and the perturbed states) on the 1D line. This relative importance of the different basic features is quantified as “importance scores” for the different features, which in turn tell us what to study and where to study. Such knowledge enables us to rapidly identify the key factors responsible for the non-linearly large EOS transport in PMETAC-brush-grafted nanochannels.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

An implementation of neural simulation-based inference for parameter estimation in ATLAS

Neural simulation-based inference (NSBI) is a powerful class of machine-learning-based methods for statistical inference that naturally handles high-dimensional parameter estimation without the need to bin data into low-dimensional summary histograms. Such methods are promising for a range of measurements, including at the Large Hadron Collider, where no single observable may be optimal to scan over the entire theoretical phase space under consideration, or where binning data into histograms could result in a loss of sensitivity. This work develops a NSBI framework for statistical inference, using neural networks to estimate probability density ratios, which enables the application to a full-scale analysis. It incorporates a large number of systematic uncertainties, quantifies the uncertainty due to the finite number of events in training samples, develops a method to construct confidence intervals, and demonstrates a series of intermediate diagnostic checks that can be performed to validate the robustness of the method. As an example, the power and feasibility of the method are assessed on simulated data for a simplified version of an off-shell Higgs boson couplings measurement in the four-lepton final states. This approach represents an extension to the standard statistical methodology used by the experiments at the Large Hadron Collider, and can benefit many physics analyses.

frequentist statistics↗
NASA NTRS1991

A method for classification of multisource data using interval-valued probabilities and its application to HIRIS data

A method of classifying multisource data in remote sensing is presented. The proposed method considers each data source as an information source providing a body of evidence, represents statistical evidence by interval-valued probabilities, and uses Dempster's rule to integrate information based on multiple data source. The method is applied to the problems of ground-cover classification of multispectral data combined with digital terrain data such as elevation, slope, and aspect. Then this method is applied to simulated 201-band High Resolution Imaging Spectrometer (HIRIS) data by dividing the dimensionally huge data source into smaller and more manageable pieces based on the global statistical correlation information. It produces higher classification accuracy than the Maximum Likelihood (ML) classification method when the Hughes phenomenon is apparent.

Kim, H.↗
NASA NTRS1990

A method for classification of multisource data using interval-valued probabilities and its application to HIRIS data

A method of classifying multisource data in remote sensing is presented. The proposed method considers each data source as an information source providing a body of evidence, represents statistical evidence by interval-valued probabilities, and uses Dempster's rule to integrate information based on multiple data sources. The method is applied to the problems of ground-cover classification of multispectral data combined with digital terrain data such as elevation, slope, and aspect. Then this method is applied to simulated 201-band High Resolution Imaging Spectrometer (HIRIS) data by dividing the dimensionally huge data source into smaller and more manageable pieces based on the global statistical correlation information. It produces higher classification accuracy than the Maximum Likelihood (ML) classification method when the Hughes phenomenon is apparent.

Kim, H.↗
NASA NTRS1990

The ER-2 meteorological measurement system

The objectives of ER-2 Meteorological Measurement System (MMS) are: (1) to measure the meteorological parameters (pressure, temperature, and the three dimensional wind vector) accurately; (2) to provide high resolution data on atmospheric state variables and aircraft flight track to ER-2 investigators on a timely basis; and (3) to conduct collaborative research in atmospheric dynamics and chemistry. A summary of progress and results are presented.

Chan, K. Roland↗
NASA NTRS1992

Three-dimensional numerical simulation of the 20 June 1991, Orlando microburst

On 20 June 1991, NASA's Boeing 737, equipped with in-situ and look-ahead wind-shear detection systems, made direct low-level penetrations (300-350 m AGL) through a microburst during several stages of its evolution. This microburst was located roughly 20 km northeast of Orlando International Airport and was monitored by a Terminal Doppler Weather Radar (TDWR) located about 10 km south of the airport. The first NASA encounter with this microburst (Event 142), at approximately 2041 UTC, was during its intensification phase. At flight level, in-situ measurements indicated a peak 1-km (averaged) F-factor of approximately 0.1. The second NASA encounter (Event 143) occurred at approximately 2046 UTC, about the time of microburst peak intensity. It was during this penetration that a peak 1-km F-factor of approximately 17 was encountered, which was the largest in-situ measurement of the 1991 summer deployment. By the third encounter (Event 144), at approximately 2051 UTC, the microburst had expanded into a macroburst. During this phase of evolution, an in-situ 1-km F-factor of 0.08 was measured. The focus of this paper is to examine this microburst via numerical simulation from an unsteady, three-dimensional meteorological cloud model. The simulated high-resolution data fields of wind, temperature, radar reflectivity factor, and precipitation are closely examined so as to derive information not readily available from 'observations' and to enhance our understanding of the actual event. Characteristics of the simulated microburst evolution are compared with TDWR and in-situ measurements.

Proctor, Fred H.↗
DOE OSTI2024

Assessment of Accelerated Stress Testing Data for Silicon Photovoltaics Using Tensor Decomposition Methods

In this work, we examine the use of high-order tensor decompositions to analyze degradation pathways emerging from accelerated stress testing of silicon photovoltaic (PV) modules. Matrix-based decompositions are powerful tools for studying two-dimensional data arrays and form the foundation of a host of classical data analysis techniques. Tensors are high-order extrapolations of matrices that are able to account for more parameter dimensions, and a variety of tensor decomposition methods have been developed that similarly seek to extend insights from matrix decompositions to higher dimensions. Applying and interpreting tensor decomposition methods to sequences of PV module image data, we seek to uncover and isolate different degradation modes occurring from accelerated stress testing procedures. Further, we consider the contributions of different modes to PV module performance degradations.

data analysis↗
NASA NTRS1981

Characteristics of mean velocity in the annulus wall region at the exit of turbomachinery rotor passage

This paper reports the experimental study of the three-dimensional characteristics of the mean velocity in the annulus-wall boundary layer and wake mixing region of a moderately loaded compressor rotor blade. The measurements reported were obtained using a three-sensor hot-wire probe at the exit of the rotor. They consisted of tangential surveys at several axial and radial locations downstream of the compressor rotor blade passage. The variation of the streamwise, normal, and radial velocity profiles and their decay characteristics are derived from this data. The measurements reveal the highly complex and three-dimensional nature of the annulus-wall flow field and the nature of the interaction of the annulus-wall boundary layer, the blade boundary layers and wake, and the secondary flow associated with the annulus-wall region.

Davino, R.↗
NASA NTRS2016

Tree Canopy Light Interception Estimates in Almond and a Walnut Orchards Using Ground, Low Flying Aircraft, and Satellite Based Methods to Improve Irrigation Scheduling Programs

Canopy light interception is a main driver of water use and crop yield in almond and walnut production. Fractional green canopy cover (Fc) is a good indicator of light interception and can be estimated remotely from satellite using the normalized difference vegetation index (NDVI) data. Satellite-based Fc estimates could be used to inform crop evapotranspiration models, and hence support improvements in irrigation evaluation and management capabilities. Satellite estimates of Fc in almond and walnut orchards, however, need to be verified before incorporating them into irrigation scheduling or other crop water management programs. In this study, Landsat-based NDVI and Fc from NASA's Satellite Irrigation Management Support (SIMS) were compared with four estimates of canopy cover: 1. light bar measurement, 2. in-situ and image-based dimensional tree-crown analyses, 3. high-resolution NDVI data from low flying aircraft, and 4. orchard photos obtained via Google Earth and processed by an Image J thresholding routine. Correlations between the various estimates are discussed.

Satellite↗
NASA NTRS1988

An entropy and viscosity corrected potential method for rotor performance prediction

An unsteady Full-Potential Rotor code (FPR) has been enhanced with modifications directed at improving its drag prediction capability. The shock generated entropy has been included to provide solutions comparable to the Euler equations. A weakly interacted integral boundary layer has also been coupled to FPR in order to estimate skin-friction drag. Pressure distributions, shock positions, and drag comparisons are made with various data sets derived from two-dimensional airfoil, hovering, and advancing high speed rotor tests. In all these comparisons, the effect of the nonisentropic modification improves (i.e., weakens) the shock strength and wave drag. In addition, the boundary layer method yields reasonable estimates of skin-friction drag. Airfoil drag and hover torque data comparisons are excellent, as are predicted shock strength and positions for a high speed advancing rotor.

Bridgeman, John O.↗
NASA NTRS2022

Effective Uncertainty Quantification for Multi-Angle Polarimetric Aerosol Remote Sensing Over Ocean

Multi-angle polarimetric (MAP) measurements can enable detailed characterization of aerosol microphysical and optical properties and improve atmospheric correction in ocean color remote sensing. Advanced retrieval algorithms have been developed to obtain multiple geophysical parameters in the atmosphere–ocean system. Theoretical pixel-wise retrieval uncertainties based on error propagation have been used to quantify retrieval performance and determine the quality of data products. However, standard error propagation techniques in high-dimensional retrievals may not always represent true retrieval errors well due to issues such as local minima and the nonlinear dependence of the forward model on the retrieved parameters near the solution. In this work, we analyze these theoretical uncertainty estimates and validate them using a flexible Monte Carlo approach. The Fast Multi-Angular Polarimetric Ocean coLor (FastMAPOL) retrieval algorithm, based on efficient neural network forward models, is used to conduct the retrievals and uncertainty quantification on both synthetic HARP2 (Hyper-Angular Rainbow Polarimeter 2) and AirHARP (airborne version of HARP2) datasets. In addition, for practical application of the uncertainty evaluation technique in operational data processing, we use the automatic differentiation method to calculate derivatives analytically based on the neural network models. Both the speed and accuracy associated with uncertainty quantification for MAP retrievals are addressed in this study. Pixel-wise retrieval uncertainties are further evaluated for the real AirHARP field campaign data. The uncertainty quantification methods and results can be used to evaluate the quality of data products, as well as guide MAP algorithm development for current and future satellite systems such as NASA’s Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission.

PACE↗
NASA NTRS1975

Two-dimensional convolute integers for optical image data processing and surface fitting

An approach toward low-pass, high-pass and band-pass filtering is presented. Convolution coefficients possessing the filtering speed associated with a moving smoothing average without suffering a loss of resolution are discussed. Resolution was retained because the coefficients represented the equivalance of applying high order two-dimensional regression calculations to an image without considering the time-consuming summations associated with the usual normal equations. The smoothing (low-pass) and roughing (high-pass) aspects of the filters are a result of being derived from regression theory. The coefficients are universal integer valves completely described by filter size and surface order, and possess a number of symmetry properties. Double convolution lead to a single set of coefficients with an expanded mask which can yield band-pass filtering and the surface normal. For low order surfaces (0,1), the two-dimensional convolute integers were equivalent to a moving smoothing average.

Edwards, T. R.↗
NASA NTRS2018

Climatespark: an In-Memory Distributed Computing Framework for Big Climate Data Analytics

The unprecedented growth of climate data creates new opportunities for climate studies, and yet big climate data pose a grand challenge to climatologists to efficiently manage and analyze big data. The complexity of climate data content and analytical algorithms increases the difficulty of implementing algorithms on high performance computing systems. This paper proposes an in-memory, distributed computing framework, ClimateSpark, to facilitate complex big data analytics and time-consuming computational tasks. Chunking data structure improves parallel I/O efficiency, while a spatiotemporal index is built for the chunks to avoid unnecessary data reading and preprocessing. An integrated, multi-dimensional, array-based data model (ClimateRDD) and ETL operations are developed to address big climate data variety by integrating the processing components of the climate data lifecycle. ClimateSpark utilizes Spark SQL and Apache Zeppelin to develop a web portal to facilitate the interaction among climatologists, climate data, analytic operations and computing resources (e.g., using SQL query and Scala/Python notebook). Experimental results show that ClimateSpark conducts different spatiotemporal data queries/analytics with high efficiency and data locality. ClimateSpark is easily adaptable to other big multiple- dimensional, array-based datasets in various geoscience domains.

Hu, Fei↗
DOE OSTI2026

A score-based diffusion model approach for adaptive learning of stochastic partial differential equation solutions

In this paper, we propose a novel framework for adaptively learning the time-evolving solutions of stochastic partial differential equations (SPDEs) using score-based diffusion models within a recursive Bayesian inference setting. SPDEs play a central role in modeling complex physical systems under uncertainty, but their numerical solutions often suffer from model errors and reduced accuracy due to incomplete physical knowledge and environmental variability. To address these challenges, we encode the governing physics into the score function of a diffusion model using simulation data and incorporate observational information via a likelihood-based correction in a reverse-time stochastic differential equation. This enables adaptive learning through iterative refinement of the solution as new data becomes available. To improve computational efficiency in high-dimensional settings, we introduce the ensemble score filter, a training-free approximation of the score function designed for real-time inference. Numerical experiments on benchmark SPDEs demonstrate the accuracy and robustness of the proposed method under sparse and noisy observations.

97 MATHEMATICS AND COMPUTING↗