Search NASA⌕ Search

SEARCH · Search NASA

Results for “High dimensional data,”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Image processing tools for petabyte-scale light sheet microscopy data

Light sheet microscopy is a powerful technique for high-speed three-dimensional imaging of subcellular dynamics and large biological specimens. However, it often generates datasets ranging from hundreds of gigabytes to petabytes in size for a single experiment. Conventional computational tools process such images far slower than the time to acquire them and often fail outright due to memory limitations. To address these challenges, we present PetaKit5D, a scalable software solution for efficient petabyte-scale light sheet image processing. This software incorporates a suite of commonly used processing tools that are optimized for memory and performance. Notable advancements include rapid image readers and writers, fast and memory-efficient geometric transformations, high-performance Richardson–Lucy deconvolution and scalable Zarr-based stitching. These features outperform state-of-the-art methods by over one order of magnitude, enabling the processing of petabyte-scale image data at the full teravoxel rates of modern imaging cameras. The software opens new avenues for biological discoveries through large-scale imaging experiments.

97 MATHEMATICS AND COMPUTING↗

Efficient mapping between void shapes and stress fields using Deep Convolutional Neural Networks with sparse data

Establishing fast and accurate structure-to-property relationships is an important component in the design and discovery of advanced materials. Physics-based simulation models like the finite element method (FEM) are often used to predict deformation, stress, and strain fields as a function of material microstructure in material and structural systems. Such models may be computationally expensive and time intensive if the underlying physics of the system is complex. This limits their application to solve inverse design problems and identify structures that maximize performance. In such scenarios, surrogate models are employed to make the forward mapping computationally efficient to evaluate. However, the high dimensionality of the input microstructure and the output field of interest often renders such surrogate models inefficient, especially when dealing with sparse data. Deep convolutional neural network (CNN) based surrogate models have shown great promise in handling such high-dimensional problems. In this paper, a single ellipsoidal void structure under a uniaxial tensile load represented by a linear elastic, high-dimensional and expensive-to-query, FEM model. We consider two deep CNN architectures, a modified convolutional autoencoder framework with a fully connected bottleneck and a UNet CNN, and compare their accuracy in predicting the von Mises stress field for any given input void shape in the FEM model. Additionally, a sensitivity analysis study is performed using the two approaches, where the variation in the prediction accuracy on unseen test data is studied through numerical experiments by varying the number of training samples from 20 to 100.

surrogate modeling; convolutional neural networks;↗

Web-Based Tools for Data-Informed Remedy Optimization: Software Theory and User Guide

This report documents the development and application of two web-based decision-support tools for pump-and-treat (P&T) groundwater remediation systems: PTOLEMY (Pump-and-Treat Optimized Location Evaluation to Maximize Yields) and OPTIMA (Optimization for Pump-and-Treat Implementation, Management, & Assessment). These tools enhance remedy design and management by leveraging advanced computational methods – specifically deep learning and multi-objective optimization – within a user-friendly platform. By integrating data-driven models with established hydrogeological knowledge, PTOLEMY and OPTIMA enable more efficient evaluation of well placement and operational strategies, helping site managers balance multiple remediation objectives under complex conditions. Both tools are implemented as modules within the SOCRATES (Suite Of Comprehensive Rapid Analysis Tools for Environmental Sites) web platform, which provides data access, visualization, and analytics to support remedy optimization across sites in the U.S. Department of Energy Office of Environmental Management complex. PTOLEMY is a rapid screening module designed to identify promising locations for new extraction wells. It employs a multi-channel three-dimensional convolutional neural network (MC3D-CNN) trained on high-fidelity simulation data to predict the relative performance (in terms of contaminant mass recovery) of potential well sites. Through an interactive web interface, PTOLEMY visualizes the probability of high performance across a site, highlighting areas where an extraction well is likely to yield above-threshold contaminant removal over a multi-year period. PTOLEMY’s map-based displays and exportable results support transparent communication of screening analyses. By focusing attention on the most favorable candidate locations, the tool augments traditional engineering judgment and physics-based modeling, providing a data informed basis for subsequent detailed evaluations. OPTIMA is a multi objective optimization module designed to find wellfield layouts and operating schedules that meet various cleanup goals. It quickly evaluates thousands of candidate setups – combinations of well locations, timing, and rates – and returns a small set of best trade-off options for comparison. At its core, OPTIMA uses a U-Net-based surrogate model – a deep-learning emulator of a groundwater flow and transport simulator – to dramatically accelerate scenario evaluations. Coupling this fast surrogate with the NSGA-II (Non-dominated Sorting Genetic Algorithm II) evolutionary algorithm, OPTIMA explores a wide decision space of well locations and schedules to identify Pareto-optimal solutions that trade off key objectives (e.g., minimizing cleanup time, maximizing contaminant mass removal, and minimizing plume extent). The tool outputs a family of optimal configurations and visualizes their trade-offs (Pareto frontiers of cleanup metrics and maps of optimized well placements). Site managers can use these results to understand the range of viable strategies and to select candidate designs for more detailed verification. OPTIMA is currently under active development and not yet fully released; this guide provides early documentation to support planning and gather user feedback.

54 ENVIRONMENTAL SCIENCES↗

Full event particle-level unfolding with variable-length latent variational diffusion

The measurements performed by particle physics experiments must account for the imperfect response of the detectors used to observe the interactions. One approach, unfolding, statistically adjusts the experimental data for detector effects. Recently, generative machine learning models have shown promise for performing unbinned unfolding in a high number of dimensions. However, all current generative approaches are limited to unfolding a fixed set of observables, making them unable to perform full-event unfolding in the variable dimensional environment of collider data. A novel modification to the variational latent diffusion model (VLD) approach to generative unfolding is presented, which allows for unfolding of high- and variable-dimensional feature spaces. The performance of this method is evaluated in the context of semi-leptonic t\bar{t} t t ‾ production at the Large Hadron Collider.

Shmakov, Alexander↗

Utilization of Machine Learning Techniques for Managing the Tracking and Data Relay Satellite Constellation

National Aeronautics and Space Administration’s (NASA) Goddard Space Flight Center (GSFC) operates a constellation of ten geosynchronous Tracking and Data Relay Satellites (TDRS). The TDRS constellation consists of multiple geosynchronous communication relay satellites located around the equator so they can provide continual coverage of any mission in low earth orbit. The TDRS are located primarily in three oceanic regions around the earth. NASA’s White Sands Complex provides the ground communication support for TDRS located over the Atlantic and Pacific Oceans. Another TDRS ground station in Guam supports the TDRS over the Indian Ocean. With these satellites the TDRS network can provide continuous coverage of satellites in low-earth orbit. The NASA Space Network (SN) project office at GSFC manages the constellation of spacecraft. Major customers of the TDRS constellation include, but are not limited to, the International Space Station and the Hubble Space Telescope. The TDRS constellation has three generations of satellites and has been active for over 30 years providing reliable communication links between customer satellites and corresponding ground stations. However, one of the major concerns for TDRS, and in any space mission, is to ensure the health and safety of the spacecraft. Generally, engineers use telemetry data to monitor and analyze the performance and state of health of the spacecraft. Telemetry data contains hundreds of parameters that monitor each important component in the spacecraft, which can be utilized to recognize and characterize the behavior of the spacecraft. Each parameter contains considerable information to represent time-dependent properties of each spacecraft subsystem and component. During the entire life of a TDRS spacecraft, thousands of gigabytes of telemetry data are transmitted in real-time from the spacecraft to the ground station at the White Sands Complex in Las Cruces, New Mexico, and recorded as historical data sets for engineers to process and analyze the events that occurred on-orbit. These parameters contain the function of multiple spacecraft subsystems, such as the attitude control system (ACS), Thermal, Electrical Power Subsystem (EPS), etc. . The first and second generations have exceeded their required lifetime and NASA is keen to manage these spacecrafts carefully in order to maximize the remaining life using the spacecraft telemetry. The challenge is to know when the risk of losing a spacecraft in geosynchronous orbit exceeds the benefit of continued operations for customer support. In the TDRS fleet, the EPS is the most critical subsystem related to spacecraft operations. Failure of the EPS would strand a spacecraft in geosynchronous orbit. Since EPS provides power to the spacecraft, component failures ultimately lead to the inability to support the spacecraft loads and the communications payload. For instance, TDRS-8 has several anomalies in EPS including the Bus Voltage Limiter (BVL) shunt current, solar array loss of circuits, and failed battery cells. Any of these anomalies can cause critical issues to the spacecraft. Therefore, developing a system to analyze and perform early detection of a potential anomaly is an important issue in telemetry data analysis. In recent years, Telemetry Mining (TM) has been proposed to process telemetry data by using Data Mining (DM) techniques such as classification, clustering, regression and anomaly detection. Anomaly detection, also known as outlier detection, has been widely used in many data mining areas such as remote sensing, medical data processing and digital image processing. The goal of anomaly detection is to detect abnormal data, which contains a relatively low probability of occurrence among the entire data set. Early detection of anomalies is one of the most significant issues in managing the spacecraft configuration. If anomalies can be detected early enough, then the redundant resources can be used to extend the life of the operational spacecraft. We present an unsupervised anomaly detection method to process the EPS data extracted from TDRS-8. This is different from traditional analytical methods, which use telemetry data to illustrate behavior and physical meaning of each spacecraft component. TM connects multiple parameters as a vector and then conducts data analysis on this high dimension telemetry vector. This method is looking at the properties of a high dimensional vector that is able to consider the relationship between different parameters in the anomaly detection problem. This kind of method performs much better than the traditional limit checking method. In addition, we propose a new approach of real-time anomaly detection to process telemetry data in real-time, which can then be applied to spacecraft monitoring with high reliability, low cost and high accuracy.

Machine Learning (ML)↗

Experimental and calculated characteristics of three wings of NACA 64-210 and 65-210 airfoil sections with and without 2 degree washout

Report presents the results of an investigation conducted to determine some of the effects of airfoil section and washout on the experimental and calculated characteristics of 10-percent-thick wings. Three wings of aspect ratio 9 and ratio of root chord to tip chord 2.5 were tested. One wing had NACA 64-210 sections and 2 degree washout, the second had NACA 65-210 sections and 2 degree washout, and the third had NACA 65-210 sections and 0 degree washout. It was found that the experimental characteristics of the wings could be satisfactorily predicted from calculations based upon two-dimensional data when the airfoil contours of the wings conformed to the true airfoil sections with the same high degree of accuracy as the two-dimensional models.

AIRFOILS - NACA 65-210 - CHAR & MOMENTS LOW-DRAG↗

A procedure for rule extraction from a Self-Organising plasma disruption predictor for JET

In a previous paper, a Self-Organizing Map had proven to be able to identify the regions of the plasma operative space characterizing the pre-disruptive phase at JET without relying on any a priori information. One of the strengths of this disruption predictor lies in its inherent self-organization capability. The Self-Organizing Map discovers non-trivial relationships and captures the complicated interplay of device diagnostics on the internal plasma states directly from the experimental data. Moreover, the provided model allows the visualization of high-dimensional plasma parameters and facilitates easy interrogation of the model to understand the reasons behind its correlations. In this paper, an additional step is taken towards the interpretability of models for predicting disruptions by training a Decision Tree to classify the plasma states according to the interpretation provided by the Self-Organizing Map (stable or at high risk of disruptions). The Decision tree provides a set of rules which describe the transition of the plasma towards the pre-disruptive phase as visualized in the Self-Organizing Map. The obtained rules for the database explored in the study identify four regions in the map, two of which are at risk of disruption. These regions correspond to partitions of a 3D space based on the peaking factors of the core and divertor radiation, as well as the Locked Mode. The agreement between the Self-Organizing Map answers and the rules supplied by the Decision Tree is confirmed by the comparison of the performance exhibited by the two models in the prediction of disruptions.

Setzu, Samuele [Univ. of Cagliari, Monserrato, Cag↗

Improving Trustworthiness of Data-Driven Power Grid Contingency Analysis With Bayesian Residual Graph Neural Networks

The evolving energy landscape requires novel tools to efficiently perform contingency analysis and reliability assessment of power grids, potentially in real-time. The high computational cost of traditional power flow solvers limits their applicability in practice. Machine learning (ML) surrogates such as deep neural networks (NNs) accelerate power flow solvers computations, enabling high-order contingency analysis and real-time decision-making by learning highly nonlinear functions and integrating grid topology via graph architectures. However, (graph) NNs lack predictive power away from training data and do not provide predictive confidence estimates. Here, we present a Bayesian residual graph NN that integrates knowledge from low-fidelity data via residual training and embeds granular quantification of uncertainties, improving trustworthiness critical for high-consequence decision-making. Applying Bayesian concepts to NNs is challenging due to the high-dimensionality of both the parameter space, complicating derivation of a meaningful prior, and the output space in large grid systems, requiring enhanced techniques to assess the predicted high-dimensional uncertainties. Our contributions include: (1) Deriving a prior for fully connected and graph NNs that leverages low-fidelity data to guide mean predictions and appropriately control prior predictive uncertainty. (2) Integrating this prior within an ensembling with anchoring scheme for efficient approximate posterior inference. (3) Deriving enhanced metrics to assess accuracy of both the mean and uncertainty predictions in high dimensions, appropriately accounting for correlations propagated through graph layers. The resulting Bayesian residual graph NN is tested on a contingency analysis task for 14-bus and 118-bus grids.

24 - POWER TRANSMISSION AND DISTRIBUTION↗

Simulating Continuum-based Redshift Measurement in the Roman’s High Latitude Spectroscopic Survey

We investigate the capability of the Nancy Grace Roman Space Telescope’s (Roman) Wide-Field Instrument G150 slitless grism to detect red, quiescent galaxies based on the current reference survey. We simulate dispersed images for Roman reference High-Latitude Spectroscopic Survey (HLSS) and analyze two-dimensional spectroscopic data using the grism Redshift and Line Analysis (Grizli) software. This study focus on assessing Roman grism’s capability for continuum-level redshift measurement for a redshift range of 0.5 ≤ z ≤ 2.5. The redshift recovery is assessed by setting three requirements of: σ z = $\frac{|z–z_{true}|}{1+z}$ ≤ 0.01, signal-to-noise ratio≥ 5 and the presence of a single dominant peak in redshift likelihood function. We find that, for quiescent galaxies, the reference HLSS can reach a redshift recovery completeness of ≥50% for F158 magnitude brighter than 20.2 mag. We also explore how different survey parameters, such as exposure time and the number of exposures, influence the accuracy and completeness of redshift recovery, providing insights that could optimize future survey strategies and enhance the scientific yield of the Roman in cosmological research.

Astronomical simulations↗

KL techniques for optimal processing of time sequential imagery

Time-sequential imagery is difficult to analyze because of its high dimensionality. This paper advances a new algorithm that screens input data in an intelligent way, discards data with negligible information, and uses the remaining images to represent the sequence in an optimal compact form. Data are presented to illustrate how this algorithm can be used to do novelty filtering, novelty detection, segmentation, background independent modeling, and classification.

Vermeulen, Pieter↗

Cometary ion flow variations at Comet P/Halley as observed by the Giotto IMS experiment

The moments of the cometary-ion distributions are determined through a three-dimensional analysis of the Giotto IMS high-intensity spectrometer (HIS) data. The spectrometer is described, with emphasis on its angle analyzer and mass analyzer. The method of data analysis is outlined, with ion-flow vectors and temperatures being addressed. The results of the water group ion-flow profile are presented, and it is noted that, after crossing the cometopause region, the ions become gradually colder. At cometocentric distances larger than 130,000 km, the cometary-ion temperature is found to be in the area of 100 eV or higher, and derivations of the flow parameters are uncertain. The ion temperature and the flow speed become lower by about 50 eV after crossing the magnetic pile-up boundary. It is concluded that the observed velocity and temperature profiles can be explained on the basis of charge exchange processes.

Kettmann, G.↗

The Hyperwall

This paper presents the hyperwall, a visualization cluster that uses coordinated visualizations for interactive exploration of multidimensional data and simulations. The system strongly leverages the human eye-brain system with a generous 7x7 array offlat panel LCD screens powered by a beowulf clustel: With each screen backed by a workstation class PC, graphic and compute intensive applications can be applied to a broad range of data. Navigational tools are presented that allow for investigation of high dimensional spaces.

Biegel, Bryan A.↗

Scalable Adaptive Graphics Environment (SAGE) Software for the Visualization of Large Data Sets on a Video Wall

The use of collaborative scientific visualization systems for the analysis, visualization, and sharing of "big data" available from new high resolution remote sensing satellite sensors or four‐dimensional numerical model simulations is propelling the wider adoption of ultra‐resolution tiled display walls interconnected by high speed networks. These systems require a globally connected and well‐integrated operating environment that provides persistent visualization and collaboration services. This abstract and subsequent presentation describes a new collaborative visualization system installed for NASA's Shortterm Prediction Research and Transition (SPoRT) program at Marshall Space Flight Center and its use for Earth science applications. The system consists of a 3 x 4 array of 1920 x 1080 pixel thin bezel video monitors mounted on a wall in a scientific collaboration lab. The monitors are physically and virtually integrated into a 14' x 7' for video display. The display of scientific data on the video wall is controlled by a single Alienware Aurora PC with a 2nd Generation Intel Core 4.1 GHz processor, 32 GB memory, and an AMD Fire Pro W600 video card with 6 mini display port connections. Six mini display‐to‐dual DVI cables are used to connect the 12 individual video monitors. The open source Scalable Adaptive Graphics Environment (SAGE) windowing and media control framework, running on top of the Ubuntu 12 Linux operating system, allows several users to simultaneously control the display and storage of high resolution still and moving graphics in a variety of formats, on tiled display walls of any size. The Ubuntu operating system supports the open source Scalable Adaptive Graphics Environment (SAGE) software which provides a common environment, or framework, enabling its users to access, display and share a variety of data‐intensive information. This information can be digital‐cinema animations, high‐resolution images, high‐definition video‐teleconferences, presentation slides, documents, spreadsheets or laptop screens. SAGE is cross‐platform, community‐driven, open‐source visualization and collaboration middleware that utilizes shared national and international cyberinfrastructure for the advancement of scientific research and education.

Jedlovec, Gary↗

High Reynolds number test of a NACA 651-213, a equals 0.5 airfoil at transonic speeds

Wind-Tunnel tests were conducted in the Lockheed-Georgia Company's compressible flow facility to determine the transonic two-dimensional aerodynamic characteristics of a NACA 65 sub 1-213 a = 0.50 airfoil. The results are correlated with data obtained in the NASA-Langley 8-foot transonic pressure tunnel and the NAE high Reynolds number 15x60-inch two-dimensional test facility. The tests were conducted over a Mach number range from 0.60 to 0.80 and an angle of attack range from -1 deg to 8 deg. Reynolds numbers, based on the airfoil chord, were varied.

Burdges, K. P.↗

Flare Ribbon Signatures of Plasmoids

Solar flares are explosive space weather events that rapidly convert stored magnetic energy into bulk motion, plasma heating, and particle acceleration. Understanding the structure and dynamics of the magnetic reconnection that powers flares is critical for predicting the energy release. In particular, the amount of energy transferred to energetic particles is thought to be highly dependent on whether the reconnection is primarily turbulent (e.g., plasmoid dominated) or instead laminar. We present new high-resolution MHD simulations of three-dimensional reconnection in an eruptive flare and compare the results to recent data. Although flare reconnection is challenging to observe directly in the corona, highly detailed constraints on its dynamics can be obtained from observations of flare ribbons that track the chromospheric footpoints of newly reconnected field lines. The analogues of flare ribbons in our simulations are identified by tracking discontinuous changes in field-line magnetic connectivity due to the reconnection. In our highest-resolution calculations, we find that these ribbon analogues are highly structured and exhibit many ‘whorl’ patterns that are linked to turbulent plasmoids in the reconnecting current sheet. Such flare ribbon fine structure therefore reveals crucial information about the fundamental turbulent vs. laminar nature of the reconnection. Plasmoid-driven particle acceleration models would therefore predict co-location of energetic particle signatures with plasmoid-associated ribbon fine structure.

J T Dahlin↗

Flare Ribbon Signatures of Reconnection Plasmoids

Solar flares are explosive space weather events that rapidly convert stored magnetic energy into bulk motion, plasma heating, and particle acceleration. Understanding the structure and dynamics of the magnetic reconnection that powers flares is critical for predicting the energy release. In particular, the amount of energy transferred to energetic particles is thought to be highly dependent on whether the reconnection is primarily turbulent (e.g., plasmoid dominated) or instead laminar. We present high-resolution MHD simulations of three-dimensional reconnection in an eruptive flare and compare the results to recent data. Although flare reconnection is challenging to observe directly in the corona, highly detailed constraints on its dynamics can be obtained from observations of flare ribbons that track the chromospheric footpoints of newly reconnected field lines. The analogues of flare ribbons in our simulations are identified by tracking discontinuous changes in field-line magnetic connectivity due to the reconnection. In our highest-resolution calculations, we find that these ribbon analogues are highly structured and exhibit many ‘whorl’ patterns that are linked to turbulent plasmoids in the reconnecting current sheet. Flare ribbon fine structure therefore reveals crucial information about the fundamental turbulent vs. laminar nature of the reconnection. We discuss the implications of these results for understanding reconnection-driven energy release throughout the heliosphere.

J. T. Dahlin↗

Calculation of inviscid flow over shuttle-like vehicles at high angles of attack and comparisons with experimental data

A computer code HALIS, designed to compute the three dimensional flow about shuttle like configurations at angles of attack greater than 25 deg, is described. Results from HALIS are compared where possible with an existing flow field code; such comparisons show excellent agreement. Also, HALIS results are compared with experimental pressure distributions on shuttle models over a wide range of angle of attack. These comparisons are excellent. It is demonstrated that the HALIS code can incorporate equilibrium air chemistry in flow field computations.

Weilmuenster, K. J.↗

A Deterministic Self-Organizing Map Approach and its Application on Satellite Data based Cloud Type Classification

A self-organizing map (SOM) is a type of competitive artificial neural network, which projects the high dimensional input space of the training samples into a low dimensional space with the topology relations preserved. This makes SOMs supportive of organizing and visualizing complex data sets and have been pervasively used among numerous disciplines with different applications. Notwithstanding its wide applications, the self-organizing map is perplexed by its inherent randomness, which produces dissimilar SOM patterns even when being trained on identical training samples with the same parameters every time, and thus causes usability concerns for other domain practitioners and precludes more potential users from exploring SOM based applications in a broader spectrum. Motivated by this practical concern, we propose a deterministic approach as a supplement to the standard self-organizing map. In accordance with the theoretical design, the experimental results with satellite cloud data demonstrate the effective and efficient organization as well as simplification capabilities of the proposed approach.

Initialization method↗