Search NASA⌕ Search

SEARCH · Search NASA

Results for “Unstructured data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Asynchronous distributed-memory task-parallel algorithm for compressible flows on unstructured 3D Eulerian grids

Here, we discuss the implementation of a finite element method, used to numerically solve the Euler equations of compressible flows, using an asynchronous runtime system (RTS). The algorithm is implemented for distributed-memory machines, using stationary unstructured 3D meshes, combining data-, and task-parallelism on top of the Charm++ RTS. Charm++’s execution model is asynchronous by default, allowing arbitrary overlap of computation and communication. Task-parallelism allows scheduling parts of an algorithm independently of, or dependent on, each other. Built-in automatic load balancing enables continuous redistribution of computational load by migration of work units based on real-time CPU load measurement. The RTS also features automatic checkpointing, fault tolerance, resilience against hardware failure, and supports power-, and energy-aware computation. We demonstrate scalability up to 25 x 10 9 cells at $\mathscr{O}$10 4 compute cores and the benefits of automatic load balancing for irregular workloads. The full source code with documentation is available at https://quinoacomputing.org.

42 ENGINEERING↗

SetBERT: the deep learning platform for contextualized embeddings and explainable predictions from high-throughput sequencing

MOTIVATION: High-throughput sequencing (HTS) is a modern sequencing technology used to profile microbiomes by sequencing thousands of short genomic fragments from the microorganisms within a given sample. This technology presents a unique opportunity for artificial intelligence to comprehend the underlying functional relationships of microbial communities. However, due to the unstructured nature of HTS data, nearly all computational models are limited to processing DNA sequences individually. This limitation causes them to miss out on key interactions between microorganisms, significantly hindering our understanding of how these interactions influence the microbial communities as a whole. Furthermore, most computational methods rely on post-processing of samples which could inadvertently introduce unintentional protocol-specific bias. RESULTS: Addressing these concerns, we present SetBERT, a robust pre-training methodology for creating generalized deep learning models for processing HTS data to produce contextualized embeddings and be fine-tuned for downstream tasks with explainable predictions. By leveraging sequence interactions, we show that SetBERT significantly outperforms other models in taxonomic classification with genus-level classification accuracy of 95%. Furthermore, we demonstrate that SetBERT is able to accurately explain its predictions autonomously by confirming the biological-relevance of taxa identified by the model. AVAILABILITY AND IMPLEMENTATION: All source code is available at https://github.com/DLii-Research/setbert. SetBERT may be used through the q2-deepdna QIIME 2 plugin whose source code is available at https://github.com/DLii-Research/q2-deepdna.

Ludwig, David W↗

Rapid Exploitation and Analysis of Documents

Analysts are overwhelmed with information. They have large archives of historical data, both structured and unstructured, and continuous streams of relevant messages and documents that they need to match to current tasks, digest, and incorporate into their analysis. The purpose of the READ project is to develop technologies to make it easier to catalog, classify, and locate relevant information. We approached this task from multiple angles. First, we tackle the issue of processing large quantities of information in reasonable time. Second, we provide mechanisms that allow users to customize their queries based on latent topics exposed from corpus statistics. Third, we assist users in organizing query results, adding localized expert structure over results. Forth, we use word sense disambiguation techniques to increase the precision of matching user generated keyword lists with terms and concepts in the corpus. Fifth, we enhance co-occurrence statistics with latent topic attribution, to aid entity relationship discovery. Finally we quantitatively analyze the quality of three popular latent modeling techniques to examine under which circumstances each is useful.

97 MATHEMATICS AND COMPUTING↗

Flow Control Analysis on the Hump Model with RANS Tools

A concerted effort is underway at NASA Langley Research Center to create a benchmark for Computational Fluid Dynamic (CFD) codes. both unstructured and structured, against a data set for the hump model with actuation. The hump model was tested in the NASA Langley 0.3-m Transonic Cryogenic Tunnel. The CFD codes used for the analyses are the FUN2D (Full Unstructured Navier-Stokes 2-Dimensional) code, the structured TLNS3D (Thin-Layer Navier-Stokes 3-Dimensional) code, and the structured CFL3D code, all developed at NASA Langley. The current investigation uses the time-accurate Reynolds-Averaged Navier-Stokes (RANS) approach to predict aerodynamic performance of the active flow control experimental database for the hump model. Two-dimensional computational results verified that steady blowing and suction and oscillatory suction/blowing can be used to significantly reduce the separated flow region on the model. Discrepancies do exist between the CFD results and experimental data in the region downstream of the slot with the largest differences in the oscillatory cases. Overall, the structured CFD codes exhibited similar behavior with each other for a wide range of control conditions, with the unstructured FUN2D code showing moderately different results in the separated flow region for the suction and oscillatory cases.

Viken, Sally A.↗

Sandia Unstructured Triangle Table Generator

The software generates data tables for thermodynamic and transport properties of materials as described by a set of input models. For each input model parameterization, an associated table is created on an unstructured triangular grid. These grids all conform to the same topology. A statistical accuracy guarantee is provided for the tabular representation of the model. Details of the model and table specification are given in a XML input deck.

Carpenter, JohnH.↗

Canopus: A Paradigm Shift Towards Elastic Extreme-Scale Data Analytics on HPC

Scientific simulations on high performance computing (HPC) platforms generate large quantities of data. To bridge the widening gap between compute and I/O, and enable data to be more efficiently stored and analyzed, simulation outputs need to be refactored, reduced, and appropriately mapped to storage tiers. However, a systematic solution to support these steps has been lacking on the current HPC software ecosystem. To that end, this paper develops Canopus, a progressive JPEGlike data management scheme for storing and analyzing big scientific data. It co-designs the data decimation, compression and data storage, taking the hardware characteristics of each storage tier into considerations. With reasonably low overhead, our approach refactors simulation data into a much smaller, reduced-accuracy base dataset, and a series of deltas that is used to augment the accuracy if needed. The base dataset and deltas are compressed and written to multiple storage tiers. Data saved on different tiers can then be selectively retrieved to restore the level of accuracy that satisfies data analytics. Thus, Canopus provides a paradigm shift towards elastic data analytics and enables end users to make trade-offs between analysis speed and accuracy on-the-fly. We evaluate the impact of Canopus on unstructured triangular meshes, a pervasive data model used by scientific modeling and simulations. In particular, we demonstrate the progressive data exploration of Canopus using the “blob detection” use case on the fusion simulation data.

Lu, Tao↗

A Big Data Analytics Framework for HPC Log Data: Three Case Studies Using the Titan Supercomputer Log

Reliability, availability and serviceability (RAS) logs of high performance computing (HPC) resources, when closely investigated in spatial and temporal dimensions, can provide invaluable information regarding system status, performance, and resource utilization. These data are often generated from multiple logging systems and sensors that cover many components of the system. The analysis of these data for finding persistent temporal and spatial insights faces two main difficulties: the volume of RAS logs makes manual inspection difficult and the unstructured nature and unique properties of log data produced by each subsystem adds another dimension of difficulty in identifying implicit correlation among recorded events. To address these issues, we recently developed a multi-user Big Data analytics framework for HPC log data at Oak Ridge National Laboratory (ORNL). This paper introduces three in-progress data analytics projects that leverage this framework to assess system status, mine event patterns, and study correlations between user applications and system events. We describe the motivation of each project and detail their workflows using three years of log data collected from ORNL's Titan supercomputer.

Park, Byung↗

Adding Data Management Services to Parallel File Systems

The objective of this project, called DAMASC for “Data Management in Scientific Computing”, is to coalesce data management with parallel file system management to present a declarative interface to scientists for managing, querying, and analyzing extremely large data sets efficiently and predictably. Managing extremely large data sets is a key challenge of exascale computing. The overhead, energy, and cost of moving massive volumes of data demand designs where computation is close to storage. In current architectures, compute/analysis clusters access data in a physically separate parallel file system and largely leave it scientist to reduce data movement. Over the past decades the high-end computing community has adopted middleware with multiple layers of abstractions and specialized file formats such as NetCDF-4 and HDF5. These abstractions provide a limited set of high-level data processing functions, but have inherent functionality and performance limitations: middleware that provides access to the highly structured contents of scientific data files stored in the (unstructured) file systems can only optimize to the extent that file system interfaces permit; the highly structured formats of these files often impedes native file system performance optimizations. We are developing Damasc, an enhanced high-performance file system with native rich data management services. Damasc will enable efficient queries and updates over files stored in their native byte-stream format while retaining the inherent performance of file system data storage via declarative queries and updates over views of underlying files. Damasc has four key benefits for the development of data-intensive scientific code: (1) applications can use important data-management services, such as declarative queries, views, and provenance tracking, that are currently available only within database systems; (2) the use of these services becomes easier, as they are provided within a familiar file-based ecosystem; (3) common optimizations, e.g., indexing and caching, are readily supported across several file formats, avoiding effort duplication; and (4) performance improves significantly, as data processing is integrated more tightly with data storage. Our key contributions are: SciHadoop which explores changes to MapReduce assumption by taking advantage of semantics of structured data while preserving MapReduce’s failure and resource management; DataMods which extends common abstractions of parallel file systems so they become programmable such that they can be extended to natively support a variety of data models and can be hooked into emerging distributed runtimes such as Stanford’s Legion; and Miso which combines Hadoop and relational data warehousing to minimize time to insight, taking into account the overhead of ingesting data into data warehousing.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Sandia Unstructured Triangle Tabular Interpolation Package v 0.1 beta

The software interpolates tabular data, such as for equations of state, provided on an unstructured triangular grid. In particular, interpolation occurs in a two dimensional space by looking up the triangle in which the desired evaluation point resides and then performing a linear interpolation over the n-tuples associated with the nodes of the chosen triangle. The interface to the interpolation routines allows for automated conversion of units from those tabulated to the desired output units. when multiple tables are included in a data file, new tables may be generated by on-the-fly mixing of the provided tables

Carpenter, JohnH.↗

An Accelerated Clip Algorithm for Unstructured Meshes: A Batch-Driven Approach

The clip technique is a popular method for visualizing complex structures and phenomena within 3D unstructured meshes. Meshes can be clipped by specifying a scalar isovalue to produce an output unstructured mesh with its external surface as the isovalue. Similar to isocontouring, the clipping process relies on scalar data associated with the mesh points, including scalar data generated by implicit functions such as planes, boxes, and spheres, which facilitates the visualization of results interior to the grid. In this paper, we introduce a novel batch-driven parallel algorithm based on a sequential clip algorithm designed for high-quality results in partial volume extraction. Our algorithm comprises five passes, each progressively processing data to generate the resulting clipped unstructured mesh. The novelty lies in the use of fixed-size batches of points and cells, which enable rapid workload trimming and parallel processing, leading to a significantly improved memory footprint and run-time performance compared to the original version. On a 32-core CPU, the proposed batch-driven parallel algorithm demonstrates a run-time speed-up of up to 32.6x and a memory footprint reduction of up to 4.37x compared to the existing sequential algorithm. The software is currently available under an open-source license in the VTK visualization system.

Tsalikis, Spiros↗

Implementation of Flow Tripping Capability in the USM3D Unstructured Flow Solver

A flow tripping capability is added to an established NASA tetrahedral unstructured parallel Navier-Stokes flow solver, USM3D. The capability is based on prescribing an appropriate profile of turbulence model variables to energize the boundary layer in a plane normal to a specified trip region on the body surface. We demonstrate this approach using the k-e two-equation turbulence model of USM3D. Modification to the solution procedure primarily consists of developing a data structure to identify all unstructured tetrahedral grid cells located in the plane normal to a specified surface trip region and computing a function based on the mean flow solution to specify the modified profile of the turbulence model variables. We leverage this data structure and also show an adjunct approach that is based on enforcing a laminar flow condition on the otherwise fully turbulent flow solution in user specified region. The latter approach is applied for the solutions obtained using other one- and two-equation turbulence models of USM3D. A key ingredient of the present capability is the use of a graphical user-interface tool PREDISC to define a trip region on the body surface in an existing grid. Verification of the present modifications is demonstrated on three cases, namely, a flat plate, the RAE2822 airfoil, and the DLR F6 wing-fuselage configuration.

Pandya, Mohagna J.↗

Implementation of Flow Tripping Capability in the USM3D Unstructured Flow Solver

A flow tripping capability is added to an established NASA tetrahedral unstructured parallel Navier-Stokes flow solver, USM3D. The capability is based on prescribing an appropriate profile of turbulence model variables to energize the boundary layer in a plane normal to a specified trip region on the body surface. We demonstrate this approach using the k-epsilon two-equation turbulence model of USM3D. Modification to the solution procedure primarily consists of developing a data structure to identify all unstructured tetrahedral grid cells located in the plane normal to a specified surface trip region and computing a function based on the mean flow solution to specify the modified profile of the turbulence model variables. We leverage this data structure and also show an adjunct approach that is based on enforcing a laminar flow condition on the otherwise fully turbulent flow solution in user-specified region. The latter approach is applied for the solutions obtained using other one-and two-equation turbulence models of USM3D. A key ingredient of the present capability is the use of a graphical user-interface tool PREDISC to define a trip region on the body surface in an existing grid. Verification of the present modifications is demonstrated on three cases, namely, a flat plate, the RAE2822 airfoil, and the DLR F6 wing-fuselage configuration.

Pandya, Mohagna J.↗

An efficient approach to unstructured mesh hydrodynamics on the cell broadband engine (u)

Unstructured mesh physics for the Cell Broadband Engine (CBE) has received little or no attention to date, largely because the CBE architecture poses particular challenges for unstructured mesh algorithms. SPU memory management strategies such as data preloading cannot be applied to the irregular memory storage patterns of unstructured meshes; and the SPU vector instruction set does not support the indirect addressing needed by connectivity arrays. This paper presents an approach to unstructured mesh physics that addresses these challenges, by creating a new mesh data structure and reorganizing code to give efficient CBE performance. The approach is demonstrated on the FLAG production hydrodynamics code using standard test problems, and results show an average speedup of more than 5x over the original code.

99 GENERAL AND MISCELLANEOUS↗

Facility Detection and Popularity Assessment from Text Classification of Social Media and Crowdsourced Data

Advances in technology have continually progressed our understanding of where people are, how they use the environment around them, and why they are at their current location. Having a better knowledge of when various locations become popular through space and time could have large impacts on research fields like urban dynamics and energy consumption. In this paper, we discuss the ability to identify and locate various facility types (e.g. restaurant, airport, stadiums) using social media, and assess methods in determining when these facilities become popular over time. We use natural language processing tools and machine learning classifiers to interpret geotagged Twitter text and determine if a user is seemingly at a location of interest when the tweet was sent. On average our classifiers are approximately 85% accurate varying across multiple facility types, with a peak precision of 98%. By using these methods to classify unstructured text, geotagged social media data can be an extremely useful tool to better understanding the composition of places and how and when people use them.

crowd-sourced data↗

USM3D Unstructured Grid Solutions for CAWAPI at NASA LaRC

In support the Cranked Arrow Wing Aerodynamic Project International (CAWAPI) to improve the Technology Readiness Level of flow solvers by comparing results with measured F-16XL-1 flight data, NASA Langley employed the TetrUSS unstructured grid solver, USM3D, to obtain solutions for all seven flight conditions of interest. A newly available solver version that incorporates a number of turbulence models, including the two-equation linear and non-linear k-epsilon, was used in this study. As a first test, a choice was made to utilize only a single grid resolution with the solver for the simulation of the different flight conditions. Comparisons are presented with three turbulence models in USM3D, flight data for surface pressure, boundary-layer profiles, and skin-friction results, as well as limited predictions from other solvers. A result of these comparisons is that the USM3D solver can be used in an engineering environment to predict flow physics on a complex configuration at flight Reynolds numbers with a two-equation linear k-epsilon turbulence model.

Lamar, John E.↗

Adaptive wavelet compression of large additive manufacturing experimental and simulation datasets

New manufacturing technologies such as additive manufacturing require research and development to minimize the uncertainties in the produced parts. The research involves experimental measurements and large simulations, which result in huge quantities of data to store and analyze. We address this challenge by alleviating the data storage requirements using lossy data compression. We select wavelet bases as the mathematical tool for compression. Unlike images, additive manufacturing data is often represented on irregular geometries and unstructured meshes. Thus, we use Alpert tree-wavelets as bases for our data compression method. We first analyze different basis functions for the wavelets and find the one that results in maximal compression and miminal error in the reconstructed data. We then devise a new adaptive thresholding method that is data-agnostic and allows a priori estimation of the reconstruction error. Finally, we propose metrics to quantify the global and local errors in the reconstructed data. One of the error metrics addresses the preservation of physical constraints in reconstructed data fields, such as divergence-free stress field in structural simulations. While our compression and decompression method is general, we apply it to both experimental and computational data obtained from measurements and thermal/structural modeling of the sintering of a hollow cylinder from metal powders using a Laser Engineered Net Shape process. The results show that monomials achieve optimal compression performance when used as wavelet bases. The new thresholding method results in compression ratios that are two to seven times larger than the ones obtained with commonly used thresholds. Overall, adaptive Alpert tree-wavelets can achieve compression ratios between one and three orders of magnitude depending on the features in the data that are required to preserve. Furthermore, these results show that Alpert tree-wavelet compression is a viable and promising technique to reduce the size of large data structures found in both experiments and simulations.

42 ENGINEERING↗

Numerical Solutions for the CAWAPI Configuration on Unstructured Grids at NASA LaRC, United States

In support of the Cranked Arrow Wing Aerodynamic Project International (CAWAPI) with its goal of improving the Technology Readiness Level of flow solvers by comparing results with measured F-16XL-1 flight data, NASA Langley employed the TetrUSS unstructured grid solver, USM3D, to obtain solutions for all seven flight conditions of interest. A newly available solver version that incorporates a number of turbulence models, including the two-equation linear and non-linear k- , was used in this study. As a first test, a choice was made to utilize only a single grid resolution with the solver for the simulation of the different flight conditions. Comparisons are presented with three turbulence models in USM3D, flight data for surface pressure, boundary-layer profiles, and skin-friction distribution, as well as limited predictions from other solvers. A result of these comparisons is that the USM3D solver can be used in an engineering environment to predict vortex-flow physics on a complex configuration at flight Reynolds numbers with a two-equation linear k- turbulence model.

Lamar, John E.↗

TomoPyUI : a user-friendly tool for rapid tomography alignment and reconstruction

The management and processing of synchrotron and neutron computed tomography data can be a complex, labor-intensive and unstructured process. Users devote substantial time to both manually processing their data ( i.e. organizing data/metadata, applying image filters etc. ) and waiting for the computation of iterative alignment and reconstruction algorithms to finish. In this work, we present a solution to these problems: TomoPyUI , a user interface for the well known tomography data processing package TomoPy . This highly visual Python software package guides the user through the tomography processing pipeline from data import, preprocessing, alignment and finally to 3D volume reconstruction. The TomoPyUI systematic intermediate data and metadata storage system improves organization, and the inspection and manipulation tools (built within the application) help to avoid interrupted workflows. Notably, TomoPyUI operates entirely within a Jupyter environment. Herein, we provide a summary of these key features of TomoPyUI , along with an overview of the tomography processing pipeline, a discussion of the landscape of existing tomography processing software and the purpose of TomoPyUI , and a demonstration of its capabilities for real tomography data collected at SSRL beamline 6-2c.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗