Search NASA⌕ Search

SEARCH · Search NASA

Results for “data representation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Real-Time Interactive 4D-STEM Phase-Contrast Imaging From Electron Event Representation Data: Less computation with the right representation

The arrival of direct electron detectors (DED) with high frame-rates in the field of scanning transmission electron microscopy has enabled many experimental techniques that require collection of a full diffraction pattern at each scan position, a field which is subsumed under the name four dimensional-scanning transmission electron microscopy (4D-STEM). DED frame rates approaching 100 kHz require data transmission rates and data storage capabilities that exceed commonly available computing infrastructure. Current commercial DEDs allow the user to make compromises in pixel bit depth, detector binning or windowing to reduce the per-frame file size and allow higher frame rates. This change in detector specifications requires decisions to be made before data acquisition that may reduce or lose information that could have been advantageous during data analysis. The 4D Camera, a DED with 87 kHz frame-rate developed at Lawrence Berkeley National Laboratory, reduces the raw data to a linear-index encoded electron event representation (EER). Here we show with experimental data from the 4D Camera that linear-index encoded EER and its direct use in 4D-STEM phase contrast imaging methods enables real-time, interactive phase-contrast from large-area 4D-STEM datasets. Furthermore, we detail the computational complexity advantages of the EER and the necessary computational steps to achieve real-time interactive ptychography and center-of-mass differential phase contrast using commonly available hardware accelerators.

4D-STEM↗
DOE OSTI2020

Multimodal Data Representation with Deep Learning for Extracting Cancer Characteristics from Clinical Text

This paper presents a multimodal data representation to improve the performance of deep learning models for extracting cancer key characteristics from unstructured text in pathology reports. Specifically, in addition to using the text as the input to deep learning models, we use concept unique identifiers (CUIs) as another source of information to the models. We analyze the performance of different text and CUI data representations, including word embeddings and bag of embeddings (BOE), with a convolutional neural network (CNN) and a fully connected multilayer perceptron neural network (MLP-NN). The high level document embeddings from text and CUI inputs are combined by concatenating them and then applying a classifier. The model is used for extracting cancer subsite and histology from pathology reports. These two classification tasks have a large number of labels, i.e. 317 for subsite and 556 for histology, with extreme class imbalance. We compare the performance of the developed DL models across the two tasks based on micro- and macro-F1 scores. The evaluation shows that a multi-channel DL model that utilizes text represented by word embeddings and CUIs represented by BOE outperforms other DL models. Also, this approach significantly improves the model performance on low prevalence classes.

Alawad, Mohammed↗
NASA NTRS2017

An Innovative Infrastructure with a Universal Geo-Spatiotemporal Data Representation Supporting Cost-Effective Integration of Diverse Earth Science Data

The SpatioTemporal Adaptive Resolution Encoding (STARE) is a unifying scheme encoding geospatial and temporal information for organizing data on scalable computing/storage resources, minimizing expensive data transfers. STARE provides a compact representation that turns set-logic functions into integer operations, e.g. conditional sub-setting, taking into account representative spatiotemporal resolutions of the data in the datasets. STARE geo-spatiotemporally aligns data placements of diverse data on massive parallel resources to maximize performance. Automating important scientific functions (e.g. regridding) and computational functions (e.g. data placement) allows scientists to focus on domain-specific questions instead of expending their efforts and expertise on data processing. With STARE-enabled automation, SciDB (Scientific Database) plus STARE provides a database interface, reducing costly data preparation, increasing the volume and variety of interoperable data, and easing result sharing. Using SciDB plus STARE as part of an integrated analysis infrastructure dramatically eases combining diametrically different datasets.

Rilee, Michael Lee↗
DOE OSTI2022

Analysis of Interpretable Data Representations for 4D-STEM Using Unsupervised Learning

Abstract Understanding the structure of materials is crucial for engineering devices and materials with enhanced performance. Four-dimensional scanning transmission electron microscopy (4D-STEM) is capable of mapping nanometer-scale local crystallographic structure over micron-scale field of views. However, 4D-STEM datasets can contain tens of thousands of images from a wide variety of material structures, making it difficult to automate detection and classification of structures. Traditional automated analysis pipelines for 4D-STEM focus on supervised approaches, which require prior knowledge of the material structure and cannot describe anomalous or deviant structures. In this article, a pipeline for engineering 4D-STEM feature representations for unsupervised clustering using non-negative matrix factorization (NMF) is introduced. Each feature is evaluated using NMF and results are presented for both simulated and experimental data. It is shown that some data representations more reliably identify overlapping grains. Additionally, real space refinement is applied to identify spatially distinct sample regions, allowing for size and shape analysis to be performed. This work lays the foundation for improved analysis of nanoscale structural features in materials that deviate from expected crystallographic arrangement using 4D-STEM.

Bruefach, Alexandra (ORCID:0000000209323477)↗
NASA NTRS1977

Data representations in PDP-10 MacLISP

The internal representations of the various MacLISP data types are presented and discussed. Certain implementation tradeoffs are considered. The ultimate decisions on these tradeoffs are discussed in the light of MacLISP's prime objective of being an efficient high-level language for the implementation of large systems such as MACSYMA. The basic strategy of garbage collection is outlined, with reference to the specific representations involved. Certain clever tricks are explained and justified. The address space crunch is explained and some alternative solutions explored.

Steele, G. L., Jr.↗
NASA NTRS1993

Material Data Representation of Hysteresis Loops for Hastelloy X Using Artificial Neural Networks

The artificial neural network (ANN) model proposed by Rumelhart, Hinton, and Williams is applied to develop a functional approximation of material data in the form of hysteresis loops from a nickel-base superalloy, Hastelloy X. Several different ANN configurations are used to model hysteresis loops at different cycles for this alloy. The ANN models were successful in reproducing the hysteresis loops used for its training. However, because of sharp bends at the two ends of hysteresis loops, a drift occurs at the corners of the loops where loading changes to unloading and vice versa (the sharp bends occurred when the stress-strain curves were reproduced by adding stress increments to the preceding values of the stresses). Therefore, it is possible only to reproduce half of the loading path. The generalization capability of the network was tested by using additional data for two other hysteresis loops at different cycles. The results were in good agreement. Also, the use of ANN led to a data compression ratio of approximately 22:1.

Alam, Javed↗
DOE OSTI2020

Joint Analysis of Program Data Representations using Machine Learning for Improved Software Assurance and Development Capabilities

We explore the use of multiple deep learning models for detecting flaws in software programs. Current, standard approaches for flaw detection rely on a single representation of a software program (e.g., source code or a program binary). We illustrate that, by using techniques from multimodal deep learning, we can simultaneously leverage multiple representations of software programs to improve flaw detection over single representation analyses. Specifically, we adapt three deep learning models from the multimodal learning literature for use in flaw detection and demonstrate how these models outperform traditional deep learning models. We present results on detecting software flaws using the Juliet Test Suite and Linux Kernel.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Striped Data Analysis Framework

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. We have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

Gutsche, Oliver↗
DOE OSTI2026

Metadata Standards for the NSE: Extended Field Standards

This standard presents a set of optional metadata fields for managed digital objects within the Nuclear Security Enterprise (NSE) and provides a deeper look at data representation in metadata by looking at the representation of 1) Records Management required metadata, and 2) common representations of technical/scientific data. Metadata standardization is a critical enabler for effectively sharing data, documents, and other digital objects between NSE sites, and for tracing the digital thread at the object level. Standardization is necessary for both schemas and vocabularies, meaning that both field standards and value standards must be specified. This document serves as a complementary field standard, recommending an optional set of fields that should be uniformly built for all managed digital objects within the NSE. This document specifically focuses on extending the shared discovery layer defined in the first white paper by introducing additional descriptive and data representation fields that improve cross-site search and interpretation.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1985

An algorithm for computing the number of distinct spectral vectors in thematic mapper data

A computationally efficient method was developed to compute the number of distinct spectral vectors and their frequency of occurrence in Landsat-4 Thematic Mapper (TM) data. The algorithm first partitions the image into spectrally disjoint subsets and then computes the frequency distribution of distinct spectral vectors within each subset from a multidimensional histogram. The overall frequency distribution is tabulated by accumulating the results from each subset. The number of distinct spectral vectors could be used as a measure of potential storage compaction of alternate data representations for data compression, or as a measure of information content in the comparison of spectral band combinations and/or spatial resolutions for an image. Results from processing three 512 x 512 pixel Landsat-4 TM images and one Landsat-4 Multispectral Scanner (MSS) image are presented as examples. An algorithm for computing the frequency distribution of distinct spectral vectors in MSS data is given in the Appendix.

Wharton, S. W.↗
NASA NTRS1986

"Noiseless" Data-Compression Algorithm

Gamma-ray spectrometer data compressed to enable more frequent sampling. Proposed data-compression algorithm efficiently represents gamma-ray spectrometer spectra at any spectrum collection interval from 5 seconds to 5 minutes. Data representations "noiseless" (Data exactly constructed). Techniques useful in designing datacompression algorithms for other spectral instruments, which have varying data-rate requirements.

Rice, R. F.↗
DOE OSTI2021

Effects of raster terrain representation on GIS shortest path analysis

Spatial analysis extracts meaning and insights from spatially referenced data, where the results are highly dependent on the quality of the data used and the manipulations on the data when preparing it for analysis. Users should understand the impacts that data representations may have on their results in order to prevent distortions in their outcomes. We study the consequences of two common data preparations when locating a linear feature performing shortest path analysis on raster terrain data: 1) the connectivity of the network generated by connecting raster cells to their neighbors, and 2) the range of the attribute scale for assigning costs. Such analysis is commonly used to locate transmission lines, where the results could have major implications on project cost and its environmental impact. Experiments in solving biobjective shortest paths show that results are highly dependent on the parameters of the data representations, with exceedingly variable results based on the choices made in reclassifying attributes and generating networks from the raster. Based on these outcomes, we outline recommendations for ensuring geographic information system (GIS) data representations maintain analysis results that are accurate and unbiased.

Medrano, F. Antonio (ORCID:000000025913632X)↗
NASA NTRS2008

A Data Type for Efficient Representation of Other Data Types

A self-organizing, monomorphic data type denoted a sequence has been conceived to address certain concerns that arise in programming parallel computers. A sequence in the present sense can be regarded abstractly as a vector, set, bag, queue, or other construct. Heretofore, in programming a parallel computer, it has been necessary for the programmer to state explicitly, at the outset, what parts of the program and the underlying data structures must be represented in parallel form. Not only is this requirement not optimal from the perspective of implementation; it entails an additional requirement that the programmer have intimate understanding of the underlying parallel structure. The present sequence data type overcomes both the implementation and parallel structure obstacles. In so doing, the sequence data type provides unified means by which the programmer can represent a data structure for natural and automatic decomposition to a parallel computing architecture. Sequences exhibit the behavioral and structural characteristics of vectors, but the underlying representations are automatically synthesized from combinations of programmers advice and execution use metrics. Sequences can vary bidirectionally between sparseness and density, making them excellent choices for many kinds of algorithms. The novelty and benefit of this behavior lies in the fact that it can relieve programmers of the details of implementations. The creation of a sequence enables decoupling of a conceptual representation from an implementation. The underlying representation of a sequence is a hybrid of representations composed of vectors, linked lists, connected blocks, and hash tables. The internal structure of a sequence can automatically change from time to time on the basis of how it is being used. Those portions of a sequence where elements have not been added or removed can be as efficient as vectors. As elements are inserted and removed in a given portion, then different methods are utilized to provide both an access and memory strategy that is optimized for that portion and the use to which it is put.

James, Mark↗
NASA NTRS1988

Representation and matching of knowledge to design digital systems

A knowledge-based expert system is described that provides an approach to solve a problem requiring an expert with considerable domain expertise and facts about available digital hardware building blocks. To design digital hardware systems from their high level VHDL (Very High Speed Integrated Circuit Hardware Description Language) representation to their finished form, a special data representation is required. This data representation as well as the functioning of the overall system is described.

Jones, J. U.↗