LibFeature: a software library for quickly generating feature vectors on the fly from structured data
We have developed a software library, LibFeature, that greatly simplifies the task of extracting feature vectors from raw data.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
We have developed a software library, LibFeature, that greatly simplifies the task of extracting feature vectors from raw data.
The Fibonacci grid, proposed by Swinbank and Purser (see companion abstract), provides attractive properties for global numerical atmospheric prediction by offering an optimally homogeneous, geometrically regular, and approximately isotropic discretization, with only the polar regions requiring special numerical treatment. It is a mathematical idealization, applied to the sphere, of the multi-spiral patterns often found in botanical structures, such as in pine cones and sunflower heads. Computationally, it is natural to organize the domain, into zones, in each of which the same pair, or triple, of "Fibonacci spirals" dominate. But the further subdivision of such zones into "tiles" of a shape and size suitable for distribution to the processors of a massively parallel computer requires very careful consideration if the subsequent spatial computations along the respective spirals, especially those computations (such as compact differencing schemes) that involve recursion, can be implemented in an efficient "load-balanced "manner without requiring excessive amounts of inter-processor communications. In this paper we show how certain "number theoretic" properties of the Fibonacci sequence (whose numbers prescribe the multiplicity of successive spirals) may be exploited in the decomposition of grid zones into tidy arrangements of triangular grid tiles, each tile possessing one side approximately parallel to the constant-latitude zone boundary. We also describe how the spatially recursive processes may be decomposed across such a tiling, and the directionality of the recursions reversed on alternate grid lines, to ensure a very high degree of load balancing throughout the execution of the computations required for one time step of a global model.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
In HPC applications, memory access behavior is one of the main factors affecting performance. Improving an application’s memory access behavior involves optimizing data layout and/or restructuring code, and requires studying spatial-temporal data locality. Existing data locality analyses focus on single-location metrics and are restricted to evaluating temporal locality. We introduce spatial-temporal affinity metrics that quantify temporal access proximity, forward access correlation, and nearby access correlation between pairs of memory locations. We describe methods for distinguishing between potential vs. realized affinity and for reasoning about affinity at multiple resolutions (3D, 2D, 1D). Finally, we construct spatial-temporal affinity signatures that classify memory behavior and that be used to reason about changes in software (data relayout, code refactoring) or hardware (caching, prefetching). We describe methods for signature visualization, interpretation, and quantitative comparison of signatures. We evaluate our methodology using applications with variants that contrast data structures, data layouts and algorithms. We show that spatial-temporal affinity analysis provides novel insights and enables predictive reasoning about application performance when contrasted with reuse distance analysis.
Consideration is given to the hardware and software needed to manage the enormous amount and complexity of data that the next generation of space-borne sensors will provide. An anthology is presented illustrating the evolution of artificial intelligence, science data processing, and management from the 1960s to the near future. Problems and limitations of technologies, data structures, data standards, and conceptual thinking are addressed. The development of an end-to-end Intelligent Information Fusion System that embodies knowledge of the user's domain-specific goals is proposed.
Recognizing the significance of NASA remote sensing Earth science data in monitoring and better understanding our planet s natural environment, NASA has implemented the Decision Support Through Earth Science Research Results program (NASA ROSES solicitations). a) This successful program has yielded several monitoring, surveillance, and decision support systems through collaborations with benefiting organizations. b) The Goddard Space Flight Center (GSFC) Earth Sciences Data and Information Services Center (GES DISC) has participated in this program on two projects (one complete, one ongoing), and has had opportune ad hoc collaborations gaining much experience in the formulation, management, development, and implementation of decision support systems utilizing NASA Earth science data. c) In addition, GES DISC s understanding of Earth science missions and resulting data and information, including data structures, data usability and interpretation, data interoperability, and information management systems, enables the GES DISC to identify challenges that come with bringing science data to decision makers. d) The purpose of this presentation is to share GES DISC decision support system project experiences in regards to system sustainability, required data quality (versus timeliness), data provider understanding of how decisions are made, and the data receivers willingness to use new types of information to make decisions, as well as other topics. In addition, defining metrics that really evaluate success will be exemplified.
No abstract available
An open source computer program has been developed to satisfy a need for simplified organization of structured input data for scientific simulation programs. Typically, such input data are parsed in from a flat American Standard Code for Information Interchange (ASCII) text file into computational data structures. Also typically, when a graphical user interface (GUI) is used, there is a need to completely duplicate the input information while providing it to a user in a more structured form. Heretofore, the duplication of the input information has entailed duplication of software efforts and increases in susceptibility to software errors because of the concomitant need to maintain two independent input-handling mechanisms. The present program implements a method in which the input data for a simulation program are completely specified in an Extensible Markup Language (XML)-based text file. The key benefit for XML is storing input data in a structured manner. More importantly, XML allows not just storing of data but also describing what each of the data items are. That XML file contains information useful for rendering the data by other applications. It also then generates data structures in the C++ language that are to be used in the simulation program. In this method, all input data are specified in one place only, and it is easy to integrate the data structures into both the simulation program and the GUI. XML-to-C is useful in two ways: 1. As an executable, it generates the corresponding C++ classes and 2. As a library, it automatically fills the objects with the input data values.
This recommendation contains the detailed specification of the logic required to carry out the Command Operations Procedures of the Transfer Layer. The Recommendation for Telecommand--Part 2, Data Routing Service contains the standard data structures and data communication procedures used by the intermediate telecommand system layers (the Transfer and Segmentation Layers). In particular, it contains a brief description of the Command Operations Procedures (COP) within the Transfer Layer. This recommendation contains the detailed definition of the COP's in the form of state tables, along with definitions of the terms used. It is assumed that the reader of this document is familiar with the data structures and terminology of part 2. In case of conflict between the description of the COP's in part 2 and in this recommendation, the definition in this recommendation will take precedence. In particular, this document supersedes section 4.3.3.1 through 4.3.3.4 of part 2.
Unsupervised clustering is a fundamental building block in numerous image processing applications. One of the most popular and widely used clustering schemes for remote sensing applications is the ISOCLUS algorithm, which is based on the ISODATA method. The algorithm is given a set of n data points in d-dimensional space, an integer k indicating the initial number of clusters, and a number of additional parameters. The general goal is to compute the coordinates of a set of cluster centers in d-space, such that those centers minimize the mean squared distance from each data point to its nearest center. This clustering algorithm is similar to another well-known clustering method, called k-means. One significant feature of ISOCLUS over k-means is that the actual number of clusters reported might be fewer or more than the number supplied as part of the input. The algorithm uses different heuristics to determine whether to merge lor split clusters. As ISOCLUS can run very slowly, particularly on large data sets, there has been a growing .interest in the remote sensing community in computing it efficiently. We have developed a faster implementation of the ISOCLUS algorithm. Our improvement is based on a recent acceleration to the k-means algorithm of Kanungo, et al. They showed that, by using a kd-tree data structure for storing the data, it is possible to reduce the running time of k-means. We have adapted this method for the ISOCLUS algorithm, and we show that it is possible to achieve essentially the same results as ISOCLUS on large data sets, but with significantly lower running times. This adaptation involves computing a number of cluster statistics that are needed for ISOCLUS but not for k-means. Both the k-means and ISOCLUS algorithms are based on iterative schemes, in which nearest neighbors are calculated until some convergence criterion is satisfied. Each iteration requires that the nearest center for each data point be computed. Naively, this requires O(kn) time, where k denotes the current number of centers. Traditional techniques for accelerating nearest neighbor searching involve storing the k centers in a data structure. However, because of the iterative nature of the algorithm, this data structure would need to be rebuilt with each new iteration. Our approach is to store the data points in a kd-tree data structure. The assignment of points to nearest neighbors is carried out by a filtering process, which successively eliminates centers that can not possibly be the nearest neighbor for a given region of space. This algorithm is significantly faster, because large groups of data points can be assigned to their nearest center in a single operation. Preliminary results on a number of real Landsat datasets show that our revised ISOCLUS-like scheme runs about twice as fast.
The following topics are discussed: requirements for dynamic mesh adaption; linked-list data structure; edge-based data structure; adaptive-grid data structure; three types of element subdivision; mesh refinement; mesh coarsening; additional constraints for coarsening; anisotropic error indicator for edges; unstructured-grid Euler solver; inviscid 3-D wing; and mesh quality for solution-adaptive grids. The discussion is presented in viewgraph form.
Applying binaural simulation techniques to structural acoustic data can be very computationally intensive as the number of discrete noise sources can be very large. Typically, Head Related Transfer Functions (HRTFs) are used to individually filter the signals from each of the sources in the acoustic field. Therefore, creating a binaural simulation implies the use of potentially hundreds of real time filters. This paper details two methods of reducing the number of real-time computations required by: (i) using the singular value decomposition (SVD) to reduce the complexity of the HRTFs by breaking them into dominant singular values and vectors and (ii) by using equivalent source reduction (ESR) to reduce the number of sources to be analyzed in real-time by replacing sources on the scale of a structural wavelength with sources on the scale of an acoustic wavelength. The ESR and SVD reduction methods can be combined to provide an estimated computation time reduction of 99.4% for the structural acoustic data tested. In addition, preliminary tests have shown that there is a 97% correlation between the results of the combined reduction methods and the results found with the current binaural simulation techniques
The authors describe the capabilities of McIDAS, an interactive visualization system that is vastly increasing the ability of earth scientists to manage and analyze data from remote sensing instruments and numerical simulation models. McIDAS provides animated three-dimensionsal images and highly interactive displays. The software can manage, analyze, and visualize large data sets that span many physical variables (such as temperature, pressure, humidity, and wind speed), as well as time and three spatial dimensions. The McIDAS system manages data from at least 100 different sources. The data management tools consist of data structures for storing different data types in files, libraries of routines for accessing these data structures, system commands for performing housekeeping functions on the data files, and reformatting programs for converting external data to the system's data structures. The McIDAS tools for three-dimensional visualization of meteorological data run on an IBM mainframe and can load up to 128-frame animation sequences into the workstations. A highly interactive version of the system can provide an interactive window into data sets containing tens of millions of points produced by numerical models and remote sensing instruments. The visualizations are being used for teaching as well as by scientists.
XB-70-1 mass, structural, and aerodynamic data were updated to reflect as closely as possible the characteristics of the airplane at three specific flight conditions which were actually flown; a nominal Mach number of 0.90 at an altitude of 25,000 feet (two cases) and a nominal Mach number of 1.6 at an altitude of 40,000 feet (one case). In-flight response characteristics at a number of points on the vehicle were obtained by exciting a pair of shaker vanes on the nose of the airplane. Data were recorded with the basic stability augmentation system (SAS) operating both alone and together with the identical location of accelerometer and force (ILAF) structural mode control system. Detailed total vehicle weight, mass characteristics, structural frequencies, generalized masses, all aerodynamic data used in the present analyses, and a description of the actual mode shapes are tabulated and presented.
Statistical model for application of maneuver flight loads data to structural design data
Structure of crust and upper part of earth mantle from geophysical data
Development of quadtree as hierarchical data structuring technique for representing spatial data (like points, regions, surfaces, lines, curves, volumes, etc.) has been motivated to a large extent by storage requirements of images, maps, and other multidimensional (spatially structured) data. For many spatial algorithms, time-efficiency of quadtrees in terms of execution may be as important as their space-efficiency concerning storage conditions. Briefly, the quadtree is a class of hierarchical data structures which is based on the recursive partition of a square region into quadrants and sub-quadrants until a predefined limit. Beyond the wide applicability of quadtrees in image processing, spatial information analysis, and building digital databases (processes becoming ordinary for the astronomical community), there may be numerous further applications in astronomy. Some of these practicable applications based on quadtree representation of astronomical data are presented and suggested for further considerations. Examples are shown for use of point as well as region quadtrees. Statistics of different leaf and non-leaf nodes (homogeneous and heterogeneous sub-quadrants respectively) at different levels may provide useful information on spatial structure of astronomical data in question. By altering the principle guiding the decomposition process, different types of spatial data may be focused on. Finally, a sampling method based on quadtree representation of an image is proposed which may prove to be efficient in the elaboration of sampling strategy in a region where observations were carried out previously either with different resolution or/and in different bands.