Search NASA⌕ Search

SEARCH · Search NASA

Results for “database management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

X-1 to X-Wings: Developing a Parametric Cost Model

In todays cost-constrained environment, NASA needs an X-Plane database and parametric cost model that can quickly provide rough order of magnitude predictions of cost from initial concept to first fight of potential X-Plane aircraft. This paper takes a look at the steps taken in developing such a model and reports the results. The challenges encountered in the collection of historical data and recommendations for future database management are discussed. A step-by-step discussion of the development of Cost Estimating Relationships (CERs) is then covered.

parametric cost model↗

Implementing Connected Component Labeling as a User Defined Operator for SciDB

We have implemented a flexible User Defined Operator (UDO) for labeling connected components of a binary mask expressed as an array in SciDB, a parallel distributed database management system based on the array data model. This UDO is able to process very large multidimensional arrays by exploiting SciDB's memory management mechanism that efficiently manipulates arrays whose memory requirements far exceed available physical memory. The UDO takes as primary inputs a binary mask array and a binary stencil array that specifies the connectivity of a given cell to its neighbors. The UDO returns an array of the same shape as the input mask array with each foreground cell containing the label of the component it belongs to. By default, dimensions are treated as non-periodic, but the UDO also accepts optional input parameters to specify periodicity in any of the array dimensions. The UDO requires four stages to completely label connected components. In the first stage, labels are computed for each subarray or chunk of the mask array in parallel across SciDB instances using the weighted quick union (WQU) with half-path compression algorithm. In the second stage, labels around chunk boundaries from the first stage are stored in a temporary SciDB array that is then replicated across all SciDB instances. Equivalences are resolved by again applying the WQU algorithm to these boundary labels. In the third stage, relabeling is done for each chunk using the resolved equivalences. In the fourth stage, the resolved labels, which so far are "flattened" coordinates of the original binary mask array, are renamed with sequential integers for legibility. The UDO is demonstrated on a 3-D mask of O(1011) elements, with O(108) foreground cells and O(106) connected components. The operator completes in 19 minutes using 84 SciDB instances.

UDO↗

Autonomous landmark based spacecraft navigation system

An autonomous landmark based spacecraft navigation scheme is presented. This new schme involves the following data processing steps: image selection and planning; landmark detection; preliminary image matching; crater matching; database management; and finally landmark based orbit determination.

autonomous↗

Leveraging STARE for Co-aligned Data Locality with netCDF and Python MPI

We have leveraged STARE indexing to package partitioned data chunks from diverse datasets into netCDF files, distributed them on a cluster of 16 lightweight nodes with their placements spatiotemporally co-aligned, and demonstrated a few integrative analyses using netCDF parallel I/O and Python MPI, with single-user performance and scalability comparable to, or even better than, that of a parallel array database management system (ADBMS) such as SciDB. However, records of the node location and STARE index ranges for each data chunk, similar to the chunk maps of SciDB, must be maintained and consulted by the I/O and analysis code for coordinating the analytic operations in parallel, in order to achieve the good performance and scalability.

Kwo-Sen Kuo↗

Parallel In Situ Indexing for Data-intensive Computing

As computing power increases exponentially, vast amount of data is created by many scientific re- search activities. However, the bandwidth for storing the data to disks and reading the data from disks has been improving at a much slower pace. These two trends produce an ever-widening data access gap. Our work brings together two distinct technologies to address this data access issue: indexing and in situ processing. From decades of database research literature, we know that indexing is an effective way to address the data access issue, particularly for accessing relatively small fraction of data records. As data sets increase in sizes, more and more analysts need to use selective data access, which makes indexing an even more important for improving data access. The challenge is that most implementations of in- dexing technology are embedded in large database management systems (DBMS), but most scientific datasets are not managed by any DBMS. In this work, we choose to include indexes with the scientific data instead of requiring the data to be loaded into a DBMS. We use compressed bitmap indexes from the FastBit software which are known to be highly effective for query-intensive workloads common to scientific data analysis. To use the indexes, we need to build them first. The index building procedure needs to access the whole data set and may also require a significant amount of compute time. In this work, we adapt the in situ processing technology to generate the indexes, thus removing the need of read- ing data from disks and to build indexes in parallel. The in situ data processing system used is ADIOS, a middleware for high-performance I/O. Our experimental results show that the indexes can improve the data access time up to 200 times depending on the fraction of data selected, and using in situ data processing system can effectively reduce the time needed to create the indexes, up to 10 times with our in situ technique when using identical parallel settings.

97 MATHEMATICS AND COMPUTING↗

PARLO: PArallel Run-Time Layout Optimization for Scientific Data Explorations with Heterogeneous Access Pattern

Download Citation Email Print Request Permissions Save to Project The size and scope of cutting-edge scientific simulations are growing much faster than the I/O and storage capabilities of their run-time environments. The growing gap is exacerbated by exploratory, data-intensive analytics, such as querying simulation data with multivariate, spatio-temporal constraints, which induces heterogeneous access patterns that stress the performance of the underlying storage system. Previous work addresses data layout and indexing techniques to improve query performance for a single access pattern, which is not sufficient for complex analytics jobs. We present PARLO a parallel run-time layout optimization framework, to achieve multi-level data layout optimization for scientific applications at run-time before data is written to storage. The layout schemes optimize for heterogeneous access patterns with user-specified priorities. PARLO is integrated with ADIOS, a high-performance parallel I/O middleware for large-scale HPC applications, to achieve user-transparent, light-weight layout optimization for scientific datasets. It offers simple XML-based configuration for users to achieve flexible layout optimization without the need to modify or recompile application codes. Experiments show that PARLO improves performance by 2 to 26 times for queries with heterogeneous access patterns compared to state-of-the-art scientific database management systems. Compared to traditional post-processing approaches, its underlying run-time layout optimization achieves a 56% savings in processing time and a reduction in storage overhead of up to 50%. PARLO also exhibits a low run-time resource requirement, while also limiting the performance impact on running applications to a reasonable level.

Gong, Zhenhuan↗

Monitoring the Long-Term Performance of Engineered Containment Systems for Radioactive Waste: What can We Learn From Ecological Monitoring Approaches?

Due to technical and economic limitations, chemical and radioactive wastes will require isolation in engineered containment facilities at many sites including the former nuclear weapons facilities within the Department of Energy (DOE) complex. Also, while our experience with currently favored designs is only a few decades at best, the materials being isolated may require effective containment for hundreds to thousands of years. It is becoming clear that without monitoring of both the engineered barriers and the associated institutional controls and intervention, engineered containment systems will ultimately fail. Monitoring of engineered barriers is often limited to groundwater monitoring and physical inspections. The database managed by the DOE Office of Legacy Management provides information concerning barrier performance at uranium mill tailings sites over several years to decades. The current monitoring approach that focuses on groundwater, while necessary for regulatory compliance, does not provide an early warning. Rather, it provides detection of barrier failure as opposed to indications of barrier degradation. Identification and monitoring of precursors to failure are needed. Furthermore, existing monitoring strategies typically omit ecological monitoring. Additional understanding of the interaction between barrier degradation and ecological dynamics could provide a basis for monitoring and managing barriers as well. Monitoring the long-term performance of systems of interest is a challenge not only to the Department of Energy but to other institutions and federal agencies. National Environmental Research Parks (NERP), National Estuarine Research Reserves (NERR), and Long Term Ecological Research (LTER) areas operated by the US government agencies are designed with the specific goal of intensely monitoring and conducting research over long time scales and within diverse geographical settings. This presentation will address the importance of understanding ecological interactions as they are expressed over decades and centuries in respect to federal long-term monitoring programs, and to delineate how multifaceted sets of ecological data can be used to interpret the overall integrity of ecosystems. Furthermore, important components of these programs will be identified that may have merit in the design of long-term monitoring programs for hazardous chemical and radioactive waste sites. (authors)

54 ENVIRONMENTAL SCIENCES↗

3D Visualisation of Un-georeferenced, Coordinate-poor Spatial Data in Complex D and D and WM Environments - 18294

This paper describes the development of an algorithm-based utility tool using Visual Basic that examines un-georeferenced or GPS-deprived spatial datasets (3D datasets lacking one or more of the key x, y and z components). Taking into account any known geometrical parameters which are input via a user-friendly front-end, the new code automatically evaluates attributes, including any spatial coordinate components, detects invalid survey data points and fills in bad or missing data points utilising data-specific interpolation (e.g. cubic or linear spline) of valid data points, such that local extrema are not created. Previously troublesome 2D data can be easily converted to realistic 3D contour display, integrated with other time-series data to create 4D movies, and incorporated into other existing 3D datasets. More importantly those processed and 'more complete' data can be easily stored in 3D or 4D spatial database management systems for future interrogation by using clear, realistic and intuitive 3D visualisation. Test cases are presented to demonstrate the value and versatility of the newly developed utility tool which include the 3D visualisation of: surveyed wall thicknesses of a helical coil, radiation mapping of a plant and aerial survey mapping of radiation in a region near Fukushima. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Explicitly Accounting for Multiple Spatial Scales in Data Informing Kd Distribution Development - 18414

This work provides an example of incorporating information about spatial scale into input parameter distribution development for probabilistic performance assessment (PA) models and highlights the importance of having an interdisciplinary team working together in the distribution development process. The novel approach is demonstrated for contaminant partitioning coefficients for water/rock (Kds), providing a specific context for discussing the approach and challenges, but the strategy is general and applies to many other PA model inputs. In this example, a PA model of a radioactive waste disposal site is assumed to have large cells representing spatial scales of up to thousands of square meters in area and several to tens of meters in depth. Spatial variability in Kds is expected to occur at scales much smaller than the units in the model domain over which values are applied, and therefore values are intended to represent possible averages over the modeled volume (i.e., spatial scale represented by a PA model cell). Information about possible Kd values for each material category comes from a variety of literature sources, and the reported results used to inform the distribution are available at a wide range of scales (e.g., individual batch experiment results from small-scale samples, averages based on samples collected over site-scales, averages based on samples from large regions, etc.). Effectively addressing the mismatch in spatial scale between available information and the PA Model is a challenge within PA distribution development and it takes effort from database managers, subject matter experts, modelers, and statisticians. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Data Parallel Bin-Based Indexing for Answering Queries on Multi-Core Architectures

The multi-core trend in CPUs and general purpose graphics processing units (GPUs) offers new opportunities for the database community. The increase of cores at exponential rates is likely to affect virtually every server and client in the coming decade, and presents database management systems with a huge, compelling disruption that will radically change how processing is done. This paper presents a new parallel indexing data structure for answering queries that takes full advantage of the increasing thread-level parallelism emerging in multi-core architectures. In our approach, our Data Parallel Bin-based Index Strategy (DP-BIS) first bins the base data, and then partitions and stores the values in each bin as a separate, bin-based data cluster. In answering a query, the procedures for examining the bin numbers and the bin-based data clusters offer the maximum possible level of concurrency; each record is evaluated by a single thread and all threads are processed simultaneously in parallel. We implement and demonstrate the effectiveness of DP-BIS on two multi-core architectures: a multi-core CPU and a GPU. The concurrency afforded by DP-BIS allows us to fully utilize the thread-level parallelism provided by each architecture--for example, our GPU-based DP-BIS implementation simultaneously evaluates over 12,000 records with an equivalent number of concurrently executing threads. In comparing DP-BIS's performance across these architectures, we show that the GPU-based DP-BIS implementation requires significantly less computation time to answer a query than the CPU-based implementation. We also demonstrate in our analysis that DP-BIS provides better overall performance than the commonly utilized CPU and GPU-based projection index. Finally, due to data encoding, we show that DP-BIS accesses significantly smaller amounts of data than index strategies that operate solely on a column's base data; this smaller data footprint is critical for parallel processors that possess limited memory resources (e.g., GPUs).

97 MATHEMATICS AND COMPUTING↗

VIEWCACHE: An incremental pointer-based access method for autonomous interoperable databases

One of the biggest problems facing NASA today is to provide scientists efficient access to a large number of distributed databases. Our pointer-based incremental data base access method, VIEWCACHE, provides such an interface for accessing distributed datasets and directories. VIEWCACHE allows database browsing and search performing inter-database cross-referencing with no actual data movement between database sites. This organization and processing is especially suitable for managing Astrophysics databases which are physically distributed all over the world. Once the search is complete, the set of collected pointers pointing to the desired data are cached. VIEWCACHE includes spatial access methods for accessing image datasets, which provide much easier query formulation by referring directly to the image and very efficient search for objects contained within a two-dimensional window. We will develop and optimize a VIEWCACHE External Gateway Access to database management systems to facilitate database search.

Roussopoulos, N.↗

VIEWCACHE: An incremental pointer-based access method for autonomous interoperable databases

One of biggest problems facing NASA today is to provide scientists efficient access to a large number of distributed databases. Our pointer-based incremental database access method, VIEWCACHE, provides such an interface for accessing distributed data sets and directories. VIEWCACHE allows database browsing and search performing inter-database cross-referencing with no actual data movement between database sites. This organization and processing is especially suitable for managing Astrophysics databases which are physically distributed all over the world. Once the search is complete, the set of collected pointers pointing to the desired data are cached. VIEWCACHE includes spatial access methods for accessing image data sets, which provide much easier query formulation by referring directly to the image and very efficient search for objects contained within a two-dimensional window. We will develop and optimize a VIEWCACHE External Gateway Access to database management systems to facilitate distributed database search.

Roussopoulos, N.↗

A Prescribed Fire Emission Factors Database for Land Management and Air Quality Applications

The Clean Air Act, its amendments, and air quality regulations require that prescribed fire managers estimate the quantity of emissions that a prescribed fire will produce. Information on emissions is available for these calculations; however, it is often incomplete or difficult to find. Tables and computer models can also provide some of this information, but the quality and applicability of the data to a specific site are unknown. In conjunction with three research projects developing new emissions data and meteorological tools to assist prescribed fire managers, the Resource Conservation and Climate Change Program Area of the Department of Defense's Strategic Environmental Research and Development Program is supporting development of a database that contains emissions information related to prescribed burning. Ultimately the vetted database will be available on the Internet and will contain emissions information that has been developed from laboratory and field-scale measurements and has been published.

Lincoln, Emily↗

A Database for Radwaste Management in Germany-30 Years of Experience by TUV NORD - 20461

From the very beginning in the late 1980's TUV NORD EnSys is involved as an independent expert in quality assurance processes concerning interim storage facilities and the disposal for radioactive waste with negligible heat generation in Germany. A database has been developed since these times by TUV NORD EnSys. The development of this database over a long time will be discussed as well as the lessons learned within this process. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Applications of interest : a relational database approach to managing control system software applications.

Large accelerator facilities such as the Advanced Photon Source (APS) typically are operated by a diverse set of integrated control systems, such as front-end controllers, PLCs, and FPGAs. This type of control system structure encompasses numerous engineering documents, distributed real-time control system databases, source code, user displays, and other components. The complexity of the control system is further increased as the life cycle of a control system is never ending, change is constant. And the accelerator itself generates new operational problems on a regular basis. This overall controls environment begs the question of how best to provide a means for control system engineers to easily and quickly troubleshoot unique functions of the control system, find relevant information, and understand the impact of changes to one part of the control system on other applications. The answer to this question lies in being able to associate pertinent drawings, manuals, source code, hardware, and expert developers in an efficient and logical manner. Applications of Interest is a relational database software tool created for the purpose of providing alternative views of the supporting information behind each distinct control system application at the APS.

43 PARTICLE ACCELERATORS↗

EXTENSIBLE DATABASE FRAMEWORK FOR MANAGEMENT OF UNSTRUCTURED AND SEMI-STRUCTURED DOCUMENTS

Method and system for querying a collection of Unstructured or semi-structured documents to identify presence of, and provide context and/or content for, keywords and/or keyphrases. The documents are analyzed and assigned a node structure, including an ordered sequence of mutually exclusive node segments or strings. Each node has an associated set of at least four, five or six attributes with node information and can represent a format marker or text, with the last node in any node segment usually being a text node. A keyword (or keyphrase) is specified. and the last node in each node segment is searched for a match with the keyword. When a match is found at a query node, or at a node determined with reference to a query node, the system displays the context andor the content of the query node.

Gawdiak, Yuri O.↗