Search NASA⌕ Search

SEARCH · Search NASA

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

Promoting Data Stewardship Through Best Practices

The ecological and environmental sciences are comprised of many different disciplines, each with their own methods, theories, and culture. A characteristic that most of these different disciplines share, however, is a lack of culture for good stewardship of data. Characteristics of good data stewardship include understanding the importance of data management, using best practices for managing data, and recognizing the value of data sharing and data reuse for the future of ecology and the environmental sciences. The Data Observation Network for Earth (DataONE) is actively developing a community database of best practices that can be easily accessed and adopted by scientists to promote good data stewardship practices and lead to high quality data products. Here we introduce DataONE s approach to developing the best practices database and provide a data management primer that contains examples relevant to all elements of the data life cycle.

Strasser, C. A.↗

SciDAC - Center for Simulation of Wave Interactions with MHD -- General Atomics Support of ORNL Collaboration

The Center for Simulation of Wave Interactions with Magnetohydrodynamics (SWIM) project is dedicated to conduct research on integrated multi-physics simulations. The Integrated Plasma Simulator (IPS) is a framework that was created by the SWIM team. It provides an integration infrastructure for loosely coupled component-based simulations by facilitating services for code execution coordination, computational resource management, data management, and inter-component communication. The IPS framework features improving resource utilization, implementing application-level fault tolerance, and support of the concurrent multi-tasking execution model. The General Atomics (GA) team worked closely with other team members on this contract, and conducted research in the areas of computational code monitoring, meta-data management, interactive visualization, and user interfaces. The original website to monitor SWIM activity was developed in the beginning of the project. Due to the amended requirements, the software was redesigned and a revision of the website was deployed into production in April of 2010. Throughout the duration of this project, the SWIM Monitoring Portal (http://swim.gat.com:8080/) has been a critical production tool for supporting the project's physics goals.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

How to handle 6GBytes a night and not get swamped

The Macho Project has undertaken a 5 year effort to search for dark matter in the halo of the Galaxy by scanning the Magellanic Clouds for micro-lensing events. Each evening's raw image data will be reduced in real-time into the observed stars' photometric measurements. The actual search for micro-lensing events will be a post-processing operation. The theoretical prediction of the rate of such events necessitates the collection of a large number of repeated exposures. The project designed camera subsystem delivers 64 Mbytes per exposure with exposures typically occurring every 500 seconds. An ideal evening's observing will provide 6 Gbytes of raw image data and 40 Mbytes of reduced photometric measurements. Recognizing the difficulty of digging out from a snowballing cascade of raw data, the project requires the real-time reduction of each evening's data. The software team's implementation strategy centered on this non-negotiable mandate. Accepting the reality that 2 full time people needed to implement the core real-time control and data management system within 6 months, off-the-shelf vendor components were explored to provide quick solutions to the classic needs for file management, data management, and process control. Where vendor solutions were lacking, state-of-the-art models were used for hand tailored subsystems. In particular, petri nets manage process control, memory mapped bulletin boards provide interprocess communication between the multi-tasked processes, and C++ class libraries provide memory mapped, disk resident databases. The differences between the implementation strategy and the final implementation reality are presented. The necessity of validating vendor product claims are explored. Both the successful and hindsight decisions enabling the collection and processing of the nightly data barrage are reviewed.

Allsman, R.↗

Dynamo: Handling Scientific Data Across Sites and Storage Media

Abstract Dynamo is a full-stack software solution for scientific data management. Dynamo’s architecture is modular, extensible, and customizable, making the software suitable for managing data in a wide range of installation scales, from a few terabytes stored at a single location to hundreds of petabytes distributed across a worldwide computing grid. This article documents the core system design of Dynamo and describes the applications that implement various data management tasks. A brief report is also given on the operational experiences of the system at the CMS experiment at the CERN Large Hadron Collider and at a small-scale analysis facility.

Iiyama, Yutaro (ORCID:0000000282975930)↗

Relational Information Management Data-Base System

DBMS with several features particularly useful to scientists and engineers. RIM5 interfaced with any application program written in language capable of Calling FORTRAN routines. Applications include data management for Space Shuttle Columbia tiles, aircraft flight tests, high-pressure piping, atmospheric chemistry, census, university registration, CAD/CAM Geometry, and civil-engineering dam construction.

Storaasli, O. O.↗

Data base management study

Data base management techniques and applicable equipment are described. Recommendations which will assist potential NASA data users in selecting and using appropriate data base management tools and techniques are presented. Classes of currently available data processing equipment ranging from basic terminals to large minicomputer systems were surveyed as they apply to the needs of potential SEASAT data users. Cost and capabilities projections for this equipment through 1985 were presented. A test of a typical data base management system was described, as well as the results of this test and recommendations to assist potential users in determining when such a system is appropriate for their needs. The representative system tested was UNIVAC's DMS 1100.

Source record↗

Design versus manufacturing data base management requirements

Data base management systems are valuable manufacturing and design tools as these disciplines are exceptionally information intensive, requiring precise organization and control of data processing and utilization. One such data base manager is the IPAD* system, which was originally developed to support the design process but was expanded to incorporate the additional needs of manufacturing. To set the stage, an overview of the design and manufacturing process is presented. The different functions of computers in these processes are then discussed. Finally, the design and manufacturing requirements for a data base manager are compared and contrasted.

Mckenna, E. G.↗

Dynamic object management for distributed data structures

In distributed-memory multiprocessors, remote memory accesses incur larger delays than local accesses. Hence, insightful allocation and access of distributed data can yield substantial performance gains. The authors argue for the use of dynamic data management policies encapsulated within individual distributed data structures. Distributed data structures offer performance, flexibility, abstraction, and system independence. This approach is supported by data from a trace-driven simulation study of parallel scientific benchmarks. Experimental data on memory locality, message count, message volume, and communication delay suggest that data-structure-specific data management is superior to a single, system-imposed policy.

Totty, Brian K.↗

WATCHMAN: A Data Warehouse Intelligent Cache Manager

Data warehouses store large volumes of data which are used frequently by decision support applications. Such applications involve complex queries. Query performance in such an environment is critical because decision support applications often require interactive query response time. Because data warehouses are updated infrequently, it becomes possible to improve query performance by caching sets retrieved by queries in addition to query execution plans. In this paper we report on the design of an intelligent cache manager for sets retrieved by queries called WATCHMAN, which is particularly well suited for data warehousing environment. Our cache manager employs two novel, complementary algorithms for cache replacement and for cache admission. WATCHMAN aims at minimizing query response time and its cache replacement policy swaps out entire retrieved sets of queries instead of individual pages. The cache replacement and admission algorithms make use of a profit metric, which considers for each retrieved set its average rate of reference, its size, and execution cost of the associated query. We report on a performance evaluation based on the TPC-D and Set Query benchmarks. These experiments show that WATCHMAN achieves a substantial performance improvement in a decision support environment when compared to a traditional LRU replacement algorithm.

Scheuermann, Peter↗

A software package for the data-independent management of multidimensional data

The Common Data Format (CDF), a structure which provides true data independence for applications software and has been developed at the National Space Science Data Center, is discussed. The background to the CDF is reviewed, and the CDF is described. The conceptual organization of the CDF is discussed, and a sample CDF structure is shown and described. The implementation of CDF, its status, and its applications are examined.

Treinish, Lloyd A.↗

A Science Data Gateway for Environmental Management

Science data gateways are effective in providing complex science data collections to the world-wide user Q1 communities. In this paper we describe a gateway for the Advanced Simulation Capability for Environmental Management (ASCEM) framework. Built on top of established web service technologies, the ASCEM data gateway is specifically designed for environmental modeling applications. Its key distinguishing feature sinclude (1) handling of complex spatiotemporal data, (2) offering a variety of selective data access mechanisms, (3) providing state-of-the-art plotting and visualization of spatiotemporal data records, and (4) integrating seamlessly with a distributed workflow system using a RESTful interface. ASCEM projectscientists have been using this data gateway since 2011.

ASCEM, Akuna, Database Management↗

Standard Energy Efficiency Data Platform

The SEED platform is expected to be a building energy performance data management tool that provides federal, state and local governments, building owners and operators with an easy, flexible and cost-effective method to collect information about groups of buildings, oversee compliance with energy disclosure laws and demonstrate the economic and environmental benefits of energy efficiency. It will allow users to leverage a local application to manage data disclosure and large data sets without the IT investment of developing custom applications. The first users of SEED will be agencies that need to collect, store, and report/share large data sets generated by benchmarking, energy auditing, retro-commissioning or retrofitting of many buildings. Similarly, building owners and operators will use SEED to manage their own energy data in a common format and centralized location. SEED users will also control the disclosure of their information for compliance requirements, recognition programs such as ENERGY STAR, or data sharing with the Buildings Performance Database and/or other third parties at their discretion.

Cheifetz, D.Magnus↗

Standard Energy Efficiency Data Platform

The SEED platform is expected to be a building energy performance data management tool that provides federal, state and local governments, building owners and operators with an easy, flexible and cost-effective method to collect information about groups of buildings, oversee compliance with energy disclosure laws and demonstrate the economic and environmental benefits of energy efficiency. It will allow users to leverage a local application to manage data disclosure and large data sets without the IT investment of developing custom applications. The first users of SEED will be agencies that need to collect, store, and report/share large data sets generated by benchmarking, energy auditing, retro-commissioning or retrofitting of many buildings. Similarly, building owners and operators will use SEED to manage their own energy data in a common format and centralized location. SEED users will also control the disclosure of their information for compliance requirements, recognition programs such as ENERGY STAR, or data sharing with the Buildings Performance Database and/or other third parties at their discretion.

Cheifetz, D. Magnus↗

The Fabric for Frontier Experiments Project at Fermilab

The FabrIc for Frontier Experiments (FIFE) project is a new, far-reaching initiative within the Fermilab Scientific Computing Division to drive the future of computing services for experiments at FNAL and elsewhere. It is a collaborative effort between computing professionals and experiment scientists to produce an end-to-end, fully integrated set of services for computing on the grid and clouds, managing data, accessing databases, and collaborating within experiments. FIFE includes 1) easy to use job submission services for processing physics tasks on the Open Science Grid and elsewhere, 2) an extensive data management system for managing local and remote caches, cataloging, querying, moving, and tracking the use of data, 3) custom and generic database applications for calibrations, beam information, and other purposes, 4) collaboration tools including an electronic log book, speakers bureau database, and experiment membership database. All of these aspects will be discussed in detail. FIFE sets the direction of computing at Fermilab experiments now and in the future, and therefore is a major driver in the design of computing services worldwide.

Kirby, Michael↗

Review of City Energy and Emissions Analysis Needs, Methods, and Tools

Currently over $300B is spent in US city economies to pay for energy. Many US cities are taking leading roles in exploring and promoting activities to improve energy efficiency and reduce green house gas (GHG) emissions. This paper summarizes a series of interviews with several leading cities regarding their needs, methods and tools they are using to model energy use and evaluate policies to reduce GHG. We also present a review of several analysis tools evaluated and used to explore urban scale design scenarios for two new major developments in the San Francisco area. We found that cities face great challenges managing data on their building stock, obtaining energy use data, and evaluating the different tools that are available to them. There is a need for better data management systems that allow tools to be more interoperable. The wide variety and features of today’s tools, and the fact that many of them are not open data models, create sub-optimal environments to conduct the energy analysis many cities are seeking to conduct.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Data base management systems activities

The Data Management System-1100 is designed to operate in conjunction with the UNIVAC 1100 Series Operating System on any 1100 Series computer. DMS-1100 is divided into the following four major software components: (1) Data Definition Languages (DDL); (2) Data Management Routine (DMR); (3) Data Manipulation Languages (DML); and (4) Data Base Utilities (DBU). These software components are described in detail.

Source record↗

Recent evolutions in the LTDP CDF project

In the latest years, CNAF (the national center of the Italian Institute for Nuclear Physics INFN dedicated to Research and Development on Information and Communication Technologies) has been working on the Long Term Data Preservation (LTDP) project for the CDF experiment, active at Fermilab from 1990 to 2011. The main aims of the project are to protect the most relevant part of the CDF RUN-2 data collected between 2001 and 2011 and already stored on tape at CNAF (4 PB), as well as to ensure the availability and the access to the analysis facility to those data over time. Lately, the CDF database, hosting information about CDF datasets such as their structure, file locations and metadata, has been imported from Fermilab to CNAF. Also, the Sequential Access via Metadata (SAM) station data handling tool for CDF data management, that allows to manage data transfers and to retrieve information from the CDF database, has been properly installed and configured at CNAF. This was a fundamental step in the perspective of a complete decommissioning of CDF services on Fermilab side. An access system has been designed and tested to submit CDF analysis jobs, using CDF software distributed via CERN Virtual Machine File System (CVMFS) and requesting delivery of CDF files stored on CNAF tapes, as well as data present only on Fermilab storage archive. Moreover, the availability and the correctness of all CDF data stored on CNAF tapes has been verified. This paper describes all these recent evolutions in detail, presenting the future plans for the LTDP project at CNAF.

Fornari, Federico↗