SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
In this talk we will give an overview of the new features of the upcoming HDF5 release 1.12.0, and outline the HDF5 roadmap for the next year. We will demonstrate new open source file drivers to access HDF5 files via Amazon Simple Storage Service (Amazon S3) and on Hadoop Distributed File system (HDFS). We will use this presentation to get feedback on the HDF5 roadmap from the ESDIS users and application developers.
In this work, we are developing a lunar terrain database to enable fast rendering of sun illumination and earth visibility for a proposed coverage analysis tool. This development will advance lunar mission design and formulation for current and future communications architectures, and will aid in lunar surface mission planning and communications/navigation operations. Our effort can be described in three steps: (1) we parallelize a brute force algorithm, which computes elevation masks from laser altimetry data acquired by the Lunar Reconnaissance Orbiter’s (LRO) Lunar Orbiter Laser Altimeter (LOLA); (2) we investigate parallel I/O methods to store terrain mask information from step (1) into a parallel file system; and (3) we finally deliver data to the terrain coverage analysis tool.
Scientists generate petabytes of data daily to help uncover environmental trends or behaviors that are hard to predict. For example, understanding climate simulations based on the long-term average of temperature, precipitation, and other environmental variables is essential to predicting and establishing root causes of future undesirable scenarios and assessing possible mitigation strategies. While supercomputer centers provide a powerful infrastructure for generating petabytes of simulation output, accessing and analyzing these datasets interactively remains challenging on multiple fronts. This paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hid-ing the complexities of dealing with file systems or cloud services.We also optimize network utilization while streaming from petas-cale repositories through state-of-the-art progressive compression algorithms. Based on this abstraction, we provide customizable dashboards that can be accessed from any device with any inter-net connection, enabling interactive visual analysis of vast amounts of data to a wide range of users - from top scientists with access to leadership-class computing environments to undergraduate students of disadvantaged backgrounds from minority-serving institutions. We focus on NASA’s use of petascale climate datasets as an example of particular societal impact and, therefore, a case where achieving equity in science participation is critical. We validate our approach by improving the ability of climate scientists to visually explore their data via two fully interactive dashboards. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution.These dashboards, released in simplified form to the general public, contribute significantly to a broader push to democratize the access and use of climate data.
There is growing evidence in the scientifc computing community that parallel file systems are not sufficient for all HPC storage workloads. This realization has motivated extensive research in new storage system designs. The question of which alternative we should turn towards implies that there could be a single answer satisfying a wide range of very diverse applications. We argue that such a generic solution does not exist. Instead, custom data services should be designed and tailored to the needs of specific applications on specific hardware. Furthermore, custom data services should be designed in close collaboration with users. In this paper, we present a methodology for the rapid development of such data services. This methodology promotes the design of reusable building blocks that can be composed together efficiently through a runtime based on high-performance threading, tasking, and remote procedure calls. We illustrate the success of our methodology by showcasing three examples of data services designed from the same building blocks, yet targeting entirely different applications.
System interrogates and maintains user-definable data files from remote terminals, using English-like, free-form query language easily learned by persons not proficient in computer programming. System operates in asynchronous mode, allowing any number of inquiries within limitation of available core to be active concurrently.
In this paper, the author presents the minimum functions required for an on-board file management system. We explore file manipulation processes and demonstrate how the file transfer along with the file management system will be utilized to support flight operations and data delivery.
The objective of the Distributed Access View Integrated Database (DAVID) project is the development of an easy to use computer system with which NASA scientists, engineers and administrators can uniformly access distributed heterogeneous databases. Basically, DAVID will be a database management system that sits alongside already existing database and file management systems. Its function is to enable users to access the data in other languages and file systems without having to learn the data manipulation languages. Given here is an outline of a talk on the DAVID project and several charts.
MPISS is an automatic file transfer system that implements a combination of standard and mission-unique transfer protocols required by the Global Precipitation Measurement Mission (GPM) Precipitation Processing System (PPS) to control the flow of data between the MOC and the PPS. The primary features of MPISS are file transfers (both with and without PPS specific protocols), logging of file transfer and system events to local files and a standard messaging bus, short term storage of data files to facilitate retransmissions, and generation of file transfer accounting reports. The system includes a graphical user interface (GUI) to control the system, allow manual operations, and to display events in real time. The PPS specific protocols are an enhanced version of those that were developed for the Tropical Rainfall Measuring Mission (TRMM). All file transfers between the MOC and the PPS use the SSH File Transfer Protocol (SFTP). For reports and data files generated within the MOC, no additional protocols are used when transferring files to the PPS. For observatory data files, an additional handshaking protocol of data notices and data receipts is used. MPISS generates and sends to the PPS data notices containing data start and stop times along with a checksum for the file for each observatory data file transmitted. MPISS retrieves the PPS generated data receipts that indicate the success or failure of the PPS to ingest the data file and/or notice. MPISS retransmits the appropriate files as indicated in the receipt when required. MPISS also automatically retrieves files from the PPS. The unique feature of this software is the use of both standard and PPS specific protocols in parallel. The advantage of this capability is that it supports users that require the PPS protocol as well as those that do not require it. The system is highly configurable to accommodate the needs of future users.
The CCSDS File Delivery Protocol (CFDP) Standard could reshape ground support architectures by enabling applications to communicate over the space link using reliable-symmetric transport services. JPL utilized the CFDP standard to support the Deep Impact Mission. The architecture was based on layering the CFDP applications on top of the CCSDS Space Link Extension Services for data transport from the mission control centers to the ground stations. On July 4, 2005 at 1:52 A.M. EDT, the Deep Impact impactor successfully collided with comet Tempel 1. During the final 48 hours prior to impact, over 300 files were uplinked to the spacecraft, while over 6 thousand files were downlinked from the spacecraft using the CFDP. This paper uses the Deep Impact Mission as a case study in a discussion of the CFDP architecture, Deep Impact Mission requirements, and design for integrating the CFDP into the JPL deep space support services. Issues and recommendations for future missions using CFDP are also provided.
Disclosed are systems and software that provide a high-performance, extensible file format and web API for remote data access and a visual interface for data viewing, query, and analysis. The described system can support storage of raw spectroscopic data such as neural recording data, MSI data, metadata, and derived analyses in a single, self-describing format that may be compatible by a large range of analysis software.
The present invention provides a fail-over file transfer process to handle data file transfer when the transfer is unsuccessful in order to avoid unnecessary network congestion and enhance reliability in an automated data file transfer system. If a file cannot be delivered after attempting to send the file to a receiver up to a preset number of times, and the receiver has indicated the availability of other backup receiving locations, then the file delivery is automatically attempted to one of the backup receiving locations up to the preset number of times. Failure of the file transfer to one of the backup receiving locations results in a failure notification being sent to the receiver, and the receiver may retrieve the file from the location indicated in the failure notification when ready.
Errors in files detected and corrected during operation. Permanent File Validation (PFVAL) utility computer program provides CDC CYBER NOS sites with mechanism to verify integrity of permanent file base. Locates and identifies permanent file errors in Mass Storage Table (MST) and Track Reservation Table (TRT), in permanent file catalog entries (PFC's) in permit sectors, and in disk sector linkage. All detected errors written to listing file and system and job day files. Program operates by reading system tables , catalog track, permit sectors, and disk linkage bytes to vaidate expected and actual file linkages. Used extensively to identify and locate errors in permanent files and enable online correction, reducing computer-system downtime.
As mass storage technology becomes more affordable for sites smaller than supercomputer centers, understanding their file access patterns becomes crucial for developing systems to store rarely used data on tertiary storage devices such as tapes and optical disks. This paper presents a new way to collect and analyze file system statistics for UNIX-based file systems. The collection system runs in user-space and requires no modification of the operating system kernel. The statistics package provides details about file system operations at the file level: creations, deletions, modifications, etc. The paper analyzes four months of file system activity on a university file system. The results confirm previously published results gathered from supercomputer file systems, but differ in several important areas. Files in this study were considerably smaller than those at supercomputer centers, and they were accessed less frequently. Additionally, the long-term creation rate on workstation file systems is sufficiently low so that all data more than a day old could be cheaply saved on a mass storage device, allowing the integration of time travel into every file system.
The purpose of this project was to advance the existing ICESS/Bren School computing infrastructure to allow scientists, students, and research trainees the opportunity to interact with environmental data and simulations in near-real time. Improvements made with the funding from this project have helped to strengthen the research efforts within both units, fostered graduate research training, and helped fortify partnerships with government and industry. With this funding, we were able to expand our computational environment in which computer resources, software, and data sets are shared by ICESS/Bren School faculty researchers in all areas of Earth system science. All of the graduate and undergraduate students associated with the Donald Bren School of Environmental Science and Management and the Institute for Computational Earth System Science have benefited from the infrastructure upgrades accomplished by this project. Additionally, the upgrades fostered a significant number of research projects (attached is a list of the projects that benefited from the upgrades). As originally proposed, funding for this project provided the following infrastructure upgrades: 1) a modem file management system capable of interoperating UNIX and NT file systems that can scale to 6.7 TB, 2) a Qualstar 40-slot tape library with two AIT tape drives and Legato Networker backup/archive software, 3) previously unavailable import/export capability for data sets on Zip, Jaz, DAT, 8mm, CD, and DLT media in addition to a 622Mb/s Internet 2 connection, 4) network switches capable of 100 Mbps to 128 desktop workstations, 5) Portable Batch System (PBS) computational task scheduler, and vi) two Compaq/Digital Alpha XP1000 compute servers each with 1.5 GB of RAM along with an SGI Origin 2000 (purchased partially using funds from this project along with funding from various other sources) to be used for very large computations, as required for simulation of mesoscale meteorology or climate.
The TidGen Power System generates emission-free electricity from tidal currents and connects directly into existing grids using smart grid technology. The power system consists of three major subsystems: shore-side power electronics, mooring system, and turbine generator unit (TGU) device. This submission contains supporting CFD files, case files and geometry for the Advanced TidGen. TSR = Tip speed ratio Cp = Power coefficient Cl = Lift coefficient Cd = Drag coefficient
The performance gap between magnetic disks and data processing on HPC systems has become that huge that an efficient data processing can only be achieved by introducing non-volatile memory (NVRAM) as a new storage tier. Although the benefits of hierarchical storage have been adequately demonstrated to the point that the newest leadership class HPC systems will employ burst buffers, critical questions remain for supporting hierarchical storage systems, including: How should we present hierarchical storage systems to user applications, such that they are easy to use and that application code is portable across systems? How should we manage data movement through a storage hierarchy for best performance and resilience of data? How do the particular I/O use cases mandate the way we manage data? There have been many efforts to explore this space in the form of file systems, with increasingly more implemented at the user level. This is because it is relatively easy to swap in new, specialized user-level file systems for use by applications on a case-by-case basis, as opposed to the current mainstream approach of using general-purpose, system-level file systems which may not be optimized for HPC workloads and must be installed by administrators. In contrast, file systems at the user level can be tailored for specific HPC workloads for high performance and can be used by applications without administrator intervention. Many user-level file system developers have found themselves “having to reinvent the wheel” to implement various optimizations in their file systems. Thus, a main goal of this meeting was to bring together experts in I/O performance, file systems, and storage, and collectively explore the space of current and future problems and solutions for I/O on hierarchical storage systems in order to begin a community effort in enabling user-level file system support for HPC systems. We had a lively week of learning about each other’s approaches as well as unique I/O use cases that can influence the design of a community-driven file and storage system standards. The agenda for this meeting contained talks from participants on the following high level topics: HPC storage and I/O support today; what do HPC users need for I/O; existing userlevel file system efforts; object stores and other alternative storage systems; and components for building user-level file systems. The talks were short and intended to be conversation starters for more in-depth discussions with the whole group. The participants engaged in lengthy discussions on various questions that arose from the talks including: Are we ready to program to a memory hierarchy versus block devices? Are the needs of HPC users reflected in our existing file systems and storage systems? Should we drop or keep POSIX moving forward? What do we mean when we say “user-level file system”? Do we all mean the same thing? How should the IO 500 benchmark be defined so it is fair and useful? and How are stage-in and stage-out actually going to work? The report for this seminar contains a record of the talks from the participants as well as the resulting discussions. Our hope is that the effort initiated during this seminar will result in long-term collaborations that will benefit the HPC community as a whole.
Modern piloted flight simulators require detailed models of many aircraft components, such as the airframe, propulsion system, flight deck controls and instrumentation, as well as motion drive and visual display systems. The amount of computing power necessary to implement these systems can exceed that offered by dedicated mainframe computers. One approach to this problem is through the use of distributed computing, where parts of the simulation are assigned to computing subsystems, such as microcomputers. One such subsystem, such as microcomputers. One such subsystem, a real-time, portable, microcomputer-based jet engine simulator, is described in this paper. The simulator will be used at the NASA Ames Vertical Motion Simulator facility to perform calculations previously done on the facility's mainframe computer. The mainframe will continue to do all other system calculations and will interface to the engine simulator through analog I/0. The engine simulator hardware includes a 16-bit microcomputer and floating-point coprocessor. There is an 8 channel analog input board and an 8 channel analog output board. A model of a small turboshaft engine/control is coded in floating-point FORTRAN. The FORTRAN code and a data monitoring program run under the control of an assembly language real-time executive. The monitoring program allows the user to isplay and/or modify simulator variables on-line through a data terminal. A dual disk drive system is used for mass storage of programs and data. The CP/M-86 operating system provides file management and overall system control. The frame time for the simulator is 30 milliseconds, which includes all analog I/0 operations.
Representing water head information in power system model files, can provide a more realistic model of the system and thereby inform operation and planning personnel in the decision-making process. This article describes a procedure for modifying the power system model files (steady-state and dynamic) to represent water head information. Additionally, the impact of representing the water head on power system reliability studies including contingency analysis, cascading failure analysis and dynamic frequency response analysis has been investigated, using the modified power system models. This paper considers the detailed Western Electricity Coordination Council model during summer and winter conditions as the test system for the impact analysis. Results show that under reduced water head: 1) the number of critical voltage and branch flow violations increases; 2) chances of cascading failure and island formation increases; and 3) frequency nadir decreases as compared to those of the base cases where the water head information is not represented.