Search NASA⌕ Search

SEARCH · Search NASA

Results for “File System”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Java Library for Input and Output of Image Data and Metadata

A Java-language library supports input and output (I/O) of image data and metadata (label data) in the format of the Video Image Communication and Retrieval (VICAR) image-processing software and in several similar formats, including a subset of the Planetary Data System (PDS) image file format. The library does the following: It provides low-level, direct access layer, enabling an application subprogram to read and write specific image files, lines, or pixels, and manipulate metadata directly. Two coding/decoding subprograms ("codecs" for short) based on the Java Advanced Imaging (JAI) software provide access to VICAR and PDS images in a file-format-independent manner. The VICAR and PDS codecs enable any program that conforms to the specification of the JAI codec to use VICAR or PDS images automatically, without specific knowledge of the VICAR or PDS format. The library also includes Image I/O plugin subprograms for VICAR and PDS formats. Application programs that conform to the Image I/O specification of Java version 1.4 can utilize any image format for which such a plug-in subprogram exists, without specific knowledge of the format itself. Like the aforementioned codecs, the VICAR and PDS Image I/O plug-in subprograms support reading and writing of metadata.

Deen, Robert↗
NASA NTRS2013

Dtest Testing Software

This software runs a suite of arbitrary software tests spanning various software languages and types of tests (unit level, system level, or file comparison tests). The dtest utility can be set to automate periodic testing of large suites of software, as well as running individual tests. It supports distributing multiple tests over multiple CPU cores, if available. The dtest tool is a utility program (written in Python) that scans through a directory (and its subdirectories) and finds all directories that match a certain pattern and then executes any tests in that directory as described in simple configuration files.

Jain, Abhinandan↗
NASA NTRS2018

More Efficient Precipitation Rate Data Acquisition with the STORM Swath-Based Analysis Tool

A multitude of locations exist for acquiring satellite-derived precipitation rate data. Because the volume of data exceeds what can reasonably be analyzed manually, online tools have been developed to help researchers make more informed data acquisition decisions. Some interfaces provide sample images generated from prioritized variables to enable efficient visual inspection, while others include the ability to subset geographically to reduce the volume of the data to the area of interest. Building upon such value-added services is the STORM (the NASA Precipitation Processing System (PPS) data product ordering system) Swath-Based Analysis Tool (https://storm.pps.eosdis.nasa.gov/storm/Analysis.jsp). With it, users can compare overflights of multiple precipitation satellites simultaneously and from this make determinations about which swaths will best serve their research. They can preview these overflights with two distinct visualization tools. Finally, they can then download geographically subset swaths that have precipitation rate characteristics matching their research needs. A user starts by making up to six selections. They then select their date range of interest going back to 1997. Finally, they select a geographical region either by inputting latitude-longitude corners of a rectangle or drawing that rectangle on an interactive map. The system processes each file, determining aggregate statistics about the precipitation rates within the selected region as observed by each instrument. These statistical values are added to a time series graph as a "time remaining" counter is continuously updated. The chart is fully interactive, with detailed values displayed upon mouseover, and numerous options available upon clicking each point. Since the user is likely to need the relevant files for their own research, they can generate an order from their request. All files are geographically subset to the region of interest, and the user can use the statistical values to limit which files are added to the order. This flexibility helps to ensure researchers are only downloading the files they need for their work, making for a more efficient process both from a time-consumption and a storage perspective. This paper will introduce the Swath-Based Analysis Tool and explore its numerous features. It will discuss several use cases for precipitation scientists.

remote sensing↗
NASA NTRS2018

Heasim and Skyback Simulation Tools and Their Application to the Hitomi Mission

We present an introduction to the heasim multimission observation and skyback background, high-energy pseudo Monte Carlo astrophysical simulation tools. Heasim may be used to accurately and efficiently construct flexible image transport system (FITS) event files for simple or composite sources with a wide range of standard and user-defined spatial, spectral, and temporal characteristics. Skyback is designed to enable users to assess the impact of background discrete and diffuse emission on prospective observations, and skyback output may be directly input into heasim. We present a brief overview of heasim and skyback input, algorithms, usage, and output. We also introduce the sxsbranch tool that computes Hitomi soft X-ray spectrometer resolution grade branching ratios, emphasizing its application to simulations. We include several examples of particular relevance to the Hitomi mission.

Loewenstein, Michael↗
NASA NTRSDate not supplied

ASDC’s Python-Based Metadata Extraction Pipeline for Suborbital Campaigns

The FAIRness of data products, especially findability and accessibility depend on rich metadata which, when extracted, can allow for proper curation. Over the past few years, the Atmospheric Science Data Center (ASDC) suborbital science support team has developed a metadata extraction pipeline to ensure the required metadata can be retrieved systematically, effectively, and efficiently to ensure the data can be used by a broad community. The development of a pipeline has presented many, but necessary, challenges to support archival and distribution of ASDC’s 30+ suborbital missions. Though sufficient metadata is provided by instrument scientists, the metadata may not be readily machine actionable due to different formats and templates. Further complicating metadata extraction, our team has found that the nature of metadata can be quite diverse given the difference in measurement types, instruments, and measurement platforms. A metadata extraction pipeline has been developed to provide an efficient, plugin-in based, method for adding new parsers, a configuration system that lets non-developers customize how files are processed, and a system for identifying and logging metadata quality issues to ensure they are readily found and addressed. The metadata extraction pipeline identifies critical pieces of metadata that are needed to promote data FAIRness, including location, file revision, measurement start/end datetime and can be easily modified to extract further information (such as variables). Given the wide-ranging datasets, the pipeline has been modified to accommodate multiple file formats, including multiple versions of ICARTT (International Consortium for Atmospheric Research on Transport and Transformation), HDF (Hierarchical Data Format), netCDF (network Common Data Form), and multiple versions of the Ames File Format. The pipeline also supports building metadata for file formats that cannot have metadata easily extracted from them, such as PDF (Portable Document Format) and GIF (Graphics Interchange Format). The pipeline has allowed our team to maintain a consistent flow of data and metadata to archival and distribution services, ensuring the ASDC meets the needs of the suborbital science community. This presentation will highlight the ASDC’s suborbital metadata extraction pipeline, its development, how it’s been modified to support data FAIRness, and plans for maintaining the pipeline and adding new features.

Abraham Porter↗
DOE OSTI2020

Data Transfer Tool (HPC Annual Report V.3.1)

Sandia has been developing and supporting data transfer tools for over 20 years and has the expertise to take DOE into the Extreme Scale era. In looking at Exascale and beyond (Extreme Scale Computing), data sets can be thousands of 500TBs in size, a single file can be in the 100TB range, and billions of files are expected. Huge bursts of data need to be transferred, even today. While data archiving is often not thought about, it is an integral part of the full data management path when data is generated on HPC systems. In order to move generated data to its final resting place (data archive) or to transfer between file systems, a capable data transfer tool is required.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1996

Analysis of the access patterns at GSFC distributed active archive center

The Goddard Space Flight Center (GSFC) Distributed Active Archive Center (DAAC) has been operational for more than two years. Its mission is to support existing and pre Earth Observing System (EOS) Earth science datasets, facilitate the scientific research, and test Earth Observing System Data and Information System (EOSDIS) concepts. Over 550,000 files and documents have been archived, and more than six Terabytes have been distributed to the scientific community. Information about user request and file access patterns, and their impact on system loading, is needed to optimize current operations and to plan for future archives. To facilitate the management of daily activities, the GSFC DAAC has developed a data base system to track correspondence, requests, ingestion and distribution. In addition, several log files which record transactions on Unitree are maintained and periodically examined. This study identifies some of the users' requests and file access patterns at the GSFC DAAC during 1995. The analysis is limited to the subset of orders for which the data files are under the control of the Hierarchical Storage Management (HSM) Unitree. The results show that most of the data volume ordered was for two data products. The volume was also mostly made up of level 3 and 4 data and most of the volume was distributed on 8 mm and 4 mm tapes. In addition, most of the volume ordered was for deliveries in North America although there was a significant world-wide use. There was a wide range of request sizes in terms of volume and number of files ordered. On an average 78.6 files were ordered per request. Using the data managed by Unitree, several caching algorithms have been evaluated for both hit rate and the overhead ('cost') associated with the movement of data from near-line devices to disks. The algorithm called LRU/2 bin was found to be the best for this workload, but the STbin algorithm also worked well.

Johnson, Theodore↗
NASA NTRS1992

A proof-of-concept implementation of persistence in a hierarchical storage system

The concept of providing transparent access to a collection of files in a mass storage system is a familiar one. The goal of this project was to investigate the feasibility of providing similar access to a collection of persistent, complex objects. We describe an architecture for interfacing a persistent store of complex objects to a hierarchical storage system. Persistent object stores support the uniform creation, storage, and access of complex objects, regardless of their lifetimes. In other words, a mechanism is provided so that persistent objects outlive the processes which create them and can be accessed in a uniform manner by other processes. We validated this architecture by implementing a proof-of-concept system and testing the system on two stores of data. These tests indicate that this architecture supports the creation, storage, and access of very large persistent object stores.

Grossman, Robert↗
NASA NTRS1975

Evaluation of a data dictionary system

The usefulness was investigated of a data dictionary/directory system for achieving optimum benefits from existing and planned investments in computer data files in the Data Systems Development Branch and the Institutional Data Systems Division. Potential applications of the data catalogue system are discussed along with an evaluation of the system. Other topics discussed include data description, data structure, programming aids, programming languages, program networks, and test data.

Driggers, W. G.↗
NASA NTRS1993

Measurements over distributed high performance computing and storage systems

A strawman proposal is given for a framework for presenting a common set of metrics for supercomputers, workstations, file servers, mass storage systems, and the networks that interconnect them. Production control and database systems are also included. Though other applications and third part software systems are not addressed, it is important to measure them as well.

Williams, Elizabeth↗
NASA NTRS1978

CELFE: Coupled Eulerian-Lagrangian Finite Element program for high velocity impact. Part 2: Program user's manual

The CELFE computer program and user's manual, together with the execution of the CELFE/NASTRAN system, are described. The execution procedure and the transfer of data between the CELFE and NASTRAN programs are controlled through the use of DATA files in the Univac 1100 system. Five data files are used to control the runstream and data transfer, and three files are used to hold the programs. These files are contained on a single tape. Changes in NASTRAN routines required by the present analysis are also discussed in this report. All the program listings, except the last two files (where the absolute and relocatable elements are stored), are included in the appendixes.

Lee, C. H.↗
DOE OSTI2018

MyEMSL MTS File Cache Manager

MTS manager for retrieving files from MyEMSL; The MyEMSL MTS File Cache Manager is a part of PRISM, the Proteomics Research Information and Management System. The software processes the MyEMSL Download Queue in the Mass and Time Tag System (MTS) to download files required for processing.

Monroe, Matthew↗
DOE OSTI2024

A flexible data acquisition system architecture for the Nab experiment

Here, the Nab experiment will measure the electron–neutrino correlation and Fierz interference term in free neutron beta decay to test the Standard Model and probe Beyond the Standard Model physics. Using National Instrument’s PXIe-5171 Reconfigurable Oscilloscope module, we have developed a data acquisition system that is not only capable of meeting Nab’s specifications, but flexible enough to be adapted in situ as the experimental environment dictates. The L1 and L2 trigger logic can be reconfigured to optimize the system for coincidence event detection at runtime through configuration files and LabVIEW controls. This system is capable of identifying L1 triggers at a rate of at least 1 MHz, while reading out a peak signal rate of approximately 2 GB/s. During the commissioning phase of the experiment, the system ran at a sustained readout rate of 400 MB/s of detector signal data originating from roughly 6 kHz L2 triggers, well within the peak performance of the system.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

SLICE

For most of their lifetime, pebble-bed reactors (PBRs) operate at an equilibrium state in which the core is filled with fuel pebbles at various levels of burnup. A fuel pebble travels multiple times in so-called passes through the reactor before it reaches its target discharge burnup and is replaced with a fresh fuel pebble. Given the stochastic nature of the fuel pebble travel paths and consequently the individual fuel pebble histories, it is not possible with standard methods developed for traditional reactor concepts to calculate the fuel inventory in the reactor core. An iterative approach, the SCALE Leap-In method for Cores at Equilibrium (SLICE), was developed to generate region-average fuel inventory for a PBR. The SLICE code enables automatic generation of input files for the SCALE code system (https://www.ornl.gov/scale), management of the SCALE result files, and analysis of results.

Bostelmann, Friederike [Oak Ridge National Laborat↗
DOE OSTI2023

Usability and Optimization Improvements in MOOSE

The Multiphysics Object-Oriented Simulation Environment (MOOSE) framework is a foundational capability used by the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program to create over 15 different simulation tools for advanced nuclear reactors. Due to MOOSE’s broad use, improvements to the framework in support of modeling and simulation goals are critical to the program. Such improvements can take many forms, including optimization, improved user experience, streamlined application programming interfaces (APIs), parallelism, and new capabilities. The work described in this report was conducted in direct support of the simulation tools and has already been deployed. The capabilities were implemented in the same order as they are covered in this report: multiple nonlinear systems in the same input file, implementation of generic field transfers and other transfer system enhancements, support for stateful material property redistribution for adaptivity and distributed meshes, and dynamic linking and loading of individually compiled applications. These four additions are fundamental capabilities that will be leveraged by many NEAMS applications.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1970

Information retrieval system

Generalized information storage and retrieval system capable of generating and maintaining a file, gathering statistics, sorting output, and generating final reports for output is reviewed. File generation and file maintenance programs written for the system are general purpose routines.

Berg, R. F.↗
NASA NTRS1987

UNIX-based data management system for the Mobile Satellite Propagation Experiment (PiFEx)

A new method is presented for handling data resulting from Mobile Satellite propagation experiments such as the Pilot Field Experiment (PiFEx) conducted by JPL. This method uses the UNIX operating system and C programming language. The data management system is implemented on a VAX minicomputer. The system automatically divides the large data file housing data from various experiments under a predetermined format into various individual files containing data from each experiment. The system also has a number of programs written in C and FORTRAN languages to allow the researcher to obtain meaningful quantities from the data at hand.

Kantak, Anil V.↗