SEARCH · Search NASA

Results for “Data archiving software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

All sources NASA NTRS DOE OSTI

NASA NTRS2013

Hyperspectral Microwave Atmospheric Sounder (HyMAS) Architecture and Design Accommodations

The Hyperspectral Microwave Atmospheric Sounder (HyMAS) is being developed at Lincoln Laboratories and accommodated by the Goddard Space Flight Center for a flight opportunity on a NASA research aircraft. The term "hyperspectral microwave" is used to indicate an all-weather sounding that performs equivalent to hyperspectral infrared sounders in clear air with vertical resolution of approximately 1 km. Deploying the HyMAS equipped scanhead with the existing Conical Scanning Microwave Imaging Radiometer (CoSMIR) shortens the path to a flight demonstration. Hyperspectral microwave is achieved through the use of independent RF antennas that sample the volume of the Earth s atmosphere through various levels of frequencies, thereby producing a set of dense, spaced vertical weighting functions. The simulations proposed for HyMAS 118/183-GHz system should yield surface precipitation rate and water path retrievals for small hail, soft hail, or snow pellets, snow, rainwater, etc. with accuracies comparable to those of the Advanced Technology Microwave Sounder. Further improvements in retrieval methodology (for example, polarization exploitation) are expected. The CoSMIR instrument is a packaging concept re-used on HyMAS to ease the integration features of the scanhead. The HyMAS scanhead will include an ultra-compact Intermediate Frequency Processor (IFP) module that is mounted inside the door to improve thermal management. The IFP is fabricated with materials made of Low-Temperature Co-fired Ceramic (LTCC) technology integrated with detectors, amplifiers, A/D conversion and data aggregation. The IFP will put out 52 channels of 16 bit data comprised of 4-9 channel data streams for temperature profiles and 2-8 channel streams for water vapor. With the limited volume of the existing CoSMIR scanhead and new HyMAS front end components, the HyMAS team at Goddard began preliminary layout work inside the new drum. Importing and re-using models of the shell, the scan head computer, and the slip rings developed for CoSMIR was the starting point. The next step was to modify the antenna faceplate to accommodate the dimensions of the three dual polarization Gaussian Optics Antenna (GOA) assemblies. Two mechanical concepts for the core technology, the hyperspectral IFP, were captured in a design tradeoff. Connector models considered minimum bend radii for the IFP analog connectors. Hyperspectral imaging is accomplished by strategically using a short wavelength intermediate frequency of 18-29 GHz, and thus reducing the size of components in the connection of the front end to the IFP. The SMK (2.92mm) Series connector will lay near the hinge line to minimize its flexing. The digital output of the IFP will use a Serial Peripheral Interface (SPI) that must be accommodated by the scan head computer. To make that computer more reliable, maintainable, and forward compatible with the 52 HyMAS channels, a testbed of the scan head, calibration, and archive computers and the PIC24 microprocessor that resides on the IFP is in development. The computers will be programmed using a new framework application called Interoperable Remote Component (IRC). This software allows flexibility to program computers that communicate with each other and can adapt easily to the emerging HyMAS requirements for data format, algorithms, and graphical user interface (GUI). It is expected that the CoSMIR instrument will cut over to the IRC after it is adapted on an updated CoSMIR testbed.

Hilliard, Lawrence↗

NASA NTRS2001

Simple, Scalable, Script-Based Science Processor (S4P)

The development and deployment of data processing systems to process Earth Observing System (EOS) data has proven to be costly and prone to technical and schedule risk. Integration of science algorithms into a robust operational system has been difficult. The core processing system, based on commercial tools, has demonstrated limitations at the rates needed to produce the several terabytes per day for EOS, primarily due to job management overhead. This has motivated an evolution in the EOS Data Information System toward a more distributed one incorporating Science Investigator-led Processing Systems (SIPS). As part of this evolution, the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) has developed a simplified processing system to accommodate the increased load expected with the advent of reprocessing and launch of a second satellite. This system, the Simple, Scalable, Script-based Science Processor (S42) may also serve as a resource for future SIPS. The current EOSDIS Core System was designed to be general, resulting in a large, complex mix of commercial and custom software. In contrast, many simpler systems, such as the EROS Data Center AVHRR IKM system, rely on a simple directory structure to drive processing, with directories representing different stages of production. The system passes input data to a directory, and the output data is placed in a "downstream" directory. The GES DAAC's Simple Scalable Script-based Science Processing System is based on the latter concept, but with modifications to allow varied science algorithms and improve portability. It uses a factory assembly-line paradigm: when work orders arrive at a station, an executable is run, and output work orders are sent to downstream stations. The stations are implemented as UNIX directories, while work orders are simple ASCII files. The core S4P infrastructure consists of a Perl program called stationmaster, which detects newly arrived work orders and forks a job to run the appropriate executable (registered in a configuration file for that station). Although S4P is written in Perl, the executables associated with a station can be any program that can be run from the command line, i.e., non-interactively. An S4P instance is typically monitored using a simple Graphical User Interface. However, the reliance of S4P on UNIX files and directories also allows visibility into the state of stations and jobs using standard operating system commands, permitting remote monitor/control over low-bandwidth connections. S4P is being used as the foundation for several small- to medium-size systems for data mining, on-demand subsetting, processing of direct broadcast Moderate Resolution Imaging Spectroradiometer (MODIS) data, and Quick-Response MODIS processing. It has also been used to implement a large-scale system to process MODIS Level 1 and Level 2 Standard Products, which will ultimately process close to 2 TB/day.

Lynnes, Christopher↗

NASA NTRS1994

Managing People's Data

Just imagine a mass storage system that consists of a machine with 2 CPUs, 1 Gigabyte (GB) of memory, 400 GB of disk space, 16800 cartridge tapes in the automated tape silos, 88,000 tapes located in the vault, and the software to manage the system. This system is designed to be a data repository; it will always have disk space to store all the incoming data. Currently 9.14 GB of new data per day enters the system with this rate doubling each year. To assure there is always disk space available for new data, the system. has to move data reside from the expensive disk to a much less expensive medium such as the 3480 cartridge tapes. Once the data is archived to tape, it should be able to move back to disk when someone wants to access it and the data movement should be transparent to the user. Now imagine all the tasks that a system administrator must perform to keep this system running 24 hour a day, 7 days a week. Since the filesystem maintains the illusion of unlimited disk space, data that comes to the system must get moved to tapes in an efficient manner. This paper will describe the mass storage system running at the Numerical Aerodynamic Simulation (NAS) at NASA Ames Research Center in both software and hardware aspects, then it will describe all of the tasks the system administrator has to perform on this system.

Le, Diana↗

NASA NTRS1993

LDEF materials special investigation group's data bases

The Long Duration Exposure Facility (LDEF) was composed of and contained a wide array of materials, representing the largest collection of materials flown for space exposure and returned for ground-based analyses to date. The results and implications of the data from these materials are the foundation on which future space missions will be built. The LDEF Materials Special Investigation Group (MSIG) has been tasked with establishing and developing data bases to document these materials and their performance to assure not only that the data are archived for future generations but also that the data are available to the space user community in an easily accessed, user-friendly form. The format and content of the data bases developed or being developed to accomplish this task are discussed. The hardware and software requirements for each of the three data bases are discussed along with current availability of the data bases.

Strickland, John W.↗

NASA NTRS1989

A programmable image compression system

A programmable image compression system which has the necessary flexibility to address diverse imaging needs is described. It can compress and expand single frame video images (monochrome or color) as well as documents and graphics (black and white or color) for archival or transmission applications. Through software control, the compression mode can be set for lossless or controlled quality coding; the image size and bit depth can be varied; and the image source and destination devices can be readily changed. Despite the large combination of image data types, image sources, and algorithms, the system provides a simple consistent interface to the programmer. This system (OPTIPAC) is based on the TITMS320C25 digital signal processing (DSP) chip and has been implemented as a co-processor board for an IBM PC-AT compatible computer. The underlying philosophy can readily be applied to different hardware platforms. By using multiple DSP chips or incorporating algorithm specific chips, the compression and expansion times can be significantly reduced to meet performance requirements.

Farrelle, Paul M.↗

NASA NTRS1991

Data systems and computer science programs: Overview

An external review of the Integrated Technology Plan for the Civil Space Program is presented. The topics are presented in viewgraph form and include the following: onboard memory and storage technology; advanced flight computers; special purpose flight processors; onboard networking and testbeds; information archive, access, and retrieval; visualization; neural networks; software engineering; and flight control and operations.

Smith, Paul H.↗

NASA NTRS1993

LDEF materials data bases

The Long Duration Exposure Facility (LDEF) and the accompanying experiments were composed of and contained a wide variety of materials representing the largest collection of materials flown in low Earth orbit (LEO) and retrieved for ground based analysis to date. The results and implications of the mechanical, thermal, optical, and electrical data from these materials are the foundation on which future LEO space missions will be built. The LDEF Materials Special Investigation Group (MSIG) has been charged with establishing and developing data bases to document these materials and their performance to assure not only that the data are archived for future generations but also that the data are available to the spacecraft user community in an easily accessed, user-friendly form. This paper discusses the format and content of the three data bases developed or being developed to accomplish this task. The hardware and software requirements for each of these three data bases are discussed along with current availability of the data bases. This paper also serves as a user's guide to the MAPTIS LDEF Materials Data Base.

Funk, Joan G.↗

NASA NTRS2006

EOS MLS Science Data Processing System: A Description of Architecture and Capabilities

This paper describes the architecture and capabilities of the Science Data Processing System (SDPS) for the EOS MLS. The SDPS consists of two major components--the Science Computing Facility and the Science Investigator-led Processing System. The Science Computing Facility provides the facilities for the EOS MLS Science Team to perform the functions of scientific algorithm development, processing software development, quality control of data products, and scientific analyses. The Science Investigator-led Processing System processes and reprocesses the science data for the entire mission and delivers the data products to the Science Computing Facility and to the Goddard Space Flight Center Earth Science Distributed Active Archive Center, which archives and distributes the standard science products.

Microwave Limb Sounder (MLS)↗

NASA NTRS1993

High-performance mass storage system for workstations

Reduced Instruction Set Computer (RISC) workstations and Personnel Computers (PC) are very popular tools for office automation, command and control, scientific analysis, database management, and many other applications. However, when using Input/Output (I/O) intensive applications, the RISC workstations and PC's are often overburdened with the tasks of collecting, staging, storing, and distributing data. Also, by using standard high-performance peripherals and storage devices, the I/O function can still be a common bottleneck process. Therefore, the high-performance mass storage system, developed by Loral AeroSys' Independent Research and Development (IR&D) engineers, can offload a RISC workstation of I/O related functions and provide high-performance I/O functions and external interfaces. The high-performance mass storage system has the capabilities to ingest high-speed real-time data, perform signal or image processing, and stage, archive, and distribute the data. This mass storage system uses a hierarchical storage structure, thus reducing the total data storage cost, while maintaining high-I/O performance. The high-performance mass storage system is a network of low-cost parallel processors and storage devices. The nodes in the network have special I/O functions such as: SCSI controller, Ethernet controller, gateway controller, RS232 controller, IEEE488 controller, and digital/analog converter. The nodes are interconnected through high-speed direct memory access links to form a network. The topology of the network is easily reconfigurable to maximize system throughput for various applications. This high-performance mass storage system takes advantage of a 'busless' architecture for maximum expandability. The mass storage system consists of magnetic disks, a WORM optical disk jukebox, and an 8mm helical scan tape to form a hierarchical storage structure. Commonly used files are kept in the magnetic disk for fast retrieval. The optical disks are used as archive media, and the tapes are used as backup media. The storage system is managed by the IEEE mass storage reference model-based UniTree software package. UniTree software will keep track of all files in the system, will automatically migrate the lesser used files to archive media, and will stage the files when needed by the system. The user can access the files without knowledge of their physical location. The high-performance mass storage system developed by Loral AeroSys will significantly boost the system I/O performance and reduce the overall data storage cost. This storage system provides a highly flexible and cost-effective architecture for a variety of applications (e.g., realtime data acquisition with a signal and image processing requirement, long-term data archiving and distribution, and image analysis and enhancement).

Chiang, T.↗

NASA NTRS1985

Planetary image conversion task

The Planetary Image Conversion Task group processed 12,500 magnetic tapes containing raw imaging data from JPL planetary missions and produced an image data base in consistent format on 1200 fully packed 6250-bpi tapes. The output tapes will remain at JPL. A copy of the entire tape set was delivered to US Geological Survey, Flagstaff, Ariz. A secondary task converted computer datalogs, which had been stored in project specific MARK IV File Management System data types and structures, to flat-file, text format that is processable on any modern computer system. The conversion processing took place at JPL's Image Processing Laboratory on an IBM 370-158 with existing software modified slightly to meet the needs of the conversion task. More than 99% of the original digital image data was successfully recovered by the conversion task. However, processing data tapes recorded before 1975 was destructive. This discovery is of critical importance to facilities responsible for maintaining digital archives since normal periodic random sampling techniques would be unlikely to detect this phenomenon, and entire data sets could be wiped out in the act of generating seemingly positive sampling results. Reccomended follow-on activities are also included.

Martin, M. D.↗

NASA NTRS1992

High performance compression of science data

In the future, NASA expects to gather over a tera-byte per day of data requiring space for levels of archival storage. Data compression will be a key component in systems that store this data (e.g., optical disk and tape) as well as in communications systems (both between space and Earth and between scientific locations on Earth). We propose to develop algorithms that can be a basis for software and hardware systems that compress a wide variety of scientific data with different criteria for fidelity/bandwidth tradeoffs. The algorithmic approaches we consider are specially targeted for parallel computation where data rates of over 1 billion bits per second are achievable with current technology.

Storer, James A.↗

NASA NTRS1993

High performance compression of science data

Storer, James A.↗

NASA NTRS1996

ACTS (Advanced Communications Technology Satellite) Propagation Experiment: Preprocessing Software User's Manual

The preprocessing software manual describes the Actspp program originally developed to observe and diagnose Advanced Communications Technology Satellite (ACTS) propagation terminal/receiver problems. However, it has been quite useful for automating the preprocessing functions needed to convert the terminal output to useful attenuation estimates. Prior to having data acceptable for archival functions, the individual receiver system must be calibrated and the power level shifts caused by ranging tone modulation must be received. Actspp provides three output files: the daylog, the diurnal coefficient file, and the file that contains calibration information.

Crane, Robert K.↗

NASA NTRS2019

Data Preservation: Final Step in the Life Cycle of a Mission

Many NASA Earth Observing System (EOS) have either already reached the end of their active life or are nearing it. Preservmissionsation of data products is a fairly well defined task for the NASA EOS Data Centers or DAACs.The Goddard Earth Sciences Data and Information Services Center (GES-DISC) has implemented a repository system, which is capable of long-term archive of documentation artifacts and other associated digital content. The existing GES-DISC Repository System is based on Fedora Commons, an open-source repository management software, for cost savings and flexibility.The first mission to utilize the GES-DISC Repository System was the High Resolution Dynamics Limb Sounder (HIRDLS) on the Aura spacecraft. Since then, the GES DISC has gathered documentation from the UARS and TOMS into the Repository. The Microwave Limb Sounder (MLS) team has begun delivering some early pre-launch documents to the GES-DISC Repository System as well. Other missions in planning or progress include AIRS, OMI, SORCE, SNPP Sounder, and TRMM.

Johnson, J.↗

NASA NTRS1984

Studies of lightning data in conjunction with geostationary satellite data

The archiving of a more complete data base to perform lightning studies is outlined. This effort includes: (1) continued archiving of bureau of land management LLP, geostationary satellite, and NWS radar data; (2) expansion of the McIDAS real-time LLP access to other networks. Additional processing tools for display and analysis of lightning location data in conjunction with geostationary satellite data are developed which entails adapting existing McIDAS software to allow the production of statistical summaries and contouring of lightning characteristics over user defined areas or storms. The plotting of three dimensional displays of lightning statistics versus satellite and radar data, and the performance of an error analysis of lightning location data using overlapping regions of the BLM network are discussed.

Suchman, D.↗

NASA NTRS1994

The Rings Node for the Planetary Data System

The Planetary Data System's Rings Node is devoted to the archiving and distributing of scientific data sets relevant to planetary ring systems. The two major classes of ring data are images and occultation profiles, although a variety of additional data types (e.g. spectra, particle absorption signatures, etc.) are also of interest. A large fraction of our data sets are from the Voyager missions to the outer planets, but Earth-based and Hubble Space Telescope data sets are also represented. Archiving work often includes re-formatting the data into standardized formats and reconstructing some of the data processing steps. The Rings Node also performs a variety of services to support research into these data sets. These services include developing on-line catalogs and information systems, filling orders for data, developing software tools, and coordinating special observing campaigns.

Showalter, Mark R.↗

NASA NTRS2008

Sequence History Update Tool

The Sequence History Update Tool performs Web-based sequence statistics archiving for Mars Reconnaissance Orbiter (MRO). Using a single UNIX command, the software takes advantage of sequencing conventions to automatically extract the needed statistics from multiple files. This information is then used to populate a PHP database, which is then seamlessly formatted into a dynamic Web page. This tool replaces a previous tedious and error-prone process of manually editing HTML code to construct a Web-based table. Because the tool manages all of the statistics gathering and file delivery to and from multiple data sources spread across multiple servers, there is also a considerable time and effort savings. With the use of The Sequence History Update Tool what previously took minutes is now done in less than 30 seconds, and now provides a more accurate archival record of the sequence commanding for MRO.

Khanampompan, Teerapat↗

NASA NTRS1995

Interplanetary space science data base and access/display tool on the NSSDC heliospheric CD-ROM

The National Space Science Data Center (NSSDC) has accumulated a rich archive of heliospheric, magnetospheric, and ionospheric data, as well as data from most other NASA-involved science disciplines. To facilitate access to and use of these data, NSSDC has begun to put selected data onto CD-ROM's. This paper describes one such CD-ROM, and the access and display software developed at NSSDC to support its use. The data on the CD-ROM consist primarily of hourly solar wind magnetic field and plasma data from many near-Earth spacecraft (OMNI) and deep space spacecraft (Voyagers, Pioneers, Helios, Pioneer Venus Orbiter). In addition, 5-minute resolution IMP-8 and ISEE-3 magnetic field and plasma data are also included. Data are stored in both ASCII and CDF formats.

Papitashvili, N. E.↗