SEARCH · Search NASA

Results for “Data archiving software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

All sources NASA NTRS DOE OSTI

NASA NTRS2002

Digital echocardiography 2002: now is the time

The ability to acquire echocardiographic images digitally, store and transfer these data using the DICOM standard, and routinely analyze examinations exists today and allows the implementation of a digital echocardiography laboratory. The purpose of this review article is to outline the critical components of a digital echocardiography laboratory, discuss general strategies for implementation, and put forth some of the pitfalls that we have encountered in our own implementation. The major components of the digital laboratory include (1) digital echocardiography machines with network output, (2) a switched high-speed network, (3) a high throughput server with abundant local storage, (4) a reliable low-cost archive, (5) software to manage information, and (6) support mechanisms for software and hardware. Implementation strategies can vary from a complete vendor solution providing all components (hardware, software, support), to a strategy similar to our own where standard computer and networking hardware are used with specialized software for management of image and measurement information.

Review↗

NASA NTRS2017

Atmospheric Correction for Hyperspectral Ocean Color Retrieval with Application to the Hyperspectral Imager for the Coastal Ocean (HICO)

The classical multi-spectral Atmospheric Correction (AC) algorithm is inadequate for the new generation of spaceborne hyperspectral sensors such as NASA's first hyperspectral Ocean Color Instrument (OCI) onboard the anticipated Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) satellite mission. The AC process must estimate and remove the atmospheric path radiance contribution due to the Rayleigh scattering by air molecules and scattering by aerosols from the measured top-of-atmosphere (TOA) radiance, compensate for the absorption by atmospheric gases, and correct for reflection and refraction of the air-sea interface. In this work, we present and evaluate an improved AC for hyperspectral sensors developed within NASA's Sea-viewing Wide Field-of-view Sensor (SeaWiFS) Data Analysis System software package (SeaDAS). The improvement is based on combining the classical AC approach of multi-spectral capabilities to correct for the atmospheric path radiance, extended to hyperspectral, with a gas correction algorithm to compensate for absorbing gases in the atmosphere, including water vapor. The SeaDAS-hyperspectral version is capable of operationally processing the AC of any hyperspectral airborne or spaceborne sensor. The new algorithm development was evaluated and assessed using the Hyperspectral Imager for Coastal Ocean (HICO) scenes collected at the Marine Optical BuoY (MOBY) site, and other SeaWiFS Bio-optical Archive and Storage System (SeaBASS) and AERosol Robotic NETwork - Ocean Color (AERONET-OC) coastal sites. A hyperspectral vicarious calibration was applied to HICO, showing the validity and consistency of HICO's ocean color products. The hyperspectral AC capability is currently available in SeaDAS to the scientific community at https://oceancolor.gsfc.nasa.gov/.

Ocean Color↗

NASA NTRS2005

Testing the Archivas Cluster (Arc) for Ozone Monitoring Instrument (OMI) Scientific Data Storage

The Ozone Monitoring Instrument (OMI) launched on NASA's Aura Spacecraft, the third of the major platforms of the EOS program on July 15,2004. In addition to the long term archive and distribution of the data from OM1 through the Goddard Earth Science Distributed Active Archive Center (GESDAAC), we are evaluating other archive mechanisms that can archive the data in a more immediately available method where it can be used for futher data production and analysis. In 2004, Archivas, Inc. was selected by NASA s Small Business Innovative Research (SBIR) program for the development of their Archivas Cluster (ArC) product. Arc is an online disk based system utilizing self-management and automation on a Linux cluster. Its goal is to produce a low cost solution coupled with the ease of management. The OM1 project is an application partner of the SBIR program, and has deployed a small cluster (5TB) based on the beta Archwas software. We performed extensive testing of the unit using production OM1 data since launch. In 2005, Archivas, Inc. was funded in SBIR Phase II for further development, which will include testing scalability with the deployment of a larger (35TB) cluster at Goddard. We plan to include Arc in the OM1 Team Leader Computing Facility (TLCF) hosting OM1 data for direct access and analysis by the OMI Science Team. This presentation will include a brief technical description of the Archivas Cluster, a summary of the SBIR Phase I beta testing results, and an overview of the OMI ground data processing architecture including its interaction with the Phase II Archivas Cluster and hosting of OMI data for the scientists.

Tilmes, Curt↗

NASA NTRSDate not supplied

Interoperability of Tools at the CCMC

The CCMC has a diverse set of tools in many languages that support utilization of simulation outputs accessible through CCMC interactive archives. Some model output post-processing and analysis tools are delivered to tCCMC by the community. One such model output post-processing/utilization tool developed by the CCMC is the open source Kamodo package. Kamodo is developed primarily in Python (with some C) and has already established strong interoperability with other Python libraries inside PyHC and out. While that interoperability is important, interoperability with other languages and tools is as important. Many models are written in Fortran or C, and their ability to pull in data from a Python tool or export directly to other analysis software in a different language will greatly increase scientific productivity. Interoperability within Python is important, but broader interoperability is just as important.

CCMC↗

NASA NTRS2003

Time Analyzer for Time Synchronization and Monitor of the Deep Space Network

A software package has been developed to measure, monitor, and archive the performance of timing signals distributed in the NASA Deep Space Network. Timing signals are generated from a central master clock and distributed to over 100 users at distances up to 30 kilometers. The time offset due to internal distribution delays and time jitter with respect to the central master clock are critical for successful spacecraft navigation, radio science, and very long baseline interferometry (VLBI) applications. The instrument controller and operator interface software is written in LabView and runs on the Linux operating system. The software controls a commercial multiplexer to switch 120 separate timing signals to measure offset and jitter with a time-interval counter referenced to the master clock. The offset of each channel is displayed in histogram form, and "out of specification" alarms are sent to a central complex monitor and control system. At any time, the measurement cycle of 120 signals can be interrupted for diagnostic tests on an individual channel. The instrument also routinely monitors and archives the long-term stability of all frequency standards or any other 1-pps source compared against the master clock. All data is stored and made available for

Cole, Steven↗

NASA NTRS2015

Airborne Doppler Wind Lidar Post Data Processing Software DAPS-LV

Systems, methods, and devices of the present invention enable post processing of airborne Doppler wind LIDAR data. In an embodiment, airborne Doppler wind LIDAR data software written in LabVIEW may be provided and may run two versions of different airborne wind profiling algorithms. A first algorithm may be the Airborne Wind Profiling Algorithm for Doppler Wind LIDAR ("APOLO") using airborne wind LIDAR data from two orthogonal directions to estimate wind parameters, and a second algorithm may be a five direction based method using pseudo inverse functions to estimate wind parameters. The various embodiments may enable wind profiles to be compared using different algorithms, may enable wind profile data for long haul color displays to be generated, may display long haul color displays, and/or may enable archiving of data at user-selectable altitudes over a long observation period for data distribution and population.

Beyon, Jeffrey Y.↗

NASA NTRS2012

Science Operations with the James Webb Space Telescope

The James Webb Space Telescope (JWST) will be a powerful space observatory whose four science instruments will deliver rich imaging and multiplexed spectroscopic datasets to the astronomical and planetary science communities. The ground segment for JWST, now being designed and built, will carry out JWST's science operations. The ground segment includes: (1) software that the scientific community will use to propose and specify new observations; (2) software that will schedule both science and calibration observations in a way that optimizes observing efficiency while managing the accumulation of momentum; (3) the infrastructure to regularly measure and maintain the telescope's wavefront; (4) orbit determination, ranging, and tracking; (5) communication via the Deep Space Network to command the observatory and retrieve scientific data; (6) onboard scripts that execute each observing program in an event-driven fashion, with occasional interruptions for targets of opportunity or time-critical observations; and (7) a system that processes and calibrates the data into science ready products, automatically recalibrates when calibrations improve, and archives the data for timely access by the principal investigator and later worldwide access by the scientific community. This ground system builds on experience from operating the Hubble Space Telescope, while solving challenges that are unique to the James Webb Space Telescope. In this paper, we describe the elements of the JWST ground system, how it will work operationally from the perspective of the observatory itself, and how a typical user will interact with the system to turn his/her idea into scientific discovery.

Rigby, Jane↗

DOE OSTI2025

dCache: The Storage System of Choice for Data-Intensive Applications

The ever-increasing volumes of data produced by modern scientific facilities like EuXFEL and LHC put significant stress on data management infrastructure operated by laboratories and research centers. The challenges to be addressed span the entire data life cycle, from ingest and efficient data analysis to long-term preservation, typically involving large tape libraries. dCache, a storage system developed in collaboration between the Deutsches Elektronen-Synchrotron (DESY), Fermi National Accelerator Laboratory, and Nordic e-Infrastructure Collaboration (NeIC), is designed to manage a large number of disk servers and to facilitate transparent data migration to and from archival storage. Its multifaceted approach offers a unified method to support a variety of scientific use cases with the same storage infrastructure, including high-throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters, and long-term data preservation on tertiary storage. Initially developed for high energy physics (HEP) experiments, dCache is now used by various scientific communities, including astrophysics, biomedical research, and life sciences, each having specific requirements. This paper presents architecture, deployment strategies, performance and scalability enhancements, and recent advancements in dCache addressing the needs of scientific communities. Finally, we touch on the development and release process, ensuring the software’s high quality.

DCache↗

NASA NTRS1994

Designing an autonomous environment for mission critical operation of the EUVE satellite

Since the launch of NASA's Extreme Ultraviolet Explorer (EUVE) satellite in 1992, there has only been a handful of occurrences that have warranted manual intervention in the EUVE Science Operations Center (ESOC). So, in an effort to reduce costs, the current environment is being redesigned to utilize a combination of off-the-shelf packages and recently developed artificial intelligence (AI) software to automate the monitoring of the science payload and ground systems. The successful implementation of systemic automation would allow the ESOC to evolve from a seven day/week, three shift operation, to a seven day/week one shift operation. First, it was necessary to identify all areas considered mission critical. These were defined as follows: (1) The telemetry stream must be monitored autonomously and anomalies identified. (2) Duty personnel must be automatically paged and informed of the occurrence of an anomaly. (3) The 'basic' state of the ground system must be assessed. (4) Monitors should check that the systems and processes needed to continue in a 'healthy' operational mode are working at all times. (5) Network loads should be monitored to ensure that they stay within established limits. (6) Connectivity to Goddard Space Flight Center (GSFC) systems should be monitored as well, not just for connectivity of the network itself but also for the ability to transfer files. (7) All necessary peripheral devices should be monitored. This would include the disks, routers, tape drives, printers, tape carousel, and power supplies. (8) System daemons such as the archival daemon, the Sybase server, the payload monitoring software, and any other necessary processes should be monitored to ensure that they are operational. (9) The monitoring system needs to be redundant so that the failure of a single machine will not paralyze the monitors. (10) Notification should be done by means of looking though a table of the pager numbers for current 'on call' personnel. The software should be capable of dialing out to notify, sending email, and producing error logs. (11) The system should have knowledge of when real-time passes and tape recorder dumps will occur and should know that these passes and data transmissions are successful. Once the design criteria were established, the design team split into two groups: one that addressed the tracking, commanding, and health and safety of the science payload and another group that addressed the ground systems and communications aspects of the overall system.

Abedini, Annadiana↗

DOE OSTI2021

DEPRECATED AI-Batt-OS (Autonomous Identification of Battery Life Models - Open Source) [SWR 21-17]

DEPRECATED. This repository was archived by the owner on Jun 30, 2026. It is now read-only. Open source implementation of some of the methods utilized by AI-Batt, a battery lifetime modeling and analysis toolkit provided by the National Laboratory of the Rockies (NLR). This software demonstrates the use of bi-level optimization and symbolic regression techniques to semi-autonomously identify algebraic models predicting the capacity fade of lithium-ion batteries during calendar aging. Modeling the degradation of batteries is a complex task, due to the difficulty in separating the time-dependent and time-independent factors impacting cell level degradation, across multiple data series with different numbers of measurements and/or data quality. Bi-level optimization enables model parameters to be optimized to either the entire data set or to individual data series, allowing statistical disambiguation of global behaviors (data series independent) and local behaviors (data series dependent). Symbolic regression is used to automatically search for optimal low-dimesional models predicting the variation of locally optimized parameters versus time-independent experimental variables from millions of possible models, resulting in a more accurate and repeatable model identification process than is possible by a manual search. The provided tools also implement cross-validation and bootstrap resampling schemes, empowering statistical model comparison/selection and quantification of model uncertainties. An example script replicates the results from the manuscript "Challenging Practices of Algebraic Battery Life Models through Statistical Validation and Model Identification via Machine-Learning", submitted to ECS. All code is written in MATLAB. Requires the Statistics and Machine Learning Toolbox. Contact Dr. Paul Gasper at Paul.Gasper@nlr.gov for any questions.

Gasper, Paul [National Renewable Energy Lab. (NREL↗

DOE OSTI2024

Myna

The additive manufacturing (AM) community has been developing digital factory tools over the past decade to better leverage the multi-modal process data coming out of the advanced manufacturing process. As a result, numerous databases of additive manufacturing process data exist in the literature and in the archival storage of disparate research groups. While some efforts have been made to create a standard ontology for storing and sharing AM data, in practice a variety of data structures are used to store AM build data, even within a single institution. This causes many problems for maintainability and extensibility when attempting to integrate computational modeling tools with experimental data to either validate models or to provide further insight into results and trends. Myna is a Python-based framework that aims to decrease the effort needed to connect individual computational models to the variety of AM process data that exist in different research groups and institutions. This type of software is sometimes referred to as "middleware" or “glueware,” in that it connects disparate databases and applications into a single computational ecosystem. Instead of maintaining unique interfaces between each application and each database, developers can create a single interface from each application to Myna and thereby gain access to the implemented database connections. Similarly, developing a database connection in Myna provides access to the developed simulation applications. This framework greatly simplifies the maintainability of model applications that rely on experimental data. Using external simulation tools, users will also be able to run pre-configured workflows using the built-in workflow manager. Several examples of input files are provided with Myna for different workflows, including melt pool geometry predictions and detailed melt pool and solidification microstructure predictions.

Knapp, GerryL. [Oak Ridge National Laboratory (ORN↗

NASA NTRS2001

Description of Selected Algorithms and Implementation Details of a Concept-Demonstration Aircraft VOrtex Spacing System (AVOSS)

A ground-based system has been developed to demonstrate the feasibility of automating the process of collecting relevant weather data, predicting wake vortex behavior from a data base of aircraft, prescribing safe wake vortex spacing criteria, estimating system benefit, and comparing predicted and observed wake vortex behavior. This report describes many of the system algorithms, features, limitations, and lessons learned, as well as suggested system improvements. The system has demonstrated concept feasibility and the potential for airport benefit. Significant opportunities exist however for improved system robustness and optimization. A condensed version of the development lab book is provided along with samples of key input and output file types. This report is intended to document the technical development process and system architecture, and to augment archived internal documents that provide detailed descriptions of software and file formats.

Hinton, David A.↗

NASA NTRS2006

Swift-BAT: The First Year of Gamma-Ray Burst Detections

The Burst Alert Telescope (BAT) on the Swift has been detecting gamma-ray bursts (GRBs) since Dec. 17,2004 and automated burst alerts have been distributed since Feb. 14,2005. Since commissioning the BAT has triggered on more than 100 GRBs, nearly all of which have been followed up by the narrow-field instruments on Swift through automatic repointing, and by ground and other satellite telescopes after rapid notification. Within seconds of a trigger the BAT produces and relays to the ground a position good to three arc minutes and a four channel light curve. A full ten minutes of event data follows on subsequent ground station passes. The burst archive has allowed us to determine ensemble burst parameters such as fluence, peak flux and duration. An overview of the properties of BAT bursts and BAT'S performance as a burst monitor will be presented in this talk. BAT is a coded aperture imaging system with a wide (approx.2 sr) field of view consisting of a large coded mask located 1 m above a 5200 cm2 array of 32.768 CdZnTe detectors. All electronics and other hardware systems on the BAT have been operating well since commissioning and there is no sign of any degradation on orbit. The flight and ground software have proven similarly robust and allow the real time localization of all bursts and the rapid derivation of burst light curves, spectra and spectral fits on the ground.

Krimm, Hans A.↗

NASA NTRS2011

Towards a Standard for Provenance and Context for Preservation of Data for Earth System Science

Long-term data sets with data from many missions are needed to study trends and validate model results that are typical in Earth System Science research. Data and derived products originate from multiple missions (spaceborne, airborne and/or in situ) and from multiple organizations. During the missions as well as well past their termination, it is essential to preserve the data and products to support future studies. Key aspects of preservation are: preserving bits and ensuring data are uncorrupted, preserving understandability with appropriate documentation, and preserving reproducibility of science with appropriate documentation and other artifacts. Computer technology provides adequate standards to ensure that, with proper engineering, bits are preserved as hardware evolves. However, to ensure understandability and reproducibility, it is essential to plan ahead to preserve all the relevant data and information. There are currently no standards to identify the content that needs to be preserved, leading to non-uniformity in content and users not being sure of whether preserved content is comprehensive. Each project, program or agency can specify the items to be preserved as a part of its data management requirements. However, broader community consensus that cuts across organizational or national boundaries would be needed to ensure comprehensiveness, uniformity and long-term utility of archived data. The Federation of Earth Science Information Partners (ESIP), a diverse network of scientists, data stewards and technology developers, has a forum for ESIP members to collaborate on data preservation issues. During early 2011, members discussed the importance of developing a Provenance and Context Content Standard (PCCS) and developed an initial list of content items. This list is based on the outcome of a NASA and NOAA meeting held in 1998 under the auspices of the USGCRP, documentation requirements from NOAA and our experience with some of the NASA Earth science missions. The items are categorized into the following 8 high level categories: Preflight/Pre-Operations, Products (Data), Product Documentation, Mission Calibration, Product Software, Algorithm Input, Validation, Software Tools.

Ramaprian, Hampapuram K.↗

NASA NTRS2011

Evaluating Cloud Computing in the Proposed NASA DESDynI Ground Data System

The proposed NASA Deformation, Ecosystem Structure and Dynamics of Ice (DESDynI) mission would be a first-of-breed endeavor that would fundamentally change the paradigm by which Earth Science data systems at NASA are built. DESDynI is evaluating a distributed architecture where expert science nodes around the country all engage in some form of mission processing and data archiving. This is compared to the traditional NASA Earth Science missions where the science processing is typically centralized. What's more, DESDynI is poised to profoundly increase the amount of data collection and processing well into the 5 terabyte/day and tens of thousands of job range, both of which comprise a tremendous challenge to DESDynI's proposed distributed data system architecture. In this paper, we report on a set of architectural trade studies and benchmarks meant to inform the DESDynI mission and the broader community of the impacts of these unprecedented requirements. In particular, we evaluate the benefits of cloud computing and its integration with our existing NASA ground data system software called Apache Object Oriented Data Technology (OODT). The preliminary conclusions of our study suggest that the use of the cloud and OODT together synergistically form an effective, efficient and extensible combination that could meet the challenges of NASA science missions requiring DESDynI-like data collection and processing volumes at reduced costs.

Benchmark↗

NASA NTRS1995

Development and analysis of SCR requirements tables for system scenarios

We describe the use of scenarios to develop and refine requirement tables for parts of the Earth Observing System Data and Information System (EOSDIS). The National Aeronautics and Space Administration (NASA) is developing EOSDIS as part of its Mission-To-Planet-Earth (MTPE) project to accept instrument/platform observation requests from end-user scientists, schedule and perform requested observations of the Earth from space, collect and process the observed data, and distribute data to scientists and archives. Current requirements for the system are managed with tools that allow developers to trace the relationships between requirements and other development artifacts, including other requirements. In addition, the user community (e.g., earth and atmospheric scientists), in conjunction with NASA, has generated scenarios describing the actions of EOSDIS subsystems in response to user requests and other system activities. As part of a research effort in verification and validation techniques, this paper describes our efforts to develop requirements tables from these scenarios for the EOSDIS Core System (ECS). The tables specify event-driven mode transitions based on techniques developed by the Naval Research Lab's (NRL) Software Cost Reduction (SCR) project. The SCR approach has proven effective in specifying requirements for large systems in an unambiguous, terse format that enhance identification of incomplete and inconsistent requirements. We describe development of SCR tables from user scenarios and identify the strengths and weaknesses of our approach in contrast to the requirements tracing approach. We also evaluate the capabilities of both approach to respond to the volatility of requirements in large, complex systems.

Callahan, John R.↗

NASA NTRS1993

Value added data archiving

Researchers in the Molecular Sciences Research Center (MSRC) of Pacific Northwest Laboratory (PNL) currently generate massive amounts of scientific data. The amount of data that will need to be managed by the turn of the century is expected to increase significantly. Automated tools that support the management, maintenance, and sharing of this data are minimal. Researchers typically manage their own data by physically moving datasets to and from long term storage devices and recording a dataset's historical information in a laboratory notebook. Even though it is not the most efficient use of resources, researchers have tolerated the process. The solution to this problem will evolve over the next three years in three phases. PNL plans to add sophistication to existing multilevel file system (MLFS) software by integrating it with an object database management system (ODBMS). The first phase in the evolution is currently underway. A prototype system of limited scale is being used to gather information that will feed into the next two phases. This paper describes the prototype system, identifies the successes and problems/complications experienced to date, and outlines PNL's long term goals and objectives in providing a permanent solution.

Berard, Peter R.↗

NASA NTRS2007

Pointing History Engine for the Spitzer Space Telescope

The Pointing History Engine (PHE) is a computer program that provides mathematical transformations needed to reconstruct, from downlinked telemetry data, the attitude of the Spitzer Space Telescope (formerly known as the Space Infrared Telescope Facility) as a function of time. The PHE also serves as an example for development of similar pointing reconstruction software for future space telescopes. The transformations implemented in the PHE take account of the unique geometry of the Spitzer telescope-pointing chain, including all data on relative alignments of components, and all information available from attitude-determination instruments. The PHE makes it possible to coordinate attitude data with observational data acquired at the same time, so that any observed astronomical object can be located for future reference and re-observation. The PHE is implemented as a subroutine used in conjunction with telemetry-formatting services of the Mission Image Processing Laboratory of NASA s Jet Propulsion Laboratory to generate the Boresight Pointing History File (BPHF). The BPHF is an archival database designed to serve as Spitzer s primary astronomical reference documenting where the telescope was pointed at any time during its mission.

Bayard, David↗