SEARCH · Search NASA

Results for “data processing and archiving”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

All sources NASA NTRS DOE OSTI

NASA NTRS1990

High Resolution Imaging Spectrometer (HIRIS) - A major advance in imaging spectrometry

HIRIS, a facility instrument on the Earth Observing System Platform, will enable scientists to both identify and interpret changes, in the global environment, at a level of detail unavailable with previous instrumentation. This instrument will make localized measurements of biological, ecological, climatological, hydrological, and geological properties, and, with its finer wavelength sampling and full spectral coverage over the wavelength region of interest, will allow these properties to be identified and studied quantitatively from space. The data obtained will be downlinked from space, processed, catalogued, stored in active archives for easy access, and distributed to users.

Rockey, Donald E.↗

NASA NTRS2005

CRYSTAL-FACE Polarization Lidar Research

The University of Alaska Fairbanks (UAF) Polarization Diversity Lidar (PDL, Sassen 1994) participated in the July 2002 CRYSTAL-FACE field campaign, shortly after the PI moved from the University of Utah to UAF. The truck-mounted PDL is an advanced dual wavelength (1.06 and 0.532 micron), high resolution (0.1-s by 1.5-m), scanning lidar system designed as a testbed for evaluating laser backscatter depolarization techniques for the study of clouds and aerosols in the atmosphere. The main goals identified in our proposal for the CRYSTAL-FACE experiment were, i) the characterization of Florida thunderstorm anvil macrophysical and microphysical properties from lidar backscattering and depolarization, ii) the study of thin to subvisual tropopause-topped subtropical cirrus, iii) the search for indirect cloud effects of trans- Atlantic advected Saharan dust storm aerosols on clouds, and iv) the investigation of melting layer effects on lidar and multi-wavelength Doppler radar measurements in precipitation. Although we experienced adversity in the field during the campaign, sufficient data was collected to begin addressing these topics, and several conference presentations, three journal articles, and one book chapter have resulted from the data analysis effort supported by this grant. (PDL operations were delayed by FAA concerns over the initial sighting at the Kendall-Tamiami Airport, and a brief but major laser breakdown was experienced during the re- setup at the remote Ochopee Everglades site that also supported the N-POL radar.) All lidar data collected by the PDL system were processed and quality checked, and submitted to the CRYSTAL-FACE data archive in a timely manner.

Sassen, Kenneth↗

NASA NTRS2006

Antarctic Iceberg Tracking Based on Time Series of Aqua AMSRE Microwave Brightness Temperature Measurements

Observations of icebergs are identified as one of the requirements for the GEOSS (Global Earth Observation System of Systems) in the area of reducing loss of life and property from natural and human-induced disasters. However, iceberg observations are not included among targets in the GEOSS 10-Year Implementation Plan, and thus there is an unfulfilled need for iceberg detection and tracking in the near future. Large Antarctic icebergs have been tracked by the National Ice Center and by the academic community using a variety of satellite sensors including both passive and active microwave imagers, such as SSM/I (Special Sensor Microwave/Imager) deployed on the DMSP (Defense Meteorological Satellite Program) spacecraft. Improvements provided in recent years by NASA and non-NASA satellite radars, scatterometers, and radiometers resulted in an increased number of observed icebergs and even prompted a question: Is The Number of Antarctic Icebergs Really Increasing? [D.G. Long, J. Ballantyne, and C. Bertoia, Eos, Transactions of the American Geophysical Union 83 (42): 469 & 474, 15 October 2002]. AMSR-E (Advanced Microwave Scanning Radiometer for the Earth Observing System) represents an improvement over SSM/I, its predecessor. AMSR-E has more measurement channels and higher spatial resolution than SSM/I. For example, the instantaneous field of view of the AMSR-E s 89-GHz channels is 6 km by 4 km versus 16 km by 14 km for SSM/I s comparable 85-GHz channels. AMSR-E, deployed on the Aqua satellite, scans across a 1450-km swath and provides brightness temperature measurements with nearglobal coverage every one or two days. In polar regions, overlapping swaths generate coverage up to multiple times per day and allow for creation of image time series with high temporal resolution. Despite these advantages, only incidental usage of AMSR-E data for iceberg tracking has been reported so far, none in an operational environment. Therefore, an experiment was undertaken in the RPC (Rapid Prototyping Capability) of NASA s (National Aeronautics and Space Administration) Applied Science Program to demonstrate that passive microwave brightness temperature measurements acquired by AMSR-E can be effectively used to track iceberg movement around Antarctica. The RPC s robust computation environment enabled processing of terabytes of data products available from the NASA Distributed Active Archive Center at the National Snow and Ice Data Center. Iceberg tracking based on the AMSR-E Level 2A data product was compared with records from the National Ice Center for currently existing icebergs. Some icebergs as small as roughly 10 km in size were easily observed, but tracking of many others, even larger ones, was obscured by presence of sea ice surrounding the icebergs. The best results, such as for the large iceberg A22A, were achieved when an iceberg was in open ocean.

Blonski, Slawomir↗

NASA NTRS2012

Evaluation and Validation of Global Land Surface Products Derived from Landsat, MODIS, and VIIRS

Data at medium and coarse resolution from the Landsat, MODIS, and VIIRS instruments provide crucial and indispensable time-series for the land component of the study of global change. This talk will be divided into two main sections. In the first part, a summary of the status of the processing, archiving, and early (Launch +6 months) on-orbit evaluation of the VIIRS Land Environmental Data Records (EDRs), will be presented. In the second part, results from an uncertainty analysis of MODIS- and Landsat-based albedo retrievals, based on collocated comparisons with tower and airborne multiangular measurements collected at the Cloud and Radiation Testbed (CART) site during the 2007 Cloud and Land Surface Interaction Campaign (CLASIC'07), will be discussed.

Roman, Miguel O.↗

NASA NTRSDate not supplied

Enabling Interoperability in Earth System Digital Twins (ESDT): Integrating Observations, Models, and AI for Actionable Insights Through NASA'S Intelligent Systems Technology Program

NASA’s Intelligent Systems Technology Program (IST) is driving a paradigm shift in Earth science through the development of Earth System Digital Twins (ESDT). These integrated information systems create a dynamic "digital replica" of the Earth by harmonizing continuous, multi-source observations with high-fidelity models and state-of-the-art artificial intelligence (AI) that enable “What now?”, “What next?”, and “What if?” scenario building. These scenarios are reflected in NASA IST’s series of ESDTs, from the Coastal Zone Digital Twin that integrates complex data on the current state of the Chesapeake Bay to the Terrestrial Environmental Rapid-Replication and Assimilation Hydrometeorological (TerraHydro) AI-based ESDT that forecasts water movement across Earth’s surface, to the Agriculture Land Information System (AgLIS) which can be used to assess optimal planting dates and crop yield estimates. By bridging the gap between vast data archives and actionable insights, these projects enable a system-of-systems approach to understanding complex, interacting Earth processes. This poster will highlight recent innovations and future directions from NASA’s ESDT initiatives: Continuous Data Assimilation & Multi-Source Fusion. A core requirement of the ESDT work is the transition from static models to dynamic "living" replicas. This involves creating frameworks for the continual assimilation of near-real-time data from uncoordinated, heterogeneous sources, including satellite observations and airborne assets, and ground-based Internet of Things (IoT) sensors. These systems link design, operational status, and environmental data, ensuring the digital twin accurately reflects the current state of the physical Earth system. High-Fidelity Hybrid Modeling & Computational Acceleration to enable interactive "what-if" explorations, programs are moving beyond traditional, slow physical solvers by developing fast surrogate machine learning models and Deep Generative Models (DGMs). These hybrid approaches use neural networks to emulate complex physics, such as cloud feedback or ocean dynamics, at a fraction of the original computing cost, often leveraging advanced hardware like Graphics Processing Units (GPUs) to achieve the necessary scale. Federated Ecosystems & Interoperable Frameworks rather than building isolated tools, NASA IST is moving toward federated ESDTs and reusable analytic collaborative frameworks. This theme focuses on interoperability standards and common ontologies that allow specialized digital twins to interact and share data. This system-of-systems architecture supports multi-discipline investigations, such as analyzing how upstream watershed changes impact downstream urban flooding or how wildfire emissions affect regional air quality. By leveraging these advancements, ESDTs empower researchers and decision-makers to conduct real-time analysis and run complex hypothetical scenarios, ultimately improving our understanding of Earth’s evolving systems and informing critical real-world applications.

Earth System↗

NASA NTRS2016

Kepler Archive Manual

A description of Kepler, its design, performance and operational constraints may be found in the Kepler Instrument Handbook (KIH, Van Cleve Caldwell 2016). A description of Kepler calibration and data processing is described in the Kepler Data Processing Handbook (KDPH, Jenkins et al. 2016; Fanelli et al. 2011). Science users should also consult the special ApJ Letters devoted to early Kepler results and mission design (April 2010, ApJL, Vol. 713 L79-L207). Additional technical details regarding the data processing and data qualities can be found in the Kepler Data Characteristics Handbook (KDCH, Christiansen et al. 2013) and the Data Release Notes (DRN). This archive manual specifically documents the file formats, as they exist for the last data release of Kepler, Data Release 25(KSCI-19065-002). The earlier versions of the archive manual and data release notes act as documentation for the earlier versions of the data files.

Kepler↗

NASA NTRSDate not supplied

Enabling Biological Discovery Through Biospecimen Sharing: The Nasa Biological Institutional Scientific Collection

Understanding biological impacts from spaceflight hazards and the subsequent development of countermeasures are a high priority to enable humanity to venture back to the Moon, and then to Mars and beyond. Experiments have been conducted with model organisms flown to space and analogous investigations terrestrially, to identify biological mechanistic impacts from spaceflight hazards and to develop mitigation countermeasures, thus contributing towards basic and applied science goals. However, sending organisms into space is a costly endeavor. To maximize scientific return, all biospecimens not required by spaceflight-relevant Principal Investigators are harvested, preserved, and archived in the NASA Biological Institutional Scientific Collection (NBISC). Biospecimens are collected and preserved according to well-established standard operating procedures to maintain scientific quality and are available on-request by the international scientific community. NBISC currently stores over 32,000 biospecimens from Shuttle, International Space Station, and ground-based space analog investigations. Tissue sharing has resulted in at least 33 publications since 2011 and 51 requests since 2016. Many requests for NBISC biospecimens come from first-time investigators who subsequently submit grants as their point-of-entry into the field of spaceflight biology and health. The NBISC biorepository is part of the NASA ‘Open Science for Life in Space’ collaborative group of projects, which includes NASA Genelab, the Space Biology Program’s Biospecimen Sharing Program, Physical Sciences Informatics, and the Ames Life Sciences Data Archive. NBISC biospecimens have been awarded to NASA Genelab, who then generated various open access science ‘omics datasets through the GeneLab Sample Processing laboratory, with resulting data widely used for biological study. Other NBISC biospecimen awards have led to studies on fecal microbiome analysis, DNA damage analysis using single-cell DNA sequencing, enzymatic-pathway identification involved in spaceflight muscle atrophy, and characterization of ocular morphological changes. Of note, NBISC is expanded to include a new Space Microbial Culture Collection (SMCC) for the collection, identification, documentation, long-term preservation, and distribution of space-related microbial isolates.

Biospecimens↗

DOE OSTI2025

Model data for a watershed-scale study in the Portage River Basin (OH) examining the effects of subsurface drainage on the hydrologic response of an agricultural watershed.

This study builds on Rathore et al. (2024, WRR) and investigates the role of artificial tile-drainage on various aspects of watershed hydrological response, with a particular focus on peakflow. The model-data for the original modeling-focused paper (Rathore et al., 2024, WRR) is archived at Rathore et al. (2024, ESS-DIVE). Hence, this model-data archive provides scripts that are specific to this study that includes model updates, processing and analysis scripts. For details and models files of original model, readers are referred to Rathore et al. (2024, ESS-DIVE). The key difference between the model configuration in this study and Rathore et al. (2024, WRR) is that the tile drains are applied to the entire domain, to study the impact of tile-drains on different aspects of hydrological response. Additional scenario considering intensified precipitation after a dry period was also simulated. The Watershed Workflow package is implemented in Python3. The Jupyter notebooks can be executed through multiple open-source tools, for example, Anaconda Jupyter Lab, VS Studio Code, etc. Other data files include CSV and HDF5 files, which can be read through Python scripts.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1974

Skylab Medical Data Center and Archives

The founding of the Skylab medical data center and archives as a central area to house medical data from space flights is described. Skylab program strip charts, various daily reports and summaries, experiment reports and logs, status report on Skylab data quality, raw data digital tapes, processed data microfilm, and other Skylab documents are housed in the data center. In addition, this memorandum describes how the data center acted as a central point for the coordination of preflight and postflight baseline data and how it served as coordinator for all data processing through computation and analysis. Also described is a catalog identifying Skylab medical experiments and all related data currently archived in the data center.

Spross, F. R.↗

NASA NTRS1999

Polar Geophysics Products Derived from AVHRR: The "AVHRR Polar Pathfinder

This NOAA/NASA Pathfinder effort was established to locate, acquire, and process Advanced Very High Resolution Radiometer (AVHRR) imagery into geo-located and calibrated radiances, cloud masks, surface clear-sky broadband albedo, clear-sky skin temperatures, satellite viewing times, and viewing and solar geometry for the, high-latitude portions of the northern and southern hemispheres (all area north of 48N and south of 53S). AVHRR GAC data for August 1981 - July 1998 were acquired, with some gaps remaining, and processed into twice-daily 5-km grids, with some products also provided at 25-km resolution. AVHRR LAC data for 3.5 years of coverage in the northern hemisphere and 2.75 years of coverage in the southern hemisphere were processed into 1.25-km grids for the same suite of products. The resulting data sets are presently being transferred to the National Snow and Ice Data Center (NSIDC) for archiving and distribution. Using these data, researchers now have at their disposal an extensive AVHRR data set for investigations of high-latitude processes. In addition, the data lend themselves to development and testing of algorithms. The products are particularly relevant for climate research and algorithm development as applied to relatively long time periods and large areas.

Maslanik, James↗

NASA NTRS1996

Queuing Models of Tertiary Storage

Large scale scientific projects generate and use large amounts of data. For example, the NASA Earth Observation System Data and Information System (EOSDIS) project is expected to archive one petabyte per year of raw satellite data. This data is made automatically available for processing into higher level data products and for dissemination to the scientific community. Such large volumes of data can only be stored in robotic storage libraries (RSL's) for near-line access. A characteristic of RSL's is the use of a robot arm that transfers media between a storage rack and the read/write drives, thus multiplying the capacity of the system. The performance of the RSL's can be a critical limiting factor for the performance of the archive system. However, the many interacting components of an RSL make a performance analysis difficult. In addition, different RSL components can have widely varying performance characteristics. This paper describes our work to develop performance models of an RSL in isolation. Next we show how the RSL model can be incorporated into a queuing network model. We use the models to make some example performance studies of archive systems. The models described in this paper, developed for the NASA EODIS project, are implemented in C with a well defined interface. The source code, accompanying documentation, and also sample JAVA applets are available at: http://www.cis.ufl.edu/~ted/

Johnson, Theodore↗

NASA NTRS1973

Man-computer Inactive Data Access System (McIDAS)

A technical description is given of the effort to design, develop, fabricate, and test the two dimensional data processing system, McIDAS. The system has three basic sections: an access and data archive section, a control section, and a display section. Areas reported include hardware, system software, and applications software.

Source record↗

NASA NTRS1980

Evaluation of NESS soundings used in the FGGE special effort

A special effort to produce high quality edited and enhanced global data sets is being conducted for the two special observing periods of the first GARP global experiment. More than 60 cases were selected for enhancement on the basis of meteorological interest. These cases include situations of blocking, cut off low development, cyclogenesis, and tropical circulations. The sounding data enhancement process consists of supplementing the operational satellite sounding data set with higher resolution sounding in meteorologically active regions and with new soundings where data voids or soundings of questionable quality exist. After the enhancement process is completed, a final editing and quality assessment of the enhanced data are performed. The data is archived for further analyses. Subjective evaluation of the effects of sounding data enhancement indicate that: (1) enhanced and operational IR retrievals are similar in most situations; (2) large improvements in 1000 to 300 mb thickness and mandatory level temperature and intensification of atmospheric thermal gradients occasionally occur; (3) interactive processing can fill in major gaps in the TIROS-N coverage; and (4) assimilation of special effort soundings can result in major modifications to large scale analyses and prognoses.

Atlas, R.↗

NASA NTRS1994

OOD/OOP experience in the Science Operations Center part of the ground system for X ray Timing Explorer mission

The Science Operations Center (SOC) for the X-ray Timing Explorer (XTE) mission is an important component of the XTE ground system. Its mandate includes: (1) command and telemetry for the three XTE instruments, using CCSDS standards; (2) monitoring of the real-time science operations, reconfiguration of the experiment and the instruments, and real-time commanding to address the targets of opportunity (TOO) and alternate observations; and (3) analysis, processing, and archival of the XTE telemetry, and the timely delivery of the data products to the principal investigator (PI) teams and the guest observers (GO). The SOC has two major components: the science operations facility (SOF) that addresses the first two objectives stated above and the guest observer facility (GOF) that addresses the third. The SOF has subscribed to the object oriented design and implementation; while the GOF uses the traditional approach in order to take advantage of the existing software developed in support of previous missions. This paper details the SOF development using the object oriented design (OOD), and its implementation using the object oriented programming (OOP) in C++ under Unix environment on client-server architecture using Sun workstations. It also illustrates how the object oriented (OO) and the traditional approaches coexist in SOF and GOF, the lessons learned, and how the OOD facilitated the distributed software development collaboratively by four different teams. Details are presented for the SOF system, its major subsystems, its interfaces with the rest of the XTE ground data system, and its design and implementation approaches.

Choudhary, Abdur Rahim↗

NASA NTRS2007

Simple, Script-Based Science Processing Archive

The Simple, Scalable, Script-based Science Processing (S4P) Archive (S4PA) is a disk-based archival system for remote sensing data. It is based on the data-driven framework of S4P and is used for data transfer, data preprocessing, metadata generation, data archive, and data distribution. New data are automatically detected by the system. S4P provides services such as data access control, data subscription, metadata publication, data replication, and data recovery. It comprises scripts that control the data flow. The system detects the availability of data on an FTP (file transfer protocol) server, initiates data transfer, preprocesses data if necessary, and archives it on readily available disk drives with FTP and HTTP (Hypertext Transfer Protocol) access, allowing instantaneous data access. There are options for plug-ins for data preprocessing before storage. Publication of metadata to external applications such as the Earth Observing System Clearinghouse (ECHO) is also supported. S4PA includes a graphical user interface for monitoring the system operation and a tool for deploying the system. To ensure reliability, S4P continuously checks stored data for integrity, Further reliability is provided by tape backups of disks made once a disk partition is full and closed. The system is designed for low maintenance, requiring minimal operator oversight.

Lynnes, Christopher↗

NASA NTRS2010

The Nuclear Spectroscopic Telescope Array (NuSTAR)

The Nuclear Spectroscopic Telescope Array (NuSTAR) is a NASA Small Explorer mission that will carry the first focusing hard X-ray (5 - 80 keV) telescope to orbit. NuSTAR will offer a factor 50 - 100 sensitivity improvement compared to previous collimated or coded mask imagers that have operated in this energy band. In addition, NuSTAR provides sub-arcminute imaging with good spectral resolution over a 12-arcminute field of view. After launch, NuSTAR will carry out a two-year primary science mission that focuses on four key programs: studying the evolution of massive black holes through surveys carried out in fields with excellent multiwavelength coverage, understanding the population of compact objects and the nature of the massive black hole in the center of the Milky Way, constraining explosion dynamics and nucleosynthesis in supernovae, and probing the nature of particle acceleration in relativistic jets in active galactic nuclei. A number of additional observations will be included in the primary mission, and a. guest observer program will be proposed for an extended mission to expand the range of scientific targets. The payload consists of two co-aligned depth-graded multilayer coated grazing incidence optics focused onto solid state CdZnTe pixel detectors. To be launched in early 2012 on a Pegasus rocket into a low-inclination Earth orbit. NuSTAR largely avoids SAA passages, and will therefore have low and stable detector backgrounds. The telescope achieves a 10.15-meter focal length through on-orbit deployment of all mast. An aspect and alignment metrology system enable reconstruction of the absolute aspect and variations in the telescope alignment resulting from mast flexure during ground data processing. Data will be publicly available at GSFC's High Energy Astrophysics Science Archive Research Center (HEASARC) following validation at the science operations center located at Caltech.

Harrison, Fiona A.↗

NASA NTRS1992

Monthly mean global satellite data sets available in CCM history tape format

Satellite data for climate monitoring have become increasingly important over the past decade, especially with increasing concern for inadvertent antropogenic climate change. Although most satellite based data are of short record, satellites can provide the global coverage that traditional meteorological observations network lack. In addition, satellite data are invaluable for the validation of climate models, and they are useful for many diagnostic studies. Herein, several satellite data sets were processed and transposed into 'history tape' format for use with the Community Climate Model (CCM) modular processor. Only a few of the most widely used and best documented data sets were selected at this point, although future work will expand the number of data sets examined as well as update the archived data sets. An attempt was made to include data of longer record and only monthly averaged data were processed. For studies using satellite data over an extended period, it is important to recognize the impact of changes in instrumentation, drift in instrument calibration, errors introduced by retrieval algorithms and other sources of errors such as those resulting from insufficient space and/or time sampling.

Hurrell, James W.↗

NASA NTRS2020

Perspectives on Data Reproducibility and Replicability in Paleoclimate and Climate Science

This paper summarizes the current state of reproducibility and replicability in the fields of climate and paleoclimate science, including brief histories of their development and applications in climate science, new and recent approaches towards improvement of reproducibility and replicability, and challenges. Recommendations for addressing those challenges include: development of searchable, auto-updated, interlinked, multi-archive public paleoclimate repositories for raw and processed digital datasets; cross-center standardized code base cases, improved data storage techniques, and a focus on replicability for climate simulation storage and access; and support of the development and community awareness of findable, accessible, interoperable and reusable (FAIR) principles by funding agencies and publishers. This paper is largely based on the May 2018 presentations of a panel of researchers to the Committee on Reproducibility and Replicability in Science, part of the National Academies of Science, Engineering, and Medicine. The commentary and recommendations made here are in alignment with those of its Consensus Study Report on Reproducibility and Replicability in Science (2019).

data repositories↗