Search NASA⌕ Search

SEARCH · Search NASA

Results for “data processing and archiving”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

NASA ESDS Citizen Science Data Working Group

This document provides guidelines for legal, policy, and ethical issues; standards for citizen science data collection and management; information on ensuring usability of citizen science data and communication regarding its use; and best practices for long-term archival of citizen science data.Section1 contains a detailed discussion of policy, ethical, and legal considerations influencing citizen science data collection. Section 2 considers standards for documentation, including documentation of instrumentation, procedures, and the data itself. It concludes with a discussion of how citizen science data should be attributed. Section 3 provides guidance about how to ensure citizen science data are collected and stored in a useable way. It also considers how NASA and data producers should notify the scientific community, including citizen scientists and the public, about citizen science datasets and the scientific conclusions reached using them. Finally, Section 4 provides detailed information regarding what should be archived from projects using a citizen science approach, including data and code. It provides guidance about archive location, process, and timeframe, as well as information about data access and distribution services provided by NASA that may be relevant to data producers working with citizen scientists.

Citizen Science↗

Preservation and Enhancement of the Spacewatch Data Archives

In March of 1998, the asteroid 1997 XF11 was announced to be potentially hazardous after being tracked over 90 days. A potential two year wait for confirming observations was shortened to under 24 hours because of the existence of archived photographic prediscovery images. Spacewatch was a pioneer in using CCD scanning and possesses a valuable digital archive of its scans. Unfortunately these data are aging on magnetic tape and will soon be lost. Since 1990, the Spacewatch project gathered some 1.5 Terabytes of scan data covering roughly 75,000 degrees of sky to a limiting magnitude of V = 21.5. The data have not yet been mined for all of their asteroids for scientific studies and orbit determination. Spacewatch's real-time motion detection program MODP was constrained by the computers of the era to use simplified image processing algorithms at a reduced efficiency. Jedicke and Herron estimated MODP's efficiency at finding asteroids to be approximately 60 percent to V=18 and improving somewhat thereafter. This lead to a substantial bias correction in their analyses. Larsen has developed a MODP replacement capable in excess of 90 percent efficiency in the same range and able to push a magnitude fainter in completeness. We propose a program of post-processing and re-archiving Spacewatch data. Our scans would be transferred from tape to CD-ROMs and converted to FITS images -- establishing a consistent data format and media for both past and future Spacewatch observations. Larsen's MODP replacement would mine these data for previously undetected motions, which would be made available to the Minor Planet Center and our ongoing asteroid population studies. A searchable observation record would be made generally available for prediscovery work. We estimate the net asteroid yield of this proposal is equivalent to three full years of Spacewatch operations.

Larsen, Jeffrey A.↗

Improving Data Discovery, Analysis, and Visualizations With Cloud-Based User Services

The Global Hydrometeorology Resource Center (GHRC) Distributed Active Archive Center (DAAC) is one of 12 DAACs managed by the United States National Aeronautics and Space Administration (NASA) Earth Science Data and Information System (ESDIS) project [1]. GHRC and the other DAACs are designed to process, archive, document, and distribute NASA Earth-observing data, ranging from satellite missions to field campaigns [2]. A major goal of the DAACs is to enable science with these data. Science enabling can be difficult as datasets can be very large, use multiple formats, come from numerous platforms, and require three-dimensional visualization. GHRC is using its expertise with cloud-based technologies to develop open source and open science tools to empower users to explore, coincidentally visualize, and analyze multiple datasets. Being open source, the user community can develop visualizations for their own datasets. This presentation will expand on this objective and highlight the capabilities available to the international community now.

GHRC↗

A Complete UV Atlas of Standard Stars

This program is an Astrophysics Data Program (ADP) contract, together with a follow-on contract (S- 57791 -Z), intended to produce a comprehensive ultraviolet spectral atlas of stars based on the data in the IUE Final Archive. The data in the Final Archive were uniformly processed by the NEWSIPS pipeline system. The NEWSIPS gives an improved signal-to-noise ratio for the data that will be especially beneficial to low signal portions of the spectra; and it also gives more accurate fluxes for stars by adopting the latest absolute calibration and measured instrument parameters. Therefore, this atlas provides more uniform and accurate data than the version previously published.

Wu, Chi-Chao↗

Data compression for data archival, browse or quick-look

Soon after space and Earth science data is collected, it is stored in one or more archival facilities for later retrieval and analysis. Since the purpose of the archival process is to keep an accurate and complete record of data, any data compression used in an archival system must be lossless, and protect against propagation of error in the storage media. A browse capability for space and Earth science data is needed to enable scientists to check the appropriateness and quality of particular data sets before obtaining the full data set(s) for detailed analysis. Browse data produced for these purposes could be used to facilitate the retrieval of data from an archival facility. Quick-look data is data obtained directly from the sensor for either previewing the data or for an application that requires very timely analysis of the space or Earth science data. Two main differences between data compression techniques appropriate to browse and quick-look cases, are that quick-look can be more specifically tailored, and it must be limited in complexity by the relatively limited computational power available on space platforms.

Dozier, Jeff↗

The Suomi National Polar-Orbiting Partnership (SNPP): Continuing NASA Research and Applications

The Suomi National Polar-orbiting Partnership (SNPP) satellite was successfully launched into a polar orbit on October 28, 2011 carrying 5 remote sensing instruments designed to provide data to improve weather forecasts and to increase understanding of long-term climate change. SNPP provides operational continuity of satellite-based observations for NOAA's Polar-orbiting Operational Environmental Satellites (POES) and continues the long-term record of climate quality observations established by NASA's Earth Observing System (EOS) satellites. In the 2003 to 2011 pre-launch timeframe, NASA's SNPP Science Team assessed the adequacy of the operational Raw Data Records (RDRs), Sensor Data Records (SDRs), and Environmental Data Records (EDRs) from the SNPP instruments for use in NASA Earth Science research, examined the operational algorithms used to produce those data records, and proposed a path forward for the production of climate quality products from SNPP. In order to perform these tasks, a distributed data system, the NASA Science Data Segment (SDS), ingested RDRs, SDRs, and EDRs from the NOAA Archive and Distribution and Interface Data Processing Segments, ADS and IDPS, respectively. The SDS also obtained operational algorithms for evaluation purposes from the NOAA Government Resource for Algorithm Verification, Independent Testing and Evaluation (GRAVITE). Within the NASA SDS, five Product Evaluation and Test Elements (PEATEs) received, ingested, and stored data and performed NASA's data processing, evaluation, and analysis activities. The distributed nature of this data distribution system was established by physically housing each PEATE within one of five Climate Analysis Research Systems (CARS) located at either at a NASA or a university institution. The CARS were organized around 5 key EDRs directly in support of the following NASA Earth Science focus areas: atmospheric sounding, ocean, land, ozone, and atmospheric composition products. The PEATES provided the system level interface with members of the NASA SNPP Science Team and other science investigators within each CARS. A sixth Earth Radiation Budget CARS was established at NASA Langley Research Center (NASA LaRC) to support instrument performance, data evaluation, and analysis for the SNPP Clouds and the Earth's Radiant Budget Energy System (CERES) instrument. Following the 2011 launch of SNPP, spacecraft commissioning, and instrument activation, the NASA SNPP Science Team evaluated the operational RDRs, SDRs, and EDRs produced by the NOAA ADS and IDPS. A key part in that evaluation was the NASA Science Team's independent processing of operational RDRs and SDRs to EDRs using the latest NASA science algorithms. The NASA science evaluation was completed in the December 2012 to April 2014 timeframe with the release of a series of NASA Science Team Discipline Reports. In summary, these reports indicated that the RDRs produced by the SNPP instruments were of sufficiently high quality to be used to create data products suitable for NASA Earth System science and applications. However, the quality of the SDRs and EDRs were found to vary greatly when considering suitability for NASA science. The need for improvements in operational algorithms, adoption of different algorithmic approaches, greater monitoring of on-orbit instrument calibration, greater attention to data product validation, and data reprocessing were prominent findings in the reports. In response to these findings, NASA, in late 2013, directed the NASA SNPP Science Team to use SNPP instrument data to develop data products of sufficiently high quality to enable the continuation of EOS time series data records and to develop innovative, practical applications of SNPP data. This direction necessitated a transition of the SDS data system from its pre-launch assessment mode to one of full data processing and production. To do this, the PEATES, which served as NASA's data product testing environment during the prelaunch and early on-orbit periods, were transitioned to Science Investigator-led Processing Systems (SIPS). The distributed data architecture was maintained in this new system by locating the SIPS at the same institutions at which the CARS and PEATES were located. The SIPS acquire raw SNPP instrument Level 0 (i.e. RDR) data over the full SNPP mission from the NOAA ADS and IDPS through the NASA SDS Data Distribution and Depository Element (SD3E). The SIPS process those data into NASA Level 1, Level 2, and global, gridded Level 3 standard products using peer-reviewed algorithms provided by members of the NASA Science Team. The SIPS work with the NASA SNPP Science Team in obtaining enhanced, refined, or alternate real-time algorithms to support the capabilities of the Direct Readout Laboratory (DRL). All data products, algorithm source codes, coefficients, and auxiliary data used in product generation are archived in an assigned NASA Distributed Active Archive Center (DAAC).

remote sensing↗

New developments in space radiation research at NASA: Annotating data using a novel radiation biology ontology

Like many interdisciplinary sciences, data producers and consumers in the field of radiation biology often use a wide variety of terminology to describe their experiments and data. Furthermore, space systems and technologies are rapidly evolving, and a shared understanding and common terminology for these is also lacking. The efficiency of research organizations can be enhanced by standardizing metadata through the use of knowledge resources like ontologies. Employing a sophisticated model such as a formal ontology to standardize metadata enables automated data acquisition processes and supports more complete, accurate meta-analysis through more efficient and complete data discovery and retrieval, particularly when using multiple data sources. Thus, we developed the Radiation Biology Ontology (RBO) in order to improved radiation biology metadata uniformity and transparency. We used open-source software (the Ontology Development Kit, Protégé and WebProtégé) and worked within the OBO Foundry framework, which includes a set of ontology development principles and practices for ontology consistency, uniformity, and accountability. The RBO has now been incorporated into two radiation research data repositories, NASA’s GeneLab omics database (https://genelab.nasa.gov), and the European Commission STORE database (https://www.storedb.org/). Continuous build integration tools allowed our international RBO collaboration to be more efficient and focus its efforts on semantic model design. Currently, the RBO contains over 300 annotated classes and individuals specific to the study of radiation on biological systems, as well as imports of many additional classes from other OBO Foundry ontologies that relate to and/or provide context for these RBO entities. We publish the RBO through the OBO Foundry, so that it is available for browsing, download, and querying through NCBI Bioportal web site and application programming interface. The NASA Ames Life Science Data Archive (ALSDA) is also in the process of adopting use of the RBO, taking NASA one step closer to a knowledge-based system for space biology data. It is our hope that the global communities of radiation research Investigators, data curators and data analysts can similarly leverage the RBO and will contribute to its further development.

radiation↗

NASA Tech Briefs, December 2007

Topics include: Ka-Band TWT High-Efficiency Power Combiner for High-Rate Data Transmission; Reusable, Extensible High-Level Data-Distribution Concept; Processing Satellite Imagery To Detect Waste Tire Piles; Monitoring by Use of Clusters of Sensor-Data Vectors; Circuit and Method for Communication Over DC Power Line; Switched Band-Pass Filters for Adaptive Transceivers; Noncoherent DTTLs for Symbol Synchronization; High-Voltage Power Supply With Fast Rise and Fall Times; Waveguide Calibrator for Multi-Element Probe Calibration; Four-Way Ka-Band Power Combiner; Loss-of-Control-Inhibitor Systems for Aircraft; Improved Underwater Excitation-Emission Matrix Fluorometer; Metrology Camera System Using Two-Color Interferometry; Design and Fabrication of High-Efficiency CMOS/CCD Imagers; Foam Core Shielding for Spacecraft CHEM-Based Self-Deploying Planetary Storage Tanks Sequestration of Single-Walled Carbon Nanotubes in a Polymer PPC750 Performance Monitor Application-Program-Installer Builder Using Visual Odometry to Estimate Position and Attitude Design and Data Management System Simple, Script-Based Science Processing Archive Automated Rocket Propulsion Test Management Online Remote Sensing Interface Fusing Image Data for Calculating Position of an Object Implementation of a Point Algorithm for Real-Time Convex Optimization Handling Input and Output for COAMPS Modeling and Grid Generation of Iced Airfoils Automated Identification of Nucleotide Sequences Balloon Design Software Rocket Science 101 Interactive Educational Program Creep Forming of Carbon-Reinforced Ceramic-Matrix Composites Dog-Bone Horns for Piezoelectric Ultrasonic/Sonic Actuators Benchtop Detection of Proteins Recombinant Collagenlike Proteins Remote Sensing of Parasitic Nematodes in Plants Direct Coupling From WGM Resonator Disks to Photodetectors Using Digital Radiography To Image Liquid Nitrogen in Voids Multiple-Parameter, Low-False-Alarm Fire-Detection Systems Mosaic-Detector-Based Fluorescence Spectral Imager Plasmoid Thruster for High Specific-Impulse Propulsion Analysis Method for Quantifying Vehicle Design Goals Improved Tracking of Targets by Cameras on a Mars Rover Sample Caching Subsystem Multistage Passive Cooler for Spaceborne Instruments GVIPS Models and Software Stowable Energy-Absorbing Rocker-Bogie Suspensions

Source record↗

The Kepler Data Processing Handbook: A Field Guide to Prospecting for Habitable Worlds

The Kepler telescope hurtled into orbit in March 2009, initiating NASA's first mission to discover Earth-size planets orbiting Sun-like stars. Kepler simultaneously collected data for approximately 165,000 target stars at a time over its four-year mission, identifying over 4700 planet candidates, over 2300 confirmed or validated planets, and over 2100 eclipsing binaries. While Kepler was designed to discover exoplanets, the long-term, ultrahigh photometric precision measurements it achieved made it a premier observational facility for stellar astrophysics, especially in the field of asteroseismology, and for variable stars, such as RR Lyrae. The Kepler Science Operations Center (SOC) was developed at NASA Ames Research Center to process the data acquired by Kepler from pixel-level calibrations all the way to identifying transiting planet signatures and subjecting them to a suite of diagnostic tests to establish or break confidence in their planetary nature. Detecting small, rocky planets transiting Sun-like stars presents a variety of daunting challenges, including achieving an unprecedented photometric precision of ~20 ppm on 6.5-hour timescales, and supporting the science operations, management, processing, and repeated reprocessing of the accumulating data stream. A newly revised and expanded version of the Kepler Data Processing Handbook (KDPH) has been released to support the legacy archival products. The KDPH details the theory, design and performance of the algorithms supporting each data processing step. This paper presents an overview of the KDPH and features illustrations of several key algorithms in the Kepler Science Data Processing Pipeline. Kepler was selected as the 10th mission of the Discovery Program. Funding for this mission is provided by NASA, Science Mission Directorate.

high performance computing↗

Science data processing in the Mission to Planet Earth era

The data-processing systems developed to support NASA's Mission to Planet Earth (MPE) are described in an overview of the system architectures. The MPE program is discussed in terms of flight segments and data types to examine the required elements of the information systems. The data-processing segments are described for the Atmospheric Laboratory for Applications and Science, the Tropical Rainfall Measuring Mission, the Earth Observing System, and the Geostationary Earth Observatory. The data and information systems for the separate segments provide some redundant processing services. Distributed active archive centers are employed for each segment composed of three subelements: (1) a product-generation system, (2) a data archive and distribution system, and (3) an information management system. The data processing design for the MPE is expected to facilitate the understanding of the entire earth system on a global scale.

Goodman, H. M.↗

Demonstrating Acquisition of Real-Time Thermal Data over Fires Utilizing UAVs

NASA-Ames Research Center, in collaboration with General Atomics Aeronautical Systems, Inc. has been developing real-time data acquisition and information delivery systems employing uninhabited aerial vehicle (UAV) technology for disaster mitigation and assessment demonstrations. Working in conjunction with the US Forest Service, a disaster community agency responsible for wildfire management and mitigation, we developed a large-scale wildfire demonstration called the First Response Experiment (FIRE). During that experiment in late summer 2001, the participants demonstrated the melding of innovative technologies such as UAV platforms, real-time data processing, and data telemetry for quick analysis of a disaster event. The General Atomics ALTUS UAV, the Airborne Infrared Disaster Assessment System (AIRDAS) and Over-The-Horizon (OTH) satellite data telemetry equipment were employed over a controlled burn to test the feasibility of a disaster monitoring and mitigation platform for hazardous duty. The ALTUS UAV was employed to demonstrate the long duration, altitude, and payload capability of unmanned platforms for acquiring disaster related data. The ALTUS has an operational altitude to 45,000 feet (13,700 in), with a flight duration of twenty-four hours and a payload capacity of over 300 lbs. (148.5 kg). This allows the platform to operate under the conditions that would be necessary for monitoring and mitigating disaster events throughout the Unites States. The four channel AIRDAS data (calibrated thermal infrared digital imagery of the fire event) was sent from the ALTUS UAV via a satellite communications system (NERA transponder and INMARSAT satellite) to a data archive server and an image processing work station at NASA-Ames Research Center, 400 miles away.

Wegener, Steven S.↗

Design of Space Station era mass storage systems

The architecture, components, and technical approach for a Space Station era mass storage system prototype are described. The primary data storage needs call for such functions as line outage recording, rate buffering, data archiving, and level zero type processing. The prototype mass storage system hardware will be configurable to support three modes of operation at up to 300 Mbps. The system architecture consists of two distinct components: a configurable high-level mass storage control system and a basic storage kernal.

Horner, Ward P.↗

Radio-science performance analysis software

The Radio Science Systems Group (RSSG) provides various support functions for several flight project radio-science teams. Among these support functions are uplink and sequence planning, real-time operations monitoring and support, data validation, archiving and distribution functions, and data processing and analysis. This article describes the support functions that encompass radio-science data performance analysis. The primary tool used by the RSSG to fulfill this support function is the STBLTY program set. STBLTY is used to reconstruct observable frequencies and calculate model frequencies, frequency residuals, frequency stability in terms of Allan deviation, reconstructed phase, frequency and phase power spectral density, and frequency drift rates. In the case of one-way data, using an ultrastable oscillator (USO) as a frequency reference, the program set computes the spacecraft transmitted frequency and maintains a database containing the in-flight history of the USO measurements. The program set also produces graphical displays. Some examples and discussions on operating the program set on Galileo and Ulysses data will be presented.

Morabito, D. D.↗

A Complete UV Atlas of Standard Stars

The general objective of this project is to provide a comprehensive ultraviolet spectral atlas of stars based on the data in the IUE Final Archive. The data have been uniformly processed by the IUE NEWSIPS pipeline system. The NEWSIPS gives an improved signal to noise ratio for the data that will be especially beneficial to low signal portions of the spectra; and it also gives more accurate fluxes for stars by adopting the latest absolute calibration and measured instrument parameters. Our new atlas, therefore, provides more uniform and accurate data than the version previously published. In addition, considerable efforts were devoted to obtain reliable spectral types, V,B-V, and E(B-V). Our goal is to provide the information which can be used with reasonable confidence by scientists.

Wu, Chi-Chao↗

Tracking Provenance of Earth Science Data

Tremendous volumes of data have been captured, archived and analyzed. Sensors, algorithms and processing systems for transforming and analyzing the data are evolving over time. Web Portals and Services can create transient data sets on-demand. Data are transferred from organization to organization with additional transformations at every stage. Provenance in this context refers to the source of data and a record of the process that led to its current state. It encompasses the documentation of a variety of artifacts related to particular data. Provenance is important for understanding and using scientific datasets, and critical for independent confirmation of scientific results. Managing provenance throughout scientific data processing has gained interest lately and there are a variety of approaches. Large scale scientific datasets consisting of thousands to millions of individual data files and processes offer particular challenges. This paper uses the analogy of art history provenance to explore some of the concerns of applying provenance tracking to earth science data. It also illustrates some of the provenance issues with examples drawn from the Ozone Monitoring Instrument (OMI) Data Processing System (OMIDAPS) run at NASA's Goddard Space Flight Center by the first author.

Tilmes, Curt↗

The Open Data Repository's Data Publisher

We have quickly gathered a diverse set of databases that have significant activity. Researchers are using them on a daily basis from collection through all phases of research. Focus on: (1) Linked data and semantic web integration are fundamental to our long term plans. (2) Citation system is critical before our first full release. Snapshot-based citation generation for data sets or objects. As we expand our feature set, expect the system will: (1) Provide a very high level of provenance for any data housed in the software. (2) Make data management beneficial to the researcher throughout the research process. (3) Create living archives that allow citable snapshots of data and summaries of the changes since the snapshot was created.

Stone, N.↗

The ICESat-2 Mission: Land, Ocean, and Inland Water Data Products for Middle and Low Latitude Science and Applications

NASA’s Ice, Cloud, and Land Elevation Satellite (ICESat-2) is a polar orbiting mission, launched on September 15, 2018, with over two years of nearly continuous observations. The sole instrument onboard is the Advanced Topographic Laser Altimeter System (ATLAS), a micropulse, high repetition rate, six-beam, 532 nm Lidar with photon-counting technology. Although designed primarily for detecting height changes in ice caps and sea ice in the high latitudes, it continuously observes all terrain in its track including middle and low latitude regions as well, during approximately 15 orbits per day. The official ICESat-2 products include not only cryosphere data but also global high resolution parameters associated with tree canopies, land surface, oceans, and over 1.5 million inland water bodies consisting of lakes, rivers and coastal waters. This presentation provides an overview and the status of the ICESat-2 mission including: i) a summary of the salient technological features and orbit design, ii) the official ICESat-2 science data products for ice, vegetation canopy, ocean sea level, and inland water including several examples in the middle and low latitudes, iii) a list of NASA sponsored software tools for globally browsing the two year archive and for processing both the raw data and the official ICESat-2 geophysical data products, and iv) avenues for interested users to connect with the ICESat-2 Applications Program for answering questions and assistance in possibly using ICESat-2 data in your particular science or application. The overall goal is to facilitate the use of ICESat-2 data not only for science investigations but also for improved decision support applications with societal benefit. Additional ICESat-2 information, documentation and data products are publicly available at https://nsidc.org/data/icesat-2

ICESat-2↗