SEARCH · Search NASA

Results for “data processing and archiving”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS2011

Landsat Data Continuity Mission

The Landsat Data Continuity Mission (LDCM) is currently under development and is on schedule to launch the 8th satellite in the Landsat series in December of 2012. LDCM is a joint project between the National Aeronautics and Space Administration (NASA) and the United States Geological Survey (USGS). NASA is responsible for developing and launching the flight hardware and on-orbit commissioning and USGS is responsible for developing the ground system and operating the system onorbit after commissioning. Key components of the flight hardware are the Operational Land Imager (OLI), nearing completion by Ball Aerospace & Technologies Corp in Boulder, CO, the Thermal Infrared Sensor (TIRS), being built by NASA's Goddard Space Flight Center and the spacecraft, undergoing integration at Orbital Sciences Corp in Gilbert, Arizona. The launch vehicle will be an Atlas-5 with launch services provided by NASA's Kennedy Space Center. Key ground systems elements are the Mission Operations Element, being developed by the Hammers Corporation, and the Collection Activity Planning Element, Ground Network Element, and Data Processing and Archive System, being developed internally by the USGS Earth Resources Observations and Science (EROS) Center. The primary measurement goal of LDCM is to continue the global coverage of moderate spatial resolution imagery providing continuity with the existing Landsat record. The science goal for this imagery is to monitor land use and land cover, particularly as it relates to global climate change. Together the OLI and TIRS instruments on LDCM replace the ETM+ instrument on Landsat-7 with significant enhancements. The OLI is a pushbroom design instrument where the scanning mechanism of the ETM+ is effectively replaced by a long line of detectors. The OLI has 9 spectral bands with similar spatial resolution to ETM+: 7 of them similar to the reflective spectral bands on ETM+ and two new bands. The two new bands cover (1) the shorter wavelength blue part of the spectrum to help with coastal studies and aerosol analyses/atmospheric correction and (2) an atmospheric water absorption band, where the Earth surface is generally not visible, but Cirrus clouds are, to aid in cloud detection and screening. The radiometry of OLI benefits from improved SNR, dynamic range and quantization. OLI is undergoing system testing with a delivery scheduled for Spring 2011. The TIRS is also a pushbroom design and used QWIPS detectors that require cooling to 43K using a cryocooler. It.has two spectral bands, effectively splitting the ETM+ band 6 in half, that can be used as a split window to aid in atmospheric correction. It has nominally 100 m spatial resolution as opposed to the 60 m of Landsat-7 ETM+: TIRS has commenced integration and test, with a delivery to the spacecraft vendor scheduled for Winter 2011-2012. The Orbital spacecraft currently being integrated for LDCM will have improved capabilities for pointing over previous missions. These capabilities will allow the OLI and TIRS instruments to point off-nadir the equivalent of one WRS-2 path to increase the chances of coverage for high priority targets, particularly in the event of natural disasters. Also, the pointing capability will allow the calibration of the OLI using the sun (roughly weekly), the moon (monthly), stars (during commissioning) and the Earth (at 90 deg from normal orientation, a.k.a., side slither) quarterly. The solar calibration will be used for OLI absolute and relative calibration, the moon for trending the stability of the OLI response, the stars will be used for Line of Sight determination and the side slither will be an alternate OLI and relative gain determination methodology. The spacecraft is scheduled to begin integration with the OLI instrument in Summer 2011. The LDCM data processing and archive system (DPAS), located at USGS EROS, generates the products for distribution to users. Like Landsat-7 this includes an image assessment system for characrizing instrument performance and updating calibration parameters. Products will be generated that include the spectral bands from both instruments, terrain corrected and registered to the geoid. Also, like Landsat-7, data products will be distributed at no charge to the user. The current status and plans of the space and ground segments of the LDCM project will be presented along with performance predictions as available. More detailed information on the two instruments is intended to be presented in separate papers.

Markham, Brian↗

NASA NTRSDate not supplied

New Ways of Facilitating Improved Data Discovery and Access for NASA's Suborbital Earth Science Observations

NASA conducts field research in various Earth Science disciplines utilizing airborne and other non-satellite platforms to acquire in situ and remotely sensed observations indicative of physical processes across a range of scales. Field efforts are key in the development and validation of instruments and satellite algorithm refinements. The heterogeneous data, with a range of file formats, scales, and acquisition methods, support research in several science areas. NASA’s archive process assigns data products to discipline-oriented Distributed Active Archive Centers (DAACs) for stewardship. Over time, individual DAACs have developed tools for data browsing and serving disparate user bases. As science becomes more interdisciplinary, researchers need to incorporate observations from multiple campaigns, and multiple DAACs, into their work. Motivated in part by this shifting paradigm of needs, the Catalog of Archived Suborbital Earth Science Investigations (CASEI) was created. CASEI provides a single starting point to browse, search, and discover airborne and field data. Contextual metadata are organized and inter-linked allowing intuitive, integrated exploration across all NASA DAACs. Campaign science objectives, platform and instrument configurations, geographical details, geophysical concepts, and more are tracked in CASEI’s database, facilitating multi-parameter search, browse, and discovery of relevant data products. Researchers are able to directly access associated data products, via DOI links, regardless of the DAAC where they reside. Significant events, key time periods of high science interest within the longer-duration campaign effort, are also indicated and allow for a more efficient identification of critical data subsets. This presentation describes CASEI’s development, intensive metadata curation process, and demonstrates the web interface experience. Initial content metrics and plans for continued maintenance will also be discussed.

metadata↗

NASA NTRS2003

Protein Crystal Growth

In order to rapidly and efficiently grow crystals, tools were needed to automatically identify and analyze the growing process of protein crystals. To meet this need, Diversified Scientific, Inc. (DSI), with the support of a Small Business Innovation Research (SBIR) contract from NASA s Marshall Space Flight Center, developed CrystalScore(trademark), the first automated image acquisition, analysis, and archiving system designed specifically for the macromolecular crystal growing community. It offers automated hardware control, image and data archiving, image processing, a searchable database, and surface plotting of experimental data. CrystalScore is currently being used by numerous pharmaceutical companies and academic and nonprofit research centers. DSI, located in Birmingham, Alabama, was awarded the patent Method for acquiring, storing, and analyzing crystal images on March 4, 2003. Another DSI product made possible by Marshall SBIR funding is VaporPro(trademark), a unique, comprehensive system that allows for the automated control of vapor diffusion for crystallization experiments.

Source record↗

NASA NTRS2011

Evaluating Cloud Computing in the Proposed NASA DESDynI Ground Data System

The proposed NASA Deformation, Ecosystem Structure and Dynamics of Ice (DESDynI) mission would be a first-of-breed endeavor that would fundamentally change the paradigm by which Earth Science data systems at NASA are built. DESDynI is evaluating a distributed architecture where expert science nodes around the country all engage in some form of mission processing and data archiving. This is compared to the traditional NASA Earth Science missions where the science processing is typically centralized. What's more, DESDynI is poised to profoundly increase the amount of data collection and processing well into the 5 terabyte/day and tens of thousands of job range, both of which comprise a tremendous challenge to DESDynI's proposed distributed data system architecture. In this paper, we report on a set of architectural trade studies and benchmarks meant to inform the DESDynI mission and the broader community of the impacts of these unprecedented requirements. In particular, we evaluate the benefits of cloud computing and its integration with our existing NASA ground data system software called Apache Object Oriented Data Technology (OODT). The preliminary conclusions of our study suggest that the use of the cloud and OODT together synergistically form an effective, efficient and extensible combination that could meet the challenges of NASA science missions requiring DESDynI-like data collection and processing volumes at reduced costs.

Benchmark↗

NASA NTRSDate not supplied

Hydrological Data at the NASA GES DISC: Current Capabilities and New Opportunities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of twelve NASA Earth science data centers that document, process, archive and distribute data from Earth observation missions and projects. GES DISC maintains an archive of several hydrology datasets, including the Land Data Assimilation Systems (LDAS) and the Gravity Recovery and Climate Experiment (GRACE) Data Assimilation for Drought Monitoring (GRACE-DA-DM) data products. These datasets include model output of heat fluxes, rain, snow, soil temperature, soil moisture, and runoff; and observational forcing data, including surface pressure, temperature, precipitation, downward shortwave and longwave radiation, humidity, and wind. The temporal resolution of the hydrology data at GES DISC ranges from hourly to monthly, and spatial resolutions range from 0.1° to 1.0°. The GES DISC provides services which enable users to aggregate, temporally and spatially subset, regrid, and visualize archived data including the GES DISC Subsetter, Hydrology Data Rods, and the Geospatial Interactive Online Visualization and Analysis Infrastructure (GIOVANNI). The Hydrology Data Rods service optimally reorganizes large hydrological data sets as extended time series, providing more efficient access for the hydrological community. The time series data (aka “data rods”) were integrated into hydrology community tools, such as the Data Rods Explorer on HydroShare. Furthermore, the GES DISC is in the process of migrating its data and services to the cloud. Hydrological data available at the GES DISC are now available in the Amazon Web Services (AWS) cloud (us-west-2 region) providing users Direct S3 data access and the capability for cloud computing operations. In this presentation, the hydrology data products and services currently available at the GES DISC will be summarized. Also discussed are the migration to the cloud, user support through this transition, and the status of migrating the data rods service to the cloud.

Ashley Heath↗

NASA NTRS1996

Radarsat Processing System at ASF

Radarsat is a Canadian polar orbiting remote sensing satellite scheduled for launch in September 1995. Its lone instrument on-board is a synthetic aperture radar (SAR) that is capable of operating in a number of imaging modes including the first operational ScanSAR mode. As one of the data reception, processing and archive facility for Radarsat data, Alaska SAR Facility (ASF) has responded to its Science users by establishing a Radarsat processing system to handle the data processing of all Radarsat modes. This task involves enhancements to the high throughput hardware based Alaska SAR Processor (ASP) to handle standard mode Radarsat data in addition to its existing ERS and JERS capabilities, the addition of the new ScanSAR Processor (SSP) to process the Radarsat ScanSAR mode data, and the introduction of a Precision Processor (PP) to accommodate the special Radarsat modes such as fine resolution and wide swath. For raw data ingestion and distribution to the appropriate SAR processor, a new Control Processor (CP) and Raw Data Scanner (RDS) subsystem is also incorporated.

Radarsat↗

NASA NTRS2019

Assimilated Hydrological Data at NASA GES DISC with Examples of Extreme Events

Extreme weather and climate events, such as heavy rainfall, heatwave, floods and droughts, and strong wind, can have devastating impacts on society. NASA and NOAA, based on independent analyses, recently announced that global surface temperatures in 2018 are the fourth warmest since 1880, behind only those of 2016, 2017, and 2015 (nasa.gov). Also in 2018, the United States experienced 14 billion-dollar disasters, ranking as the fourth highest total number of such events, behind only the years 2017, 2011, and 2016 (climate.gov). Many research studies have focused on acquiring observational and modeling data, to reveal linkages between increasing extreme events, global water and energy cycle, and global climate change. However, draw conclusions is still a challenge. NASA Goddard Earth Sciences Data and Information Services Center is one of twelve NASA Earth Observing System (EOS) data centers that process, archive, document, and distribute data from Earth science missions and related projects. The GES DISC hosts a wide range of remotely-sensed and model data and provides reliable and robust data access and services to users worldwide. This presentation provides a few examples of extreme event study that use Land Surface Model (LSM) assimilated, quality-controlled, and spatially and temporally consistent, hydrological data from the GES DISC. Also provided is a summary table for the hydrological data holdings, along with discussions of recent updates to data and data services.

Rui, Hualan↗

NASA NTRS2020

Famine Early Warning Systems Network (FEWS NET) Land Data Assimilation System (LDAS) and Other Assimilated Hydrological Data at NASA GES DISC

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) provides science support for several data sets relevant to agriculture and food security, including the Famine Early Warning Systems Network (FEWS NET) Land Data Assimilation System (LDAS), or FLDAS data set. The GES DISC is one of twelve NASA Earth Observing System (EOS) data centers that process, archive, document, and distribute data from Earth science missions and related projects. The GES DISC hosts a wide range of remote sensing and model data, and provides reliable and robust data access and other services to users worldwide. Beyond data archive and access, the GES DISC offers many services to visualize and analyze the data. This presentation provides a summary of the hydrological data available at the GES DISC, along with an overview of related data services. Specifically, the FLDAS data set has been adapted to work with domains, data streams, and monitoring and forecast requirements associated with food security assessment in data-sparse, developing country settings. The FLDAS global monthly data have a 0.1 x 0.1 degree spatial resolution covering the period from January 1982 to present. Global FLDAS monthly anomaly and monthly climatology data are also available at the GES DISC to evaluate how current conditions compare to averages over the FLDAS 35-year period. Several case studies using the FLDAS soil moisture, evapotranspiration, rainfall, runoff, and surface temperature data will be presented.

Loeser, Carlee↗

NASA NTRSDate not supplied

An Update on the Hydrological Land Surface Data and Services at NASA GES DISC

"The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of twelve NASA Earth Observing System data centers that document, process, archive and distribute data from Earth science missions and related projects. The GES DISC hosts many hydrological land surface data products including North American Land Data Assimilation System (NLDAS), Global Land Data Assimilation System (GLDAS), the Famine Early Warning Systems Network (FEWS NET) Land Data Assimilation System (FLDAS), and assimilated-Gravity Recovery and Climate Experiment (GRACE). The Land Surface Model (LSM) products contain model output, including heat fluxes, rain, snow, soil temperature, soil moisture and runoff; and observational forcing data, including surface pressure, temperature, precipitation, downward shortwave and longwave radiation, humidity, and wind. The temporal resolution of the hydrology data at the GES DISC ranges from hourly to monthly and the spatial resolution 0.1 degree to 1.0 degree. The variability of Earth’s water and energy cycles can be characterized with these spatially and temporally consistent, and quality controlled hydrological land surface data. These data could support ongoing modeling activities, and help improve our understanding of land-surface-atmosphere interactions and their impact on climate. Over the years, new data is added, and older data is reprocessed and updated. This presentation provides a summary table of the hydrological data and discusses recent updates to the data and data services at the GES DISC. New data products include (1) GRACE Global Version 3.0 drought indicators and (2) FLDAS monthly global and Central Asia daily data. Reprocessed and updated products include (1) NLDAS Version 2.0 and (2) GRACE United States Version 4.0 drought indicators. Lastly, this presentation will discuss the opportunities for accessing some of our hydrology data from the cloud. As the hydrology land surface data is moved to the cloud, users can expect faster service for longer time series, notably the hydrology data rods time series service."

Ashley Heath↗

NASA NTRSDate not supplied

How GES DISC is Opening Doors to Open Science

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of 12 NASA Earth Observing System data centers that document, process, archive, and distribute data from Earth science missions and related projects. GES DISC hosts numerous Earth Science products through various data formats and supports their access with several programming and website tools. The User Needs team at GES DISC is responsible for producing tutorials and resources to teach these tools, collecting and engaging with user feedback, and collaborating with other NASA data centers to enable science creation for everyone. This poster presentation outlines how GES DISC, and the User Needs team, are committed to enabling open science.

Christopher Battisto↗

NASA NTRSDate not supplied

How GES DISC is Opening Doors to Open Science

Christopher Battisto↗

NASA NTRS2000

BOREAS Level-4b AVHRR-LAC Ten-Day Composite Images: At-sensor Radiance

The BOReal Ecosystem-Atmosphere Study (BOREAS) Staff Science Satellite Data Acquisition Program focused on providing the research teams with the remotely sensed satellite data products they needed to compare and spatially extend point results. Manitoba Remote Sensing Center (MRSC) and BOREAS Information System (BORIS) personnel acquired, processed, and archived data from the Advanced Very High Resolution Radiometer (AVHRR) instruments on the National Oceanic and Atmospheric Administration (NOAA-11) and -14 satellites. The AVHRR data were acquired by CCRS and were provided to BORIS for use by BOREAS researchers. These AVHRR level-4b data are gridded, 10-day composites of at-sensor radiance values produced from sets of single-day images. Temporally, the 10- day compositing periods begin 11-Apr-1994 and end 10-Sep-1994. Spatially, the data cover the entire BOREAS region. The data are stored in binary image format files.

Cihlar, Josef↗

NASA NTRS2000

BOREAS Level-4c AVHRR-LAC Ten-Day Composite Images: Surface Parameters

The BOReal Ecosystem-Atmosphere Study (BOREAS) Staff Science Satellite Data Acquisition Program focused on providing the research teams with the remotely sensed satellite data products they needed to compare and spatially extend point results. Manitoba Remote Sensing Center (MRSC) and BOREAS Information System (BORIS) personnel acquired, processed, and archived data from the Advanced Very High Resolution Radiometer (AVHRR) instruments on the NOAA-11 and -14 satellites. The AVHRR data were acquired by CCRS and were provided to BORIS for use by BOREAS researchers. These AVHRR level-4c data are gridded, 10-day composites of surface parameters produced from sets of single-day images. Temporally, the 10-day compositing periods begin 11-Apr-1994 and end 10-Sep-1994. Spatially, the data cover the entire BOREAS region. The data are stored in binary image format files. Note: Some of the data files on the BOREAS CD-ROMs have been compressed using the Gzip program.

Cihlar, Josef↗

NASA NTRS1991

Version 0 EOSDIS - An overview

Attention is given to NASA's Earth Observing System Data and Information System (EOSDIS), which is to be a single, distributed but internally consistent evolutionary system to support the planning and execution of EOS data acquisitions and to process, archive, and distribute EOS data products and selected non-EOS data to enable interdisciplinary studies of the earth. V0 EOSDIS, a logical step in this evolutionary process, is to address both technical and managerial challenges. Technical challenges include developing a multidiscipline, distributed system for searching and ordering data in a heterogeneous environment, and standardizing data formats and distribution techniques among differing communities and organizations. Managerial challenges include establishing and maintaining a structure consisting of geographically distributed entities such that cooperative development is carried out effectively despite organizational differences, maintaining interactions with the scientific community to ensure its close involvement despite its size and diversity, and keeping the expectations for V0 consistent with its schedules and resources.

Ramapriyan, H. K.↗

DOE OSTI2024

Synthetic Streamflow Datasets to Support Emulation of Water Allocations via LSTM

This archive is the data companion to the bonney_et-al_2026_erc metarepo which generates synthetic data, trains an LSTM model, and generates performance metrics on the trained model. While the generation of the synthetic data is fully reprodicible, it is a computationally expensive process. This data archive contains the synthetic datasets needed for training and testing an LSTM model and reproduction of figures and tables. In addition, supplemenatary data products generating and visualizing results is also included, such as geospatial data for the basin. Contents There are two high level directories: `WRAP_archive/` and `repo_data/`. The `WRAP_archive` directory contains compressed intermediate dataproducts from the dataset generation workflow (marked as "I_Dataset_Generation" in the metarepo). These data products are not required by any scripts in the metarepo, but they are archived as they are expensive to generate and may have useful information for other analyses. The `repo_data` directory contains the necessary data for reproducing the workflow in the metarepo and should be decompressed and moved into the top level of the metarepo. Additional details are provided in README.md.

drought↗

NASA NTRS2011

Automated X-ray and Optical Analysis of the Virtual Observatory and Grid Computing

We are developing a system to combine the Web Enabled Source Identification with X-Matching (WESIX) web service, which emphasizes source detection on optical images,with the XAssist program that automates the analysis of X-ray data. XAssist is continuously processing archival X-ray data in several pipelines. We have established a workflow in which FITS images and/or (in the case of X ray data) an X-ray field can be input to WESIX. Intelligent services return available data (if requested fields have been processed) or submit job requests to a queue to be performed asynchronously. These services will be available via web services (for non-interactive use by Virtual Observatory portals and applications) and through web applications (written in the Django web application framework). We are adding web services for specific XAssist functionality such as determining .the exposure and limiting flux for a given position on the sky and extracting spectra and images for a given region. We are improving the queuing system in XAssist to allow for "watch lists" to be specified by users, and when X-ray fields in a user's watch list become publicly available they will be automatically added to the queue. XAssist is being expanded to be used as a survey planning 1001 when coupled with simulation software, including functionality for NuStar, eRosita, IXO, and the Wide Field Xray Telescope (WFXT), as part of an end to end simulation/analysis system. We are also investigating the possibility of a dedicated iPhone/iPad app for querying pipeline data, requesting processing, and administrative job control.

Ptak, A.↗

NASA NTRS1993

High-performance mass storage system for workstations

Reduced Instruction Set Computer (RISC) workstations and Personnel Computers (PC) are very popular tools for office automation, command and control, scientific analysis, database management, and many other applications. However, when using Input/Output (I/O) intensive applications, the RISC workstations and PC's are often overburdened with the tasks of collecting, staging, storing, and distributing data. Also, by using standard high-performance peripherals and storage devices, the I/O function can still be a common bottleneck process. Therefore, the high-performance mass storage system, developed by Loral AeroSys' Independent Research and Development (IR&D) engineers, can offload a RISC workstation of I/O related functions and provide high-performance I/O functions and external interfaces. The high-performance mass storage system has the capabilities to ingest high-speed real-time data, perform signal or image processing, and stage, archive, and distribute the data. This mass storage system uses a hierarchical storage structure, thus reducing the total data storage cost, while maintaining high-I/O performance. The high-performance mass storage system is a network of low-cost parallel processors and storage devices. The nodes in the network have special I/O functions such as: SCSI controller, Ethernet controller, gateway controller, RS232 controller, IEEE488 controller, and digital/analog converter. The nodes are interconnected through high-speed direct memory access links to form a network. The topology of the network is easily reconfigurable to maximize system throughput for various applications. This high-performance mass storage system takes advantage of a 'busless' architecture for maximum expandability. The mass storage system consists of magnetic disks, a WORM optical disk jukebox, and an 8mm helical scan tape to form a hierarchical storage structure. Commonly used files are kept in the magnetic disk for fast retrieval. The optical disks are used as archive media, and the tapes are used as backup media. The storage system is managed by the IEEE mass storage reference model-based UniTree software package. UniTree software will keep track of all files in the system, will automatically migrate the lesser used files to archive media, and will stage the files when needed by the system. The user can access the files without knowledge of their physical location. The high-performance mass storage system developed by Loral AeroSys will significantly boost the system I/O performance and reduce the overall data storage cost. This storage system provides a highly flexible and cost-effective architecture for a variety of applications (e.g., realtime data acquisition with a signal and image processing requirement, long-term data archiving and distribution, and image analysis and enhancement).

Chiang, T.↗

DOE OSTI2025

Utah FORGE: LBNL Reports on VEMP Electromagnetic Data Collection and Processing - 2024

This archive contains reports related to Vertical Electromagnetic Profiling (VEMP) tool data collection and processing at Utah FORGE in 2024. The first report describes LBNL's effort to collect electromagnetic geophysical data with the tool in well 78-32B and a downhole electrode in well 16A. The second report describes the final data acquisition and processing of the VEMP electromagnetic data collected at the Utah FORGE site in May of 2024. Also included are a noise analysis as well as a comparison of the data to numerical models. This was originally presented as a paper at the 2025 Stanford Geothermal Workshop.

15 GEOTHERMAL ENERGY↗