Search NASA⌕ Search

SEARCH · Search NASA

Results for “Remote Sensing; Data Storage;”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Real-time and post-hoc compression for data from Distributed Acoustic Sensing

Distributed Acoustic Sensing (DAS) is an emerging sensing technology that records the strain-rate along fiber optic cables at high spatial and temporal resolution. This technique is becoming a popular tool in seismology, hydrology, and other subsurface monitoring applications. However, due to the large coverage (10’s of km) and high density of measurements (1m spacing at 100’s of Hz), a DAS installation could produce terabytes of data records per day. Because many DAS instruments are deployed in remote locations, this large data size poses significant challenges to its transfer and storage. In this paper, we explore lossless compression methods to reduce the storage requirement in both real-time and post-hoc scenarios. Here we propose a two-stage compression method to improve the compression ratio and compression speed. This two-stage compression method could reduce the storage requirement by 40%, which is 20% more than other lossless methods, such as ZSTD. We demonstrate that the compression method could complete its operation well before the DAS instrument needs to output the next file, making it suitable for real-time DAS acquisition. We also implement a parallel compression method for a post-hoc scenario and demonstrate that our method could effectively utilize a parallel computer. With 256 CPU cores, our parallel compression method achieves the speed of 26GB/second.

58 GEOSCIENCES↗
NASA NTRS1985

Imaging spectrometry for earth remote sensing

Initial results of the novel remote earth sensing technique of imaging spectrometry, which is technically feasible from both spacecraft and aircraft platforms, indicate that the direct identification of surface materials on a picture-element basis is possible through proper sampling of absorption features in the reflectance spectrum. Sensors of this type are able to acquire images simultaneously in 100-200 contiguous spectral bands. Computerized data reduction and storage techniques are available for the large data sets thus generated, and novel analytic techniques are under development to maximize information content extraction.

Goetz, A. F. H.↗
NASA NTRS2019

Using Big Data Technologies with Earth Science Data in HDF5: HDF5 Scalable Solutions

HDF5 (Hierarchical Data Format 5) is open-source, high-performance software that consists of an abstract data model, library, and fileformat used for storing and managing extremely large and/or complex data collections. NASA Earth Observing System (EOS) Data and Information Systems use HDF5 as an archival format to store remote sensing data from EOS satellites. HDF5 is also used to store other types of Geoscience and Strophysical data, e.g., seismic data and data from Low-Frequency Array (LOFAR) radio telescopes. Data stored in HDF5 has reached tens of petabytes and is growing at an accelerated rate.With the growing amout of HDF5 Earth Science data to analyze and process, scientists need to adopt big data technologies including new storage paradigms such as cloud and object storage. To run models and perform data analysis they also need to utilizied efficient and diverse ways to access data, from high-performance computing's (HPC) Message Passing Interface (MPI) I/O and deep memory hierarchies (DMH) to non-HPC frameworks such as Apache Hadoop, Spark, and Drill. The HDF Group continually works to enable usage of big data technologies in HDF software.

Knox, Larry↗
NASA NTRS2002

The CCDS Data Compression Recommendations: Development and Status

The Consultative Committee for Space Data Systems (CCSDS) has been engaging in recommending data compression standards for space applications. The first effort focused on a lossless scheme that was adopted in 1997. Since then, space missions benefiting from this recommendation range from deep space probes to near Earth observatories. The cost savings result not only from reduced onboard storage and reduced bandwidth, but also in ground archive of mission data. In many instances, this recommendation also enables more science data to be collected for added scientific value. Since 1998, the compression sub-panel of CCSDS has been investigating lossy image compression schemes and is currently working towards a common solution for a single recommendation. The recommendation will fulfill the requirements for remote sensing conducted on space platforms.

Yeh, Pen-Shu↗
NASA NTRS2015

Intelligent Observation Strategies for Geosynchronous Remote Sensing for Natural Hazards

Geosynchronous satellites offer a unique perspective for monitoring environmental factors important to understanding natural hazards and supporting the disasters management life cycle, namely forecast, detection, response, recovery and mitigation. In the NASA decadal survey for Earth science, the GEO-CAPE mission was proposed to address coastal and air pollution events in geosynchronous orbit, complementing similar initiatives in Asia by the South Koreans and by ESA in Europe, thereby covering the northern hemisphere. In addition to analyzing the challenges of identifying instrument capabilities to meet the science requirements, and the implications of hosting the instrument payloads on commercial geosynchronous satellites, the GEO-CAPE mission design team conducted a short study to explore strategies to optimize the science return for the coastal imaging instrument. The study focused on intelligent scheduling strategies that took into account cloud avoidance techniques as well as onboard processing methods to reduce the data storage and transmission loads. This paper expands the findings of that study to address the use of intelligent scheduling techniques and near-real time data product acquisition of both the coastal water and air pollution events. The topics include the use of onboard processing to refine and execute schedules, to detect cloud contamination in observations, and to reduce data handling operations. Analysis of state of the art flight computing capabilities will be presented, along with an assessment of cloud detection algorithms and their performance characteristics. Tools developed to illustrate operational concepts will be described, including their applicability to environmental monitoring domains with an eye to the future. In the geostationary configuration, the payload becomes a networked thing with enough connectivity to exchange data seamlessly with users. This allows the full field of view to be sensed at very high rate under the control of ground infrastructure, resulting in improved efficiencies, accuracy and science benefits. Hence a remote sensing payload and its data may become one of millions of connected objects in the emerging Internet of Things (IoT), and be as easily accessible by a users smart phone as any other smart appliance.

0↗
NASA NTRS2011

Miniaturized Airborne Imaging Central Server System

In recent years, some remote-sensing applications require advanced airborne multi-sensor systems to provide high performance reflective and emissive spectral imaging measurement rapidly over large areas. The key or unique problem of characteristics is associated with a black box back-end system that operates a suite of cutting-edge imaging sensors to collect simultaneously the high throughput reflective and emissive spectral imaging data with precision georeference. This back-end system needs to be portable, easy-to-use, and reliable with advanced onboard processing. The innovation of the black box backend is a miniaturized airborne imaging central server system (MAICSS). MAICSS integrates a complex embedded system of systems with dedicated power and signal electronic circuits inside to serve a suite of configurable cutting-edge electro- optical (EO), long-wave infrared (LWIR), and medium-wave infrared (MWIR) cameras, a hyperspectral imaging scanner, and a GPS and inertial measurement unit (IMU) for atmospheric and surface remote sensing. Its compatible sensor packages include NASA s 1,024 1,024 pixel LWIR quantum well infrared photodetector (QWIP) imager; a 60.5 megapixel BuckEye EO camera; and a fast (e.g. 200+ scanlines/s) and wide swath-width (e.g., 1,920+ pixels) CCD/InGaAs imager-based visible/near infrared reflectance (VNIR) and shortwave infrared (SWIR) imaging spectrometer. MAICSS records continuous precision georeferenced and time-tagged multisensor throughputs to mass storage devices at a high aggregate rate, typically 60 MB/s for its LWIR/EO payload. MAICSS is a complete stand-alone imaging server instrument with an easy-to-use software package for either autonomous data collection or interactive airborne operation. Advanced multisensor data acquisition and onboard processing software features have been implemented for MAICSS. With the onboard processing for real time image development, correction, histogram-equalization, compression, georeference, and data organization, fast aerial imaging applications, including the real time LWIR image mosaic for Google Earth, have been realized for NASA fs LWIR QWIP instrument. MAICSS is a significant improvement and miniaturization of current multisensor technologies. Structurally, it has a complete modular and solid-state design. Without rotating hard drives and other moving parts, it is operational at high altitudes and survivable in high-vibration environments. It is assembled from a suite of miniaturized, precision-machined, standardized, and stackable interchangeable embedded instrument modules. These stackable modules can be bolted together with the interconnection wires inside for the maximal simplicity and portability. Multiple modules are electronically interconnected as stacked. Alternatively, these dedicated modules can be flexibly distributed to fit the space constraints of a flying vehicle. As a flexibly configurable system, MAICSS can be tailored to interface a variety of multisensor packages. For example, with a 1,024x1,024 pixel LWIR and a 8,984x6,732 pixel EO payload, the complete MAICSS volume is approximately 7x9x11 in. (=18x23x28 cm), with a weight of 25 lb (=11.4 kg).

Sun, Xiuhong↗
NASA NTRSDate not supplied

Evaluating Changes in Freshwater Health in the Tonlé Sap Basin using Remotely Sensed Data

The Tonlé Sap lake and river basin provide freshwater resources to about 4.5 million people in central Cambodia. Accelerating dam construction, intensifying agriculture, deforestation, and changing climate patterns threaten the health of the lake and the food security of nearby communities. NASA DEVELOP collaborated with Conservation International to calculate multiple components of the ecosystem vitality indicator of the Freshwater Health Index for Tonlé Sap lake from 2000 – 2020 using remote sensing. Landcover classifications, lake level altimetry, groundwater storage and nutrient flows were derived from multiple satellites including Landsat 5, Landsat 7, Landsat 8, Sentinel-2, the Global Precipitation Measurement (GPM)mission, and the Gravity Recovery and Climate Experiment (GRACE). Additionally, the Soil and Water Assessment Tool (SWAT) was used to analyze nutrient flow and water quality in the basin. A breakdown in the volume and regularity of annual lake levels from 2000-2020 was observed, reflecting increased pressure on water supply and agricultural productivity. A decrease in landcover naturalness was also observed and at least 8% of forested areas in the basin were lost while rice harvest intensity increased throughout the study period. Additionally, an initial water quality analysis was demonstrated using SWAT. Continued collaboration with Conservation International and local representatives will guide the remaining water quality analyses and allow them to make informed decisions regarding freshwater management and resource allocation in the Tonlé Sap lake and river basin.

Adriana Le Compte↗
NASA NTRS1978

Use of change detection in assessing development plans - A Philippine example

An aircraft/Landsat change-detection study conducted 1948-1972 on Marinduque Province, Republic of the Philippines, is discussed, and a procedure using both remote sensing and information systems for collection, spatial analysis, and display of periodic data is described. Each of the 4,008 25-hectare cells representing Marinduque were observed, and changes in and between variables were measured and tested using nonparametric statistics to determine the effect of specific land cover changes. Procedures using Landsat data to obtain a more continuous updating of the data base are considered. The system permits storage and comparison of historical and current data.

Coiner, J. C.↗
NASA NTRS1993

A browse facility for Earth science remote sensing data: Center director's discretionary fund final report

An image data visual browse facility is developed for a UNIX platform using the X Windows 11 system. It allows one to visually examine reduced resolution image data to determine which data are applicable for further research. Links with a relational data base manager then allow one to extract not only the full resolution image data, but any other ancillary data related to the case study. Various techniques are examined for compression of the image data in order to reduce data storage requirements and time necessary to transmit the data on the internet. Data used were from the WetNet project.

Meyer, P. J.↗
NASA NTRS1988

Moorable spectroradiometers in the BIOWATT experiment

A new oceanographic instrument, the MER-2020, has been developed for the long term measurement of bio-optical properties. This instrument is a self-contained reflectance spectroradiometer with 40 megabytes of internal data storage and battery power permitting deployments for as long as 6 months. It can serve as the control and recording center for a variety of other sensors including fluorometers, transmissometers, temperature and conductivity probes. Instruments of this type were deployed at depths to 70 meters on a 5000 meter mooring during the BIOWATT experiment between February and November 1987. Measurements of the spectral diffuse attenuation coefficient and of the remote sensed reflectance from the ten month period in 1987 in the Northern Sargasso Sea are presented. In addition, the temperature, pressure, package orientation and sunlight stimulated natural fluorescence of phytoplankton are discussed. This instrument will be a key part of in situ optical calibration for future remote color sensors, and presently permits direct measurement of long term optical variability, important in visibility and communication problems.

Booth, C. R.↗
NASA NTRS2023

Using Satellite-Based Terrestrial Water Storage Data: A Review

Land water storage plays a key role for the Earth’s climate, natural ecosystems, and human activities. Since the launch of the first Gravity Recovery and Climate Experiment (GRACE) mission in 2002, spaceborne observations of changes in terrestrial water storage (TWS) have provided a unique, global perspective on natural and human-induced changes in freshwater resources. Even though they have become much used within the broader Earth system science community, space-based TWS datasets still incorporate important and case-specific limitations which may not always be clear to users not familiar with the underlying processing algorithms. Here, we provide an accessible and illustrated overview of the measurement concept, of the main available data products, and of some frequently encountered technical terms and concepts. We summarize concrete recommendations on how to use TWS data in combination with other hydrological or climatological datasets, and guidance on how to avoid possible pitfalls. Finally, we provide an overview of some of the main applications of GRACE TWS data in the fields of hydrology and climate science. This review is written with the intention of supporting future research and facilitating the use of satellite-based terrestrial water storage datasets in interdisciplinary contexts.

Terrestrial water storage↗
NASA NTRS2010

Future Plans in US Flight Missions: Using Laser Remote Sensing for Climate Science Observations

Laser Remote Sensing provides critical climate science observations necessary to better measure, understand, model and predict the Earth's water, carbon and energy cycles. Laser Remote Sensing applications for studying the Earth and other planets include three dimensional mapping of surface topography, canopy height and density, atmospheric measurement of aerosols and trace gases, plume and cloud profiles, and winds measurements. Beyond the science, data from these missions will produce new data products and applications for a multitude of end users including policy makers and urban planners on local, national and global levels. NASA Missions in formulation including Ice, Cloud, and land Elevation Satellite (ICESat 2) and the Deformation, Ecosystem Structure, and Dynamics of Ice (DESDynI), and future missions such as the Active Sensing of CO2 Emissions over Nights, Days and Seasons (ASCENDS), will incorporate the next generation of LIght Detection And Ranging (lidar) instruments to measure changes in the surface elevation of the ice, quantify ecosystem carbon storage due to biomass and its change, and provide critical data on CO 2 in the atmosphere. Goddard's plans for these instruments and potential uses for the resulting data are described below. For the ICESat 2 mission, GSFC is developing a micro-pulse multi-beam lidar. This instrument will provide improved ice elevation estimates over high slope and very rough areas and result in improved lead detection for sea ice estimates. Data about the sea ice and predictions related to sea levels will continue to help inform urban planners as the changes in the polar ice accelerate. DESDynI is planned to be launched in 2017 and includes both lidar and radar instruments. GSFC is responsible for the lidar portion of the DESDynI mission and is developing a scanning laser altimeter that will measure the Earth's topography, the structure of tree canopies, biomass, and surface roughness. The DESDynI lidar will also measure and predict the response of ice masses to climate change and impact on sea level. Data from the lidar will ultimately be fused with radar data products with heretofore unseen results and applications. The 3-D structure of forests is critical to understanding the impact of land use and associated landscape changes on the habitat of life forms and consequently on their biodiversity. Lidar instruments are also under development to measure trace gases in the atmospheric such as CO2 and methane. GSFC is developing an active measurement approach to determine the CO2 column density and surface pressure for the proposed ASCENDS mission. The objective of this approach is to produce data on the amounts of anthropogenic and organic CO2 in the atmosphere with sufficient accuracy to meet the needs of target users including state, federal and international users as well as policy-related legislative, regulatory, and voluntary carbon-related management groups local to international interests. In summary, NASA will continue to rely on laser remote sensing for critical climate science observations and is committed to the development of the next generation of lidar instruments for a range of applications.

Callahan, Lisa W.↗
NASA NTRS2005

NASA Remote Sensing Data in Earth Sciences: Processing, Archiving, Distribution, Applications at the GES DISC

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of the major Distributed Active Archive Centers (DAACs) archiving and distributing remote sensing data from the NASA's Earth Observing System. In addition to providing just data, the GES DISC/DAAC has developed various value-adding processing services. A particularly useful service is data processing a t the DISC (i.e., close to the input data) with the users' algorithms. This can take a number of different forms: as a configuration-managed algorithm within the main processing stream; as a stand-alone program next to the on-line data storage; as build-it-yourself code within the Near-Archive Data Mining (NADM) system; or as an on-the-fly analysis with simple algorithms embedded into the web-based tools (to avoid downloading unnecessary all the data). The existing data management infrastructure at the GES DISC supports a wide spectrum of options: from data subsetting data spatially and/or by parameter to sophisticated on-line analysis tools, producing economies of scale and rapid time-to-deploy. Shifting processing and data management burden from users to the GES DISC, allows scientists to concentrate on science, while the GES DISC handles the data management and data processing at a lower cost. Several examples of successful partnerships with scientists in the area of data processing and mining are presented.

Leptoukh, Gregory G.↗
NASA NTRS1975

A study of application of remote sensing to river forecasting. Volume 2: Detailed technical report, NASA-IBM streamflow forecast model user's guide

The Model is described along with data preparation, determining model parameters, initializing and optimizing parameters (calibration) selecting control options and interpreting results. Some background information is included, and appendices contain a dictionary of variables, a source program listing, and flow charts. The model was operated on an IBM System/360 Model 44, using a model 2250 keyboard/graphics terminal for interactive operation. The model can be set up and operated in a batch processing mode on any System/360 or 370 that has the memory capacity. The model requires 210K bytes of core storage, and the optimization program, OPSET (which was used previous to but not in this study), requires 240K bytes. The data band for one small watershed requires approximately 32 tracks of disk storage.

Source record↗
NASA NTRSDate not supplied

Open Source Application of Fusing Aerosol Products from GEO and LEO Satellites

Retrieving aerosol optical depths (AODs) from sun-synchronous polar orbiting (aka low earth orbit, LEO) satellites, such as MODISs, and VIIRSs, OMI, TROPOMI, etc, has become well-established as a tool for extracting information on particulate matter (PM) and related processes in the atmosphere. However, with recently launched geostationary satellites (GEO), such as GOES-16/17/18, and Himawari-8/9, and Meteosat Third Generation (MTG) they provide a much higher temporal resolution (order of 10 minutes), typically an image once or more per hour during daylight compared to LEO once per day. By combining these observations, we may be able to characterize the diurnal cycle of global AOD at the local, regional and global scale. While the science community is still exploring the new data from GEO observations, we have been thinking about how to properly combine/merge/fuse those data considering differences in their spatial and temporal resolutions. However, this poses a “Big Data” challenge. The big data challenge is not just about data storage, but also about data discoverability, and accessibility, and even more, about data migration/mirroring in the cloud-computing environment. This paper is merely showing some of the efforts and approaches we have attempted in fusing six satellites’ Level 2 aerosol data (three are from GEO (GOES-16/17 and Himawari-8), and the other three are from LEO (TERRA/MODIS, AQUA/MODIS, SNPP-VIIRS) from Dark Target (DT) aerosol retrieval algorithm. Having the on-demand capability of fusing remote sensing products onto the desired temporal and spatial domain enables researchers and application practitioners to better manipulate and work with satellite and sensor data. It is our hopeWe hope that by making such an open-source package, and the accompanying functionality, the scientific community will be granted easier access to aerosol data processing resources. The MEaSUREs Program (Making Earth System Data Records for Use in Research Environments) expands our understanding of the Earth's current system through atmospheric and surface measurements. In an effort to aid the scientific research component and improve open source methods, this project developed Python code for fusing six satellite Level 2 aerosol data (three are from geostationary satellites (GEO), and the other three are from low earth orbital satellites (LEO)) from Dark Target Aerosol Retrieval Algorithm.

Jennifer Wei↗
NASA NTRS1974

Application of remote sensing techniques in land-use planning: Flood-plain delineation

The delineation of areas subject to inundation by means of remotely-sensed data acquisition represents a considerable saving in personnel time. Repeated input from aerial sensor sources provides the planner with a potent tool for the formation of a data base and for the monitoring of land use patterns over a period of time. The primary output of this project was a set of base map overlays at a scale of 1:62,500 delineating areas which require special regulations when proposed for land use involving human habitation or certain classes of storage. A secondary product of the study was county-wide maps of watershed configurations and of soil hydrologic groups. Further research is anticipated to extend the mapping of watershed areas outside the political boundaries of Cochise County, which will provide data for subsequent rainfall-runoff relationship studies in the area. All of the data provided will be incorporated into the Cochise County composite computer mapping project now operational. Results of this project have improved the pool of information available to the planning staff of Cochise County.

Altenstadter, J.↗
NASA NTRS1994

A comparative study of SAR data compression schemes

The amount of data collected from spaceborne remote sensing has substantially increased in the last years. During same time period, the ability to store or transmit data has not increased as quickly. At this time, there is a growing interest in developing compression schemes that could provide both higher compression ratios and lower encoding/decoding errors. In the case of the spaceborne Synthetic Aperture Radar (SAR) earth observation system developed by the French Space Agency (CNES), the volume of data to be processed will exceed both the on-board storage capacities and the telecommunication link. The objective of this paper is twofold: to present various compression schemes adapted to SAR data; and to define a set of evaluation criteria and compare the algorithms on SAR data. In this paper, we review two classical methods of SAR data compression and propose novel approaches based on Fourier Transforms and spectrum coding.

Lambert-Nebout, C.↗
NASA NTRS2016

Extreme Water Deficit in Brazil Detected from Space

Extreme droughts have caused significant socioeconomic and environmental damage worldwide. In Brazil, ineffective energy development and water management policies have magnified the impacts of recent severe droughts, which include massive agricultural losses, water supply restrictions, and energy rationing. Spaceborne remote sensing data advance our understanding of the spatiotemporal variability of large-scale droughts and enhance the detection and monitoring of extreme water-related events. In this study, data derived from the Gravity Recovery and Climate Experiment (GRACE) mission are used to detect and quantify an extended major drought over eastern Brazil and provide estimates of impacted areas and region-specific water deficits. Two structural breakpoint detection methods were applied to time series of GRACE-based terrestrial water storage anomalies (TWSA), determining when two abrupt changes occurred. One, in particular, defines the beginning of the current drought. Using TWSA, a water loss rate of 26.1 cmyr21 over southeastern Brazil was detected from 2012 to 2015. Based on analysis of Global Land Data Assimilation System(GLDAS) outputs, the extreme drought is mostly related to lower-than-usual precipitation rates, resulting in high soil moisture depletion and lower-than-usual rates of evapotranspiration. A reduction of 2023 of precipitation over an extended period of 3 years is enough to raise serious water scarcity conditions in the country. Correlations between monthly time series of both grid-based TWSA and ground-based water storage measurements at 16 reservoirs located within southeastern Brazil varied from 0.42 to 0.82. Differences are mainly explained by reservoir sizes and proximity to the drought nucleus.

terrestrial↗