Search NASA⌕ Search

SEARCH · Search NASA

Results for “data masking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1995

An improved land mask for the SSM/I grid

This paper discusses the development of a new land/ocean/coastline mask for use with Defense Meteorological Satellite Program (DMSP) Special Sensor Microwave/Imager (SSM/I) data, and other types of data which are mapped to the polar stereographic SSM/I grid. Pre-existing land masks were found to disagree, to lack certain land features, and to disagree with land boundaries that are visible in high resolution sensor imagery, such as imagery from the Synthetic Aperture Radar (SAR) on the Earth Resources Satellite (ERS-1). The Digital Chart of the World (DCW) database was initially selected as a source of shoreline data for this effort. Techniques for developing a land mask from these shoreline data are discussed. The resulting land mask, although not perfect, is seen to exhibit significant improvement over previous land mask products.

Martino, Michael G.↗
NASA NTRS1994

Can salt on an optical rain gauge lens affect performance?

The optical rain gauge (ORG) by ScTI is designed to be tolerant to reduction of infrared beam intensity due to a dirty lens. Recently there is interest in long term use of optical gauges onboard buoys at sea. Because of logistics, these systems are serviced infrequently, i.e., every several months. Due to the proximity of the gauges to the sea surface, salt can be expected to be deposited on the lens. To obtain an indication of the potential for dirty lens to affect the ORG calibration, two simple experiments were conducted. In the control experiment, a sample ORG was compared to three other ORG's during natural rain events. Next a translucent mask was placed on the transmitter lens of the sample ORG and again data were collected under natural rain conditions. The mask reduced the gain of the perturbed ORG by about 30%. The perturbed ORG operated rather well in that the mask only causes a change in the gain and does not cause data drop out at low rain rates. However, the reduced gain would seriously impact an assessment of rain statistics.

Bliven, Larry F.↗
NASA NTRSDate not supplied

TPSAS-NF1676L-32467-DND

With the launch of a new generation of Geostationary satellites (GEO) such as Himawari and GOES-16 & 17, cloud detection using satellite imager data has been greatly enhanced with increased spectral bands, and higher temporal and spatial resolutions. A concern for all geostationary sensors, however, are changes in instrument sensitivity and algorithm performance at different viewing for daytime and nighttime. CALIPSO lidar observations provide a valuable reference for assessing these impacts as the satellite flies in a sun-synchronous orbit and crosses a wide range of GEO viewing angles each day. This paper will present the cloud mask results using the imager data from Himawari (AHI) and GOES-16&17 (ABI). The detection algorithms have been adapted from the Cloud and Earth’s Radiant Energy System (CERES) MODIS Edition 4 cloud mask, and adjusted and tuned to geo-satellites. They are used operationally for the CERES Time and Space Averaging (TISA) gridded cloud products and for near-real-time retrievals for weather and nowcasting applications.

Qing Z. Trepte↗
NASA NTRS1972

Some new techniques for processing remotely obtained images by self-generated spectral masks.

An extension of a new technique that makes possible parallel, simultaneous processing of remotely obtained images is presented. The technique holds out promise for automatic onboard classification of data. The central feature involves the generation of binary masks, directly from the image, based on object reflectance data, that group objects into equivalence classes. These masks, called equivalence class masks, can be used in various logical combinations to isolate classes of objects with a priori known reflectance or radiance signatures. Experimental verification of the technique is furnished for simple scenes. A computational scheme, based on a sequence of integrated irradiance measurements on the image, that makes it possible to identify objects within an equivalence class is suggested.

Stark, H.↗
NASA NTRSDate not supplied

Development of a Consistent Cross-Platform GEO-Satellite Cloud Mask to Support CERES

Geostationary satellites provide continuous cloud and meteorological observations over a fixed portion of the Earth’s surface, allowing them to monitor the movement of storm systems and their diurnal variation. For climate studies, geostationary observations provide valuable insight of cloud formation and evolution and how they influence the Earth’s radiation budget. The Geo-Satellite Edition 4 cloud mask (GEO Ed4) is used operationally in NASA’s Cloud and Earth’s Radiant Energy System (CERES) project to help account for diurnal variations in cloudiness on Earth’s radiation budget. The Ed4 cloud mask was applied to imager data on MSG (MeteoSat Second Generation), Himawari, GOES-West, and GOES-East satellites using as much spectral information as possible. That approach led to some discontinuities when a more modern satellite replaced an older satellite with less spectral information. A different strategy is being investigated for the CERES GEO Ed5 cloud mask that only uses spectral channels common on every satellite. Thus, a 3-channel (0.6, 3.9, 11 µm) algorithm for daytime cloud detection, and a 2-channels (3.9 and 11 µm) algorithm for nighttime cloud detection have been implemented and tested for Ed5. The goal of Ed5 cloud mask is to achieve global cloud amount consistency across five geo-satellites, and a smooth transition from an old satellite to a new satellite over each geo-location. This paper compares cloud mask results between Ed4 and a preliminary Ed5 version over the GOES-East region, using GOES-8, GOES-13, and GOES-16 satellites, each with a different spectral channel complement. Instantaneous and monthly regional mean inter-comparisons are performed and evaluated with CALIPSO cloud products to assess the accuracy and consistency of the Ed5 approach relative to that taken in Ed4. Results will be presented and discussed at the conference.

clouds↗
NASA NTRS2019

Improving Satellite Global Chlorophyll a [alpha] Data Products Through Algorithm Refinement and Data Recovery

A recently developed algorithm to estimate surface ocean chlorophyll a concentrations (Chl in milligrams per cubic meter), namely, the ocean color index (OCI) algorithm, has been adopted by the U.S. National Aeronautics and Space Administration to apply to all satellite ocean color sensors to produce global Chl maps. The algorithm is a hybrid between a band‐difference color index algorithm for low‐Chl waters and the traditional band‐ratio algorithms (OCx) for higher‐Chl waters. In this study, the OCI algorithm is revisited for its algorithm coefficients and for its algorithm transition between color index and OCx using a merged data set of high‐performance liquid chromatography and fluorometric Chl. Results suggest that the new OCI algorithm (OCI2) leads to lower Chl estimates than the original OCI (OCI1) for Chl less than 0.05 milligrams per cubic meter, but smoother algorithm transition for Chl between 0.25 and 0.40 milligrams per cubic meter. Evaluation using in situ data suggests that similar to OCI1, OCI2 has significantly improved image quality and cross‐sensor consistency between SeaWiFS (Sea-viewing Wide Field-of-view Sensor), MODISA (Moderate Resolution Imaging Spectroradiometer on Aqua), and VIIRS (Visible Infrared Imaging Radiometer Suite) over the OCx algorithms for oligotrophic oceans. Mean cross‐sensor difference in monthly Chl data products over global oligotrophic oceans reduced from approximately 10 percent for OCx to 1-2 percent for OCI2. More importantly, data statistics suggest that the current straylight masking scheme used to generate global Chl maps can be relaxed from 7 by 5 to 3 by 3 pixels without losing data quality in either Chl or spectral remote sensing reflectance (R (sub rs) by lambda (sensor wavelength), per steradian (sr (sup −1)) for not just oligotrophic oceans but also more productive waters. Such a relaxed masking scheme yields an average relative increase of 39 percent in data quantity for global oceans, thus making it possible to reduce data product uncertainties and fill data gaps.

Hu, Chuanmin↗
NASA NTRS2004

Cloud Detection from Satellite Imagery: A Comparison of Expert-Generated and Automatically-Generated Decision Trees

Automated cloud detection and tracking is an important step in assessing global climate change via remote sensing. Cloud masks, which indicate whether individual pixels depict clouds, are included in many of the data products that are based on data acquired on- board earth satellites. Many cloud-mask algorithms have the form of decision trees, which employ sequential tests that scientists designed based on empirical astrophysics studies and astrophysics simulations. Limitations of existing cloud masks restrict our ability to accurately track changes in cloud patterns over time. In this study we explored the potential benefits of automatically-learned decision trees for detecting clouds from images acquired using the Advanced Very High Resolution Radiometer (AVHRR) instrument on board the NOAA-14 weather satellite of the National Oceanic and Atmospheric Administration. We constructed three decision trees for a sample of 8km-daily AVHRR data from 2000 using a decision-tree learning procedure provided within MATLAB(R), and compared the accuracy of the decision trees to the accuracy of the cloud mask. We used ground observations collected by the National Aeronautics and Space Administration Clouds and the Earth s Radiant Energy Systems S COOL project as the gold standard. For the sample data, the accuracy of automatically learned decision trees was greater than the accuracy of the cloud masks included in the AVHRR data product.

Shiffman, Smadar↗
NASA NTRSDate not supplied

ARM Data as A Resource for Validation of NASA PACE Cloud Retrievals

NASA’s Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission will launch in January 2024 and continue and improve upon satellite data records in its eponymous domains. PACE will carry a broad-swath hyperspectral imager, OCI, which will provide MODIS/VIIRS-type cloud data products (i.e. a cloud mask, top height, visible optical thickness, droplet effective radius, phase, and derived water path). It will also carry two multi-angle polarimeters (HARP2 and SPEXone) which will not only provide the above but also enable retrievals of additional cloud properties (e.g. droplet effective variance, ice crystal asymmetry parameter). Validating satellite-based cloud retrievals is challenging. We plan to use several ARM data streams to evaluate PACE cloud data products and are prototyping our analyses using retrievals from MODIS on the Aqua satellite and OLCI on the Sentinel-3A satellite. This poster shows how we plan to use ARM data to evaluate liquid water path (via MWRRET) and cloud top height retrievals (via KARZASRCL), with example results from these proxy sensors and ARM data from the SGP, ENA, and NSA sites. We seek comments from and collaborations with the ARM community to get the most out of our respective data streams.

ARM↗
NASA NTRS1994

The classification of the Arctic Sea ice types and the determination of surface temperature using advanced very high resolution radiometer data

The accurate quantification of new ice and open water areas and surface temperatures within the sea ice packs is a key to the realistic parameterization of heat, moisture, and turbulence fluxes between ocean and atmosphere in the polar regions. Multispectral NOAA advanced very high resolution radiometer/2 (AVHRR/2) satellite images are analyzed to evaluate how effectively the data can be used to characterize sea ice in the Bering and Greenland seas, both in terms of surface type and physical temperature. The basis of the classification algorithm, which is developed using a late wintertime Bering Sea ice cover data, is that frequency distributions of 10.8- micrometers radiances provide four distinct peaks, represeting open water, new ice, young ice, and thick ice with a snow cover. The results are found to be spatially and temporally consistent. Possible sources of ambiguity, especially associated with wider temporal and spatial application of the technique, are discussed. An ice surface temperature algorithm is developed for the same study area by regressing thermal infrared data from 10.8- and 12.0- micrometers channels against station air temperatures, which are assumed to approximate the skin temperatures of adjacent snow and ice. The standard deviations of the results when compared with in situ data are about 0.5 K over leads and polynyas to about 0.5-1.5 K over thick ice. This study is based upon a set of in situ data limited in scope and coverage. Cloud masks are applied using a thresholding technique that utilizes 3.74- and 10.8- micrometers channel data. The temperature maps produced show coherence with surface features like new ice and leads, and consistency with corresponding surface type maps. Further studies are needed to better understand the effects of both the spatial and temporal variability in emissivity, aerosol and precipitable atmospheric ice particle distribution, and atmospheric temperature inversions.

Massom, Robert↗
NASA NTRS1995

Cloud Masking and Surface Temperature Distribution in the Polar Regions Using AVHRR and other Satellite Data

Surface temperature is one of the key variables associated with weather and climate. Accurate measurements of surface air temperatures are routinely made in meteorological stations around the world. Also, satellite data have been used to produce synoptic global temperature distributions. However, not much attention has been paid on temperature distributions in the polar regions. In the polar regions, the number of stations is very sparse. Because of adverse weather conditions and general inaccessibility, surface field measurements are also limited. Furthermore, accurate retrievals from satellite data in the region have been difficult to make because of persistent cloudiness and ambiguities in the discrimination of clouds from snow or ice. Surface temperature observations are required in the polar regions for air-sea-ice interaction studies, especially in the calculation of heat, salinity, and humidity fluxes. They are also useful in identifying areas of melt or meltponding within the sea ice pack and the ice sheets and in the calculation of emissivities of these surfaces. Moreover, the polar regions are unique in that they are the sites of temperature extremes, the location of which is difficult to identify without a global monitoring system. Furthermore, the regions may provide an early signal to a potential climate change because such signal is expected to be amplified in the region due to feedback effects. In cloud free areas, the thermal channels from infrared systems provide surface temperatures at relatively good accuracies. Previous capabilities include the use of the Temperature Humidity Infrared Radiometer (THIR) onboard the Nimbus-7 satellite which was launched in 1978. Current capabilities include the use of the Advance Very High Resolution Radiometer (AVHRR) aboard NOAA satellites. Together, these two systems cover a span of 16 years of thermal infrared data. Techniques for retrieving surface temperatures with these sensors in the polar regions have been developed. Errors have been estimated to range from 1K to 5K mainly due to cloud masking problems. With many additional channels available, it is expected that the EOS-Moderate Resolution Imaging Spectroradiometer (MODIS) will provide an improved characterization of clouds and a good discrimination of clouds from snow or ice surfaces.

Comiso, Joey C.↗
NASA NTRS2009

Analysis Resistant Cipher Method and Apparatus

A system for encoding and decoding data words including an anti-analysis encoder unit for receiving an original plaintext and producing a recoded data, a data compression unit for receiving the recoded data and producing a compressed recoded data, and an encryption unit for receiving the compressed recoded data and producing an encrypted data. The recoded data has an increased non-correlatable data redundancy compared with the original plaintext in order to mask the statistical distribution of characters in the plaintext data. The system of the present invention further includes a decryption unit for receiving the encrypted data and producing a decrypted data, a data decompression unit for receiving the decrypted data and producing an uncompressed recoded data, and an anti-analysis decoder unit for receiving the uncompressed recoded data and producing a recovered plaintext that corresponds with the original plaintext.

Oakley, Ernest C.↗
NASA NTRS1998

Emission Lines and the High Energy Continuum

Quasars show many striking relationships between line and continuum radiation whose origins remain a mystery. FeII, [OIII], Hbeta, and HeII emission line properties correlate with high energy continuum properties such as the relative strength of X-ray emission, and X-ray continuum slope. At the same time, the shape of the high energy continuum may vary with luminosity. An important tool for studying global properties of Quasi Stellar Objects (QSOs) is the co-addition of data for samples of QSOS. We use this to show that X-ray bright (XB) QSOs show stronger emission lines in general, but particularly from the narrow line region. The difference in the [OIII]/Hbeta ratio is particularly striking, and even more so when blended FeII emission is properly subtracted. Weaker narrow forbidden lines ([OII] and NeV) are enhanced by factors of 2 to 3 in both UV and optical XB composite spectra. The physical origin of these diverse and interrelated correlations has yet to be determined. Unfortunately, many physically informative trends intrinsic to QSOs may be masked by dispersion in the data due to either low signal-to-noise or variability. An important tool for studying global properties of QSOs is the co-addition of data for samples of QSOS. We use this to show that X-ray bright (XB) QSOs show stronger emission lines in general, but particularly from the narrow line region. The difference in the [OIII]/Hbeta ratio is particularly striking, and even more so when blended Fell emission is properly subtracted. Weaker narrow forbidden lines ([OII] and NeV) are enhanced by factors of 2 to 3 in both UV and optical XB composite spectra. We describe a large-scale effort now underway to probe these effects in large samples, using both data and analysis as homogeneous as possible. Using an HST FOS Atlas of QSO spectra, with primary comparison to ROSAT PSPC spectral constraints, we will model the Big Blue Bump, its relationship to luminosity and QSO type, and we will analyze and contrast line emission and UV/X-ray continuum properties. Absorption of the continuum near the broad emission line region may play a profound role, which we will be able to constrain by direct analysis of observed UV/X-ray spectral absorption.

Green, Paul↗
NASA NTRS1996

Seasonal Characteristics of the Perennial Ice Cover of the Beaufort Sea

By definition, ice which survives the summer is classified as multiyear ice. Thus the area covered by multiyear ice during the winter should be nearly equivalent to the ice area during the previous summer's minima. This condition provides a reasonable criterion for the evaluation of ice concentration and ice type retrieval algorithms using remote-sensing data sets. From special sensor microwave imager (SSM/I) data the NASA Team algorithm estimates the multiyear, first-year, and total ice concentrations during the winter using combinations of the polarization and spectral gradient ratios. The Team algorithm provides only estimates of ice concentration in the summer. From ERS 1 synthetic aperture radar (SAR) data the remarkably stable contrast between multiyear ice and first-year ice in winter provides consistent estimates of multiyear ice concentrations. In the summer, multiyear ice concentration cannot be estimated from SAR or SSM/I data because free water on the surface effectively masks the backscatter and emissivity signature of this ice type. From SAR data a technique which takes advantage of the high backscatter of wind-roughened open water as a discrimination feature is used to estimate the total ice concentration in the summer. With a year-long (January 1992 to January 1993) data set from the Beaufort Sea we found that the multiyear ice concentration estimates from the SAR data are stable and are nearly equivalent to the ice concentration estimated at the end of the previous summer. We contrast this with the variability of the multiyear ice concentration and ice friction estimates obtained using SSM/I data. The Team algorithm produces ice concentration and multiyear ice estimates which are consistently lower than those from the SAR data. We discuss reasons for these discrepancies and the implications of the higher than previously noted multiyear ice concentrations.

Kwok, R.↗
DOE OSTI2020

Variability and Diversity Load Model Tool [SWR-20-03]

The motivation for the development of this tool and the underlying algorithms and methods was to enable the development of high-temporal resolution, realistic time-series data for quasi-static time-series (QSTS) analysis of distribution systems. Often, aggregated load profile data for a distribution circuit is available (e.g. feeder loading data collected via SCADA at the utility substation) and, while this data is typically accurate it masks the considerable variability of the 100’s or 1000’s of individual loads connected on the circuit. This tool was developed to model both the increased variability expected for these individual loads (e.g. the load of a single distribution transformer connected to 8-12 houses) and the expected diversity between loads on the circuit. It is important to note that the difference in variability and diversity, in the context of this tool, is that variability modeling only adds representative variability due to disaggregated load characteristics (e.g. the presence in the load profile of loads turning off and on like an air conditioner/oven) while the average energy profile remains the same as the user supplied power profile. Diversity modeling generates multiple individual load profiles which, in aggregate, sum to the user supplied power profile. Diversity is effectively variability in the energy usage over longer periods of time than seen in the variability model. Put another way, variability modeling supplies the expected variability due to the operation of various end-use loads and diversity modeling supplies the usage differences due to human behavior, schedules, etc. This load modeling tool was developed for use in generating data for distribution systems. Modeling is summarized by two major functions: 1) taking low resolution load profiles and adding intra-seconds variability onto the profiles, and 2) taking a user supplied load profile and distribution factors and adding both diversity and variability to the user supplied profile.

Zhu, Xiangqi↗
NASA NTRS1995

SeaWiFS technical report series. Volume 28: SeaWiFS algorithms, part 1

This document provides five brief reports that address several algorithm investigations sponsored by the Calibration and Validation Team (CVT) within the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) Project. This volume, therefore, has been designated as the first in a series of algorithm volumes. Chapter 1 describes the initial suite of masks, used to prevent further processing of contaminated radiometric data, and flags, which are employed to mark data whose quality (due to a variety of factors) may be suspect. In addition to providing the mask and flag algorithms, this chapter also describes the initial strategy for their implementation. Chapter 2 evaluates various strategies for the detection of clouds and ice in high latitude (polar and sub-polar regions) using Coastal Zone Color Scanner (CZCS) data. Chapter 3 presents an algorithm designed for detecting and masking coccolithosphore blooms in the open ocean. Chapter 4 outlines a proposed scheme for correcting the out-of-band response when SeaWiFS is in orbit. Chapter 5 gives a detailed description of the algorithm designed to apply sensor calibration data during the processing of level-1b data.

Hooker, Stanford B.↗
NASA NTRS1987

Edge detection - Image-plane versus digital processing

To optimize edge detection with the familiar Laplacian-of-Gaussian operator, it has become common to implement this operator with a large digital convolution mask followed by some interpolation of the processed data to determine the zero crossings that locate edges. It is generally recognized that this large mask causes substantial blurring of fine detail. It is shown that the spatial detail can be improved by a factor of about four with either the Wiener-Laplacian-of-Gaussian filter or an image-plane processor. The Wiener-Laplacian-of-Gaussian filter minimizes the image-gathering degradations if the scene statistics are at least approximately known and also serves as an interpolator to determine the desired zero crossings directly. The image-plane processor forms the Laplacian-of-Gaussian response by properly combining the optical design of the image-gathering system with a minimal three-by-three lateral-inhibitory processing mask. This approach, which is suggested by Marr's model of early processing in human vision, also reduces data processing by about two orders of magnitude and data transmission by up to an order of magnitude.

Huck, Friedrich O.↗
NASA NTRS2003

A Limited Comparison of the Thermal Durability of Polyimide Candidate Matrix Polymers with PMR-15

Studies were conducted with six different candidate high-temperature neat matrix resin specimens of varied geometric shapes to investigate the mechanisms involved in the thermal degradation of polyimides like PMR-15. The metrics for assessing the quality of these candidates were chosen to be glass transition temperature (T(sub g)), thermo-oxidative stability, dynamic mechanical properties, microstructural changes, and dimensional stability. The processing and mechanical properties were not investigated in the study reported herein. The dimensional changes and surface layer growth were measured and recorded. The data were in agreement with earlier published data. An initial weight increase reaction was observed to be dominating at the lower temperatures. However, at the more elevated temperatures, the weight loss reactions were prevalent and probably masked the weight gain reaction. These data confirmed the findings of the existence of an initial weight gain reaction previously reported. Surface- and core-dependent weight losses were shown to control the polymer degradation at the higher temperatures.

Bowles, Kenneth J.↗
DOE OSTI2024

Summit Darshan Archival Dataset

Summit Darshan Archival Dataset contains 2021 Summit Darshan log data for 25 applications and is grouped into science domains. The dataset is processed, and all the propriety fields are anonymized. The resultant data is converted into a tabular structure and saved in parquet file format. In this notebook, we demonstrate how to access the data. Data Organization: The data is organized into two directories: Darshan total (`darshan_total`): List all the high levels generated by the `darshan-parser --total` command on `.darshan` files. There is one parquet file for each application. Note: `uid` and `exe` field are masked Darshan detail (`darshan_detail`): This data contains detailed job level log information extracted by command `darshan-parser` on the raw `.darshan` files. The data is sorted by directory hierarchy in the order of `year/month/day (2021/12/07)`. For instance, to get the data for a `job_id` 3819766 of application `App11`, which was executed on `2021-12-07`can be accessed as follows. Note:`uid` and `filename` fields are masked

97 MATHEMATICS AND COMPUTING↗