Search NASA⌕ Search

SEARCH · Search NASA

Results for “data masking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2

Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masking algorithms have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. This paper summarizes results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10–30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), are evaluated within the CMIX. Those algorithms vary in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs are evaluated against existing reference cloud mask datasets. Those datasets vary in sampling methods, geographical distribution, sample unit (points, polygons, full image labels), and generation approaches (experts, machine learning, sky images). Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in how the reference datasets were produced. The algorithms were in good agreement for thick cloud detection, which were opaque and had lower uncertainties in their identification, in contrast to thin/semi-transparent clouds detection. Not only did CMIX allow identification of strengths and weaknesses of existing algorithms and potential areas of improvements, but also the problems associated with the existing reference datasets. The paper concludes with recommendations on generating new reference datasets, metrics, and an analysis framework to be further exploited and additional input datasets to be considered by future CMIX activities.

Sergii Skakun↗

AVHRR-Based Polar Pathfinder Products: Evaluation, Enhancement and Transition to MODIS

The Advanced Very High Resolution Radiometer (AVHRR)-Based Polar Pathfinder (APP) products include calibrated AVHRR channel data, surface temperatures, albedo, satellite scan and solar geometries, and cloud mask, all composited into twice-per-day images, and daily averaged fields of sea ice motion, for regions poleward of 50 latitude. Our general goals under this grant: (1) Quantify the APP accuracy and sources of error by comparing Pathfinder products with field measurements; (2) Determine the consistency of mean fields and trends in comparison with longer time series of available station data and forecast model output; (3) Investigate the consistency of the products between the different AVHRR instruments over the 1982-present period of the NOAA program; and (4) Compare and annual cycle of the APP products with MODIS to establish a baseline for extending Pathfinder-type products into the new ESE period.

Fowler, Charles↗

MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series

In this paper, we introduce Masked Anomaly Detection (MAD), a general self-supervised learning task for multivariate time series anomaly detection. With the increasing availability of sensor data from industrial systems, being able to detecting anomalies from streams of multivariate time series data is of significant importance. Given the scarcity of anomalies in real-world applications, the majority of literature has been focusing on modeling normality. The learned normal representations can empower anomaly detection as the model has learned to capture certain key underlying data regularities. A typical formulation is to learn a predictive model, i.e., use a window of time series data to predict future data values. In this paper, we propose an alternative self-supervised learning task. By randomly masking a portion of the inputs and training a model to estimate them using the remaining ones, MAD is an improvement over the traditional left-to-right next step prediction (NSP) task. Our experimental results demonstrate that MAD can achieve better anomaly detection rates over traditional NSP approaches when using exactly the same neural network (NN) base models, and can be modified to run as fast as NSP models during test time on the same hardware, thus making it an ideal upgrade for many existing NSP-based NN anomaly detection models.

97 MATHEMATICS AND COMPUTING↗

Assessment of Airframe Noise Reduction Technologies Based on EPNL from Flight Tests

The acoustic performance of various airframe noise reduction technologies – Adaptive Compliant Trailing Edge flap, main landing gear fairings, and gear cavity treatments – was determined, individually and in combination, using the Effective Perceived Noise Level metric. These noise measurements and calculations closely follow the Federal Aviation Administration aircraft noise certification standards, specifically for the approach noise measurement point. The flyover data correspond to pole-mounted, single-microphone measurements obtained during a series of flight tests, conducted under the NASA Flight Demonstrations and Capabilities project, that evaluated flap and landing gear noise reduction technologies. To minimize contributions from the propulsion system, the aircraft was flown along the approach path with engine thrust set at ground idle. Although contamination from engine, background, and secondary airframe noise sources partially masked the true performance of the tested technologies, the resulting acoustic data clearly showed substantial noise reductions relative to baseline levels. The acoustic benefits measured by the single microphones are consistent with previously reported trends in acoustic levels obtained from phased microphone array data.

Ravetta, Patricio A.↗

Assessment of computer techniques for processing digital LANDSAT MSS data for lithological discrimination of Serra do Ramalho, State of Bahia

Enhancement techniques and thematic classifications were applied to the metasediments of Bambui Super Group (Upper Proterozoic) in the Region of Serra do Ramalho, SW of the state of Bahia. Linear contrast stretch, band-ratios with contrast stretch, and color-composites allow lithological discriminations. The effects of human activities and of vegetation cover mask and limit, in several ways, the lithological discrimination with digital MSS data. Principal component images and color composite of linear contrast stretch of these products, show lithological discrimination through tonal gradations. This set of products allows the delineations of several metasedimentary sequences to a level superior to reconnaissance mapping. Supervised (maximum likelihood classifier) and nonsupervised (K-Means classifier) classification of the limestone sequence, host to fluorite mineralization show satisfactory results.

Paradella, W. R.↗

Remote Sensing of Radiative and Microphysical Properties of Clouds During TC (sup 4): Results from MAS, MASTER, MODIS, and MISR

The Moderate Resolution Imaging Spectroradiometer (MODIS) Airborne Simulator (MAS) and MODIS/Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) Airborne Simulator (MASTER) were used to obtain measurements of the bidirectional reflectance and brightness temperature of clouds at 50 discrete wavelengths between 0.47 and 14.2 microns (12.9 microns for MASTER). These observations were obtained from the NASA ER-2 aircraft as part of the Tropical Composition, Cloud and Climate Coupling (TC4) experiment conducted over Central America and surrounding Pacific and Atlantic Oceans between 17 July and 8 August 2007. Multispectral images in eleven distinct bands were used to derive a confidence in clear sky (or alternatively the probability Of cloud) over land and ocean ecosystems. Based on the results of individual tests run as part of the cloud mask, an algorithm was developed to estimate the phase of the clouds (liquid water, ice, or undetermined phase). The cloud optical thickness and effective radius were derived for both liquid water and ice clouds that were detected during each flight, using a nearly identical algorithm to that implemented operationally to process MODIS Cloud data from the Aqua and Terra satellites (Collection 5). This analysis shows that the cloud mask developed for operational use on MODIS, and tested using MAS and MASTER data in TC(sup 4), is quite capable of distinguishing both liquid water and ice clouds during daytime conditions over both land and ocean. The cloud optical thickness and effective radius retrievals use five distinct bands of the MAS (or MASTER), and these results were compared with nearly simultaneous retrievals of marine liquid water clouds from MODIS on the Terra spacecraft. Finally, this MODIS-based algorithm was adapted to Multiangle Imaging SpectroRadiometer (MISR) data to infer the cloud optical thickness Of liquid water clouds from MISR. Results of this analysis are compared and contrasted.

King, Michael D.↗

Experimental masking of RBV images to reduce stationary residual inaccuracies in radiometric correction

Return beam vidicon tube calibration data are used to correct spatially non-uniform radiance response in individual vidicons, during post-acquisition signal processing. However, examination of different ERTS-1 RBV-1, -2, and -3 image scenes showed stationary, repetitive residual inaccuracies in shading correction, large enough to affect qualitative and quantitative image analyses. Photographic masks designed to reduce residual shading were prepared from RBV-1, -2, and -3 images of a relatively cloud-free ocean scene of uniform reflectance. When applied to other RBV scenes, the masks enabled more closely corrected RBV images to be printed. Illustrations are given of RBV-1 image before and after correction, compared with an equivalent MSS spectral image of the same scene.

Ross, D. S.↗

Marine Boundary Layer Cloud Property Retrievals from High-Resolution ASTER Observations: Case Studies and Comparison with Terra MODIS

A research-level retrieval algorithm for cloud optical and microphysical properties is developed for the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) aboard the Terra satellite. It is based on the operational MODIS algorithm. This paper documents the technical details of this algorithm and evaluates the retrievals for selected marine boundary layer cloud scenes through comparisons with the operational MODIS Data Collection 6 (C6) cloud product. The newly developed, ASTERspecific cloud masking algorithm is evaluated through comparison with an independent algorithm reported in Zhao and Di Girolamo (2006). To validate and evaluate the cloud optical thickness (tau) and cloud effective radius (r(sub eff)) from ASTER, the high-spatial-resolution ASTER observations are first aggregated to the same 1000m resolution as MODIS. Subsequently, tau(sub aA) and r(sub eff, aA) retrieved from the aggregated ASTER radiances are compared with the collocated MODIS retrievals. For overcast pixels, the two data sets agree very well with Pearson's product-moment correlation coefficients of R greater than 0.970. However, for partially cloudy pixels there are significant differences between r(sub eff, aA) and the MODIS results which can exceed 10 micrometers. Moreover, it is shown that the numerous delicate cloud structures in the example marine boundary layer scenes, resolved by the high-resolution ASTER retrievals, are smoothed by the MODIS observations. The overall good agreement between the research-level ASTER results and the operational MODIS C6 products proves the feasibility of MODIS-like retrievals from ASTER reflectance measurements and provides the basis for future studies concerning the scale dependency of satellite observations and three-dimensional radiative effects.

marine boundary layer↗

Pseudonymization at Scale: OLCF’s Summit Usage Data Case Study

The analysis of vast amounts of data and the processing of complex computational jobs have traditionally relied upon high performance computing (HPC) systems, which offer reliable and efficient management of large-scale computational and data resources. Understanding these analyses’ needs is paramount for designing solutions that can lead to better science, and similarly, understanding the characteristics of the user behavior on those systems is important for improving user experiences on HPC systems. A common approach to gathering data about user behavior is to extract workload characteristics from system log data available only to system administrators. Recently at Oak Ridge Leadership Computing Facility (OLCF), however, we unveiled user behavior about the Summit supercomputer by collecting data from a user’s point of view with ordinary Unix commands.In this paper, we discuss the process, challenges, and lessons learned while preparing this dataset for publication and submission to an open data challenge. The original dataset contains personal identifiable information (PII) about the users of OLCF which needed be masked prior to publication, and we determined that anonymization, which scrubs PII completely, destroyed too much of the structure of the data to be interesting for the data challenge. We instead chose to pseudonymize the dataset, which reduced the linkability of the dataset to the users’ identities. Pseudonymization is significantly more computationally expensive than anonymization, and the size of our dataset, which is approximately 175 million lines of raw text, necessitated the development of a parallelized workflow that could be reused on different HPC machines. We demonstrate the scaling behavior of the workflow on two leadership class HPC systems at OLCF, and we show that we were able to bring the overall makespan time from an impractical 20+ hours on a single node down to around 2 hours. As a result of this work, we release the entire pseudonymized dataset and make the workflows and source code publicly available.

Maheshwari, Ketan↗

Incorporating Yearly Derived Winter Wheat Maps Into Winter Wheat Yield Forecasting Model

Wheat is one of the most important cereal crops in the world. Timely and accurate forecast of wheat yield and production at global scale is vital in implementing food security policy. Becker-Reshef et al. (2010) developed a generalized empirical model for forecasting winter wheat production using remote sensing data and official statistics. This model was implemented using static wheat maps. In this paper, we analyze the impact of incorporating yearly wheat masks into the forecasting model. We propose a new approach of producing in season winter wheat maps exploiting satellite data and official statistics on crop area only. Validation on independent data showed that the proposed approach reached 6% to 23% of omission error and 10% to 16% of commission error when mapping winter wheat 2-3 months before harvest. In general, we found a limited impact of using yearly winter wheat masks over a static mask for the study regions.

MODIS↗

Sensitivity of the MAR Regional Climate Model Snowpack to the Parameterization of the Assimilation of Satellite-Derived Wet-Snow Masks on the Antarctic Peninsula

Both regional climate models (RCMs) and remote sensing (RS) data are essential tools in understanding the response of polar regions to climate change. RCMs can simulate how certain climate variables, such as surface melt, runoff and snowfall, are likely to change in response to different climate scenarios but are subject to biases and errors. RS data can assist in reducing and quantifying model uncertainties by providing indirect observations of the modeled variables on the present climate. In this work, we improve on an existing scheme to assimilate RS wet snow occurrence data with the “Modèle Atmosphérique Régional” (MAR) RCM and investigate the sensitivity of the RCM to the parameters of the scheme. The assimilation is performed by nudging the MAR snowpack temperature to match the presence of liquid water observed by satellites. The sensitivity of the assimilation method is tested by modifying parameters such as the depth to which the MAR snowpack is warmed or cooled, the quantity of water required to qualify a MAR pixel as “wet” (0.1 % or 0.2 % of the snowpack mass being water), and assimilating different RS datasets. Data assimilation is carried out on the Antarctic Peninsula for the 2019–2021 period. The results show an increase in meltwater production (+66.7 % on average, or +95 Gt), along with a small decrease in surface mass balance (SMB) (−4.5 % on average, or −20 Gt) for the 2019–2020 melt season after assimilation. The model is sensitive to the tested parameters, albeit with varying orders of magnitude. The prescribed warming depth has a larger impact on the resulting surface melt production than the liquid water content (LWC) threshold due to strong refreezing occurring within the top layers of the snowpack. The values tested for the LWC threshold are lower than the LWC for typical melt days (approximately 1.2 %) and impact results mainly at the beginning and end of the melting period. The assimilation method will allow for the estimation of uncertainty in MAR meltwater production and will enable the identification of potential issues in modeling near-surface snowpack processes, paving the way for more accurate simulations of snow processes in model projections.

regional climate models↗

Remote Sensing of Liquid Water and Ice Cloud Optical Thickness and Effective Radius in the Arctic: Application of Airborne Multispectral MAS Data

A multispectral scanning spectrometer was used to obtain measurements of the reflection function and brightness temperature of clouds, sea ice, snow, and tundra surfaces at 50 discrete wavelengths between 0.47 and 14.0 microns. These observations were obtained from the NASA ER-2 aircraft as part of the FIRE Arctic Clouds Experiment, conducted over a 1600 x 500 km region of the north slope of Alaska and surrounding Beaufort and Chukchi Seas between 18 May and 6 June 1998. Multispectral images of the reflection function and brightness temperature in 11 distinct bands of the MODIS Airborne Simulator (MAS) were used to derive a confidence in clear sky (or alternatively the probability of cloud), shadow, and heavy aerosol over five different ecosystems. Based on the results of individual tests run as part of the cloud mask, an algorithm was developed to estimate the phase of the clouds (water, ice, or undetermined phase). Finally, the cloud optical thickness and effective radius were derived for both water and ice clouds that were detected during one flight line on 4 June. This analysis shows that the cloud mask developed for operational use on MODIS, and tested using MAS data in Alaska, is quite capable of distinguishing clouds from bright sea ice surfaces during daytime conditions in the high Arctic. Results of individual tests, however, make it difficult to distinguish ice clouds over snow and sea ice surfaces, so additional tests were added to enhance the confidence in the thermodynamic phase of clouds over the Beaufort Sea. The cloud optical thickness and effective radius retrievals used 3 distinct bands of the MAS, with the newly developed 1.62 and 2.13 micron bands being used quite successfully over snow and sea ice surfaces. These results are contrasted with a MODIS-based algorithm that relies on spectral reflectance at 0.87 and 2.13 micron.

King, Michael D.↗

A Satellite-Derived Climate-Quality Data Record of the Clear-Sky Surface Temperature of the Greenland Ice Sheet

We have developed a climate-quality data record of the clear-sky surface temperature of the Greenland Ice Sheet using the Moderate-Resolution Imaging Spectroradiometer (MODIS) Terra ice-surface temperature (1ST) algorithm. A climate-data record (CDR) is a time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change. We present daily and monthly Terra MODIS ISTs of the Greenland Ice Sheet beginning on 1 March 2000 and continuing through 31 December 2010 at 6.25-km spatial resolution on a polar stereographic grid within +/-3 hours of 17:00Z or 2:00 PM Local Solar Time. Preliminary validation of the ISTs at Summit Camp, Greenland, during the 2008-09 winter, shows that there is a cold bias using the MODIS IST which underestimates the measured surface temperature by approximately 3 C when temperatures range from approximately -50 C to approximately -35 C. The ultimate goal is to develop a CDR that starts in 1981 with the Advanced Very High Resolution (AVHRR) Polar Pathfinder (APP) dataset and continues with MODIS data from 2000 to the present. Differences in the APP and MODIS cloud masks have so far precluded the current IST records from spanning both the APP and MODIS IST time series in a seamless manner though this will be revisited when the APP dataset has been reprocessed. The Greenland IST climate-quality data record is suitable for continuation using future Visible Infrared Imager Radiometer Suite (VIIRS) data and will be elevated in status to a CDR when at least 9 more years of climate-quality data become available either from MODIS Terra or Aqua, or from the VIIRS. The complete MODIS IST data record will be available online in the summer of 2011.

Hall, Dorothy K.↗

Use of MODIS Satellite Data to Evaluate Juniperus spp. Pollen Phenology to Support a Pollen Dispersal Model, PREAM, to Support Public Health Allergy Alerts

Juniperus spp. pollen is a significant aeroallergen that can be transported 200-600 km from the source. Local observations of Juniperus spp. phenology may not be consistent with the timing and source of pollen collected by pollen sampling instruments. Methods: The Dust REgional Atmospheric Model (DREAM)is a verified model for atmospheric dust transport modeling using MODIS data products to identify source regions and quantities of dust. We successfully modified the DREAM model to incorporate pollen transport (PREAM) and used MODIS satellite images to develop Juniperus ashei pollen input source masks. The Pollen Release Potential Source Map, also referred to as a source mask in model applications, may use different satellite platforms and sensors and a variety of data sets other than the USGS GAP data we used to map J. ashei cover type. MODIS derived percent tree cover is obtained from MODIS Vegetation Continuous Fields (VCF) product (collection 3 and 4, MOD44B, 500 and 250 m grid resolution). We use updated 2010 values to calculate pollen concentration at source (J. ashei ). The original MODIS derived values are converted from native approx. 250 m to 990m (approx. 1 km) for the calculation of a mask to fit the model (PREAM) resolution. Results: The simulation period is chosen following the information that in the last 2 weeks of December 2010. The PREAM modeled near-surface concentrations (Nm-3) shows the transport patterns of J. ashei pollen over a 5 day period (Fig. 2). Typical scales of the simulated transport process are regional.

Luvall, J. C.↗

Electronic Coherence and Coherent Dephasing in the Optical Control of Electrons in Graphene

Electronic coherence is of utmost importance for the access and control of quantum-mechanical solid-state properties. Using a purely electronic observable, the photocurrent, we measure a lower bound of the electronic coherence time of 22 ± 4 fs in graphene. The photocurrent is ideally suited to measure electronic coherence, as it is a direct result of coherent quantum-path interference, controlled by the delay between two ultrashort two-color laser pulses. The maximum delay for which interference between the population amplitude injected by the first pulse interferes with that generated by the second pulse determines the electronic coherence time. In particular, numerical simulations reveal that the experimental data yields a lower bound on the electronic coherence time, masked by coherent dephasing due to the broadband absorption in graphene. Overall, we expect that our results will significantly advance the understanding of coherent quantum control in solid-state systems ranging from excitation with weak fields to strongly driven systems.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Expectation-propagation for weak radionuclide identification at radiation portal monitors

We propose a sparsity-promoting Bayesian algorithm capable of identifying radionuclide signatures from weak sources in the presence of a high radiation background. The proposed method is relevant to radiation identification for security applications. In such scenarios, the background typically consists of terrestrial, cosmic, and cosmogenic radiation that may cause false positive responses. We evaluate the new Bayesian approach using gamma-ray data and are able to identify weapons-grade plutonium, masked by naturally-occurring radioactive material (NORM), in a measurement time of a few seconds. We demonstrate this identification capability using organic scintillators (stilbene crystals and EJ-309 liquid scintillators), which do not provide direct, high-resolution, source spectroscopic information. Compared to the EJ-309 detector, the stilbene-based detector exhibits a lower identification error, on average, owing to its better energy resolution. Organic scintillators are used within radiation portal monitors to detect gamma rays emitted from conveyances crossing ports of entry. The described method is therefore applicable to radiation portal monitors deployed in the field and could improve their threat discrimination capability by minimizing “nuisance” alarms produced either by NORM-bearing materials found in shipped cargoes, such as ceramics and fertilizers, or radionuclides in recently treated nuclear medicine patients.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

Airborne measurements of tropospheric ozone destruction and particulate bromide formation in the Arctic

Aircraft profiles of O3 concentrations over the Arctic ice pack in spring exhibit a depletion of O3 beneath the surface temperature inversion. One such profile from the NOAA WP-3D Arctic Gas and Aerosol Sampling Program (AGASP) flights in April, 1986 north of Alert, NWT (YLT, 82.5 N) is shown. The gradient of O3 across the temperature inversion, which is essentially a step function from tropospheric values (35 to 40 ppbv) to 0, is somewhat masked by a 1-min running mean applied to the data. Evidence is presented that O3 destruction beneath the Arctic temperature inversion is the result of a photochemical reaction between gaseous Br compounds and O3 to produce particulate Br aerosol. It is noted that in springtime, O3 at the Alert Baseline Station regularly decreases from 30 to 40 ppbv to near 0 over the period of a few hours to a day. At the same time, there is a production of particulate Br with a near 1.0 anti-correlation to O3 concentration. Surface concentrations of bromoform in the Arctic exhibit a rapid decrease following polar sunrise. AGASP aircraft measurements of filterable bromine particulates in the Arctic (March-April, 1983 and 1986) are shown. The greatest concentrations of Br aerosol (shown as enrichment factors relative to to Na in seawater, EFBR (Na)) were observed in samples collected beneath the surface temperature inversion over ice. Samples collected at the same altitude over open ocean (off Spitzbergen) labeled Marine did not exhibit similar Br enrichments. A second region of particulate Br enrichment was observed in the lower stratosphere, which regularly descends to below 500 mb (5.5 km) in the high Arctic. The NOAA WP-3D flew in the stratosphere on all AGASP flights and occasionally measured O3 concentrations in excess of 300 ppbv.

Schnell, Russell C.↗

The Design and Evaluation of "CAPTools"--A Computer Aided Parallelization Toolkit

Writing applications for high performance computers is a challenging task. Although writing code by hand still offers the best performance, it is extremely costly and often not very portable. The Computer Aided Parallelization Tools (CAPTools) are a toolkit designed to help automate the mapping of sequential FORTRAN scientific applications onto multiprocessors. CAPTools consists of the following major components: an inter-procedural dependence analysis module that incorporates user knowledge; a 'self-propagating' data partitioning module driven via user guidance; an execution control mask generation and optimization module for the user to fine tune parallel processing of individual partitions; a program transformation/restructuring facility for source code clean up and optimization; a set of browsers through which the user interacts with CAPTools at each stage of the parallelization process; and a code generator supporting multiple programming paradigms on various multiprocessors. Besides describing the rationale behind the architecture of CAPTools, the parallelization process is illustrated via case studies involving structured and unstructured meshes. The programming process and the performance of the generated parallel programs are compared against other programming alternatives based on the NAS Parallel Benchmarks, ARC3D and other scientific applications. Based on these results, a discussion on the feasibility of constructing architectural independent parallel applications is presented.

Yan, Jerry↗