Search NASA⌕ Search

SEARCH · Search NASA

Results for “scene understanding”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Robotic Understanding of Spatial Relationships Using Neural-Logic Learning

Understanding spatial relations of objects is critical in many robotic applications such as grasping, manipulation, and obstacle avoidance. Humans can simply reason object's spatial relations from a glimpse of a scene based on prior knowledge of spatial constraints. The proposed method enables a robot to comprehend spatial relationships among objects from RGB-D data. This paper proposed a neural-logic learning framework to learn and reason spatial relations from raw data by following logic rules on spatial constraints. The neural-logic network consists of three blocks: grounding block, spatial logic block, and inference block. The grounding block extracts high-level features from the raw sensory data. The spatial logic blocks can predicate fundamental spatial relations by training a neural network with spatial constraints. The inference block can infer complex spatial relations based on the predicated fundamental spatial relations. Simulations and robotic experiments evaluated the performance of the proposed method.

Wang, Dali↗
NASA NTRS1980

LANDSAT signature development program

LANDSAT signature development program (LSDP) automatically produces unsupervised classification of scene from LANDSAT data tape. Program is effective enough to be useful to sophisticated remote sensing analyst yet is simple enough to be utilized by ground truth investigators who have only basic understanding of computer and remote sensing procedures.

Bland, R. A.↗
NASA NTRS2018

On the Ability of Space- Based Passive and Active Remote Sensing Observations of CO2 to Detect Flux Perturbations to the Carbon Cycle

Space-borne observations of CO2 are vital to gaining understanding of the carbon cycle in regions of the world that are difficult to measure directly, such as the tropical terrestrial biosphere, the high northern and southern latitudes, and in developing nations such as China. Measurements from passive instruments such as GOSAT (Greenhouse Gases Observing Satellite) and OCO-2 (Orbiting Carbon Observatory 2), however, are constrained by solar zenith angle limitations as well as sensitivity to the presence of clouds and aerosols. Active measurements such as those in development for the Active Sensing of CO2 Emissions over Nights, Days and Seasons (ASCENDS) mission show strong potential for making measurements in the high-latitude winter and in cloudy regions. In this work we examine the enhanced flux constraint provided by the improved coverage from an active measurement such as ASCENDS. The simulation studies presented here show that with sufficient precision, ASCENDS will detect permafrost thaw and fossil fuel emissions shifts at annual and seasonal time scales, even in the presence of transport errors, representativeness errors, and biogenic flux errors. While OCO-2 can detect some of these perturbations at the annual scale, the seasonal sampling provided by ASCENDS provides the stronger constraint. Plain Language Summary: Active and passive remote sensors show the potential to provide unprecedented information on the carbon cycle. With the all-season sampling, active remote sensors are more capable of constraining high-latitude emissions. The reduced sensitivity to cloud and aerosol also makes active sensors more capable of providing information in cloudy and polluted scenes with sufficient accuracy. These experiments account for errors that are fundamental to the top-down approach for constraining emissions, and even including these sources of error, we show that satellite remote sensors are critical for understanding the carbon cycle.

tropical terrestrial biosphere↗
NASA NTRS2020

Comparison of Cloud Detection Algorithms for Sentinel-2 Imagery

Accurate, automated cloud and cloud shadow detection is a key component of the processing needed to prepare optical satellite imagery for scientific analysis. Many existing cloud detection algorithms rely on temperature information to identify clouds, making detection difficult for imagers that lack a thermal band, like Sentinel-2. To get maximum benefit from Sentinel-2 products it is critical to understand which algorithms best identify clouds and their shadows in images. We examined the relative performance of five different cloud-masking algorithms (Sen2Cor, MAJA, LaSRC, Fmask and Tmask) in 6 Sentinel-2 scenes (28 total images) distributed across the Eastern Hemisphere. Expanding on these comparisons, we tested ensemble approaches to improve results. We tested three ensemble approaches to cloud and shadow classification based on the outputs of the five initial algorithms using the cloud masks in: (1) a majority prediction model; (2) a random forests model; and (3) a conditional logic model. Accuracy assessments show a trade-off between omission and commission errors in cloud detection for individual algorithms across all sites, and some algorithms are better at detecting either clouds or cloud shadows. No single algorithm outperforms the others for both clouds and shadows. Aggregating the results from multiple algorithms produces fewer undetected clouds and higher overall accuracy than any single algorithm, with as high as 2.7% improvement over the top-performing algorithm, suggesting an ensemble approach may be the most useful for processing of Sentinel-2 data.

Sentinel-2↗
NASA NTRS2022

Polarization performance simulation for the GeoXO atmospheric composition instrument: NO2 retrieval impacts

NOAA’s Geostationary Extended Observations (GeoXO) constellation will continue and expand on the capabilities of the current generation of geostationary satellite systems to support US weather, ocean, atmosphere, and climate operations. It is planned to consist of a dedicated atmospheric composition instrument (ACX) to support air quality forecasting and monitoring by providing similar capabilities to missions such as TEMPO (Tropospheric Emission: Monitoring Pollution), currently planned to launch in 2023, and Ozone Monitoring Instrument (OMI), TROPOMI (TROPOspheric Monitoring Instrument), and GEMS (Geostationary Environment Monitoring Spectrometer) currently in operation. As the early phases of ACX development are progressing, design trade-offs are being considered to understand the relationship between instrument design choices and trace gas retrieval impacts. Some of these choices will affect the instrument polarization sensitivity (PS), which can have radiometric impacts on environmental satellite observations. We conducted a study to investigate how such radiometric impacts can affect NO2 retrievals by exploring their sensitivities to time of day, location, and scene type with an ACX instrument model that incorporates PS. The study addresses the basic steps of operational NO2 retrievals: the spectral fitting step and the conversion of slant column to vertical column via the air mass factor (AMF). The spectral fitting step was performed by generating at-sensor radiance from a clear sky scene with a known NO2 amount, the application of an instrument model including both instrument PS and noise, and a physical retrieval. The spectral fitting step was found to mitigate the impacts of instrument PS. The AMF-related step was considered for clear sky and partially cloudy scenes, where instrument PS can lead to errors in interpreting the cloud content, propagating to AMF errors and finally to NO2 retrieval errors. For this step, the NO2 retrieval impacts were small but non-negligible for high NO2 amounts; we estimated that a typical high NO2 amount can cause maximum of 0.25 x 10^15 molecules/cm^2 for a PS of 5%. These simulation capabilities were designed to aid in the development of a GeoXO atmospheric composition instrument that will improve our ability to monitor and understand the Earth’s atmosphere.

trace gases↗
NASA NTRS2022

Polarization Performance Simulation for the GeoXO Atmospheric Composition Instrument: NO2 Retrieval Impacts

NOAA's Geostationary Extended Observations (GeoXO) constellation will continue and expand on the capabilities of the current generation of geostationary satellite systems to support US weather, ocean, atmosphere, and climate operations. It is planned to consist of a dedicated atmospheric composition instrument (ACX) to support air quality forecasting and monitoring by providing capabilities similar to missions such as TEMPO (Tropospheric Emission: Monitoring Pollution), currently planned to launch in 2023, as well as OMI (Ozone Monitoring Instrument), TROPOMI (TROPOspheric Monitoring Instrument), and GEMS (Geostationary Environment Monitoring Spectrometer) currently in operation. As the early phases of ACX development are progressing, design trade-offs are being considered to understand the relationship between instrument design choices and trace gas retrieval impacts. Some of these choices will affect the instrument polarization sensitivity (PS), which can have radiometric impacts on environmental satellite observations. We conducted a study to investigate how such radiometric impacts can affect NO2 retrievals by exploring their sensitivities to time of day, location, and scene type with an ACX instrument model that incorporates PS. The study addresses the basic steps of operational NO2 retrievals: the spectral fitting step and the conversion of slant column to vertical column via the air mass factor (AMF). The spectral fitting step was performed by generating at-sensor radiance from a clear-sky scene with a known NO2 amount, the application of an instrument model including both instrument PS and noise, and a physical retrieval. The spectral fitting step was found to mitigate the impacts of instrument PS. The AMF-related step was considered for clear-sky and partially cloudy scenes, for which instrument PS can lead to errors in interpreting the cloud content, propagating to AMF errors and finally to NO2 retrieval errors. For this step, the NO2 retrieval impacts were small but non-negligible for high NO2 amounts; we estimated that a typical high NO2 amount can cause a maximum retrieval error of 0.25×1015 molec. cm−2 for a PS of 5 %. These simulation capabilities were designed to aid in the development of a GeoXO atmospheric composition instrument that will improve our ability to monitor and understand the Earth's atmosphere.

Aaron Pearlman↗
NASA NTRS2021

Quantifying and Characterizing Striping of Microwave Humidity Sounder with Observation and Simulation

Striping has been observed in the MetOp-A Microwave Humidity Sounder (MHS) data since its degradation in November 2018. However, accurate striping quantification and characterization remain challenging due to the large scene dynamics observed at W/G bands of MHS. Here we have developed a set of novel algorithms for striping quantification, decomposition, characterization, and simulation. Our algorithm extracts striping from the warm-load and cold-space scenes that are relatively stable. We break down the striping into two parts of thermal and 1/f noise and quantify their absolute magnitude and relative ratio. We found a significant increase in striping at 157 GHz that has more than quadrupled by October 2019 relative to its normal level. Regardless of the degradation, the ratio of thermal and 1/f noise remains the same. Our simulation reproduces all the characteristics of striping against observation. It is shown that 1/f noise generates sharp, non-periodic stripes, while thermal noise also generates stripes but with smoother band features. The latter is due to the periodic calibration that has a chopping effect. The striping percentage, defined as the ratio of 1/f to total noise, shows no dependence on the scene temperature. Striping is pronounced not only in 157 GHz but also in 89 and190 GHz with the striping percentage over 50%, while lower in183 GHz of 20%. The results provide insights for quantifying and understanding striping. Our algorithm can be applied to other radiometers and to simulate striping for evaluating its impact in data assimilation and science products.

John Xun Yang↗
DOE OSTI2024

Self-Supervised Cloud Classification

Abstract Low-level marine clouds play a pivotal role in Earth’s weather and climate through their interactions with radiation, heat and moisture transport, and the hydrological cycle. These interactions depend on a range of dynamical and microphysical processes that result in a broad diversity of cloud types and spatial structures, and a comprehensive understanding of cloud morphology is critical for continued improvement of our atmospheric modeling and prediction capabilities moving forward. Deep learning has recently accelerated our ability to study clouds using satellite remote sensing, and machine learning classifiers have enabled detailed studies of cloud morphology. A major limitation of deep learning approaches to this problem, however, is the large number of hand-labeled samples that are required for training. This work applies a recently developed self-supervised learning scheme to train a deep convolutional neural network (CNN) to map marine cloud imagery to vector embeddings that capture information about mesoscale cloud morphology and can be used for satellite image classification. The model is evaluated against existing cloud classification datasets and several use cases are demonstrated, including training cloud classifiers with very few labeled samples, interrogation of the CNN’s learned internal feature representations, cross-instrument application, and resilience against sensor calibration drift and changing scene brightness. The self-supervised approach learns meaningful internal representations of cloud structures and achieves comparable classification accuracy to supervised deep learning methods without the expense of creating large hand-annotated training datasets. Significance Statement Marine clouds heavily influence Earth’s weather and climate, and improved understanding of marine clouds is required to improve our atmospheric modeling capabilities and physical understanding of the atmosphere. Recently, deep learning has emerged as a powerful research tool that can be used to identify and study specific marine cloud types in the vast number of images collected by Earth-observing satellites. While powerful, these approaches require hand-labeling of training data, which is prohibitively time intensive. This study evaluates a recently developed self-supervised deep learning method that does not require human-labeled training data for processing images of clouds. We show that the trained algorithm performs competitively with algorithms trained on hand-labeled data for image classification tasks. We also discuss potential downstream uses and demonstrate some exciting features of the approach including application to multiple satellite instruments, resilience against changing image brightness, and its learned internal representations of cloud types. The self-supervised technique removes one of the major hurdles for applying deep learning to very large atmospheric datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Detecting damaged buildings using real-time crowdsourced images and transfer learning

After significant earthquakes, we can see images posted on social media platforms by individuals and media agencies owing to the mass usage of smartphones these days. These images can be utilized to provide information about the shaking damage in the earthquake region both to the public and research community, and potentially to guide rescue work. This paper presents an automated way to extract the damaged buildings images after earthquakes from social media platforms such as Twitter and thus identify the particular user posts containing such images. Using transfer learning and ~ 6500 manually labelled images, we trained a deep learning model to recognize images with damaged buildings in the scene. The trained model achieved good performance when tested on newly acquired images of earthquakes at different locations and when ran in near real-time on Twitter feed after the 2020 M7.0 earthquake in Turkey. Furthermore, to better understand how the model makes decisions, we also implemented the Grad-CAM method to visualize the important regions on the images that facilitate the decision.

58 GEOSCIENCES↗
NASA NTRS1987

Evaluation of flow topology from numerical data

Results obtained from numerical calculations and modern (optical) diagnostics are often too complicated for manual inspection, manipulation and display. A simpler but still accurate description of these results is needed to facilitate data understanding. The paper discusses preliminary investigations into methods for the decomposition of 2-D and 3-D fluid flow data bases into elementary structures for purposes of description, analysis and comparison. An approach which involves the development of scene-like representation of the flow topology is presented. Using features such as critical points and dividing streamlines as a basis, a representation of the global topology of the flow is generated. The topology is then represented by a graph with the various structures represented by the nodes and their relationships in the flow by the connecting lines of the graph. Once the flow field has been placed in this form, it can be studied and compared with other data sets using techniques of syntactic pattern recognition or displayed using 3-D graphics techniques.

Hesselink, Lambertus↗
NASA NTRS1980

Instrument scanning and controlling: Using eye movement data to understand pilot behavior and strategies

Eye movement data and other parameters including instrument readings, aircraft state and position variables, and control maneuvers were recorded while pilots flew ILS simulations in a B 737. The experiment itself employed seven airline pilots, each of whom flew approximately 40 approach/landing sequences. The simulator was equipped with a night visual scene but the scene was fogged out down to approximately 60 meters (200 ft). The instrument scanning appeared to follow aircraft parameters not physical position of instruments. One important implication of the results is: pilots look for categories or packets of information. Control inputs were tabulated according to throttle, wheel position, column, and pitch trim changes. Three seconds of eye movements before and after the control input were then obtained. Analysis of the eye movement data for the controlling periods showed clear patterns. The results suggest a set of miniscan patterns which are used according to the specific details of the situation. A model is developed which integrates scanning and controlling. Differentiations are made between monitoring and controlling scans.

Dick, A. O.↗
NASA NTRS1997

Investigation of Cloud Properties and Atmospheric Profiles with MODIS

The WINter Cloud Experiment (WINCE) was directed and supported by personnel from the University of Wisconsin in January and February. Data sets of good quality were collected by the MODIS Airborne Simulator (MAS) and other instruments on the NASA ER2; they will be used to develop and validate cloud detection and cloud property retrievals over winter scenes (especially over snow). Software development focused on utilities needed for all of the UW product executables; preparations for Version 2 software deliveries were almost completed. A significant effort was made, in cooperation with SBRS and MCST, in characterizing and understanding MODIS PFM thermal infrared performance; crosstalk in the longwave infrared channels continues to get considerable attention.

Menzel, Paul↗
NASA NTRS1995

Aerolian erosion, transport, and deposition of volcaniclastic sands among the shifting sand dunes, Christmas Lake Valley, Oregon: TIMS image analysis

Remote sensing is a tool that, in the context of aeolian studies, offers a synoptic view of a dune field, sand sea, or entire desert region. Blount et al. (1990) presented one of the first studies demonstrating the power of multispectral images for interpreting the dynamic history of an aeolian sand sea. Blount's work on the Gran Desierto of Mexico used a Landsat TM scene and a linear spectral mixing model to show where different sand populations occur and along what paths these sands may have traveled before becoming incorporated into dunes. Interpretation of sand transport paths and sources in the Gran Desierto led to an improved understanding of the origin and Holocene history of the dunes. With the anticipated advent of the EOS-A platform and ASTER thermal infrared capability in 1998, it will become possible to look at continental sand seas and map sand transport paths using 8-12 mu m bands that are well-suited to tracking silicate sediments. A logical extension of Blount's work is to attempt a similar study using thermal infrared images. One such study has already begun by looking at feldspar, quartz, magnetite, and clay distributions in the Kelso Dunes of southern California. This paper describes the geology and application of TIMS image analysis of a less-well known Holocene dune field in south central Oregon using TIMS data obtained in 1991.

Edgett, Kenneth S.↗
NASA NTRS1973

Computer-Generated Movies for Mission Planning

Computer-generated movies help the viewer to understand mission dynamics and get quantitative details. Sample movie frames demonstrate the uses and effectiveness of movies in mission planning. Tools needed for movie-making include computer programs to generate images on film and film processing to give the desired result. Planning scenes to make an effective product requires some thought and experience. Viewpoints and timing are particularly important. Lessons learned so far and problems still encountered are discussed.

Roberts, P. H., Jr.↗
NASA NTRSDate not supplied

Comparing Tropopause-Penetrating Convection Identifications Derived from NEXRAD and GOES over the Contiguous United States

Overshooting cloud tops (OTs) are a result of powerful updrafts that rapidly transport air from the troposphere to the lower stratosphere. Aside from the well-known relationships between strong updrafts and severe weather, the boundaries of updrafts, including the OT, are turbulent, which leads to irreversible mixing of air between the stratosphere and troposphere, altering the composition of both layers. In order to better understand the impact of OTs, knowing precisely when and where they occur is vital. Within the last decade remote-sensing observations from satellites and radars have been used to identify OTs, but the results thus far have not been entirely consistent due to challenges with accurately identifying OTs with these sensor datasets. This study compares OTs identified using NEXRAD reflectivity data from the GridRad dataset with an approach that uses GOES infrared (IR) data. The GridRad method estimates the altitude of echo tops directly from the radar pointing geometry, merging a combination of radars that enables products every 5 mins, with 2 km spatial and 1 km vertical grid spacing. The GOES satellite method uses spatial patterns in tropopause-relative IR brightness temperature to derive an OT likelihood. The study region covers a large part of the contiguous United States during selected active convection dates in 2017 and includes data from GOES-13/16. A large sample of OTs observed by NEXRAD and GOES-13/16 satellites are analyzed to better understand NEXRAD and IR observations of OTs, quantify agreement between the OT detection methods, and demonstrate how the 4x increase in spatial resolution from GOES-13 to GOES-16 impacts OT algorithm detection performance. The results show that for nearly time-matched scenes, with many severe storms, GOES-13 and -16 detection accuracy is quite comparable; however, some missed detections are attributed to poorer GOES-13 image resolution. It was found that GOES-13 OT regions are ~2 K warmer than GOES-16 on average, but such differences exceed 5 K for the most intense OT regions. It is found that false GOES IR-based detections are unavoidable, given that cold cloud pixels significantly colder than the tropopause from nearby or recently decayed OTs can persist within anvils for 15 minutes or more.

John W. Cooney↗
NASA NTRS1989

Active microwave classification of sea ice

Radar backscatter studies of Arctic sea ice have been carried out over a number of years with the intent to acquire physical property information through the examination of microwave signatures. The breadth of these studies continues to expand; as an example, measurements are now conducted at frequencies from 500 MHz to about 100 GHz. One of the scientific goals of this work has been to develop an improved outstanding of the scattering processes at play. A second, equally important goal has been to apply the knowledge gained in examining the backscatter response of ice and snow made in conjunction with the detailed scene characterizations, the insight gained through theoretical modeling and parametric study, and the data entered into the radar signature library to develop procedures to convert microwave signal information (available in the very near future) into valuable data products. This should ultimately provide a better understanding of the environment. The author discusses what has been learned through the many efforts associated with the near-surface scatterometer measurement programs and how the knowledge gained is assisting in the development of future sea ice type satellite algorithms. The logic and mechanisms used in discriminating sea ice types are presented.

Onstott, Robert G.↗
NASA NTRS2015

Next-Generation Angular Distribution Models for Top-of-Atmosphere Radiative Flux Calculation from the CERES Instruments: Methodology

The top-of-atmosphere (TOA) radiative fluxes are critical components to advancing our understanding of the Earth's radiative energy balance, radiative effects of clouds and aerosols, and climate feedback. The Clouds and the Earth's Radiant Energy System (CERES) instruments provide broadband shortwave and longwave radiance measurements. These radiances are converted to fluxes by using scene-type-dependent angular distribution models (ADMs). This paper describes the next-generation ADMs that are developed for Terra and Aqua using all available CERES rotating azimuth plane radiance measurements. Coincident cloud and aerosol retrievals, and radiance measurements from the Moderate Resolution Imaging Spectroradiometer (MODIS), and meteorological parameters from Goddard Earth Observing System (GEOS) data assimilation version 5.4.1 are used to define scene type. CERES radiance measurements are stratified by scene type and by other parameters that are important for determining the anisotropy of the given scene type. Anisotropic factors are then defined either for discrete intervals of relevant parameters or as a continuous functions of combined parameters, depending on the scene type. Significant differences between the ADMs described in this paper and the existing ADMs are over clear-sky scene types and polar scene types. Over clear ocean, we developed a set of shortwave (SW) ADMs that explicitly account for aerosols. Over clear land, the SW ADMs are developed for every 1 latitude1 longitude region for every calendar month using a kernel-based bidirectional reflectance model. Over clear Antarctic scenes, SW ADMs are developed by accounting the effects of sastrugi on anisotropy. Over sea ice, a sea-ice brightness index is used to classify the scene type. Under cloudy conditions over all surface types, the longwave (LW) and window (WN) ADMs are developed by combining surface and cloud-top temperature, surface and cloud emissivity, cloud fraction, and precipitable water. Compared to the existing ADMs, the new ADMs change the monthly mean instantaneous fluxes by up to 5Wm(exp -2) on a regional scale of 1 deg. latitude x 1 deg. longitude, but the flux changes are less than 0.5Wm(exp -2) on a global scale.

Su, W.↗
NASA NTRS1975

IPL processing of the Mariner 10 images of Mercury

This paper describes the digital processing performed on the images of Mercury returned to earth from Mariner 10. Each image contains considerably more information than can be displayed in a single picture. Several specialized processing techniques and procedures are utilized to display the particular information desired for specific scientific analyses: radiometric decalibration for photometric investigations, high-pass filtering to characterize morphology, modulation transfer function restoration to provide the highest possible resolution, scene-dependent filtering of the terminator images to provide maximum feature discriminability in the regions of low illumination, and rectification to cartographic projections to provide known geometric relationships between features. A principal task was the construction of full disk mosaics as an aid to the understanding of surface structure on a global scale.

Soha, J. M.↗