Search NASA⌕ Search

SEARCH · Search NASA

Results for “scene understanding”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Towards Geospatial Knowledge Graph Infused Neuro-Symbolic AI for Remote Sensing Scene Understanding

Deep learning has proven its effectiveness in numerous tasks for remote sensing scene understanding. However there is an increasing interest to explore fusion of domain-specific background information to the deep neural network to further improve its performance. Remote sensing researchers are also working towards developing models that generalize and adapt to multiple applications. Generalization challenges coupled with the scarcity of large corpora of high-quality noise-free labelled data, have together fueled an interest for leveraging background information. Knowledge graphs serve as excellent choice to represent domain-specific information in a structured, standardized and extensible manner. Integrating symbolic knowledge representations in the form of Knowledge Graph Embedding (KGE) to perform neuro-symbolic reasoning is an emerging research direction promising significant impacts. This vision paper seeks to position ideas and provoke early thoughts toward advancing neuro-symbolic artificial intelligence in the context of geospatial challenges. Specifically, it conceptualizes and elaborates on an architecture for infusing geospatial knowledge from knowledge graph in a deep neural network pipeline. As guiding case studies - land-use land-cover classification, object detection and instance segmentation can benefit from infusing spatio-contextual information with remote sensing imagery. The discussion further reflects on and articulates the challenges and explainable AI opportunities anticipated when scaling and maintaining large-scale geospatial knowledge graphs.

Potnis, Abhishek↗
NASA NTRS1977

Agricultural scene understanding

The author has identified the following significant results. The LACIE field measurement data were radiometrically calibrated. Calibration enabled valid comparisons of measurements from different dates, sensors, and/or locations. Thermal band canopy results included: (1) Wind velocity had a significant influence on the overhead radiance temperature and the effect was quantized. Biomass and soil temperatures, temperature gradient, and canopy geometry were altered. (2) Temperature gradient was a function of wind velocity. (3) Temperature gradient of the wheat canopy was relatively constant during the day. (4) The laser technique provided good quality geometric characterization.

Landgrebe, D. A.↗
NASA NTRS1979

Computer image processing and recognition

A systematic introduction to the concepts and techniques of computer image processing and recognition is presented. Consideration is given to such topics as image formation and perception; computer representation of images; image enhancement and restoration; reconstruction from projections; digital television, encoding, and data compression; scene understanding; scene matching and recognition; and processing techniques for linear systems.

Hall, E. L.↗
NASA NTRS1988

Improved canopy reflectance modeling and scene inference through improved understanding of scene pattern

The Li-Strahler reflectance model, driven by LANDSAT Thematic Mapper (TM) data, provided regional estimates of tree size and density within 20 percent of sampled values in two bioclimatic zones in West Africa. This model exploits tree geometry in an inversion technique to predict average tree size and density from reflectance data using a few simple parameters measured in the field (spatial pattern, shape, and size distribution of trees) and in the imagery (spectral signatures of scene components). Trees are treated as simply shaped objects, and multispectral reflectance of a pixel is assumed to be related only to the proportions of tree crown, shadow, and understory in the pixel. These, in turn, are a direct function of the number and size of trees, the solar illumination angle, and the spectral signatures of crown, shadow and understory. Given the variance in reflectance from pixel to pixel within a homogeneous area of woodland, caused by the variation in the number and size of trees, the model can be inverted to give estimates of average tree size and density. Because the inversion is sensitive to correct determination of component signatures, predictions are not accurate for small areas.

Franklin, Janet↗
DOE OSTI2024

So you think you can track?

This work introduces a multi-camera tracking dataset consisting of 234 hours of video data recorded concurrently from 234 overlapping HD cameras covering a 4.2 mile stretch of 8-10 lane interstate highway near Nashville, TN. Video is recorded in cooperation with Tennessee State Department of Transportation and its policies. The video is recorded during a period of high traffic density with 500+ objects typically visible within the scene and typical object longevities of 3-15 minutes. GPS trajectories from 270 vehicle passes through the scene are manually corrected in the video data to provide a set of ground-truth trajectories for recall-oriented tracking metrics, and object detections are provided for each camera in the scene (159 million total before cross-camera fusion). Initial benchmarking of tracking-by-detection algorithms is performed against the GPS trajectories, and a best HOTA of only 9.5% is obtained (best recall 75.9% at IOU 0.1, 47.9 average IDs per ground truth object), indicating the benchmarked trackers do not perform sufficiently well at the long temporal and spatial durations required for traffic scene understanding. Video data, scene information, and vehicle trajectories are made publicly available at i24motion.org.

42 ENGINEERING↗
NASA NTRS1984

Comprehensive Understanding for Vegetated Scene Radiance Relationships

The improvement of our fundamental understanding of the dynamics of directional scattering properties of vegetation canopies through analysis of field data and model simulation data is discussed. Directional reflectance distributions spanning the entire existance hemisphere were measured in two field studies; one using a Mark III 3-band radiometer and one using rapid scanning bidirectional field instrument called PARABOLA. Surfaces measured included corn, soybeans, bare soils, grass lawn, orchard grass, alfalfa, cotton row crops, plowed field, annual grassland, stipa grass, hard wheat, salt plain shrubland, and irrigated wheat. Some structural and optical measurements were taken. Field data show unique reflectance distributions ranging from bare soil to complete vegetation canopies. Physical mechanisms causing these trends are proposed based on scattering properties of soil and vegetation. Soil exhibited a strong backscattering peak toward the Sun. Complete vegetation exhibited a bowl distribution with the minimum reflectance near nadir. Incomplete vegetation canopies show shifting of the minimum reflectance off of nadir in the forward scattering direction because both the scattering properties or the vegetation and soil are observed.

Kimes, D. S.↗
NASA NTRS2018

Online Self-Supervised Long-Range Scene Segmentation for MAVs

Recently, there have been numerous advances in the development of payload and power constrained lightweight Micro Aerial Vehicles (MAVs). As these robots aspire for high-speed autonomous flights in complex dynamic environments, robust scene understanding at long-range becomes critical. The problem is heavily characterized by either the limitations imposed by sensor capabilities for geometry-based methods, or the need for large-amounts of manually annotated training data required by data-driven methods. This motivates the need to build systems that have the capability to alleviate these problems by exploiting the complimentary strengths of both geometry and data-driven methods. In this paper, we take a step in this direction and propose a generic framework for adaptive scene segmentation using self-supervised online learning. We present this in the context of vision-based autonomous MAV flight, and demonstrate the efficacy of our proposed system through extensive experiments on benchmark datasets and real world field tests.

Matthies, Larry↗
DOE OSTI2021

2021 Smoky Mountains Conference Data Challenge Synthetic-to-Real Domain Adaptation for Autonomous Driving Dataset

The dataset is comprised of both real and synthetic images from a vehicle's forward-facing camera. Each camera image is accompanied by a corresponding pixel-level semantic segmentation image (all files are .png files). In total, the dataset contains 5600 images in the training/validation set and 1400 images in the testing set. The training dataset contains mostly synthetic RGB images collected with a wide range of weather and lighting conditions using the CARLA simulator [1]. In addition, the training data also includes a small pre-selected subset of data from the Cityscapes training dataset – which is comprised of RGB-segmentation image pairs from driving scenarios in various European cities [2]. The testing data is split into three sets. The first set contains synthetic CARLA images with weather/lighting conditions that were not present in the training set. The second set is a subset of the Cityscapes testing dataset. Finally, the third set is an unknown testing set which will not be revealed to the participants until after the submission deadline. [1] Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, October). CARLA: An open urban driving simulator. In Conference on robot learning (pp. 1-16). PMLR. [2] Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., ... and Schiele, B. (2016). The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3213-3223).

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS1980

Analytical design of multispectral sensors

An optimal design based on the criterion of minimum mean square representation error using the Karhunen-Loeve expansion was developed to represent the spectral response functions from a stratum based upon a stochastic process scene model. From the overall pattern recognition system perspective, the effect of the representation accuracy on a typical performance criterion (the probability of correct classification) is investigated. The optimum sensor design provides a standard against which practical (suboptimum) operational sensors can be compared. An example design is provided and its performance is illustrated. Although developed primarily for the purpose of sensor design, the procedure has potential for making important contributions to scene understanding. Spectral channels which have narrow bandwidths relative to current sensor systems may be necessary to provide adequate spectral representation and improved classification performance.

Wiersma, D. J.↗
NASA NTRS1986

Optical processing and Space Station automation

Optical information processing research aimed at Space Station automation applications is reviewed. The emphasis of the NASA Ames Research Center program is on intelligent optical pattern recognition and optical control processing. Attention is given to the primary functions of an overall scene understanding system: distortion-invariant optical feature generation, dimensionality reduction, object classification, and contextual information processing. A method of using synthetic discriminant functions to facilitate learning in a high-speed optical correlator is described. A discussion is presented of candidate analog and digital architectures for the optical implementation of state-estimation algorithms needed for the control of high-dimension dynamic systems. The multivariate system chosen for the optical control technology demonstration - a segmented, adaptive mirror and interferometrically based wavefront sensor - is also described.

Ennis, D. J.↗
NASA NTRS1992

Encoding spatial images: A fuzzy set theory approach

As the use of fuzzy set theory continues to grow, there is an increased need for methodologies and formalisms to manipulate obtained fuzzy subsets. Concepts involving relative position of fuzzy patterns are acknowledged as being of high importance in many areas. In this paper, we present an approach based on the concept of dominance in fuzzy set theory for modelling relative positions among fuzzy subsets of a plane. In particular, we define the following spatial relations: to the left (right), in front of, behind, above, below, near, far from, and touching. This concept has been implemented to define spatial relationships among fuzzy subsets of the image plane. Spatial relationships based on fuzzy set theory, coupled with a fuzzy segmentation, should therefore yield realistic results in scene understanding.

Sztandera, Leszek M.↗
NASA NTRS1990

Knowledge/geometry-based Mobile Autonomous Robot Simulator (KMARS)

Ongoing applied research is focused on developing guidance system for robot vehicles. Problems facing the basic research needed to support this development (e.g., scene understanding, real-time vision processing, etc.) are major impediments to progress. Due to the complexity and the unpredictable nature of a vehicle's area of operation, more advanced vehicle control systems must be able to learn about obstacles within the range of its sensor(s). A better understanding of the basic exploration process is needed to provide critical support to developers of both sensor systems and intelligent control systems which can be used in a wide spectrum of autonomous vehicles. Elcee Computek, Inc. has been working under contract to the Flight Dynamics Laboratory, Wright Research and Development Center, Wright-Patterson AFB, Ohio to develop a Knowledge/Geometry-based Mobile Autonomous Robot Simulator (KMARS). KMARS has two parts: a geometry base and a knowledge base. The knowledge base part of the system employs the expert-system shell CLIPS ('C' Language Integrated Production System) and necessary rules that control both the vehicle's use of an obstacle detecting sensor and the overall exploration process. The initial phase project has focused on the simulation of a point robot vehicle operating in a 2D environment.

Cheng, Linfu↗
NASA NTRS1984

Comprehensive Understanding for Vegetated Scene Radiance Relationships

Directional reflectance distributions spanning the entire existent hemisphere were measured in two field studies; one using a Mark III 3-band radiometer and one using the rapid scanning bidirectional field instrument called PARABOLA. Surfaces measured included corn, soybeans, bare soils, grass lawn, orchard grass, alfalfa, cotton row crops, plowed field, annual grassland, stipa grass, hard wheat, salt plain shrubland, and irrigated wheat. Analysis of field data showed unique reflectance distributions ranging from bare soil to complete vegetation canopies. Physical mechanisms causing these trends were proposed. A 3-D model was developed and is unique in that it predicts: (1) the directional spectral reflectance factors as a function of the sensor's azimuth and zenith angles and the sensor's position above the canopy; (2) the spectral absorption as a function of location within the scene; and (3) the directional spectral radiance as a function of the sensor's location within the scene. Initial verification of the model as applied to a soybean row crop showed that the simulated directional data corresponded relatively well in gross trends to the measured data. The model was expanded to include the anisotropic scattering properties of leaves as a function of the leaf orientation distribution in both the zenith and azimuth angle modes.

Kimes, D. S.↗
DOE OSTI2022

Tackling the Challenges in Scene Graph Generation With Local-to-Global Interactions

In this work, we seek new insights into the underlying challenges of the scene graph generation (SGG) task. Quantitative and qualitative analysis of the visual genome (VG) dataset implies: 1) ambiguity: even if interobject relationship contains the same object (or predicate), they may not be visually or semantically similar; 2) asymmetry: despite the nature of the relationship that embodied the direction, it was not well addressed in previous studies; and 3) higher-order contexts: leveraging the identities of certain graph elements can help generate accurate scene graphs. Motivated by the analysis, we design a novel SGG framework, Local-to-global interaction networks (LOGINs). Locally, interactions extract the essence between three instances of subject, object, and background, while baking direction awareness into the network by explicitly constraining the input order of subject and object. Globally, interactions encode the contexts between every graph component (i.e., nodes and edges). Finally, Attract and Repel loss is utilized to fine-tune the distribution of predicate embeddings. By design, our framework enables predicting the scene graph in a bottom-up manner, leveraging the possible complementariness. To quantify how much LOGIN is aware of relational direction, a new diagnostic task called Bidirectional Relationship Classification (BRC) is also proposed. Overall, experimental results demonstrate that LOGIN can successfully distinguish relational direction than existing methods (in BRC task), while showing state-of-the-art results on the VG benchmark (in SGG task).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Role of depth in optical diffractive neural networks

Free-space all-optical diffractive neural networks have emerged as promising systems for neuromorphic scene classification. Understanding the fundamental properties of these systems is important to establish their ultimate performance. Here we consider the case of diffraction by subwavelength apertures and study the behavior of the system as a function of the number of diffractive layers by employing a co-design modeling approach. We show that adding depth allows the system to achieve high classification accuracies with a reduced number of diffractive features compared to a single layer, but that it does not allow the system to surpass the performance of an optimized single layer. The improvement from depth is found to be limited to the first few layers. These properties originate from the constraints imposed by the physics of light, in particular the weakening electric field with distance from the aperture.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS1982

An overview of computer vision

An overview of computer vision is provided. Image understanding and scene analysis are emphasized, and pertinent aspects of pattern recognition are treated. The basic approach to computer vision systems, the techniques utilized, applications, the current existing systems and state-of-the-art issues and research requirements, who is doing it and who is funding it, and future trends and expectations are reviewed.

Gevarter, W. B.↗