Search NASA⌕ Search

SEARCH · Search NASA

Results for “perceptual losses”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

11 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Method of Ground-Based Cloud Motion Predict: CCLSTM + SR-Net

Ground-based cloud images can provide information on weather and cloud conditions, which play an important role in cloud cover monitoring and photovoltaic power generation forecasting. However, the cloud motion prediction of ground-based cloud images still lacks advanced and complete methods, and traditional technologies based on image processing and motion vector calculation are difficult to predict cloud morphological changes. In this paper, we propose a cloud motion prediction method based on Cascade Causal Long Short-Term Memory (CCLSTM) and Super-Resolution Network (SR-Net). Firstly, CCLSTM is used to estimate the shape and speed of cloud motion. Secondly, the Super-Resolution Network is built based on perceptual losses to reconstruct the result of CCLSTM and, finally, make it clearer. We tested our method on Atmospheric Radiation Measurement (ARM) Climate Research Facility TSI (total sky imager) images. The experiments showed that the method is able to predict the sky cloud changes in the next few steps.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

ESM data downscaling: a comparison of super-resolution deep learning models

Abstract Climate projections at fine spatial resolutions are required to conduct accurate risk assessment for critical infrastructure and design adaptation planning. Generating these projections using advanced Earth system models (ESM) requires significant computational resources. To address this issue, various statistical downscaling techniques have been introduced to generate fine-resolution data from coarse-resolution simulations. In this study, we evaluate and compare five deep learning-based downscaling techniques, namely, super-resolution convolutional neural networks, fast super-resolution convolutional neural network ESM, efficient sub-pixel convolutional neural network, enhanced deep residual network (EDRN), and super-resolution generative adversarial network (SRGAN). These techniques are applied to a dataset generated by the Energy Exascale Earth System Model (E3SM), focusing on key surface variables such as surface temperature, shortwave heat flux, and longwave heat flux. Models are trained and validated using paired fine-resolution (0.25 $$^{\circ }$$ ∘ ) and coarse-resolution (1 $$^{\circ }$$ ∘ ) monthly data obtained from a 9-year simulation. Next, blind testing is performed using monthly data obtained from two different years outside of the training and validation set. To evaluate the efficiency of each technique, different statistical metrics are used, including mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The results show that EDRN outperforms other algorithms in terms of PSNR, SSIM, and MSE, but struggles to capture fine-scale features in the data. In contrast, SRGAN, a generative model that uses perceptual loss, excels in capturing fine details at boundaries and internal structures, resulting in lower LPIPS than other methods.

Pawar, Nikhil M. (ORCID:0000000211613289)↗
NASA NTRS2023

Neural-Based Compression Scheme for Solar Image Data

Studying the solar system and especially the Sun relies on the data gathered daily from space missions. These missions are data-intensive and compressing this data to make them efficiently transferable to the ground station is a twofold decision to make. Stronger compression methods, by distorting the data, can increase data throughput at the cost of accuracy which could affect scientific analysis of the data. On the other hand, preserving subtle details in the compressed data requires a high amount of data to be transferred, reducing the desired gains from compression. In this work, we propose a neural network-based lossy compression method to be used in NASA’s data-intensive imagery missions. We chose NASA’s Solar Dynamics Observatory (SDO) mission which transmits 1.4 terabytes of data each day as a proof of concept for the proposed algorithm. In this work, we propose an adversarially trained neural network, equipped with local and non-local attention modules to capture both the local and global structure of the image resulting in a better trade-off in rate-distortion (RD) compared to conventional hand-engineered codecs. The RD variational autoencoder used in this work is jointly trained with a channel-dependent entropy model as a shared prior between the analysis and synthesis transforms to make the entropy coding of the latent code more effective. We also studied how optimizing perceptual losses could help our neural compressor to preserve high-frequency details of the data in the reconstructed compressed image. Our neural image compression algorithm outperforms currently-in-use and state-of-the-art codecs such as JPEG and JPEG-2000 in terms of the RD performance when compressing extreme-ultraviolet (EUV) data. As a proof of concept for use of this algorithm in SDO data analysis, we have performed coronal hole (CH) detection using our compressed images, and generated consistent segmentations, even at a compression rate of ∼ 0.1 bits per pixel (compared to 8 bits per pixel on the original data) using EUV data from SDO.

Image coding↗
NASA NTRS2021

The Role of Tactile Cueing in Multimodal Displays: Application in Complex Task Environments for Space Exploration

This paper reviews the potential role of tactile cueing in multimodal displays that will need to be developed to enable Extravehicular Activity (EVA) and telerobotic exploration for both Moon and Mars missions. Specific missions to be accomplished during EVA surface and space operations will include construction and assembly, surface and geologic exploration, and excavation for protective shelter from radiation and significant solar particle events. EVA crews have limited time, resources, mobility, visibility and dexterity, and activities take place in a perceptually impoverished environment. Limitations include a loss of spatial reference frame leading to spatial disorientation, a loss of information normally provided by the auditory and somato-sensory systems, and a restricted field of view. In addition, in non-surface operations crew must be able to navigate and operate in a fully six degree of freedom (DoF) environment. Fatigue resulting from suited EVA tasks will also critically affect performance and safety. In order to mitigate these factors, it is important that crews be well trained prior to executing actual missions. There is a critical need to design multimodal interfaces to optimize performance under these conditions for both training and actual operations. Because of the high level of complexity of the EVA environment on the one hand, and the “sensory restrictions” on the other, it makes sense to envisage enhancements to the astronauts’ visual environment. Such an enriched environment may include virtual auditory as well as tactile inputs.

multimodal displays↗
NASA NTRS1984

Utilization of Historic Information in an Optimisation Task

One of the basic components of a discrete model of motor behavior and decision making, which describes tracking and supervisory control in unitary terms, is assumed to be a filtering mechanism which is tied to the representational principles of human memory for time-series information. In a series of experiments subjects used the time-series information with certain significant limitations: there is a range-effect; asymmetric distributions seem to be recognized, but it does not seem to be possible to optimize performance based on skewed distributions. Thus there is a transformation of the displayed data between the perceptual system and representation in memory involving a loss of information. This rules out a number of representational principles for time-series information in memory and fits very well into the framework of a comprehensive discrete model for control of complex systems, modelling continuous control (tracking), discrete responses, supervisory behavior and learning.

Boesser, T.↗
DOE OSTI2021

Inpainting radar missing data regions with deep learning

Abstract. Missing and low-quality data regions are a frequent problem for weather radars. They stem from a variety of sources: beam blockage, instrument failure, near-ground blind zones, and many others. Filling in missing data regions is often useful for estimating local atmospheric properties and the application of high-level data processing schemes without the need for preprocessing and error-handling steps – feature detection and tracking, for instance. Interpolation schemes are typically used for this task, though they tend to produce unrealistically spatially smoothed results that are not representative of the atmospheric turbulence and variability that are usually resolved by weather radars. Recently, generative adversarial networks (GANs) have achieved impressive results in the area of photo inpainting. Here, they are demonstrated as a tool for infilling radar missing data regions. These neural networks are capable of extending large-scale cloud and precipitation features that border missing data regions into the regions while hallucinating plausible small-scale variability. In other words, they can inpaint missing data with accurate large-scale features and plausible local small-scale features. This method is demonstrated on a scanning C-band and vertically pointing Ka-band radar that were deployed as part of the Cloud Aerosol and Complex Terrain Interactions (CACTI) field campaign. Three missing data scenarios are explored: infilling low-level blind zones and short outage periods for the Ka-band radar and infilling beam blockage areas for the C-band radar. Two deep-learning-based approaches are tested, a convolutional neural network (CNN) and a GAN that optimize pixel-level error or combined pixel-level error and adversarial loss respectively. Both deep-learning approaches significantly outperform traditional inpainting schemes under several pixel-level and perceptual quality metrics.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2009

Auditory and Vestibular Issues Related to Human Spaceflight

Human spaceflight provides unique opportunities to study human vestibular and auditory systems. This session will discuss 1) vestibular adaptive processes reflected by pronounced perceptual and motor coordination problems during, and after, space missions; 2) vestibular diagnostic and rehabilitative techniques (used to promote recovery after living in altered gravity environments) that may be relevant to treatment of vestibular disorders on earth; and 3) unique acoustical challenges to hearing loss prevention and crew performance during spaceflight missions.

Danielson, Richard W.↗
NASA NTRS1989

Image gathering and restoration - Information and visual quality

A method is investigated for optimizing the end-to-end performance of image gathering and restoration for visual quality. To achieve this objective, one must inevitably confront the problems that the visual quality of restored images depends on perceptual rather than mathematical considerations and that these considerations vary with the target, the application, and the observer. The method adopted in this paper is to optimize image gathering informationally and to restore images interactively to obtain the visually preferred trade-off among fidelity resolution, sharpness, and clarity. The results demonstrate that this method leads to significant improvements in the visual quality obtained by the traditional digital processing methods. These traditional methods allow a significant loss of visual quality to occur because they treat the design of the image-gathering system and the formulation of the image-restoration algorithm as two separate tasks and fail to account for the transformations between the continuous and the discrete representations in image gathering and reconstruction.

Mccormick, Judith A.↗
NASA NTRS1993

Manual control in space research on perceptual-motor functions under zero gravity conditions (L-10)

Are human abilities to control vehicles and other machines the same in space as those on Earth? The L-10 Manual Control Experiment of the First Materials Processing Tests (FMPT) started from this question. Suppose a pilot has the task to align the head of a space vehicle toward a target. His actions are to look at the target, to determine the vehicle movement, and to operate the manipulator. If the activity of the nervous system were the same as on Earth, the movements, of the eye and hand would become excessive because the muscles do not have to oppose gravity. The timing and amount of movement must be arranged for appropriate actions. The sensation of motion would also be affected by the loss of gravity because the mechanism of the otolith, the major acceleration sensor, depends on gravity. The possible instability of the sensation of direction may cause mistakes in the direction of control of manipulator movement. Thus, the experimental data can be used for designing man-machine systems in space, as well as for investigation of physiological mechanisms. In this experiment, the direction of vehicle heading is expressed by a light spot on an array of light emitting diodes and the manipulator is of a finger stick type. As the light spot moves up and down, the Japanese Payload Specialist, and the subject, must move the manipulator forward and backward to keep the movement of the light spot within the neighborhood of the central point of the display. The position of the light spot is computed in such a manner that when the stick is kept at the neutral position, a motion whose acceleration is proportional to the angle of deflection is added to the movement of the light spot. The Operator Describing Function, which is an expression of human control characteristics, can be calculated from 2 minutes of raw data of the light spot position and stick deflection. The 2 minutes of operation is called a run, and 8 runs with resting periods composes a session. The on-orbit experiment will be conducted on the second, fourth, and seventh days. One session of experiment of each of these days is conducted following the L-4 experiment, which uses the same apparatus. The Payload Specialist, aided by a Mission Specialist, will take our apparatus from a rack container, set up the apparatus, attach electrodes for measurements of eye movement and muscle activity, conduct the L-4 Visual Stability Experiment, conduct one session of the manual control experiment, and then dissemble and stow the apparatus. In addition to the flight experiment, pre-flight and post-flight experiments will be conducted. The data of three sessions on orbit will reflect adaption of physiological systems to microgravity. The data of post-flight experiments, on the other hand, will reflect re-adaptation of physiological systems to the gravity condition on the ground. Control data collected with and without psychological tension will be scheduled just prior to and long before launch.

Tada, Akira↗
NASA NTRS1990

A discrepancy within primate spatial vision and its bearing on the definition of edge detection processes in machine vision

The visual perception of form information is considered to be based on the functioning of simple and complex neurons in the primate striate cortex. However, a review of the physiological data on these brain cells cannot be harmonized with either the perceptual spatial frequency performance of primates or the performance which is necessary for form perception in humans. This discrepancy together with recent interest in cortical-like and perceptual-like processing in image coding and machine vision prompted a series of image processing experiments intended to provide some definition of the selection of image operators. The experiments were aimed at determining operators which could be used to detect edges in a computational manner consistent with the visual perception of structure in images. Fundamental issues were the selection of size (peak spatial frequency) and circular versus oriented operators (or some combination). In a previous study, circular difference-of-Gaussian (DOG) operators, with peak spatial frequency responses at about 11 and 33 cyc/deg were found to capture the primary structural information in images. Here larger scale circular DOG operators were explored and led to severe loss of image structure and introduced spatial dislocations (due to blur) in structure which is not consistent with visual perception. Orientation sensitive operators (akin to one class of simple cortical neurons) introduced ambiguities of edge extent regardless of the scale of the operator. For machine vision schemes which are functionally similar to natural vision form perception, two circularly symmetric very high spatial frequency channels appear to be necessary and sufficient for a wide range of natural images. Such a machine vision scheme is most similar to the physiological performance of the primate lateral geniculate nucleus rather than the striate cortex.

Jobson, Daniel J.↗
NASA NTRS2014

Straight Ahead in Microgravity

This joint ESA-NASA study will address adaptive changes in spatial orientation related to the subjective straight ahead, and the use of a vibrotactile sensory aid to reduce perceptual errors. The study will be conducted before and after long-duration expeditions to the International Space Station (ISS) to examine how spatial processing of target location is altered following exposure to microgravity. This project specifically addresses the sensorimotor research gap "What are the changes in sensorimotor function over the course of a mission?" Six ISS crewmembers will be requested to participate in three preflight sessions (between 120 and 60 days prior to launch) and then three postflight sessions on R+0/1 day, R+4 +/-2 days, and R+8 +/-2 days. The three specific aims include: (a) fixation of actual and imagined target locations at different distances; (b) directed eye and arm movements along different spatial reference frames; and (c) the vestibulo-ocular reflex during translation motion with fixation targets at different distances. These measures will be compared between upright and tilted conditions. Measures will then be compared with and without a vibrotactile sensory aid that indicates how far one has tilted relative to the straight-ahead direction. The flight study was been approved by the medical review boards and will be implemented in the upcoming Informed Crew Briefings to solicit flight subject participation. Preliminary data has been recorded on 6 subjects during parabolic flight to examine the spatial coding of eye movements during roll tilt relative to perceived orientations while free-floating during the microgravity phase of parabolic flight or during head tilt in normal gravity. Binocular videographic recordings obtained in darkness allowed us to quantify the mean deviations in gaze trajectories along both horizontal and vertical coordinates relative to the aircraft and head orientations. During some parabolas, a vibrotactile sensory aid provided feedback of body orientation relative to the plane coordinates. RESULTS Both variability and curvature of gaze trajectories increased during roll tilt compared to the upright position. The saccades were less accurate during parabolic flight compared to measurements obtained in normal gravity. Although subjects were instructed to look off in the distance while performing the eye movements, fixation distance varied with vertical gaze direction independent of whether the saccades were made along perceived aircraft or head orientations. The increased errors in gaze trajectories along both perceived orientations during microgravity can be attributed to the otolith's role in spatial coding of eye movements. A change in an individual's egocentric reference might have negative consequences on evaluating the direction of an approaching object or on the accuracy of reaching movements or locomotion. Consequently, investigating how microgravity affects the target location will have theoretical, operational and even clinical implications for future space exploration missions. The use of vibrotactile feedback as a sensorimotor countermeasure is applicable to balance therapy applications for vestibular loss patients and the elderly to mitigate risks due to loss of orientation.

Wood, S. J.↗