Search NASA⌕ Search

SEARCH · Search NASA

Results for “Visual Perception”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
NASA NTRS2001

Spatial orientation and balance control changes induced by altered gravitoinertial force vectors

To better understand the mechanisms of human adaptation to rotating environments, we exposed 19 healthy subjects and 8 vestibular-deficient subjects ("abnormal"; four bilateral and four unilateral lesions) to an interaural centripetal acceleration of 1 g (resultant 45 degrees roll-tilt of 1.4 g) on a 0.8-m-radius centrifuge for periods of 90 min. The subjects sat upright (body z-axis parallel to centrifuge rotation axis) in the dark with head stationary, except during 4 min of every 10 min, when they performed head saccades toward visual targets switched on at 3- to 5-s intervals at random locations (within +/- 30 degrees) in the earth-horizontal plane. Eight of the normal subjects also performed the head saccade protocol in a stationary chair adjusted to a static roll-tilt angle of 45 degrees for 90 min (reproducing the change in orientation but not the magnitude of the gravitoinertial force on the centrifuge). Eye movements, including voluntary saccades directed along perceived earth- and head-referenced planes, were recorded before, during, and immediately after centrifugation. Postural center of pressure (COP) and multisegment body kinematics were also gathered before and within 10 min after centrifugation. Normal subjects overestimated roll-tilt during centrifugation and revealed errors in perception of head-vertical provided by directed saccades. Errors in this perceptual response tended to increase with time and became significant after approximately 30 min. Motion-sickness symptoms caused approximately 25% of normal subjects to limit their head movements during centrifugation and led three normal subjects to stop the test early. Immediately after centrifugation, subjects reported feeling tilted 10 degrees in the opposite direction, which was in agreement with the direction of their earth-referenced directed saccades. Postural COP, segmental body motion amplitude, and hip-sway frequency increased significantly after centrifugation. These postural effects were short-lived, however, with a recovery time of several postural test trials (minutes). There were also asymmetries in the direction of postcentrifugation COP and head tilt which depended on the subject's orientation during the centrifugation adaptation period (left ear or right ear out). The amount of total head movements during centrifugation correlated poorly or inversely with postcentrifugation postural stability, and the most unstable subject made no head movements. There was no decrease in postural stability after static tilt, although these subjects also reported a perceived tilt briefly after return to upright, and they also had COP asymmetries. Abnormal subjects underestimated roll-tilt during centrifugation, and their directed saccades revealed permanent spatial distortions. Bilateral abnormal subjects started out with poor postural control, but showed no postural decrements after centrifugation, while unilateral abnormal subjects had varying degrees of postural decrement, both in their everyday function and as a result of experiencing the centrifugation. In addition, three unilateral, abnormal subjects, who rode twice in opposite orientations, revealed a consistent orthogonal pattern of COP offsets after centrifugation. These results suggest that both orientation and magnitude of the gravitoinertial vector are used by the central nervous system for calibration of multiple orientation systems. A change in the background gravitoinertial force (otolith input) can rapidly initiate postural and perceptual adaptation in several sensorimotor systems, independent of a structured visual surround.

Clinical Trial↗
NASA NTRS2021

Multi-Resolution Elevation Mapping and Safe Landing Site Detection with Applications to Planetary Rotorcraft

In this paper, we propose a resource-efficient approach to provide an autonomous UAV with an on-board perception method to detect safe, hazard-free landing sites during flights over complex 3D terrain. We aggregate 3D measurements acquired from a sequence of monocular images by a Structure-from-Motion approach into a local, robot-centric, multi-resolution elevation map of the overflown terrain, which fuses depth measurements according to their lateral surface resolution (pixel-footprint) in a probabilistic framework based on the concept of dynamic Level of Detail. Map aggregation only requires depth maps and the associated poses, which are obtained from an on-board Visual Odometry algorithm. An efficient landing site detection method then exploits the features of the underlying multi-resolution map to detect safe landing sites based on the slope, roughness, and quality of the reconstructed terrain surface. The evaluation of the performance of the mapping and landing site detection modules are analyzed independently and jointly in simulated and realworld experiments in order to establish the efficacy of the proposed approach.

Brockers, Roland↗
NASA NTRS2021

Dense 3D-Reconstruction from Monocular Image Sequences for Computationally Constrained UAS

The ability to find safe landing sites over complex 3D terrain is an essential safety feature for fully autonomous small unmanned aerial systems (UAS), which requires on-board perception for 3D reconstruction and terrain analysis if the overflown terrain is unknown. This is a challenge for UAS that are limited in size, weight and computational power, such as small rotorcrafts executing autonomous missions on Earth, or in planetary applications such as the Mars Helicopter. For such a computationally constraint system, we propose a structure from motion approach that uses inputs from a single downward facing camera to produce dense point clouds of the overflown terrain in real time. In contrast to existing approaches, our method uses metric pose information from a visual-inertial odometry algorithm as camera pose priors, which allows deploying a fast pose refinement step to align camera frames such that a conventional stereo algorithm can be used for dense 3D reconstruction. We validate the performance of our approach with extensive evaluations in simulation, and demonstrate the feasibility with data from UAS flights.

Brockers, Roland↗
NASA NTRS1999

Yaw and pitch visual-vestibular interaction in weightlessness

Both yaw and pitch visual-vestibular interactions at two separate frequencies of chair rotation (0.2 and 0.8 Hz) in combination with a single velocity of optokinetic stimulus (36 degrees/s) were used to investigate the effects of sustained weightlessness on neural strategies adopted by astronaut subjects to cope with the stimulus rearrangement of spaceflight. Pitch and yaw oscillation in darkness at 0.2 and 0.8 Hz without optokinetic stimulation, and constant velocity linear optokinetic stimulation at 18, 36, and 54 degrees/s presented relative to the head with the subject stationary, were used as controls for the visual-vestibular interactions. The results following 8 days of space flight showed no significant changes in: (1) either the horizontal and vertical vestibulo-ocular reflex (VOR) gain, phase, or bias; (2) the yaw visual-vestibular response (VVR); or (3) the horizontal or vertical optokinetic (OKN) slow phase velocity (SPV). However, significant changes were observed: (1) when during pitch VVR at 0.2 Hz late inflight, the contribution of the optokinetic input to the combined oculomotor response was smaller than during the stationary OKN SPV measurements, followed by an increased contribution during the immediate postflight testing; and (2) when during pitch VVR at 0.8 Hz, the component of the combined oculomotor response due to the underlying vertical VOR was more efficiently suppressed early inflight and less suppressed immediately postflight compared with preflight observations. The larger OKN response during pitch VVR at 0.2 Hz and the better suppression of VOR during pitch VVR at 0.8 Hz postflight are presumably due to the increased role of vision early inflight and immediately after spaceflight, as previously observed in various studies. These results suggest that the subjects adopted a neural strategy to structure their spatial orientation in weightlessness by reweighting visual, otolith, and perhaps tactile/somatic signals.

NASA Center JSC↗
NASA NTRS1984

Rheoceptive mediators of graviperception in a water flea: Morphological implications of antennal-socket setae in daphnia magna

Aquatic microcrustaceans of the genus Daphnia are known to orient to light during the day. At night, in the absence of visual cues, daphnids were suspected of maintaining equilibrium by monitoring the direction of gravity through their swimming antennae. Recent investigations using simulated, weightlessness conditions coupled with absence of illumination revealed hair like structures or setae on the basal, articulating socket of the antennae that, when surgically removed, resulted in disorientation. Given the simulated weightlessness or neutrally buoyant condition that eliminated sinking of the normally negatively buoyant Daphnia, it was proposed that the antennal socket setae function as rheoceptors stimulated by the upward rush of water currents during gravity induced, sinking phase of daphnid swimming movements. This rheoceptively mediated, gravity perception hypothesis is further supported by morphological investigations. Scanning electron micrographs indicate that antennal socket setae are anatomically similar to proprioceptors used by higher crustaceans to monitor gravitational direction.

Meyers, D. G.↗
NASA NTRS1991

Visually guided control of movement in the context of multimodal stimulation

Flight simulation has been almost exclusively concerned with simulating the motions of the aircraft. Physically distinct subsystems are often combined to simulate the varieties of aircraft motion. Visual display systems simulate the motion of the aircraft relative to remote objects and surfaces (e.g., other aircraft and the terrain). 'Motion platform' simulators recreate aircraft motion relative to the gravitoinertial vector (i.e., correlated rotation and tilt as opposed to the 'coordinated turn' in flight). 'Control loaders' attempt to simulate the resistance of the aerodynamic medium to aircraft motion. However, there are few operational systems that attempt to simulate the motion of the pilot relative to the aircraft and the gravitoinertial vector. The design and use of all simulators is limited by poor understanding of postural control in the aircraft and its effect on the perception and control of flight. Analysis of the perception and control of flight (real or simulated) must consider that: (1) the pilot is not rigidly attached to the aircraft; and (2) the pilot actively monitors and adjusts body orientation and configuration in the aircraft. It is argued that this more complete approach to flight simulation requires that multimodal perception be considered as the rule rather than the exception. Moreover, the necessity of multimodal perception is revealed by emphasizing the complementarity rather than the redundancy among perceptual systems. Finally, an outline is presented for an experiment to be conducted at NASA ARC. The experiment explicitly considers possible consequences of coordination between postural and vehicular control.

Riccio, Gary E.↗
NASA NTRS1972

Visual light flash phenomenon, part C

The Apollo 16 mission provided the opportunity to obtain additional data on the characteristics and frequency of the light flashes and also provided the first opportunity to obtain a direct physical record of incident cosmic ray particles with the Apollo light flash moving emulsion detector (ALFMED). The ALFMED is an electromechanical device that is worn on the head like a helmet and supports cosmic radiation-sensitive emulsions around the head of the test subject. Two light flash observation sessions were conducted during the mission: one during translunar coast and the second during transearth coast. Characteristics of the light flashes observed and reported by the Apollo crew members were generally similar to those reported on previous missions. Analyses of the ALFMED emulsion plates are proceeding as scheduled, although results are not yet available. The ALFMED results should provide conclusive evidence establishing the correlation, if any, between the incident cosmic ray particles and the perception of light flashes as reported by Apollo crew members.

Richard E. Benson↗
NASA NTRS1995

Human Performance Issues of Lunar-Sited Teleoperations

Teleoperations in terrestrial environments present a number of challenges to system operators and designers. Transmission lags, restricted visual fields, and reduced or distorted tactile/kinesthetic feedback can compromise performance, especially for innerloop control tasks (e.g., vehicle or manipulator control). These problems are likely to be exacerbated in lunar operations, since teleoperation may occur across large distances. Further, the lunar environment will introduce unique concerns. For example, the teleoperated systems will reflect the reduced gravity of the moon. In addition to the novelty of these dynamics, operators will often have to cope with them while physically located in a terrestrial or microgravity environment. Similarly, the optical characteristics of the lunar environment differ from our usual experience (e.g., lack of atmospheric attenuation) in ways that may impact normative depth, distance, and motion perception. These human factors issues are related to the question of humans adapting to a lunar environment. However, teleoperations requires the operator to maintain functionality in both the control station and end-effector environments, defeating more straightforward environmental adaptation strategies.

Kaiser, Mary K.↗
NASA NTRS1991

Percepts of rigid motion within and across apertures

Humans consistently err in their percepts of rotational motion viewed through an aperture. Such errors provide insight into the constraints observers use to interpret retinal images. In the first of two experiments, Ss consistently perceived the fixed center of rotation for an unmarked line viewed through an aperture as located on the line, regardless of its actual location. Accuracy greatly improved with visible line endings. This finding was extended to explain why a square appears nonrigid when it rotates behind a partial occluder. This illusion may result from observers misperceiving the center of rotation of the unmarked square sides. In this situation, Ss seemed unable to apply an object rigidity constraint across apertures. These findings support a conceptualization of the visual system in which consistent local information must be clearly present before prior knowledge can be used to interpret retinal stimulation.

Shiffrar, Maggie↗
NASA NTRS1985

Model of human visual-motion sensing

A model of how humans sense the velocity of moving images is proposed. The model exploits constraints provided by human psychophysics, notably that motion-sensing elements appear tuned for two-dimensional spatial frequency, and by the frequency spectrum of a moving image, namely, that its support lies in the plane in which the temporal frequency equals the dot product of the spatial frequency and the image velocity. The first stage of the model is a set of spatial-frequency-tuned, direction-selective linear sensors. The temporal frequency of the response of each sensor is shown to encode the component of the image velocity in the sensor direction. At the second stage, these components are resolved in order to measure the velocity of image motion at each of a number of spatial locations and spatial frequencies. The model has been applied to several illustrative examples, including apparent motion, coherent gratings, and natural image sequences. The model agrees qualitatively with human perception.

Watson, A. B.↗
NASA NTRS2020

Evaluation of Low Cost, User-Centered Alerting Devices for the Mitigation of Flight Crew Spatial Disorientation

The National Aeronautics and Space Administration (NASA) is conducting research into technologies which have the potential to reduce flight crew Spatial Disorientation (SD). While flight deck technology has advanced rapidly over the past fifty (50) years, the reported occurrences of flight crew SD have not decreased. The Cost-Effective Devices for Alerting Research (CEDAR) effort has focused on the identification and development of low cost, user-centered alerting solutions for the purpose of mitigating the occurrence of flight crew SD. This effort seeks to evaluate both existing technologies as well as new and emerging technologies which have a viable path to implementation in commercial aviation within the next five (5) years. This research is intended to develop a proof-of-concept for real-time SD mitigation which could eventually be utilized to improve safety in future air transport operations. NASA has partnered with the United States Naval Aeromedical Research Unit in Dayton, OH (NAMRU-D) to conduct flight crew SD research utilizing their Disorientation Research Device (DRD), dubbed “The Kraken”. This high-tech simulator features never-before-seen capabilities for side-by-side commercial aviation flight crew SD research, and is capable of re-creating the forces necessary to induce SD illusions in a safe and repeatable manner. Prototype SD mitigation solutions were incorporated into this simulator for scenario-based testing and evaluation using airline pilots within a contextually representative operational environment. SD mitigation technology prototypes designed to provide haptic feedback for both alerting and guidance, as well as innovative visual and aural alerting displays were evaluated. In December 2014, as a result of analyzing eighteen (18) loss-of-control events, the Commercial Aviation Safety Team (CAST) recommended research into flight deck technologies that have potential to mitigate the problems and contributing factors that lead to flight crew loss of airplane state awareness (ASA) and conditions likely to produce SD. The aviation community (government, industry and academia) have been charged with conducting research into cost-effective, user-centered flight deck alerting systems to alert flight crews; especially for the two conditions that produced SD (sub-threshold rolls and the somatogravic illusion). Somatogravic illusions are defined as illusions in which “there is a false perception of attitude on exposure to a force vector that differs in direction and/or magnitude from the normal gravitational force”. The sub-threshold roll illusion (aka the “leans” illusion) is a false sensation of roll attitude. A prolonged roll of less than two degrees becomes physiologically imperceptible to the pilot, and can lead to an incorrect perception of aircraft orientation. The sub-threshold roll illusion forms the entry point for the “graveyard spiral”, a maneuver in which the pilot’s corrective action (based on their false perception of aircraft orientation) causes the aircraft to spiral down into the ground. In both of these SD illusions, it is when the pilot attempts to correct the aircraft’s attitude (based on a false perception of its state) that problems arise, sometimes with lethal consequences.

Ronald J Daiker↗
NASA NTRS1991

Percepts of rigid motion within and across apertures

Humans consistently err in their percepts of rotational motion viewed through an aperture. Such errors provide insight into the constraints observers use to interpret retinal images. In the first of two experiments, the subjects consistently perceived the fixed center of rotation for an unmarked line viewed through an aperture as located on the line, regardless of its actual location. Accuracy greatly improved with visible line endings. This finding was extended to explain why a square appears nonrigid when it rotates behind a partial occluder. This illusion is theorized to result from observers misperceiving the center of rotation of the unmarked square sides. In this situation, the subjects seemed unable to apply an object rigidity constraint across apertures. These findings support a conceptualization of the visual system in which consistent local information must be clearly present before prior knowledge can be used to interpret retinal stimulation.

Shiffrar, Maggie↗
NASA NTRS1996

Eye Movements Reveal Hierarchical Motion Processing

Purpose: In the analysis of visual motion, local features such as orientation are analyzed early in the cortical processing stream (V1), while integration across orientation and space is thought to occur in higher cortical areas such as MT, MST, etc. If all areas provide inputs to eye movement control centers, we would expect that local properties would drive eye movements with relatively short latencies, while global properties would require longer latencies. When such latencies are observed, they can provide information about when (and where?) various stimulus properties are analyzed. Methods: The stimulus employed was an elliptical Gabor patch with a drifting carrier, in which the orientations of the carrier grating and the contrast window were varied independently. We have previously demonstrated that the directional percepts evoked by this stimulus vary between the "grating direction" (the normal to the grating's orientation) and the "window direction", and that similar effects can be observed in reflexive eye movements. Subjects viewed such a stimulus while attempting to maintain steady fixation on the center of the pattern, and the small reflexive eye movements ("stare OKN") were recorded. In the middle of the trial, the orientation of either the grating or the window was rotated smoothly by 30 degrees. Results: Responses to the shift of both grating orientation and window orientation are seen in the average OKN slow phase velocity. Grating rotations produce a rapid OKN rotation to the grating direction (100 ms latency, 300 ms time constant), followed by a slower rebound to the steady state perceived direction midway between the grating and window directions. Window rotations, on the other hand, evoke a slower response (200 ms latency, 500 ms time constant). Conclusions: The results demonstrate multiple cortical inputs to eye movement control: a fast, early input driven by orientation, and a slower input from higher areas sensitive to global stimulus properties.

Mulligan, Jeffrey B.↗
DOE OSTI2026

Comparative Analysis of Report-Back of Research Results Strategies for Personal Chemical Exposure Data

Background. Report-back of research results (RBRR) is ethically supported and highly requested by participants yet lacks broadly transferable guidelines for RBRR. Effective RBRR must be responsive to target audience needs and may not be addressed by a ‘one-size-fits-all’ approach. Objective. Within a subset of our 19 studies on RBRR, we had the unique opportunity to carry out a comparative analysis of RBRR strategies across cohorts with similar development and evaluation methods, yet distinct in life stage, geography, number and type of chemicals assessed, and community contexts. Methods. We highlight key outcomes from three environmental health studies: an ongoing New York, NY cohort (Fair Start; n=486) and a Detroit, MI cohort (CLEAR; n=34) assessing exposure to ambient urban pollution during pregnancy, and a longitudinal cohort in Houston, TX (Houston-3H) following Hurricane Harvey (n=312). Focus group and survey data were analyzed to identify lessons learned and explore how RBRR supports understanding of environmental health. Results. Commonalities emerged in RBRR development, design, organization, and data visualization, as well as in how RBRR can contribute to an understanding of health-environment connections. Differences included preferences for individual versus community level findings, as well as distinguishable contextual considerations. For pregnancy cohorts, messaging was framed with cultural sensitivity, and to avoid unintended consequences of parental guilt due to prenatal exposures. In the post-disaster Houston-3H study, participants requested additional transparency regarding sampling design and study rationale. Significance. All RBRR case studies reported chemicals without known regulatory or health guidelines, so results were contextualized within the study population. Participants across cohorts requested multi-study comparisons to better understand their results beyond their communities. While foundational RBRR elements (e.g. plain language, graphic organizers) may supersede cohort-specific differences, RBRR should be personalized to encompass perceptions of health across different life-stage, cultural, and environmental contexts.

Vogel, Taylor J.↗
NASA NTRS1997

Eye Movement Latencies to Direction Change for Different Classes of Motion

In the analysis of visual motion, local features such as orientation are analyzed early in the cortical processing stream (V1), while integration across orientation and space is thought to occur in higher cortical areas such as MT, MST, etc. If all areas provide inputs to eye movement control centers, we would expect that local properties would drive eye movements with relatively short latencies, while global properties would require longer latencies. When such latencies are observed, they can provide information about when (and where?) various stimulus properties are analyzed. To this end, a stimulus was employed in which local and global properties determining perceived direction-of-motion could be manipulated independently: an elliptical Gabor patch with a drifting carrier, with variable orientation of the carrier grating and the contrast window. We have previously demonstrated that the directional percepts evoked by this stimulus vary between the "grating direction" (the normal to the grating's orientation) and the "window direction" (ARVO 91, 94), and that similar effects can be observed in reflexive eye movements (ARVO 95). Subjects viewed such a stimulus while attempting to maintain steady fixation on the center of the pattern, and the small reflexive eye movements ("stare OKN") were recorded. In the middle of the trial, the orientation of either the grating or the window was rotated smoothly by 30 degrees. Responses to the shift of both grating orientation and window orientation are seen in the average OKN slow phase velocity. Grating rotations produce a rapid OKN rotation to the grating direction (100 ms latency, 300 ms time constant), followed by a slower rebound to the steady state perceived direction midway between the grating and window directions. Window rotations, on the other hand, evoke a slower response (200 ms latency, 500 ms time constant). The results demonstrate multiple cortical inputs to eye movement control: a taste early input driven by orientation, and a slower input from higher areas sensitive to global stimulus properties.

Mulligan, Jeffrey B.↗
NASA NTRSDate not supplied

FT-01: Autonomous Helicopter Flight Testing Over Long Island Sound Special Session - Hazard Perception and Avoidance Tech Area

This presentation is included as part of a panel reporting on the Integration of Automated Systems (IAS) flight test series. The IAS series concluded in October 2023 and was conducted under NASA’s Advanced Air Mobility project and in partnership with Sikorsky and the Defense Advanced Research Projects Agency (DARPA). The flight test effort included two crewed rotorcraft platforms. The first, a modified S-76B helicopter, served as the “ownship” for the duration of the flight test. The second vehicle, a modified S-70, served as the intruder aircraft. One Sikorsky pilot and one NASA test pilot was onboard each aircraft for every test point, with the NASA test pilot responsible for interacting with the research systems under test. This portion of the panel presentation focuses on the results of the flight test that assessed the Federal Aviation Administration’s (FAA) next-generation collision avoidance system, the Airborne Collision Avoidance System X for Rotorcraft (ACAS Xr). The two configurations available within ACAS Xr – the Collision Avoidance System (CAS) configuration and the Detect and Avoid (DAA) configuration – were flown with an onboard pilot under Visual Flight Rules in controlled airspace over the Long Island Sound (Connecticut, USA). A total of 33 flight test cards were flown with ACAS Xr active. Results showed that the ACAS Xr alerting and guidance was largely effective and rated positively by the NASA test pilots, exemplified by zero instances of the pilots overriding an ACAS Xr Resolution Advisory (RA). Key areas of improvement, however, were noted, particularly with regards to the lack of an aural alert indicating a need to accelerate when receiving an RA at low speed and the occurrence of multiple RAs that the pilots found to be unacceptable.

detect and avoid↗
NASA NTRS1997

A Cognitive System Model for Human/Automation Dynamics in Airspace Management

NASA has initiated a significant thrust of research and development focused on providing the flight crew and air traffic managers automation aids to increase capacity in en route and terminal area operations through the use of flexible, more fuel-efficient routing, while improving the level of safety in commercial carrier operations. In that system development, definition of cognitive requirements for integrated multi-operator dynamic aiding systems is fundamental. In order to support that cognitive function definition, we have extended the Man Machine Integrated Design and Analysis System (MIDAS) to include representation of multiple cognitive agents (both human operators and intelligent aiding systems) operating aircraft, airline operations centers and air traffic control centers in the evolving airspace. The demands of this application require representation of many intelligent agents sharing world-models, and coordinating action/intention with cooperative scheduling of goals and actions in a potentially unpredictable world of operations. The MIDAS operator models have undergone significant development in order to understand the requirements for operator aiding and the impact of that aiding in the complex nondeterminate system of national airspace operations. The operator model's structure has been modified to include attention functions, action priority, and situation assessment. The cognitive function model has been expanded to include working memory operations including retrieval from long-term store, interference, visual-motor and verbal articulatory loop functions, and time-based losses. The operator's activity structures have been developed to include prioritization and interruption of multiple parallel activities among multiple operators, to provide for anticipation (knowledge of the intention and action of remote operators), and to respond to failures of the system and other operators in the system in situation-specific paradigms. The model's internal representation has been be modified so that multiple, autonomous sets of equipment will function in a scenario as the single equipment sets do now. In order to support the analysis requirements with multiple items of equipment, it is necessary for equipment to access the state of other equipment objects at initialization time (a radar object may need to access the position and speed of aircraft in its area, for example), and as a function of perception and sensor system interaction. The model has been improved to include multiple world-states as a function of equipment am operator interaction. The model has been used -1o predict the impact of warning and alert zones in aircraft operation, and, more critic-ally, the interaction of flight-deck based warning mechanisms and air traffic controller action in response to ground-based conflict prediction and alerting systems. In this operation, two operating systems provide alerting to two autonomous, but linked sets of operators, whose view of the system and whose dynamics in response are radically different. System stability and operator action was predicted using the MIDAS model.

Corker, Kevin M.↗
NASA NTRSDate not supplied

Autonomous Ocean World Exploration: Advancement of a Virtual Testbed

The search for life (extinct or extant) and potentially habitable bodies in our solar system and beyond is one of the 12 priority science questions outlined in the National Acadamies’ 2022 decadal survey [5]. Extraterrestrial destinations containing liquid water present an opportunity to search for life as we know it, and in recent years an increasing number of such locations have been discovered within our solar system. Several Jovian moons—Europa, Ganymede, and Callisto [10]—and the Saturnian moons Enceladus [8] and Titan [9] are known or suspected to harbor massive subsurface oceans. Of these "ocean worlds", Europa is the focus of at least one planned NASA orbiter mission, Europa Clipper [4], and an early lander mission concept, the Europa Lander [2, 3]. Whereas most robotic missions to the Moon and Mars (e.g. orbiters, rovers, landers) to date have had ground controllers on Earth tightly involved in mission operations, missions to more distant worlds will require a high degree of onboard autonomy due to long communication lags and blackouts, harsh environments (radiation, cold), and more limited battery and hardware life. The past decade has seen great advances in both AI technologies and computing scalability and performance that offer promising solutions for spacecraft autonomy and motivate the software system and research programs described in this paper. The Ocean Worlds Autonomy Testbed for Exploration, Research, and Simulation (OceanWATERS) [1], which has been in development at the NASA Ames Research Center since 2018, is a virtual environment for testing lander autonomy solutions. It is built on the Robot Operating System (ROS), runs on consumer-grade Linux workstations, and was released as open source in 2020. OceanWATERS provides a physical and visual simulation of a prototypical lander in a Europa-like environment (Figure 1). The lander was modeled after requirements and specifications made in JPL’s Europa Lander Study of 2016 [3]. Simulated lander systems include stereo cameras and spotlights mounted on an antenna mast that pans and tilts, a 6 degrees of freedom (DoF) robotic arm with a force-torque sensor and two interchangeable end effectors, and a battery pack power system. The environment consists of multiple terrain models including a highly detailed model sourced from the FROST dataset [11], simulation of surrounding planetary bodies based on an ephemeris model, and lighting from the sun with associated surface illumination, reflectance, and shadows. Operations supported by OceanWATERS include panoramic and directed imaging of the environment and lander workspace, Cartesian and joint-level arm commanding, grinding of the terrain surface (e.g. digging a trench), and scooping of ground material (Figure 2) which can be discarded or collected as science samples in a receptacle that can be emptied (science operations themselves are not simulated). These operations are realized as ROS Actions and are complimented by a wide selection of telemetry that is continually produced by each lander subsystem. The power system model is driven by the open-source Generic Software Architecture for Prognostics (GSAP) [11] that predicts the battery’s remaining useful life and other characteristics. As a testbed for high-level autonomy, OceanWATERS provides an execution framework based on PLEXIL [12], an open-source plan specification language and execution engine developed largely at Ames. NASA's initial development of OceanWATERS, as well the Ocean Worlds Lander Autonomy Testbed (OWLAT) [6], a complimentary physical testbed developed at JPL, was the first step in a plan for realizing candidate onboard autonomy solutions for such planetary landers. In 2020 NASA solicited applications for its Autonomous Robotics Research for Ocean Worlds (ARROW) program, and in 2021 the similar Concepts for Ocean worlds Life Detection Technology (COLDTech) program. Collectively six research teams, based in universities and companies across the United States, were awarded grants to develop and demonstrate autonomy solutions on OceanWATERS and OWLAT. These 1–2-year projects have now finished or are nearing completion, and a wide variety of autonomy challenges in ocean world surface missions were addressed. Prototyped and demonstrated solutions have included autonomous discovery, response and adaptation to system faults and unexpected environmental events, world model synthesis through perception, plan synthesis using learned models, methods to optimize sample target selection and prioritize science data transmission, extension of PLEXIL for stochastic decision-making, and an integration of a model of JPL’s mission-ready COLDArm [7]. Technologies used in these projects include many forms of machine learning, causal reasoning, automated planning, Markov decision processes, formal methods, and other advanced techniques. A more detailed summary of the ARROW and COLDTech projects is given herein. OceanWATERS has had significant enhancements since its open-source release in 2020. Many of its new features were driven or shaped by feedback from the ARROW and COLDTech teams and requirements of their projects. In support of enabling autonomous adaptation to spacecraft faults (a specific capability solicited by both programs), a fault injection and detection framework was developed that supports a wide and growing range of fault types such as locked joints, image loss, and battery failures. The power system model was completed and integrated into the simulator, starting as a single-cell battery model and later upgraded to a multi-cell model with associated faults such as cell disconnection. Arm/terrain interaction was improved by adding a force-torque sensor and associated faults, and an analytic dig force model based on the Balovnev bucket force equations. Environment fidelity was increased by modeling terrain deformation resulting from digging and scooping; visual improvements were made in textures, lighting, and shadows. To facilitate interoperation with OWLAT, a unified command and telemetry interface between the testbeds was developed at the ROS level, along with a PLEXIL interface. The number of lander operations was greatly expanded (e.g. with Cartesian-based arm and antenna movement), and a framework was designed for users to build their own lander actions. A GUI for PLEXIL plan selection was created (Figure 3), and an expansive set of plans were added, such as those that illustrate patterns for fault handling. This paper provides a self-contained high-level description of OceanWATERS, focusing on more detailed coverage of the aforementioned enhancements. It provides a high-level summary of the projects undertaken by participants in the ARROW and COLDTech programs and how these efforts have helped shape OceanWATERS. Finally, potential future work and directions for the testbed are listed, as likely informed by the recent planetary science decadal survey [5].

K Michael Dalal↗