SEARCH · Search NASA
Results for “human activity recognition”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Lunar Mining and Processing: Considerations for Responsible Space Mining & Connections to Terrestrial Mining
The National Aeronautics and Space Administration (NASA) of the United States of America (US) has initiated the Artemis Moon to Mars program to send astronauts (the first woman and person of color) back to the lunar surface, create a sustainable human lunar exploration program, and lead the first human exploration mission to the Mars surface in the late 2030’s [1]. Besides reinvigorating human exploration beyond low Earth orbit not seen since the Apollo program and enabling new scientific activities and discoveries, a major objective of this program is to characterize the resources that exist on the Moon and Mars, and learn how to utilize them for human exploration and the commercialization of cis-lunar space. Commonly known as In Situ Resource Utilization (ISRU), the search for, acquisition, and processing of resources in space has the potential to greatly reduce the dependency on transporting mission consumables and infrastructure from Earth, thereby reducing mission costs, risks, and dependency on Earth. With the launch of Artemis I in November 2022 and the anticipation of several robotic missions to the Moon under the Commercial Lunar Payload Services (CLPS) program, greater recognition and excitement about NASA’s Artemis program and lunar exploration activities is growing in the public. With the recognition that past statements and concept videos of human exploration of the Moon are actually becoming real, there is also a growing awareness of the possible positive and negative consequences and impacts these exploration activities may have on the Moon and Mars. On the positive side, the development of ISRU and lunar mining and processing can enable and grow lunar surface exploration and cis-lunar commercial activities, as well as provide benefits to terrestrial industries through spin-in and spin-back of advanced technologies and autonomous operations. On the negative side, there is a perception that space mining will impact the lunar surface and environment negatively for science, and that cultural beliefs about the Moon need to be addressed and considered before these operations occur. This paper will begin to explore the potential driving attributes and guidelines that will address how best to maximize the lessons and connections to terrestrial mining to reduce the risk and cost of lunar ISRU and space commercial activities, enhance efforts to achieve the terrestrial ‘mine of the future’, and provide viable markets for space-derived technologies until commercial space mining is established. This paper will also begin to explore the potential driving attributes and guidelines that could address how to minimize the environmental and surface impacts of lunar ISRU and foster ‘responsible’ space mining that can be implemented until more official agreements and treaties are signed. The existing robust mining regulations adopted globally will be used as a basis for this examination and suggestions will be presented to adopt these agreements for use in space mining.
Speech Acquisition and Automatic Speech Recognition for Integrated Spacesuit Audio Systems
A voice-command human-machine interface system has been developed for spacesuit extravehicular activity (EVA) missions. A multichannel acoustic signal processing method has been created for distant speech acquisition in noisy and reverberant environments. This technology reduces noise by exploiting differences in the statistical nature of signal (i.e., speech) and noise that exists in the spatial and temporal domains. As a result, the automatic speech recognition (ASR) accuracy can be improved to the level at which crewmembers would find the speech interface useful. The developed speech human/machine interface will enable both crewmember usability and operational efficiency. It can enjoy a fast rate of data/text entry, small overall size, and can be lightweight. In addition, this design will free the hands and eyes of a suited crewmember. The system components and steps include beam forming/multi-channel noise reduction, single-channel noise reduction, speech feature extraction, feature transformation and normalization, feature compression, model adaption, ASR HMM (Hidden Markov Model) training, and ASR decoding. A state-of-the-art phoneme recognizer can obtain an accuracy rate of 65 percent when the training and testing data are free of noise. When it is used in spacesuits, the rate drops to about 33 percent. With the developed microphone array speech-processing technologies, the performance is improved and the phoneme recognition accuracy rate rises to 44 percent. The recognizer can be further improved by combining the microphone array and HMM model adaptation techniques and using speech samples collected from inside spacesuits. In addition, arithmetic complexity models for the major HMMbased ASR components were developed. They can help real-time ASR system designers select proper tasks when in the face of constraints in computational resources.
Monitoring issues from a modeling perspective
Recognition that earth's climate and biogeophysical conditions are likely changing due to human activities has led to a heightened awareness of the need for improved long-term global monitoring. The present long-term measurement efforts tend to be spotty in space, inadequately calibrated in time, and internally inconsistent with respect to other instruments and measured quantities. In some cases, such as most of the biosphere, most chemicals, and much of the ocean, even a minimal monitoring program is not available. Recently, it has become painfully evident that emerging global change issues demand information and insights that the present global monitoring system simply cannot supply. This is because a monitoring system must provide much more than a statement of change at a given level of statistical confidence. It must describe changes in diverse parts of the entire earth system on regional to global scales. It must be able to provide enough input to allow an integrated physical characterization of the changes that have occurred. Finally, it must allow a separation of the observed changes into their natural and anthropogenic parts. The enormous policy significance of global change virtually guarantees an unprecedented level of scrutiny of the changes in the earth system and why they are happening. These pressures create a number of emerging challenges and opportunities. For example, they will require a growing partnership between the observational programs and the theory/modeling community. Without this partnership, the scientific community will likely fall short in the monitoring effort. The monitoring challenge before us is not to solve the problem now, but rather to set appropriate actions in motion so as to create the required framework for solution. Each individual piece needs to establish its role in the large problem and how the required interactions are to take place. Below, we emphasize some of the needs and opportunities that could and should be addressed through participation by the theoreticians and modelers in the global change monitoring effort.
Automatic Speech Acquisition and Recognition for Spacesuit Audio Systems
NASA has a widely recognized but unmet need for novel human-machine interface technologies that can facilitate communication during astronaut extravehicular activities (EVAs), when loud noises and strong reverberations inside spacesuits make communication challenging. WeVoice, Inc., has developed a multichannel signal-processing method for speech acquisition in noisy and reverberant environments that enables automatic speech recognition (ASR) technology inside spacesuits. The technology reduces noise by exploiting differences between the statistical nature of signals (i.e., speech) and noise that exists in the spatial and temporal domains. As a result, ASR accuracy can be improved to the level at which crewmembers will find the speech interface useful. System components and features include beam forming/multichannel noise reduction, single-channel noise reduction, speech feature extraction, feature transformation and normalization, feature compression, and ASR decoding. Arithmetic complexity models were developed and will help designers of real-time ASR systems select proper tasks when confronted with constraints in computational resources. In Phase I of the project, WeVoice validated the technology. The company further refined the technology in Phase II and developed a prototype for testing and use by suited astronauts.
Long-term modifications of synaptic efficacy in the human inferior and middle temporal cortex
The primate temporal cortex has been demonstrated to play an important role in visual memory and pattern recognition. It is of particular interest to investigate whether activity-dependent modification of synaptic efficacy, a presumptive mechanism for learning and memory, is present in this cortical region. Here we address this issue by examining the induction of synaptic plasticity in surgically resected human inferior and middle temporal cortex. The results show that synaptic strength in the human temporal cortex could undergo bidirectional modifications, depending on the pattern of conditioning stimulation. High frequency stimulation (100 or 40 Hz) in layer IV induced long-term potentiation (LTP) of both intracellular excitatory postsynaptic potentials and evoked field potentials in layers II/III. The LTP induced by 100 Hz tetanus was blocked by 50-100 microM DL-2-amino-5-phosphonovaleric acid, suggesting that N-methyl-D-aspartate receptors were responsible for its induction. Long-term depression (LTD) was elicited by prolonged low frequency stimulation (1 Hz, 15 min). It was reduced, but not completely blocked, by DL-2-amino-5-phosphonovaleric acid, implying that some other mechanisms in addition to N-methyl-DL-aspartate receptors were involved in LTD induction. LTD was input-specific, i.e., low frequency stimulation of one pathway produced LTD of synaptic transmission in that pathway only. Finally, the LTP and LTD could reverse each other, suggesting that they can act cooperatively to modify the functional state of cortical network. These results suggest that LTP and LTD are possible mechanisms for the visual memory and pattern recognition functions performed in the human temporal cortex.
Multimodal Neuroelectric Interface Development
This project aims to improve performance of NASA missions by developing multimodal neuroelectric technologies for augmented human-system interaction. Neuroelectric technologies will add completely new modes of interaction that operate in parallel with keyboards, speech, or other manual controls, thereby increasing the bandwidth of human-system interaction. We recently demonstrated the feasibility of real-time electromyographic (EMG) pattern recognition for a direct neuroelectric human-computer interface. We recorded EMG signals from an elastic sleeve with dry electrodes, while a human subject performed a range of discrete gestures. A machine-teaming algorithm was trained to recognize the EMG patterns associated with the gestures and map them to control signals. Successful applications now include piloting two Class 4 aircraft simulations (F-15 and 757) and entering data with a "virtual" numeric keyboard. Current research focuses on on-line adaptation of EMG sensing and processing and recognition of continuous gestures. We are also extending this on-line pattern recognition methodology to electroencephalographic (EEG) signals. This will allow us to bypass muscle activity and draw control signals directly from the human brain. Our system can reliably detect P-rhythm (a periodic EEG signal from motor cortex in the 10 Hz range) with a lightweight headset containing saline-soaked sponge electrodes. The data show that EEG p-rhythm can be modulated by real and imaginary motions. Current research focuses on using biofeedback to train of human subjects to modulate EEG rhythms on demand, and to examine interactions of EEG-based control with EMG-based and manual control. Viewgraphs on these neuroelectric technologies are also included.
Multimodal neuroelectric interface development
We are developing electromyographic and electroencephalographic methods, which draw control signals for human-computer interfaces from the human nervous system. We have made progress in four areas: 1) real-time pattern recognition algorithms for decoding sequences of forearm muscle activity associated with control gestures; 2) signal-processing strategies for computer interfaces using electroencephalogram (EEG) signals; 3) a flexible computation framework for neuroelectric interface research; and d) noncontact sensors, which measure electromyogram or EEG signals without resistive contact to the body.
A simulation system for Space Station extravehicular activity
A simulation program has been developed which addresses the human factors required to provide a crewmember with productive information during Space Station EVA. The operation of the voice recognition and control system and helmet-mounted projection display in the Space Station Extravehicular Mobility Unit (EMU) is reviewed. The features of the simulation program are discussed, including logic flow, information types, and the man-machine interface techniques used in the simulation program, voice recognizer, and helmet-mounted display.
NASA Johnson Space Center Usability Testing and Analysis Facility (UTAF) Overview
The Usability Testing and Analysis Facility (UTAF) is part of the Space Human Factors Laboratory at the NASA Johnson Space Center in Houston, Texas. The facility provides support to the Office of Biological and Physical Research, the Space Shuttle Program, the International Space Station Program, and other NASA organizations. In addition, there are ongoing collaborative research efforts with external businesses and universities. The UTAF provides human factors analysis, evaluation, and usability testing of crew interfaces for space applications. This includes computer displays and controls, workstation systems, and work environments. The UTAF has a unique mix of capabilities, with a staff experienced in both cognitive human factors and ergonomics. The current areas of focus are: human factors applications in emergency medical care and informatics; control and display technologies for electronic procedures and instructions; voice recognition in noisy environments; crew restraint design for unique microgravity workstations; and refinement of human factors processes. This presentation will provide an overview of ongoing activities, and will address how the projects will evolve to meet new space initiatives.
NASA Johnson Space Center Usability Testing and Analysis Facility (WAF) Overview
The Usability Testing and Analysis Facility (UTAF) is part of the Space Human Factors Laboratory at the NASA Johnson Space Center in Houston, Texas. The facility provides support to the Office of Biological and Physical Research, the Space Shuttle Program, the International Space Station Program, and other NASA organizations. In addition, there are ongoing collaborative research efforts with external businesses and universities. The UTAF provides human factors analysis, evaluation, and usability testing of crew interfaces for space applications. This includes computer displays and controls, workstation systems, and work environments. The UTAF has a unique mix of capabilities, with a staff experienced in both cognitive human factors and ergonomics. The current areas of focus are: human factors applications in emergency medical care and informatics; control and display technologies for electronic procedures and instructions; voice recognition in noisy environments; crew restraint design for unique microgravity workstations; and refinement of human factors processes. This presentation will provide an overview of ongoing activities, and will address how the projects will evolve to meet new space initiatives.
NASA Johnson Space Center Usability Testing and Analysis facility (UTAF) Overview
The Usability Testing and Analysis Facility (UTAF) is part of the Space Human Factors Laboratory at the NASA Johnson Space Center in Houston, Texas. The facility performs research for NASA's HumanSystems Integration Program, under the HumanSystems Research and Technology Division. Specifically, the UTAF provides human factors support for space vehicles, including the International Space Station, the Space Shuttle, and the forthcoming Crew Exploration Vehicle. In addition, there are ongoing collaborative research efforts with external corporations and universities. The UTAF provides human factors analysis, evaluation, and usability testing of crew interfaces for space applications. This includes computer displays and controls, workstation systems, and work environments. The UTAF has a unique mix of capabilities, with a staff experienced in both cognitive human factors and ergonomics. The current areas of focus are: human factors applications in emergency medical care and informatics; control and display technologies for electronic procedures and instructions; voice recognition in noisy environments; crew restraint design for unique microgravity workstations; and refinement of human factors processes and requirements. This presentation will provide an overview of ongoing activities, and will address how the UTAF projects will evolve to meet new space initiatives.
Risks of Increased Earth Independence
As human space exploration begins to extend beyond the immediate vicinity of Earth, the ability of expertise on the ground to support operations will progressively decrease. Ground control expertise has been the primary countermeasure preventing loss of life and mission over the past sixty years and it will be gradually stripped away as expeditions reach further into space. The large, flexible and adaptive teams of experts on the ground, provide not only engineering analysis and problem-solving but also greatly increased work capacity. Monitoring of the large and fast moving stream of telemetry data is carried out around the clock by 20-30 flight controllers per shift. Analysis of that data and troubleshooting is carried out by an additional 50-100 engineers per shift. Artificial intelligence capabilities will need to be part of the solution but will not the entire solution. As progress continues to be made on intelligent systems, especially those for use in complex, dynamic environments where humans will remain a part of the activity, there is an ever increasing need to focus on how those systems will interact with the humans. Although capabilities such as natural language processing and facial recognition have become common place, even basic aspects of problem-solving, causal reasoning and generative decision-making remain well-beyond of our current capabilities. Advanced data visualization, procedure execution support, and new training approaches will be needed to close the gap between what is currently provided by experts on the ground and what the crew will need to do in increasingly autonomous ways with greater distance from Earth.
Investigation of air transportation technology at the Massachusetts Institute of Technology, 1988-1989
There are four areas of research being pursued in 1988 under sponsorship of the FAA/NASA Joint University Research Program, and one area which has been completed. The four active areas were: (1) Automatic Speech Recognition (ASR) in Air Traffic Control. The purpose of this research effort is to demonstrate the feasibility of using ASR technology within the ATC environment and to address the problems involved, especially the relevant human factors issues. (2) A Rule-Based Planning and Scheduling System. Planning denotes the formulation of a detailed scheme, program, or method worked out beforehand for the accomplishment of a goal. It involves the analysis of the desired goal and its division into sub-goals which are subsequently treated in the same way until a set of primitive objectives is obtained. (3) Modeling of Ice Accretion on Aircraft in Glaze Icing Conditions. The work in aircraft icing over the past year has focused on the fundamental aspects of glaze ice accretion, with the goal of improving analytical ice accretion models. Over the past year, studies have been conducted on the generation of surface roughness on accreting ice surfaces with the goal of providing a deterministic surface roughness in the ice accretion models. (4) Cockpit Display of Hazardous Weather Information. Information transfer and display issues associated with the dissemination of hazardous weather warnings are studied in the context of windshear alerts.
CATS-based Air Traffic Controller Agents
This report describes intelligent agents that function as air traffic controllers. Each agent controls traffic in a single sector in real time; agents controlling traffic in adjoining sectors can coordinate to manage an arrival flow across a given meter fix. The purpose of this research is threefold. First, it seeks to study the design of agents for controlling complex systems. In particular, it investigates agent planning and reactive control functionality in a dynamic environment in which a variety perceptual and decision making skills play a central role. It examines how heuristic rules can be applied to model planning and decision making skills, rather than attempting to apply optimization methods. Thus, the research attempts to develop intelligent agents that provide an approximation of human air traffic controller behavior that, while not based on an explicit cognitive model, does produce task performance consistent with the way human air traffic controllers operate. Second, this research sought to extend previous research on using the Crew Activity Tracking System (CATS) as the basis for intelligent agents. The agents use a high-level model of air traffic controller activities to structure the control task. To execute an activity in the CATS model, according to the current task context, the agents reference a 'skill library' and 'control rules' that in turn execute the pattern recognition, planning, and decision-making required to perform the activity. Applying the skills enables the agents to modify their representation of the current control situation (i.e., the 'flick' or 'picture'). The updated representation supports the next activity in a cycle of action that, taken as a whole, simulates air traffic controller behavior. A third, practical motivation for this research is to use intelligent agents to support evaluation of new air traffic control (ATC) methods to support new Air Traffic Management (ATM) concepts. Current approaches that use large, human-in-the-loop simulations are unquestionably valuable for this purpose, but pose considerable logistical, fiscal, and experimental control problems. First, data analysis is extremely complicated, owing simply to the large number of participants and data sources in such simulations. In addition, experienced human air traffic controllers working adjacent sectors tend to flexibly adapt to the evolving control problem - potentially shifting to other strategies than those under investigation. In addition, their performance is tightly coupled to the control interface, which in the development phase may support some concepts and supporting strategies better than others. A simple shift in strategy by one controller can change the character of a particular traffic scenario dramatically, which makes experimental comparison of ATC performance under different traffic scenarios difficult. Training a given team of controllers on operations under a new ATM concept for a sufficient period of time could avert such difficulties, but instituting an adequate training program is expensive and logistically difficult.
Designing for Our Future in Space
Over the past several years, the disciplines of architecture and human factors have been increasingly recognized as specialties that have focused upon "human-centered design" in the development of spacecraft and surface habitats. These specialties have been instrumental in the conceptual design of overall spacecraft configurations and layouts, as well as habitability outfitting hardware, such as the galley, hygiene facility, sleep quarters, or the layout of displays and controls. From the human-centered perspective, this approach to design assists in the mitigation of risk when designing for an extreme environment such as space. It takes into account the human s physical and cognitive capabilities and limitations, the human s performance in the context of human space flight, the human s interaction with machines that are both physically and cognitively complex, the activities required of the human to accomplish the goals of missions, and the use of design practices that promote products to enable human activity. It is this latter aspect - the use of design practices that promote products to enable human activity - that is the focus of the approach used by the Rhode Island School of Design (RISD) in collaboration with the Habitability and Human Factors Branch (HHFB) at the NASA Johnson Space Center (JSC). During the past few years, there has been a growing recognition of the value added by utilizing industrial designers to further the conceptual development of space hardware, that when used in conjunction with architecture and human factors, provides a robust solution to the design challenge. The "Design for Extreme Environments" Studio at RISD has taken suggested design topics from the NASA JSC HHFB and asked the students to investigate solutions to these challenges. The topics have demanded that the student pay particular attention to a variety of aspects of the space environment and understand how the human responds to each. The student must then adapt the design to these responses. The studio environment has been one way to introduce these challenges, but providing for an "in-residence" opportunity at JSC has given the students a broader vision and set of experiences. The accompanying presentation highlights the studio as well as in-residence work that has been accomplished.
Essential Autonomous Science Inference on Rovers (EASIR)
Existing constraints on time, computational, and communication resources associated with Mars rover missions suggest on-board science evaluation of sensor data can contribute to decreasing human-directed operational planning, optimizing returned science data volumes, and recognition of unique or novel data. All of which act to increase the scientific return from a mission. Many different levels of science autonomy exist and each impacts the data collected and returned by, and activities of, rovers. Several computational algorithms, designed to recognize objects of interest to geologists and biologists, are discussed. The algorithms represent various functions that producing scientific opinions and several scenarios illustrate how the opinions can be used.
Sensor fusion II: Human and machine strategies; Proceedings of the Meeting, Philadelphia, PA, Nov. 6-9, 1989
Various papers on human and machine strategies in sensor fusion are presented. The general topics addressed include: active vision, measurement and analysis of visual motion, decision models for sensor fusion, implementation of sensor fusion algorithms, applying sensor fusion to image analysis, perceptual modules and their fusion, perceptual organization and object recognition, planning and the integration of high-level knowledge with perception, using prior knowledge and context in sensor fusion.