Results for “Multi-modal integration” — Page 2

NASA NTRS2023

Progressively Enabling Earth Independent Medical Operations (EIMO)

This panel presents the findings from a series of Technical Interchange Meetings (TIMs) hosted by the Exploration Medical Capability Element (ExMC) in NASA’s Human Research Program. The topics for the TIMs were derived from a 2-day conference of senior leaders and subject matters experts that collectively outlined a multi-faceted strategy designed to optimize crew health and performance through an increasingly autonomous medical approach. The first abstract in this panel outlines the scope of issues related to data collection, usage, transmission and computing capacity to facilitate EIMO. The second presentation provides an overview of the challenges in developing curricula and advanced training tools to baseline knowledge, skills and abilities (KSA), verify clinical competency and assure retention during prolonged durations inherent in exploration-class missions. An overview of the complicated medical supply and resource chain necessary to facilitate EIMO is provided in the third presentation of this panel. The final presentation in this EIMO panel surveys the breadth and depth of demands on cognitive load expected to be experienced by crew on an exploration mission and proposes strategies to mitigate the prospect of cognitive overload through methods to shift task load from the crew to multi-modal artificial intelligence based medical support systems. Taken together, these presentations summarize the challenges to be expected and potential solution spaces to be explored and developed to progressively enable increasing autonomous medical operations to support crewed missions beyond low earth orbit. Through EIMO focused pre-mission planning, integrated data architecture design, innovative training development and AI-assisted task load management, the gradual transition of medical care and decision making from terrestrial to space-based assets enabling support of astronaut health and performance and reducing overall mission risk is achievable.

Jay Lemery↗

NASA NTRSDate not supplied

Progressively Enabling Earth Independent Medical Operations (EIMO)

This panel presents the findings from a series of Technical Interchange Meetings (TIMs) hosted by the Exploration Medical Capability Element (ExMC) in NASA’s Human Research Program. The topics for the TIMs were derived from a 2-day conference of senior leaders and subject matters experts that collectively outlined a multi-faceted strategy designed to optimize crew health and performance through an increasingly autonomous medical approach. The first abstract in this panel outlines the scope of issues related to data collection, usage, transmission and computing capacity to facilitate EIMO. The second presentation provides an overview of the challenges in developing curricula and advanced training tools to baseline knowledge, skills and abilities (KSA), verify clinical competency and assure retention during prolonged durations inherent in exploration-class missions. An overview of the complicated medical supply and resource chain necessary to facilitate EIMO is provided in the third presentation of this panel. The final presentation in this EIMO panel surveys the breadth and depth of demands on cognitive load expected to be experienced by crew on an exploration mission and proposes strategies to mitigate the prospect of cognitive overload through methods to shift task load from the crew to multi-modal artificial intelligence based medical support systems. Taken together, these presentations summarize the challenges to be expected and potential solution spaces to be explored and developed to progressively enable increasing autonomous medical operations to support crewed missions beyond low earth orbit. Through EIMO focused pre-mission planning, integrated data architecture design, innovative training development and AI-assisted task load management, the gradual transition of medical care and decision making from terrestrial to space-based assets enabling support of astronaut health and performance and reducing overall mission risk is achievable.

John Lemery↗

NASA NTRS1988

Presentation planning using an integrated knowledge base

A description is given of user interface research aimed at bringing together multiple input and output modes in a way that handles mixed mode input (commands, menus, forms, natural language), interacts with a diverse collection of underlying software utilities in a uniform way, and presents the results through a combination of output modes including natural language text, maps, charts and graphs. The system, Integrated Interfaces, derives much of its ability to interact uniformly with the user and the underlying services and to build its presentations, from the information present in a central knowledge base. This knowledge base integrates models of the application domain (Navy ships in the Pacific region, in the current demonstration version); the structure of visual displays and their graphical features; the underlying services (data bases and expert systems); and interface functions. The emphasis is on a presentation planner that uses the knowledge base to produce multi-modal output. There has been a flurry of recent work in user interface management systems. (Several recent examples are listed in the references). Existing work is characterized by an attempt to relieve the software designer of the burden of handcrafting an interface for each application. The work has generally focused on intelligently handling input. This paper deals with the other end of the pipeline - presentations.

Arens, Yigal↗

NASA NTRS2014

NASA Wearable Technology CLUSTER 2013-2014 Report

Wearable technology has the potential to revolutionize the way humans interact with one another, with information, and with the electronic systems that surround them. This change can already be seen in the dramatic increase in the availability and use of wearable health and activity monitors. These devices continuously monitor the wearer using on-‐body sensors and wireless communication. They provide feedback that can be used to improve physical health and performance. Smart watches and head mounted displays are also receiving a great deal of commercial attention, providing immediate access to information via graphical displays, as well as additional sensing features. For the purposes of the Wearable Technology CLUSTER, wearable technology is broadly defined as any electronic sensing, human interfaces, computing, or communication that is mounted on the body. Current commercially available wearable devices primarily house electronics in rigid packaging to provide protection from flexing, moisture, and other contaminants. NASA mentors are interested in this approach, but are also interested in direct integration of electronics into clothing to enable more comfortable systems. For human spaceflight, wearable technology holds a great deal of promise for significantly improving safety, efficiency, autonomy, and research capacity for the crew in space and support personnel on the ground. Specific capabilities of interest include: Continuous biomedical monitoring for research and detection of health problems. Environmental monitoring for individual exposure assessments and alarms. Activity monitoring for responsive robotics and environments. Multi-modal caution and warning using tactile, auditory, and visual alarms. Wireless, hands-free, on-demand voice communication. Mobile, on-demand access to space vehicle and robotic displays and controls. Many technical challenges must be overcome to realize these wearable technology applications. For example, to make a wearable device that is both functional and comfortable for long duration wear, developers must strive to reduce electronic mass and volume while also addressing constraints imposed by the body attachment method. Depending on the application, the device must be placed in a location that the user can see and reach, and that provides the appropriate access to air and the wearer's skin. Limited power is available from body-‐worn batteries and heat must be managed to prevent discomfort. If the clothing is to be washed, there are additional durability and washability hurdles that traditional electronics are not designed to address. Finally, each specific capability has unique technical challenges that will likely require unique solutions. In addition to the technical challenges, development of wearable devices is made more difficult by the diversity of skills required and the historic lack of collaboration across domains. Wearable technology development requires expertise in textiles engineering, apparel design, software and computer engineering, electronic design and manufacturing, human factors engineering, and application-‐specific fields such as acoustics, medical devices, and sensing. Knowledge from each of these domains must be integrated to create functional and comfortable devices. For this reason, the diversity of knowledge and experience represented in the Wearable Technology is critical to overcoming the fundamental challenges in the field.

Simon, Cory↗

NASA NTRS1995

Multi-modal virtual environment research at Armstrong Laboratory

One mission of the Paul M. Fitts Human Engineering Division of Armstrong Laboratory is to improve the user interface for complex systems through user-centered exploratory development and research activities. In support of this goal, many current projects attempt to advance and exploit user-interface concepts made possible by virtual reality (VR) technologies. Virtual environments may be used as a general purpose interface medium, an alternative display/control method, a data visualization and analysis tool, or a graphically based performance assessment tool. An overview is given of research projects within the division on prototype interface hardware/software development, integrated interface concept development, interface design and evaluation tool development, and user and mission performance evaluation tool development.

Eggleston, Robert G.↗

NASA NTRSDate not supplied

High Fidelity Adaptively Refined CFD and Reduced Order Models of a High Aspect Ratio Aeroelastic Wing Wind-Tunnel Model

The NASA Advanced Air Transport Technology (AATT) goal of reduced fuel burn for transport aircraft has led to the NASA N+3 High Aspect Ratio Wing (HARW) subproject. This project requires identifying, developing, and demonstrating key technologies and integrated multidisciplinary solutions to enable a safe, high performance, aeroelastic wing. Since this aircraft will have a high aspect ratio wing, aeroelasticity is expected to be a major issue in the design. In this paper high fidelity computational fluid dynamics (CFD) is performed with flow adapted meshes. A system identification of the aerodynamics is developed using both a multi-modal multi-sine time-marching and a multi-mode linear frequency domain method. GLA, MLA and flutter suppression simulations will be performed.

Robert Bartels↗

NASA NTRSDate not supplied

Earth Independent Medical Operations (EIMO) Datascope: Challenges and Potential Solutions

Data flows and storage/retrieval capacity are severely constrained during missions in space and challenges will become even greater during exploration class missions. There is a need for an artificial intelligence (AI)-based clinical decision support system (CDSS) to monitor and analyze data to provide real-time consultative support for crew medical officer (CMO) decision-making. EIMO is defined as the gradual transition of medical care and decision making from terrestrial to space-based assets, enabling support of astronaut health and performance and reducing overall mission risk. While a hallmark of this paradigm shift from low-earth orbit is that on-board care will increasingly become the responsibility of the astronauts for primary management and decision making, terrestrial assets will continue to be paramount in pre-mission screening and planning, as well as prevention, health maintenance and long-term care contingencies. New capabilities and systems that enable progressively more robust and resilient systems and crews will be necessary to reduce risk and increase probability of deep space exploration mission success. An aspiration for EIMO is to develop AI-enhanced solutions for analysis of crew health & performance data and to facilitate clinical decision support for autonomous medical operations. A “system of systems” approach is envisioned whereby EIMO will deploy AI-supported natural language processing and machine learning (ML) techniques to utilize embedded reference databases and real-time data streams [input vectors] from multiple data sources. Constituent input vectors may include environmental controls, countermeasures data, behavioral data, physiologic wearables, point-of-care laboratory tests, personalized medical records, inventory trade space risk assessments, COTS medical databases, and ground support inputs. An ideal AI capability would possess trained fusion algorithms to cross reference input vectors with medical ‘knowledge’ [cultivated database] to stratify relevant data streams for predictive and actionable capabilities. In addition, EIMO will feature mobility, in that it can be accessed and can push/pull data within and between multiple vehicles/habitats. Large amounts and variable sources of data can be leveraged to diagnose, inform treatment strategies, and potentially predict medical events and performance decrements. Inclusion of advanced training tools using extended reality will enable increasingly autonomous medical care to aid a CMO when ground support is unavailable or time-delayed beyond required action window, e.g., emergent medical situations. EIMO CDSS would require very large datasets to train pre-flight and significant amounts of data are needed to support ML via in-flight CDSS operations. An additional challenge will be to find sufficient data to train a model relevant to astronaut demographics. The rapid, accelerating evolution of this field creates a propitious solution space to leverage multi-modal AI through public-private partnership(s). The status of multi-modal AI systems today would preclude their use for long duration missions as they remain unreliable and are subject to “digital hallucinations” and other errors that could pose operational risk. A federated labs structure is being considered to test and optimize data flow from the multiple input vectors leading to field testing in suitable ground/flight analogs. Critical to the success of an EIMO CDSS will be integration and interoperability and success will be defined by a system that can serve as an in-flight medical consult for the CMO providing critical support during medical contingencies. Benefits to terrestrial medicine may be significant as an outflow of the EIMO medical system, particularly for remote areas and communities lacking significant infrastructure, personnel and resources.

J Lemery↗

NASA NTRS2024

Earth Independent Medical Operations (EIMO) Datascope: Challenges and Potential Solutions

Data flows and storage/retrieval capacity are severely constrained during missions in space and challenges will become even greater during exploration class missions. There is a need for an artificial intelligence (AI)-based clinical decision support system (CDSS) to monitor and analyze data to provide real-time consultative support for crew medical officer (CMO) decision-making. EIMO is defined as the gradual transition of medical care and decision making from terrestrial to space-based assets, enabling support of astronaut health and performance and reducing overall mission risk. While a hallmark of this paradigm shift from low-earth orbit is that on-board care will increasingly become the responsibility of the astronauts for primary management and decision making, terrestrial assets will continue to be paramount in pre-mission screening and planning, as well as prevention, health maintenance and long-term care contingencies. New capabilities and systems that enable progressively more robust and resilient systems and crews will be necessary to reduce risk and increase probability of deep space exploration mission success. An aspiration for EIMO is to develop AI-enhanced solutions for analysis of crew health & performance data and to facilitate clinical decision support for autonomous medical operations. A “system of systems” approach is envisioned whereby EIMO will deploy AI-supported natural language processing and machine learning (ML) techniques to utilize embedded reference databases and real-time data streams [input vectors] from multiple data sources. Constituent input vectors may include environmental controls, countermeasures data, behavioral data, physiologic wearables, point-of-care laboratory tests, personalized medical records, inventory trade space risk assessments, COTS medical databases, and ground support inputs. An ideal AI capability would possess trained fusion algorithms to cross reference input vectors with medical ‘knowledge’ [cultivated database] to stratify relevant data streams for predictive and actionable capabilities. In addition, EIMO will feature mobility, in that it can be accessed and can push/pull data within and between multiple vehicles/habitats. Large amounts and variable sources of data can be leveraged to diagnose, inform treatment strategies, and potentially predict medical events and performance decrements. Inclusion of advanced training tools using extended reality will enable increasingly autonomous medical care to aid a CMO when ground support is unavailable or time-delayed beyond required action window, e.g., emergent medical situations. EIMO CDSS would require very large datasets to train pre-flight and significant amounts of data are needed to support ML via in-flight CDSS operations. An additional challenge will be to find sufficient data to train a model relevant to astronaut demographics. The rapid, accelerating evolution of this field creates a propitious solution space to leverage multi-modal AI through public-private partnership(s). The status of multi-modal AI systems today would preclude their use for long duration missions as they remain unreliable and are subject to “digital hallucinations” and other errors that could pose operational risk. A federated labs structure is being considered to test and optimize data flow from the multiple input vectors leading to field testing in suitable ground/flight analogs. Critical to the success of an EIMO CDSS will be integration and interoperability and success will be defined by a system that can serve as an in-flight medical consult for the CMO providing critical support during medical contingencies. Benefits to terrestrial medicine may be significant as an outflow of the EIMO medical system, particularly for remote areas and communities lacking significant infrastructure, personnel and resources.

Medical Operations↗

NASA NTRS1980

Multi-modal information processing for visual workload relief

The simultaneous performance of two single-dimensional compensatory tracking tasks, one with the left hand and one with the right hand, is discussed. The tracking performed with the left hand was considered the primary task and was performed with a visual display or a quickened kinesthetic-tactual (KT) display. The right-handed tracking was considered the secondary task and was carried out only with a visual display. Although the two primary task displays had afforded equivalent performance in a critical tracking task performed alone, in the dual-task situation the quickened KT primary display resulted in superior secondary visual task performance. Comparisons of various combinations of primary and secondary visual displays in integrated or separated formats indicate that the superiority of the quickened KT display is not simply due to the elimination of visual scanning. Additional testing indicated that quickening per se also is not the immediate cause of the observed KT superiority.

Burke, M. W.↗

NASA NTRS2018

NeMO-Net - The Neural Multi-Modal Observation & Training Network for Global Coral Reef Assessment

In the past decade, coral reefs worldwide have experienced unprecedented stresses due to climate change, ocean acidification, and anthropomorphic pressures, instigating massive bleaching and die-off of these fragile and diverse ecosystems. Furthermore, remote sensing of these shallow marine habitats is hindered by ocean wave distortion, refraction and optical attenuation, leading invariably to data products that are often of low resolution and signal-to-noise (SNR) ratio. However, recent advances in UAV and Fluid Lensing technology have allowed us to capture multispectral 3D imagery of these systems at sub-cm scales from above the water surface, giving us an unprecedented view of their growth and decay. By combining spatial and spectral information from varying resolutions, we seek to augment and improve the classification accuracy of previously low-resolution datasets at large temporal scales.NeMO-Net, the first open-source deep convolutional neural network (CNN) and interactive learning and training software, currently being developed at NASA Ames, is aimed at assessing the present and past dynamics of coral reef ecosystems through determination of percent living cover and morphology. The latest iteration uses fully convolutional networks to segment and identify coral imagery taken by UAVs and satellites, including WorldView-2 and Sentinel. We present results taken from the Indian Ocean where classification accuracy has exceeded 91% for 24 geomorphological classes given ample training data. In addition, we utilize deep Laplacian Pyramid Super-Resolution Networks (LapSRN) to reconstruct high resolution information from low resolution imagery, trained from various UAV and satellite datasets. Finally, in the case of insufficient training data, we have developed an interactive online platform that allows users to easily segment and submit their classifications, which has been integrated with the current NeMO-Net workflow. Specifically, we present results from the Fiji islands in which preliminary user data has allowed for the accurate identification of 9 separate classes, despite issues such as cloud shadowing and spectral variation. The project is being supported by NASA's Earth Science Technology Office (ESTO) Advanced Information Systems Technology (AIST-16) Program.

Neural↗

NASA NTRS2015

Integrated Assessments of the Impact of Climate Change on Agriculture: An Overview of AgMIP Regional Research in South Asia

South Asia encompasses a wide and highly varied geographic region, and includes climate zones ranging from the mountainous Himalayan territory to the tropical lowland and coastal zones along alluvial floodplains. The region's climate is dominated by a monsoonal circulation that heralds the arrival of seasonal rainfall, upon which much of the regional agriculture relies. The spatial and temporal distribution of this rainfall is, however, not uniform over the region. Northern South Asia, central India, and the west coast receive much of their rainfall during the southwest monsoon season, between June and September. These rains partly result from the moisture transport accompanying the monsoonal winds, which move in the southwesterly direction from the equatorial Indian Ocean. Regions further south, such as south/southeast India and Sri Lanka, may receive rains from both the southwest monsoon, and also during the northeast monsoon season between October and December (with northeasterly monsoon wind flow and moisture flux), which results in a bi- or multi-modal rainfall distribution. In addition, rainfall across South Asia displays a large amount of intraseasonal and interannual variability. Interannual variability is influenced by many drivers, both natural (e.g., El Ni-Southern Oscillation; ENSO) and man-made (e.g., rising temperatures due to increasing greenhouse gas concentrations), and it is challenging to obtaining accurate time-series of annual rainfall, even amongst various observed data products, which display inconsistencies amongst themselves. These climatic and rainfall variations can further complicate South Asia's agricultural and water management. Agriculture employs at least 65 of the workforce in most South Asian countries, and nearly 80 of South Asia's poor inhabit rural areas. Understanding the response of current agricultural production to climate variability and future climate change is of utmost importance in securing food and livelihoods for South Asia's growing population. In order to assess the future of food and livelihood security across South Asia, the Agricultural Model Intercomparison and Improvement Project (AgMIP) has undertaken integrated climate-crop-economic assessments of the impact of climate change on food security and poverty in South Asia, encompassing Bangladesh, India, Nepal, Pakistan, and Sri Lanka. AgMIP has funded, on a competitive basis, four South Asian regional research teams (RRTs) and one South Asian coordination team (CT) to undertake climate-crop-economic integrated assessments of food security for many districts in each of these countries, with the goal of characterizing the state of food security and poverty across the region, and projecting how these are subject to change under future climate change conditions.

mountains↗

NASA NTRSDate not supplied

Datascope to Enable Earth Independent Medical Operations (EIMO)

BACKGROUND: NASA has amassed sixty years of knowledge and experience relevant to maintenance of crew health and performance in low earth orbit. The Apollo Program introduced the importance of ensuring progressively autonomous operational capability. Earth Independent Medical Operations (EIMO) will require a gradual shift in the balance of medical responsibility, management, and authority from terrestrial to space-based assets. Terrestrial assets will continue to be essential for pre-mission screening and planning in addition to maintenance of crew health and performance. However, new capabilities are needed to enable EIMO and the amount of data required to support these systems, and mitigate the impacts of data transmission delays and reduced bandwidth coupled with lack of cloud-like resources and on-board computing capacity that is currently unclear or operationally insufficient. OVERVIEW: The overall goal of EIMO is to develop artificial intelligence (AI)-based solutions to analyze crew health and performance data utilizing a clinical decision support system (CDSS) to provide crew medical officers (CMO) with the equivalent of real-time, on-board medical consults. The EIMO ecosystem is envisioned as a “system of systems” where embedded reference databases and real-time data streams from multiple input vectors continuously and seamlessly assess crew health and performance. EIMO will be designed to make recommendations to the CMO using multi-modal AI-based natural language processing and machine learning methods with interoperability to push/pull data within and between multiple vehicle and habitat architectures. DISCUSSION: Data flows and storage/retrieval capacity are severely constrained during space missions and the challenges will become even greater during exploration missions. Just as each past program from Mercury to the International Space Station (ISS) required rethinking the interaction between ground-based controllers and space-based crew, so too will future missions to the Moon and Mars. While the NASA High-Performance Spaceflight Computing Processor project aims to increase computational capacity by 100 times over current spaceflight computers, the projected deliverable still lags considerably behind what will be needed to enable an AI-driven CDSS. Restrictions in processing speed and data storage capacity, coupled with transmission bottlenecks and delays, necessitate definition and optimization of an integrated data architecture to enable a progressively autonomous medical capability.

Medical operations↗

NASA NTRS2020

NeMO-Net – The Neural Multi-Modal Observation & Training Network for Global Coral Reef Assessment

We present NeMO-Net, the Srst open-source deep convolutional neural network (CNN) and interactive learning and training software aimed at assessing the present and past dynamics of coral reef ecosystems through habitat mapping into 10 biological and physical classes. Shallow marine systems, particularly coral reefs, are under significant pressures due to climate change, ocean acidification, and other anthropogenic pressures, leading to rapid, often devastating changes, in these fragile and diverse ecosystems. Historically, remote sensing of shallow marine habitats has been limited to meter-scale imagery due to the optical effects of ocean wave distortion, refraction, and optical attenuation. NeMO-Net combines 3D cm-scale distortion-free imagery captured using NASA FluidCam and Fluid lensing remote sensing technology with low resolution airborne and spaceborne datasets of varying spatial resolutions, spectral spaces, calibrations, and temporal cadence in a supercomputer-based machine learning framework. NeMO-Net augments and improves the benthic habitat classification accuracy of low-resolution datasets across large geographic ad temporal scales using high-resolution training data from FluidCam.NeMO-Net uses fully convolutional networks based upon ResNet and ReSneNet to perform semantic segmentation of remote sensing imagery of shallow marine systems captured by drones, aircraft, and satellites, including WorldView and Sentinel. Deep Laplacian Pyramid Super-Resolution Networks (LapSRN) alongside Domain Adversarial Neural Networks (DANNs) are used to reconstruct high resolution information from low resolution imagery, and to recognize domain-invariant features across datasets from multiple platforms to achieve high classification accuracies, overcoming inter-sensor spatial, spectral and temporal variations.Finally, we share our online active learning and citizen science platform, which allows users to provide interactive training data for NeMO-Net in 2D and 3D, integrated within a deep learning framework. We present results from the PaciSc Islands including Fiji, Guam and Peros Banhos 1 1 2 1 3 1 where 24-class classification accuracy exceeds 91%.

Chirayath, Ved↗

NASA NTRS2017

NeMO-Net The Neural Multi-Modal Observation Training Network for Global Coral Reef Assessment

In the past decade, coral reefs worldwide have experienced unprecedented stresses due to climate change, ocean acidification, and anthropomorphic pressures, instigating massive bleaching and die-off of these fragile and diverse ecosystems. Furthermore, remote sensing of these shallow marine habitats is hindered by ocean wave distortion, refraction and optical attenuation, leading invariably to data products that are often of low resolution and signal-to-noise (SNR) ratio. However, recent advances in UAV and Fluid Lensing technology have allowed us to capture multispectral 3D imagery of these systems at sub-cm scales from above the water surface, giving us an unprecedented view of their growth and decay. Exploiting the fine-scaled features of these datasets, machine learning methods such as MAP, PCA, and SVM can not only accurately classify the living cover and morphology of these reef systems (below 8 error), but are also able to map the spectral space between airborne and satellite imagery, augmenting and improving the classification accuracy of previously low-resolution datasets.We are currently implementing NeMO-Net, the first open-source deep convolutional neural network (CNN) and interactive active learning and training software to accurately assess the present and past dynamics of coral reef ecosystems through determination of percent living cover and morphology. NeMO-Net will be built upon the QGIS platform to ingest UAV, airborne and satellite datasets from various sources and sensor capabilities, and through data-fusion determine the coral reef ecosystem makeup globally at unprecedented spatial and temporal scales. To achieve this, we will exploit virtual data augmentation, the use of semi-supervised learning, and active learning through a tablet platform allowing for users to manually train uncertain or difficult to classify datasets. The project will make use of Pythons extensive libraries for machine learning, as well as extending integration to GPU and High-End Computing Capability (HECC) on the Pleiades supercomputing cluster, located at NASA Ames. The project is being supported by NASAs Earth Science Technology Office (ESTO) Advanced Information Systems Technology (AIST-16) Program.

Remote Sensin↗

NASA NTRS2017

NeMO-Net: The Neural Multi-Modal Observation and Training Network for Global Coral Reef Assessment

In the past decade, coral reefs worldwide have experienced unprecedented stresses due to climate change, ocean acidification, and anthropomorphic pressures, instigating massive bleaching and die-off of these fragile and diverse ecosystems. Furthermore, remote sensing of these shallow marine habitats is hindered by ocean wave distortion, refraction and optical attenuation, leading invariably to data products that are often of low resolution and signal-to-noise (SNR) ratio. However, recent advances in UAV and Fluid Lensing technology have allowed us to capture multispectral 3D imagery of these systems at sub-cm scales from above the water surface, giving us an unprecedented view of their growth and decay. Exploiting the fine-scaled features of these datasets, machine learning methods such as MAP, PCA, and SVM can not only accurately classify the living cover and morphology of these reef systems (below 8 percent error), but are also able to map the spectral space between airborne and satellite imagery, augmenting and improving the classification accuracy of previously low-resolution datasets. We are currently implementing NeMO-Net, the first open-source deep convolutional neural network (CNN) and interactive active learning and training software to accurately assess the present and past dynamics of coral reef ecosystems through determination of percent living cover and morphology. NeMO-Net will be built upon the QGIS platform to ingest UAV, airborne and satellite datasets from various sources and sensor capabilities, and through data-fusion determine the coral reef ecosystem makeup globally at unprecedented spatial and temporal scales. To achieve this, we will exploit virtual data augmentation, the use of semi-supervised learning, and active learning through a tablet platform allowing for users to manually train uncertain or difficult to classify datasets. The project will make use of Pythons extensive libraries for machine learning, as well as extending integration to GPU and High-End Computing Capability (HECC) on the Pleiades supercomputing cluster, located at NASA Ames. The project is being supported by NASAs Earth Science Technology Office (ESTO) Advanced Information Systems Technology (AIST-16) Program.

NeMO-Net↗

NASA NTRS1989

A layered abduction model of perception: Integrating bottom-up and top-down processing in a multi-sense agent

A layered-abduction model of perception is presented which unifies bottom-up and top-down processing in a single logical and information-processing framework. The process of interpreting the input from each sense is broken down into discrete layers of interpretation, where at each layer a best explanation hypothesis is formed of the data presented by the layer or layers below, with the help of information available laterally and from above. The formation of this hypothesis is treated as a problem of abductive inference, similar to diagnosis and theory formation. Thus this model brings a knowledge-based problem-solving approach to the analysis of perception, treating perception as a kind of compiled cognition. The bottom-up passing of information from layer to layer defines channels of information flow, which separate and converge in a specific way for any specific sense modality. Multi-modal perception occurs where channels converge from more than one sense. This model has not yet been implemented, though it is based on systems which have been successful in medical and mechanical diagnosis and medical test interpretation.

Josephson, John R.↗

NASA NTRS2006

Single- and Multiple-Objective Optimization with Differential Evolution and Neural Networks

Genetic and evolutionary algorithms have been applied to solve numerous problems in engineering design where they have been used primarily as optimization procedures. These methods have an advantage over conventional gradient-based search procedures became they are capable of finding global optima of multi-modal functions and searching design spaces with disjoint feasible regions. They are also robust in the presence of noisy data. Another desirable feature of these methods is that they can efficiently use distributed and parallel computing resources since multiple function evaluations (flow simulations in aerodynamics design) can be performed simultaneously and independently on ultiple processors. For these reasons genetic and evolutionary algorithms are being used more frequently in design optimization. Examples include airfoil and wing design and compressor and turbine airfoil design. They are also finding increasing use in multiple-objective and multidisciplinary optimization. This lecture will focus on an evolutionary method that is a relatively new member to the general class of evolutionary methods called differential evolution (DE). This method is easy to use and program and it requires relatively few user-specified constants. These constants are easily determined for a wide class of problems. Fine-tuning the constants will off course yield the solution to the optimization problem at hand more rapidly. DE can be efficiently implemented on parallel computers and can be used for continuous, discrete and mixed discrete/continuous optimization problems. It does not require the objective function to be continuous and is noise tolerant. DE and applications to single and multiple-objective optimization will be included in the presentation and lecture notes. A method for aerodynamic design optimization that is based on neural networks will also be included as a part of this lecture. The method offers advantages over traditional optimization methods. It is more flexible than other methods in dealing with design in the context of both steady and unsteady flows, partial and complete data sets, combined experimental and numerical data, inclusion of various constraints and rules of thumb, and other issues that characterize the aerodynamic design process. Neural networks provide a natural framework within which a succession of numerical solutions of increasing fidelity, incorporating more realistic flow physics, can be represented and utilized for optimization. Neural networks also offer an excellent framework for multiple-objective and multi-disciplinary design optimization. Simulation tools from various disciplines can be integrated within this framework and rapid trade-off studies involving one or many disciplines can be performed. The prospect of combining neural network based optimization methods and evolutionary algorithms to obtain a hybrid method with the best properties of both methods will be included in this presentation. Achieving solution diversity and accurate convergence to the exact Pareto front in multiple objective optimization usually requires a significant computational effort with evolutionary algorithms. In this lecture we will also explore the possibility of using neural networks to obtain estimates of the Pareto optimal front using non-dominated solutions generated by DE as training data. Neural network estimators have the potential advantage of reducing the number of function evaluations required to obtain solution accuracy and diversity, thus reducing cost to design.

Rai, Man Mohan↗

NASA NTRS2018

Miniature Exercise Device-2 (MED-2): Preliminary ISS Evaluation Results for a Compact Motorized Resistive and Aerobic Rowing Exercise Device

Future human missions beyond Low Earth Orbit (LEO) will require onboard equipment to provide exercise capabilities for the crew to counter the adverse physiological effects of long-duration microgravity. To accomplish this within the physical constraints of a space vehicle or transit module, a single miniature device that provides both resistive and aerobic exercise modalities is required. To meet this need, Johnson Space Center’s (JSC) Software, Robotics, and Simulation Division (ER) developed the Miniature Exercise Device-2 (MED-2). MED-2 integrates a torque-controlled servomotor and a series-elastic actuator to provide highly-controllable load profiles and a large magnitude output performance in a very small package. This innovative technology is derived from years of JSC/ER design, development and operational experience with cutting-edge robotics, motor controllers, software and actuator/sensor miniaturization, including Robonaut 2 and MED-1. MED-2 was presented at the 2016 ISS R&D Conference. This is an update now that the last of six crewmembers will have completed planned MED-2 sessions on the International Space Station (ISS) in May 2018.Current state-of-the-art ISS exercise equipment consists of two treadmills, a resistive exercise device and two cycle ergometers with a total mass of several thousand pounds and a total volume of several cubic yards. This equipment has proven vital to mitigate the musculoskeletal and cardiovascular degradation effects of microgravity. However, due to the large operational volume and mass of these ISS devices, tailoring them for smaller vehicles, such as Orion, is not possible. In addition, each of the current ISS devices targets a single specific modality. Compared to the existing spaceflight (and even terrestrial) exercise equipment, MED-2 is a new archetype altogether. The combined features of compact size, multi-modality and high-performance is attributable to its innovative series elastic actuator and motor controller. Following its arrival on ISS in 2016, MED-2 was evaluated in two parts. The first and shorter evaluation was an engineering functional checkout of the hardware. As this was a novel exercise device previously never used on ISS, the initial checkout assessed the operation of the hardware and ensured the motion and dynamic range of the crew did not present any collision or other hazards. The second portion of the study collected the heart rates, kinematics and utilized operational volumes of six astronauts to determine the quality of both the resistive and aerobic exercise modalities as delivered by MED-2. Investigators from JSC Biomedical Research and Environmental Science Division (SK) and Glenn Research Center are currently evaluating the data and preparing preliminary results. For the resistive exercise modality, MED-2 demonstrated a range of constant resistive loads from 10-150 lbf. With a displacement range of 84 inches, the MED-2 accommodates users from 5th percentile Japanese female through 95th percentile American male for all of its certified exercises. The displacement measurement accuracy has also been verified within 2.5 percent full range. The crew was able to successfully perform all prescribed resistive exercises, except Goblet Squats which were not feasible with a constant load profile. For the aerobic exercise modality, MED-2 simulated a rowing motion with prescribed and user-selected resistance levels. It has demonstrated rates up to 60 strokes per minute on the ground. MED-2 loads and displacements performance are the same as those cited for the resistive modality. Although each of the crew was able to perform the prescribed aerobic rowing sets, there was considerable variability in the rowing motion among different crewmembers. Also, as expected, the crew was unable to get the full benefits of a typical terrestrial rowing stroke because the current configuration does not allow the user to reach past their feet. These observations have already informed the requirements for other microgravity rowing devices currently in development. One of the unique features of the MED-2 device is the intuitive touch-screen control system. This One Portal graphical user interface (GUI) was developed based on JSC/ER’s heritage knowledge and experience of developing and sustaining the current ISS exercise equipment. Through this interface, the crew easily performed prepared prescriptions as well as had the ability to adjust exercise modality, load and other exercise details such as number of repetitions and number of sets. This touch-screen and GUI fulfilled the MED-2 project goal to simplify the interaction between the user and the device. Furthermore, the extent to which MED-2 utilizes a touchscreen and GUI to control exercise equipment is unmatched among the existing ISS exercise devices. As a motorized device, MED-2 technology can provide a customizable force profile that can be varied as a function of strap displacement, strap velocity or a combination of these and other variables. During 2017, JSC/ER developed and flight-certified a resistive exercise algorithm that mimics the 1-G inertial effects of free-weights and enables adjustable eccentric-to-concentric loading ratios. Subsequent development will explore varying the load profiles and incorporating additional exercises beyond the current list of certified movements.

Exercise↗