Search NASA⌕ Search

SEARCH · Search NASA

Results for “vision systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Avionic Pictorial Tunnel-/Pathway-/Highway-In-The-Sky Workshops

In 1994-96, Langley Research Center held a series of interactive workshops investigating highway-in-the-sky concepts, which enable precise flight path control. These workshops brought together government and industry display designers and pilots to discuss and fly various concepts in an iterative manner. The primary emphasis of the first workshops was the utility and usability of pathways and the pros and cons of various features available. The final workshops were focused on the specific applications to the eXternal Visibility System (XVS) of the NASA High-speed Research Program, which was concerned with replacement of the forward windows in a High-speed Civil Transport with electronic displays and high resolution video cameras to enable a "No-Droop" configuration. The primary concerns in the XVS application were the prevention of display clutter and obscuration of hazards, as the camera image was the primary means of traffic separation in clear visibility conditions. These concerns were not so prominent in the first workshops, which assumed a Synthetic Vision System application in which hazard locations are known and obscuration is handled easily. The resulting consensus concept has been used since in simulation and flight test activities of many Government programs. and other concepts have been influenced by the workshop discussions.

Parrish, Russell V.↗
NASA NTRS2004

Pathway Design Effects on Synthetic Vision Head-Up Displays

NASA s Synthetic Vision Systems (SVS) project is developing technologies with practical applications that will eliminate low visibility conditions as a causal factor to civil aircraft accidents while replicating the operational benefits of clear day flight operations, regardless of the actual outside visibility condition. A major thrust of the SVS project involves the development/demonstration of affordable, certifiable display configurations that provide intuitive out-the-window terrain and obstacle information with advanced pathway guidance for transport aircraft. This experiment evaluated the influence of different tunnel and guidance concepts upon pilot situation awareness (SA), mental workload, and flight path tracking performance for Synthetic Vision display concepts using a Head-Up Display (HUD). Two tunnel formats (dynamic, minimal) were evaluated against a baseline condition (no tunnel) during simulated IMC approaches to Reno-Tahoe International airport. Two guidance cues (tadpole, follow-me aircraft) were also evaluated to assess their influence on the tunnel formats. Results indicated that the presence of a tunnel on an SVS HUD had no effect on flight path performance but that it did have significant effects on pilot SA and mental workload. The dynamic tunnel concept with the follow-me aircraft guidance symbol produced the lowest workload and provided the highest SA among the tunnel concepts evaluated.

Kramer, Lynda J.↗
NASA NTRS2005

Advanced Pathway Guidance Evaluations on a Synthetic Vision Head-Up Display

NASA's Synthetic Vision Systems (SVS) project is developing technologies with practical applications to potentially eliminate low visibility conditions as a causal factor to civil aircraft accidents while replicating the operational benefits of clear day flight operations, regardless of the actual outside visibility condition. A major thrust of the SVS project involves the development/demonstration of affordable, certifiable display configurations that provide intuitive out-the-window terrain and obstacle information with advanced guidance for commercial and business aircraft. This experiment evaluated the influence of different pathway and guidance display concepts upon pilot situation awareness (SA), mental workload, and flight path tracking performance for Synthetic Vision display concepts using a Head-Up Display (HUD). Two pathway formats (dynamic and minimal tunnel presentations) were evaluated against a baseline condition (no tunnel) during simulated instrument meteorological conditions approaches to Reno-Tahoe International airport. Two guidance cues (tadpole, follow-me aircraft) were also evaluated to assess their influence. Results indicated that the presence of a tunnel on an SVS HUD had no effect on flight path performance but that it did have significant effects on pilot SA and mental workload. The dynamic tunnel concept with the follow-me aircraft guidance symbol produced the lowest workload and provided the highest SA among the tunnel concepts evaluated.

Kramer, Lynda J.↗
NASA NTRS1985

Control of a free-flying robot manipulator system

The goal of the research is to develop and test control strategies for a self-contained, free flying space robot. Such a robot would perform operations in space similar to those currently handled by astronauts during extravehicular activity (EVA). The focus of the work is to develop and carry out a program of research with a series of physical Satellite Robot Simulator Vehicles (SRSV's), two-dimensionally freely mobile laboratory models of autonomous free-flying space robots such as might perform extravehicular functions associated with operation of a space station or repair of orbiting satellites. The development of the SRSV and of some of the controller subsystems are discribed. The two-link arm was fitted to the SRSV base, and researchers explored the open-loop characteristics of the arm and thruster actuators. Work began on building the software foundation necessary for use of the on-board computer, as well as hardware and software for a local vision system for target identification and tracking.

Alexander, H.↗
NASA NTRS2012

Single Lens Dual-Aperture 3D Imaging System: Color Modeling

In an effort to miniaturize a 3D imaging system, we created two viewpoints in a single objective lens camera. This was accomplished by placing a pair of Complementary Multi-band Bandpass Filters (CMBFs) in the aperture area. Two key characteristics about the CMBFs are that the passbands are staggered so only one viewpoint is opened at a time when a light band matched to that passband is illuminated, and the passbands are positioned throughout the visible spectrum, so each viewpoint can render color by taking RGB spectral images. Each viewpoint takes a different spectral image from the other viewpoint hence yielding a different color image relative to the other. This color mismatch in the two viewpoints could lead to color rivalry, where the human vision system fails to resolve two different colors. The difference will be closer if the number of passbands in a CMBF increases. (However, the number of passbands is constrained by cost and fabrication technique.) In this paper, simulation predicting the color mismatch is reported.

bipartite filter↗
DOE OSTI2023

Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition

Deep neural networks have surpassed human performance in key visual challenges such as object recognition, but require a large amount of energy, computation, and memory. In contrast, spiking neural networks (SNNs) have the potential to improve both the efficiency and biological plausibility of object recognition systems. Here we present a SNN model that uses spike-latency coding and winner-take-all inhibition (WTA-I) to efficiently represent visual stimuli using multi-scale parallel processing. Mimicking neuronal response properties in early visual cortex, images were preprocessed with three different spatial frequency (SF) channels, before they were fed to a layer of spiking neurons whose synaptic weights were updated using spike-timing-dependent-plasticity. We investigate how the quality of the represented objects changes under different SF bands and WTA-I schemes. We demonstrate that a network of 200 spiking neurons tuned to three SFs can efficiently represent objects with as little as 15 spikes per neuron. Furthermore, studying how core object recognition may be implemented using biologically plausible learning rules in SNNs may not only further our understanding of the brain, but also lead to novel and efficient artificial vision systems.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Automated Waterbox Inspection for Nuclear Power Plants Using Computer Vision - Based Change Detection

Nuclear power plant waterboxes require regular inspection for leaks, missing components, and structural damage during maintenance outages. Traditional manual inspection is time-consuming and poses safety risks from confined space entry. We developed an automated computer vision system for drone-based waterbox inspection in partnership with Florida Light and Power. Our approach uses feature detection and matching to identify critical changes between baseline and current inspection images, automatically flagging additions (leaks/debris), removals (missing plugs), and translations (displaced components) while compensating for drone movement and environmental variations. We systematically evaluated six feature matching methods, from classical approaches (SIFT+BF) to state-of-the-art neural networks (SuperPoint+SuperGlue), using both standard benchmarks (HPatches) and waterbox-specific validation with real-world augmentations. SuperPoint+SuperGlue achieved superior performance with 7.82 pixels RMSE and 100% success rate—2.8x better accuracy than our baseline. While the pre-trained model has commercial licensing restrictions for nuclear deployment, our findings validate this architecture for custom training. We implemented a real-time GUI demonstrating the SIFT+BF approach for immediate deployment, processing drone feeds at 30 FPS with color-coded change visualization. Future work includes training a custom SuperPoint+SuperGlue model on waterbox data and integrating Vision-Language Models for automated reporting and maintenance guidance.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗
DOE OSTI2020

A Benthic Habitat Monitoring Approach for Marine and Hydrokinetic Sites (Final Technical Report)

This final technical report summarizes the work completed as part of the Standardized and Cost-Effective Benthic Habitat Mapping and Monitoring Tools for MHK Environmental Assessments project funded by U.S. Department of Energy (DOE) under contract DE-EE007826 to Integral Consulting Inc. The overall goal of this project was to demonstrate a consistent, repeatable, and semi-automated seafloor and sediment mapping approach for rapidly characterizing benthic physical and biological/habitat conditions to support marine environmental assessments for marine and hydrokinetic energy sites. The approach evaluated combines sediment profile imaging and plan view (SPI/PV) technology with multibeam echosounder surveys as an effective and low-cost benthic habitat mapping protocol. A key innovation was the development of a semi-automated computer vision system that standardizes the extraction of data from the SPI/PV images. Other elements of the project were SPI camera hardware modifications, including the design and fabrication of a prototype “power” SPI camera to improve camera prism penetration in firm substrates, and outreach to agency regulators and other stakeholders on this habitat mapping approach. This technical report consists of five main subsections that summarize: 1) the benthic mapping approach; 2) benthic mapping results from the three areas’ surveys; 3) the development and performance of the image processing algorithms; 4) SPI camera hardware improvements and prototype testing; and 5) the regulatory outreach efforts.

16 TIDAL AND WAVE POWER↗
NASA NTRS2011

Recent Developments in the Design, Capabilities and Autonomous Operations of a Lightweight Surface Manipulation System and Test-bed

The first generation of a versatile high performance device for performing payload handling and assembly operations on planetary surfaces, the Lightweight Surface Manipulation System (LSMS), has been designed and built. Over the course of its development, conventional crane type payload handling configurations and operations have been successfully demonstrated and the range of motion, types of operations and the versatility greatly expanded. This enhanced set of 1st generation LSMS hardware is now serving as a laboratory test-bed allowing the continuing development of end effectors, operational techniques and remotely controlled and automated operations. This paper describes the most recent LSMS and test-bed development activities, that have focused on two major efforts. The first effort was to complete a preliminary design of the 2nd generation LSMS that has the capability for limited mobility and can reposition itself between lander decks, mobility chassis, and fixed base locations. A major portion of this effort involved conducting a study to establish the feasibility of, and define, the specifications for a lightweight cable-drive waist joint. The second effort was to continue expanding the versatility and autonomy of large planetary surface manipulators using the 1st generation LSMS as a test-bed. This has been accomplished by increasing manipulator capabilities and efficiencies through both design changes and tool and end effector development. A software development effort has expanded the operational capabilities of the LSMS test-bed to include; autonomous operations based on stored paths, use of a vision system for target acquisition and tracking, and remote command and control over a communications bridge.

Dorsey, John T.↗
NASA NTRS2013

Robonaut 2 - Preparing for Intra-Vehicular Mobility on the International Space Station

Robonaut 2 (R2) has been undergoing experimental trials on board the International Space Station (ISS) for more than a year. This upper-body anthropomorphic robotic system shown in Figure 1 has been making steady progress after completing its initial checkout. R2 demonstrated free space motion, physically interacted with its human crew mates, manipulated interfaces on its task board and has even used its first tool. This steady growth in capability will lead R2 to its next watershed milestone. Developers are currently testing prototype robotic climbing appendages and a battery backpack in preparation of sending flight versions of both subsystems to the ISS in late 2013. Upon integration of its new components, R2 will be able to go mobile inside the space station with a twofold agenda. First, R2 will learn to maneuver in microgravity in the best possible laboratory for such a task. Second, it will start providing early payback to the ISS program by helping with intra-vehicular (IVA) maintenance tasks. The experience gained inside the ISS will be invaluable in reducing risk when R2 moves to its next stage and is deployed as an extra-vehicular (EVA) tool. Even on its current fixed base stanchion, R2 has already shown its capability of performing several maintenance tasks on the ISS. It has measured the air flow through one of the stations vents and provided previously unavailable real time flow data to ground operators. R2 has cleaned its first handrail, exciting some crew members that perhaps Saturday morning housekeeping on the station may someday become a task they can hand off to their robotic colleague. Other tasks, including using radio frequency identification (RFID) tools for inventory tasks or vacuuming air filters, have also been suggested and will be explored. Once mobile, R2 will take on these tasks and more to free up crew time for more important science and exploration pursuits. In addition to task exploration, research and testing is happening on orbit to prepare for R2 mobility operations. The current vision system in R2 s head is being used to identify and localize IVA handrails throughout the US Lab and ground control software is being updated and integrated in advance of supporting mobility operations.

Badger, Julia↗
DOE OSTI2025

Large‐Scale 2D Perovskite Nanocrystals Photodetector Array via Ultrasonic Spray Synthesis

Abstract 2D perovskite (PVSK) single crystals have received significant attention due to their unique optical and optoelectronic properties. However, current synthesis methods face limitations, particularly in large‐area fabrication, which remain critical barriers to practical applications. In this study, the synthesis of red/green/purple‐blue‐colored 2D PVSK nanocrystals over a large area (4‐inch wafer) and the fabrication of high‐performance photodetector arrays are presented via a facile yet efficient spray‐coating approach with a liquid‐bridge transport effect. The photodetector array achieves 100% working yield, high photo‐responsivity (1.5 × 10 6 A W −1 ) and specific‐detectivity (1.1 × 10 16 Jones) with competitive photomapping characteristics. An intelligent vision system for automatic shape recognition is further demonstrated with a recognition rate exceeding 90%. This study provides significant advances in the scalable synthesis of nanoscale 2D PVSK crystals, their integration into large‐area optoelectronic devices, and their potential use in artificial‐intelligence systems.

Lee, Yoon Ho [Davidson School of Chemical Engineer↗
DOE OSTI2022

Using dorsal surface for individual identification of dairy calves through 3D deep learning algorithms

Advances in machine learning techniques have allowed the development of computer vision systems (CVS) that can accurately predict several phenotypes of interest for livestock operations. In this context, 3D images taken from a top-down view are particularly useful for estimating body condition score, growth development, and body biometrics in cattle. Frequently, such CVS rely on identification (ID) systems, such as electronic tags, as a way to match animal ID and the predicted phenotype. However, the same 3D images used to predict body weight and other animal biometrics could be adopted for animal recognition as well. Such alternative would optimize CVS to recognize animal ID and monitor growth development simultaneously while leveraging the same hardware infrastructure. Furthermore, this strategy could be used to recognize animals with similar color patterns. Nonetheless, growing animals are continuously changing body shape, which could limit its use as an invariant feature for pattern recognition. Thus, the objectives of this study were: (1) to compare algorithms for different 3D object representations to identify individual animals; and (2) to evaluate how short-term changes in body shape due to animal growth affect the predictive performance of these algorithms. For objective 1, the algorithms were trained (n = 4,558) and tested (n = 1,139) using images from 38 Holstein calves. For objective 2, we designed three different experiments using images (n = 2,347) from five Holstein calves taken over six weeks during their growing period, always training and testing on different weeks. Each experiment evaluated how changing a different parameter of the image capturing procedure affected the predictive ability of the trained algorithms. In the first experiment, we varied the total number of images per animal in the training set; in the second experiment, we varied the number of weeks while keeping a fixed number of images in the training set; and in the third experiment, we skipped weeks between images in the training and test sets. The F 1 score for objective (1) was up to 0.804 when testing with the last frames of each video, and up to 0.959 when using random frames for testing. For objective (2), the F 1 score was up to 0.947 for the first experiment when using 130 images per animal; up to 0.979 for the second experiment when using all five weeks; and up to 0.917 when not skipping weeks between training and testing. In conclusion, these results show that deep learning algorithms can be used to identify individual animals through their dorsal area 3D surfaces, and, from our experiments using calves in their growing period, that they are robust enough to account for changes in body shape and size, making them a promising tool for animal recognition during growth.

3D neural networks↗
DOE OSTI2023

GWAS identifies candidate genes controlling adventitious rooting in Populus trichocarpa

Adventitious rooting (AR) is critical to the propagation, breeding, and genetic engineering of trees. The capacity for plants to undergo this process is highly heritable and of a polygenic nature; however, the basis of its genetic variation is largely uncharacterized. To identify genetic regulators of AR, we performed a genome-wide association study (GWAS) using 1148 genotypes of Populus trichocarpa. GWASs are often limited by the abilities of researchers to collect precise phenotype data on a high-throughput scale; to help overcome this limitation, we developed a computer vision system to measure an array of traits related to adventitious root development in poplar, including temporal measures of lateral and basal root length and area. GWAS was performed using multiple methods and significance thresholds to handle non-normal phenotype statistics and to gain statistical power. These analyses yielded a total of 277 unique associations, suggesting that genes that control rooting include regulators of hormone signaling, cell division and structure, reactive oxygen species signaling, and other processes with known roles in root development. Numerous genes with uncharacterized functions and/or cryptic roles were also identified. These candidates provide targets for functional analysis, including physiological and epistatic analyses, to better characterize the complex polygenic regulation of AR.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

3D reconstruction and neural rendering for adversarial machine learning

While evasion attacks on computer vision systems have been widely studied, creating attacks that remain effective under significant changes in viewpoint continues to be challenging. Traditional approaches often rely on affine transformations of images, but these approaches degrade at larger perspective shifts and often produce unrealistic or ineffective perturbations. Recent methods use differentiable renderers to improve viewpoint robustness, but they typically depend on manually constructed 3D models. We introduce a semi-automated pipeline that generates physically printable and perspective-invariant adversarial patches using only a small set of 2D images. Our method integrates 3D reconstruction, neural rendering, adversarial patch optimization, and an object detection victim model into a unified workflow. We use 2D Gaussian Splatting for high fidelity mesh reconstruction and FlexPara for surface parameterization that produces texture maps suitable for patch editing. Together, these components form a fully differentiable pipeline in PyTorch3D that links texture modification to model outputs, enabling efficient optimization of patches that remain effective across many viewpoints. The complete process, from image capture to patch printing and physical evaluation, can be completed within a few hours. We demonstrate the effectiveness of the resulting patches through attacks on the YOLOv8 object detection model and discuss remaining challenges and opportunities for improving robustness and scalability.

Singhvi, Vivaan [ORNL] (ORCID:0009000586288221)↗
DOE OSTI2025

AI-powered municipal solid waste management: a comprehensive review from generation to utilization

The accumulation of municipal solid waste (MSW) continues to rise due to burgeoning population, rapid global urbanization and economic growth, intensifying ecological concerns associated with landfills and greenhouse gas (GHG) emissions. Over the past 2 decades, global waste generation has surged by 50%, with one-third remaining uncollected and about 70% sent to landfills. This review examines the critical role of integrating emerging technologies, such as advanced sensors and artificial intelligence (AI), into end-to-end MSW management to alleviate landfill burdens. The suitability of various AI tools for different stages of MSW management is assessed, alongside the deployment of advanced sensors including hyperspectral cameras, computer vision systems, and internet of things (IoT) devices for material identification. Applications of genetic algorithms and reinforcement learning for optimizing collection routes, reducing costs, and lowering emissions are highlighted. Life cycle assessment (LCA) across all stages of MSW management is also reviewed, along with future trends in leveraging generative AI, natural language processing (NLP), and agent-based AI systems to analyze waste generation patterns and public sentiment. Efficient collection and handling can be enhanced through route optimization with geographic information systems and real-time bin-level monitoring. Furthermore, sensor-embedded, real-time object detection systems paired with robotics enable material characterization and automated sorting, thereby lowering costs and diverting waste from landfills into value-added products for diverse industrial sectors including packaging, chemicals, textiles, metals and glass, transportation, and electronics industries. Without intervention, global waste is projected to reach 4.54 billion tons by 2050, contributing direct economic costs of $\$$400 billion and roughly 2.38 billion tons of CO 2 -equivalent emissions annually. This review demonstrates how AI-driven, end-to-end solutions for MSW management can mitigate economic and environmental challenges, while directly supporting the United Nations Sustainable Development (UNDP) goals related to innovation and infrastructure (SDG 9), sustainable cities (SDG 11), responsible consumption and production (SDG 12), and climate action (SDG 13).

09 BIOMASS FUELS↗
DOE OSTI2022

Development of an Energy Efficient and Cost Effective Autonomous Vehicle Research Platform

Commercialization of autonomous vehicle technology is a major goal of the automotive industry, thus research in this space is rapidly expanding across the world. However, despite this high level of research activity, literature detailing a straightforward and cost-effective approach to the development of an AV research platform is sparse. To address this need, we present the methodology and results regarding the AV instrumentation and controls of a 2019 Kia Niro which was developed for a local AV pilot program. This platform includes a drive-by-wire actuation kit, Aptiv electronically scanning radar, stereo camera, MobilEye computer vision system, LiDAR, inertial measurement unit, two global positioning system receivers to provide heading information, and an in-vehicle computer for driving environment perception and path planning. Robotic Operating System software is used as the system middleware between the instruments and the autonomous application algorithms. After selection, installation, and integration of these components, our results show successful utilization of all sensors, drive-by-wire functionality, a total additional power* consumption of 242.8 Watts (*Typical), and an overall cost of $118,189 USD, which is a significant saving compared to other commercially available systems with similar functionality. This vehicle continues to serve as our primary AV research and development platform.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS1978

Vision-based guidance for an automated roving vehicle

A controller designed to guide an automated vehicle to a specified target without external intervention is described. The intended application is to the requirements of planetary exploration, where substantial autonomy is required because of the prohibitive time lags associated with closed-loop ground control. The guidance algorithm consists of a set of piecewise-linear control laws for velocity and steering commands, and is executable in real time with fixed-point arithmetic. The use of a previously-reported object tracking algorithm for the vision system to provide position feedback data is described. Test results of the control system on a breadboard rover at the Jet Propulsion Laboratory are included.

Griffin, M. D.↗
NASA NTRS1982

Definition of display/control requirements for assault transport night/adverse weather capability

A Helicopter Night Vision System was developed to improve low-altitude night and/or adverse weather assult transport capabilities. Man-in-the-loop simulation experiments were performed to define the minimum display and control requirements for the assult transport mission and investigate forward looking infrared sensor requirements, along with alternative displays such as panel mounted displays (PMD) helmet mounted displays (HMD), and integrated control display units. Also explored were navigation requirements, pilot/copilot interaction, and overall cockpit arrangement. Pilot use of an HMD and copilot use of a PMD appear as both the preferred and most effective night navigation combination.

Milelli, R. J.↗