Search NASA⌕ Search

SEARCH · Search NASA

Results for “Computer Vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,441 records · Page 80
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Vision Foundation Models in Remote Sensing: A survey

Artificial intelligence (AI) technologies have profoundly transformed the field of remote sensing (RS), revolutionizing data collection, processing, and analysis. Traditionally reliant on manual interpretation and task-specific models, RS research has been significantly enhanced by the advent of foundation models (FMs)—large-scale pretrained AI models capable of performing a wide array of tasks with unprecedented accuracy and efficiency. This article provides a comprehensive survey of FMs in the RS domain. We categorize these models based on their architectures, pretraining datasets, and methodologies. Through detailed performance comparisons, we highlight emerging trends and the significant advancements achieved by those FMs. Additionally, we discuss technical challenges, practical implications, and future research directions, addressing the need for high-quality data, computational resources, and improved model generalization. Our research also finds that pretraining methods, particularly self-supervised learning (SSL) techniques like contrastive learning (CL) and masked autoencoders (MAEs), remarkably enhance the performance and robustness of FMs. This survey aims to serve as a resource for researchers and practitioners by providing a panorama of advances and promising pathways for the continued development and application of FMs in RS.

data models↗
NASA NTRS2019

The TASAR Project: Launching Aviation on an Optimized Route Toward Aircraft Autonomy

The Traffic Aware Strategic Aircrew Request (TASAR) concept applies onboard automation for the purpose of advising the pilot of route modifications that would be beneficial to the flight. Leveraging onboard computing platforms with connectivity to avionics and diverse data sources on and off the aircraft, TASAR introduces a new, powerful capability for in-flight trajectory management to the cockpit and its flight crew that is anticipated to induce a significant culture change in airspace operations. Flight crews empowered by TASAR and its derivative technologies could transform from today’s flight plan followers to proactive trajectory managers, taking an initial critical step towards increasing autonomy in the airspace system. TASAR was developed as a catalyst for operational autonomy, a future vision where the responsibilities and authorities of trajectory management reside with the aircraft operator and are distributed among participating aircraft, thus fulfilling a vision dating back decades and enabling a fully scalable airspace system. This NASA Technical Paper maps TASAR to its foundational vision and traces its research and development from initial concept generation to an operational evaluation by a U.S. airline in revenue service, the final stage before technology transfer and commercialization.

Wing, David J.↗
DOE OSTI2014

An automated miniature robotic vehicle inspection system

A novel, autonomous reconfigurable robotic inspection system for quantitative NDE mapping is presented. The system consists of a fleet of wireless (802.11g) miniature robotic vehicles, each approximately 175 × 125 × 85 mm with magnetic wheels that enable them to inspect industrial structures such as storage tanks, chimneys and large diameter pipe work. The robots carry one of a number of payloads including a two channel MFL sensor, a 5 MHz dry coupled UT thickness wheel probe and a machine vision camera that images the surface. The system creates an NDE map of the structure overlaying results onto a 3D model in real time. The authors provide an overview of the robot design, data fusion algorithms (positioning and NDE) and visualization software.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

A systems engineering vision for floating offshore wind cost optimization

The U.S. offshore wind resource potential is immense, and often in close proximity to densely-populated coastal load centers. In many U.S. coastal areas, water depths favor the deployment of floating over fixed-bottom offshore wind technology. Floating offshore wind plants have the potential to be cost-competitive with fixed-bottom installations, but because the technology has not yet been deployed at commercial scale, it is not clear when and with what configurations this potential cost parity can be achieved. This article first reviews the state of floating offshore wind technology and deployments to identify key gaps that must be addressed to bring down the overall cost of energy produced. The article then puts forth a long-term vision for a research program and design methodology that may be able to push floating wind plants toward a lower levelized cost of energy than fixed-bottom offshore wind. The method involves a fully integrated systems-engineering and techno-economic design approach to capture the complex interactions between the physics, manufacturing, installation, and operation of floating wind turbines to achieve transformational cost reductions. The approach also envisions multifidelity and uncertainty management strategies to examine the most robust and viable concepts in the design trade-space. To better focus the computational resources, engineering lessons learned from existing offshore wind systems and concept studies are used to develop a set of criteria that can be applied to prefilter candidate technology building blocks that have the greatest cost reduction potential.

17 WIND ENERGY↗
DOE OSTI2026

Tula: Optimizing Time, Cost, and Generalization in Distributed Large-Batch Training

Distributed training increases the number of batches processed per iteration either by scaling-out (adding more nodes) or scaling-up (increasing the batch-size). However, the largest configuration does not necessarily yield the best performance. Horizontal scaling introduces additional communication overhead, while vertical scaling is constrained by computation cost and device memory limits. Thus, simply increasing the batch-size leads to diminishing returns: training time and cost decrease initially but eventually plateaus, creating a knee-point in the time/cost vs. batch-size pareto curve. The optimal batch-size therefore depends on the underlying model, data and available compute resources. Large batches also suffer from worse model quality due to the well-known “generalization gap”. In this paper, we present Tula, an online service that automatically optimizes time, cost, and convergence quality for large-batch training of convolutional models. It combines parallel-systems modeling with statistical performance prediction to identify the optimal batchsize. Tula predicts training time and cost within 7.5−14% error across multiple models, and achieves up to 20× overall speedup and improves test accuracy by ≈9% on average over standard large-batch training on various vision tasks, thus successfully mitigating the generalization gap and accelerating training at the same time.

Tyagi, Sahil [ORNL] (ORCID:0009000783144745)↗
DOE OSTI2022

Meta-optic accelerators for object classifiers

Rapid advances in deep learning have led to paradigm shifts in a number of fields, from medical image analysis to autonomous systems. These advances, however, have resulted in digital neural networks with large computational requirements, resulting in high energy consumption and limitations in real-time decision-making when computation resources are limited. Here, we demonstrate a meta-optic–based neural network accelerator that can off-load computationally expensive convolution operations into high-speed and low-power optics. In this architecture, metasurfaces enable both spatial multiplexing and additional information channels, such as polarization, in object classification. End-to-end design is used to co-optimize the optical and digital systems, resulting in a robust classifier that achieves 93.1% accurate classification of handwriting digits and 93.8% accuracy in classifying both the digit and its polarization state. This approach could enable compact, high-speed, and low-power image and information processing systems for a wide range of applications in machine vision and artificial intelligence.

42 ENGINEERING↗
DOE OSTI2022

Lens-free compound eye cameras based on angle-sensitive meta-surfaces

A lens-free ultrathin imaging sensor using a compound-eye vision modality can be formed by forming a metasurface on each pixel of an array of pixels in a solid state imaging sensor. The metasurface can be configured to form a diffraction grating that directs light incident on the metasurface at a predefined angle to excite surface plasmon polaritons into the solid state imaging sensor and light incident at any other angle is reflected or diffracted away from the metasurface. Each pixel of the imaging sensor can be configured using the metasurface to only receive light incident from a different portion of a field of view. A computational imaging system can be used to construct the image from the individual pixels.

Paiella, Roberto↗
NASA NTRS1989

Using multiple sensors for printed circuit board insertion

As more and more activities are performed in space, there will be a greater demand placed on the information handling capacity of people who are to direct and accomplish these tasks. A promising alternative to full-time human involvement is the use of semi-autonomous, intelligent robot systems. To automate tasks such as assembly, disassembly, repair and maintenance, the issues presented by environmental uncertainties need to be addressed. These uncertainties are introduced by variations in the computed position of the robot at different locations in its work envelope, variations in part positioning, and tolerances of part dimensions. As a result, the robot system may not be able to accomplish the desired task without the help of sensor feedback. Measurements on the environment allow real time corrections to be made to the process. A design and implementation of an intelligent robot system which inserts printed circuit boards into a card cage are presented. Intelligent behavior is accomplished by coupling the task execution sequence with information derived from three different sensors: an overhead three-dimensional vision system, a fingertip infrared sensor, and a six degree of freedom wrist-mounted force/torque sensor.

Sood, Deepak↗
NASA NTRS2007

High-Performance, Radiation-Hardened Electronics for Space Environments

The Radiation Hardened Electronics for Space Environments (RHESE) project endeavors to advance the current state-of-the-art in high-performance, radiation-hardened electronics and processors, ensuring successful performance of space systems required to operate within extreme radiation and temperature environments. Because RHESE is a project within the Exploration Technology Development Program (ETDP), RHESE's primary customers will be the human and robotic missions being developed by NASA's Exploration Systems Mission Directorate (ESMD) in partial fulfillment of the Vision for Space Exploration. Benefits are also anticipated for NASA's science missions to planetary and deep-space destinations. As a technology development effort, RHESE provides a broad-scoped, full spectrum of approaches to environmentally harden space electronics, including new materials, advanced design processes, reconfigurable hardware techniques, and software modeling of the radiation environment. The RHESE sub-project tasks are: SelfReconfigurable Electronics for Extreme Environments, Radiation Effects Predictive Modeling, Radiation Hardened Memory, Single Event Effects (SEE) Immune Reconfigurable Field Programmable Gate Array (FPGA) (SIRF), Radiation Hardening by Software, Radiation Hardened High Performance Processors (HPP), Reconfigurable Computing, Low Temperature Tolerant MEMS by Design, and Silicon-Germanium (SiGe) Integrated Electronics for Extreme Environments. These nine sub-project tasks are managed by technical leads as located across five different NASA field centers, including Ames Research Center, Goddard Space Flight Center, the Jet Propulsion Laboratory, Langley Research Center, and Marshall Space Flight Center. The overall RHESE integrated project management responsibility resides with NASA's Marshall Space Flight Center (MSFC). Initial technology development emphasis within RHESE focuses on the hardening of Field Programmable Gate Arrays (FPGA)s and Field Programmable Analog Arrays (FPAA)s for use in reconfigurable architectures. As these component/chip level technologies mature, the RHESE project emphasis shifts to focus on efforts encompassing total processor hardening techniques and board-level electronic reconfiguration techniques featuring spare and interface modularity. This phased approach to distributing emphasis between technology developments provides hardened FPGA/FPAAs for early mission infusion, then migrates to hardened, board-level, high speed processors with associated memory elements and high density storage for the longer duration missions encountered for Lunar Outpost and Mars Exploration occurring later in the Constellation schedule.

Keys, Andrew S.↗
DOE OSTI2024

Toward the validation of crowdsourced experiments for lightness perception

Crowdsource platforms have been used to study a range of perceptual stimuli such as the graphical perception of scatterplots and various aspects of human color perception. Given the lack of control over a crowdsourced participant’s experimental setup, there are valid concerns on the use of crowdsourcing for color studies as the perception of the stimuli is highly dependent on the stimulus presentation. Here, we propose that the error due to a crowdsourced experimental design can be effectively averaged out because the crowdsourced experiment can be accommodated by the Thurstonian model as the convolution of two normal distributions, one that is perceptual in nature and one that captures the error due to variability in stimulus presentation. Based on this, we provide a mathematical estimate for the sample size needed to produce a crowdsourced experiment with the same power as the corresponding in-person study. We tested this claim by replicating a large-scale, crowdsourced study of human lightness perception with a diverse sample with a highly controlled, in-person study with a sample taken from psychology undergraduates. Our claim was supported by the replication of the results from the latter. These findings suggest that, with sufficient sample size, color vision studies may be completed online, giving access to a larger and more representative sample. With this framework at hand, experimentalists have the validation that choosing either many online participants or few in person participants will not sacrifice the impact of their results.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2003

Three-dimensional ocular kinematics during eccentric rotations: evidence for functional rather than mechanical constraints

Previous studies have reported that the translational vestibuloocular reflex (TVOR) follows a three-dimensional (3D) kinematic behavior that is more similar to visually guided eye movements, like pursuit, rather than the rotational VOR (RVOR). Accordingly, TVOR rotation axes tilted with eye position toward an eye-fixed reference frame rather than staying relatively fixed in the head like in the RVOR. This difference arises because, contrary to the RVOR where peripheral image stability is functionally important, the TVOR like pursuit and saccades cares to stabilize images on the fovea. During most natural head and body movements, both VORs are simultaneously activated. In the present study, we have investigated in rhesus monkeys the 3D kinematics of the combined VOR during yaw rotation about eccentric axes. The experiments were motivated by and quantitatively compared with the predictions of two distinct hypotheses. According to the first (fixed-rule) hypothesis, an eye-position-dependent torsion is computed downstream of a site for RVOR/TVOR convergence, and the combined VOR axis would tilt through an angle that is proportional to gaze angle and independent of the relative RVOR/TVOR contributions to the total eye movement. This hypothesis would be consistent with the recently postulated mechanical constraints imposed by extraocular muscle pulleys. According to the second (image-stabilization) hypothesis, an eye-position-dependent torsion is computed separately for the RVOR and the TVOR components, implying a processing that takes place upstream of a site for RVOR/TVOR convergence. The latter hypothesis is based on the functional requirement that the 3D kinematics of the combined VOR should be governed by the need to keep images stable on the fovea with slip on the peripheral retina being dependent on the different functional goals of the two VORs. In contrast to the fixed-rule hypothesis, the data demonstrated a variable eye-position-dependent torsion for the combined VOR that was different for synergistic versus antagonistic RVOR/TVOR interactions. Furthermore, not only were the eye-velocity tilt slopes of the combined VOR as much as 10 times larger than what would be expected based on extraocular muscle pulley location, but also eye velocity during antagonistic RVOR/TVOR combinations often tilted opposite to gaze. These results are qualitatively and quantitatively consistent with the image-stabilization hypothesis, suggesting that the eye-position-dependent torsion is computed separately for the RVOR and the TVOR and that the 3D kinematics of the combined VOR are dependent on functional rather than mechanical constraints.

Non-NASA Center↗
NASA NTRS2008

Model of Image Artifacts from Dust Particles

A mathematical model of image artifacts produced by dust particles on lenses has been derived. Machine-vision systems often have to work with camera lenses that become dusty during use. Dust particles on the front surface of a lens produce image artifacts that can potentially affect the performance of a machine-vision algorithm. The present model satisfies a need for a means of synthesizing dust image artifacts for testing machine-vision algorithms for robustness (or the lack thereof) in the presence of dust on lenses. A dust particle can absorb light or scatter light out of some pixels, thereby giving rise to a dark dust artifact. It can also scatter light into other pixels, thereby giving rise to a bright dust artifact. For the sake of simplicity, this model deals only with dark dust artifacts. The model effectively represents dark dust artifacts as an attenuation image consisting of an array of diffuse darkened spots centered at image locations corresponding to the locations of dust particles. The dust artifacts are computationally incorporated into a given test image by simply multiplying the brightness value of each pixel by a transmission factor that incorporates the factor of attenuation, by dust particles, of the light incident on that pixel. With respect to computation of the attenuation and transmission factors, the model is based on a first-order geometric (ray)-optics treatment of the shadows cast by dust particles on the image detector. In this model, the light collected by a pixel is deemed to be confined to a pair of cones defined by the location of the pixel s image in object space, the entrance pupil of the lens, and the location of the pixel in the image plane (see Figure 1). For simplicity, it is assumed that the size of a dust particle is somewhat less than the diameter, at the front surface of the lens, of any collection cone containing all or part of that dust particle. Under this assumption, the shape of any individual dust particle artifact is the shape (typically, circular) of the aperture, and the contribution of the particle to the attenuation factor for a given pixel is the fraction of the cross-sectional area of the collection cone occupied by the particle. Assuming that dust particles do not overlap, the net transmission factor for a given pixel is calculated as one minus the sum of attenuation factors contributed by all dust particles affecting that pixel. In a test, the model was used to synthesize attenuation images for random distributions of dust particles on the front surface of a lens at various relative aperture (F-number) settings. As shown in Figure 2, the attenuation images resembled dust artifacts in real test images recorded while the lens was aimed at a white target.

Willson, Reg↗
NASA NTRS1993

Emerging computer technologies and the news media of the future

The media environment of the future may be dramatically different from what exists today. As new computing and communications technologies evolve and synthesize to form a global, integrated communications system of networks, public domain hardware and software, and consumer products, it will be possible for citizens to fulfill most information needs at any time and from any place, to obtain desired information easily and quickly, to obtain information in a variety of forms, and to experience and interact with information in a variety of ways. This system will transform almost every institution, every profession, and every aspect of human life--including the creation, packaging, and distribution of news and information by media organizations. This paper presents one vision of a 21st century global information system and how it might be used by citizens. It surveys some of the technologies now on the market that are paving the way for new media environment.

Vrabel, Debra A.↗
DOE OSTI2024

Anti-distortion bioinspired camera with an inhomogeneous photo-pixel array

The bioinspired camera, comprising a single lens and a curved image sensor—a photodiode array on a curved surface—, was born of flexible electronics. Its economical build lends itself well to space-constrained machine vision applications. The curved sensor, much akin to the retina, helps image focusing, but the curvature also creates a problem of image distortion, which can undermine machine vision tasks such as object recognition. Here we report an anti-distortion single-lens camera, where 4096 silicon photodiodes arrayed on a curved surface in a nonuniform pattern assimilated to the distorting optics are the key to anti-distortion engineering. That is, the photo-pixel distribution pattern itself is warped in the same manner as images are warped, which correctively reverses distortion. Acquired images feature no appreciable distortion across a 120° horizontal view, as confirmed by their neural-network recognition accuracies. This distortion correction via photo-pixel array reconfiguration is a form of in-sensor computing.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2021

Assessment of the High Flux Isotope Reactor Cybersecurity Initiative

Recent cyber-attacks on industrial control systems, and inadvertent exposure of nuclear plant systems to cyber-exploits underscore the need for plant operators to adopt and deploy cyber-security defense solutions made for industrial control systems. Of increasing concern is the fact that international cyber hackers are beginning to target critical infrastructure, and because these more modern controls systems depend on advanced use of digital systems, they are more vulnerable than ever before to cyber-attacks. Traditional cyber defense strategies and products that have been available for decades are tailored for use on IT or corporate networks but can cause interruptions and catastrophic damage when deployed on industrial control system networks. The Department of Energy (DOE) Office of Nuclear Energy established the Gateway for Accelerated Innovation in Nuclear (GAIN) program to provide private companies pursuing innovative nuclear energy technologies with access to the technical support necessary to move toward commercialization. One of these GAIN small business vouchers was awarded to Dragos, Inc. to enable collaboration with Oak Ridge National Laboratory (ORNL) to evaluate the Dragos Platform on a production nuclear reactor test bed, hence laying the path for future commercial adoption. The vision was to provide a guide for industrial operators on implementing an industrial monitoring solution and to show how these solutions can be deployed without causing safety and reliability issues. This report documents the results of the collaboration between ORNL and Dragos, Inc.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

The VIBES Are Shifting: Assessing Emergent Capabilities in Multi-Modal Models

Researchers assessing open-source domains such as the internet, and particularly those studying information conflict, often have no single prescribed workflow. In the course of their research, they may need to perform a diverse array of tasks far beyond simply identifying an ever-changing set of objects. These analytical tasks can include ascertaining the provenance of images, understand an image in the context of accompanying text-based data, or identifying indicators of digital image manipulation. They must further be able to do this at the scale of tens of thousands of images or more. Traditional machine vision models have typically lacked the flexibility and breadth of performance sufficient for these needs. The research team from Pacific Northwest National Laboratory assessed the performance of a single baseline CLIP ViT-L model against a series of analytical tasks relevant for the study of online information conflict.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1994

Comparisons of luminaires: Efficacies and system design

Lighting designs for architectural (aesthetic) purposes, vision and safety, and plant growth have many features in common but several crucial ones that are not. The human eye is very sensitive to the color (wavelength) of light, whereas plants are less so. There are morphological reactions, particularly to the red and blue portions of the light spectrum but, in general, plants appear to accept and use light for photosynthesis everywhere over the PAR region of the spectrum. In contrast, the human eye interprets light intensity on a logarithmic scale, making people insensitive to significant differences of light intensity. As a rough rule, light intensity must change by 30 to 50% for the human eye to recognize the difference. Plants respond much more linearly to light energy, at least at intensities below photosynthetic saturation. Thus, intensity differences not noticeable to the human eye can have significant effects on total plant growth and yield, and crop timing. These factors make luminaire selection and lighting system design particularly important when designing supplemental lighting systems for plant growth. Supplemental lighting for plant growth on the scale of commercial greenhouses is a relatively expensive undertaking. Light intensities are often much higher than required for task (vision) lighting, which increases both installation and operating costs. However, and especially in the northern regions of the United States (and Canada, Europe, etc.), supplemental lighting during winter may be necessary to produce certain crops (e.g., tomatoes) and very useful to achieve full plant growth potential and crop timing with most other greenhouse crops. Operating costs over the life of a luminaire typically will exceed the initial investment, making lighting efficacy a major consideration. This report reviews tests completed to evaluate the efficiencies of various commercially-available High-Pressure Sodium luminaires, and then describes the results of using a commercial lighting design computer program, Lumen-Micro, to explore how to place luminaires within greenhouses and plant growth chambers to achieve light (PAR) uniformity and relatively high lighting efficacies. Several suggestions are presented which could encourage systematic design of plant lighting systems.

Albright, L. D.↗
DOE OSTI2020

Deep learning approaches for neural decoding across architectures and recording modalities

Decoding behavior, perception or cognitive state directly from neural signals is critical for brain–computer interface research and an important tool for systems neuroscience. In the last decade, deep learning has become the state-of-the-art method in many machine learning tasks ranging from speech recognition to image segmentation. The success of deep networks in other domains has led to a new wave of applications in neuroscience. In this article, we review deep learning approaches to neural decoding. Here, we describe the architectures used for extracting useful features from neural recording modalities ranging from spikes to functional magnetic resonance imaging. Furthermore, we explore how deep learning has been leveraged to predict common outputs including movement, speech and vision, with a focus on how pretrained deep networks can be incorporated as priors for complex decoding targets like acoustic speech or images. Deep learning has been shown to be a useful tool for improving the accuracy and flexibility of neural decoding across a wide range of tasks, and we point out areas for future scientific development.

59 BASIC BIOLOGICAL SCIENCES↗