SEARCH · Search NASA

Results for “Vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2025

ORBIT-2 Dataset for Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling

This dataset release corresponds to the work conducted in ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling, where large-scale AI methods were applied to improve climate and weather resolution. The collection integrates four widely used, publicly available datasets: ERA5, PRISM, DAYMET, and IMERG. To prepare the data for ORBIT-2 model training and evaluation, we applied a preprocessing pipeline that generates paired low-resolution and high-resolution samples, enabling supervised downscaling experiments. The transformation from coarse to fine scales was performed using bilinear regridding, consistent with the procedures described in WeatherBench2, a community benchmark for weather and climate AI models. This dataset supports the development and evaluation of foundation models designed for weather and climate downscaling at exascale. Additional details on methodology and applications can be found in Wang et al., ORBIT-2 (arXiv:2505.04802, 2025).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Monitoring installation of partially occluded subassemblies in modular construction factories using BIM, ray tracing, and computer vision

Modular and offsite construction methods are being increasingly adopted due to the advantages they offer in terms of project completion time, quality, and energy-efficiency. Despite these advantages, the current state of monitoring systems in modular construction factories highly relies on labor-intensive, subjective, and error-prone observational methods. A large body of research has aimed to automate the monitoring process using an array of sensors, such as IMUs and RFIDs, during the past two decades. Recently, computer vision-based methods have gained increasing interest as a non-intrusive technology to monitor the process inside modular construction factories. However, partial occlusion challenges have impeded their practical application on a large scale. This challenge is specifically important for monitoring the installation of subassemblies since they can obstruct the view of the monitoring camera, especially those that enable long-term monitoring like closed-circuit television (CCTV) fixed-view surveillance cameras. Here, this paper aims to address this challenge by proposing a novel computer vision-based method to monitor the installation of new subassemblies inside modular factories in highly occluded scenes. The proposed methodology identifies the subassemblies in the CCTV video footage using computer vision, analyzes the occlusions using BIM and ray casting techniques, and estimates the progress of assembly by comparing the BIM model with the detected subassemblies in the video. The proposed methodology was successfully validated on surveillance videos captured from a volumetric modular construction factory in the U.S., achieving 93% accuracy in identifying the installation of subassemblies. The results from this research show that the integration of BIM and computer vision is a promising method for monitoring the installation processes inside modular factories under severe occlusion.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Pre-training Vision Models for the Classification of Alerts from Wide-field Time-domain Surveys

Modern wide-field time-domain surveys facilitate the study of transient, variable and moving phenomena by conducting image differencing and relaying alerts to their communities. Machine learning tools have been used on data from these surveys and their precursors for more than a decade, and convolutional neural networks (CNNs), which make predictions directly from input images, saw particularly broad adoption through the 2010s. Since then, continually rapid advances in computer vision have transformed the standard practices around using such models. It is now commonplace to use standardized architectures pre-trained on large corpora of everyday images (e.g., ImageNet). In contrast, time-domain astronomy studies still typically design custom CNN architectures and train them from scratch. Here, we explore the effects of adopting various pre-training regimens and standardized model architectures on the performance of alert classification. We find that the resulting models match or outperform a custom, specialized CNN like what is typically used for filtering alerts. Moreover, our results show that pre-training on galaxy images from Galaxy Zoo tends to yield better performance than pre-training on ImageNet or training from scratch. We observe that the design of standardized architectures are much better optimized than the custom CNN baseline, requiring significantly less time and memory for inference despite having more trainable parameters. On the eve of the Legacy Survey of Space and Time and other image-differencing surveys, these findings advocate for a paradigm shift in the creation of vision models for alerts, demonstrating that greater performance and efficiency, in time and in data, can be achieved by adopting the latest practices from the computer vision field.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

DeepAndes: A Self-Supervised Vision Foundation Model for Multispectral Remote Sensing Imagery of the Andes

By mapping sites at large scales usingremotely sensed data, archaeologists can generate unique insights into long-term demographic trends, interregional social networks, and human adaptations in the past. Remote sensing surveys complement field-based approaches, and their reach can be especially great when combined with deep learning and computer vision techniques. However, conventional supervised deep learning methods face challenges in annotating fine-grained archaeological features at scale. In addition, while recent vision foundation models have shown remarkable success in learning large-scale remote sensing data with minimal annotations, most off-the-shelf solutions are designed for RGB images rather than multispectral satellite imagery, such as the eight-band data used in our study. In this article, we introduce DeepAndes, a transformer-based vision foundation model trained on three million multispectral satellite images, specifically tailored for Andean archaeology. DeepAndes incorporates a customized DINOv2 self-supervised learning algorithm optimized for eight-band multispectral imagery, marking the first foundation model designed explicitly for the Andes region. We evaluate its image understanding performance through imbalanced image classification, image instance retrieval, and pixel-level semantic segmentation tasks. Our experiments show that DeepAndes achieves superior F1 scores, mean average precision, and Dice scores in few-shot learning scenarios, significantly outperforming models trained from scratch or pretrained on smaller datasets. This underscores the effectiveness of large-scale self-supervised pretraining in archaeological remote sensing.

Guo, Junlin [Vanderbilt Univ., Nashville, TN (Unit↗

DOE OSTI2026

A Smart Vision-Aided RICH (Robotic Interface Control and Handling) System for VULCAN

High-flux neutron beams and high-efficiency detectors enable rapid neutron diffraction measurements at the Engineering Materials Diffractometer (VULCAN) at the Spallation Neutron Source (SNS), Oak Ridge National Laboratory (ORNL). To optimize beam time utilization, efficient sample exchange, alignment, and automated measurements are essential. Recent advances in artificial intelligence (AI) have expanded the capabilities of robotic systems. Here, we report the development of a Robotic Interactive Control and Handling (RICH) system for sample handling at VULCAN, designed to support high-throughput experiments and reduce overhead time. The RICH system employs a six-axis desktop robot integrated with AI-based computer vision models capable of recognizing and localizing samples in real time from instrument and depth-resolving cameras. Vision algorithms combine these detections to align samples with designated measurement positions or place them within complex sample environments such as furnaces. This integration of machine learning-assisted vision with robotic handling demonstrates the feasibility of autonomous sample detection and preparation, offering a pathway toward fully unmanned neutron scattering experiments.

automation↗

DOE OSTI2026

Building an AI-native Research Ecosystem for Experimental Particle Physics: A Community Vision

Experimental particle physics seeks to understand the universe by probing its fundamental particles and forces and exploring how they govern the large-scale processes that shape cosmic evolution. This whitepaper presents a vision for how Artificial Intelligence (AI) can accelerate discovery in this field. We outline grand challenges that must be addressed to enable transformative breakthroughs and describe how current and planned experimental facilities can implement this vision to advance our understanding of the vast and complex physical world from the smallest to the largest scales. We show how facilities currently under construction, such as the HL-LHC, DUNE and soon EIC, can both benefit from and serve as proving grounds for this vision, while also enabling a longer-term goal for how future experiments -- like FCC-ee at CERN, IceCube-Gen2, a Muon Collider in the U.S., and smaller to mid-scale projects -- can be fully AI-native. We describe how a truly national-scale collaboration, jointly managed across large funding partners, and involving both DOE laboratories and universities, can make this happen.

Aarrestad, Thea Klaeboe [ETH, Zurich (main)]↗

DOE OSTI2024

Improvements in Optical Surface Measurement Using Reflected Computer Vision Targets

Since 2021, NREL has been developing a system to measure heliostats by measuring the deflection of printed computer vision targets, called the Reflected Target Non-intrusive Assessment (ReTNA) [6], [7]. While this system will have lower resolution than a fringe deflectometry system, it has several important advantages that make it a complimentary technology: 2D surface slope measurement can be generated from a single image, it can operate in ambient lighting, target points can be directly located in 3D space with photogrammetry allowing for a non-flat target, and it's well suited to using a smaller target, and multiple images to measure larger optical surfaces. ReTNA has undergone several significant changes and improvements, described below. This talk will summarize new system layouts designed for commercial use, new computer vision algorithms used to automate the analysis process and validation campaigns for the ReTNA software.

computer vision↗

DOE OSTI2025

Advanced Fuels Campaign: Strategic Vision

The Advanced Fuels Campaign (AFC) is dedicated to propelling the United States to the forefront of nuclear fuel technology through a comprehensive strategic vision. The vision is structured around five key goals: driving U.S. leadership in nuclear fuel technology, expanding nuclear energy production from the existing fleet, completing the qualification basis for advanced reactor fuel technology, driving innovation in advanced nuclear fuel technology, and enabling a high-performing organization. By focusing on these goals, AFC aims to define and implement cutting-edge nuclear fuel technologies, support industry advancements, achieve critical fuel qualifications, foster innovative research, and ensure effective program management and stakeholder engagement. This strategic approach will solidify the United States’ position as a global leader in nuclear energy and fuel technology, while also preparing for the future of environmentally friendly and reliable nuclear power.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2025

Advanced Fuels Campaign: Strategic Vision

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2025

Automated Waterbox Inspection for Nuclear Power Plants Using Computer Vision - Based Change Detection

Nuclear power plant waterboxes require regular inspection for leaks, missing components, and structural damage during maintenance outages. Traditional manual inspection is time-consuming and poses safety risks from confined space entry. We developed an automated computer vision system for drone-based waterbox inspection in partnership with Florida Light and Power. Our approach uses feature detection and matching to identify critical changes between baseline and current inspection images, automatically flagging additions (leaks/debris), removals (missing plugs), and translations (displaced components) while compensating for drone movement and environmental variations. We systematically evaluated six feature matching methods, from classical approaches (SIFT+BF) to state-of-the-art neural networks (SuperPoint+SuperGlue), using both standard benchmarks (HPatches) and waterbox-specific validation with real-world augmentations. SuperPoint+SuperGlue achieved superior performance with 7.82 pixels RMSE and 100% success rate—2.8x better accuracy than our baseline. While the pre-trained model has commercial licensing restrictions for nuclear deployment, our findings validate this architecture for custom training. We implemented a real-time GUI demonstrating the SIFT+BF approach for immediate deployment, processing drone feeds at 30 FPS with color-coded change visualization. Future work includes training a custom SuperPoint+SuperGlue model on waterbox data and integrating Vision-Language Models for automated reporting and maintenance guidance.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

DOE OSTI2024

Structural and functional dynamics of human cone cGMP-phosphodiesterase important for photopic vision

Cone cGMP-phosphodiesterase (PDE6) is the key effector enzyme for daylight vision, and its properties are critical for shaping distinct physiology of cone photoreceptors. We determined the structures of human cone PDE6C in various liganded states by single-particle cryo-EM that reveal essential functional dynamics and adaptations of the enzyme. Our analysis exposed the dynamic nature of PDE6C association with its regulatory γ-subunit (Pγ) which allows openings of the catalytic pocket in the absence of phototransduction signaling, thereby controlling photoreceptor noise and sensitivity. We demonstrate evolutionarily recent adaptations of PDE6C stemming from residue substitutions in the Pγ subunit and the noncatalytic cGMP binding site and influencing the Pγ dynamics in holoPDE6C. Thus, our structural analysis sheds light on the previously unrecognized molecular evolution of the effector enzyme in cones that advances adaptation for photopic vision.

Science & Technology - Other Topics↗

DOE OSTI2025

Spatial profiling of the interplay between cell type- and vision-dependent transcriptomic programs in the visual cortex

How early sensory experience during “critical periods” of postnatal life affects the organization of the mammalian neocortex at the resolution of neuronal cell types is poorly understood. We previously reported that the functional and molecular profiles of layer 2/3 (L2/3) cell types in the primary visual cortex (V1) are vision-dependent [S. Chenget al.,Cell185, 311–327.e24 (2022)]. Here, we characterize the spatial organization of L2/3 cell types with and without visual experience. Spatial transcriptomic profiling based on 500 genes recapitulates the zonation of L2/3 cell types along the pial–ventricular axis in V1. By applying multitasking theory, we suggest that the spatial zonation of L2/3 cell types is linked to the continuous nature of their gene expression profiles, which can be represented as a 2D manifold bounded by three archetypal cell types. By comparing normally reared and dark reared L2/3 cells, we show that visual deprivation-induced transcriptomic changes comprise two independent gene programs. The first, induced specifically in the visual cortex, includes immediate-early genes and genes associated with metabolic processes. It manifests as a change in cell state that is orthogonal to cell-type-specific gene expression programs. By contrast, the second program impacts L2/3 cell-type identity, regulating a subset of cell-type-specific genes and shifting the distribution of cells within the L2/3 cell-type manifold. Through an integrated analysis of spatial transcriptomics with single-nucleus RNA-seq data, we describe how vision patterns cortical L2/3 cell types during the critical period.

Science & Technology - Other Topics↗

DOE OSTI2026

Uncertainty quantification of fireball features extracted from nuclear test films using computer vision

Films from the US’s historic nuclear testing era comprise the only extensive collection of imagery depicting high-yield detonations. These films offer unique insights into the characteristics of flows occurring on scales that are difficult to replicate experimentally, and they are a valuable source of data for the validation of models used to describe nuclear detonations. In recent work, we implemented modern computer vision and machine learning techniques to extract features of the fireball following nuclear detonation. With a training dataset of fireball films, we fine-tuned a You Only Look Once 11 (YOLO11) model to detect and track the fireball. Applied to a video, the outer bounding box produced in each frame by YOLO11 is used as an input prompt to Meta’s Segment Anything Model 2 (SAM2), which is shown to accurately predict the boundary of the fireball over time with high resolution. These state-of-the-art computer vision foundation models exhibit impressive visual accuracy in their results but lack an output of values that robustly quantify uncertainty in scientific applications. In this paper, we develop procedures for uncertainty quantification of extracted fireball features. We outline the application of a parallel attention mechanism to calculate uncertainty ranges that complement and better pose model validation data. This higher quality fireball validation data may serve to improve prognostic models describing nuclear detonations in support of nuclear forensic and emergency response activities.

Khristy, Joel [ORNL] (ORCID:0000000209963060)↗

DOE OSTI2026

ObstacleSense: Low-Power Neuromorphic Vision for Corridor Obstacle Awareness in Low-Level ADAS

The automotive industry’s pursuit of Level 5 autonomy is constrained by substantial perception-compute power requirements, often reaching 1, 000 + watts in full autonomy stacks. Reducing this energy burden requires rethinking perception not only at the high-end autonomy level, but also at the foundational Advanced Driver Assistance Systems (ADAS) level where low-power, safety-critical sensing can have broad impact. Neuromorphic vision provides a promising starting point: HD Dynamic Vision Sensors (DVS) can operate below 100 mW at the sensor level by reporting only asynchronous brightness changes. However, low-power sensing alone is insufficient if downstream perception reintroduces dense, energy-intensive computation. In particular, many event-driven object-detection pipelines still rely on CNN backbones, while purely spiking alternatives often trade away accuracy or ignore deployment constraints. We introduce ObstacleSense, a highly compact, CNN-free hybrid ANN–SNN framework for Level 0–1 forward-corridor obstacle awareness. Instead of performing full-scene object detection with a convolutional feature backbone, ObstacleSense targets the safety-critical question of whether the ego corridor is occupied and how far the nearest obstacle is. The architecture combines polarity-conditioned event encoding, lightweight temporal spiking dynamics, axial spatial mixing, and coarse-to-fine range estimation within a regular fixed-grid compute pattern. This design avoids the dense CNN backbone commonly used in event-based detection while maintaining a small state footprint suitable for eventual small-FPGA deployment. Before hardware mapping, we evaluate the software implementation using a model-side power proxy derived from MACs, weight and activation traffic, and spiking state updates under shared FP16 assumptions. On simulated CARLA event corpora, the deployment-oriented model achieves 0.9464 objectness F1, 0.9978 grid-level mAP, and 0.8987 m distance Mean Absolute Error at an estimated 1.92 mW proxy cost, while maintaining performance on unseen generalization test sequences.

Johnson-Scott, Zac [ORNL]↗

DOE OSTI2025

Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval

Code for ‘Lost in OCR Translation?’: robust document retrieval under degradation. Compares OCR-based, vision-only, and hybrid pipelines; includes SambaNova LLaMA Vision OCR, Nougat, and ViDoRe baselines. Provides QA data generation, RAG evaluation, and metrics (Levenshtein, nDCG@k, Recall@k, EM/F1) with reproducible scripts. Includes dataset guides

Bhattarai, Manish [Los Alamos National Labs]↗

DOE OSTI2025

Catalyst-Vision (PEM Catalyst Layer Image Analysis Tool) [SWR-25-100]

Catalyst-Vision (PEM Catalyst Layer Image Analysis Tool) provides an advanced Python-based tool, primarily designed for use in a Jupyter/Colab notebook, for the quantitative morphological analysis of pre-segmented shapes. While developed for analyzing PEM catalyst layers from microscopy, its methodology is suitable for characterizing any grayscale object provided on a uniform white background. The tool uses a robust computer vision pipeline based on the Euclidean Distance Transform and skeletonization to accurately measure local thickness and tortuosity, providing a comprehensive characterization of an object's geometry and internal texture. If you find this code useful, please cite our preprint as: Chan, Ai-Lin and Hayden, Steven and Harvey, Steven P. and Smeaton, Michelle and Okrucky, Caleb and Watt, John and Ulična, Soňa and Spurgeon, Steven and Jungjohann, Katherine and Alia, Shaun, Mechanism-informed breakdown: understanding degradation by controlling voltage hold patterns in PEM water electrolyzers. Preprint (2025).

Spurgeon, Steven [National Laboratory of the Rocki↗

DOE OSTI2025

Sequence length scaling in vision transformers for scientific images on frontier

Vision Transformers (ViTs) are pivotal for foundational models in scientific imagery, including Earth science applications, due to their capability to process large sequence lengths. While transformers for text have inspired scaling sequence lengths in ViTs, adapting these for ViTs introduces unique challenges. We develop distributed sequence parallelism for ViTs, enabling them to handle up to 1M tokens. Our approach, leveraging DeepSpeed-Ulysses and Long-Sequence-Segmentation with model sharding, is the first to apply sequence parallelism in ViT training, achieving a 94% batch scaling efficiency on 2,048 AMD-MI250X GPUs. Evaluating sequence parallelism in ViTs, particularly in models up to 10B parameters, highlighted substantial bottlenecks. We countered these with hybrid sequence, pipeline, and flash attention strategies, to scale beyond single GPU memory limits. Our method significantly enhances climate modeling accuracy by 20% in temperature predictions, marking the first training of a vision transformer model to convergence with a sequence length of 188K tokens, using full self-attention.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗

DOE OSTI2024

Visioning Energy: Science Fiction Author-Energy Researcher Collaboration Workshop Recap

The Visioning Energy: Science Fiction Author-Energy Researcher Collaboration Workshop brought together speculative fiction authors and NREL researchers to examine possible scenarios of the future. The goals of the workshop were to support out-of-the-box thinking and creative future visioning for the researchers and to provide authors with insights into the latest clean energy technologies and their potential. This report outlines the proceedings of the workshop and insights from the collaborative brainstorming activity, highlighting themes and opportunities for further exploration.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗