Search NASA⌕ Search

SEARCH · Search NASA

Results for “vision transformers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
NASA NTRS1987

Efficiency of a model human image code

Hypothetical schemes for neural representation of visual information can be expressed as explicit image codes. Here, a code modeled on the simple cells of the primate striate cortex is explored. The Cortex transform maps a digital image into a set of subimages (layers) that are bandpass in spatial frequency and orientation. The layers are sampled so as to minimize the number of samples and still avoid aliasing. Samples are quantized in a manner that exploits the bandpass contrast-masking properties of human vision. The entropy of the samples is computed to provide a lower bound on the code size. Finally, the image is reconstructed from the code. Psychophysical methods are derived for comparing the original and reconstructed images to evaluate the sufficiency of the code. When each resolution is coded at the threshold for detection artifacts, the image-code size is about 1 bit/pixel.

Watson, Andrew B.↗
NASA NTRS2015

“PowerCell”: The Interface Between Mars Resources and Human Exploration

The barriers to forming human settlements on Mars are high but surmountable within our lifetime. While the Apollo astronauts carried their life support with them, our success in exploring and forming settlements on Mars depends on our ability to use local Martian resources to generate the materials and conditions humans need to survive, so-called in situ resource utilization (ISRU). On Earth, biology provides us with food, shelter, oxygen, and other materials. Off-planet, synthetic biology will enable numerous parallel productions: optimized food production, water treatment, air treatment, environmental monitoring, regolith biomining, waste management, cell based biomaterial production, biocementation, and in situ synthesis based on received DNA sequences. How will the organisms responsible for these synthetic production systems obtain organic carbon and fixed nitrogen in the hostile Martian environment? We envision a synthetic-biology enabled Martian colony and introduce here the critical intermediate component a biological power source needed to transform the in situ resources found on Mars into biological feedstocks to enable growth of production organisms. Here, we present our first PowerCell, a photosynthetic and nitrogen-fixing filamentous cyanobacterium engineered to provide a carbon-rich fuel source for a biological life support system on Mars. We provide a vision of how the PowerCell system will operate in a Martian colony based on ground experiments and preparations for testing in space as a NASA secondary payload aboard the upcoming DLR Eu:CROPIS satellite mission experiments.

Rothschild, Lynn J.↗
NASA NTRS1989

Hybrid vision for automated spacecraft landing

A hyrbid real-time vision system concept is proposed for Mars lander guidance and control for the Mars Rover/Sample Return mission. The system includes digital and optical processing methods with high speed digital image warping to preprocess a video image for optical correlation. The system also includes position estimation by synthetic estimation filtering, image stabilization by joint transform optical correlation, and hazard identification from measurements of optical flow by sequential image subtraction.

Juday, Richard D.↗
NASA NTRS1992

Shared visions: Partnership of Rockwell International and NASA Cost Effectiveness Enhancements (CEE) for the space shuttle system integration program

As a result of limited resources and tight fiscal constraints over the past several years, the defense and aerospace industries have experienced a downturn in business activity. The impact of fewer contracts being awarded has placed a greater emphasis for effectiveness and efficiency on industry contractors. It is clear that a reallocation of resources is required for America to continue to lead the world in space and technology. The key to technological and economic survival is the transforming of existing programs, such as the Space Shuttle Program, into more cost efficient programs so as to divert the savings to other NASA programs. The partnership between Rockwell International and NASA and their joint improvement efforts that resulted in significant streamlining and cost reduction measures to Rockwell International Space System Division's work on the Space Shuttle System Integration Contract is described. This work was a result of an established Cost Effectiveness Enhancement (CEE) Team formed initially in Fiscal Year 1991, and more recently expanded to a larger scale CEE Initiative in 1992. By working closely with the customer in agreeing to contract content, obtaining management endorsement and commitment, and involving the employees in total quality management (TQM) and continuous improvement 'teams,' the initial annual cost reduction target was exceeded significantly. The CEE Initiative helped reduce the cost of the Shuttle Systems Integration contract while establishing a stronger program based upon customer needs, teamwork, quality enhancements, and cost effectiveness. This was accomplished by systematically analyzing, challenging, and changing the established processes, practices, and systems. This examination, in nature, was work intensive due to the depth and breadth of the activity. The CEE Initiative has provided opportunities to make a difference in the way Rockwell and NASA work together - to update the methods and processes of the organizations. The future success of NASA space programs and Rockwell hinges upon the ability to adopt new, more efficient and effective work processes. Efficiency, proficiency, cost effectiveness, and teamwork are a necessity for economic survival. Continuous improvement initiatives like the CEE are, and will continue to be, vehicles by which the road can be traveled with a vision to the future.

Bejmuk, Bohdan I.↗
NASA NTRS1994

Controlling telerobots with video data and compensating for time-delayed video using Omniview

Remote viewing is critical for teleoperations, but the inherent limitations of standard video reduce the operator's effectiveness. These limitations have been compensated for in many ways, from using the operator's adaptability, to augmenting his capability with feedback from a variety of sensors and simulations. Omniview can overcome some of these limitations and improve the operator's efficiency without adding additional sensors or computational burden. It can minimize the potential collisions with facility equipment, provide peripheral vision, and display multiple images simultaneously from a single input device. The Omniview technology provides electronic pan, tilt, magnify, and rotational orientation within a hemispherical field-of-view without any moving parts. Image sizes, viewing directions, scale, offset, etc., may be adjusted to fit operator needs. This paper discusses the derivation of the image transformation, the design of the electronics, and two applications to telepresence that are under development. These are Video Emulated Tweening (VET), and Manipulator Guidance and Positioning (ManGAP). The VET effort uses Omniview to compensate for time-delayed video in teleoperation of remote vehicles. In ManGAP two Omniview systems are used to provide two sets of orientation vectors to points in the field-of-view (FOV). These vectors then provide absolute position information to both control the position of the telerobot, and to avoid collisions with the work sight equipment.

Kuban, Dan↗
NASA NTRS2004

An X-Band Radar Terrain Feature Detection Method for Low-Altitude SVS Operations and Calibration Using LiDAR

To enable safe use of Synthetic Vision Systems at low altitudes, real-time range-to-terrain measurements may be required to ensure the integrity of terrain models stored in the system. This paper reviews and extends previous work describing the application of x-band radar to terrain model integrity monitoring. A method of terrain feature extraction and a transformation of the features to a common reference domain are proposed. Expected error distributions for the extracted features are required to establish appropriate thresholds whereby a consistency-checking function can trigger an alert. A calibration-based approach is presented that can be used to obtain these distributions. To verify the approach, NASA's DC-8 airborne science platform was used to collect data from two mapping sensors. An Airborne Laser Terrain Mapping (ALTM) sensor was installed in the cargo bay of the DC-8. After processing, the ALTM produced a reference terrain model with a vertical accuracy of less than one meter. Also installed was a commercial-off-the-shelf x-band radar in the nose radome of the DC-8. Although primarily designed to measure precipitation, the radar also provides estimates of terrain reflectivity at low altitudes. Using the ALTM data as the reference, errors in features extracted from the radar are estimated. A method to estimate errors in features extracted from the terrain model is also presented.

Young, Steve↗
DOE OSTI2026

Integrated System Planning: Emerging Software Requirements in the Power Industry

Power system planning software remains fragmented across organizational boundaries, with specialized tools for capacity expansion, production cost modeling, power flow, and dynamic analysis operating on incompatible data models and assumptions. This article argues that the fragmentation is not merely a technical problem but a predictable consequence of Conway's law: software architectures mirror the departmental structures within which they are developed. Regulatory milestones like Federal Energy Regulatory Commission (FERC) Order 888 formalized these divisions, but the roots trace back to the distinct engineering disciplines-mechanical, chemical, and electrical-that staffed generation and transmission planning departments in vertically integrated utilities. As the industry moves toward integrated system planning (ISP) that coordinates generation, transmission, and distribution investment decisions, the software ecosystem must evolve accordingly. We identify five categories of software requirements to enable this transition: coherent data inputs decoupled from individual applications, unified and extensible data schemas, modular component representations that support multiple abstraction levels, lifecycle management of planning datasets, and well-defined application programming interface (API) contracts that separate data exchange from algorithmic control. We examine how these requirements interact with three common workflow patterns-serial gate clearing, sequential multiapplication, and convergence oriented-and discuss the interface design principles each demands. We then outline a vision for platform-based planning architectures where specialized analytical services compose through standardized interfaces and where artificial intelligence (AI)/machine learning (ML) tools augment decision support within a disciplined software infrastructure. The practices proposed here offer a path from today's siloed tool collections toward collaborative planning ecosystems capable of handling the complexity of modern power system transformation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2020

The Science Enabled by a Dedicated Solar System Space Telescope

The National Academy Committee on Astrobiology and Planetary Science (CAPS) made a recommendation to study a large/medium-class dedicated space telescope for planetary science, going beyond the Discovery-class dedicated planetary space telescope endorsed in Visions and Voyages. Such a telescope would observe targets across the entire solar system, engaging a broad spectrum of the science community. It would ensure that the high-resolution, high-sensitivity observations of the solar system in visible and UV wavelengths revolutionized by the Hubble Space Telescope could be extended. A dedicated telescope for solar system science would a) transform our understanding of time-dependent phenomena in our solar system that cannot be studied currently under programs to observe and visit new targets and b) enable a comprehensive survey and spectral characterization of minor bodies across the solar system, which requires a large time allocation not supported by existing facilities. The time-domain phenomena to be explored are critically reliant on UV observations and include: interaction of planetary magnetospheres with the solar wind and internal plasma sources, Venus and giant planet atmospheric dynamics, icy satellite geologic activity and surface evolution, cometary evolution, and evolving ring phenomena. This paper presents science themes and key questions that require a long-lasting space telescope dedicated to planetary science that can capture high-quality, consistent data at the required cadences that are free from the complicating effects of the terrestrial atmosphere and differences across observing facilities. Such a telescope would have excellent synergy with astrophysical facilities by placing planetary discoveries made by astrophysics assets in temporal context, as well as triggering detailed follow-up observations using larger telescopes. The telescope would also support future missions to the Ice Giants, Ocean Worlds, and minor bodies across the solar system by placing the results of such targeted missions in the context of longer records of temporal activities and larger sample populations.

C. L. Young↗
DOE OSTI2025

Tracking Dendritic Growth in Hydrogen-Based Hematite Reduction via Computer Vision

The reduction of hematite to metallic iron using hydrogen (H2) as a reducing agent presents a promising pathway for decarbonizing steel production. In this study, we employ a combination of in situ confocal scanning laser microscopy (CSLM) and advanced computer vision techniques to quantitatively analyze dendritic growth of ferrite during H2-based reduction of iron oxide at high temperatures. A workflow integrating Watershed Image Segmentation (WIS) and Lucas-Kanade Optical Flow (LKOF) is developed to extract both global and local kinetic information from time-resolved micrograph sequences. H2 reduction experiments conducted at 1400 degrees C and 1500 degrees C demonstrate a clear correlation between temperature and reduction rate, as evidenced by accuracy of fitted Johnson-Mehl-Avrami-Kolmogorov (JMAK) parameters. Optical flow analysis further elucidates the anisotropic and branched nature of dendritic growth, providing spatially resolved velocity fields that correlate well with global transformation kinetics. The proposed methodology demonstrates strong agreement with experimental measurements and literature values, offering a robust framework for automated image-based analysis to study kinetics through microstructural evolution in the reduction of iron ore, and likely other reaction-diffusion phenomena.

08 HYDROGEN↗
NASA NTRSDate not supplied

Urban Air Mobility (UAM) Vision Concept of Operations (ConOps) UAM Maturity Level (UML)-4 Overview

The Vision ConOps is intended as a foundation to engage members of the UAM community and provide a consensus on the future vision of UAM operations. It provides a concept for more detailed discussion and a basis for the exploration of ideas using a common framework to inform the continued development and integration of UAM as part of the broader transportation system. Advanced Air Mobility (AAM) encompasses a range of innovative aviation technologies (small drones, electric aircraft, automated air traffic management, etc.) that are transforming aviation’s role in everyday life, including the movement of goods and people. Urban Air Mobility (UAM) represents one of the most exciting and complex AAM concepts with highly automated aircraft, providing commercial services to the public over densely populated cities.

Brian Hill↗
NASA NTRS2014

Visual Image Sensor Organ Replacement

This innovation is a system that augments human vision through a technique called "Sensing Super-position" using a Visual Instrument Sensory Organ Replacement (VISOR) device. The VISOR device translates visual and other sensors (i.e., thermal) into sounds to enable very difficult sensing tasks. Three-dimensional spatial brightness and multi-spectral maps of a sensed image are processed using real-time image processing techniques (e.g. histogram normalization) and transformed into a two-dimensional map of an audio signal as a function of frequency and time. Because the human hearing system is capable of learning to process and interpret extremely complicated and rapidly changing auditory patterns, the translation of images into sounds reduces the risk of accidentally filtering out important clues. The VISOR device was developed to augment the current state-of-the-art head-mounted (helmet) display systems. It provides the ability to sense beyond the human visible light range, to increase human sensing resolution, to use wider angle visual perception, and to improve the ability to sense distances. It also allows compensation for movement by the human or changes in the scene being viewed.

Maluf, David A.↗
DOE OSTI2025

Poster Abstract: Leveraging Large Language Models to Reveal Interpretable Cooling Behaviors from Smart Thermostat Data

Frequent heatwaves and hot summers increasingly challenge occupant comfort, health, and energy grid stability. Addressing these challenges requires a detailed understanding of household cooling behaviors, such as thermostat adjustments and adaptive responses to extreme conditions. Traditional analyses often rely on aggregated numerical metrics that overlook subtle but important household-specific variations. In this study, we introduce a generalizable methodology that integrates large language models (LLMs) with vision capabilities to enable scalable and detailed analysis of residential thermostat data. Using Ecobee's Donate Your Data (DYD) dataset—which provides five-minute records of indoor temperatures, thermostat setpoints, and HVAC runtimes—we focus on two U.S. cities with contrasting summer climates : Austin (TX) and Phoenix (AZ). Because raw time-series data are not well suited for direct LLM analysis, we transform them into visual representations, such as daily indoor temperature trajectories and weekly runtime histograms, to better capture behavioral variations. Leveraging LLMs' visual interpretation, we extract descriptive behavioral features, including temperature preferences, time-of-day cooling orientation, anticipatory versus reactive heatwave responses, and behavioral consistency. These semantic features support unsupervised clustering to identify distinct occupant archetypes at scale, revealing differences—such as morning-centric anticipatory coolers versus households that shift toward warmer setpoints during heatwaves—that can inform demand response, resilience planning, and health-aware interventions. By converting raw numerical data into interpretable behavioral patterns, this methodology enables scalable and practical analysis of occupant behavior, supporting actionable insights for comfort, resilience, and energy management.

Nihar, Kopal↗
NASA NTRS2013

Ten Years of Land Cover Change on the California Coast Detected using Landsat Satellite Image Analysis

Landsat satellite imagery was analyzed to generate a detailed record of 10 years of vegetation disturbance and regrowth for Pacific coastal areas of Marin and San Francisco Counties. The Landsat Ecosystem Disturbance Adaptive Processing System (LEDAPS) methodology, a transformation of Tasseled-Cap data space, was applied to detected changes in perennial coastal shrubland, woodland, and forest cover from 1999 to 2009. Results showed several principal points of interest, within which extensive contiguous areas of similar LEDAPS vegetation change (either disturbed or restored) were detected. Regrowth areas were delineated as burned forest areas in the Point Reyes National Seashore (PRNS) from the 1995 Vision Fire. LEDAPS-detected disturbance patterns on Inverness Ridge, PRNS in areas observed with dieback of tanoak and bay laurel trees was consistent with defoliation by sudden oak death (Phytophthora ramorum). LEDAPS regrowth pixels were detected over much of the predominantly grassland/herbaceous cover of the Olema Valley ranchland near PRNS. Extensive restoration of perennial vegetation cover on Crissy Field, Baker Beach and Lobos Creek dunes in San Francisco was identified. Based on these examples, the LEDAPS methodology will be capable of fulfilling much of the need for continual, low-cost monitoring of emerging changes to coastal ecosystems.

Land Cover Change↗
NASA NTRS1981

Laser electro-optic system for rapid three-dimensional /3-D/ topographic mapping of surfaces

It is pointed out that the generic utility of a robot in a factory/assembly environment could be substantially enhanced by providing a vision capability to the robot. A standard videocamera for robot vision provides a two-dimensional image which contains insufficient information for a detailed three-dimensional reconstruction of an object. Approaches which supply the additional information needed for the three-dimensional mapping of objects with complex surface shapes are briefly considered and a description is presented of a laser-based system which can provide three-dimensional vision to a robot. The system consists of a laser beam array generator, an optical image recorder, and software for controlling the required operations. The projection of a laser beam array onto a surface produces a dot pattern image which is viewed from one or more suitable perspectives. Attention is given to the mathematical method employed, the space coding technique, the approaches used for obtaining the transformation parameters, the optics for laser beam array generation, the hardware for beam array coding, and aspects of image acquisition.

Altschuler, M. D.↗
NASA NTRS2003

Terrain Model Registration for Single Cycle Instrument Placement

This paper presents an efficient and robust method for registration of terrain models created using stereo vision on a planetary rover. Our approach projects two surface models into a virtual depth map, rendering the models as they would be seen from a single range sensor. Correspondence is established based on which points project to the same location in the virtual range sensor. A robust norm of the deviations in observed depth is used as the objective function, and the algorithm searches for the rigid transformation which minimizes the norm. An initial coarse search is done using rover pose information from odometry and orientation sensing. A fine search is done using Levenberg-Marquardt. Our method enables a planetary rover to keep track of designated science targets as it moves, and to hand off targets from one set of stereo cameras to another. These capabilities are essential for the rover to autonomously approach a science target and place an instrument in contact in a single command cycle.

Deans, Matthew↗
NASA NTRS1990

Extensions of algebraic image operators: An approach to model-based vision

Researchers extend their previous research on a highly structured and compact algebraic representation of grey-level images which can be viewed as fuzzy sets. Addition and multiplication are defined for the set of all grey-level images, which can then be described as polynomials of two variables. Utilizing this new algebraic structure, researchers devised an innovative, efficient edge detection scheme. An accurate method for deriving gradient component information from this edge detector is presented. Based upon this new edge detection system researchers developed a robust method for linear feature extraction by combining the techniques of a Hough transform and a line follower. The major advantage of this feature extractor is its general, object-independent nature. Target attributes, such as line segment lengths, intersections, angles of intersection, and endpoints are derived by the feature extraction algorithm and employed during model matching. The algebraic operators are global operations which are easily reconfigured to operate on any size or shape region. This provides a natural platform from which to pursue dynamic scene analysis. A method for optimizing the linear feature extractor which capitalizes on the spatially reconfiguration nature of the edge detector/gradient component operator is discussed.

Lerner, Bao-Ting↗
NASA NTRS2007

Stellar Imager

The Stellar Imager (SI) is one of NASA's "Vision Missions" - concepts for future, space-based, strategic missions that could enormously increase our capabilities for observing the Cosmos. SI is designed as a UV/Optical Interferometer which will enable 0.1 milli-arcsecond (mas) spectral imaging of stellar surfaces and, via asteroseismology, stellar interiors and of the Universe in general. The ultra-sharp images of the Stellar Imager will revolutionize our view of many dynamic astrophysical processes by transforming point sources into extended sources, and snapshots into evolving views. SI, with a characteristic angular resolution of 0.1 milli-arcseconds at 2000 Angstroms, represents an advance in image detail of several hundred times over that provided by the Hubble Space Telescope. The Stellar Imager will zoom in on what today-with few exceptions - we only know as point sources, revealing processes never before seen, thus providing a tool as fundamental to astrophysics as the microscope is to the study of life on Earth. SI's science focuses on the role of magnetism in the Universe, particularly on magnetic activity on the surfaces of stars like the Sun. It's prime goal is to enable long-term forecasting of solar activity and the space weather that it drives, in support of the Living With a Star program in the Exploration Era. SI will also revolutionize our understanding of the formation of planetary systems, of the habitability and climatology of distant planets, and of many magneto-hydrodynamically controlled processes in the Universe. Stellar Imager is included as a "Flagship and Landmark Discovery Mission" in the 2005 Sun Solar System Connection (SSSC) Roadmap and as a candidate for a "Pathways to Life Observatory" in the Exploration of the Universe Division (EUD) Roadmap (May, 2005) and as such is a candidate mission for the 2025-2030 timeframe. An artist's drawing of the current "baseline" concept for SI is presented.

Carpenter, Kenneth↗
DOE OSTI2025

VISION: a modular AI assistant for natural human-instrument interaction at scientific user facilities

Scientific user facilities, such as synchrotron beamlines, are equipped with a wide array of hardware and software tools that require a codebase for human-computer-interaction. This often necessitates developers to be involved to establish connection between users/researchers and the complex instrumentation. The advent of generative AI presents an opportunity to bridge this knowledge gap, enabling seamless communication and efficient experimental workflows. Here we present a modular architecture for the Virtual Scientific Companion by assembling multiple AI-enabled cognitive blocks that each scaffolds large language models (LLMs) for a specialized task. With VISION, we performed LLM-based operation on the beamline workstation with low latency and demonstrated the first voice-controlled experiment at an x-ray scattering beamline. The modular and scalable architecture allows for easy adaptation to new instruments and capabilities. Development on natural language-based scientific experimentation is a building block for an impending future where a science exocortex—a synthetic extension to the cognition of scientists—may radically transform scientific practice and discovery.

36 MATERIALS SCIENCE↗