SEARCH · Search NASA

Results for “vision transformers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

All sources NASA NTRS DOE OSTI

NASA NTRS1990

Networks for image acquisition, processing and display

The human visual system comprises layers of networks which sample, process, and code images. Understanding these networks is a valuable means of understanding human vision and of designing autonomous vision systems based on network processing. Ames Research Center has an ongoing program to develop computational models of such networks. The models predict human performance in detection of targets and in discrimination of displayed information. In addition, the models are artificial vision systems sharing properties with biological vision that has been tuned by evolution for high performance. Properties include variable density sampling, noise immunity, multi-resolution coding, and fault-tolerance. The research stresses analysis of noise in visual networks, including sampling, photon, and processing unit noises. Specific accomplishments include: models of sampling array growth with variable density and irregularity comparable to that of the retinal cone mosaic; noise models of networks with signal-dependent and independent noise; models of network connection development for preserving spatial registration and interpolation; multi-resolution encoding models based on hexagonal arrays (HOP transform); and mathematical procedures for simplifying analysis of large networks.

Ahumada, Albert J., Jr.↗

NASA NTRS2003

Design of a Borescope for Extravehicular Non-Destructive Applications

Anomalies such as corrosion, structural damage, misalignment, cracking, stress fiactures, pitting, or wear can be detected and monitored by the aid of a borescope. A borescope requires a source of light for proper operation. Today s current lighting technology market consists of incandescent lamps, fluorescent lamps and other types of electric arc and electric discharge vapor lamp. Recent advances in LED technology have made LEDs viable for a number of applications, including vehicle stoplights, traffic lights, machine-vision-inspection, illumination, and street signs. LEDs promise significant reduction in power consumption compared to other sources of light. This project focused on comparing images taken by the Olympus IPLEX, using two different light sources. One of the sources is the 50-W internal metal halide lamp and the other is a 1 W LED placed at the tip of the insertion tube. Images acquired using these two light sources were quantitatively compared using their histogram, intensity profile along a line segment, and edge detection. Also, images were qualitatively compared using image registration and transformation [l]. The gray-level histogram, edge detection, image profile and image registration do not offer conclusive results. The LED light source, however, produces good images for visual inspection by an operator. Analysis using pattern recognition using Eigenfaces and Gaussian Pyramid in face recognition may be more useful.

Bachnak, Rafic↗

NASA NTRS2005

Jet Propulsion Laboratory: Annual Report 2004

Once or twice in an age, a year comes along that the historians proclaim as an Annus Mirabilis - a year of wonders. For the Jet Propulsion Laboratory, 2004 was just that sort of time. From beginning to end, it was a nonstop experience of wondrous events in space. Imagine that two robot rovers embark on cross-country rambles across Mars, scrutinizing rocks for signs of past water on the now-arid world. A flagship spacecraft brakes into orbit at Saturn to begin longterm surveillance of the ringed world, preparing to drop a sophisticated probe to the surface of its haze-shrouded largest moon. Another craft makes the closest-ever pass by the nucleus of a comet, collecting sample particles as it goes. Two new space telescopes peer into the depths of the universe far beyond our solar system, viewing stars, nebulas and galaxies in invisible light beyond the spectrum our eyes can see. A pair of instruments is lofted on a NASA Earth-orbiting satellite to monitor air quality and the protective layer of ozone blanketing our home planet. A small probe brings samples of the solar wind to Earth for in-depth study. While JPL was absorbed with all of these ventures on other worlds, NASA and the White House unveiled an ambitious new plan of space exploration. The Vision for Space Exploration announced in January foresees a program of robotic and astronaut missions leading to a human return to the Moon by 2020, and eventual crewed expeditions to Mars. The vision also calls for more robotic missions to the moons of the outer planets; spaceborne observatories that will search for Earth-like planets around other stars and explore the formation and evolution of the universe; and continued study of our home planet. In order to accomplish all of this, NASA must perfect many as-yet-uninvented technologies and space transportation capabilities. JPL has a great deal to bring to this vision. Robotic exploration of Mars will lead the way for missions that will carry women and men to the red planet. Our engineers and scientists are formulating spacecraft that could use nuclear power to enable exploration missions of the future. And even now we are designing formations of space telescopes that will capture family portraits of the planets around neighboring stars. Those are only some of the ways that the Laboratory is contributing to NASA's broader goals. During 2004, JPL made a distinctive contribution to agencywide initiatives in areas such as safety, NASA transformation, the agency's Internet portal and NASA's Explorer Schools programs. Years like 2004 pose a special challenge for us. It would be easy to say that this was a once-in-a-decade high point of mission activities, but I believe that this would miss an opportunity. We are fortunate to have many space projects in the works that have the promise of being just as exhilarating as the great mission successes that we celebrated this year. The challenge and opportunity for us now is to make every year like this one.

National Aeronautics and Space Administration (NAS↗

NASA NTRS1989

A knowledge-based machine vision system for space station automation

A simple knowledge-based approach to the recognition of objects in man-made scenes is being developed. Specifically, the system under development is a proposed enhancement to a robot arm for use in the space station laboratory module. The system will take a request from a user to find a specific object, and locate that object by using its camera input and information from a knowledge base describing the scene layout and attributes of the object types included in the scene. In order to use realistic test images in developing the system, researchers are using photographs of actual NASA simulator panels, which provide similar types of scenes to those expected in the space station environment. Figure 1 shows one of these photographs. In traditional approaches to image analysis, the image is transformed step by step into a symbolic representation of the scene. Often the first steps of the transformation are done without any reference to knowledge of the scene or objects. Segmentation of an image into regions generally produces a counterintuitive result in which regions do not correspond to objects in the image. After segmentation, a merging procedure attempts to group regions into meaningful units that will more nearly correspond to objects. Here, researchers avoid segmenting the image as a whole, and instead use a knowledge-directed approach to locate objects in the scene. The knowledge-based approach to scene analysis is described and the categories of knowledge used in the system are discussed.

Chipman, Laure J.↗

NASA NTRS1994

Higher-Order Neural Networks Applied to 2D and 3D Object Recognition

A Higher-Order Neural Network (HONN) can be designed to be invariant to geometric transformations such as scale, translation, and in-plane rotation. Invariances are built directly into the architecture of a HONN and do not need to be learned. Thus, for 2D object recognition, the network needs to be trained on just one view of each object class, not numerous scaled, translated, and rotated views. Because the 2D object recognition task is a component of the 3D object recognition task, built-in 2D invariance also decreases the size of the training set required for 3D object recognition. We present results for 2D object recognition both in simulation and within a robotic vision experiment and for 3D object recognition in simulation. We also compare our method to other approaches and show that HONNs have distinct advantages for position, scale, and rotation-invariant object recognition. The major drawback of HONNs is that the size of the input field is limited due to the memory required for the large number of interconnections in a fully connected network. We present partial connectivity strategies and a coarse-coding technique for overcoming this limitation and increasing the input field to that required by practical object recognition problems.

Spirkovska, Lilly↗

NASA NTRS2012

Single Lens Dual-Aperture 3D Imaging System: Color Modeling

In an effort to miniaturize a 3D imaging system, we created two viewpoints in a single objective lens camera. This was accomplished by placing a pair of Complementary Multi-band Bandpass Filters (CMBFs) in the aperture area. Two key characteristics about the CMBFs are that the passbands are staggered so only one viewpoint is opened at a time when a light band matched to that passband is illuminated, and the passbands are positioned throughout the visible spectrum, so each viewpoint can render color by taking RGB spectral images. Each viewpoint takes a different spectral image from the other viewpoint hence yielding a different color image relative to the other. This color mismatch in the two viewpoints could lead to color rivalry, where the human vision system fails to resolve two different colors. The difference will be closer if the number of passbands in a CMBF increases. (However, the number of passbands is constrained by cost and fabrication technique.) In this paper, simulation predicting the color mismatch is reported.

bipartite filter↗

DOE OSTI2026

MARVEL Reactor Digital Engineering Developments

The MARVEL reactor project has served to introduce a new generation of engineers to the processes required to transform a reactor design from simply an idea on paper into what will be an approved, constructed, and operational nuclear power system. Much as there have been advances in materials, analysis, and evaluation methodologies over the 50 years since the last reactor was built at INL, so too has the technology for managing the engineering process itself advanced. Digital Engineering tools and methods provide improved coordination between previously siloed engineering disciplines, reduced burdens of non-value-added data transcription processes and bring forward insights and improvements that might otherwise fall later in the design stage, where changes are much more costly. While the tools and techniques to support the full digital engineering vision are not yet complete, the MARVEL design processes provide valuable demonstrations and validations of key aspects and illuminate further areas for implementation by subsequent projects.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗

DOE OSTI2025

AI Applications to Physics Experiments at Jefferson Lab

We survey how AI/ML is being deployed across Jefferson Lab's experimental and accelerator programs. In EPSCI, Hydra applies computer vision to automate real-time data-quality monitoring across all four experimental halls, replacing manual inspection of hundreds to thousands of histograms per shift. AIEC (AI Experiment Controls) uses ML to stabilize drift chamber gains and is now part of standard CEBAF production running, while AI Optimized Polarization (AIOP) targets autonomous control of polarized targets and photon beam angular alignment. In CASA, cavity fault classification models identify faulted cavities and trip types from waveform data with ~85% and ~78% agreement to labeled data, respectively, and are deployed in production; a separate effort applies LLMs and hybrid search to make the CEBAF operations logbook AI-ready. QCD-focused work includes transformer- and GAN-based generative models for particle-level event simulation, with distributed GAN training scaling studies on Polaris. Additional efforts span ML-on-FPGA for the EIC and a new Data Science Department coordinating anomaly detection, uncertainty quantification, and HPC-scalable ML lab-wide. Collectively, these projects illustrate AI's growing role in improving efficiency across JLab's nuclear physics mission.

Mei, Xinxin [Thomas Jefferson National Accelerator↗

NASA NTRS1993

Extracting heading and temporal range from optic flow: Human performance issues

Pilots are able to extract information about their vehicle motion and environmental structure from dynamic transformations in the out-the-window scene. In this presentation, we focus on the information in the optic flow which specifies vehicle heading and distance to objects in the environment, scaled to a temporal metric. In particular, we are concerned with modeling how the human operators extract the necessary information, and what factors impact their ability to utilize the critical information. In general, the psychophysical data suggest that the human visual system is fairly robust to degradations in the visual display, e.g., reduced contrast and resolution or restricted field of view. However, extraneous motion flow, i.e., introduced by sensor rotation, greatly compromises human performance. The implications of these models and data for enhanced/synthetic vision systems are discussed.

Kaiser, Mary K.↗

DOE OSTI2026

Toward memory-efficient melt pool monitoring: a classification framework using event-based imaging and sparse sensing technique

Vision sensors like CMOS and CCD cameras are often used for in-process monitoring of melt pools in laser-based additive and welding processes, but they require transferring large amounts of data and computational processing resources. Event-based neuromorphic imagery, on the other hand, detects only the change in pixel intensity, thus potentially reducing the data amount and latency. With an event imager, this study develops a framework for melt pool condition classification, including image construction, time scale selection, optimal pixel selection, and sparse classification, to achieve a highly memory-efficient scheme. These are based on sparse sensing techniques with singular value decomposition (SVD) and QR pivoting, the two fundamental matrix transformations for linear dimensionality reduction. The framework is then validated by classifying a controlled experiment by exciting various mode shapes of liquid gallium pools of varying depths (3, 6, and 8 mm). At 200 pixels, the classifier can reach overall accuracy of 75%, while at 2000 pixels (0.013% of the total possible pixels), the accuracy is nearly 90% (89.86%). At the same number of pixels, random selection can only achieve 46% and 67%, respectively. The memory savings of the sparsely sampled event data compared to a conventional imager is about 500 times. In addition to performance, implementation and limitations of the framework are also discussed.

42 ENGINEERING↗

NASA NTRS1992

Human low vision image warping - Channel matching considerations

We are investigating the possibility that a video image may productively be warped prior to presentation to a low vision patient. This could form part of a prosthesis for certain field defects. We have done preliminary quantitative studies on some notions that may be valid in calculating the image warpings. We hope the results will help make best use of time to be spent with human subjects, by guiding the selection of parameters and their range to be investigated. We liken a warping optimization to opening the largest number of spatial channels between the pixels of an input imager and resolution cells in the visual system. Some important effects are not quantified that will require human evaluation, such as local 'squashing' of the image, taken as the ratio of eigenvalues of the Jacobian of the transformation. The results indicate that the method shows quantitative promise. These results have identified some geometric transformations to evaluate further with human subjects.

Juday, Richard D.↗

DOE OSTI2026

Computer vision-based rock bolt detection in orthomosaic imagery obtained in the Waste Isolation Pilot Plant underground facility

Assessing structural integrity of large underground tunnel facilities is often a time consuming and human-labor intensive task. Thus, research using various modes of sensing and automated detection of key structural components in mines is posed to aid in safety assessments and establishing overall structural health. We propose an approach utilizing off-the-shelf camera and lidar technology fixed to a custom sensing platform, image stitching techniques, fine-tuned object detection models, and specialized model-inference methods to automatically detect, count, and map roof bolts for assessment of structural safety in man-made underground tunnels. Results show a novel workflow for effective object counting in orthomosaic tunnel ceiling images generated from collections in GPS-denied mining environments. Additionally, we demonstrate effective fine-tuning of EfficientDet object detectors utilizing state-of-the-art image augmentation techniques known as the mosaic and mixup transformations. Our work is demonstrated on sensed data and imagery collected from the Department of Energy (DOE) Waste Isolation Pilot Plant (WIPP) where miles of tunnel ceiling must be assessed for structural integrity.

42 ENGINEERING↗

NASA NTRSDate not supplied

Earth-Independent Medical Operations (EIMO) Concept of Operations

In contrast to the current crew health paradigm for low-Earth orbit and Lunar missions, which depends on real-time communication with Mission Control, deep-space exploration missions will require a significant shift in medical operations. This shift is driven by the constraints of operating at a considerable distance from Earth, such as resource limitations—lack of resupply, restricted mass, power, volume, and data—as well as communication delays and the inability to evacuate back to Earth during emergencies. To move toward a more self-reliant medical model, a strategy is needed to gradually increase space-based crew autonomy and reduce risks to mission success in the challenging environment of deep space. This transformative change, known as "Earth-Independent Medical Operations" (EIMO), explores the gradual transfer of medical care and decision-making from Earth-based support to space-based systems. The goal of this transition is to enhance astronaut health and performance while minimizing mission risks. EIMO requires the development of a medical system that integrates seamlessly with mission planning, vehicle and spacesuit design, and data architecture. This integration is crucial for building a robust medical infrastructure that not only safeguards astronaut well-being but also ensures overall mission success. The Human Research Program (HRP) Exploration Medical Capability (ExMC) Element has revised the EIMO model-based Concept of Operations (ConOps) which outlines an initial vision for EIMO. The ConOps, which is built on the stakeholders’ need, system goals, and objectives (NGOs), presents an array of in-mission scenarios that span a wide range of medical conditions demonstrating the system’s capabilities from basic to complex events. Developed by a multidisciplinary team of systems engineers, scientists, and clinicians within ExMC, the ConOps revision includes two new scenarios(Barotrauma and Self-Medical Management and Behavioral Health and Chronic Medical Care), and implementation of findings from EIMO technical interchange meetings that focused on data and training. The envisioned EIMO Medical System (MS) operates as a system of systems, gathering data from various sources such as reference databases, real-time wearable sensors, point-of-care diagnostics, and environmental controls. The MS also incorporates advanced training tools to support autonomous medical care, assisting the Crew Medical Officer (CMO) during medical events where Ground Support is either unavailable or communication-delayed beyond practicality. Furthermore, MS functions and capabilities were decomposed from the scenarios to establish foundational requirements for EIMO and traced to the NASA Spaceflight Human-System Standard(NASA-STD-3001, Volumes 1 and 2). These traces were performed to gain insights on the alignment of EIMO requirements with the NASA standard. This work serves as an initial recommendation to increase crew autonomy gradually and safely for Mars missions and future deep-space exploration.

medical system↗

DOE OSTI2024

The Digital Engineering Vision for DOME: Facilitating Design, Deployment, and Operations [Poster]

DOME is a planned microreactor test facility at INL’s Materials and Fuels Complex. It is a complex system with several interdependent sub-systems such as the reactor (up to 20 MWth), radioactive confinement, temperature and pressure regulation system, ventilation system, etc. The engineering design process for such a system traditionally involves several documents from various sources and the system information is scattered across these documents. Digital engineering represents a paradigm shift through which systems are designed using digital models and integrated data. The digital engineering vision for DOME utilizes a model-based systems engineering (MBSE) approach. The system architecture, physical components, control logic, and verification experiments are all designed using MathWorks MATLAB and Simulink. This hierarchical model can combine data from multiple sources at various levels of abstraction. It can be used to simulate the facility’s operations and to test the system using different sets of parameters. Its capabilities can be expanded by interfacing it with high-fidelity multi-physics models, risk analysis tools, etc. The same model can evolve into a digital twin that can monitor operations and conduct predictive analysis using real-time sensor data from the facility. The eventual goal of this effort is to transform the end-to-end engineering of nuclear facilities in every phase of their lifecycle, including design, deployment, and operations.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2024

SinhaRoy_TechPresentation_2024 [Slides]

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

NASA NTRS2014

Gravitational-wave Mission Study

In November 2013, ESA selected the science theme, the "Gravitational Universe," for its third large mission opportunity, known as L3, under its Cosmic Vision Programme. The planned launch date is 2034. ESA is considering a 20% participation by an international partner, and NASA's Astrophysics Division has indicated an interest in participating. We have studied the design consequences of a NASA contribution, evaluated the science benefits and identified the technology requirements for hardware that could be delivered by NASA. The European community proposed a strawman mission concept, called eLISA, having two measurement arms, derived from the well studied LISA (Laser Interferometer Space Antenna) concept. The US community is promoting a mission concept known as SGO Mid (Space-based Gravitational-wave Observatory Mid-sized), a three arm LISA-like concept. If NASA were to partner with ESA, the eLISA concept could be transformed to SGO Mid by the addition of a third arm, augmenting science, reducing risk and reducing non-recurring engineering costs. The characteristics of the mission concepts and the relative science performance of eLISA, SGO Mid and LISA are described. Note that all results are based on models, methods and assumptions used in NASA studies

identified the technology requirements↗

NASA NTRS2006

Real-time Enhancement, Registration, and Fusion for an Enhanced Vision System

Over the last few years NASA Langley Research Center (LaRC) has been developing an Enhanced Vision System (EVS) to aid pilots while flying in poor visibility conditions. The EVS captures imagery using two infrared video cameras. The cameras are placed in an enclosure that is mounted and flown forward-looking underneath the NASA LaRC ARIES 757 aircraft. The data streams from the cameras are processed in real-time and displayed on monitors on-board the aircraft. With proper processing the camera system can provide better-than-human-observed imagery particularly during poor visibility conditions. However, to obtain this goal requires several different stages of processing including enhancement, registration, and fusion, and specialized processing hardware for real-time performance. We are using a real-time implementation of the Retinex algorithm for image enhancement, affine transformations for registration, and weighted sums to perform fusion. All of the algorithms are executed on a single TI DM642 digital signal processor (DSP) clocked at 720 MHz. The image processing components were added to the EVS system, tested, and demonstrated during flight tests in August and September of 2005. In this paper we briefly discuss the EVS image processing hardware and algorithms. We then discuss implementation issues and show examples of the results obtained during flight tests.

Hines, Glenn D.↗

NASA NTRS2010

Feature Extraction and Selection Strategies for Automated Target Recognition

Several feature extraction and selection methods for an existing automatic target recognition (ATR) system using JPLs Grayscale Optical Correlator (GOC) and Optimal Trade-Off Maximum Average Correlation Height (OT-MACH) filter were tested using MATLAB. The ATR system is composed of three stages: a cursory region of-interest (ROI) search using the GOC and OT-MACH filter, a feature extraction and selection stage, and a final classification stage. Feature extraction and selection concerns transforming potential target data into more useful forms as well as selecting important subsets of that data which may aide in detection and classification. The strategies tested were built around two popular extraction methods: Principal Component Analysis (PCA) and Independent Component Analysis (ICA). Performance was measured based on the classification accuracy and free-response receiver operating characteristic (FROC) output of a support vector machine(SVM) and a neural net (NN) classifier.

computer vision↗